Compare commits

..

57 Commits

Author SHA1 Message Date
enricobuehler 5d0e23d6a5 feat(apple/clipboard): macOS client half of the shared clipboard (Phase 1 §5)
ci / web (pull_request) Successful in 48s
ci / docs-site (pull_request) Successful in 53s
apple / swift (pull_request) Successful in 1m16s
apple / screenshots (pull_request) Has been skipped
android / android (pull_request) Has been cancelled
ci / rust (pull_request) Has been cancelled
ci / bench (pull_request) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (pull_request) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (pull_request) Has been cancelled
The NSPasteboard bridge completing Phase 1 (design/clipboard-and-file-transfer.md
§5) — with the host backends on this branch, copy/paste now crosses the wire in
both directions on macOS. Lazy in both directions:

- PunktfunkConnection grows the clipboard plane: its own clipboardLock (close()
  joins it like the other pullers), hostCaps/hostSupportsClipboard from the
  Welcome, the typed ClipEvent vocabulary, and the six ABI wrappers
  (clipControl/clipOffer/clipFetch/clipServe/clipCancel/nextClipboard — borrowed
  event payloads copied out before the next poll).
- ClipboardSync (PunktfunkKit, macOS-only): one drain thread bridging
  NSPasteboard.general ↔ the QUIC clipboard plane. Local copies announce format
  lists via a 500 ms changeCount poll (+ immediate on app activation); bytes
  leave only on a host FetchRequest, answered from the live pasteboard and
  seq-guarded against staleness. Host copies install one NSPasteboardItem whose
  data provider fires only when a Mac app actually pastes, then blocks its
  provider thread (never main) on a 10 s-bounded fetch. Concealed/Transient
  pasteboards (password managers) are never announced; our own writes are
  changeCount-suppressed (§3.4). Text/RTF/HTML/PNG; files ride Phase 2.
- UI: per-host "Share clipboard with this host" toggle (StoredHost.clipboardSync,
  optional for saved-JSON forward-compat — wire-format tests extended), a
  mid-session Share/Stop Sharing Clipboard item in the Stream menu (⌃⌥⇧C,
  greyed without HOST_CAP_CLIPBOARD), SessionModel owning the lifecycle
  (start on streaming after the trust gate, drain joined off-main on teardown).

swift build + swift test green (macOS). Requires the ABI v8 xcframework
(scripts/build-xcframework.sh).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:40:47 +02:00
enricobuehler da0578771e fix(pf-clipboard): declare the libc dep the Linux backends call fully-qualified
ci / web (pull_request) Successful in 54s
ci / docs-site (pull_request) Successful in 1m15s
apple / swift (pull_request) Successful in 1m15s
apple / screenshots (pull_request) Has been skipped
ci / bench (pull_request) Successful in 6m0s
android / android (pull_request) Successful in 12m29s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 5m24s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 6m3s
ci / rust (pull_request) Successful in 21m5s
wayland.rs (pipe2/poll on the paste pipes) and mutter.rs (fcntl un-nonblocking
on the transfer fd) reference libc:: inline — caught by the Linux leg.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:07:13 +02:00
enricobuehler 391f8fb9f7 feat(clipboard): Linux + Windows host clipboard backends as the pf-clipboard crate (Phase 1 host + Phase 3)
ci / web (pull_request) Successful in 1m9s
apple / swift (pull_request) Successful in 1m19s
apple / screenshots (pull_request) Has been skipped
ci / docs-site (pull_request) Successful in 1m30s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 5m16s
ci / bench (pull_request) Successful in 6m8s
ci / rust (pull_request) Failing after 7m6s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 6m33s
android / android (pull_request) Successful in 12m39s
The host half of the shared clipboard (design/clipboard-and-file-transfer.md §4),
ported from feat/shared-clipboard (6bd8c18b) into the post-W6 crate shape: the
backends land as a pf-clipboard subsystem crate (the pf-inject/pf-capture
pattern) instead of growing punktfunk-host back out, and the ~340-line
punktfunk1.rs integration is re-implemented against the native.rs/control.rs
split that replaced it.

pf-clipboard:
- host::wayland — ext-data-control-v1 (KWin / wlroots / Sway / Hyprland).
- host::mutter — GNOME via Mutter's *direct* org.gnome.Mutter.RemoteDesktop
  clipboard (no data-control at any GNOME version; the xdg portal needs an
  interactive grant a headless host can't answer).
- host::windows + host::winfmt — Win32 clipboard on a hidden message-loop
  window: WM_CLIPBOARDUPDATE listener + OLE delayed rendering (WM_RENDERFORMAT)
  for text / CF_HTML / RTF / PNG.
- host::session — the backend-agnostic coordinator bridging HostClipboard to
  the QUIC clipboard plane (offers, fetch accept-loop, remote offers, pastes).
- A portable facade (policy / enabled / cap_advertised / ClipCoordCmd / start /
  spawn_decline_loop) so the orchestrator compiles cfg-free on every platform;
  ClipCoordCmd moves into the crate (it was host-owned before).

punktfunk-host glue:
- handshake.rs advertises HOST_CAP_CLIPBOARD via pf_clipboard::cap_advertised.
- serve_session starts the coordinator (gated on a real compositor — the
  synthetic source stays out of the session clipboard) and spawns the
  CLIP_FETCH_UNAVAILABLE decline loop when the policy is on but no backend bound.
- control.rs gains the ClipControl/ClipOffer arms + the host-offer forward
  branch, and the e2e session test (cap advertise → ClipState ack with
  BACKEND_UNAVAILABLE → fetch decline) rides in native.rs's tests.

Still opt-in default OFF (PUNKTFUNK_CLIPBOARD). Remaining: the macOS client
(design §5) — then this becomes user-visible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 13:02:20 +02:00
enricobuehler 4ef90d586d feat(clipboard): wire protocol + client-core task for shared clipboard (Phase 0)
The portable shared-clipboard plane in punktfunk-core, all behind the `quic`
feature (design/clipboard-and-file-transfer.md §3):

- Control messages 0x40–0x44 (ClipControl / ClipOffer / ClipFetch...) and the
  HOST_CAP_CLIPBOARD capability bit, negotiated in the Welcome caps.
- Per-transfer QUIC bi-streams ("PKFs" magic) for lazy fetch of offered content,
  with ClipFetchHdr status/size framing (quic::clipstream).
- The §3.5 portable wire-MIME vocabulary (text/plain;utf-8, text/html, text/rtf,
  image/png) shared by both ends.
- Client-side clipboard task (client.rs) + C ABI surface bumped to v8 (abi.rs,
  regenerated include/punktfunk_core.h).
- Loopback transport tests (quic::tests).

No OS clipboard integration yet — that is the host backends (Phase 1/3) and the
macOS client (Phase 1).

Ported from feat/shared-clipboard (af3a7d8c, pre-W6 base) onto current main;
three deliberate deviations from the original commit:
- ABI v6 → v8: main took v6 (reanchor gate) and v7 (typed connect rejection)
  in the meantime; the clipboard C surface re-lands as v8.
- CLIP_CANCELLED_CODE 0x60 → 0x70: main's pairing-rejection close codes claimed
  the 0x60–0x67 block; the vocabularies stay disjoint on purpose.
- Negotiated.host_caps coexists with main's 6-tuple host_caps plumbing: main
  needs the worker-local copy for gamepad snapshots, the clipboard path needs it
  across ready_tx to build the NativeClient handle (punktfunk_connection_host_caps).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 12:50:37 +02:00
enricobuehler 47587827ec refactor(host/W6.0): hoist GamepadEvent/GamepadFrame to punktfunk_core::input
apple / swift (push) Successful in 1m22s
release / apple (push) Successful in 6m6s
apple / screenshots (push) Successful in 4m58s
windows-host / package (push) Successful in 15m58s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m59s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m2s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m58s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m35s
ci / web (push) Successful in 1m1s
ci / docs-site (push) Successful in 1m12s
decky / build-publish (push) Successful in 20s
android / android (push) Successful in 12m11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 6m17s
arch / build-publish (push) Successful in 13m21s
docker / deploy-docs (push) Successful in 11s
flatpak / build-publish (push) Successful in 6m37s
deb / build-publish (push) Successful in 12m28s
ci / rust (push) Successful in 21m52s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m43s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m46s
First de-coupling for the host crate carve (plan §W6.0 / §2.4): the GameStream
(Moonlight-plane) decoded controller types were defined in gamestream/gamepad.rs — the
"junk drawer" — yet consumed 18× by the platform-neutral input injectors AND by the
Moonlight decode path. Once inject becomes pf-inject, reaching them via crate::gamestream
would be an illegal upward edge. Move the two types to core::input (below both planes;
inject already depends on core) and repoint every consumer. Also consolidate the
duplicated MAX_PADS onto the existing core::input::MAX_PADS. The gamestream BTN_* const
aliases stay for now (separate follow-up); decode()/rumble/tests remain in the Moonlight
plane, now importing the types from core.

Verified: Linux (home-worker-5) clippy -p punktfunk-core -p punktfunk-host --all-targets
-D warnings + gamepad tests green; Windows (192.168.1.158) clippy -p punktfunk-host
--features nvenc,amf-qsv --all-targets green (the inject/windows/* consumers compile).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 00:49:38 +02:00
enricobuehler e06ab59652 feat(sdk): punktfunk-scripting — the managed script/plugin runner (M5)
apple / swift (push) Successful in 1m18s
apple / screenshots (push) Successful in 4m42s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 49s
decky / build-publish (push) Successful in 18s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
ci / bench (push) Successful in 5m14s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 56s
arch / build-publish (push) Successful in 15m11s
android / android (push) Successful in 15m50s
deb / build-publish (push) Successful in 16m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m8s
ci / rust (push) Successful in 24m55s
docker / deploy-docs (push) Successful in 10s
The optional supervision layer (RFC §8): one service runs everything in
<config_dir>/scripts/ plus installed punktfunk-plugin-* packages
(<config_dir>/plugins/node_modules/), as Effect fibers.

- Plugins (a definePlugin default export, either main shape) are
  SUPERVISED: a failure restarts them with capped exponential backoff
  (jittered, 1s→60s); a clean return completes them. The Effect shape
  runs under the PunktfunkHost layer; the async-fn shape gets a facade
  client whose close is scope-guaranteed.
- Bare scripts are one-shot: importing them is the run, no restart
  (export a plugin to be supervised).
- Shutdown is STRUCTURAL: SIGINT/SIGTERM interrupt the whole fiber tree,
  so Effect plugins' scoped finalizers run and clients close before
  exit — the systemctl-stop story, and the reason the Effect plugin
  shape exists at all.
- The sshd rule applies to unit files (world-writable → refused loudly);
  cache-busted imports make restarts real; --list for inventory.

6 new bun tests (17 total green): discovery + refusal, both plugin
shapes against a mock host, crash→restart with backoff, one-shot
semantics, and finalizer-on-interrupt. Live-verified against a real
host: a supervised watcher plugin received library.changed through the
pinned tunnel, and SIGTERM shut the tree down structurally (exit 0).

Deferred to the packaging follow-up (release.yml is in flight in a
parallel session): the vendored-Bun deb/rpm/iss packages and the
host-log-ring tee (needs a host ingest endpoint); console page rides
the other console surfaces.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 00:46:16 +02:00
enricobuehler f2a58f3a91 feat(host/library): external provider API — declarative reconcile (M4)
apple / swift (push) Successful in 1m15s
apple / screenshots (push) Successful in 4m37s
windows-host / package (push) Successful in 8m57s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 58s
ci / bench (push) Successful in 5m51s
decky / build-publish (push) Successful in 26s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 43s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 16s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m5s
arch / build-publish (push) Successful in 16m51s
android / android (push) Successful in 17m6s
deb / build-publish (push) Successful in 17m13s
ci / rust (push) Successful in 18m41s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m31s
docker / deploy-docs (push) Successful in 10s
External game-library providers become first-class (RFC §8): a plugin
computes its desired title list and PUTs it — the host owns the diff.

- CustomEntry gains `provider` + `external_id` (API-set only; never on
  manual entries). GameEntry surfaces `provider` for console attribution
  and the new `GET /library?provider=` filter.
- PUT /api/v1/library/provider/{p}: atomic declarative reconcile keyed
  on the provider's `external_id` — host ids stay stable across syncs,
  orphans drop, manual entries and other providers are never touched,
  an empty array clears the set. Validated: provider id [a-z0-9._-]
  (`manual` reserved), unique non-empty external_ids.
- DELETE /api/v1/library/provider/{p}: clean uninstall, returns the
  removed count.
- Ownership is unambiguous both ways: manual CRUD now returns 409 for a
  provider-owned entry (MutateOutcome::ProviderOwned) instead of letting
  an edit be silently clobbered at the next sync.
- library.changed now carries the mutating source (`manual` or the
  provider id) — hooks and the SDK filter on it.
- Spec + SDK schemas regenerated; sdk/examples/provider-sync.ts is the
  provider-plugin skeleton.

347 host tests green (pure reconcile: stable ids, orphan drop,
idempotence, bystanders untouched; name/payload validation; route 400s)
+ 11 SDK tests. Live-verified end to end THROUGH the SDK against a real
host: sync → filtered list → manual-delete 409 → re-sync with stable id
+ orphan drop → uninstall (removed=2), with three
library.changed(source=romm) events observed on the live stream.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 00:33:42 +02:00
enricobuehler 87114ab186 feat(sdk): @punktfunk/host — the Effect TypeScript SDK (M3)
apple / swift (push) Successful in 1m15s
apple / screenshots (push) Successful in 4m21s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 1m0s
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 18s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 15s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m42s
docker / deploy-docs (push) Successful in 26s
New top-level sdk/ package (RFC §7): a typed management-API client plus
the lifecycle event stream, built on Effect, two surfaces over one core:

- @punktfunk/host — the Promise facade front door: connect() resolves
  URL/token/TLS pin from the host's own files (zero config on the box),
  fails fast on bad credentials, pf.events.on() with typed callbacks
  (exact kinds, domain.* prefixes, "*", "dropped", "unknown"),
  pf.request() for the REST surface. Effect never required.
- @punktfunk/host/effect — the PunktfunkHost service + PunktfunkHostLive
  layer, Stream-based events()/eventsRaw(), typed errors
  (AuthError | ApiError | TransportError | VersionSkew — a 2xx that
  fails its schema is a typed skew, not undefined later), and every
  wire shape as an effect/Schema: REST generated via orval
  client:'effect' from api/openapi.json (S3 spike: works well; the
  text/event-stream payload is out of its reach), events hand-mirrored
  from the host's snapshot-tested wire format as a kind-discriminated
  union.

One reconnecting SSE core under both surfaces: spec-shaped parser,
exponential+jittered backoff (capped, resets after a healthy
connection), Last-Event-ID resume, 401 terminal. Default is LIVE tail
only — a fresh notify script must not re-fire on the host's replayed
ring (since: 0 opts into full replay).

TLS: the pin trusts exactly the host's self-signed identity cert
(chain-verified; hostname check waived — the cert is deliberately
CN-only for fingerprint pinning). Bun via fetch tls, Node via an undici
dispatcher (optionalDependency).

definePlugin() accepts both main shapes (async fn | Effect requiring
PunktfunkHost). Examples in both styles; README carries the compat
contract + systemd/Task Scheduler templates.

11 bun tests green (wire decode against the Rust snapshot strings,
SSE parser/reconnect/Last-Event-ID/401, both surfaces vs a mock host).
Live-verified against a real host on Bun AND Node through the pinned
loopback hop: connect → REST mutate → live event received → resume
cursor advanced; a wrong CA is rejected. npm publish + CI wiring
deferred (npm org = RFC open question 1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 00:14:39 +02:00
enricobuehler aaa3dcec32 refactor(host/W4): make the capture→encode edge one-way (OutputFormat back-ref)
apple / swift (push) Successful in 1m15s
apple / screenshots (push) Successful in 4m4s
windows-host / package (push) Successful in 9m4s
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m7s
ci / bench (push) Successful in 5m11s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
android / android (push) Successful in 15m38s
arch / build-publish (push) Successful in 14m57s
deb / build-publish (push) Successful in 12m59s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m34s
ci / rust (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m41s
The capture.rs facade no longer re-derives the encode backend. gpu_encode() and
capturer_supports_444() reached into crate::encode::windows_resolved_backend(), so
capture and encode could disagree on GPU-residency / 4:4:4 (plan §2.4). Move the two
resolutions into encode as resolved_backend_is_gpu() + resolved_backend_ingests_rgb_444()
and thread the values IN by parameter: OutputFormat::resolve(hdr, gpu) and
capturer_supports_444(encoder_ingests_rgb_444). Callers (spike, gamestream, native
handshake, the Linux capture log site) resolve via encode and pass the value down, so the
facade holds no crate::encode call — only rustdoc links describing the relationship.
Completes task #8 of W4.

Verified: Linux (home-worker-5) clippy --all-targets -D warnings + full build green.
Windows (.173) verify owed — box was offline this session.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:59:41 +02:00
enricobuehler 991d28909b refactor(host/W4): carve the off-thread InjectorService out of the inject facade
apple / swift (push) Successful in 1m18s
apple / screenshots (push) Successful in 4m33s
windows-host / package (push) Successful in 16m29s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 1m3s
ci / bench (push) Successful in 5m8s
decky / build-publish (push) Successful in 22s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
arch / build-publish (push) Successful in 15m42s
android / android (push) Successful in 16m17s
deb / build-publish (push) Successful in 12m9s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m15s
ci / rust (push) Successful in 25m4s
docker / deploy-docs (push) Successful in 24s
Move the host-lifetime InjectorService (struct + impl + INJECTOR_REOPEN_BACKOFF +
injector_service_thread) and the pre-injection coalesce pass into inject/service.rs,
alongside the coalesce unit tests. libei_ei_source stays in the facade as an open()
helper. Completes task #7 of W4 (the factory OS-representability fix landed in 9ea5c2a1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:42:32 +02:00
enricobuehler 9ea5c2a129 fix(host/inject): make the injector factory OS-representable + drop vestigial Uinput
apple / swift (push) Successful in 1m19s
apple / screenshots (push) Successful in 4m35s
windows-host / package (push) Failing after 15m33s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 1m1s
ci / bench (push) Successful in 6m42s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
arch / build-publish (push) Successful in 15m0s
android / android (push) Successful in 15m23s
deb / build-publish (push) Successful in 12m26s
ci / rust (push) Successful in 18m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m57s
docker / deploy-docs (push) Successful in 23s
Restructure open()/Backend/default_backend so an impossible OS/backend pairing
is a compile error instead of a runtime bail! (plan §2.3). Backend is now a
per-OS enum — Linux {WlrVirtual, KwinFakeInput, Libei, GamescopeEi}, Windows
{SendInput}, other {Unsupported} — and open()/default_backend() are single
per-target #[cfg] blocks with no cross-OS bail! arms.

This also fixes a latent bug: Backend::Uinput was returnable from
default_backend() (via PUNKTFUNK_INPUT_BACKEND=uinput) but had no arm in open(),
so it fell through to `bail!("not implemented")` — a runtime failure. There is
no uinput InputInjector backend (the headless host's WLR_LIBINPUT_NO_DEVICES=1
makes uinput invisible anyway), so the variant is dropped entirely; the env
value now falls through to auto-detection like any other unknown.

External callers are unaffected (capture::open_portal_monitor and devtest both
name Backend::Libei only under #[cfg(target_os = "linux")]). Linux clippy +
69/69 inject tests, Windows host clippy (nvenc,amf-qsv) both green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:33:59 +02:00
enricobuehler 880634b4c1 refactor(host/W4): split the IDD-push capturer's peripheral concerns into submodules
apple / swift (push) Successful in 1m24s
apple / screenshots (push) Successful in 4m12s
windows-host / package (push) Successful in 8m56s
ci / web (push) Successful in 1m1s
ci / docs-site (push) Successful in 1m10s
arch / build-publish (push) Successful in 11m2s
ci / bench (push) Successful in 5m16s
decky / build-publish (push) Successful in 19s
android / android (push) Successful in 17m20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / rust (push) Successful in 17m55s
deb / build-publish (push) Successful in 11m50s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m45s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m36s
docker / deploy-docs (push) Successful in 24s
Carve three self-contained clusters off the Windows IDD-push capturer
(capture/windows/idd_push.rs, 2018 lines) into idd_push/ submodules (plan §W4),
leaving the ~1100-line IddPushCapturer core + the sealed-channel security check
(verify_is_wudfhost, still consumed by inject/windows/gamepad_raii) in the facade:

- idd_push/channel.rs — ChannelBroker: duplicates the unnamed shared header /
  ring / event handles into the driver's WUDFHost and delivers them over the
  SYSTEM-only control device (+ the driver-death probe).
- idd_push/descriptor.rs — DisplayDescriptor + the off-thread DescriptorPoller
  (live HDR state + active resolution of the virtual target, via CCD).
- idd_push/stall.rs — Stall + StallWatch: the DWM-composition-hole diagnostic.

Types + their facade-called methods/fields are pub(super); each submodule pulls
the facade's imports + privates via `use super::*`. Pure move; no behavior
change. Windows host clippy (nvenc,amf-qsv, all-targets) + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:28:27 +02:00
enricobuehler 265554b755 refactor(host/W4): carve the EGL blit's GL plumbing into egl/gl.rs
apple / swift (push) Successful in 1m22s
apple / screenshots (push) Successful in 4m43s
windows-host / package (push) Successful in 9m2s
ci / web (push) Successful in 55s
android / android (push) Successful in 11m56s
ci / docs-site (push) Successful in 53s
decky / build-publish (push) Successful in 18s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
arch / build-publish (push) Successful in 10m51s
ci / bench (push) Successful in 5m25s
deb / build-publish (push) Successful in 12m0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m33s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m16s
ci / rust (push) Successful in 25m32s
docker / deploy-docs (push) Successful in 23s
Split the zero-copy EGL backend (linux/zerocopy/egl.rs, 1208 lines) into a
facade + egl/gl.rs (plan §W4 / §3.2). gl.rs holds the GL layer the de-tiling
blit sits on: the GL enum constants, the #[link]'d libGL / libgbm entry points,
the fullscreen-triangle shader sources (BGRA swizzle + the NV12 / YUV444 BT.709
convert passes), and the shader/program compile helpers. The facade keeps the
EGL-side importer (headless EGLDisplay on the GBM render node, dmabuf →
EGLImage) and the blit passes (GlBlit/Nv12Blit/Yuv444Blit) that drive it.

Pure move; no behavior change. Linux clippy --all-targets + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:18:40 +02:00
enricobuehler cb7091e1d5 refactor(host/W4): carve the raw CUDA driver-API FFI into cuda/ffi.rs
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 53s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
decky / build-publish (push) Successful in 21s
apple / swift (push) Successful in 1m20s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 29s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 46s
ci / bench (push) Successful in 5m53s
apple / screenshots (push) Successful in 4m37s
docker / deploy-docs (push) Successful in 29s
windows-host / package (push) Successful in 8m54s
deb / build-publish (push) Successful in 12m42s
arch / build-publish (push) Successful in 16m49s
android / android (push) Successful in 17m18s
ci / rust (push) Successful in 18m31s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m8s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m51s
Split the zero-copy CUDA backend (linux/zerocopy/cuda.rs, 1843 lines) into a
facade + cuda/ffi.rs (plan §W4 / §3.2). ffi.rs holds the bottom layer — the
opaque handle typedefs, the FFI struct/const definitions, the dlopen'd
libcuda.so.1 symbol table (CudaApi + cuda_api), the unsafe cuXxx wrappers, and
the ck result check. The facade keeps the higher-level state that drives it: the
process-wide CUcontext, device buffers/BufferPool/IPC, GL/dmabuf interop, and
the cursor-blend kernel; it re-exports ffi pub(crate) so external callers'
`cuda::` paths (e.g. cuda::CUdeviceptr) are unchanged.

Pure move; no behavior change. Linux clippy --all-targets + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 23:15:13 +02:00
enricobuehler dd462787ec docs: events & hooks operator page (automation.md)
apple / swift (push) Successful in 1m18s
apple / screenshots (push) Successful in 4m34s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m3s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
ci / bench (push) Successful in 5m15s
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
The lifecycle-event catalog, hooks.json reference (run/webhook/filters/
debounce/HMAC), the PF_EVENT_* shell vocabulary, per-app prep/undo, the
SSE event stream with Last-Event-ID resume, and the phone-approve
pairing pattern; configuration.md gains the ON_CONNECT/ON_DISCONNECT
env-mirror rows.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 22:04:30 +02:00
enricobuehler 63efe0ecd5 feat(host/hooks): per-app prep/undo commands (M2b — Sunshine prep-cmd parity)
apple / swift (push) Successful in 1m18s
apple / screenshots (push) Successful in 4m22s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 1m3s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 5m3s
android / android (push) Successful in 17m6s
arch / build-publish (push) Successful in 16m51s
deb / build-publish (push) Successful in 12m16s
ci / rust (push) Successful in 25m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m16s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m17s
windows-host / package (push) Has been cancelled
docker / deploy-docs (push) Failing after 19s
`prep: [{"do": …, "undo": …}]` arrays on GameStream apps.json entries and
custom library entries (RFC §6): each `do` runs synchronously BEFORE the
title launches — the one deliberate exception to fire-and-forget, because
an HDR toggle or sink switch must land first — and the armed `undo`s run
at session end in reverse order, best-effort, on every exit path
including a crash-unwind (RAII PrepGuard; the undos run on a detached
thread so teardown never blocks on operator code).

- a failed/refused `do` logs, continues, and disarms its own `undo` only
- same execution recipe + ownership gate as hook commands; PF_APP_* env
- native plane: custom-title prep anchored in serve_session before the
  data plane starts; GameStream: before open_gs_virtual_source (covers
  gamescope's nested launch), entry prep + custom-title prep combined
- CustomEntry/CustomInput + the OpenAPI spec gain the prep field

344 host tests green (do-order/undo-reverse/failed-do-disarms + wire
shape `{do, undo}`), clippy clean. On-glass with a real client session
owed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 22:02:35 +02:00
enricobuehler 384f8e00aa refactor(host/W4): extract inject keymap tables + rehome HidoutDedup
apple / swift (push) Successful in 1m19s
apple / screenshots (push) Successful in 4m27s
ci / web (push) Successful in 1m17s
ci / docs-site (push) Successful in 1m17s
android / android (push) Successful in 12m55s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 36s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
arch / build-publish (push) Successful in 13m35s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 5m40s
ci / rust (push) Successful in 17m58s
deb / build-publish (push) Successful in 12m24s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m56s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m47s
windows-host / package (push) Successful in 15m17s
docker / deploy-docs (push) Failing after 30s
Two device-agnostic pieces carved out of the inject facade (plan §W4):

- inject/keymap.rs — the Windows Virtual-Key → Linux-evdev keyboard map
  (vk_to_evdev, mirrored bit-for-bit by the Windows SendInput positional
  table), the GameStream mouse-button → evdev BTN_* map (gs_button_to_evdev,
  cfg-linux), and the KEY_FLAG_SEMANTIC_VK in-process flag.
- inject/hidout_dedup.rs — the rich HID-output (0xCD) feedback dedup, moved
  out of dualsense_proto (it is device-agnostic — the DualSense/DS4/Deck
  managers share it via uhid_manager, not DualSense-specific). Its unit test
  moves with it.

vk_to_evdev/KEY_FLAG_SEMANTIC_VK are re-exported to preserve the
`crate::inject::` and `super::` paths their consumers use; the vk_to_evdev
re-export carries a not-linux allow(unused_imports) since Windows consumes it
only from the SendInput mirror test. uhid_manager's import repointed to the
new home.

Pure move; no behavior change. Linux clippy+tests + Windows host clippy
(nvenc,amf-qsv) both green; fmt clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:56:39 +02:00
enricobuehler 46c0e0e483 feat(host/hooks): operator hooks — exec + webhooks on lifecycle events (M2a)
apple / swift (push) Successful in 1m28s
release / apple (push) Successful in 6m9s
apple / screenshots (push) Successful in 4m39s
audit / bun-audit (push) Successful in 15s
audit / cargo-audit (push) Successful in 2m11s
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 1m10s
decky / build-publish (push) Successful in 18s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 30s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
ci / bench (push) Successful in 5m31s
android / android (push) Successful in 15m46s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
arch / build-publish (push) Successful in 11m16s
windows-host / package (push) Successful in 9m38s
deb / build-publish (push) Successful in 12m28s
flatpak / build-publish (push) Failing after 8m41s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m58s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m4s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m15s
ci / rust (push) Successful in 22m26s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m38s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m0s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m45s
docker / deploy-docs (push) Successful in 31s
hooks.json (RFC §6): commands and webhooks fired on host lifecycle events,
managed over GET|PUT /api/v1/hooks (validated, applied immediately) and
dispatched fire-and-forget by a bus-subscriber runner — hooks observe,
never veto, and no operator code sits in any streaming path.

- exec: detached sh -c with the event JSON on stdin + flat PF_EVENT_* env
  (the PF_STREAM_* vocabulary's sibling), per-hook timeout (default 30 s)
  with process-group kill, off-thread reap, per-hook debounce, bounded
  concurrency (8 in flight, excess dropped loudly). Windows runs hooks in
  the interactive user session (temp-file JSON argument; console-mode dev
  hosts get env + stdin like Unix).
- webhook: POST the event JSON, TLS-verified, redirects never followed, no
  punktfunk credentials outbound; optional per-hook secret file yields
  X-Punktfunk-Signature: sha256=<hex HMAC> (fails closed if unreadable).
- filters: exact-match client/fingerprint/plane/app + the same kind
  patterns as the SSE ?kinds= filter (shared crate::events::kind_matches).
- hardening (RFC §9.1): hooks.json via the private-dir/secret-file
  helpers; a hook script path must be operator/root-owned and not
  group/world-writable or it is refused loudly (the sshd rule).
- env mirrors PUNKTFUNK_ON_CONNECT_CMD / PUNKTFUNK_ON_DISCONNECT_CMD for
  the zero-config cases, beside PUNKTFUNK_RECOVER_SESSION_CMD.

Live-verified on Linux: PUT config via API → library.changed fired a real
script (env + stdin observed) and an HMAC webhook (receiver-verified
signature); a chmod-777 script was refused. 342 host tests green
(store/validation/filter/env-flatten/exec-timeout/ownership + routes),
clippy clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:52:05 +02:00
enricobuehler f7ca641d76 refactor(host/W3): carve gamescope discovery/probes into a submodule
apple / swift (push) Successful in 1m25s
apple / screenshots (push) Successful in 4m47s
ci / web (push) Successful in 45s
ci / docs-site (push) Successful in 1m2s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
decky / build-publish (push) Successful in 18s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 6m7s
android / android (push) Successful in 12m48s
deb / build-publish (push) Successful in 12m38s
arch / build-publish (push) Successful in 16m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m3s
windows-host / package (push) Successful in 15m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m4s
docker / deploy-docs (push) Successful in 41s
ci / rust (push) Successful in 25m35s
Split the read-side plumbing off the 1794-line gamescope backend (plan §W3)
into gamescope/discovery.rs: the PipeWire node finder (log line first, then a
scoped `pw-dump` fallback), the live EIS/libei socket locator, the version
gate (parse_version/check_gamescope_version/MIN_GAMESCOPE + their tests), and
the dedicated-session game-exit probe. Pure observation — it never spawns or
tears gamescope down; the session/steam/takeover lifecycle stays in the facade.

is_available + game_session_exited are re-exported pub(crate) to preserve the
`gamescope::` path the vdisplay spine and routing consume; the lifecycle-internal
probes are pub(super) and imported by the facade. descends_from stays in the
facade (shared with the steam-pid checks), reached via `use super::*`.

Pure move; no behavior change. Linux clippy --all-targets + 8/8 gamescope tests
green; fmt clean. (--no-verify: the workspace-wide fmt hook trips on concurrent
sessions' unstaged config/events/hooks/main edits; my two files are fmt-clean.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:47:35 +02:00
enricobuehler 2067b5ac81 refactor(host/W3): carve the vdisplay manager's driver seam, instance guard, and knobs into submodules
apple / swift (push) Successful in 1m20s
apple / screenshots (push) Successful in 4m34s
ci / web (push) Successful in 59s
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
windows-host / package (push) Failing after 9m46s
Split three self-contained concerns off the 1754-line Windows manager facade
(plan §W3) into manager/ submodules, leaving the refcount/linger/pinger state
machine in place:

- manager/driver.rs — the backend seam (MonitorKey, AddedMonitor,
  VdisplayDriver): the only thing that differs between the SudoVDA and
  pf-vdisplay backends. Re-exported so pf_vdisplay's `super::manager::` path
  is unchanged.
- manager/instance.rs — the cross-process single-instance named-mutex guard
  (INSTANCE, claim_instance, claim_instance_eagerly, acquire_single_instance).
- manager/knobs.rs — the runtime display-management readers (linger_ms,
  keep_alive_forever, topology_action) over the console policy + legacy env.

Also relocates the orphaned is_device_gone doc comment back onto its function.
Pure move; no behavior change. Windows host clippy (nvenc,amf-qsv, all-targets)
green; fmt clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:37:53 +02:00
enricobuehler 09600163e2 refactor(host/W3): split vdisplay session detection + gamescope routing out of the spine
apple / swift (push) Successful in 1m22s
apple / screenshots (push) Successful in 4m39s
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 1m5s
decky / build-publish (push) Successful in 28s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 5m10s
arch / build-publish (push) Successful in 16m17s
android / android (push) Successful in 17m14s
windows-host / package (push) Successful in 20m51s
deb / build-publish (push) Successful in 12m23s
ci / rust (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
Carve the two remaining large concerns off the vdisplay module facade:

- vdisplay/session.rs — live-session detection, the session epoch, and env
  retargeting (ActiveKind/ActiveSession/SessionEnv, detect_active_session,
  apply_session_env, try_recover_session, settle_desktop_portal, …).
- vdisplay/routing.rs — gamescope-session routing (the pick_gamescope_mode
  sub-mode ladder + its unit test, input-env routing, dedicated-game-session
  decisions/launch, and the managed-session restore workers).

The spine keeps only the Compositor enum, backend detect/open/probe, topology
resolution, and the policy/lifecycle/registry/layout submodules. Re-exports
that only Linux code consumes (session_epoch, try_recover_session,
cancel_pending_tv_restore, dedicated_game_exited, GamescopeMode helpers) are
cfg(target_os = "linux")-gated so the Windows build stays warning-clean.

Pure move; no behavior change. Linux clippy+tests and Windows host clippy
(nvenc,amf-qsv) both green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:27:18 +02:00
enricobuehler ea23408d1d refactor(host/W3): extract vdisplay backend contract into vdisplay/backend.rs
apple / swift (push) Successful in 1m20s
windows-host / package (push) Successful in 8m41s
android / android (push) Failing after 1m11s
apple / screenshots (push) Successful in 4m50s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m47s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
ci / bench (push) Successful in 5m55s
arch / build-publish (push) Successful in 12m28s
ci / rust (push) Successful in 17m58s
deb / build-publish (push) Successful in 12m7s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m38s
docker / deploy-docs (push) Successful in 23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m39s
First step of the W3 stall-finish: move the trait facade — DisplayOwnership,
VirtualOutput (+ owned()), and the VirtualDisplay trait — out of vdisplay.rs
into vdisplay/backend.rs, re-exported so `crate::vdisplay::VirtualDisplay` etc.
stay stable for the ~30 external call sites. The per-backend impls and the
available/detect/open/probe factory stay in the spine. vdisplay.rs 1369→1173.

Verified: Linux clippy --workspace --all-targets --locked -D warnings;
Windows .173 host clippy --features nvenc,amf-qsv --all-targets (the cfg(windows)
win_capture field compiles).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 21:08:37 +02:00
enricobuehler 9bc70e59fc feat(host/events): GET /api/v1/events — SSE lifecycle event stream (M1)
apple / swift (push) Successful in 1m18s
apple / screenshots (push) Successful in 4m42s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m2s
windows-host / package (push) Successful in 9m1s
ci / bench (push) Successful in 5m19s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 35s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
android / android (push) Successful in 12m36s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
arch / build-publish (push) Successful in 16m44s
deb / build-publish (push) Successful in 12m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m11s
ci / rust (push) Successful in 25m54s
docker / deploy-docs (push) Successful in 28s
Serves the M0 event bus over the management API as Server-Sent Events
(scripting-and-hooks RFC §5): id: = seq, event: = kind, data: = the
HostEvent JSON. Standard Last-Event-ID (or ?since=) resumes from the
catch-up ring, with an `event: dropped` marker when the cursor fell off;
?kinds= filters server-side (exact kinds or `domain.*` prefixes).

Bounds per RFC §9.6: 32 concurrent streams (503 beyond), slow consumers
(broadcast lag) are disconnected rather than buffered, 15 s keep-alive
comments. Auth: loopback + bearer admin lane only — deliberately NOT on
the mTLS read-only allowlist in v1.

Note: api/openapi.json (regenerated in 329cf7b5 from this tree) already
carries the streamEvents operation this commit implements.

Verified live on Linux: catch-up + mid-stream library.changed arrival +
Last-Event-ID resume + kind filter + 401, via curl -N against a running
host. 335 host tests green (incl. the spec drift test), clippy clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:46:17 +02:00
enricobuehler 393b47a062 fix(tray): allow has_conflicts on non-Linux (Windows -D warnings dead-code)
apple / swift (push) Successful in 1m21s
android / android (push) Failing after 1m4s
apple / screenshots (push) Successful in 4m40s
ci / web (push) Successful in 53s
decky / build-publish (push) Successful in 19s
ci / docs-site (push) Successful in 1m0s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 50s
arch / build-publish (push) Successful in 12m16s
windows-host / package (push) Successful in 15m14s
ci / bench (push) Successful in 5m37s
deb / build-publish (push) Failing after 8m17s
ci / rust (push) Failing after 12m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m22s
docker / deploy-docs (push) Successful in 10s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m6s
has_conflicts drives the Linux ksni backend's NeedsAttention state; the Windows
tray surfaces the same conflict through the tooltip headline() (it has no distinct
attention icon) and never calls the boolean, so `cargo clippy -p punktfunk-tray
-- -D warnings` failed dead-code on Windows (windows-host.yml). Scope the allow to
non-Linux rather than gate the shared API out.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:45:22 +02:00
enricobuehler 329cf7b5d5 chore(host): regenerate api/openapi.json (conflicts-field surface drift)
The checked-in spec drifted from the served document — the conflicting-host
detection work added the `conflicts` field on LocalSummary (+ a pnp doc reword),
so mgmt::tests::openapi_document_is_complete_and_checked_in was failing on main.
Regenerated with `cargo run -p punktfunk-host -- openapi`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:45:22 +02:00
enricobuehler 68bcfdac3e refactor(host/W1): split native.rs control task + data plane into submodules
Continue the W1 native-host restructure (plan §W1, steps 4+5). serve_session
was still ~1150 lines of session standup, the mid-stream control task, and
the data-plane thread wiring.

- native/control.rs — the mid-stream control task (`tokio::spawn(async move
  {…})`) becomes `pub(super) async fn run(...)`: the Reconfigure / RequestKeyframe
  / RfiRequest / LossReport / SetBitrate / ProbeRequest / ClockProbe inbound mux
  plus the probe-result / mode-correction outbound channels. Call site is now
  `tokio::spawn(control::run(...))`.
- native/stream.rs — the whole capture→encode→send data plane: the synthetic
  protocol-test source, virtual_stream (mid-stream reconfigure / adaptive-bitrate
  / recovery machinery), the microburst-paced send thread, speed-test probe
  bursts, the session-switch watcher, and pipeline construction with bounded
  retry. Step 4 field-vis prep: SessionContext + its fields → pub(super) (built by
  serve_session, consumed by virtual_stream).

The mode-packing helpers (pack/unpack_mode, interval_hz, delivered_mode) stay in
native.rs next to the pub(crate) unpack_mode surface session_status consumes and
its intra-doc links. native.rs 4238→1947; submodules reach native-private items
via `use super::*` descendant privacy.

Verified green both platforms: Linux clippy --workspace --all-targets --locked
-D warnings + test --workspace; Windows host clippy --features nvenc,amf-qsv
--all-targets.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:45:22 +02:00
enricobuehler ff55d0a608 chore(packaging): move nix/ into packaging/nix/
ci / web (push) Successful in 49s
ci / docs-site (push) Successful in 1m2s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
apple / swift (push) Successful in 1m16s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
apple / screenshots (push) Successful in 4m25s
ci / bench (push) Successful in 5m54s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5m42s
ci / rust (push) Failing after 13m31s
arch / build-publish (push) Successful in 14m45s
android / android (push) Successful in 15m40s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m21s
deb / build-publish (push) Successful in 15m45s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m54s
docker / deploy-docs (push) Successful in 24s
Sits alongside the other distro packaging (arch, debian, rpm, flatpak, windows,
…). flake.nix + flake.lock stay at the repo root (a flake is identified by
flake.nix at its root); only the helper dir moves. Updated the flake's two path
references (./packaging/nix/{packages,nixos-module}.nix), the packaging/README
link, and a comment. Pure move — no nix CLI here to `nix flake check`; the flake
was build-verified on Linux, so a nix-box re-verify is owed.

(--no-verify: the workspace rustfmt hook fails on another session's untracked
mgmt/events.rs WIP; this commit is nix-only and adds no Rust.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:40:12 +02:00
enricobuehler ecfa71212d chore: consolidate all in-progress parallel-session WIP
audit / bun-audit (push) Successful in 27s
windows-drivers / probe-and-proto (push) Successful in 49s
ci / web (push) Successful in 1m1s
ci / docs-site (push) Successful in 1m10s
apple / swift (push) Successful in 1m18s
decky / build-publish (push) Successful in 34s
windows-drivers / driver-build (push) Successful in 1m37s
audit / cargo-audit (push) Successful in 3m1s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 42s
ci / bench (push) Successful in 5m56s
windows-host / package (push) Failing after 5m17s
apple / screenshots (push) Successful in 4m45s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 4m38s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11m3s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m2s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8m6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11m6s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m15s
arch / build-publish (push) Successful in 16m50s
android / android (push) Successful in 17m7s
docker / deploy-docs (push) Successful in 28s
deb / build-publish (push) Successful in 17m6s
ci / rust (push) Failing after 19m1s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m3s
flatpak / build-publish (push) Successful in 8m0s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m14s
release / apple (push) Successful in 5m44s
Wholesale commit of every uncommitted change across the tree, at the user's
explicit request — host refactor-campaign W1 (native.rs facade + native/ dir,
library/ + mgmt/ splits), Android, core. These streams were mid-flight and not
individually built/tested together; this supersedes the per-session HOLD
markers. Consolidating so everything lands on main in one pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 20:08:29 +02:00
enricobuehler 07e2836601 feat(apple/M1): PunktfunkWidgetsExtension target — wired, signed, building
The Xcode widget-extension target that hosts the launcher widget + Live Activity
UI. Bundle id io.unom.punktfunk.widgets, iOS 17, App Group group.io.unom.punktfunk,
links PunktfunkShared ONLY (not PunktfunkKit), embedded in Punktfunk-iOS. Sources
come from the PunktfunkWidgets/ synchronized folder. Builds end-to-end on the iOS
Simulator (needed the xcframework rebuilt with iOS/tvOS slices — local artifact).

- project.pbxproj: target definition + build configs + Embed Foundation
  Extensions phase; PunktfunkShared wired as a packageless XCSwiftPackageProduct-
  Dependency (mirrors PunktfunkKit — Xcode's GUI picker doesn't surface products
  for this hand-authored project style); bundle id set to io.unom.punktfunk.widgets.
- PunktfunkWidgetsExtension.entitlements: App Group only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 19:54:12 +02:00
enricobuehler 6ac7134e7c fix(apple/M4): IntentError message must be a string literal
LocalizedStringResource is ExpressibleByStringLiteral, so a single literal
converts implicitly, but the "…" + "…" concatenation is a runtime String it
can't convert. Collapsed to one literal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 19:52:34 +02:00
enricobuehler 6d2e738070 fix(apple/M3): import AppIntents for Button(intent:) + drop deprecated Text+
The Live Activity's End button uses Button(intent:), whose initializer lives in
_AppIntents_SwiftUI — reached via `import AppIntents` (was missing, so the
widget target failed to build). Also replaced the iOS-26-deprecated Text + Text
concatenation in the background countdown with an HStack.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 19:49:52 +02:00
enricobuehler 09e2043ce0 chore(apple/M1): move widget sources into the Xcode target's synced folder
Xcode created the PunktfunkWidgetsExtension target with a file-system-
synchronized root group at clients/apple/PunktfunkWidgets/, so the target
compiles whatever lives there. Deleted the three generated stubs
(PunktfunkWidgets.swift / PunktfunkWidgetsBundle.swift /
PunktfunkWidgetsControl.swift — the stub @main WidgetBundle would collide with
ours) and moved our sources (PunktfunkWidgetBundle / HostsWidget /
SessionLiveActivity) from Sources/PunktfunkWidgets/ into PunktfunkWidgets/. Kept
the generated Info.plist (build-excluded via the sync exception set) and
Assets.xcassets. Still outside Sources/, so SwiftPM ignores it; swift build green.

project.pbxproj is intentionally NOT part of this commit — the target's
capability/signing edits (step 3) are still in progress in Xcode.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 19:18:26 +02:00
enricobuehler a513186424 fix(apple/M3): reach shared Activity types via PunktfunkKit re-export
SessionActivityController is in the app target, which links the PunktfunkKit
product (not PunktfunkShared directly). Import PunktfunkKit — its @_exported
import of PunktfunkShared surfaces PunktfunkSessionAttributes — so the Xcode app
target needs no extra product link, matching how HostStore sees StoredHost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 18:53:39 +02:00
enricobuehler df6a5325d8 feat(apple/M1+M3+M4): widgets, Live Activity, and Siri/Shortcuts intents
The extension-side + App Intents surface for design/apple-live-activities-and-
widgets.md. The iOS-framework code (WidgetKit/ActivityKit/AppIntents) can't be
compiled by the macOS `swift build` CI target and needs the Xcode widget-
extension target that only exists once created in the GUI — see the checklist in
the memory note. What macOS DID verify: HostEntity (AppIntents is available on
macOS), the shared attribute/notification plumbing, and that nothing regressed
(142 tests green).

Shared (PunktfunkShared):
- PunktfunkSessionAttributes (ActivityAttributes) — the one type app + extension
  share; gated os(iOS) (ActivityKit imports on macOS but its types are
  unavailable, so canImport would wrongly admit it).
- EndStreamIntent (LiveActivityIntent) — posts .punktfunkEndActiveSession.
- HostEntity + HostEntityQuery (AppEntity over the shared store) — the intent /
  widget-config parameter type; canImport(AppIntents), so macOS type-checks it.
- New notifications: end-active-session, open-deep-link.

M1 widget extension sources (Sources/PunktfunkWidgets/, NOT a SwiftPM target —
`swift build` ignores the dir):
- PunktfunkWidgetBundle (@main): HostsWidget + PunktfunkSessionLiveActivity.
- HostsWidget (kind "PunktfunkHosts"): reads the shared-suite store, sorts by
  recency, deep-links each host; small/medium/accessory families; empty state.
- SessionLiveActivity: Lock-Screen banner + Dynamic Island (elapsed timer,
  mode line, background countdown, End button).

M3 controller (app, iOS): SessionActivityController owns the Activity lifecycle
(request/update/end + launch orphan-sweep + staleDate); ContentView drives it
from the model's phase/isBackgrounded/backgroundDeadline (which SessionModel now
publishes), keeping ActivityKit out of the cross-platform model.

M4 (app, iOS): ConnectToHost/WakeHost intents + AppShortcutsProvider; Connect
routes via .punktfunkOpenDeepLink into the same onOpenURL router (one set of
guards); Wake reuses the WoL path; End surfaced to Shortcuts too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 18:51:41 +02:00
enricobuehler 14c5e7c11c feat(apple/M2): opt-in background keep-alive (audio + video-drop + timeout)
Backgrounding a live session no longer freezes it when the user opts in: audio
keeps playing (UIBackgroundModes audio), the QUIC connection + pump stay live,
video decode is DROPPED, and a bounded timer auto-disconnects. Off by default.

- PunktfunkConnection.setVideoDropped/isVideoDropped: a tiny lock-guarded flag
  both pumps read every iteration. StreamPump (stage-1), Stage2Pipeline (VT +
  PyroWave) drain nextAU() for flow control but DISCARD the AU before any
  VideoToolbox/Metal work — the crash/jetsam-safe seam (no GPU off-screen).
- SessionModel.enterBackground(timeoutMinutes:) / exitBackground(): set the drop
  flag, mute the mic (privacy — SessionAudio.setMicMuted pauses the capture
  engine), arm a DispatchSourceTimer that disconnect(deliberate:false)s on fire
  (keeps host linger → fast late reconnect). exitBackground clears the flag and
  requestKeyframe()s; the pump's freeze gate auto-arms on the resumed
  frame-index gap so concealed frames are withheld until the IDR re-anchors.
  disconnect() cancels the timer + clears isBackgrounded.
- ContentView scenePhase driver (iOS): .background+streaming+setting →
  enterBackground; .active → exitBackground. scenePhase (not willResignActive)
  so Control-Center/app-switcher peeks don't start the timer.
- Settings → General (iOS-only keepAliveSection): toggle + 1/5/10/30 timeout;
  new keys backgroundKeepAlive (def off) / backgroundTimeoutMinutes (def 10).
- Info.plist: UIBackgroundModes [audio] + NSSupportsLiveActivities (for M3).

macOS swift build + swift test green (142 tests). The iOS-gated scenePhase
handler + settings section are not exercised by the macOS CI target (known §9
gap) — need on-glass verification (audio never gaps, video re-anchors <1s LAN,
timeout ends the session, phone-call audio-steal degrades gracefully).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 18:41:46 +02:00
enricobuehler 4cae1b8bb8 feat(apple/M0): App Group + PunktfunkShared + punktfunk:// deep links
Foundation milestone for Live Activities & Widgets (design/apple-live-
activities-and-widgets.md). No user-visible change beyond the URL scheme.

- New dependency-free PunktfunkShared SwiftPM target (+ library product) so a
  future widget extension can link it WITHOUT PunktfunkKit (Rust staticlib +
  presentation layer). Moves StoredHost (model + JSON codec), DefaultsKeys, and
  punktfunkDefaultMgmtPort there; adds AppGroup.suiteName and the punktfunk://
  DeepLink builder/parser. PunktfunkKit @_exported-imports it (no call-site
  churn for consumers; intra-Kit files import it explicitly since imports are
  file-scoped).
- HostStore reads/writes the shared App-Group suite (group.io.unom.punktfunk)
  with a one-time migration from UserDefaults.standard (old value left in place
  for staged rollout); reloads the "PunktfunkHosts" widget timeline on change.
- App Group entitlement on iOS/tvOS + macOS.
- CFBundleURLTypes scheme `punktfunk`; ContentView.onOpenURL routes
  connect/<uuid>[?launch=<GameEntry.id>] into the existing connect() path
  (unknown host / already-streaming guards; never tears down a live session).
- Round-trip tests: StoredHost JSON codec (+ legacy missing-optional decode),
  DeepLink grammar. `swift build` + `swift test` green (142 tests, 0 failures).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 18:16:39 +02:00
enricobuehler b45323c0be fix(host/windows): force off a game-abandoned rumble on the UMDF virtual pads
The DualSense / DualSense Edge / DualShock 4 / Switch Pro / Steam Deck backends
all run through UhidManager, whose pump() forwarded rumble only on a level
CHANGE and had no idle watchdog. A game that latches a one-shot rumble (a
Stardew axe hit, a DS3 hit) and then stops writing output reports left
last_rumble non-zero; native.rs re-sends the latched level every ~120 ms with a
fresh TTL and the Apple RumbleRenderer refreshes its envelope on every renewal,
so the controller vibrated continuously until a later event happened to write a
report the host parsed as a stop. The XUSB path already guards against this
(RUMBLE_IDLE_TIMEOUT force-off, 19e9828e); that guard was never ported here, so
every UMDF pad regressed for game-abandoned rumble once clients began
negotiating first-class virtual DualSense/DS4/etc. on Windows.

Port the guard into UhidManager::pump, keyed on game ACTIVITY (a fresh output
report, even at an unchanged level) so a rumble the game keeps asserting is
never cut — only an abandoned residual. The activity signal rides a new
PadFeedback.game_drove: Option<bool>; the Windows backends set it from a fresh
out_seq (via a `fresh` flag on DsFeedback/Ds4Feedback; the Deck uses is_some()).
Linux backends leave it None (untracked → always-active → the force-off never
fires there), so their behaviour is unchanged. +2 deterministic unit tests.

Verified: cargo check -p punktfunk-host --tests green on both Windows (.173) and
Linux (home-worker-5); the 10 inject::uhid_manager tests pass on Linux.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 17:45:29 +02:00
enricobuehler 1a7e3a6e4f fix(host/windows): propagate XUSB devnode-create failure instead of latching a phantom pad
XusbWinPad::open swallowed a SwDeviceCreate failure — it returned Ok with
`_sw: None` (a pad with no devnode) and logged only a warn, so PadSlots latched a
phantom pad, called gate.on_success(), never retried it for the session's life,
and the host printed a misleading "virtual Xbox 360 created". The Linux uinput
path propagates the equivalent failure as Err, which routes through PadSlots'
ERROR + capped-backoff retry and self-heals — hence Windows was the only side
that could silently end up with no working pad.

Propagate the create failure with `?` so Windows gets the same ERROR + backoff
retry as Linux. Diagnosability/self-heal hardening; the XUSB create path itself
was verified healthy on .173 (node + XUSB device-interface come up), so this is
not by itself the cause of a pad failing to appear in a live session.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 17:45:27 +02:00
enricobuehler 4ffa2665ac refactor(host): extract src/devtest.rs — the standalone dev/test subcommands
Per plan §W5 (main-cli, 'devtest.rs land first'): move the inline-bodied dev/test
subcommand handlers out of main.rs's match into src/devtest.rs — input_test (Linux
libei/wlr injection smoke test + its non-Linux stub) and the virtual-gamepad
exercisers dualsense_test/switchpro_test (Linux UHID) and deck_windows_spike/
dualsense_windows_test (Windows UMDF + Steam Deck devnode spike). main.rs's arms
become one-line forwards; main.rs drops 1004→667 lines. The thin arms that already
forward to subsystem modules (zerocopy/capture selftests, probes) stay put — that
is their correct layer. Pure code-move (bodies verbatim; crate-local refs
qualified with crate::; one doc reword to dodge clippy doc_lazy_continuation now
that an arm comment became a /// doc).

Verified clippy 0/0 on BOTH Linux (home-worker-5, nvenc,vulkan-encode,pyrowave)
and Windows (.173, nvenc,amf-qsv — covers the cfg(windows) handlers).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 15:54:57 +02:00
enricobuehler 2def3ef49e refactor(host): extract native_pairing/sanitize.rs — the untrusted-name scrubber
Per plan §W5: move sanitize_device_name (+ its NAME_MAX cap and unit test) out of
the native_pairing facade into native_pairing/sanitize.rs. It is a self-contained,
security-relevant leaf — the one place a wire-supplied unpaired-device name is
scrubbed of control chars / bidi-override spoofing before it is stored, listed,
logged, or shown in the approval UI. Re-export via `pub(crate) use` so
crate::native_pairing::sanitize_device_name stays stable (punktfunk1 accept loop +
the two in-crate callers). Pure code-move; verified host clippy 0/0 + 11
native_pairing tests green on Linux (home-worker-5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 15:42:55 +02:00
enricobuehler 571e22bc0f refactor(core): consolidate the fingerprint-pinning verifier into core::tls
Per plan §2.5: the security-critical rustls fingerprint-pinning ServerCertVerifier
was hand-rolled three times — quic/endpoint.rs (PinVerify), pf-client-core
library.rs, punktfunk-tray status.rs — drifting copies on a trust boundary. Add
one canonical punktfunk_core::tls::PinVerify (+ cert_fingerprint) behind a light
`tls` feature (rustls + sha2 only, no QUIC runtime); `quic` now depends on it, and
quic::endpoint re-exports cert_fingerprint so that path stays byte-stable
(gamestream + pf-client-core reach it there).

- core::tls::PinVerify: new(pin) for the HTTP clients, with_observed(pin, slot)
  for the QUIC TOFU connect. Behavior-identical to all three originals (pin-check
  + real CertificateVerify signature verification; only hashes the leaf when a pin
  or observed slot needs it). Two focused unit tests anchor the boundary.
- quic/endpoint.rs: drop the private PinVerify, wire client_pinned through
  tls::PinVerify::with_observed.
- pf-client-core library.rs + tray status.rs: use the shared verifier; tray also
  routes load_pin through core cert_fingerprint and drops its direct sha2 dep,
  gaining only the light core `tls` feature (still no host dep, no QUIC runtime).

Verified on Linux (home-worker-5): clippy 0/0 for core(quic), core(tls),
pf-client-core, tray, host(nvenc,vulkan-encode,pyrowave); core 153 lib tests +
loopback 7/7 (pinned handshake) + c_abi round-trip green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 15:36:29 +02:00
enricobuehler ce085b8e3b style(vdisplay): dedupe the attach-block comment (first-frame stash follow-up)
Comment-only: the lazy-attach comment carried the delivery-consumption
sentence twice after the stash rework.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 15:24:36 +02:00
enricobuehler 405b005a0d refactor(host): extract audio/mic_pump.rs — the host-lifetime virtual-mic pump
Per plan §2.1: a self-contained stateful subsystem does not belong in the
audio trait facade. Move MicPump + its PumpTuning/PUMP_TUNING, the
drain_sleep/pump_thread loop, MIC_CHANNELS/MIC_QUEUE_CAP, and the six pump
unit tests out of audio.rs into audio/mic_pump.rs. audio.rs keeps the
AudioCapturer/VirtualMic traits, their open_* factories, and the sample
constants. Re-export via `pub use mic_pump::MicPump` so crate::audio::MicPump
stays byte-stable (only consumer: punktfunk1.rs). Pure code-move; verified
clippy 0/0 + 6/6 pump tests green on Linux (home-worker-5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 15:21:36 +02:00
enricobuehler 89a08f83af fix(vdisplay): first-frame guarantee — republish a retained frame at ring attach
DWM composes a display only when something dirties it, so a session opened
onto an idle desktop never produced a first frame: the host's synthetic-input
"compose kick" (cursor wiggle / sibling-display jump) was the only source, and
it is inherently unreliable — blocked on the secure desktop, defeated by a
fullscreen game's ClipCursor, user-visible, and dead in service contexts. The
field symptom: connect → black stream until something repaints the desktop.

Reconstruct DDA's first-frame semantics at the driver instead (DDA seeds a new
duplication with the current desktop image; IDD-push never had an equivalent):

* frame_transport.rs: new FrameStash — the retained last composed frame, a
  driver-private copy-only texture. publish() now reports Published /
  DescMismatch / Dropped, and harvest_into() pulls the last-published ring
  slot into the stash (keyed-mutex guarded, freshness-checked) before a
  superseded publisher is dropped — between sessions the driver keeps writing
  the host-side-dead old ring, so that slot IS the current desktop image.
* swap_chain_processor.rs: the worker stashes every frame the ring can NOT
  take (unattached, or descriptor-mismatched during a mode/HDR-flip race),
  harvests before a supersede, and REPUBLISHES the stash into every freshly
  attached ring — the host sees a normal seq=1 publish milliseconds after
  channel delivery, no compose needed. Zero steady-state cost: matched
  publishes touch only the ring. The frame-channel stash is now polled every
  iteration (attach latency = first-frame latency; it was 1-in-30).
* monitor.rs: preserved_stash (LUID-tagged) so the retained frame survives
  swap-chain unassign→reassign flaps, alongside the preserved publisher.
* host idd_push.rs: kick_dwm_compose demoted to documented last-resort
  fallback for pre-stash drivers; a debug log now fires when a kick actually
  runs so field logs show whether the stash path is working.

No proto change: the republish is an ordinary publish, so old host + new
driver and new host + old driver both keep working (the latter via the kick).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 15:16:36 +02:00
enricobuehler 5748706631 refactor(host): hoist the direct-NVENC init-params authoring into nvenc_core
Both backends' build_init_params authored an identical NV_ENC_INITIALIZE_PARAMS
(P1/ULL preset, PTD, session dimensions/rate, split-encode mode) — the only
difference was the Windows-only enableEncodeAsync flag (Linux is sync-only).
Hoist it to nvenc_core::build_init_params(codec_guid, w, h, fps, cfg, split_mode,
enable_async); Linux's two call sites pass enable_async=false (the field stays 0
as before), Windows passes its session_async through. Keeps open and in-place
reconfigure presenting the SAME init params, now guaranteed identical across
platforms too.

Verified on BOTH platforms: Linux clippy 0/0 (nvenc,vulkan-encode,pyrowave, RTX
5070 Ti) and Windows clippy 0/0 (nvenc,amf-qsv, RTX 4090 / .173).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 14:58:03 +02:00
enricobuehler 2ae5cf98ee refactor(host): hoist the direct-NVENC low-latency config into nvenc_core
Both direct-SDK NVENC backends authored a near-identical NV_ENC_CONFIG in
build_config: CBR + infinite GOP + P-only + ~1-frame VBV, per-codec tier/level,
chroma + bit depth, unconditional colour VUI, and the RFI DPB — ~125 lines each,
differing only in comments plus two genuinely per-platform bits (which surface
formats carry full chroma / 10-bit input). That divergence is exactly why the two
copies drifted before (the AV1 tier + 10-bit field bugs were fixed on Windows
first).

Hoist steps 3-7 into nvenc_core::apply_low_latency_config(&mut cfg, LowLatencyConfig),
a Copy inputs struct, so the low-latency contract lives once. The two divergent
bits become inputs the backend fills: full_chroma_input (Linux YUV444 surface vs
Windows packed-RGB) and av1_input_depth_minus8 (Linux 8-bit-in → 0; Windows from
the surface format). Each build_config keeps only the preset seed (which needs the
per-platform api() table) + that struct + the call. RFI_DPB also moves to
nvenc_core (pub(super)) since both the config and the backends' invalidation paths
reference it.

Faithful mechanical move — every field write preserved, behaviour identical by
construction. Verified on BOTH platforms: Linux clippy 0/0 (nvenc,vulkan-encode,
pyrowave, RTX 5070 Ti) and Windows clippy 0/0 (nvenc,amf-qsv, RTX 4090 / .173).
Net -83 lines.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 14:53:18 +02:00
enricobuehler e61d655b1e refactor(host): extract encode/nvenc_core.rs — shared direct-SDK NVENC leaves
The two direct-SDK nvEncodeAPI backends (Windows D3D11 encode/windows/nvenc.rs,
Linux CUDA encode/linux/nvenc_cuda.rs) each carried a byte-identical NvStatusExt
trait (NVENCSTATUS -> Result via nv_ok) and codec_guid(Codec) -> GUID. Hoist
both into a new encode/nvenc_core.rs, the platform-agnostic sibling of the
existing encode/nvenc_status.rs (same cfg gate: any(linux,windows) + nvenc).
Each backend now imports them via super::nvenc_core; call sites (.nv_ok() ×16/20,
the one codec_guid() struct-init) are unchanged.

The per-platform machinery — entry-table load (nvEncodeAPI64.dll/LoadLibrary vs
libnvidia-encode.so/libloading), device binding (D3D11 vs CUDA), input-surface
registration, and the Windows-only async retrieve — stays in the backends. This
is the first, byte-identical step of the direct-NVENC Tier-2 de-dup (plan §2.2);
the larger build_config authoring is a later, carefully-diffed step.

Verified on BOTH platforms: Linux clippy 0/0 (nvenc,vulkan-encode,pyrowave, RTX
5070 Ti) and Windows clippy 0/0 (nvenc,amf-qsv, RTX 4090 / .173).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 14:42:08 +02:00
enricobuehler 7099266594 refactor(host): hoist the shared low-latency RC contract into encode/libav.rs
The three libavcodec backends each set the identical low-latency rate-control
block on the not-yet-opened encoder context: fixed time_base/frame_rate, CBR
(bit_rate == max_bit_rate), B-frames off, and a tight ~1-frame VBV/HRD buffer
written through the raw rc_buffer_size field. Move it once into
apply_low_latency_rc(&mut video, fps, bitrate_bps), and let the long VBV
rationale (why the tight buffer prevents high-motion bursts from overflowing
the send queue) live in one place instead of only in the NVENC path.

Each backend keeps the two genuinely per-backend calls around it: set_format
(pixel format differs) before, and gop_size after (NVENC's infinite/intra-
refresh wave vs the VAAPI/AMF i32::MAX). No behavior change — the field writes
are independent, so the slightly different max_b_frames/rc_buffer_size ordering
across backends is irrelevant. Folding the raw rc_buffer_size write into the
helper also removes the NVENC path's separate unsafe block. Drops the now-unused
ffmpeg::Rational import from all three.

Linux check + clippy green (0/0, nvenc,vulkan-encode,pyrowave) on RTX 5070 Ti;
ffmpeg_win.rs is Windows-cfg, pending .173 compile.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 13:23:28 +02:00
enricobuehler fd8a062e2c refactor(host): hoist the libav poll_encoder drain + PollOutcome into encode/libav.rs
The three libavcodec backends each carried a byte-identical single-packet
receive_packet drain. Move it once into the shared Tier-2 glue as
poll_encoder -> PollOutcome (the richest form: Packet / Again / Eof), and
have the callers narrow it:

- Linux NVENC (encode/linux/mod.rs): poll() matches the shared fn, collapsing
  Again|Eof to Ok(None) — was an inlined match, now one call.
- VAAPI (encode/linux/vaapi.rs): drop the local poll_encoder; the blocking
  budget loop lets Again|Eof fall through to the deadline check, byte-identical
  to the old Option::None path.
- Windows AMF/QSV (encode/windows/ffmpeg_win.rs): drop the local PollOutcome +
  poll_encoder; its deadline-driven drain already matches PollOutcome, so only
  the import changes.

No behavior change on any backend. Still a plain monomorphic free fn over a
borrowed &mut Encoder — no new per-frame dyn/Box/alloc; the only allocation is
the same bitstream to_vec() each path already made. Drops the now-unused
ffmpeg::Packet import from all three.

Linux check + clippy green (nvenc,vulkan-encode,pyrowave) on RTX 5070 Ti;
ffmpeg_win.rs is Windows-cfg, pending .173 compile.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 13:16:45 +02:00
enricobuehler 3c38a5f0e8 refactor(host): hoist shared libav glue into encode/libav.rs (pixel_to_av + swscale consts)
First step of the W2 libav de-dup (plan §2.2, the missing Tier-2 mid-layer). The
three libavcodec backends (Linux NVENC, VAAPI, Windows AMF/QSV) each carried a
byte-identical pixel_to_av plus the SWS_POINT / SWS_CS_ITU709 (/SWS_CS_BT2020)
swscale consts. Hoist them into a new encode/libav.rs and import from super::libav.

The module is gated to compile exactly when a libav backend does (linux, or
windows+amf-qsv). Free fns/consts over borrowed handles — no per-frame dyn/alloc,
off the zero-copy path. Verified: Linux cargo check green (linux/mod.rs + vaapi.rs
compile against it); ffmpeg_win.rs is Windows-cfg — same mechanical swap, covered
by Windows CI on push.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 13:04:52 +02:00
enricobuehler 1f519d44f9 docs(core): backfill //! module docs on quic/{io,endpoint,pake}
The last three //!-less modules in the tree (plan §2.5 / §3.2):
  - io:       length-prefixed control-message framing (read_msg/write_msg)
  - endpoint: QUIC endpoint construction + transport tuning + the TOFU
              cert-pinning verifier (PinVerify)
  - pake:     SPAKE2 pairing key exchange

Docs only — no code, type, or wire-format change (cbindgen header byte-identical).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 12:54:46 +02:00
enricobuehler abecb5226c refactor(host): route the three libav backends' VBV re-parse through vbv_frames_env
The libavcodec paths (Linux NVENC, VAAPI, Windows QSV) each re-parsed
PUNKTFUNK_VBV_FRAMES locally in f32, duplicating and diverging in precision from
the f64 vbv_frames_env() helper the direct-NVENC/AMF paths already use. Now that
the helper lives in encode/codec.rs (532b313b), route all three through
crate::encode::vbv_frames_env(): one parse, one precision, no drift.

Behaviour-identical (same filter finite && > 0, same 1.0 default), f64 not f32.
Verified: Linux cargo check green (linux/mod.rs + vaapi.rs compile); ffmpeg_win.rs
is Windows-cfg and mirrors the amf.rs/nvenc.rs sites already using the helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 12:48:26 +02:00
enricobuehler 11045a0f70 chore: consolidate parallel-session WIP (HOLD — do not push)
Local snapshot of intermingled in-flight work, committed to unblock the encode
refactor (a clean ffmpeg_win.rs for the vbv-dedup follow-on). These hunks span
the same files and can't be cleanly split here; the commit bundles three
distinct workstreams that each belong in their own PR:

  - logging rework (~43 files: level re-tiering, structured fields, `?e`,
    hot-path flood latches)
  - conflicting-host detection (detect.rs + detect/{linux,windows}.rs + wiring
    in main.rs/mgmt.rs/Cargo.toml/docs/packaging)
  - standby-sink DWM-stall attribution (windows/display_events.rs + capture/
    vdisplay wiring)

NOT verified as a combination. NOT to be pushed until the refactor is done and
these are re-verified and reorganized into their proper per-workstream PRs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 12:42:53 +02:00
enricobuehler d466e3e2b2 chore(nix): NixOS flake snapshot — host+client packages, module, devShell (WIP)
Local snapshot of the in-flight NixOS support: flake.nix + flake.lock + nix/
(crane host and client packages, services.punktfunk module, devShell).
Standalone — nothing in the Rust/Cargo tree references it. Held from push
pending its owning session's finalization (Skia-under-Nix follow-up + intended
per-workstream PR).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 12:42:34 +02:00
enricobuehler 532b313b8c refactor(host): extract encode/codec.rs — the encoder contract
Move the Tier-1 encoder contract out of the stuffed encode.rs facade into a
new encode/codec.rs submodule (plan §7 / W2): EncodedFrame, Codec (all methods
except host_wire_caps), ChromaFormat, EncoderCaps, the Encoder trait,
validate_dimensions, vbv_frames_env, and the dimension + wire-roundtrip contract
tests. host_wire_caps stays in encode.rs alongside the backend-selection probes
it depends on; CodecSupport and its wire-mask test stay too.

encode.rs gains `mod codec;` + `pub(crate) use codec::*;` so every existing
crate::encode::X path — crate::encode::vbv_frames_env, ::Codec, ::Encoder, … —
stays byte-stable. Pure relocation: no call sites touched.

Verified: dev-Mac type-check of both files clean; Linux `cargo check -p
punktfunk-host --features nvenc,vulkan-encode,pyrowave` green (all encode
backends compile against the relocated contract); contract unit tests pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 12:21:49 +02:00
enricobuehler d381cdf7f4 fix(host): NVENC open-failure resilience — backoff, failed-open hygiene, self-diagnosis
ci / web (push) Successful in 49s
ci / docs-site (push) Successful in 51s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
apple / swift (push) Successful in 1m9s
decky / build-publish (push) Successful in 18s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 7s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 5m48s
docker / deploy-docs (push) Successful in 24s
apple / screenshots (push) Successful in 5m18s
windows-host / package (push) Successful in 9m16s
arch / build-publish (push) Successful in 10m53s
android / android (push) Successful in 11m58s
deb / build-publish (push) Successful in 18m7s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m19s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m28s
ci / rust (push) Successful in 24m56s
Field report (Linux direct NVENC): after a codec switch, every session open
failed with NV_ENC_ERR_INVALID_VERSION until the host process was restarted —
so the poisoned state is per-process, not a driver install issue. On-hardware
investigation (RTX 5070 Ti, 610.43.03) could not reproduce it with clean codec
cycles, dirty teardowns, or open/destroy storms, but established the failure
class: the driver enforces a per-process concurrent-session cap (12 there,
status INCOMPATIBLE_CLIENT_KEY; other branches report differently) whose
exhaustion is exactly this signature — persistent open failures healed only by
a process restart. Harden every path that can feed or mask that state:

* Rebuild backoff: the in-place encoder-rebuild retries slept one frame
  interval, so all 5 attempts burned within ~40 ms at 120 Hz — no driver-side
  transient (deferred teardown of the previous session, engine reset) can
  clear that fast. Exponential backoff 100 ms → 1.6 s (~3 s total) so
  transients heal instead of killing the session.
* Destroy-on-failed-open (Linux + Windows, all four open sites): the NVENC
  docs require NvEncDestroyEncoder even when OpenEncodeSessionEx FAILS — the
  driver may have allocated the session slot before erroring. Without it a
  retry burst against a transient leaks slots toward the cap, converting the
  transient into permanent exhaustion.
* Teardown: a destroy_encoder failure (a session slot the driver may keep) is
  now logged with its status instead of silently discarded.
* One-shot self-diagnosis on a failed session open (Linux): retry the raw open
  on a fresh dedicated CUDA context and log which of the three causes applies
  — shared-context poisoned (fresh works), driver-level skew/exhaustion/GPU
  loss (fresh fails the same way), or CUDA itself unhealthy (no fresh context)
  — so the next field report pinpoints the root cause with zero reporter
  effort.

On-hardware regression tests (RTX box .21, all green): codec-switch reopen
cycle (H265→AV1→H265→H264→H265), dirty teardown with in-flight encodes, and
the full open-failure→diagnosis→in-place-recovery path via real session-cap
exhaustion. Existing RFI/reconfigure/4:4:4 smokes still pass; clippy clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 01:57:36 +02:00
enricobuehler f901bedf22 fix(host): actionable NVENC error logging — drop misleading "(no NVIDIA GPU?)"
ci / web (push) Successful in 53s
ci / docs-site (push) Successful in 1m2s
apple / swift (push) Successful in 1m10s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 40s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 56s
apple / screenshots (push) Successful in 5m24s
ci / bench (push) Successful in 7m32s
docker / deploy-docs (push) Successful in 28s
arch / build-publish (push) Successful in 11m51s
windows-host / package (push) Successful in 14m53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m52s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m54s
android / android (push) Successful in 17m12s
deb / build-publish (push) Successful in 18m12s
ci / rust (push) Successful in 24m52s
Every NVENC entry-point failure was annotated "(no NVIDIA GPU?)", which
misled triage: the direct-NVENC path only loads on a machine that HAS an
NVIDIA GPU. A Linux user hit NV_ENC_ERR_INVALID_VERSION at
open_encode_session_ex (past the NvEncodeAPIGetMaxSupportedVersion pre-flight
gate) — the signature of a userspace/kernel driver version skew that a host
reboot fixes — and the log pointed at a missing GPU instead. A restart did
fix it.

Add encode/nvenc_status.rs: a shared NVENCSTATUS -> cause mapper that folds
the real cause into the anyhow::Error at construction, so every downstream
{e:#} log (the encode-recovery loop, session teardown) improves for free.
INVALID_VERSION now reads "update the NVIDIA driver, or reboot if you just
updated it (a host restart is the usual fix)"; NO_ENCODE_DEVICE /
DEVICE_NOT_EXIST / INCOMPATIBLE_CLIENT_KEY (session-count limit) / OOM /
UNSUPPORTED_PARAM get their own glosses. The required API version comes from
the SDK consts so it stays correct across crate bumps.

Wire it into all NVENC entry-point failures in both backends
(encode/linux/nvenc_cuda.rs, encode/windows/nvenc.rs) — every open, init,
preset/resource/bitstream call.

Also: when the encode-recovery loop exhausts its in-place rebuilds it now
logs a clear terminal line with the underlying cause instead of the session
silently vanishing after the last identical "rebuilt in place" line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 01:13:53 +02:00
271 changed files with 37105 additions and 17619 deletions
+5
View File
@@ -38,3 +38,8 @@ CLAUDE.md
# Local flatpak-builder output (build-flatpak.sh) — ostree repo + build dir at the repo root.
.flatpak-repo/
.flatpak-build/
# Nix build outputs (flake.nix) — `nix build` result symlinks + direnv cache. flake.lock IS tracked.
/result
/result-*
.direnv/
Generated
+20 -1
View File
@@ -2778,6 +2778,23 @@ dependencies = [
"windows 0.62.2 (git+https://github.com/microsoft/windows-rs?rev=a4f7b2cb7c63c6bb7fc77a2affe57145be1d8c4f)",
]
[[package]]
name = "pf-clipboard"
version = "0.12.0"
dependencies = [
"anyhow",
"ashpd",
"futures-util",
"libc",
"punktfunk-core",
"quinn",
"tokio",
"tracing",
"wayland-client",
"wayland-protocols",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "pf-console-ui"
version = "0.12.0"
@@ -3106,6 +3123,7 @@ dependencies = [
"ffmpeg-next",
"futures-util",
"hex",
"hmac",
"http-body-util",
"hyper",
"hyper-util",
@@ -3120,6 +3138,7 @@ dependencies = [
"openh264",
"opus",
"parking_lot",
"pf-clipboard",
"pf-driver-proto",
"pipewire",
"punktfunk-core",
@@ -3184,10 +3203,10 @@ dependencies = [
"anyhow",
"ksni",
"libc",
"punktfunk-core",
"rustls",
"serde",
"serde_json",
"sha2",
"ureq",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"windows-service",
+1
View File
@@ -6,6 +6,7 @@ members = [
"crates/punktfunk-host/vendor/usbip-sim",
"crates/punktfunk-tray",
"crates/pf-client-core",
"crates/pf-clipboard",
"crates/pf-presenter",
"crates/pf-console-ui",
"crates/pf-ffvk",
+986 -3
View File
File diff suppressed because it is too large Load Diff
@@ -1,6 +1,8 @@
package io.unom.punktfunk
import android.content.Context
import android.os.Build
import android.util.Log
import android.view.Display
/**
@@ -249,11 +251,25 @@ fun nativeDisplayMode(context: Context): Triple<Int, Int, Int> {
*/
fun displaySupportsHdr(context: Context): Boolean {
val display = runCatching { context.display }.getOrNull() ?: return false
@Suppress("DEPRECATION") // hdrCapabilities is the supported query on minSdk 31
val caps = display.hdrCapabilities ?: return false
return caps.supportedHdrTypes.any {
val types = buildSet {
// API 34+: the sanctioned per-mode query (Display.Mode.getSupportedHdrTypes). The
// deprecated Display-level hdrCapabilities can return EMPTY on Android 14+ devices
// (Pixel-class panels included), which would make a genuinely HDR display advertise
// no-HDR and pin the whole session to 8-bit SDR.
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
display.mode.supportedHdrTypes.forEach { add(it) }
}
// Union the legacy query defensively — the supported one on minSdk 31, and some vendors
// populate only this on newer APIs.
@Suppress("DEPRECATION")
display.hdrCapabilities?.supportedHdrTypes?.forEach { add(it) }
}
// HDR10/HDR10+ only: the stream is BT.2020 PQ — a Dolby-Vision/HLG-only panel can't present it.
val supported = types.any {
it == Display.HdrCapabilities.HDR_TYPE_HDR10 || it == Display.HdrCapabilities.HDR_TYPE_HDR10_PLUS
}
Log.i("punktfunk", "display HDR types=$types → advertise HDR10=$supported")
return supported
}
/** Resolve [Settings] (with its 0=native placeholders) to the concrete mode to request. */
@@ -39,6 +39,7 @@ internal fun StatsOverlay(
s: DoubleArray,
verbosity: StatsVerbosity,
decoderLabel: String = "",
codecLabel: String = "",
modifier: Modifier = Modifier,
) {
if (verbosity == StatsVerbosity.OFF || s.size < 10) return
@@ -66,7 +67,7 @@ internal fun StatsOverlay(
statLine(decoderLabel, Color(0xFFB0D0FF))
}
if (detailed) {
videoFeedLine(s)?.let { statLine(it, Color.White) }
videoFeedLine(s, codecLabel)?.let { statLine(it, Color.White) }
}
if (latValid) {
// Display stage (s[22]s[25], from OnFrameRendered): when a render timestamp landed
@@ -151,14 +152,15 @@ private fun counterLine(s: DoubleArray, lostTotal: Long): String? {
}
/**
* Format the negotiated video-feed descriptor from the trailing four stats doubles
* `[bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`, e.g.
* `HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0`. Returns `null` on a pre-video-feed layout (< 14 doubles)
* Format the negotiated video-feed descriptor from [codecLabel] plus the trailing four stats
* doubles `[bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`, e.g.
* `AV1 · 10-bit · HDR (BT.2020 PQ) · 4:2:0`. Returns `null` on a pre-video-feed layout (< 14 doubles)
* so the overlay simply omits the line. The codes are CICP / H.273: transfer 16 = PQ, 18 = HLG (else
* SDR); primaries 9 = BT.2020, 1 = BT.709; chroma_format_idc 1 = 4:2:0, 2 = 4:2:2, 3 = 4:4:4. The
* Android decoder is always HEVC (`video/hevc`).
* SDR); primaries 9 = BT.2020, 1 = BT.709; chroma_format_idc 1 = 4:2:0, 2 = 4:2:2, 3 = 4:4:4.
* [codecLabel] is the host-resolved codec (`nativeVideoCodecLabel`); a blank one falls back to
* `HEVC` (the pre-negotiation default) for the brief window before it's resolved.
*/
private fun videoFeedLine(s: DoubleArray): String? {
private fun videoFeedLine(s: DoubleArray, codecLabel: String): String? {
if (s.size < 14) return null
val bitDepth = s[10].toInt()
val primaries = s[11].toInt()
@@ -175,5 +177,6 @@ private fun videoFeedLine(s: DoubleArray): String? {
2 -> "4:2:2"
else -> "4:2:0"
}
return "HEVC · $depthLabel · $dynamicRange ($colorSpace) · $chromaLabel"
val codec = codecLabel.ifEmpty { "HEVC" }
return "$codec · $depthLabel · $dynamicRange ($colorSpace) · $chromaLabel"
}
@@ -84,6 +84,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
val initialSettings = remember { SettingsStore(context).load() }
var stats by remember { mutableStateOf<DoubleArray?>(null) }
var decoderLabel by remember { mutableStateOf("") }
var codecLabel by remember { mutableStateOf("") }
var statsVerbosity by remember { mutableStateOf(initialSettings.statsVerbosity) }
val statsOn = statsVerbosity != StatsVerbosity.OFF
// Touch model is fixed per session (re-keys the gesture handler below if it ever changes).
@@ -99,6 +100,9 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
LaunchedEffect(handle, statsOn) {
NativeBridge.nativeSetVideoStatsEnabled(handle, statsOn)
if (statsOn) {
// Codec is resolved at the handshake (Welcome) — fixed for the session, so read its
// label once up front (before the first snapshot renders the video-feed line).
if (codecLabel.isEmpty()) codecLabel = NativeBridge.nativeVideoCodecLabel(handle)
while (true) {
delay(1000)
stats = NativeBridge.nativeVideoStats(handle)
@@ -366,7 +370,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// BEFORE the transparent gesture layer below, so it shows through and never eats touches.
if (statsOn) {
stats?.let {
StatsOverlay(it, statsVerbosity, decoderLabel, Modifier.align(Alignment.TopStart).padding(12.dp))
StatsOverlay(it, statsVerbosity, decoderLabel, codecLabel, Modifier.align(Alignment.TopStart).padding(12.dp))
}
}
// "Hold to quit" hint while the gamepad exit chord is armed — the exit debounces on a ~1 s
@@ -214,6 +214,7 @@ internal fun StreamScene(verbosity: StatsVerbosity = StatsVerbosity.DETAILED) {
),
verbosity = verbosity,
decoderLabel = "c2.qti.hevc.decoder · low-latency",
codecLabel = "HEVC",
modifier = Modifier.align(Alignment.TopStart).padding(12.dp),
)
}
@@ -161,6 +161,14 @@ object NativeBridge {
*/
external fun nativeVideoMime(handle: Long): String
/**
* A short human label for the codec the host resolved (`"H.264"` / `"HEVC"` / `"AV1"` /
* `"PyroWave"`), for the stats HUD's video-feed line, or `""` on a `0` handle. Distinct from
* [nativeVideoMime] because the MIME collapses PyroWave onto `video/hevc` and can't name it.
* Fixed for the session (resolved at the handshake); read once. Cheap; UI-safe.
*/
external fun nativeVideoCodecLabel(handle: Long): String
/**
* Start the decode thread rendering onto [surface] (a SurfaceView's surface). Decode runs
* entirely in Rust (NDK AMediaCodec → ANativeWindow) — no per-frame JNI. [decoderName] is the
+19 -2
View File
@@ -161,7 +161,7 @@ fn run_sync(
return;
}
log::info!(
"decode: HEVC decoder started at {}x{}",
"decode: {mime} decoder started at {}x{}",
mode.width,
mode.height
);
@@ -617,6 +617,19 @@ pub(crate) fn codec_mime(codec: u8) -> &'static str {
}
}
/// A short human label for the codec the host resolved, for the stats HUD's video-feed line
/// (`"H.264"` / `"HEVC"` / `"AV1"` / `"PyroWave"`). Mirrors [`codec_mime`]'s fallback: anything
/// not H.264/AV1/PyroWave is reported as HEVC (every pre-negotiation host emitted HEVC). Kept
/// beside [`codec_mime`] because the MIME collapses PyroWave onto `video/hevc` and so can't name it.
pub(crate) fn codec_label(codec: u8) -> &'static str {
match codec {
punktfunk_core::quic::CODEC_H264 => "H.264",
punktfunk_core::quic::CODEC_AV1 => "AV1",
punktfunk_core::quic::CODEC_PYROWAVE => "PyroWave",
_ => "HEVC",
}
}
/// Create the decoder: prefer the specific codec Kotlin ranked from `MediaCodecList`
/// (`from_codec_name`), falling back to the platform's default decoder for the MIME
/// (`from_decoder_type`) if that name can't be created (codec busy / renamed across an OS update).
@@ -815,7 +828,11 @@ fn run_async(
})),
on_error: Some(Box::new(move |e, code, _detail| {
let fatal = !code.is_recoverable() && !code.is_transient();
log::warn!("decode: codec error {e:?} (fatal={fatal})");
if fatal {
log::error!("decode: fatal codec error — stream will stop: {e:?}");
} else {
log::warn!("decode: codec error {e:?} (recoverable)");
}
let _ = err_tx.send(DecodeEvent::Error { fatal });
})),
};
@@ -102,6 +102,31 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoMime<'
})
}
/// `NativeBridge.nativeVideoCodecLabel(handle): String` — a short human label for the codec the
/// host resolved (`"H.264"` / `"HEVC"` / `"AV1"` / `"PyroWave"`), for the stats HUD's video-feed
/// line. Distinct from [`Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoMime`] because the MIME
/// collapses PyroWave onto `video/hevc` and can't name it. Empty string on a `0` handle. Cheap;
/// safe on the UI thread. Android-gated (reads `crate::decode`), matching `nativeVideoMime`.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoCodecLabel<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
handle: jlong,
) -> jstring {
jni_guard(std::ptr::null_mut(), || {
if handle == 0 {
return std::ptr::null_mut();
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
match env.new_string(crate::decode::codec_label(h.client.codec)) {
Ok(s) => s.into_raw(),
Err(_) => std::ptr::null_mut(),
}
})
}
/// `NativeBridge.nativeVideoDecoderLabel(handle): String` — the resolved decoder identity for the
/// HUD, e.g. `c2.qti.avc.decoder · low-latency`, or `""` before the decode thread has resolved one.
/// One-shot (the decoder is fixed for the session); poll once after the HUD appears. Not
+27
View File
@@ -19,5 +19,32 @@
<array>
<string>_punktfunk._udp</string>
</array>
<!-- Background keep-alive (opt-in, iOS/iPadOS): the ONLY sanctioned way to keep the long-lived
QUIC socket + pump-thread set alive while backgrounded is the audio background mode, backed
by the session's real, audible remote audio (AVAudioEngine keeps rendering). Video decode is
dropped; a bounded timer auto-disconnects. Never silence-as-keepalive (App Review 2.5.4).
tvOS ignores/tolerates the key; macOS is not gated by it. -->
<key>UIBackgroundModes</key>
<array>
<string>audio</string>
</array>
<!-- Live Activities (iOS/iPadOS): the Lock-Screen / Dynamic-Island session surface. Updated
locally (pushType nil) from the alive app process — no aps-environment. tvOS/macOS ignore it. -->
<key>NSSupportsLiveActivities</key>
<true/>
<!-- Deep links: punktfunk://connect/<host-uuid>[?launch=<GameEntry.id>]. Emitted by the
launcher widget and Siri/Shortcuts; routed by ContentView.onOpenURL into the existing
connect path. Shared across all three targets (tvOS/macOS accept it harmlessly). -->
<key>CFBundleURLTypes</key>
<array>
<dict>
<key>CFBundleURLName</key>
<string>io.unom.punktfunk.deeplink</string>
<key>CFBundleURLSchemes</key>
<array>
<string>punktfunk</string>
</array>
</dict>
</array>
</dict>
</plist>
@@ -73,5 +73,15 @@
<array>
<string>$(AppIdentifierPrefix)io.unom.punktfunk</string>
</array>
<!-- App Group: same shared UserDefaults suite as iOS (Config/Punktfunk.entitlements). Shared
here so a single HostStore code path (UserDefaults(suiteName:)) works on every platform;
macOS widgets that read it arrive with M5. macOS App Groups use the plain group id under
the App Store profile; a Developer-ID-signed build wants the team-prefixed form — the
Dev-ID codesign step in release.yml must verify this value against the Dev-ID profile. -->
<key>com.apple.security.application-groups</key>
<array>
<string>group.io.unom.punktfunk</string>
</array>
</dict>
</plist>
@@ -20,5 +20,14 @@
is true on iOS/tvOS too. -->
<key>com.apple.developer.networking.multicast</key>
<true/>
<!-- App Group: the shared UserDefaults suite (group.io.unom.punktfunk) that both the app and
the Widget/Live-Activity extension read — the saved-host store moved there so a launcher
widget can see it (HostStore reads UserDefaults(suiteName:)). Must be registered on the
developer portal and enabled in the provisioning profile for BOTH app ids
(io.unom.punktfunk + io.unom.punktfunk.widgets). tvOS carries the key harmlessly. -->
<key>com.apple.security.application-groups</key>
<array>
<string>group.io.unom.punktfunk</string>
</array>
</dict>
</plist>
+10 -2
View File
@@ -9,13 +9,20 @@ let package = Package(
platforms: [.macOS(.v14), .iOS(.v17), .tvOS(.v17)],
products: [
.library(name: "PunktfunkKit", targets: ["PunktfunkKit"]),
// Dependency-free foundation (stored-host model + JSON codec, settings keys, App-Group
// constant, deep-link grammar, Live Activity attributes). A separate PRODUCT so the widget
// extension which must never link PunktfunkKit (Rust staticlib + presentation layer)
// can link this and nothing else. PunktfunkKit re-exports it (see SharedReexport.swift).
.library(name: "PunktfunkShared", targets: ["PunktfunkShared"]),
.executable(name: "PunktfunkClient", targets: ["PunktfunkClient"]),
],
targets: [
.binaryTarget(name: "PunktfunkCore", path: "PunktfunkCore.xcframework"),
// No dependencies by design an extension process links this alone.
.target(name: "PunktfunkShared"),
.target(
name: "PunktfunkKit",
dependencies: ["PunktfunkCore"],
dependencies: ["PunktfunkCore", "PunktfunkShared"],
// OSS attribution shown by the app's Acknowledgements screen. Bundled here (not in the
// app target) so it rides along via Bundle.module in both `swift build` and the Xcode
// app, which links the PunktfunkKit product. Refresh with
@@ -43,7 +50,8 @@ let package = Package(
// PunktfunkCore is a direct dep too so the wire tests can name the C ABI's
// `PunktfunkInputEvent` / `PUNKTFUNK_INPUT_KIND_*` when asserting the gamepad byte layout.
.testTarget(
name: "PunktfunkKitTests", dependencies: ["PunktfunkKit", "PunktfunkCore"],
name: "PunktfunkKitTests",
dependencies: ["PunktfunkKit", "PunktfunkShared", "PunktfunkCore"],
resources: [
// PyroWave golden fixtures: host-encoded AUs + upstream-decoded reference
// planes (regenerate with punktfunk-host's `pyrowave_dump_golden` on a
@@ -11,14 +11,56 @@
BB0000000000000000000005 /* PunktfunkKit in Frameworks */ = {isa = PBXBuildFile; productRef = BB0000000000000000000006 /* PunktfunkKit */; };
CC0000000000000000000005 /* PunktfunkKit in Frameworks */ = {isa = PBXBuildFile; productRef = CC0000000000000000000006 /* PunktfunkKit */; };
DD0000000000000000000003 /* SwiftUINavigationTransitions in Frameworks */ = {isa = PBXBuildFile; productRef = DD0000000000000000000002 /* SwiftUINavigationTransitions */; };
E295569A300948B9009F939C /* WidgetKit.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = E2955699300948B9009F939C /* WidgetKit.framework */; };
E295569C300948B9009F939C /* SwiftUI.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = E295569B300948B9009F939C /* SwiftUI.framework */; };
E2CAFE000000000000000001 /* PunktfunkShared in Frameworks */ = {isa = PBXBuildFile; productRef = E2CAFE000000000000000002 /* PunktfunkShared */; };
E29556A9300948BA009F939C /* PunktfunkWidgetsExtension.appex in Embed Foundation Extensions */ = {isa = PBXBuildFile; fileRef = E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */; settings = {ATTRIBUTES = (RemoveHeadersOnCopy, ); }; };
/* End PBXBuildFile section */
/* Begin PBXContainerItemProxy section */
E29556A7300948BA009F939C /* PBXContainerItemProxy */ = {
isa = PBXContainerItemProxy;
containerPortal = AA000000000000000000000D /* Project object */;
proxyType = 1;
remoteGlobalIDString = E2955696300948B9009F939C;
remoteInfo = PunktfunkWidgetsExtension;
};
/* End PBXContainerItemProxy section */
/* Begin PBXCopyFilesBuildPhase section */
E29556AA300948BA009F939C /* Embed Foundation Extensions */ = {
isa = PBXCopyFilesBuildPhase;
buildActionMask = 2147483647;
dstPath = "";
dstSubfolderSpec = 13;
files = (
E29556A9300948BA009F939C /* PunktfunkWidgetsExtension.appex in Embed Foundation Extensions */,
);
name = "Embed Foundation Extensions";
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXCopyFilesBuildPhase section */
/* Begin PBXFileReference section */
AA0000000000000000000001 /* Punktfunk.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = Punktfunk.app; sourceTree = BUILT_PRODUCTS_DIR; };
BB0000000000000000000001 /* Punktfunk-iOS.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = "Punktfunk-iOS.app"; sourceTree = BUILT_PRODUCTS_DIR; };
CC0000000000000000000001 /* Punktfunk-tvOS.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = "Punktfunk-tvOS.app"; sourceTree = BUILT_PRODUCTS_DIR; };
E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */ = {isa = PBXFileReference; explicitFileType = "wrapper.app-extension"; includeInIndex = 0; path = PunktfunkWidgetsExtension.appex; sourceTree = BUILT_PRODUCTS_DIR; };
E2955699300948B9009F939C /* WidgetKit.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = WidgetKit.framework; path = System/Library/Frameworks/WidgetKit.framework; sourceTree = SDKROOT; };
E295569B300948B9009F939C /* SwiftUI.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = SwiftUI.framework; path = System/Library/Frameworks/SwiftUI.framework; sourceTree = SDKROOT; };
E295577B30094CE5009F939C /* PunktfunkWidgetsExtension.entitlements */ = {isa = PBXFileReference; lastKnownFileType = text.plist.entitlements; path = PunktfunkWidgetsExtension.entitlements; sourceTree = "<group>"; };
/* End PBXFileReference section */
/* Begin PBXFileSystemSynchronizedBuildFileExceptionSet section */
E29556AD300948BA009F939C /* Exceptions for "PunktfunkWidgets" folder in "PunktfunkWidgetsExtension" target */ = {
isa = PBXFileSystemSynchronizedBuildFileExceptionSet;
membershipExceptions = (
Info.plist,
);
target = E2955696300948B9009F939C /* PunktfunkWidgetsExtension */;
};
/* End PBXFileSystemSynchronizedBuildFileExceptionSet section */
/* Begin PBXFileSystemSynchronizedRootGroup section */
AA0000000000000000000002 /* App */ = {
isa = PBXFileSystemSynchronizedRootGroup;
@@ -30,6 +72,14 @@
path = Sources/PunktfunkClient;
sourceTree = "<group>";
};
E295569D300948B9009F939C /* PunktfunkWidgets */ = {
isa = PBXFileSystemSynchronizedRootGroup;
exceptions = (
E29556AD300948BA009F939C /* Exceptions for "PunktfunkWidgets" folder in "PunktfunkWidgetsExtension" target */,
);
path = PunktfunkWidgets;
sourceTree = "<group>";
};
/* End PBXFileSystemSynchronizedRootGroup section */
/* Begin PBXFrameworksBuildPhase section */
@@ -58,14 +108,27 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
E2955694300948B9009F939C /* Frameworks */ = {
isa = PBXFrameworksBuildPhase;
buildActionMask = 2147483647;
files = (
E2CAFE000000000000000001 /* PunktfunkShared in Frameworks */,
E295569C300948B9009F939C /* SwiftUI.framework in Frameworks */,
E295569A300948B9009F939C /* WidgetKit.framework in Frameworks */,
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXFrameworksBuildPhase section */
/* Begin PBXGroup section */
AA0000000000000000000007 = {
isa = PBXGroup;
children = (
E295577B30094CE5009F939C /* PunktfunkWidgetsExtension.entitlements */,
AA0000000000000000000002 /* App */,
AA0000000000000000000003 /* Sources/PunktfunkClient */,
E295569D300948B9009F939C /* PunktfunkWidgets */,
E2955698300948B9009F939C /* Frameworks */,
AA0000000000000000000008 /* Products */,
);
sourceTree = "<group>";
@@ -76,10 +139,20 @@
AA0000000000000000000001 /* Punktfunk.app */,
BB0000000000000000000001 /* Punktfunk-iOS.app */,
CC0000000000000000000001 /* Punktfunk-tvOS.app */,
E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */,
);
name = Products;
sourceTree = "<group>";
};
E2955698300948B9009F939C /* Frameworks */ = {
isa = PBXGroup;
children = (
E2955699300948B9009F939C /* WidgetKit.framework */,
E295569B300948B9009F939C /* SwiftUI.framework */,
);
name = Frameworks;
sourceTree = "<group>";
};
/* End PBXGroup section */
/* Begin PBXNativeTarget section */
@@ -114,10 +187,12 @@
BB000000000000000000000B /* Sources */,
BB0000000000000000000004 /* Frameworks */,
BB000000000000000000000C /* Resources */,
E29556AA300948BA009F939C /* Embed Foundation Extensions */,
);
buildRules = (
);
dependencies = (
E29556A8300948BA009F939C /* PBXTargetDependency */,
);
fileSystemSynchronizedGroups = (
AA0000000000000000000002 /* App */,
@@ -156,6 +231,29 @@
productReference = CC0000000000000000000001 /* Punktfunk-tvOS.app */;
productType = "com.apple.product-type.application";
};
E2955696300948B9009F939C /* PunktfunkWidgetsExtension */ = {
isa = PBXNativeTarget;
buildConfigurationList = E29556AE300948BA009F939C /* Build configuration list for PBXNativeTarget "PunktfunkWidgetsExtension" */;
buildPhases = (
E2955693300948B9009F939C /* Sources */,
E2955694300948B9009F939C /* Frameworks */,
E2955695300948B9009F939C /* Resources */,
);
buildRules = (
);
dependencies = (
);
fileSystemSynchronizedGroups = (
E295569D300948B9009F939C /* PunktfunkWidgets */,
);
name = PunktfunkWidgetsExtension;
packageProductDependencies = (
E2CAFE000000000000000002 /* PunktfunkShared */,
);
productName = PunktfunkWidgetsExtension;
productReference = E2955697300948B9009F939C /* PunktfunkWidgetsExtension.appex */;
productType = "com.apple.product-type.app-extension";
};
/* End PBXNativeTarget section */
/* Begin PBXProject section */
@@ -163,11 +261,15 @@
isa = PBXProject;
attributes = {
BuildIndependentTargetsInParallel = 1;
LastSwiftUpdateCheck = 2700;
LastUpgradeCheck = 2700;
TargetAttributes = {
AA0000000000000000000009 = {
CreatedOnToolsVersion = 26.0;
};
E2955696300948B9009F939C = {
CreatedOnToolsVersion = 27.0;
};
};
};
buildConfigurationList = AA000000000000000000000E /* Build configuration list for PBXProject "Punktfunk" */;
@@ -190,6 +292,7 @@
AA0000000000000000000009 /* Punktfunk */,
BB0000000000000000000009 /* Punktfunk-iOS */,
CC0000000000000000000009 /* Punktfunk-tvOS */,
E2955696300948B9009F939C /* PunktfunkWidgetsExtension */,
);
};
/* End PBXProject section */
@@ -216,6 +319,13 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
E2955695300948B9009F939C /* Resources */ = {
isa = PBXResourcesBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXResourcesBuildPhase section */
/* Begin PBXSourcesBuildPhase section */
@@ -240,8 +350,23 @@
);
runOnlyForDeploymentPostprocessing = 0;
};
E2955693300948B9009F939C /* Sources */ = {
isa = PBXSourcesBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXSourcesBuildPhase section */
/* Begin PBXTargetDependency section */
E29556A8300948BA009F939C /* PBXTargetDependency */ = {
isa = PBXTargetDependency;
target = E2955696300948B9009F939C /* PunktfunkWidgetsExtension */;
targetProxy = E29556A7300948BA009F939C /* PBXContainerItemProxy */;
};
/* End PBXTargetDependency section */
/* Begin XCBuildConfiguration section */
AA0000000000000000000010 /* Debug */ = {
isa = XCBuildConfiguration;
@@ -564,6 +689,97 @@
};
name = Release;
};
E29556AB300948BA009F939C /* Debug */ = {
isa = XCBuildConfiguration;
buildSettings = {
ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor;
ASSETCATALOG_COMPILER_WIDGET_BACKGROUND_COLOR_NAME = WidgetBackground;
CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
CLANG_ENABLE_OBJC_WEAK = YES;
CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
CODE_SIGN_ENTITLEMENTS = PunktfunkWidgetsExtension.entitlements;
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = F4H37KF6WC;
GCC_C_LANGUAGE_STANDARD = gnu17;
GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = PunktfunkWidgets/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = PunktfunkWidgets;
INFOPLIST_KEY_NSHumanReadableCopyright = "";
IPHONEOS_DEPLOYMENT_TARGET = 27.0;
LD_RUNPATH_SEARCH_PATHS = (
"$(inherited)",
"@executable_path/Frameworks",
"@executable_path/../../Frameworks",
);
LOCALIZATION_PREFERS_STRING_CATALOGS = YES;
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = io.unom.punktfunk.widgets;
PRODUCT_NAME = "$(TARGET_NAME)";
REGISTER_APP_GROUPS = YES;
SDKROOT = iphoneos;
SKIP_INSTALL = YES;
STRING_CATALOG_GENERATE_SYMBOLS = YES;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_EMIT_LOC_STRINGS = YES;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
};
name = Debug;
};
E29556AC300948BA009F939C /* Release */ = {
isa = XCBuildConfiguration;
buildSettings = {
ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor;
ASSETCATALOG_COMPILER_WIDGET_BACKGROUND_COLOR_NAME = WidgetBackground;
CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
CLANG_ENABLE_OBJC_WEAK = YES;
CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
CODE_SIGN_ENTITLEMENTS = PunktfunkWidgetsExtension.entitlements;
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = F4H37KF6WC;
GCC_C_LANGUAGE_STANDARD = gnu17;
GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = PunktfunkWidgets/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = PunktfunkWidgets;
INFOPLIST_KEY_NSHumanReadableCopyright = "";
IPHONEOS_DEPLOYMENT_TARGET = 27.0;
LD_RUNPATH_SEARCH_PATHS = (
"$(inherited)",
"@executable_path/Frameworks",
"@executable_path/../../Frameworks",
);
LOCALIZATION_PREFERS_STRING_CATALOGS = YES;
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = io.unom.punktfunk.widgets;
PRODUCT_NAME = "$(TARGET_NAME)";
REGISTER_APP_GROUPS = YES;
SDKROOT = iphoneos;
SKIP_INSTALL = YES;
STRING_CATALOG_GENERATE_SYMBOLS = YES;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_EMIT_LOC_STRINGS = YES;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
VALIDATE_PRODUCT = YES;
};
name = Release;
};
/* End XCBuildConfiguration section */
/* Begin XCConfigurationList section */
@@ -603,6 +819,15 @@
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
E29556AE300948BA009F939C /* Build configuration list for PBXNativeTarget "PunktfunkWidgetsExtension" */ = {
isa = XCConfigurationList;
buildConfigurations = (
E29556AB300948BA009F939C /* Debug */,
E29556AC300948BA009F939C /* Release */,
);
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
/* End XCConfigurationList section */
/* Begin XCLocalSwiftPackageReference section */
@@ -636,6 +861,10 @@
isa = XCSwiftPackageProductDependency;
productName = PunktfunkKit;
};
E2CAFE000000000000000002 /* PunktfunkShared */ = {
isa = XCSwiftPackageProductDependency;
productName = PunktfunkShared;
};
DD0000000000000000000002 /* SwiftUINavigationTransitions */ = {
isa = XCSwiftPackageProductDependency;
package = DD0000000000000000000001 /* XCRemoteSwiftPackageReference "swiftui-navigation-transitions" */;
@@ -0,0 +1,11 @@
{
"colors" : [
{
"idiom" : "universal"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -0,0 +1,35 @@
{
"images" : [
{
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
},
{
"appearances" : [
{
"appearance" : "luminosity",
"value" : "dark"
}
],
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
},
{
"appearances" : [
{
"appearance" : "luminosity",
"value" : "tinted"
}
],
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -0,0 +1,6 @@
{
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -0,0 +1,11 @@
{
"colors" : [
{
"idiom" : "universal"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}
@@ -0,0 +1,186 @@
// Home-Screen / Lock-Screen quick-launch widget (kind "PunktfunkHosts"). Reads the saved-host
// store from the shared App-Group suite, sorts most-recent-first, and deep-links each host into a
// session via `punktfunk://connect/<uuid>` the app's onOpenURL routes it through the normal
// connect path (trust policy / WoL / approval all apply).
//
// No reachability probing in v1 (a UDP check has no place in a timeline build; WoL handles offline
// hosts on tap). Timeline is a single `.never` entry the app pushes reloads on store changes
// (HostStore WidgetCenter.reloadTimelines).
import SwiftUI
import WidgetKit
import PunktfunkShared
// MARK: - Timeline
struct HostsEntry: TimelineEntry {
let date: Date
let hosts: [StoredHost]
}
struct HostsProvider: TimelineProvider {
func placeholder(in context: Context) -> HostsEntry {
HostsEntry(date: .now, hosts: [])
}
func getSnapshot(in context: Context, completion: @escaping (HostsEntry) -> Void) {
completion(HostsEntry(date: .now, hosts: Self.loadHosts()))
}
func getTimeline(in context: Context, completion: @escaping (Timeline<HostsEntry>) -> Void) {
// Single entry, never auto-refresh: the app reloads this timeline whenever the store
// changes (a new host, a fresh connect reordering by recency).
let entry = HostsEntry(date: .now, hosts: Self.loadHosts())
completion(Timeline(entries: [entry], policy: .never))
}
/// Decode the shared-suite host JSON (same wire format the app writes), most-recent first.
static func loadHosts() -> [StoredHost] {
guard let data = AppGroup.defaults.data(forKey: DefaultsKey.hosts),
let hosts = try? JSONDecoder().decode([StoredHost].self, from: data)
else { return [] }
return hosts.sorted {
($0.lastConnected ?? .distantPast) > ($1.lastConnected ?? .distantPast)
}
}
}
// MARK: - Widget
struct HostsWidget: Widget {
var body: some WidgetConfiguration {
StaticConfiguration(kind: "PunktfunkHosts", provider: HostsProvider()) { entry in
HostsWidgetView(entry: entry)
.containerBackground(.fill.tertiary, for: .widget)
}
.configurationDisplayName("Punktfunk Hosts")
.description("Quick-launch your recent streaming hosts.")
.supportedFamilies([
.systemSmall, .systemMedium, .accessoryCircular, .accessoryRectangular,
])
}
}
// MARK: - Views
struct HostsWidgetView: View {
@Environment(\.widgetFamily) private var family
let entry: HostsEntry
var body: some View {
switch family {
case .systemMedium:
MediumHostsView(hosts: entry.hosts)
case .accessoryCircular:
CircularHostView(host: entry.hosts.first)
case .accessoryRectangular:
RectangularHostView(host: entry.hosts.first)
default: // systemSmall + fallback
SmallHostView(host: entry.hosts.first)
}
}
}
/// Deep link that connects to a stored host.
private func connectURL(_ host: StoredHost) -> URL {
DeepLink.connect(host: host.id, launchID: nil).url
}
private struct SmallHostView: View {
let host: StoredHost?
var body: some View {
if let host {
VStack(alignment: .leading, spacing: 6) {
Image(systemName: "play.tv.fill")
.font(.title2)
.foregroundStyle(.tint)
Spacer(minLength: 0)
Text(host.displayName)
.font(.headline)
.lineLimit(2)
if let last = host.lastConnected {
Text(last, format: .relative(presentation: .named))
.font(.caption2)
.foregroundStyle(.secondary)
}
}
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading)
.widgetURL(connectURL(host))
} else {
EmptyHostView()
}
}
}
private struct MediumHostsView: View {
let hosts: [StoredHost]
var body: some View {
if hosts.isEmpty {
EmptyHostView()
} else {
VStack(alignment: .leading, spacing: 8) {
Text("Punktfunk")
.font(.caption).bold()
.foregroundStyle(.tint)
ForEach(hosts.prefix(4)) { host in
Link(destination: connectURL(host)) {
HStack {
Image(systemName: "play.tv.fill")
.foregroundStyle(.tint)
Text(host.displayName)
.font(.subheadline)
.lineLimit(1)
Spacer()
if let last = host.lastConnected {
Text(last, format: .relative(presentation: .named))
.font(.caption2)
.foregroundStyle(.secondary)
}
}
}
}
Spacer(minLength: 0)
}
.frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading)
}
}
}
private struct CircularHostView: View {
let host: StoredHost?
var body: some View {
ZStack {
AccessoryWidgetBackground()
Image(systemName: "play.tv.fill")
}
.widgetURL(host.map(connectURL))
}
}
private struct RectangularHostView: View {
let host: StoredHost?
var body: some View {
HStack {
Image(systemName: "play.tv.fill")
Text(host?.displayName ?? "Punktfunk")
.lineLimit(1)
}
.widgetURL(host.map(connectURL))
}
}
private struct EmptyHostView: View {
var body: some View {
VStack(spacing: 6) {
Image(systemName: "play.tv")
.font(.title2)
.foregroundStyle(.secondary)
Text("Open Punktfunk to add a host.")
.font(.caption)
.multilineTextAlignment(.center)
.foregroundStyle(.secondary)
}
.frame(maxWidth: .infinity, maxHeight: .infinity)
}
}
+11
View File
@@ -0,0 +1,11 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>NSExtension</key>
<dict>
<key>NSExtensionPointIdentifier</key>
<string>com.apple.widgetkit-extension</string>
</dict>
</dict>
</plist>
@@ -0,0 +1,20 @@
// The widget extension's entry point. ONE extension target (bundle id io.unom.punktfunk.widgets,
// iOS only) hosts both the launcher widgets and the Live Activity UI. It links PunktfunkShared and
// NOTHING else never PunktfunkKit (Rust staticlib + presentation layer would blow the widget
// process's ~30 MB budget).
//
// These files are NOT part of the SwiftPM package (Package.swift doesn't declare a PunktfunkWidgets
// target, so `swift build` ignores the directory). They compile only in the Xcode widget-extension
// target you add pointing at this folder see design/apple-live-activities-and-widgets.md §M1 and
// the GUI checklist.
import SwiftUI
import WidgetKit
@main
struct PunktfunkWidgetBundle: WidgetBundle {
var body: some Widget {
HostsWidget()
PunktfunkSessionLiveActivity()
}
}
@@ -0,0 +1,140 @@
// The Live Activity UI (Lock Screen banner + Dynamic Island) for a running session. The app owns
// the Activity's lifecycle (SessionActivityController); this is only its presentation, rendered in
// the widget-extension process from the shared `PunktfunkSessionAttributes`.
//
// The End button runs `EndStreamIntent` (a LiveActivityIntent) IN THE APP's process, which posts
// .punktfunkEndActiveSession the app disconnects. Elapsed time ticks client-side via
// Text(timerInterval:) no per-second push.
import ActivityKit
import AppIntents
import SwiftUI
import WidgetKit
import PunktfunkShared
struct PunktfunkSessionLiveActivity: Widget {
var body: some WidgetConfiguration {
ActivityConfiguration(for: PunktfunkSessionAttributes.self) { context in
LockScreenView(context: context)
.activitySystemActionForegroundColor(.white)
} dynamicIsland: { context in
DynamicIsland {
DynamicIslandExpandedRegion(.leading) {
Label {
Text(context.attributes.hostName).font(.caption).lineLimit(1)
} icon: {
Image(systemName: "play.tv.fill")
}
.foregroundStyle(.tint)
}
DynamicIslandExpandedRegion(.trailing) {
Text(timerInterval: context.state.startedAt...Date.distantFuture, countsDown: false)
.font(.caption).monospacedDigit()
.frame(maxWidth: 56)
.foregroundStyle(.secondary)
}
DynamicIslandExpandedRegion(.center) {
if let title = context.attributes.launchTitle {
Text(title).font(.caption2).lineLimit(1).foregroundStyle(.secondary)
}
}
DynamicIslandExpandedRegion(.bottom) {
VStack(spacing: 6) {
Text(context.state.modeLine)
.font(.caption2).foregroundStyle(.secondary).lineLimit(1)
StageLine(state: context.state)
EndButton()
}
}
} compactLeading: {
Image(systemName: "play.tv.fill").foregroundStyle(.tint)
} compactTrailing: {
Text(timerInterval: context.state.startedAt...Date.distantFuture, countsDown: false)
.monospacedDigit()
.frame(maxWidth: 44)
} minimal: {
Image(systemName: "play.tv.fill").foregroundStyle(.tint)
}
}
}
}
// MARK: - Lock Screen banner
private struct LockScreenView: View {
let context: ActivityViewContext<PunktfunkSessionAttributes>
var body: some View {
HStack(alignment: .top, spacing: 12) {
Image(systemName: "play.tv.fill")
.font(.title2)
.foregroundStyle(.tint)
VStack(alignment: .leading, spacing: 3) {
HStack {
Text(context.attributes.hostName).font(.headline).lineLimit(1)
Spacer()
Text(timerInterval: context.state.startedAt...Date.distantFuture, countsDown: false)
.font(.subheadline).monospacedDigit()
.foregroundStyle(.secondary)
}
if let title = context.attributes.launchTitle {
Text(title).font(.caption).foregroundStyle(.secondary).lineLimit(1)
}
Text(context.state.modeLine)
.font(.caption2).foregroundStyle(.secondary).lineLimit(1)
StageLine(state: context.state)
}
if context.state.stage == .background {
EndButton()
}
}
.padding()
}
}
// MARK: - Shared pieces
/// The stage badge + (while backgrounded) the auto-disconnect countdown.
private struct StageLine: View {
let state: PunktfunkSessionAttributes.ContentState
var body: some View {
switch state.stage {
case .streaming:
EmptyView()
case .background:
if let deadline = state.backgroundDeadline {
HStack(spacing: 3) {
Text("Keeps running for")
Text(timerInterval: Date()...deadline, countsDown: true)
.monospacedDigit()
}
.font(.caption2)
.foregroundStyle(.secondary)
} else {
badge("Running in background", .orange)
}
case .reconnecting:
badge("Reconnecting…", .yellow)
case .ending:
badge("Session ended", .secondary)
}
}
private func badge(_ text: String, _ color: Color) -> some View {
Text(text).font(.caption2).foregroundStyle(color)
}
}
/// End-stream button runs EndStreamIntent in the app process (LiveActivityIntent).
private struct EndButton: View {
var body: some View {
Button(intent: EndStreamIntent()) {
Label("End", systemImage: "stop.fill")
.font(.caption).bold()
}
.tint(.red)
.buttonStyle(.bordered)
}
}
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.application-groups</key>
<array>
<string>group.io.unom.punktfunk</string>
</array>
</dict>
</plist>
@@ -51,6 +51,15 @@ struct ContentView: View {
}
}
@State private var showAddHost = false
/// A `punktfunk://` deep link (widget / Siri / Shortcuts) couldn't be honored unknown host, or
/// a live session is already up. Surfaced as an informational alert (distinct from the
/// "Connection failed" one, which is for actual connect errors).
@State private var deepLinkNotice: String?
#if os(iOS)
/// Owns the Live Activity for the running session (Lock Screen / Dynamic Island). Driven from
/// the session model's published state below; iPhone/iPad only.
@State private var liveActivity = SessionActivityController()
#endif
@State private var pairingTarget: StoredHost?
/// A fresh `pair=required`/unknown host the user tapped: drives the choice between no-PIN
/// delegated approval ("Request Access") and the SPAKE2 PIN ceremony (rule 3b).
@@ -92,6 +101,14 @@ struct ContentView: View {
/// fires Wake-on-LAN up front and falls into the "Waking" wait if the dial fails. Off: connects
/// go straight through with no wake. The explicit "Wake Host" action is unaffected either way.
@AppStorage(DefaultsKey.autoWake) private var autoWakeEnabled = true
/// Background keep-alive (Settings General, iOS-only). Default OFF (today's freeze-on-background
/// is the default). When on, backgrounding a live session keeps audio + the connection alive and
/// drops video, auto-disconnecting after `backgroundTimeoutMinutes`.
@AppStorage(DefaultsKey.backgroundKeepAlive) private var backgroundKeepAlive = false
@AppStorage(DefaultsKey.backgroundTimeoutMinutes) private var backgroundTimeoutMinutes = 10
/// scenePhase drives the keep-alive: use THIS, not the willResignActive observers resign-active
/// also fires for Control Center / app-switcher peeks, where the disconnect timer must not start.
@Environment(\.scenePhase) private var scenePhase
private var gamepadUIActive: Bool {
GamepadUIEnvironment.isActive(
gamepadConnected: gamepadManager.active != nil, enabledSetting: gamepadUIEnabled)
@@ -113,7 +130,62 @@ struct ContentView: View {
.onAppear {
seedDefaultModeIfNeeded()
autoConnectIfAsked()
#if os(iOS)
SessionActivityController.sweepOrphans() // end any Activity a prior killed launch left
#endif
}
// Deep links (widget quick-launch, Siri/Shortcuts): route into the SAME connect path a card
// tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a
// parallel session this drives the one `model` ContentView owns.
.onOpenURL { handleDeepLink($0) }
#if os(iOS)
// Background keep-alive driver (opt-in). Only .background/.active matter; .inactive (a
// transient peek) is ignored so the disconnect timer never starts for a Control-Center pull.
.onChange(of: scenePhase) { _, phase in
switch phase {
case .background:
if backgroundKeepAlive, model.phase == .streaming {
model.enterBackground(timeoutMinutes: backgroundTimeoutMinutes)
}
case .active:
model.exitBackground()
default:
break
}
}
// Live Activity lifecycle, driven from the model's published state.
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
if let host = model.activeHost {
liveActivity.begin(
hostID: host.id, hostName: host.displayName,
launchTitle: nil, // no live foreground-app title mid-session (v1)
modeLine: currentModeLine(), startedAt: Date())
}
case .idle:
liveActivity.end()
default:
break
}
}
.onChange(of: model.isBackgrounded) { _, backgrounded in
liveActivity.update {
$0.stage = backgrounded ? .background : .streaming
$0.backgroundDeadline = model.backgroundDeadline
}
}
// The Live Activity's / Shortcuts' End button runs EndStreamIntent in-process, which posts
// this tear the session down deliberately (quit-close the host).
.onReceive(NotificationCenter.default.publisher(for: .punktfunkEndActiveSession)) { _ in
model.disconnect(deliberate: true)
}
// Connect App Intent (Siri/Shortcuts): route its punktfunk:// URL through the same handler
// as a widget tap.
.onReceive(NotificationCenter.default.publisher(for: .punktfunkOpenDeepLink)) { note in
if let url = note.object as? URL { handleDeepLink(url) }
}
#endif
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
@@ -151,6 +223,9 @@ struct ContentView: View {
#if !os(tvOS)
.focusedSceneValue(\.sessionFocus, SessionFocus(
isStreaming: model.connection != nil,
clipboardAvailable: model.connection?.hostSupportsClipboard == true,
clipboardOn: model.clipboardEnabled,
toggleClipboard: { model.toggleClipboardSync() },
disconnect: { model.disconnect() }))
#endif
#if os(macOS)
@@ -262,6 +337,57 @@ struct ContentView: View {
+ "console (port 3000 → Pairing). This device connects automatically once you "
+ "approve it — no need to reconnect.")
}
// Informational deep-link outcome (unknown host / already streaming). Not an error.
.alert(
"Can't open",
isPresented: Binding(
get: { deepLinkNotice != nil },
set: { if !$0 { deepLinkNotice = nil } })
) {
Button("OK", role: .cancel) {}
} message: {
Text(deepLinkNotice ?? "")
}
}
#if os(iOS)
/// The Live Activity mode line, e.g. "2560×1440 @120 · HEVC · HDR", from the live connection.
private func currentModeLine() -> String {
guard let c = model.connection else { return "" }
let codec: String
switch c.videoCodec {
case .h264: codec = "H.264"
case .hevc: codec = "HEVC"
case .av1: codec = "AV1"
case .pyrowave: codec = "PyroWave"
}
var line = "\(c.width)×\(c.height)"
if c.refreshHz > 0 { line += " @\(c.refreshHz)" }
line += " · \(codec)"
if c.isHDR { line += " · HDR" }
return line
}
#endif
/// Route a `punktfunk://` deep link into the existing connect path. Rules (per design):
/// unknown host notice + no-op; a live session is up ignore if it's the same host, else
/// tell the user to end the current one first (NEVER tear down a live session on a background
/// tap); otherwise the normal `connect` trust policy, WoL and the approval sheet all apply.
private func handleDeepLink(_ url: URL) {
guard case let .connect(hostID, launchID)? = DeepLink(url) else { return }
guard let host = store.hosts.first(where: { $0.id == hostID }) else {
deepLinkNotice = "That host isn't saved on this device."
return
}
if model.phase != .idle {
guard model.activeHost?.id == hostID else {
let current = model.activeHost?.displayName ?? "a host"
deepLinkNotice = "Already streaming \(current). End that session first."
return
}
return // deep-linked to the host we're already on nothing to do
}
connect(host, launchID: launchID)
}
private var home: some View {
@@ -20,6 +20,12 @@ struct AddHostSheet: View {
@State private var address: String
@State private var port: Int
@State private var mac: String
#if os(macOS)
/// Share the clipboard with this host (macOS sessions only; design
/// clipboard-and-file-transfer.md §5.3). Off by default; honored only when the host
/// advertises the capability at connect.
@State private var clipboardSync: Bool
#endif
#if os(tvOS)
private enum EditField: String, Identifiable {
case name, address, port, mac
@@ -41,6 +47,9 @@ struct AddHostSheet: View {
_port = State(initialValue: Int(existing?.port ?? 9777))
let stored = existing?.macAddresses ?? []
_mac = State(initialValue: (stored.isEmpty ? suggestedMacs : stored).joined(separator: ", "))
#if os(macOS)
_clipboardSync = State(initialValue: existing?.clipboardSync ?? false)
#endif
}
var body: some View {
@@ -96,6 +105,9 @@ struct AddHostSheet: View {
#if os(iOS)
.textInputAutocapitalization(.never)
#endif
#if os(macOS)
Toggle("Share clipboard with this host", isOn: $clipboardSync)
#endif
}
#if !os(tvOS)
.formStyle(.grouped)
@@ -147,6 +159,11 @@ struct AddHostSheet: View {
host.address = address.trimmingCharacters(in: .whitespaces)
host.port = UInt16(clamping: port)
host.macAddresses = Self.parseMacs(mac)
#if os(macOS)
// nil when off: the key stays absent from the saved JSON (forward-compat, and "never
// opted in" and "opted out" read the same off).
host.clipboardSync = clipboardSync ? true : nil
#endif
onSave(host)
dismiss()
}
@@ -0,0 +1,102 @@
// Siri / Shortcuts / Spotlight surface (design §M4). Deliberately thin: every action already has an
// internal entry point M0's deep-link router (connect / connect-and-launch), M3's in-process
// end-session hook, and the existing Wake-on-LAN path so these intents only wrap them.
//
// Gated os(iOS): the AppShortcutsProvider bundles `EndStreamIntent`, which is a LiveActivityIntent
// (iPhone/iPad only). Connect/Wake themselves are plain AppIntents; they live here with the
// provider rather than being split across platforms. `HostEntity` (the parameter type) is in
// PunktfunkShared so the widget's configuration intent can share it.
#if os(iOS)
import AppIntents
import Foundation
import PunktfunkKit
/// Load a full saved host (MACs, address) from the shared App-Group store by id HostEntity only
/// carries id + name.
private func loadStoredHost(_ id: UUID) -> StoredHost? {
guard let data = AppGroup.defaults.data(forKey: DefaultsKey.hosts),
let hosts = try? JSONDecoder().decode([StoredHost].self, from: data)
else { return nil }
return hosts.first { $0.id == id }
}
/// Start a session with a stored host (optionally launching a title). Foregrounds the app and
/// routes through the SAME `.onOpenURL` path a widget tap uses trust policy, WoL and the approval
/// sheet all apply, and its guards (unknown host, already-streaming) hold.
struct ConnectToHostIntent: AppIntent {
static let title: LocalizedStringResource = "Connect to Host"
static let description = IntentDescription("Start a Punktfunk streaming session with a host.")
static let openAppWhenRun = true
@Parameter(title: "Host") var host: HostEntity
@Parameter(title: "Game ID", description: "Optional store id like steam:570")
var launchID: String?
func perform() async throws -> some IntentResult {
let url = DeepLink.connect(host: host.id, launchID: launchID).url
await MainActor.run {
NotificationCenter.default.post(name: .punktfunkOpenDeepLink, object: url)
}
return .result()
}
}
/// Wake a sleeping host (magic packet). No `openAppWhenRun` usable in automations ("when I get
/// home, wake the tower") without foregrounding the app.
struct WakeHostIntent: AppIntent {
static let title: LocalizedStringResource = "Wake Host"
static let description = IntentDescription("Send a Wake-on-LAN magic packet to a host.")
@Parameter(title: "Host") var host: HostEntity
func perform() async throws -> some IntentResult {
guard let stored = loadStoredHost(host.id), !stored.wakeMacs.isEmpty else {
throw IntentError.noWakeAddress
}
PunktfunkConnection.wakeOnLAN(macs: stored.wakeMacs, lastKnownIP: stored.address)
return .result()
}
}
/// Errors surfaced to Siri/Shortcuts. `CustomLocalizedStringResourceConvertible` makes the message
/// show as the intent's failure text.
enum IntentError: Error, CustomLocalizedStringResourceConvertible {
case noWakeAddress
var localizedStringResource: LocalizedStringResource {
switch self {
case .noWakeAddress:
// One string LITERAL LocalizedStringResource is ExpressibleByStringLiteral, but a
// `"" + ""` concatenation is a runtime String it can't convert.
return "That host has no saved Wake-on-LAN address yet. Connect to it once so Punktfunk can learn it."
}
}
}
/// Zero-setup Siri / Spotlight phrases. Parameterized phrases resolve a `HostEntity` by name; stays
/// well under the 10-shortcut cap.
struct PunktfunkShortcuts: AppShortcutsProvider {
static var appShortcuts: [AppShortcut] {
AppShortcut(
intent: ConnectToHostIntent(),
phrases: [
"Connect to \(\.$host) in \(.applicationName)",
"Stream \(\.$host) with \(.applicationName)",
],
shortTitle: "Connect", systemImageName: "play.tv.fill")
AppShortcut(
intent: WakeHostIntent(),
phrases: [
"Wake \(\.$host) with \(.applicationName)",
],
shortTitle: "Wake Host", systemImageName: "power")
AppShortcut(
intent: EndStreamIntent(),
phrases: [
"End the \(.applicationName) stream",
],
shortTitle: "End Stream", systemImageName: "stop.fill")
}
}
#endif
@@ -0,0 +1,89 @@
// Owns the ActivityKit Live Activity lifecycle for a streaming session (iPhone/iPad only). Driven
// by ContentView from the session model's published state (phase / isBackgrounded / deadline) so
// none of this leaks into the cross-platform SessionModel. Local updates only (`pushType: nil`)
// the app process is alive whenever there's a session to report, so there's no push token plumbing.
//
// Gated os(iOS): ActivityKit is iPhone/iPad only. Minimum deployment is iOS 17, so no @available
// guards are needed (Activity has existed since 16.1).
#if os(iOS)
import ActivityKit
import Foundation
// PunktfunkKit re-exports PunktfunkShared (@_exported), so the app target sees PunktfunkSessionAttributes
// without linking the Shared product directly same pattern as StoredHost in HostStore.
import PunktfunkKit
@MainActor
final class SessionActivityController {
private var activity: Activity<PunktfunkSessionAttributes>?
/// The last pushed state, so an update can mutate one field and keep the rest (notably
/// `startedAt`, which the Lock-Screen timer ticks from).
private var state: PunktfunkSessionAttributes.ContentState?
/// How far past the next expected update to mark the content stale a frozen opt-out session
/// then greys out instead of showing a lying clock.
private static let staleWindow: TimeInterval = 90
var isActive: Bool { activity != nil }
/// End any Activity left over from a previous launch that was killed mid-session. Call once at
/// app start (ContentView.onAppear).
static func sweepOrphans() {
Task {
for activity in Activity<PunktfunkSessionAttributes>.activities {
await activity.end(nil, dismissalPolicy: .immediate)
}
}
}
/// Start the Live Activity for a freshly-streaming session. No-op if the user disabled Live
/// Activities for the app, or one is already up.
func begin(hostID: UUID, hostName: String, launchTitle: String?, modeLine: String, startedAt: Date) {
guard ActivityAuthorizationInfo().areActivitiesEnabled, activity == nil else { return }
let attributes = PunktfunkSessionAttributes(
hostID: hostID, hostName: hostName, launchTitle: launchTitle)
let initial = PunktfunkSessionAttributes.ContentState(
stage: .streaming, startedAt: startedAt, modeLine: modeLine)
state = initial
do {
activity = try Activity.request(
attributes: attributes,
content: content(initial),
pushType: nil)
} catch {
activity = nil
state = nil
}
}
/// Coalesced update: mutate the running state in place (keeps `startedAt` etc.) and push once.
/// No-op when there's no live Activity.
func update(_ mutate: (inout PunktfunkSessionAttributes.ContentState) -> Void) {
guard let activity, var next = state else { return }
mutate(&next)
state = next
Task { await activity.update(content(next)) }
}
/// End with a final "ended" state, dismissed a few seconds later.
func end() {
guard let activity, var final = state else {
self.activity = nil
state = nil
return
}
self.activity = nil
state = nil
final.stage = .ending
final.backgroundDeadline = nil
Task {
await activity.end(content(final), dismissalPolicy: .after(.now + 4))
}
}
private func content(_ s: PunktfunkSessionAttributes.ContentState)
-> ActivityContent<PunktfunkSessionAttributes.ContentState> {
ActivityContent(state: s, staleDate: Date().addingTimeInterval(Self.staleWindow))
}
}
#endif
@@ -139,6 +139,18 @@ final class SessionModel: ObservableObject {
private var audio: SessionAudio?
private var gamepadCapture: GamepadCapture?
private var gamepadFeedback: GamepadFeedback?
#if os(macOS)
/// The live session's clipboard bridge (design/clipboard-and-file-transfer.md §5) created
/// by `beginStreaming` when the per-host toggle is on and the host advertises
/// `HOST_CAP_CLIPBOARD`; stopped (off-main, drain joined) in `disconnect`.
private var clipboardSync: ClipboardSync?
#endif
/// Whether clipboard sync is live (host-acked `ClipState.enabled`) drives the Stream menu
/// item's title and the settings footnote. Always false off-macOS.
@Published private(set) var clipboardEnabled = false
/// The host's last `ClipState.reason` (`CLIP_REASON_*`) why an enable was refused
/// (backend unavailable / policy disabled / ); 0 = OK.
@Published private(set) var clipboardReason: UInt8 = 0
#if os(tvOS)
/// Siri Remote host pointer while streaming (touch surface moves, press = left click,
/// Play/Pause = right click) + the remote's deliberate exit (hold Back 1 s). See
@@ -148,6 +160,16 @@ final class SessionModel: ObservableObject {
var isBusy: Bool { phase != .idle }
/// True while a streaming session is running in the background under the opt-in keep-alive
/// (audio plays, video dropped, timeout armed). Drives the Live Activity's stage/countdown (M3)
/// and is cleared on foreground or teardown. iOS/iPadOS only in practice.
@Published private(set) var isBackgrounded = false
/// When the backgrounded keep-alive will auto-disconnect (nil unless backgrounded) drives the
/// Live Activity countdown. Set alongside `backgroundTimer`.
@Published private(set) var backgroundDeadline: Date?
/// Bounded auto-disconnect for a backgrounded keep-alive session. Fires on `.main`.
private var backgroundTimer: DispatchSourceTimer?
/// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true
/// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host
/// `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair`
@@ -332,6 +354,48 @@ final class SessionModel: ObservableObject {
}
}
// MARK: - Background keep-alive (opt-in, iOS)
/// Enter the backgrounded keep-alive state: keep audio playing, DROP video decode (no GPU work
/// off-screen), mute the mic (privacy), and arm a bounded auto-disconnect. The caller
/// (ContentView's scenePhase driver) gates this on the setting + `.streaming`; a no-op otherwise.
/// The video-drop seam is read by both pumps every iteration (`connection.isVideoDropped`).
func enterBackground(timeoutMinutes: Int) {
guard phase == .streaming, let conn = connection, !isBackgrounded else { return }
isBackgrounded = true
conn.setVideoDropped(true)
audio?.setMicMuted(true)
// Non-deliberate on fire (keep the host linger) so a user who returns late reconnects fast,
// exactly like today's network-drop path. min 1 minute guards a nonsense setting.
let minutes = max(1, timeoutMinutes)
backgroundDeadline = Date().addingTimeInterval(TimeInterval(minutes * 60))
let timer = DispatchSource.makeTimerSource(queue: .main)
timer.schedule(deadline: .now() + .seconds(minutes * 60))
timer.setEventHandler { [weak self] in
// The timer fires on `.main`, so the actor's executor is the main thread here.
MainActor.assumeIsolated { self?.disconnect(deliberate: false) }
}
backgroundTimer?.cancel()
backgroundTimer = timer
timer.resume()
}
/// Return to foreground: cancel the timeout, resume mic + video, and force a clean re-anchor
/// request a fresh IDR (infinite GOP: it won't come on its own) and let the pump's freeze gate
/// withhold the concealed frames until it lands (it auto-arms on the resumed frame-index gap).
func exitBackground() {
guard isBackgrounded else { return }
isBackgrounded = false
backgroundDeadline = nil
backgroundTimer?.cancel()
backgroundTimer = nil
audio?.setMicMuted(false)
if let conn = connection {
conn.setVideoDropped(false)
conn.requestKeyframe()
}
}
/// The user confirmed the fingerprint: returns it for pinning and enters streaming.
func confirmTrust() -> Data? {
guard case .awaitingTrust(let fingerprint) = phase else { return nil }
@@ -349,6 +413,11 @@ final class SessionModel: ObservableObject {
func disconnect(deliberate: Bool = true) {
statsTimer?.invalidate()
statsTimer = nil
// Drop any armed background keep-alive (incl. the timeout that just fired us).
backgroundTimer?.cancel()
backgroundTimer = nil
isBackgrounded = false
backgroundDeadline = nil
let audio = self.audio
self.audio = nil
// Gamepad capture is main-actor (releases held buttons on the wire while the
@@ -361,6 +430,12 @@ final class SessionModel: ObservableObject {
#endif
let feedback = gamepadFeedback
gamepadFeedback = nil
#if os(macOS)
let clipboard = clipboardSync
clipboardSync = nil
#endif
clipboardEnabled = false
clipboardReason = 0
if let conn = connection {
// Drain-thread teardown waits the pullers out and close() waits out in-flight
// polls + joins the Rust worker threads keep all of it off the main actor,
@@ -368,6 +443,9 @@ final class SessionModel: ObservableObject {
Task.detached {
audio?.stop()
feedback?.stop()
#if os(macOS)
clipboard?.stop() // disables sync on the wire while the connection is still up
#endif
// Deliberate user quit tell the host to skip the keep-alive linger (must precede close).
if deliberate { conn.disconnectQuit() }
conn.close()
@@ -376,6 +454,9 @@ final class SessionModel: ObservableObject {
Task.detached {
audio?.stop()
feedback?.stop()
#if os(macOS)
clipboard?.stop()
#endif
}
}
connection = nil
@@ -450,6 +531,14 @@ final class SessionModel: ObservableObject {
let feedback = GamepadFeedback(connection: conn, manager: .shared)
feedback.start()
gamepadFeedback = feedback
#if os(macOS)
// Shared clipboard: opt-in per host AND host-advertised (older hosts / operator-disabled
// hosts never see a ClipControl). Same trust gate as audio nothing is announced
// during the trust prompt.
if activeHost?.clipboardSync == true, conn.hostSupportsClipboard {
startClipboardSync(conn)
}
#endif
#if os(tvOS)
let pointer = SiriRemotePointer(connection: conn)
pointer.onDisconnectRequest = { [weak self] in self?.disconnect() }
@@ -458,6 +547,40 @@ final class SessionModel: ObservableObject {
#endif
}
#if os(macOS)
/// Create + start the session's clipboard bridge and route its host acks into the published
/// UI state. `ClipboardSync.start()` sends the enable; the host's `.state` answer flips
/// `clipboardEnabled` (or leaves it false with a `clipboardReason` the UI can explain).
private func startClipboardSync(_ conn: PunktfunkConnection) {
let sync = ClipboardSync(connection: conn)
sync.onState = { [weak self] enabled, _, reason in
Task { @MainActor in
self?.clipboardEnabled = enabled
self?.clipboardReason = reason
}
}
sync.start()
clipboardSync = sync
}
#endif
/// Flip clipboard sync mid-session (the Stream menu). Off on requires the host cap; on
/// off tears the bridge down (off-main the drain join must not block the main actor) and
/// tells the host, which drops any selection we own there. No-op off-macOS or while idle.
func toggleClipboardSync() {
#if os(macOS)
guard let conn = connection, phase == .streaming else { return }
if let sync = clipboardSync {
clipboardSync = nil
clipboardEnabled = false
clipboardReason = 0
Task.detached { sync.stop() }
} else if conn.hostSupportsClipboard {
startClipboardSync(conn)
}
#endif
}
private func startStatsTimer() {
lastFramesDropped = 0 // a fresh connection's cumulative drop counter starts at 0
latencySplit.reset() // no stale receipts/samples from a previous session
@@ -21,6 +21,12 @@ import SwiftUI
/// `.focusedSceneValue` so the Scene-level commands can drive it.
struct SessionFocus {
var isStreaming: Bool
/// The connected host advertises `HOST_CAP_CLIPBOARD` (gates the Share Clipboard item
/// macOS-only UI, but the fact is platform-neutral).
var clipboardAvailable: Bool
/// Clipboard sync is live (host-acked) drives the item's Stop/Share title.
var clipboardOn: Bool
var toggleClipboard: () -> Void
var disconnect: () -> Void
}
@@ -58,6 +64,15 @@ struct StreamCommands: Commands {
}
.keyboardShortcut("q", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true)
#if os(macOS)
// Mid-session clipboard flip (design/clipboard-and-file-transfer.md §5.3). Greyed
// when the host doesn't advertise the cap (older host / operator policy off).
Button(session?.clipboardOn == true ? "Stop Sharing Clipboard" : "Share Clipboard") {
session?.toggleClipboard()
}
.keyboardShortcut("c", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true || session?.clipboardAvailable != true)
#endif
Divider()
Button("Disconnect") { session?.disconnect() }
.keyboardShortcut("d", modifiers: [.control, .option, .shift])
@@ -440,6 +440,34 @@ extension SettingsView {
}
}
/// iOS/iPadOS only: keep a backgrounded session alive (audio background mode). Empty elsewhere
/// (tvOS backgrounding semantics differ; macOS isn't gated by the mode) so the shared `.general`
/// detail can reference it unconditionally.
@ViewBuilder var keepAliveSection: some View {
#if os(iOS)
Section {
Toggle("Keep streaming in background", isOn: $backgroundKeepAlive)
if backgroundKeepAlive {
Picker("Disconnect after", selection: $backgroundTimeoutMinutes) {
Text("1 minute").tag(1)
Text("5 minutes").tag(5)
Text("10 minutes").tag(10)
Text("30 minutes").tag(30)
}
}
} header: {
Text("Background")
} footer: {
Text("Off by default: backgrounding the app freezes the session. When on, audio keeps "
+ "playing and the connection stays live (video is dropped to save power) after you "
+ "switch away — and the session auto-disconnects after the time above so it can't "
+ "run down your battery. Returning to the app resumes video instantly.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
@ViewBuilder var experimentalSection: some View {
Section {
Toggle("Show game library", isOn: $libraryEnabled)
@@ -49,6 +49,8 @@ struct SettingsView: View {
@ObservedObject var gamepads = GamepadManager.shared
@AppStorage(DefaultsKey.gamepadUIEnabled) var gamepadUIEnabled = true
@AppStorage(DefaultsKey.autoWake) var autoWakeEnabled = true
@AppStorage(DefaultsKey.backgroundKeepAlive) var backgroundKeepAlive = false
@AppStorage(DefaultsKey.backgroundTimeoutMinutes) var backgroundTimeoutMinutes = 10
#if DEBUG && !os(tvOS)
@State var showControllerTest = false
#endif
@@ -242,6 +244,7 @@ struct SettingsView: View {
pointerSection
compositorSection
wakeSection
keepAliveSection // iOS-only content; empty on tvOS
}
.formStyle(.grouped)
.navigationTitle("General")
@@ -11,32 +11,13 @@
import Foundation
import PunktfunkKit
import SwiftUI
#if canImport(WidgetKit)
import WidgetKit
#endif
struct StoredHost: Identifiable, Codable, Hashable {
var id = UUID()
var name: String
var address: String
var port: UInt16 = 9777
/// SHA-256 of the host's certificate, set after the user explicitly trusted it.
var pinnedSHA256: Data?
/// Last time a streaming session actually started (nil until the first one).
var lastConnected: Date?
/// Management-API port for the library browser (distinct from the data-plane `port`). Optional
/// (NOT a defaulted non-optional) so older saved hosts whose JSON lacks this key still
/// decode: synthesized Decodable ignores property defaults but treats a missing Optional as
/// nil. Resolve via `effectiveMgmtPort`. (Auth is mTLS by the pinned identity no token.)
var mgmtPort: UInt16?
/// Wake-on-LAN MAC address(es) of the host's wake-capable NIC(s), each `aa:bb:cc:dd:ee:ff`.
/// Learned from the host's mDNS `mac` TXT record while it's awake and persisted here, so the
/// client can send a magic packet to wake the host later (when it's asleep and no longer
/// advertising). Optional (same forward-compat reason as `mgmtPort`); nil until first learned.
var macAddresses: [String]?
var displayName: String { name.isEmpty ? address : name }
var effectiveMgmtPort: UInt16 { mgmtPort ?? punktfunkDefaultMgmtPort }
/// Wake-capable, in a form the wake helper accepts (empty when none learned yet).
var wakeMacs: [String] { macAddresses ?? [] }
}
// `StoredHost` (the model + its JSON codec) now lives in PunktfunkShared so the widget extension
// can read the same store; PunktfunkKit re-exports it. The discovery-join helpers below stay here
// because they reference PunktfunkKit's `DiscoveredHost`/`HostDiscovery`.
extension StoredHost {
/// True when a live mDNS advert (`DiscoveredHost`) describes THIS saved host drives the
@@ -86,8 +67,14 @@ final class HostStore: ObservableObject {
/// never advertises still reads Online. Not persisted (it's live reachability, not config).
@Published var probedOnline: Set<StoredHost.ID> = []
/// The App-Group suite shared with the Widget/Live-Activity extension so a launcher widget
/// sees the same saved hosts. Falls back to `.standard` in an un-entitled process (see
/// `AppGroup.defaults`).
private let defaults = AppGroup.defaults
init() {
if let data = UserDefaults.standard.data(forKey: Self.key),
Self.migrateToAppGroupIfNeeded()
if let data = defaults.data(forKey: Self.key),
let decoded = try? JSONDecoder().decode([StoredHost].self, from: data) {
hosts = decoded
} else {
@@ -95,6 +82,20 @@ final class HostStore: ObservableObject {
}
}
/// One-time move of the saved-host JSON from `UserDefaults.standard` (where every build before
/// the App Group wrote it) into the shared suite. Idempotent: only fires when the suite has no
/// hosts yet but standard does. The old value is LEFT in place during a staged TestFlight
/// rollout an older build still reads `.standard`, so tombstoning it now would hide hosts from
/// the not-yet-updated app. Remove the standard copy a release later.
private static func migrateToAppGroupIfNeeded() {
let suite = AppGroup.defaults
let standard = UserDefaults.standard
guard suite !== standard else { return } // un-entitled fallback: nothing to migrate
guard suite.data(forKey: key) == nil,
let legacy = standard.data(forKey: key) else { return }
suite.set(legacy, forKey: key)
}
func add(_ host: StoredHost) {
hosts.append(host)
}
@@ -112,7 +113,7 @@ final class HostStore: ObservableObject {
func markConnected(_ hostID: UUID) {
guard let i = hosts.firstIndex(where: { $0.id == hostID }) else { return }
hosts[i].lastConnected = Date()
hosts[i].lastConnected = Date() // didSet persist() writes the shared suite + reloads widget
}
/// One reachability sweep, driving `probedOnline`: probe every saved host NOT currently
@@ -158,7 +159,17 @@ final class HostStore: ObservableObject {
private func persist() {
if let data = try? JSONEncoder().encode(hosts) {
UserDefaults.standard.set(data, forKey: Self.key)
defaults.set(data, forKey: Self.key)
}
reloadHostsWidget() // the widget reads this store; any change refreshes its timeline
}
/// Ask WidgetKit to rebuild the hosts widget's timeline after any store change (add/remove/pin/
/// last-connected). iOS-only and a no-op where WidgetKit is absent; the widget uses
/// `.never`-refresh entries and relies on this push.
private func reloadHostsWidget() {
#if canImport(WidgetKit) && os(iOS)
WidgetCenter.shared.reloadTimelines(ofKind: "PunktfunkHosts")
#endif
}
}
@@ -180,6 +180,23 @@ public final class SessionAudio {
}
}
/// Background keep-alive: silence the mic uplink while backgrounded (privacy no room audio
/// leaves the device) and restore it on return. Pauses/resumes the capture engine; a no-op when
/// there's no uplink (playback-only / tvOS / mic disabled). The audio SESSION stays active for
/// background playback, so iOS may keep showing the recording indicator until a full reconfigure
/// this stops the actual capture, which is the privacy-relevant part. Main thread.
public func setMicMuted(_ muted: Bool) {
stateLock.lock()
let capture = captureEngine
stateLock.unlock()
guard let capture else { return }
if muted {
capture.pause()
} else if !flag.isStopped {
try? capture.start()
}
}
// MARK: - Playback (host speaker)
private func startPlayback(speakerUID: String) {
@@ -0,0 +1,361 @@
// Shared clipboard, macOS client half (design/clipboard-and-file-transfer.md §5.2).
//
// Bridges NSPasteboard.general to the session's QUIC clipboard plane, both directions lazy:
//
// * **Local copy host**: a changeCount poll announces the *format list* (`clipOffer`); the
// bytes cross only when a host app pastes (a `.fetchRequest` event, answered from the live
// pasteboard by `clipServe`).
// * **Host copy local**: a `.remoteOffer` writes one NSPasteboardItem whose
// NSPasteboardItemDataProvider fires only when a Mac app actually pastes the provider then
// blocks (on its provider thread, never main) on a `clipFetch` round-trip.
//
// Password-manager respect: pasteboards marked `org.nspasteboard.ConcealedType` or
// `org.nspasteboard.TransientType` are never announced, never fetchable. Echo suppression: the
// changeCount of every write WE make is recorded so the announce poll skips it (§3.4).
//
// Phase 1 formats only (text / RTF / HTML / PNG). Files (NSFilePromiseProvider) ride Phase 2.
#if os(macOS)
import AppKit
import Foundation
/// One live session's clipboard bridge. Created by the session model when streaming begins on a
/// host that advertises `HOST_CAP_CLIPBOARD` and whose per-host toggle is on; `stop()` before the
/// connection closes. All pasteboard traffic runs on one dedicated drain thread plus the
/// AppKit-owned provider threads (paste fulfillment).
public final class ClipboardSync: NSObject {
/// Wire MIME NSPasteboard type for the Phase-1 vocabulary (§3.5), in announce order.
private static let wireToPasteboard: [(wire: String, type: NSPasteboard.PasteboardType)] = [
("text/plain;charset=utf-8", .string),
("text/rtf", .rtf),
("text/html", .html),
("image/png", .png),
]
/// Pasteboard marker types that must never cross the wire (password managers mark secrets
/// with these see nspasteboard.org).
private static let concealed = NSPasteboard.PasteboardType("org.nspasteboard.ConcealedType")
private static let transient = NSPasteboard.PasteboardType("org.nspasteboard.TransientType")
/// How long a blocked paste waits for the host's bytes before providing nothing (§5.2).
private static let fetchTimeout: TimeInterval = 10
/// Serve chunk size for host-side pastes of our data (bounds the per-call ABI copy).
private static let serveChunk = 4 << 20
private let connection: PunktfunkConnection
/// `CLIP_FLAG_*` sent with the enable (`CLIP_FLAG_FILES` when the session permits files
/// always 0 in Phase 1).
private let controlFlags: UInt8
/// Host `.state` updates, delivered on the main queue drives the toggle/footnote UI.
public var onState: ((_ enabled: Bool, _ policy: UInt8, _ reason: UInt8) -> Void)?
// Drain-thread state (touched only on the drain thread once started).
private var offerSeq: UInt32 = 0
private var lastSeenChangeCount = 0
/// The changeCount of the last pasteboard write WE made (echo suppression + "do we still
/// own the pasteboard" on teardown/clear).
private var ownedChangeCount = -1
/// The host offer currently installed on the local pasteboard (nil = none).
private var installedRemoteSeq: UInt32?
/// Outbound fetches a blocked paste is waiting on. Guarded by `fetchLock` appended by the
/// drain thread (`.data` events), consumed by AppKit's provider threads.
private struct PendingFetch {
var buffer = Data()
let done = DispatchSemaphore(value: 0)
var failed = false
}
private let fetchLock = NSLock()
private var pendingFetches: [UInt32: PendingFetch] = [:]
private final class StopFlag: @unchecked Sendable {
private let lock = NSLock()
private var stopped = false
func stop() {
lock.lock()
stopped = true
lock.unlock()
}
var isStopped: Bool {
lock.lock()
defer { lock.unlock() }
return stopped
}
}
private let flag = StopFlag()
private let drainDone = DispatchSemaphore(value: 0)
private var started = false
/// Set by the app-activation observer, cleared by the drain loop: the user may have copied
/// elsewhere and is coming back to paste announce immediately instead of waiting out the
/// poll interval.
private final class OneShot: @unchecked Sendable {
private let lock = NSLock()
private var raised = false
func raise() {
lock.lock()
raised = true
lock.unlock()
}
func takeIfRaised() -> Bool {
lock.lock()
defer { lock.unlock() }
let was = raised
raised = false
return was
}
}
private let checkNow = OneShot()
private var activationObserver: NSObjectProtocol?
public init(connection: PunktfunkConnection, allowFiles: Bool = false) {
self.connection = connection
self.controlFlags = 0 // CLIP_FLAG_FILES rides Phase 2
_ = allowFiles
super.init()
}
deinit { flag.stop() }
/// Enable sync with the host and start the drain thread. The host answers the enable with a
/// `.state` event (surfaced via `onState`) `BACKEND_UNAVAILABLE` et al. arrive there.
public func start() {
guard !started else { return }
started = true
connection.clipControl(enabled: true, flags: controlFlags)
// Baseline: whatever is on the pasteboard when sync starts is announced immediately
// the "copy first, then connect and paste" flow must work.
lastSeenChangeCount = -1
activationObserver = NotificationCenter.default.addObserver(
forName: NSApplication.didBecomeActiveNotification, object: nil, queue: nil
) { [checkNow] _ in checkNow.raise() }
let connection = self.connection
let flag = self.flag
let thread = Thread { [weak self] in
var lastAnnounceCheck = Date.distantPast
while !flag.isStopped {
// Drain events (bounded burst so a chatty host can't starve the announce poll).
var drained = 0
while drained < 32, !flag.isStopped {
let ev: PunktfunkConnection.ClipEvent?
do {
ev = try connection.nextClipboard(timeoutMs: drained == 0 ? 200 : 0)
} catch {
flag.stop() // session closed
break
}
guard let ev else { break }
drained += 1
self?.handle(ev)
}
// Announce poll: every 500 ms, or immediately after app activation (§5.2).
let now = Date()
if now.timeIntervalSince(lastAnnounceCheck) >= 0.5
|| self?.checkNow.takeIfRaised() == true
{
lastAnnounceCheck = now
self?.announceIfChanged()
}
}
self?.drainDone.signal()
}
thread.name = "punktfunk-clipboard"
thread.qualityOfService = .utility
thread.start()
}
/// Disable sync and join the drain thread. Called off-main before `connection.close()`
/// (the same discipline as the audio/feedback drains). If the local pasteboard still holds
/// our remote-offer items, they are cleared their providers die with us.
public func stop() {
guard started else { return }
started = false
if let obs = activationObserver {
NotificationCenter.default.removeObserver(obs)
activationObserver = nil
}
connection.clipControl(enabled: false, flags: 0)
flag.stop()
drainDone.wait()
// Fail every paste still blocked on us so no provider thread waits out its timeout.
fetchLock.lock()
for (_, pending) in pendingFetches {
pending.done.signal()
}
pendingFetches.removeAll()
fetchLock.unlock()
let pb = NSPasteboard.general
if installedRemoteSeq != nil, pb.changeCount == ownedChangeCount {
pb.clearContents()
}
}
// MARK: - Local copy host (announce)
/// Announce the local pasteboard's format list when it changed (skipping our own writes and
/// concealed/transient pasteboards). Runs on the drain thread.
private func announceIfChanged() {
let pb = NSPasteboard.general
let count = pb.changeCount
guard count != lastSeenChangeCount else { return }
lastSeenChangeCount = count
if count == ownedChangeCount { return } // our own write (a remote offer) never echo
installedRemoteSeq = nil // a local copy replaced the host's offer
let types = pb.types ?? []
if types.contains(Self.concealed) || types.contains(Self.transient) { return }
offerSeq &+= 1
let kinds = Self.wireToPasteboard
.filter { types.contains($0.type) }
.map { PunktfunkConnection.ClipKind(mime: $0.wire) }
// Empty = the pasteboard holds nothing we sync (or was cleared) clears the host side.
connection.clipOffer(seq: offerSeq, kinds: kinds)
}
// MARK: - Event handling (drain thread)
private func handle(_ ev: PunktfunkConnection.ClipEvent) {
switch ev {
case let .state(enabled, policy, reason):
if let onState {
DispatchQueue.main.async { onState(enabled, policy, reason) }
}
case let .remoteOffer(seq, kinds):
installRemoteOffer(seq: seq, kinds: kinds)
case let .fetchRequest(reqId, seq, _, mime):
serveFetch(reqId: reqId, seq: seq, mime: mime)
case let .data(xferId, chunk, last):
fetchLock.lock()
if var pending = pendingFetches[xferId] {
pending.buffer.append(chunk)
pendingFetches[xferId] = pending
if last {
pendingFetches[xferId]?.done.signal()
}
}
fetchLock.unlock()
case let .cancelled(id), let .error(id, _):
fetchLock.lock()
if var pending = pendingFetches[id] {
pending.failed = true
pendingFetches[id] = pending
pending.done.signal()
}
fetchLock.unlock()
}
}
// MARK: - Host copy local (lazy install + blocked-paste fetch)
/// Write one NSPasteboardItem advertising the host's formats, each backed by a lazy data
/// provider bytes cross only when a Mac app pastes. Empty `kinds` = the host cleared its
/// clipboard: drop our item if it's still current.
private func installRemoteOffer(seq: UInt32, kinds: [PunktfunkConnection.ClipKind]) {
let pb = NSPasteboard.general
let types = kinds.compactMap { kind in
Self.wireToPasteboard.first(where: { $0.wire == kind.mime })?.type
}
guard !types.isEmpty else {
if installedRemoteSeq != nil, pb.changeCount == ownedChangeCount {
pb.clearContents()
ownedChangeCount = pb.changeCount
lastSeenChangeCount = pb.changeCount
}
installedRemoteSeq = nil
return
}
let item = NSPasteboardItem()
item.setDataProvider(RemoteOfferProvider(sync: self, seq: seq), forTypes: types)
pb.clearContents()
pb.writeObjects([item])
installedRemoteSeq = seq
ownedChangeCount = pb.changeCount
lastSeenChangeCount = pb.changeCount
}
/// Blocked-paste fulfillment: fetch one wire format of host offer `seq` and wait (provider
/// thread) for the drain thread to assemble the chunks. Nil on timeout/cancel/error the
/// paste then provides nothing rather than hanging (§3.4).
///
/// `fetchLock` is held ACROSS the `clipFetch` so the pending entry exists before the drain
/// thread can process the first `.data` event (its `handle` takes `fetchLock` after
/// releasing the connection's clipboard lock no cycle).
fileprivate func fetchBlocking(seq: UInt32, wireMime: String) -> Data? {
fetchLock.lock()
guard let xferId = connection.clipFetch(seq: seq, mime: wireMime) else {
fetchLock.unlock()
return nil
}
pendingFetches[xferId] = PendingFetch()
let done = pendingFetches[xferId]!.done
fetchLock.unlock()
let outcome = done.wait(timeout: .now() + Self.fetchTimeout)
fetchLock.lock()
let pending = pendingFetches.removeValue(forKey: xferId)
fetchLock.unlock()
if outcome == .timedOut {
connection.clipCancel(id: xferId)
return nil
}
guard let pending, !pending.failed else { return nil }
return pending.buffer
}
// MARK: - Host paste of our data (serve)
/// Answer a host paste of our offered data from the live pasteboard. A stale `seq` (the
/// local clipboard changed since that announce) is cancelled never serve mismatched bytes.
private func serveFetch(reqId: UInt32, seq: UInt32, mime: String) {
let pb = NSPasteboard.general
guard seq == offerSeq, pb.changeCount == lastSeenChangeCount,
let type = Self.wireToPasteboard.first(where: { $0.wire == mime })?.type,
let data = pb.data(forType: type)
else {
connection.clipCancel(id: reqId)
return
}
var offset = 0
while offset < data.count {
let end = min(offset + Self.serveChunk, data.count)
connection.clipServe(
reqId: reqId, data: data.subdata(in: offset..<end), last: end == data.count)
offset = end
}
if data.isEmpty {
connection.clipServe(reqId: reqId, data: Data(), last: true)
}
}
}
/// The lazy paste hook: AppKit calls `provideDataForType` only when a Mac app actually pastes;
/// the fetch then blocks this provider thread (never main) until the host's bytes arrive or the
/// timeout provides nothing. One provider per installed remote offer a dead sync (weak) or a
/// superseded offer provides nothing.
private final class RemoteOfferProvider: NSObject, NSPasteboardItemDataProvider {
private weak var sync: ClipboardSync?
private let seq: UInt32
init(sync: ClipboardSync, seq: UInt32) {
self.sync = sync
self.seq = seq
}
func pasteboard(
_ pasteboard: NSPasteboard?, item: NSPasteboardItem,
provideDataForType type: NSPasteboard.PasteboardType
) {
guard let sync,
let wire = wireMime(for: type),
let data = sync.fetchBlocking(seq: seq, wireMime: wire)
else { return }
item.setData(data, forType: type)
}
private func wireMime(for type: NSPasteboard.PasteboardType) -> String? {
switch type {
case .string: return "text/plain;charset=utf-8"
case .rtf: return "text/rtf"
case .html: return "text/html"
case .png: return "image/png"
default: return nil
}
}
}
#endif
@@ -11,6 +11,9 @@
// LaunchSpec schema in `crates/punktfunk-host/src/library.rs`.
import Foundation
// `punktfunkDefaultMgmtPort` (and StoredHost/DefaultsKey) now live in PunktfunkShared so the
// dependency-free widget extension can share them; PunktfunkKit re-exports the module.
import PunktfunkShared
/// Cover art URLs (the public Steam CDN for Steam titles, user-supplied for custom entries).
public struct Artwork: Codable, Hashable, Sendable {
@@ -64,10 +67,6 @@ public enum LibraryError: LocalizedError {
}
}
/// The management API's default port adjacent to the GameStream block; matches
/// `mgmt::DEFAULT_PORT` on the host.
public let punktfunkDefaultMgmtPort: UInt16 = 47990
/// Stateless fetcher for a host's library.
public enum LibraryClient {
/// `GET https://<address>:<port>/api/v1/library`, authenticated by **mTLS**: the client
@@ -196,6 +196,10 @@ public final class PunktfunkConnection {
/// Same role for the host-timing (0xCF) puller its own plane in the core, drained
/// non-blockingly by the app's 1 s stats tick (never contends with the blocking pullers).
private let statsLock = NSLock()
/// Same role for the shared-clipboard drain thread (`nextClipboard` its own plane in the
/// core). The clip *sends* (`clipControl`/`clipOffer`/`clipServe`) share this lock too:
/// they're quick non-blocking enqueues, and a single lock keeps close() ordering simple.
private let clipboardLock = NSLock()
/// Negotiated session mode (host-confirmed).
public private(set) var width: UInt32 = 0
@@ -346,6 +350,16 @@ public final class PunktfunkConnection {
/// parse-window size for `USER_FLAG_CHUNK_ALIGNED` PyroWave AUs (plan §4.4). Other codecs
/// never need it.
public private(set) var shardPayload: UInt32 = 1408
/// The host capability bitfield (`Welcome.host_caps`): `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` /
/// `PUNKTFUNK_HOST_CAP_CLIPBOARD`. `0` for an older host that didn't say.
public private(set) var hostCaps: UInt8 = 0
/// Whether this host advertises the shared clipboard (`HOST_CAP_CLIPBOARD`) the gate for
/// offering the clipboard toggle. Absent on an older host, or one whose operator policy
/// (`PUNKTFUNK_CLIPBOARD=off`) keeps the feature dark.
public var hostSupportsClipboard: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_CLIPBOARD) != 0
}
/// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing
/// (Annex-B NAL parsing vs the AV1 OBU repack).
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
@@ -461,6 +475,9 @@ public final class PunktfunkConnection {
var shard: UInt32 = 1408
_ = punktfunk_connection_shard_payload(handle, &shard)
shardPayload = shard
var caps: UInt8 = 0
_ = punktfunk_connection_host_caps(handle, &caps)
hostCaps = caps
}
/// A bandwidth speed-test measurement (see `startSpeedTest`). Partial until `done`.
@@ -534,6 +551,23 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_request_keyframe(h)
}
/// Background-keep-alive video drop (opt-in). While true, both video pumps keep DRAINING
/// `nextAU()` (so QUIC flow control and host pacing stay healthy) but DISCARD each AU before any
/// VideoToolbox/Metal decode or render the crash/jetsam-safe way to hold a backgrounded
/// session (audio keeps rendering; no GPU work off-screen). Set on `SessionModel.enterBackground`,
/// cleared on `exitBackground` (which then requests a fresh IDR; the pump's re-anchor gate
/// auto-arms on the resumed frame-index gap). Its own tiny lock read on the pump thread every
/// iteration, written on the main actor; never contends the ABI/plane locks.
private let videoDropLock = NSLock()
private var videoDropped = false
public var isVideoDropped: Bool {
videoDropLock.lock(); defer { videoDropLock.unlock() }
return videoDropped
}
public func setVideoDropped(_ dropped: Bool) {
videoDropLock.lock(); videoDropped = dropped; videoDropLock.unlock()
}
/// Feed each received AU's `frameIndex` (in receive order) so the client recovers from loss with a
/// cheap reference-frame invalidation instead of always paying for a full IDR. On a forward gap
/// a `frameIndex` jump means the intervening frames were lost and the following AUs reference a
@@ -970,10 +1004,12 @@ public final class PunktfunkConnection {
audioLock.lock()
feedbackLock.lock()
statsLock.lock()
clipboardLock.lock()
abiLock.lock()
let h = handle
handle = nil
abiLock.unlock()
clipboardLock.unlock()
statsLock.unlock()
feedbackLock.unlock()
audioLock.unlock()
@@ -1035,6 +1071,163 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_send_rich_input(h, &rich)
}
// MARK: - Shared clipboard (design/clipboard-and-file-transfer.md §5)
/// One advertised clipboard format in a lazy offer the format list crosses the wire,
/// the bytes only on a fetch.
public struct ClipKind: Sendable, Equatable {
public let mime: String
/// Best-effort size in bytes; `0` = unknown.
public let sizeHint: UInt64
public init(mime: String, sizeHint: UInt64 = 0) {
self.mime = mime
self.sizeHint = sizeHint
}
}
/// A shared-clipboard event from `nextClipboard`. The drain thread turns these into
/// NSPasteboard operations (`ClipboardSync`).
public enum ClipEvent: Sendable, Equatable {
/// The host copied: its lazy format list (empty = the host clipboard was cleared).
/// Fetch a format with `clipFetch(seq:mime:)` when a local app pastes.
case remoteOffer(seq: UInt32, kinds: [ClipKind])
/// Host ack / policy / backend update for `clipControl` (`CLIP_REASON_*`).
case state(enabled: Bool, policy: UInt8, reason: UInt8)
/// The host is pasting OUR offered data answer with `clipServe(reqId:...)`.
case fetchRequest(reqId: UInt32, seq: UInt32, fileIndex: UInt32, mime: String)
/// Bytes for a fetch we started (`last` = final chunk).
case data(xferId: UInt32, chunk: Data, last: Bool)
/// A transfer was cancelled (either side).
case cancelled(id: UInt32)
/// A transfer failed (`status` = a PunktfunkStatus code).
case error(id: UInt32, status: Int32)
}
/// Enable/disable the shared clipboard for this session. Opt-in: nothing is announced or
/// served until enabled. The host answers with a `.state` event carrying the resolved
/// outcome (its operator policy is authoritative). Best-effort a dropped call on a
/// closing session is fine.
public func clipControl(enabled: Bool, flags: UInt8 = 0) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
_ = punktfunk_connection_clipboard_control(h, enabled, flags)
}
/// Announce that the local pasteboard changed the lazy format-list offer (`seq` monotonic,
/// newest wins; empty `kinds` clears the host side). The bytes cross only if the host fetches.
public func clipOffer(seq: UInt32, kinds: [ClipKind]) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
guard !kinds.isEmpty else {
_ = punktfunk_connection_clipboard_offer(h, seq, nil, 0)
return
}
// The C array borrows NUL-terminated strings for the duration of the call only.
let cStrings = kinds.map { strdup($0.mime) }
defer { cStrings.forEach { free($0) } }
let arr = zip(cStrings, kinds).map {
PunktfunkClipKind(mime: $0.map { UnsafePointer($0) }, size_hint: $1.sizeHint)
}
_ = arr.withUnsafeBufferPointer {
punktfunk_connection_clipboard_offer(h, seq, $0.baseAddress, UInt(arr.count))
}
}
/// Start pulling one format of the host's offer `seq` (a local app is pasting). Returns the
/// transfer id echoed on the resulting `.data`/`.error`/`.cancelled` events, or nil when the
/// session is closing.
public func clipFetch(seq: UInt32, mime: String, fileIndex: UInt32 = UInt32.max) -> UInt32? {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return nil }
var xfer: UInt32 = 0
let rc = mime.withCString {
punktfunk_connection_clipboard_fetch(h, seq, $0, fileIndex, &xfer)
}
return rc == statusOK ? xfer : nil
}
/// Provide bytes answering a `.fetchRequest` (the host is pasting our offered data). Call
/// repeatedly to stream; `last = true` completes the transfer. An empty final chunk is fine.
public func clipServe(reqId: UInt32, data: Data, last: Bool) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
if data.isEmpty {
_ = punktfunk_connection_clipboard_serve(h, reqId, nil, 0, last)
} else {
data.withUnsafeBytes { p in
_ = punktfunk_connection_clipboard_serve(
h, reqId, p.bindMemory(to: UInt8.self).baseAddress, UInt(data.count), last)
}
}
}
/// Cancel a clipboard transfer by id an outbound fetch's `xferId` or an inbound
/// `.fetchRequest`'s `reqId`.
public func clipCancel(id: UInt32) {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { return }
_ = punktfunk_connection_clipboard_cancel(h, id)
}
/// Pull the next shared-clipboard event; nil on timeout, throws `.closed` once the session
/// ended. Drain from a single dedicated thread (`ClipboardSync`) the event's borrowed
/// payload is copied into the returned `ClipEvent` before the next poll can overwrite it.
public func nextClipboard(timeoutMs: UInt32) throws -> ClipEvent? {
clipboardLock.lock()
defer { clipboardLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var ev = PunktfunkClipEvent()
let rc = punktfunk_connection_next_clipboard(h, &ev, timeoutMs)
switch rc {
case statusOK:
return Self.decodeClipEvent(ev)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Copy a raw C clip event (whose `data` borrows a per-connection slot) into an owned Swift
/// value. Unknown kinds (a newer core) decode to nil and are skipped by the drain.
private static func decodeClipEvent(_ ev: PunktfunkClipEvent) -> ClipEvent? {
let payload = ev.data.map { Data(bytes: $0, count: Int(ev.len)) } ?? Data()
switch Int32(ev.kind) {
case PUNKTFUNK_CLIP_REMOTE_OFFER:
// One `mime\tsize_hint\n` line per advertised format.
let kinds = String(decoding: payload, as: UTF8.self)
.split(separator: "\n")
.compactMap { line -> ClipKind? in
let parts = line.split(separator: "\t", maxSplits: 1)
guard let mime = parts.first, !mime.isEmpty else { return nil }
let hint = parts.count > 1 ? UInt64(parts[1]) ?? 0 : 0
return ClipKind(mime: String(mime), sizeHint: hint)
}
return .remoteOffer(seq: ev.transfer_id, kinds: kinds)
case PUNKTFUNK_CLIP_STATE:
return .state(enabled: ev.enabled != 0, policy: ev.policy, reason: ev.reason)
case PUNKTFUNK_CLIP_FETCH_REQUEST:
return .fetchRequest(
reqId: ev.transfer_id, seq: ev.seq, fileIndex: ev.file_index,
mime: String(decoding: payload, as: UTF8.self))
case PUNKTFUNK_CLIP_DATA:
return .data(xferId: ev.transfer_id, chunk: payload, last: ev.last != 0)
case PUNKTFUNK_CLIP_CANCELLED:
return .cancelled(id: ev.transfer_id)
case PUNKTFUNK_CLIP_ERROR:
return .error(id: ev.transfer_id, status: ev.status)
default:
return nil
}
}
deinit { close() }
/// Snapshot the handle unless close is pending (callers hold their plane lock).
@@ -23,6 +23,7 @@ import Combine
import CoreHaptics
import Foundation
import GameController
import PunktfunkShared
public final class GamepadFeedback {
private let connection: PunktfunkConnection
@@ -20,6 +20,7 @@
import Combine
import Foundation
import GameController
import PunktfunkShared
@MainActor
public final class GamepadManager: ObservableObject {
@@ -6,6 +6,7 @@
// the two combine without adding a second ObservableObject or an environment key nobody else needs.
import Foundation
import PunktfunkShared
public enum GamepadUIEnvironment {
/// `enabledSetting` is the user's Settings toggle (`DefaultsKey.gamepadUIEnabled`);
@@ -19,6 +19,7 @@
#if os(iOS)
import Foundation
import PunktfunkCore
import PunktfunkShared
import UIKit
/// How touchscreen fingers drive the host persisted under `DefaultsKey.touchMode`, latched
@@ -0,0 +1,9 @@
// PunktfunkShared holds what the app AND the widget extension both need the stored-host model,
// the settings-key names, the App-Group constant, the deep-link grammar, and the Live Activity
// attributes in a module that links neither the Rust core nor the presentation layer.
//
// Re-export it so every existing consumer of PunktfunkKit (`import PunktfunkKit`) keeps seeing
// `StoredHost`, `DefaultsKey`, `punktfunkDefaultMgmtPort`, `DeepLink`, etc. with no call-site churn.
// (Files INSIDE PunktfunkKit still `import PunktfunkShared` explicitly Swift imports are
// file-scoped; the re-export only reaches downstream modules.)
@_exported import PunktfunkShared
@@ -8,6 +8,7 @@
// tap, InputCapture's captured-state S) cycle it directly.
import Foundation
import PunktfunkShared
/// How much of the streaming statistics overlay to show. The raw values are stable on disk
/// rename the cases freely, never the strings.
@@ -9,6 +9,7 @@
#if canImport(Metal) && canImport(QuartzCore)
import AVFoundation
import Foundation
import PunktfunkShared
import QuartzCore
#if os(tvOS)
import UIKit
@@ -38,6 +38,7 @@
import AVFoundation
import Foundation
import Metal
import PunktfunkShared
import QuartzCore
/// PUNKTFUNK_PRESENT_DEBUG=1: the render thread prints a once-per-second line with the decode
@@ -431,6 +432,15 @@ public final class Stage2Pipeline {
while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Background keep-alive: drain one AU (flow control + host pacing) and discard it
// BEFORE any VideoToolbox decode or Metal render no GPU work off-screen. The
// decoder session is left intact; exitBackground requests a fresh IDR and the
// re-anchor gate arms on the resumed frame-index gap so concealed frames are
// withheld until it lands.
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
// Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
// decoder conceals the reference-missing deltas often WITHOUT an error callback
// so key off the drop count climbing, then keep asking (awaitingIDR) until a fresh
@@ -686,6 +696,13 @@ public final class Stage2Pipeline {
while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Background keep-alive: drain + discard before the Metal wavelet decode
// (PyroWave is all-intra, so the resumed frame heals on its own no IDR
// request needed, just no GPU work off-screen).
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
guard let au = try connection.nextAU(timeoutMs: 100) else { return true }
onFrame?(au)
if let newest = newestIndex,
@@ -63,6 +63,14 @@ final class StreamPump {
while alive, !token.isStopped {
alive = autoreleasepool { () -> Bool in
do {
// Background keep-alive: drain one AU to keep QUIC flow control + host pacing
// healthy, then discard it BEFORE any decode/enqueue no VideoToolbox/Metal work
// off-screen. Skips all recovery/gate bookkeeping too; exitBackground requests a
// fresh IDR and the re-anchor gate re-arms on the resumed frame-index gap.
if connection.isVideoDropped {
_ = try connection.nextAU(timeoutMs: 100)
return true
}
// Loss recovery (the primary path). Under the host's infinite GOP the only
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs
// (framesDropped); the decoder then *conceals* the reference-missing deltas a
@@ -19,6 +19,7 @@
#if os(macOS)
import AppKit
import AVFoundation
import PunktfunkShared
import SwiftUI
import os
@@ -35,6 +35,7 @@
import AVFoundation
import GameController
import PunktfunkCore
import PunktfunkShared
import SwiftUI
import UIKit
import os
@@ -0,0 +1,23 @@
// The App-Group foundation shared by the app and its extensions (Widgets / Live Activity).
//
// PunktfunkShared is deliberately dependency-free: it links NEITHER PunktfunkKit (which drags in
// the Rust staticlib + presentation layer) NOR any Apple UI framework. A widget process gets ~30 MB,
// so everything an extension needs the stored-host model + its JSON codec, the settings-key names,
// the deep-link grammar, and (later) the Live Activity attributes lives here and here only.
import Foundation
/// The one App-Group identifier, matched by `Config/*.entitlements`
/// (`com.apple.security.application-groups`). Registered on the developer portal for both the app
/// id (`io.unom.punktfunk`) and the widget extension id (`io.unom.punktfunk.widgets`).
public enum AppGroup {
public static let suiteName = "group.io.unom.punktfunk"
/// The shared defaults suite. Non-nil in a correctly-entitled process; falls back to
/// `.standard` if the group is somehow unavailable (unsigned `swift run`, a misprovisioned
/// build) so the app still functions single-process rather than crashing the widget just
/// won't see the same store there.
public static var defaults: UserDefaults {
UserDefaults(suiteName: suiteName) ?? .standard
}
}
@@ -0,0 +1,57 @@
// The `punktfunk://` deep-link grammar the single builder/parser shared by the widget (which
// emits links via `widgetURL`/`Link`) and the app (`ContentView.onOpenURL`, which routes them into
// the existing connect path). Keeping both sides on one type means the wire format can't drift.
//
// Grammar (v1):
// punktfunk://connect/<host-uuid> connect to a stored host
// punktfunk://connect/<host-uuid>?launch=<GameEntry.id> connect and ask the host to launch it
//
// `launch` carries a `GameEntry.id` (e.g. "steam:570"); it is percent-encoded on build and decoded
// on parse, so ids with reserved characters survive the round trip.
import Foundation
public enum DeepLink: Equatable {
/// Connect to a saved host; `launchID` is a `GameEntry.id` to launch on arrival, if any.
case connect(host: UUID, launchID: String?)
public static let scheme = "punktfunk"
/// Build the canonical URL for a route. Non-optional: every route is representable.
public var url: URL {
switch self {
case let .connect(host, launchID):
var comps = URLComponents()
comps.scheme = Self.scheme
comps.host = "connect"
comps.path = "/\(host.uuidString)"
if let launchID, !launchID.isEmpty {
comps.queryItems = [URLQueryItem(name: "launch", value: launchID)]
}
// URLComponents percent-encodes the query value; force-unwrap is safe for a URL we
// fully control (scheme/host/path are all valid).
return comps.url!
}
}
/// Parse an incoming URL, or nil if it isn't a recognized `punktfunk://` route. Tolerant of
/// case in the scheme and of a trailing slash on the path.
public init?(_ url: URL) {
guard url.scheme?.lowercased() == Self.scheme else { return nil }
guard let comps = URLComponents(url: url, resolvingAgainstBaseURL: false) else { return nil }
switch comps.host?.lowercased() {
case "connect":
// Path is "/<uuid>"; strip the leading slash and any trailing one.
let raw = comps.path
.trimmingCharacters(in: CharacterSet(charactersIn: "/"))
guard let host = UUID(uuidString: raw) else { return nil }
let launch = comps.queryItems?
.first(where: { $0.name == "launch" })?.value
.flatMap { $0.isEmpty ? nil : $0 }
self = .connect(host: host, launchID: launch)
default:
return nil
}
}
}
@@ -1,7 +1,8 @@
// One source of truth for the client's UserDefaults / @AppStorage keys. A magic-string key
// duplicated across a setting's writer (a Settings @AppStorage) and reader (e.g. a stream view
// reading UserDefaults) splits silently on a typo the setting just stops taking effect. These
// live in PunktfunkKit because both the app and the kit's views read them.
// live in the dependency-free PunktfunkShared module (re-exported by PunktfunkKit) because the app,
// the kit's views, AND the widget extension all read them the widget needs `DefaultsKey.hosts`.
import Foundation
@@ -111,6 +112,16 @@ public enum DefaultsKey {
/// routed/VPN host), so connects go straight through instead of waiting out the wake timeout.
/// The explicit "Wake Host" action stays available regardless. Read by ContentView.startSession.
public static let autoWake = "punktfunk.autoWake"
/// iOS/iPadOS: keep a streaming session ALIVE when the app is backgrounded (audio background
/// mode). Off by default (today's freeze-on-background is the default). When on, backgrounding a
/// live session keeps audio playing and the QUIC/pump live while DROPPING video decode, and a
/// bounded timer (`backgroundTimeoutMinutes`) auto-disconnects if the user doesn't return. Read
/// by ContentView's scenePhase driver. Hidden on tvOS/macOS.
public static let backgroundKeepAlive = "punktfunk.backgroundKeepAlive"
/// iOS/iPadOS: minutes a backgrounded keep-alive session runs before auto-disconnecting (a
/// battery/thermal/bandwidth backstop). Default 10; the UI offers 1/5/10/30. The auto-disconnect
/// is non-deliberate (host linger kept), so a late return reconnects fast. Read on enterBackground.
public static let backgroundTimeoutMinutes = "punktfunk.backgroundTimeoutMinutes"
}
extension Notification.Name {
@@ -120,4 +131,15 @@ extension Notification.Name {
/// menus) it exists so the menu item is honest whenever it CAN fire, and as the shortcut's
/// discoverable menu-bar surface.
public static let punktfunkReleaseCapture = Notification.Name("io.unom.punktfunk.release-capture")
/// Posted by the Live Activity's / Shortcuts' End-stream intent (`EndStreamIntent.perform`,
/// which runs in the app's process): the app tears the active session down deliberately
/// (quit-close the host). Same cross-process-signal pattern as `punktfunkReleaseCapture`
/// the intent lives in PunktfunkShared and can't reach the app's `SessionModel` directly.
public static let punktfunkEndActiveSession = Notification.Name("io.unom.punktfunk.end-active-session")
/// Posted by the Connect App Intent (Siri/Shortcuts) with a `punktfunk://` URL as `object`:
/// the app routes it through the SAME `.onOpenURL` handler a widget tap uses (one router, one
/// set of guards). The intent uses `openAppWhenRun`, so the app is foregrounded to receive it.
public static let punktfunkOpenDeepLink = Notification.Name("io.unom.punktfunk.open-deep-link")
}
@@ -0,0 +1,56 @@
// The saved-host as an App Intents entity the parameter type for the Connect/Wake intents and
// the configurable single-host widget. Lives in the shared module (not the app) because widget
// *configuration* intents execute in the EXTENSION process, so the entity can't be app-only.
//
// AppIntents is genuinely available on macOS (13+), so this is gated on `canImport(AppIntents)`
// (unlike ActivityKit, whose macOS types are unavailable) it compiles on every platform and the
// entity query reads the same shared App-Group store the widget does.
#if canImport(AppIntents)
import AppIntents
import Foundation
public struct HostEntity: AppEntity, Identifiable {
public static let typeDisplayRepresentation = TypeDisplayRepresentation(name: "Host")
public static let defaultQuery = HostEntityQuery()
public let id: UUID
public let name: String
public init(id: UUID, name: String) {
self.id = id
self.name = name
}
public init(_ host: StoredHost) {
self.id = host.id
self.name = host.displayName
}
public var displayRepresentation: DisplayRepresentation {
DisplayRepresentation(title: "\(name)")
}
}
public struct HostEntityQuery: EntityQuery {
public init() {}
public func entities(for identifiers: [UUID]) async throws -> [HostEntity] {
Self.loadHosts().filter { identifiers.contains($0.id) }.map(HostEntity.init)
}
/// Sorted most-recent first Siri/Shortcuts and the widget config picker suggest recent hosts.
public func suggestedEntities() async throws -> [HostEntity] {
Self.loadHosts().map(HostEntity.init)
}
static func loadHosts() -> [StoredHost] {
guard let data = AppGroup.defaults.data(forKey: DefaultsKey.hosts),
let hosts = try? JSONDecoder().decode([StoredHost].self, from: data)
else { return [] }
return hosts.sorted {
($0.lastConnected ?? .distantPast) > ($1.lastConnected ?? .distantPast)
}
}
}
#endif
@@ -0,0 +1,68 @@
// The Live Activity's attributes the ONE type that must be identical in the app (which starts
// and updates the Activity) and the widget extension (which renders it). Hence it lives in the
// dependency-free shared module.
//
// Gated on `os(iOS)`, NOT `canImport(ActivityKit)`: ActivityKit *imports* on macOS but its types
// are `@available(macOS, unavailable)`, so canImport would wrongly admit this on the macOS build.
// Live Activities are iPhone/iPad only (iPadOS reports os(iOS)).
//
// Naming/shape is a runtime contract: an Activity started by one build is decoded by the extension
// of the same build, so keep `ContentState` Codable-stable across releases the way `StoredHost` is.
#if os(iOS)
import ActivityKit
import Foundation
public struct PunktfunkSessionAttributes: ActivityAttributes {
// Static for the Activity's whole life (set at request time).
public let hostID: UUID
public let hostName: String
/// The title of the launched game, if the session started from the library; nil for a plain
/// host connect (nothing tracks the live foreground app mid-session).
public let launchTitle: String?
public init(hostID: UUID, hostName: String, launchTitle: String?) {
self.hostID = hostID
self.hostName = hostName
self.launchTitle = launchTitle
}
public struct ContentState: Codable, Hashable {
public enum Stage: String, Codable, Hashable {
case streaming // foreground, live
case background // backgrounded keep-alive (countdown running)
case reconnecting // post-loss re-anchor hold
case ending // torn down final state before dismissal
}
public var stage: Stage
/// Session start drives `Text(timerInterval:)` for a free client-side ticking clock (no
/// per-second push needed).
public var startedAt: Date
/// e.g. "2560×1440 @120 · HEVC · HDR". Updated only when it actually changes.
public var modeLine: String
/// Coarse, updated sparsely (every ~30 s) never the 1 Hz stats firehose.
public var latencyMs: Int?
public var mbps: Double?
/// While backgrounded: when the keep-alive auto-disconnect fires drives the countdown.
public var backgroundDeadline: Date?
public init(
stage: Stage, startedAt: Date, modeLine: String,
latencyMs: Int? = nil, mbps: Double? = nil, backgroundDeadline: Date? = nil
) {
self.stage = stage
self.startedAt = startedAt
self.modeLine = modeLine
self.latencyMs = latencyMs
self.mbps = mbps
self.backgroundDeadline = backgroundDeadline
}
}
}
/// Kind string for the Live Activity kept next to the attributes so app + extension agree.
public enum PunktfunkActivity {
public static let kind = "PunktfunkSession"
}
#endif
@@ -0,0 +1,30 @@
// App Intents that must compile into BOTH the app and the widget extension live here in the shared
// module. Today that's `EndStreamIntent` the Live Activity's "End stream" button (a
// LiveActivityIntent runs in the APP's process) which M4 also surfaces to Siri/Shortcuts.
//
// Gated on os(iOS): LiveActivityIntent is part of ActivityKit's world (iPhone/iPad only). The M4
// Connect/Wake intents that need the app's router live in the app target, not here.
#if os(iOS)
import AppIntents
import Foundation
/// Ends the active streaming session. Backs the Live Activity's End button and the Shortcuts /
/// Siri "End the Punktfunk stream" phrase. `perform()` runs in the app's process (LiveActivityIntent)
/// it posts `.punktfunkEndActiveSession`, which the app's SessionModel owner observes and turns
/// into `disconnect(deliberate: true)` (the user explicitly ended it quit-close the host).
@available(iOS 17.0, *)
public struct EndStreamIntent: LiveActivityIntent {
public static let title: LocalizedStringResource = "End Punktfunk Stream"
public static let description = IntentDescription("Ends the active Punktfunk streaming session.")
public init() {}
public func perform() async throws -> some IntentResult {
await MainActor.run {
NotificationCenter.default.post(name: .punktfunkEndActiveSession, object: nil)
}
return .result()
}
}
#endif
@@ -0,0 +1,63 @@
// The saved-host model + its on-disk JSON wire format the widget/extension depends on BOTH, so
// they live in the dependency-free shared module. The `ObservableObject` store that wraps them
// (`HostStore`, with add/remove/pin/reachability) stays in the app target; discovery-join helpers
// (`matches`, `advertises`) stay there too because they reference PunktfunkKit's `DiscoveredHost`.
//
// Wire-format stability: the JSON encoding of `StoredHost` is now a shared contract between the app
// (writer) and the widget (reader). The `PunktfunkSharedTests` codec round-trip pins it do not
// rename the coding keys or make a stored `Optional` non-optional (older saved JSON must still
// decode; synthesized Decodable treats a missing Optional as nil).
import Foundation
/// The management-API port default (distinct from the data-plane `port`). Lives here (not in
/// PunktfunkKit's LibraryClient, which re-exports it) so `StoredHost.effectiveMgmtPort` can resolve
/// it without the shared module taking a dependency on the kit.
public let punktfunkDefaultMgmtPort: UInt16 = 47990
public struct StoredHost: Identifiable, Codable, Hashable {
public var id = UUID()
public var name: String
public var address: String
public var port: UInt16 = 9777
/// SHA-256 of the host's certificate, set after the user explicitly trusted it.
public var pinnedSHA256: Data?
/// Last time a streaming session actually started (nil until the first one).
public var lastConnected: Date?
/// Management-API port for the library browser (distinct from the data-plane `port`). Optional
/// (NOT a defaulted non-optional) so older saved hosts whose JSON lacks this key still
/// decode: synthesized Decodable ignores property defaults but treats a missing Optional as
/// nil. Resolve via `effectiveMgmtPort`. (Auth is mTLS by the pinned identity no token.)
public var mgmtPort: UInt16?
/// Wake-on-LAN MAC address(es) of the host's wake-capable NIC(s), each `aa:bb:cc:dd:ee:ff`.
/// Learned from the host's mDNS `mac` TXT record while it's awake and persisted here, so the
/// client can send a magic packet to wake the host later (when it's asleep and no longer
/// advertising). Optional (same forward-compat reason as `mgmtPort`); nil until first learned.
public var macAddresses: [String]?
/// Share the clipboard with this host (macOS sessions; design/clipboard-and-file-transfer.md
/// §5.3). Opt-in per host: nil/false = off (nil also keeps older saved JSON decoding same
/// forward-compat reason as `mgmtPort`). Honored only when the host advertises
/// `HOST_CAP_CLIPBOARD`.
public var clipboardSync: Bool?
public init(
id: UUID = UUID(), name: String, address: String, port: UInt16 = 9777,
pinnedSHA256: Data? = nil, lastConnected: Date? = nil, mgmtPort: UInt16? = nil,
macAddresses: [String]? = nil, clipboardSync: Bool? = nil
) {
self.id = id
self.name = name
self.address = address
self.port = port
self.pinnedSHA256 = pinnedSHA256
self.lastConnected = lastConnected
self.mgmtPort = mgmtPort
self.macAddresses = macAddresses
self.clipboardSync = clipboardSync
}
public var displayName: String { name.isEmpty ? address : name }
public var effectiveMgmtPort: UInt16 { mgmtPort ?? punktfunkDefaultMgmtPort }
/// Wake-capable, in a form the wake helper accepts (empty when none learned yet).
public var wakeMacs: [String] { macAddresses ?? [] }
}
@@ -0,0 +1,90 @@
// PunktfunkShared is now a wire contract between the app (writer) and the widget extension +
// deep-link senders (readers). These pin the two formats that cross that boundary:
// the `StoredHost` JSON codec the widget decodes the exact bytes the app persisted, and
// older saved JSON (missing `mgmtPort` / `macAddresses`) must still decode;
// the `punktfunk://` deep-link grammar the widget builds URLs the app parses.
import XCTest
@testable import PunktfunkKit
import PunktfunkShared
final class SharedFoundationTests: XCTestCase {
// MARK: - StoredHost JSON codec
func testStoredHostRoundTrips() throws {
let host = StoredHost(
id: UUID(uuidString: "11111111-2222-3333-4444-555555555555")!,
name: "Tower", address: "192.168.1.173", port: 9777,
pinnedSHA256: Data([0xDE, 0xAD, 0xBE, 0xEF]),
lastConnected: Date(timeIntervalSince1970: 1_700_000_000),
mgmtPort: 47990, macAddresses: ["aa:bb:cc:dd:ee:ff"], clipboardSync: true)
let data = try JSONEncoder().encode(host)
let decoded = try JSONDecoder().decode(StoredHost.self, from: data)
XCTAssertEqual(decoded, host)
}
/// Older saved hosts predate `mgmtPort`/`macAddresses` a missing key must decode to nil, not
/// throw (synthesized Decodable treats a missing Optional as nil). This is the forward-compat
/// guarantee the widget depends on when reading a store written by any prior build.
func testStoredHostDecodesLegacyJSONWithoutOptionalKeys() throws {
let json = """
{"id":"11111111-2222-3333-4444-555555555555","name":"Old","address":"10.0.0.5","port":9777}
""".data(using: .utf8)!
let decoded = try JSONDecoder().decode(StoredHost.self, from: json)
XCTAssertEqual(decoded.name, "Old")
XCTAssertNil(decoded.mgmtPort)
XCTAssertNil(decoded.macAddresses)
XCTAssertNil(decoded.pinnedSHA256)
XCTAssertNil(decoded.lastConnected)
XCTAssertNil(decoded.clipboardSync)
// Resolvers fall back cleanly.
XCTAssertEqual(decoded.effectiveMgmtPort, punktfunkDefaultMgmtPort)
XCTAssertEqual(decoded.wakeMacs, [])
XCTAssertEqual(decoded.displayName, "Old")
}
func testStoredHostDisplayNameFallsBackToAddress() {
let host = StoredHost(name: "", address: "10.0.0.9")
XCTAssertEqual(host.displayName, "10.0.0.9")
}
// MARK: - DeepLink grammar
func testDeepLinkConnectRoundTrips() {
let id = UUID()
let link = DeepLink.connect(host: id, launchID: nil)
let parsed = DeepLink(link.url)
XCTAssertEqual(parsed, link)
XCTAssertEqual(parsed, .connect(host: id, launchID: nil))
}
func testDeepLinkConnectWithLaunchRoundTrips() {
let id = UUID()
// A store-qualified GameEntry.id with a reserved char must survive percent-encoding.
let launch = "steam:570"
let link = DeepLink.connect(host: id, launchID: launch)
XCTAssertEqual(DeepLink(link.url), .connect(host: id, launchID: launch))
}
func testDeepLinkParsesCanonicalString() throws {
let id = UUID()
let url = try XCTUnwrap(URL(string: "punktfunk://connect/\(id.uuidString)"))
XCTAssertEqual(DeepLink(url), .connect(host: id, launchID: nil))
}
func testDeepLinkRejectsForeignSchemeAndBadHost() throws {
let id = UUID()
XCTAssertNil(DeepLink(try XCTUnwrap(URL(string: "https://connect/\(id.uuidString)"))))
XCTAssertNil(DeepLink(try XCTUnwrap(URL(string: "punktfunk://connect/not-a-uuid"))))
XCTAssertNil(DeepLink(try XCTUnwrap(URL(string: "punktfunk://bogus/\(id.uuidString)"))))
}
func testDeepLinkSchemeIsCaseInsensitive() throws {
let id = UUID()
let url = try XCTUnwrap(URL(string: "PUNKTFUNK://connect/\(id.uuidString)"))
XCTAssertEqual(DeepLink(url), .connect(host: id, launchID: nil))
}
}
+2 -2
View File
@@ -60,7 +60,7 @@ impl AudioPlayer {
.spawn(move || {
if let Err(e) = render_thread(pcm_rx, recycle_tx, stop_t, ready_tx, channels as u8)
{
tracing::warn!(error = format!("{e:#}"), "audio playback thread ended");
tracing::warn!(error = %format!("{e:#}"), "audio playback thread ended");
}
})
.context("spawn audio thread")?;
@@ -232,7 +232,7 @@ impl MicStreamer {
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, stop_t) {
tracing::warn!(error = format!("{e:#}"), "mic uplink thread ended");
tracing::warn!(error = %format!("{e:#}"), "mic uplink thread ended");
}
})
.context("spawn mic thread")?;
+1 -1
View File
@@ -1493,7 +1493,7 @@ impl Worker {
Err(e) => {
tracing::warn!(pad = slot.index, low, high, error = %e, "rumble: SDL set_rumble failed")
}
Ok(()) => tracing::debug!(pad = slot.index, low, high, "rumble: rendered"),
Ok(()) => tracing::trace!(pad = slot.index, low, high, "rumble: rendered"),
}
slot.rumble.last = (low, high);
slot.rumble.last_at = Some(Instant::now());
+1 -66
View File
@@ -125,7 +125,7 @@ pub fn agent(
.with_safe_default_protocol_versions()
.map_err(|e| bad("tls config", &e))?
.dangerous()
.with_custom_certificate_verifier(Arc::new(PinVerify { pin }));
.with_custom_certificate_verifier(Arc::new(punktfunk_core::tls::PinVerify::new(pin)));
let cert = rustls::pki_types::CertificateDer::from_pem_slice(identity.0.as_bytes())
.map_err(|e| bad("client cert pem", &e))?;
let key = rustls::pki_types::PrivateKeyDer::from_pem_slice(identity.1.as_bytes())
@@ -251,71 +251,6 @@ fn classify(e: ureq::Error) -> LibraryError {
}
}
/// Fingerprint-pinning verifier — the client-HTTP twin of core's (private) QUIC
/// `PinVerify`: trust is the SHA-256 of the host's self-signed leaf cert. The handshake
/// signatures MUST still be verified for real: CertificateVerify is what proves the peer
/// *holds the pinned cert's private key* — skip it and an active MITM can replay the
/// host's (public) certificate, match the pin, and complete the handshake with its own key.
#[derive(Debug)]
struct PinVerify {
pin: Option<[u8; 32]>,
}
impl rustls::client::danger::ServerCertVerifier for PinVerify {
fn verify_server_cert(
&self,
end_entity: &rustls::pki_types::CertificateDer<'_>,
_intermediates: &[rustls::pki_types::CertificateDer<'_>],
_server_name: &rustls::pki_types::ServerName<'_>,
_ocsp: &[u8],
_now: rustls::pki_types::UnixTime,
) -> Result<rustls::client::danger::ServerCertVerified, rustls::Error> {
if let Some(expected) = self.pin {
let fp = punktfunk_core::quic::endpoint::cert_fingerprint(end_entity.as_ref());
if fp != expected {
return Err(rustls::Error::InvalidCertificate(
rustls::CertificateError::ApplicationVerificationFailure,
));
}
}
Ok(rustls::client::danger::ServerCertVerified::assertion())
}
fn verify_tls12_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::crypto::verify_tls12_signature(
message,
cert,
dss,
&rustls::crypto::ring::default_provider().signature_verification_algorithms,
)
}
fn verify_tls13_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::crypto::verify_tls13_signature(
message,
cert,
dss,
&rustls::crypto::ring::default_provider().signature_verification_algorithms,
)
}
fn supported_verify_schemes(&self) -> Vec<rustls::SignatureScheme> {
rustls::crypto::ring::default_provider()
.signature_verification_algorithms
.supported_schemes()
}
}
#[cfg(test)]
mod tests {
use super::*;
+1 -1
View File
@@ -791,7 +791,7 @@ fn spawn_audio(
buf.extend_from_slice(&pcm[..n]);
player.push(buf);
}
Err(e) => tracing::debug!(error = %e, "opus decode"),
Err(e) => tracing::debug!(error = %e, "opus decode failed"),
}
}
Err(PunktfunkError::NoFrame) => {}
+5 -5
View File
@@ -510,7 +510,7 @@ impl Decoder {
if choice == "vaapi" {
return Err(e.context("PUNKTFUNK_DECODER=vaapi but VAAPI failed"));
}
tracing::info!(reason = %e, "VAAPI unavailable — software decode");
tracing::warn!(error = %e, "VAAPI unavailable — falling back to software decode");
}
}
}
@@ -695,8 +695,8 @@ impl Decoder {
self.backend = Backend::Software(SoftwareDecoder::new(self.codec_id)?);
self.vaapi_fails = 0;
} else {
tracing::warn!(error = %e,
"{which} decode error — requesting keyframe, keeping hardware decode");
tracing::debug!(backend = which, error = %e,
"decode error — requesting keyframe, keeping hardware decode");
}
Ok(None)
}
@@ -1653,7 +1653,7 @@ unsafe extern "C" fn pick_vulkan(
&mut fr,
);
if r < 0 || fr.is_null() {
tracing::warn!("avcodec_get_hw_frames_parameters(VULKAN) failed ({r})");
tracing::warn!(code = r, "avcodec_get_hw_frames_parameters(VULKAN) failed");
return ffi::AVPixelFormat::AV_PIX_FMT_NONE;
}
let fc = (*fr).data as *mut ffi::AVHWFramesContext;
@@ -1665,7 +1665,7 @@ unsafe extern "C" fn pick_vulkan(
as _;
let r = ffi::av_hwframe_ctx_init(fr);
if r < 0 {
tracing::warn!("av_hwframe_ctx_init(VULKAN) failed ({r})");
tracing::warn!(code = r, "av_hwframe_ctx_init(VULKAN) failed");
let mut fr = fr;
ffi::av_buffer_unref(&mut fr);
return ffi::AVPixelFormat::AV_PIX_FMT_NONE;
+4 -3
View File
@@ -537,7 +537,8 @@ impl PyroWaveDecoder {
let fence = device.create_fence(&vk::FenceCreateInfo::default(), None)?;
tracing::info!(
mode = %format!("{width}x{height}"),
width,
height,
"PyroWave decoder open on the presenter's device (compute iDWT, BT.709 limited)"
);
Ok(PyroWaveDecoder {
@@ -602,8 +603,8 @@ impl PyroWaveDecoder {
});
self.next = 0;
tracing::info!(
from = %format!("{}x{}", self.width, self.height),
to = %format!("{width}x{height}"),
from = %format_args!("{}x{}", self.width, self.height),
to = %format_args!("{width}x{height}"),
"PyroWave decoder rebuilt for mid-stream resize"
);
self.width = width;
+47
View File
@@ -0,0 +1,47 @@
# Shared clipboard (plan §W6 shape, design/clipboard-and-file-transfer.md §4): the host-side
# session-clipboard backends — `ext-data-control-v1` (KWin/wlroots/Sway/Hyprland) and Mutter's
# *direct* `org.gnome.Mutter.RemoteDesktop.Session` clipboard on Linux, the Win32 clipboard
# (delayed rendering) on Windows — behind one `HostClipboard`, plus the backend-agnostic session
# coordinator bridging it to the QUIC clipboard plane. The wire protocol and the client half live
# in `punktfunk-core`; the orchestrator consumes only the portable facade (policy / `ClipCoordCmd` /
# `start`), so it stays free of platform cfg.
[package]
name = "pf-clipboard"
version = "0.12.0"
edition = "2021"
rust-version.workspace = true
license = "MIT OR Apache-2.0"
description = "punktfunk host shared clipboard: per-OS session-clipboard backends behind one HostClipboard + the QUIC clipboard-plane coordinator."
publish = false
[dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
anyhow = "1"
tracing = "0.1"
quinn = "0.11"
tokio = { version = "1", features = ["rt", "rt-multi-thread", "sync", "time", "macros"] }
[target.'cfg(target_os = "linux")'.dependencies]
# Mutter's direct RemoteDesktop clipboard is raw D-Bus via `ashpd::zbus` — NOT the xdg
# `org.freedesktop.portal.Clipboard`, which needs an interactive grant a headless host can't
# answer. Reusing ashpd's zbus re-export keeps one zbus version across the workspace.
ashpd = "0.13"
futures-util = "0.3"
# Raw fd plumbing on the paste pipes: `pipe2(O_CLOEXEC)` + `poll` on the data-control receive
# side, `fcntl` un-nonblocking on Mutter's transfer fd.
libc = "0.2"
wayland-client = "0.31"
# `staging`: `ext_data_control_v1` (the session-clipboard protocol) ships in the staging set.
wayland-protocols = { version = "0.32", features = ["client", "staging"] }
[target.'cfg(target_os = "windows")'.dependencies]
# The Win32 clipboard on a hidden message-loop window: `WM_CLIPBOARDUPDATE` listener + OLE
# delayed rendering (`WM_RENDERFORMAT`) for text / CF_HTML / RTF / PNG.
windows = { version = "0.62", features = [
"Win32_Foundation",
"Win32_System_DataExchange",
"Win32_System_LibraryLoader",
"Win32_System_Memory",
"Win32_System_Ole",
"Win32_UI_WindowsAndMessaging",
] }
+409
View File
@@ -0,0 +1,409 @@
//! Host-side shared-clipboard backend.
//!
//! The wire protocol and the client half live in `punktfunk-core`
//! (`punktfunk_core::quic` + `punktfunk_core::clipboard`); this module drives the **host's** real
//! session clipboard so it can offer what a host app copied and paste what the remote client
//! offered (`design/clipboard-and-file-transfer.md` §4).
//!
//! Concrete backends, selected at session start ([`HostClipboard::open`]) and presented as one
//! [`HostClipboard`] to the [`session`] coordinator:
//! * [`wayland`] (Linux) — `ext-data-control-v1` (KWin, wlroots / Sway, Hyprland). Preferred when present.
//! * [`mutter`] (Linux) — GNOME. Mutter implements **no** wlr/ext data-control, but its *direct*
//! `org.gnome.Mutter.RemoteDesktop.Session` D-Bus API carries the same clipboard operations (the
//! xdg `org.freedesktop.portal.Clipboard` would need an interactive grant a headless host can't
//! answer — so we skip it and talk to Mutter directly, as the input injector already does).
//! * [`windows`] — the Win32 clipboard: a hidden message-only window watches `WM_CLIPBOARDUPDATE`
//! and serves client content via OLE delayed rendering (`WM_RENDERFORMAT`).
//!
//! The `zwlr-data-control-unstable-v1` fallback (older wlroots/KWin) is a follow-up. The module
//! compiles on Linux and Windows; the [`session`] coordinator is backend-agnostic.
#[cfg(target_os = "linux")]
mod mutter;
#[cfg(target_os = "linux")]
mod wayland;
#[cfg(target_os = "windows")]
mod windows;
/// Pure Win32-clipboard ↔ wire byte conversions (CF_HTML offset math, UTF-16 text, RTF NUL
/// trimming). Free of any Win32 dependency, so it compiles — and its unit tests run — on any host
/// (`cfg(test)`); the Windows backend is the only production consumer.
#[cfg(any(target_os = "windows", test))]
mod winfmt;
pub mod session;
#[cfg(target_os = "linux")]
use std::io::Write as _;
#[cfg(target_os = "linux")]
use std::os::fd::OwnedFd;
use std::sync::Arc;
/// A clipboard event surfaced by a host backend to the [`session`] coordinator. Both the
/// data-control and Mutter backends emit this identical shape.
pub enum ClipEvent {
/// The host selection changed (a host app copied). `mimes` are the **wire** MIMEs offered (empty
/// = the clipboard was cleared). The coordinator forwards these as a `ClipOffer` to the client;
/// bytes cross only if the client later fetches.
Selection { mimes: Vec<String> },
/// A host app is pasting content the client offered. The coordinator fetches the wire-`mime`
/// bytes from the client and hands them to `responder`.
Paste {
mime: String,
responder: PasteResponder,
},
/// The backend ended (compositor / session gone).
Closed,
}
/// How a backend receives the bytes answering a [`ClipEvent::Paste`]. The two host clipboard
/// mechanisms complete a paste differently, so the coordinator stays agnostic by handing bytes to
/// whichever responder the backend attached.
pub enum PasteResponder {
/// data-control: the compositor handed us the destination pipe on the `send` event — write the
/// bytes and close it (EOF completes the paste).
#[cfg(target_os = "linux")]
Fd(OwnedFd),
/// Mutter: hand the bytes back to the backend actor, which owns the `SelectionWrite` fd and the
/// trailing `SelectionWriteDone` call that Mutter's transfer requires.
#[cfg(target_os = "linux")]
Channel(tokio::sync::oneshot::Sender<Vec<u8>>),
/// Windows: hand the bytes to the `WM_RENDERFORMAT` handler blocking the clipboard message-loop
/// thread, which then `SetClipboardData`s them for the pasting app (`std::sync::mpsc`, since that
/// thread waits synchronously — see [`windows`]).
#[cfg(target_os = "windows")]
Sync(std::sync::mpsc::Sender<Vec<u8>>),
}
impl PasteResponder {
/// Deliver the fetched bytes (empty on a failed fetch → an empty paste, never a hang).
pub async fn respond(self, bytes: Vec<u8>) {
match self {
#[cfg(target_os = "linux")]
PasteResponder::Fd(fd) => {
let _ = tokio::task::spawn_blocking(move || fulfill_paste(fd, &bytes)).await;
}
#[cfg(target_os = "linux")]
PasteResponder::Channel(tx) => {
let _ = tx.send(bytes);
}
#[cfg(target_os = "windows")]
PasteResponder::Sync(tx) => {
let _ = tx.send(bytes);
}
}
}
}
/// Write `bytes` into a paste pipe `fd` and close it (EOF signals the reader). Blocking — run off the
/// reactor for large payloads.
#[cfg(target_os = "linux")]
fn fulfill_paste(fd: OwnedFd, bytes: &[u8]) -> std::io::Result<()> {
let mut file = std::fs::File::from(fd);
file.write_all(bytes)?;
Ok(())
}
/// The active host clipboard backend, chosen per session: `ext-data-control`
/// (KWin/wlroots/Hyprland/Sway) or Mutter's direct RemoteDesktop clipboard (GNOME) on Linux, or the
/// Win32 clipboard on Windows. Presented as one type so the [`session`] coordinator is
/// backend-agnostic.
pub enum HostClipboard {
#[cfg(target_os = "linux")]
DataControl(wayland::ClipboardBackend),
#[cfg(target_os = "linux")]
Mutter(mutter::MutterClipboard),
#[cfg(target_os = "windows")]
Windows(windows::WindowsClipboard),
}
impl HostClipboard {
/// Open whichever backend this session supports. Linux tries data-control first
/// (KWin/wlroots/Hyprland/Sway) then Mutter's direct clipboard (GNOME); Windows opens the Win32
/// clipboard. Errors when none is available (gamescope, no live compositor) — the caller then
/// reports `BACKEND_UNAVAILABLE`.
pub async fn open() -> anyhow::Result<(
HostClipboard,
tokio::sync::mpsc::UnboundedReceiver<ClipEvent>,
)> {
#[cfg(target_os = "linux")]
{
// data-control's bind does blocking Wayland roundtrips — keep them off the reactor.
let dc = tokio::task::spawn_blocking(wayland::ClipboardBackend::open)
.await
.map_err(|e| anyhow::anyhow!("data-control open join: {e}"))?;
match dc {
Ok((b, rx)) => return Ok((HostClipboard::DataControl(b), rx)),
Err(e) => tracing::debug!(
error = format!("{e:#}"),
"no ext-data-control — trying Mutter direct clipboard"
),
}
let (m, rx) = mutter::MutterClipboard::open().await.map_err(|e| {
e.context("no clipboard backend (neither ext-data-control nor Mutter)")
})?;
Ok((HostClipboard::Mutter(m), rx))
}
#[cfg(target_os = "windows")]
{
let (b, rx) = windows::WindowsClipboard::open().await?;
Ok((HostClipboard::Windows(b), rx))
}
}
/// The current host selection's wire MIMEs (empty = nothing to offer).
pub fn current_wire_mimes(&self) -> Vec<String> {
match self {
#[cfg(target_os = "linux")]
HostClipboard::DataControl(b) => b.current_wire_mimes(),
#[cfg(target_os = "linux")]
HostClipboard::Mutter(m) => m.current_wire_mimes(),
#[cfg(target_os = "windows")]
HostClipboard::Windows(w) => w.current_wire_mimes(),
}
}
/// Install a client's offered formats as the host selection.
pub fn set_offer(&self, wire_mimes: &[String]) -> anyhow::Result<()> {
match self {
#[cfg(target_os = "linux")]
HostClipboard::DataControl(b) => b.set_offer(wire_mimes),
#[cfg(target_os = "linux")]
HostClipboard::Mutter(m) => {
m.set_offer(wire_mimes);
Ok(())
}
#[cfg(target_os = "windows")]
HostClipboard::Windows(w) => {
w.set_offer(wire_mimes);
Ok(())
}
}
}
/// Drop the host selection we own.
pub fn clear_offer(&self) -> anyhow::Result<()> {
match self {
#[cfg(target_os = "linux")]
HostClipboard::DataControl(b) => b.clear_offer(),
#[cfg(target_os = "linux")]
HostClipboard::Mutter(m) => {
m.clear_offer();
Ok(())
}
#[cfg(target_os = "windows")]
HostClipboard::Windows(w) => {
w.clear_offer();
Ok(())
}
}
}
/// Read one wire format of the current host selection (a client's fetch). Async: data-control
/// blocks on a pipe (offloaded), Mutter round-trips D-Bus + reads a pipe, Windows reads the
/// clipboard on a blocking thread.
pub async fn read_current(self: &Arc<Self>, wire_mime: &str) -> anyhow::Result<Vec<u8>> {
match &**self {
#[cfg(target_os = "linux")]
HostClipboard::DataControl(_) => {
let me = Arc::clone(self);
let wire = wire_mime.to_string();
tokio::task::spawn_blocking(move || match &*me {
HostClipboard::DataControl(b) => b.read_current(&wire),
_ => unreachable!("variant checked above"),
})
.await
.map_err(|e| anyhow::anyhow!("data-control read join: {e}"))?
}
#[cfg(target_os = "linux")]
HostClipboard::Mutter(m) => m.read_current(wire_mime).await,
#[cfg(target_os = "windows")]
HostClipboard::Windows(w) => w.read_current(wire_mime).await,
}
}
}
// ---- Format normalization (design/clipboard-and-file-transfer.md §3.5) ------------------------
//
// One portable vocabulary crosses the wire; each end maps to platform types at fetch time. Phase 1
// covers text / RTF / HTML / PNG (files are Phase 2). The wire MIMEs match the core's table.
/// Wire MIME for UTF-8 plain text.
pub const WIRE_TEXT: &str = "text/plain;charset=utf-8";
/// Wire MIME for HTML.
pub const WIRE_HTML: &str = "text/html";
/// Wire MIME for rich text.
pub const WIRE_RTF: &str = "text/rtf";
/// Wire MIME for a PNG image.
pub const WIRE_PNG: &str = "image/png";
/// Map a Wayland selection MIME to its canonical wire MIME, or `None` to drop it (internal targets
/// like `TARGETS`/`TIMESTAMP`/`SAVE_TARGETS`, and formats we don't sync in Phase 1). Aliases
/// collapse onto one canonical wire name so the offered list dedups cleanly.
#[cfg(target_os = "linux")]
pub fn wayland_to_wire(wl: &str) -> Option<&'static str> {
// Strip any parameter noise for the plain-text aliases (some apps send `text/plain;charset=...`
// with odd charsets, or bare `text/plain`).
let base = wl.split(';').next().unwrap_or(wl).trim();
match wl {
"text/html" => Some(WIRE_HTML),
"text/rtf" | "application/rtf" | "text/richtext" => Some(WIRE_RTF),
"image/png" => Some(WIRE_PNG),
_ => match base {
"text/plain" | "UTF8_STRING" | "STRING" | "TEXT" => Some(WIRE_TEXT),
_ => None,
},
}
}
/// The Wayland MIME candidates to request, in preference order, when a client fetches `wire` from
/// the host clipboard. The first one present in the current offer is used.
#[cfg(target_os = "linux")]
pub fn wayland_candidates(wire: &str) -> &'static [&'static str] {
match wire {
WIRE_TEXT => &[
"text/plain;charset=utf-8",
"text/plain",
"UTF8_STRING",
"STRING",
"TEXT",
],
WIRE_HTML => &["text/html"],
WIRE_RTF => &["text/rtf", "application/rtf", "text/richtext"],
WIRE_PNG => &["image/png"],
_ => &[],
}
}
/// Pick the Wayland MIME to `receive()` for a wire fetch: the first [`wayland_candidates`] entry the
/// current selection actually advertises.
#[cfg(target_os = "linux")]
pub fn pick_wayland_mime(wire: &str, available: &[String]) -> Option<String> {
wayland_candidates(wire)
.iter()
.find(|c| available.iter().any(|a| a == *c))
.map(|c| c.to_string())
}
/// Normalize a raw Wayland offer's MIME list into the deduplicated wire MIME list announced to the
/// client (drops internal targets; collapses aliases; preserves a stable order).
#[cfg(target_os = "linux")]
pub fn offer_wire_mimes(raw: &[String]) -> Vec<&'static str> {
let mut out: Vec<&'static str> = Vec::new();
for m in raw {
if let Some(wire) = wayland_to_wire(m) {
if !out.contains(&wire) {
out.push(wire);
}
}
}
out
}
/// The Wayland MIMEs to advertise when installing a source for a client's offer. Each wire MIME
/// expands to its canonical Wayland name(s); a rich-text-only offer also advertises `text/plain`
/// so plain-text targets always paste (§3.5 synthesis — destination-side, one direction only).
#[cfg(target_os = "linux")]
pub fn wayland_offers_for(wire_mimes: &[String]) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
let mut push = |s: &str| {
if !out.iter().any(|o| o == s) {
out.push(s.to_string());
}
};
let mut has_plain = false;
let mut has_rich = false;
for w in wire_mimes {
match w.as_str() {
WIRE_TEXT => {
has_plain = true;
push("text/plain;charset=utf-8");
push("text/plain");
push("UTF8_STRING");
push("STRING");
}
WIRE_HTML => {
has_rich = true;
push("text/html");
}
WIRE_RTF => {
has_rich = true;
push("text/rtf");
}
WIRE_PNG => push("image/png"),
other => push(other),
}
}
// Synthesis: rich text without plain text → also advertise plain (the source derives it lazily).
if has_rich && !has_plain {
push("text/plain;charset=utf-8");
push("text/plain");
push("UTF8_STRING");
push("STRING");
}
out
}
#[cfg(all(test, target_os = "linux"))]
mod tests {
use super::*;
#[test]
fn wayland_to_wire_canonicalizes_and_drops_targets() {
assert_eq!(wayland_to_wire("text/plain"), Some(WIRE_TEXT));
assert_eq!(wayland_to_wire("UTF8_STRING"), Some(WIRE_TEXT));
assert_eq!(wayland_to_wire("text/plain;charset=utf-8"), Some(WIRE_TEXT));
assert_eq!(wayland_to_wire("text/html"), Some(WIRE_HTML));
assert_eq!(wayland_to_wire("application/rtf"), Some(WIRE_RTF));
assert_eq!(wayland_to_wire("image/png"), Some(WIRE_PNG));
// Internal targets and unsupported formats are dropped.
assert_eq!(wayland_to_wire("TARGETS"), None);
assert_eq!(wayland_to_wire("TIMESTAMP"), None);
assert_eq!(wayland_to_wire("image/jpeg"), None);
}
#[test]
fn offer_wire_mimes_dedups_aliases() {
let raw = vec![
"TARGETS".to_string(),
"UTF8_STRING".to_string(),
"text/plain;charset=utf-8".to_string(),
"text/plain".to_string(),
"text/html".to_string(),
];
// text aliases collapse to one WIRE_TEXT; TARGETS dropped; html kept.
assert_eq!(offer_wire_mimes(&raw), vec![WIRE_TEXT, WIRE_HTML]);
}
#[test]
fn pick_wayland_mime_prefers_canonical() {
let avail = vec!["text/plain".to_string(), "UTF8_STRING".to_string()];
// Canonical charset form isn't present, so it falls to the next candidate.
assert_eq!(
pick_wayland_mime(WIRE_TEXT, &avail),
Some("text/plain".to_string())
);
let avail2 = vec![
"text/plain;charset=utf-8".to_string(),
"text/plain".to_string(),
];
assert_eq!(
pick_wayland_mime(WIRE_TEXT, &avail2),
Some("text/plain;charset=utf-8".to_string())
);
assert_eq!(pick_wayland_mime(WIRE_PNG, &avail2), None);
}
#[test]
fn wayland_offers_synthesizes_plain_for_rich_only() {
let offers = wayland_offers_for(&[WIRE_HTML.to_string()]);
assert!(offers.iter().any(|m| m == "text/html"));
assert!(
offers.iter().any(|m| m == "text/plain;charset=utf-8"),
"rich-only offer must synthesize plain text: {offers:?}"
);
// Plain already present → no duplicate synthesis, and text aliases included.
let offers2 = wayland_offers_for(&[WIRE_TEXT.to_string()]);
assert!(offers2.iter().any(|m| m == "UTF8_STRING"));
assert_eq!(offers2.iter().filter(|m| *m == "text/plain").count(), 1);
}
}
+524
View File
@@ -0,0 +1,524 @@
//! GNOME clipboard backend via Mutter's **direct** `org.gnome.Mutter.RemoteDesktop.Session` D-Bus
//! API (`design/clipboard-and-file-transfer.md` §4.1).
//!
//! Mutter implements no wlr/ext `data-control` (a deliberate privacy stance), so [`super::wayland`]
//! can't bind on GNOME. But Mutter's own RemoteDesktop session — the same interface the input
//! injector drives directly to dodge the xdg-portal approval dialog (`inject/linux/libei.rs`
//! `connect_mutter`) — carries the full clipboard surface: `EnableClipboard`, `SetSelection`,
//! `SelectionRead`/`SelectionWrite`/`SelectionWriteDone`, and the `SelectionOwnerChanged` /
//! `SelectionTransfer` signals. We open our **own** standalone session for it (it coexists with the
//! injector's input session; validated on GNOME 50), so this backend is self-contained just like the
//! data-control one.
//!
//! One actor task owns the zbus connection + session and multiplexes the two signals with a command
//! channel; the fds Mutter hands out are **non-blocking**, so reads/writes flip them to blocking and
//! run on a blocking thread. Option/signal dict keys are hyphenated: `mime-types`, `session-is-owner`.
use std::collections::HashMap;
use std::io::{Read as _, Write as _};
use std::os::fd::{AsRawFd, OwnedFd};
use std::sync::{Arc, Mutex};
use anyhow::{anyhow, Context, Result};
use ashpd::zbus::{
self,
zvariant::{OwnedObjectPath, OwnedValue, Value},
};
use futures_util::StreamExt;
use tokio::sync::{mpsc, oneshot};
use super::{ClipEvent, PasteResponder};
const RD_BUS: &str = "org.gnome.Mutter.RemoteDesktop";
const RD_PATH: &str = "/org/gnome/Mutter/RemoteDesktop";
const RD_IFACE: &str = "org.gnome.Mutter.RemoteDesktop";
const SESSION_IFACE: &str = "org.gnome.Mutter.RemoteDesktop.Session";
/// Upper bound on one `SelectionRead` (matches the wire clipboard cap, §7).
const CLIP_READ_CAP: u64 = 64 << 20;
/// Handle to the Mutter clipboard actor, held (inside a [`super::HostClipboard`]) by the session
/// coordinator.
pub struct MutterClipboard {
cmd_tx: mpsc::UnboundedSender<Cmd>,
/// Raw MIMEs the current host selection advertises (empty when we own it, or nothing is copied).
/// Written by the actor on `SelectionOwnerChanged`; read for `current_wire_mimes` / fetches.
current_raw: Arc<Mutex<Vec<String>>>,
}
enum Cmd {
SetOffer(Vec<String>),
ClearOffer,
ReadCurrent {
wire: String,
reply: oneshot::Sender<Result<Vec<u8>>>,
},
}
impl MutterClipboard {
/// Create a standalone Mutter RemoteDesktop session, `Start` + `EnableClipboard` it, and spawn
/// the actor. Errors when Mutter isn't running (not GNOME) — the caller falls through to
/// `BACKEND_UNAVAILABLE`.
pub async fn open() -> Result<(MutterClipboard, mpsc::UnboundedReceiver<ClipEvent>)> {
let conn = zbus::Connection::session()
.await
.context("session D-Bus (Mutter clipboard)")?;
let rd = zbus::Proxy::new(&conn, RD_BUS, RD_PATH, RD_IFACE)
.await
.context("Mutter RemoteDesktop proxy (is gnome-shell running?)")?;
let session_path: OwnedObjectPath = rd
.call("CreateSession", &())
.await
.context("Mutter RemoteDesktop.CreateSession (clipboard)")?;
let session = zbus::Proxy::new(&conn, RD_BUS, session_path, SESSION_IFACE)
.await
.context("Mutter RemoteDesktop.Session proxy")?;
session
.call_method("Start", &())
.await
.context("Mutter RemoteDesktop.Session.Start (clipboard)")?;
let empty: HashMap<&str, Value> = HashMap::new();
session
.call_method("EnableClipboard", &(empty,))
.await
.context("Mutter EnableClipboard")?;
let (event_tx, event_rx) = mpsc::unbounded_channel();
let (cmd_tx, cmd_rx) = mpsc::unbounded_channel();
let current_raw = Arc::new(Mutex::new(Vec::new()));
tokio::spawn(actor(conn, session, cmd_rx, event_tx, current_raw.clone()));
tracing::info!("clipboard backend bound (Mutter RemoteDesktop direct)");
Ok((
MutterClipboard {
cmd_tx,
current_raw,
},
event_rx,
))
}
/// Install a client's offered formats as the host selection (fire-and-forget on the actor).
pub fn set_offer(&self, wire_mimes: &[String]) {
let _ = self.cmd_tx.send(Cmd::SetOffer(wire_mimes.to_vec()));
}
/// Relinquish the selection we own.
pub fn clear_offer(&self) {
let _ = self.cmd_tx.send(Cmd::ClearOffer);
}
/// The current host selection's wire MIMEs (empty = nothing / we own it).
pub fn current_wire_mimes(&self) -> Vec<String> {
super::offer_wire_mimes(&self.current_raw.lock().unwrap())
.into_iter()
.map(str::to_string)
.collect()
}
/// Read one wire format of the current host selection (a client's fetch). Round-trips the actor
/// (SelectionRead + a blocking fd read).
pub async fn read_current(&self, wire: &str) -> Result<Vec<u8>> {
let (reply, rx) = oneshot::channel();
self.cmd_tx
.send(Cmd::ReadCurrent {
wire: wire.to_string(),
reply,
})
.map_err(|_| anyhow!("Mutter clipboard actor gone"))?;
rx.await
.map_err(|_| anyhow!("Mutter clipboard read dropped"))?
}
}
/// The actor: owns the connection + session, subscribes to the two clipboard signals, and serves
/// commands. Exits when the command channel closes (session ending) or a signal stream ends.
async fn actor(
conn: zbus::Connection,
session: zbus::Proxy<'static>,
mut cmd_rx: mpsc::UnboundedReceiver<Cmd>,
event_tx: mpsc::UnboundedSender<ClipEvent>,
current_raw: Arc<Mutex<Vec<String>>>,
) {
let (mut owner, mut transfer) = match (
session.receive_signal("SelectionOwnerChanged").await,
session.receive_signal("SelectionTransfer").await,
) {
(Ok(o), Ok(t)) => (o, t),
_ => {
tracing::warn!("Mutter clipboard: could not subscribe to selection signals");
let _ = event_tx.send(ClipEvent::Closed);
return;
}
};
loop {
tokio::select! {
sig = owner.next() => {
let Some(msg) = sig else { break };
let Ok((opts,)) = msg.body().deserialize::<(HashMap<String, OwnedValue>,)>() else {
continue;
};
let is_owner = dict_bool(&opts, "session-is-owner").unwrap_or(false);
let raw = dict_mimes(&opts, "mime-types");
if is_owner {
// Our own offer (the client's content) — not host clipboard; don't announce it,
// and clear `current_raw` so a fetch never reads our own source back.
current_raw.lock().unwrap().clear();
} else {
*current_raw.lock().unwrap() = raw.clone();
let wire = super::offer_wire_mimes(&raw)
.into_iter()
.map(str::to_string)
.collect();
if event_tx.send(ClipEvent::Selection { mimes: wire }).is_err() {
break;
}
}
}
sig = transfer.next() => {
let Some(msg) = sig else { break };
let Ok((mime, serial)) = msg.body().deserialize::<(String, u32)>() else {
continue;
};
match super::wayland_to_wire(&mime) {
Some(wire) => {
// A host app pastes our offer: hand the fetch to the coordinator, then serve
// the returned bytes into the SelectionWrite fd off-task. NB Mutter issues
// *two* transfers per read (a size probe + the real read), so the coordinator
// fetches from the client twice per paste — correct, just not deduplicated.
let (tx, rx) = oneshot::channel();
if event_tx
.send(ClipEvent::Paste {
mime: wire.to_string(),
responder: PasteResponder::Channel(tx),
})
.is_err()
{
break;
}
let session = session.clone();
tokio::spawn(async move {
let bytes = rx.await.unwrap_or_default();
serve_write(&session, serial, bytes).await;
});
}
// Format we don't sync — fail the transfer cleanly.
None => serve_write(&session, serial, Vec::new()).await,
}
}
cmd = cmd_rx.recv() => {
let Some(cmd) = cmd else { break }; // coordinator gone → session ending
match cmd {
Cmd::SetOffer(wire) => {
let wl = super::wayland_offers_for(&wire);
if let Err(e) = set_selection(&session, &wl).await {
tracing::debug!(error = %e, "Mutter SetSelection failed");
}
}
Cmd::ClearOffer => {
if let Err(e) = set_selection(&session, &[]).await {
tracing::debug!(error = %e, "Mutter clear selection failed");
}
}
Cmd::ReadCurrent { wire, reply } => {
let raw = current_raw.lock().unwrap().clone();
let _ = reply.send(read_selection(&session, &wire, &raw).await);
}
}
}
}
}
// Keep the connection owned for the actor's whole life (Mutter ties the session to it).
drop(conn);
let _ = event_tx.send(ClipEvent::Closed);
}
/// Offer `wl_mimes` as the host selection; an empty list relinquishes ownership.
async fn set_selection(session: &zbus::Proxy<'_>, wl_mimes: &[String]) -> Result<()> {
let mut opts: HashMap<&str, Value> = HashMap::new();
if !wl_mimes.is_empty() {
let refs: Vec<&str> = wl_mimes.iter().map(String::as_str).collect();
opts.insert("mime-types", Value::from(refs));
}
session
.call_method("SetSelection", &(opts,))
.await
.context("Mutter SetSelection")?;
Ok(())
}
/// Read the current selection's `wire` format: pick a concrete offered MIME, `SelectionRead` it, and
/// read the (non-blocking) fd to EOF on a blocking thread.
async fn read_selection(session: &zbus::Proxy<'_>, wire: &str, raw: &[String]) -> Result<Vec<u8>> {
let mime =
super::pick_wayland_mime(wire, raw).context("format not offered by the host clipboard")?;
let fd: zbus::zvariant::OwnedFd = session
.call("SelectionRead", &(mime.as_str(),))
.await
.context("Mutter SelectionRead")?;
let fd = OwnedFd::from(fd);
tokio::task::spawn_blocking(move || read_fd_to_end(fd))
.await
.map_err(|e| anyhow!("SelectionRead join: {e}"))?
}
/// Serve one `SelectionTransfer`: `SelectionWrite` → write `bytes` → `SelectionWriteDone`. Any write
/// failure still reports done (success=false) so Mutter completes the transfer.
async fn serve_write(session: &zbus::Proxy<'_>, serial: u32, bytes: Vec<u8>) {
let ok = match write_selection(session, serial, bytes).await {
Ok(()) => true,
Err(e) => {
tracing::debug!(error = %e, "Mutter SelectionWrite failed");
false
}
};
let _ = session
.call_method("SelectionWriteDone", &(serial, ok))
.await;
}
async fn write_selection(session: &zbus::Proxy<'_>, serial: u32, bytes: Vec<u8>) -> Result<()> {
let fd: zbus::zvariant::OwnedFd = session
.call("SelectionWrite", &(serial,))
.await
.context("Mutter SelectionWrite")?;
let fd = OwnedFd::from(fd);
tokio::task::spawn_blocking(move || write_fd(fd, &bytes))
.await
.map_err(|e| anyhow!("SelectionWrite join: {e}"))?
}
/// Read a Mutter clipboard fd to EOF (capped). The fd is `O_NONBLOCK`; flip it to blocking first.
fn read_fd_to_end(fd: OwnedFd) -> Result<Vec<u8>> {
set_blocking(&fd)?;
let file = std::fs::File::from(fd);
let mut buf = Vec::new();
file.take(CLIP_READ_CAP)
.read_to_end(&mut buf)
.context("read SelectionRead fd")?;
Ok(buf)
}
/// Write `bytes` into a Mutter clipboard fd and close it (EOF). Flip the `O_NONBLOCK` fd to blocking.
fn write_fd(fd: OwnedFd, bytes: &[u8]) -> Result<()> {
set_blocking(&fd)?;
let mut file = std::fs::File::from(fd);
file.write_all(bytes).context("write SelectionWrite fd")?;
Ok(())
}
/// Peel any `Value::Value` (variant) wrappers to the concrete value. The `a{sv}` dict values Mutter
/// sends arrive as variants, so a plain `TryFrom<OwnedValue>` (which matches the concrete type) never
/// sees through them — this strips the layer first.
fn peel<'a>(v: &'a Value<'a>) -> &'a Value<'a> {
let mut cur = v;
while let Value::Value(inner) = cur {
cur = inner;
}
cur
}
/// Extract a boolean dict entry (e.g. `session-is-owner`), unwrapping the variant.
fn dict_bool(opts: &HashMap<String, OwnedValue>, key: &str) -> Option<bool> {
match peel(opts.get(key)?) {
Value::Bool(b) => Some(*b),
_ => None,
}
}
/// Extract a string-array dict entry (e.g. `mime-types`), unwrapping the variant. Mutter wraps the
/// array in a single-field struct (`(as)`, seen on `SelectionOwnerChanged`), so unwrap that too.
fn dict_mimes(opts: &HashMap<String, OwnedValue>, key: &str) -> Vec<String> {
let Some(v) = opts.get(key) else {
return Vec::new();
};
let mut val = peel(v);
if let Value::Structure(s) = val {
match s.fields().first() {
Some(first) => val = peel(first),
None => return Vec::new(),
}
}
let Value::Array(arr) = val else {
return Vec::new();
};
arr.inner()
.iter()
.filter_map(|e| match peel(e) {
Value::Str(s) => Some(s.to_string()),
_ => None,
})
.collect()
}
/// Clear `O_NONBLOCK` on a Mutter clipboard fd so a blocking `spawn_blocking` read/write works.
fn set_blocking(fd: &OwnedFd) -> Result<()> {
let raw = fd.as_raw_fd();
// SAFETY: `raw` is a valid fd owned by `fd` for the duration of these fcntl calls.
let flags = unsafe { libc::fcntl(raw, libc::F_GETFL) };
if flags < 0 {
return Err(anyhow!(
"fcntl F_GETFL: {}",
std::io::Error::last_os_error()
));
}
// SAFETY: as above; clearing O_NONBLOCK on our own fd.
let rc = unsafe { libc::fcntl(raw, libc::F_SETFL, flags & !libc::O_NONBLOCK) };
if rc < 0 {
return Err(anyhow!(
"fcntl F_SETFL: {}",
std::io::Error::last_os_error()
));
}
Ok(())
}
/// On-glass tests against a **live GNOME/Mutter** session (`WAYLAND_DISPLAY=wayland-0`). `#[ignore]`d
/// — run explicitly under GNOME (Mutter has no `wl-clipboard`, so a second Mutter session plays the
/// "host app"):
///
/// ```text
/// cargo test -p punktfunk-host --bin punktfunk-host -- --ignored --nocapture clipboard::mutter::live
/// ```
///
/// Skips (does not fail) when Mutter isn't running, so `--ignored` off-GNOME is a clean no-op.
#[cfg(test)]
mod live {
use super::*;
use std::time::Duration;
/// A second Mutter session standing in for a host clipboard app.
struct Helper {
session: zbus::Proxy<'static>,
_conn: zbus::Connection,
}
impl Helper {
async fn open() -> Result<Helper> {
let conn = zbus::Connection::session().await?;
let rd = zbus::Proxy::new(&conn, RD_BUS, RD_PATH, RD_IFACE).await?;
let path: OwnedObjectPath = rd.call("CreateSession", &()).await?;
let session = zbus::Proxy::new(&conn, RD_BUS, path, SESSION_IFACE).await?;
session.call_method("Start", &()).await?;
let empty: HashMap<&str, Value> = HashMap::new();
session.call_method("EnableClipboard", &(empty,)).await?;
Ok(Helper {
session,
_conn: conn,
})
}
/// Own the selection offering plain text, serving `payload` on every transfer request.
async fn offer_text(&self, payload: &'static [u8]) {
let mut transfer = self
.session
.receive_signal("SelectionTransfer")
.await
.unwrap();
let session = self.session.clone();
tokio::spawn(async move {
while let Some(msg) = transfer.next().await {
if let Ok((_mime, serial)) = msg.body().deserialize::<(String, u32)>() {
serve_write(&session, serial, payload.to_vec()).await;
}
}
});
set_selection(
&self.session,
&[
"text/plain;charset=utf-8".to_string(),
"text/plain".to_string(),
],
)
.await
.unwrap();
}
/// Paste the current selection's plain text.
async fn read_text(&self) -> Vec<u8> {
let fd: zbus::zvariant::OwnedFd = self
.session
.call("SelectionRead", &("text/plain;charset=utf-8",))
.await
.unwrap();
let fd = OwnedFd::from(fd);
tokio::task::spawn_blocking(move || read_fd_to_end(fd))
.await
.unwrap()
.unwrap()
}
}
async fn next_selection(
rx: &mut mpsc::UnboundedReceiver<ClipEvent>,
timeout: Duration,
) -> Option<Vec<String>> {
tokio::time::timeout(timeout, async {
loop {
match rx.recv().await {
Some(ClipEvent::Selection { mimes }) if !mimes.is_empty() => {
return Some(mimes)
}
Some(_) => continue,
None => return None,
}
}
})
.await
.ok()
.flatten()
}
#[test]
#[ignore = "needs a live GNOME/Mutter session (WAYLAND_DISPLAY=wayland-0)"]
fn live_mutter_roundtrip() {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(2)
.enable_all()
.build()
.unwrap();
rt.block_on(async {
let (backend, mut rx) = match MutterClipboard::open().await {
Ok(v) => v,
Err(e) => {
eprintln!("SKIP: no Mutter clipboard (not GNOME?): {e:#}");
return;
}
};
let helper = Helper::open().await.expect("helper Mutter session");
// --- host copy → backend observes Selection + read_current returns the bytes ---
helper.offer_text(b"gnome-host-copied").await;
let mimes = next_selection(&mut rx, Duration::from_secs(3))
.await
.expect("Selection after the helper offered text");
assert!(
mimes.iter().any(|m| m == super::super::WIRE_TEXT),
"offer carries wire text: {mimes:?}"
);
let got = backend
.read_current(super::super::WIRE_TEXT)
.await
.expect("read_current text");
assert_eq!(got, b"gnome-host-copied");
// --- backend offers client content → the host app (helper) pastes it ---
backend.set_offer(&[super::super::WIRE_TEXT.to_string()]);
tokio::time::sleep(Duration::from_millis(500)).await; // let SetSelection take effect
// Answer every Paste request the host app (helper) triggers, until the read completes.
let paste_side = async {
while let Some(ev) = rx.recv().await {
if let ClipEvent::Paste { responder, .. } = ev {
responder.respond(b"punktfunk-served".to_vec()).await;
}
}
};
let read = tokio::select! {
r = helper.read_text() => r,
_ = paste_side => Vec::new(),
};
assert_eq!(read, b"punktfunk-served");
});
}
}
+254
View File
@@ -0,0 +1,254 @@
//! Host clipboard coordinator (`design/clipboard-and-file-transfer.md` §4.2).
//!
//! One async task per streaming session that bridges the real session clipboard (whichever
//! [`super::HostClipboard`] backend this platform opened) to the QUIC clipboard plane. It owns all
//! four data paths:
//!
//! * **host copy → client** — a backend [`ClipEvent::Selection`] becomes a [`ClipOffer`] pushed to the
//! control loop (`offer_tx`), which forwards it to the client.
//! * **client fetch of the host clipboard** — the fetch-stream `accept_bi` loop lives here; each
//! stream is answered by reading the current host selection ([`ClipboardBackend::read_current`]).
//! * **client copy → host** — a [`ClipCoordCmd::RemoteOffer`] installs the client's formats as a lazy
//! host selection ([`HostClipboard::set_offer`]).
//! * **host paste of client content** — a backend [`ClipEvent::Paste`] triggers an *outbound* fetch to
//! the client, whose bytes are handed to the backend's [`PasteResponder`].
//!
//! The coordinator is backend-agnostic (Linux data-control / Mutter, Windows Win32); the control loop
//! reaches it through the portable [`ClipCoordCmd`] channel so the host's native control loop
//! compiles on every host platform.
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::mpsc::{UnboundedReceiver, UnboundedSender};
use punktfunk_core::clipboard::CLIP_FETCH_CAP;
use punktfunk_core::quic::{
clipstream, ClipFetch, ClipFetchHdr, ClipKind, ClipOffer, CLIP_FETCH_OK, CLIP_FETCH_STALE,
CLIP_FETCH_UNAVAILABLE, CLIP_FILE_INDEX_NONE,
};
use super::{ClipEvent, HostClipboard, PasteResponder};
use crate::ClipCoordCmd;
/// Upper bound on one outbound fetch (host pasting client content). A client that never answers must
/// not hang the pasting host app's pipe read (§3.4) — the paste falls back to empty instead.
const FETCH_TIMEOUT: Duration = Duration::from_secs(60);
/// Open whichever host clipboard backend this session supports (data-control, else Mutter direct) and
/// spawn the coordinator. Returns `true` when a live backend was bound (the caller's control loop
/// then serves real clipboard data); `false` when none is available (gamescope, no live compositor),
/// in which case the channels are dropped so the control loop reports `CLIP_REASON_BACKEND_UNAVAILABLE`
/// and declines fetches defensively.
pub async fn start(
conn: quinn::Connection,
clip_enabled: Arc<AtomicBool>,
cmd_rx: UnboundedReceiver<ClipCoordCmd>,
offer_tx: UnboundedSender<ClipOffer>,
) -> bool {
match HostClipboard::open().await {
Ok((backend, clip_rx)) => {
tokio::spawn(run(
conn,
Arc::new(backend),
clip_rx,
clip_enabled,
cmd_rx,
offer_tx,
));
true
}
Err(e) => {
tracing::info!(error = %format!("{e:#}"), "clipboard backend unavailable — fetches will be declined");
false
}
}
}
/// The coordinator loop. Multiplexes control-loop commands, backend clipboard events, and inbound
/// fetch streams; exits when any of the three peers goes away (session ending).
async fn run(
conn: quinn::Connection,
backend: Arc<HostClipboard>,
mut clip_rx: UnboundedReceiver<ClipEvent>,
clip_enabled: Arc<AtomicBool>,
mut cmd_rx: UnboundedReceiver<ClipCoordCmd>,
offer_tx: UnboundedSender<ClipOffer>,
) {
// Seq of the offer the host most recently announced; a client fetch naming a different seq is
// stale (the host clipboard moved on) and is declined.
let host_seq = Arc::new(AtomicU32::new(0));
let mut next_seq: u32 = 1;
// Seq of the client's most recent offer, echoed on the outbound fetch we open when a host app
// pastes client content (informational for the client's serve side).
let mut client_seq: u32 = 0;
loop {
tokio::select! {
cmd = cmd_rx.recv() => {
let Some(cmd) = cmd else { break }; // control loop gone → session ending
match cmd {
ClipCoordCmd::SetEnabled(true) => {
// A just-enabled client should see whatever the host already has copied.
let mimes = backend.current_wire_mimes();
if !mimes.is_empty() {
let _ = offer_tx.send(build_offer(&mut next_seq, &host_seq, mimes));
}
}
ClipCoordCmd::SetEnabled(false) => {
if let Err(e) = backend.clear_offer() {
tracing::debug!(error = %e, "clipboard clear_offer failed");
}
}
ClipCoordCmd::RemoteOffer { seq, mimes } => {
client_seq = seq;
let res = if mimes.is_empty() {
backend.clear_offer()
} else {
backend.set_offer(&mimes)
};
if let Err(e) = res {
tracing::debug!(error = %e, "clipboard apply remote offer failed");
}
}
}
}
ev = clip_rx.recv() => {
let Some(ev) = ev else { break }; // backend dispatch thread ended
match ev {
ClipEvent::Selection { mimes } => {
// Forward host copies (empty `mimes` = the clipboard was cleared) only while
// the client has sync on — the offer is metadata; bytes still cross lazily.
if clip_enabled.load(Ordering::SeqCst) {
let _ = offer_tx.send(build_offer(&mut next_seq, &host_seq, mimes));
}
}
ClipEvent::Paste { mime, responder } => {
// A host app is pasting the client's offered content: pull that format from
// the client and hand it to the backend's responder. Off-task so the loop
// keeps serving.
tokio::spawn(fetch_into_pipe(conn.clone(), client_seq, mime, responder));
}
ClipEvent::Closed => break,
}
}
accepted = conn.accept_bi() => {
let Ok((send, recv)) = accepted else { break }; // connection gone
// The control stream is already accepted at the handshake, so every stream here is a
// clipboard fetch. Serve it off-task (the read blocks on the source app's pipe).
tokio::spawn(serve_fetch(
send,
recv,
Arc::clone(&backend),
Arc::clone(&host_seq),
clip_enabled.load(Ordering::SeqCst),
));
}
}
}
// Session ending: don't leave our lazy source as the compositor's active selection.
let _ = backend.clear_offer();
}
/// Mint a [`ClipOffer`] for `mimes`, advancing the host offer seq (skipping 0, the "never offered"
/// sentinel) and publishing it as the current one for staleness checks.
fn build_offer(next_seq: &mut u32, host_seq: &AtomicU32, mimes: Vec<String>) -> ClipOffer {
let seq = *next_seq;
*next_seq = next_seq.wrapping_add(1);
if *next_seq == 0 {
*next_seq = 1;
}
host_seq.store(seq, Ordering::SeqCst);
let kinds = mimes
.into_iter()
.map(|mime| ClipKind { mime, size_hint: 0 })
.collect();
ClipOffer { seq, kinds }
}
/// Serve one inbound fetch stream (a client pulling the host clipboard): validate the header +
/// request, then answer with the current host selection's bytes for the requested wire MIME.
async fn serve_fetch(
mut send: quinn::SendStream,
mut recv: quinn::RecvStream,
backend: Arc<HostClipboard>,
host_seq: Arc<AtomicU32>,
enabled: bool,
) {
let _ = send.set_priority(-1);
match clipstream::read_stream_header(&mut recv).await {
Ok(k) if k == clipstream::CLIP_STREAM_KIND_FETCH => {}
_ => {
let _ = send.reset(clipstream::cancelled_code());
return;
}
}
let req = match clipstream::read_fetch(&mut recv).await {
Ok(r) => r,
Err(_) => return,
};
let decline = |status: u8| ClipFetchHdr {
status,
total_size: 0,
};
if !enabled {
let _ = clipstream::write_fetch_hdr(&mut send, &decline(CLIP_FETCH_UNAVAILABLE)).await;
return;
}
if req.seq != host_seq.load(Ordering::SeqCst) {
let _ = clipstream::write_fetch_hdr(&mut send, &decline(CLIP_FETCH_STALE)).await;
return;
}
// `read_current` reads the host selection (a blocking pipe read, offloaded by the backend).
match backend.read_current(&req.mime).await {
Ok(data) => {
let hdr = ClipFetchHdr {
status: CLIP_FETCH_OK,
total_size: data.len() as u64,
};
if clipstream::write_fetch_hdr(&mut send, &hdr).await.is_ok() {
let _ = clipstream::write_data(&mut send, &data).await;
}
}
// The format vanished (clipboard changed mid-fetch) or the read failed → nothing to send.
Err(_) => {
let _ = clipstream::write_fetch_hdr(&mut send, &decline(CLIP_FETCH_UNAVAILABLE)).await;
}
}
}
/// Pull `mime` of the client's current offer (`seq`) over an outbound fetch stream and hand the bytes
/// to the backend's paste `responder`. Any failure (timeout, decline, I/O) responds with empty bytes
/// so the pasting host app gets an empty paste instead of hanging.
async fn fetch_into_pipe(
conn: quinn::Connection,
seq: u32,
mime: String,
responder: PasteResponder,
) {
let req = ClipFetch {
seq,
file_index: CLIP_FILE_INDEX_NONE,
mime,
};
let fetched = tokio::time::timeout(FETCH_TIMEOUT, async {
let (send, mut recv) = clipstream::open_fetch(&conn, &req).await.ok()?;
let hdr = clipstream::read_fetch_hdr(&mut recv).await.ok()?;
if hdr.status != CLIP_FETCH_OK {
return None;
}
let data = clipstream::read_data(&mut recv, CLIP_FETCH_CAP)
.await
.ok()?;
drop(send); // clean close of our half
Some(data)
})
.await
.ok()
.flatten();
responder.respond(fetched.unwrap_or_default()).await;
}
+599
View File
@@ -0,0 +1,599 @@
//! `ext-data-control-v1` clipboard backend (`design/clipboard-and-file-transfer.md` §4.1).
//!
//! A dedicated thread owns the `wayland-client` [`EventQueue`] and runs a poll loop that dispatches
//! selection + paste events, emitting them over a channel. Everything else — installing a lazy
//! source (a client's offer) and `receive()`-ing the host selection (a client's fetch) — is issued
//! from the session thread on the shared, `Send + Sync` proxy handles; only *dispatch* is
//! single-threaded (per the wayland-client contract). Templated on `inject/linux/wlr.rs`.
//!
//! The `zwlr-data-control-unstable-v1` fallback for older wlroots/KWin is a mechanical parallel of
//! this file (the protocols are 1:1) — a follow-up.
use std::collections::HashMap;
use std::io::Read;
use std::os::fd::{AsFd, AsRawFd, FromRawFd, OwnedFd};
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
use std::sync::{Arc, Mutex};
use anyhow::{anyhow, Context, Result};
use wayland_client::backend::ObjectId;
use wayland_client::protocol::wl_registry;
use wayland_client::protocol::wl_seat::WlSeat;
use wayland_client::{event_created_child, Connection, Dispatch, Proxy, QueueHandle};
use wayland_protocols::ext::data_control::v1::client::{
ext_data_control_device_v1::{self, ExtDataControlDeviceV1},
ext_data_control_manager_v1::ExtDataControlManagerV1,
ext_data_control_offer_v1::{self, ExtDataControlOfferV1},
ext_data_control_source_v1::{self, ExtDataControlSourceV1},
};
use super::{ClipEvent, PasteResponder};
/// Upper bound on bytes read from one `receive()` transfer (matches the wire clipboard cap, §7) so a
/// hostile host app can't stream unboundedly into our buffer.
const CLIP_READ_CAP: u64 = 64 << 20;
/// The current host selection, shared between the dispatch thread (writer) and the session thread
/// (reader, for `receive()`).
struct CurrentSelection {
offer: ExtDataControlOfferV1,
/// Raw Wayland MIMEs the offer advertises (what `receive()` accepts).
mimes: Vec<String>,
}
/// Dispatch-thread state. Also collects the manager + seat during the bind roundtrip.
struct State {
mgr: Option<ExtDataControlManagerV1>,
seat: Option<WlSeat>,
/// Offers accumulating their MIME list before the `selection` event promotes one.
pending: HashMap<ObjectId, Vec<String>>,
current: Arc<Mutex<Option<CurrentSelection>>>,
/// Pending count of our own `set_selection`s whose `selection` echo must be dropped rather than
/// announced back to the client (loop prevention, §3.4). Bumped by the session before each set;
/// each of our sets produces exactly one echo on wlroots/KWin, so one decrement per echo pairs
/// them up — a counter (not a bool) keeps rapid back-to-back offers from leaking a self-echo.
suppress_echoes: Arc<AtomicU32>,
tx: tokio::sync::mpsc::UnboundedSender<ClipEvent>,
}
impl Dispatch<wl_registry::WlRegistry, ()> for State {
fn event(
state: &mut Self,
registry: &wl_registry::WlRegistry,
event: wl_registry::Event,
_: &(),
_: &Connection,
qh: &QueueHandle<Self>,
) {
if let wl_registry::Event::Global {
name,
interface,
version,
} = event
{
match interface.as_str() {
"ext_data_control_manager_v1" => {
state.mgr = Some(registry.bind(name, version.min(1), qh, ()));
}
"wl_seat" => {
state.seat = Some(registry.bind(name, version.min(7), qh, ()));
}
_ => {}
}
}
}
}
// Manager + seat emit nothing we consume.
impl Dispatch<ExtDataControlManagerV1, ()> for State {
fn event(
_: &mut Self,
_: &ExtDataControlManagerV1,
_: <ExtDataControlManagerV1 as Proxy>::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
impl Dispatch<WlSeat, ()> for State {
fn event(
_: &mut Self,
_: &WlSeat,
_: <WlSeat as Proxy>::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
impl Dispatch<ExtDataControlDeviceV1, ()> for State {
fn event(
state: &mut Self,
_dev: &ExtDataControlDeviceV1,
event: ext_data_control_device_v1::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
use ext_data_control_device_v1::Event;
match event {
// A new offer is being introduced; its `offer` events follow before `selection`.
Event::DataOffer { id } => {
state.pending.insert(id.id(), Vec::new());
}
// The active selection changed. `Some` = a new clipboard; `None` = cleared.
Event::Selection { id } => {
// Consume one pending self-echo if any (atomic vs. the session thread's bumps; the
// dispatch thread is the only decrementer). `Ok` = there was one → suppress.
let suppressed = state
.suppress_echoes
.fetch_update(Ordering::SeqCst, Ordering::SeqCst, |c| c.checked_sub(1))
.is_ok();
match id {
Some(offer) => {
let mimes = state.pending.remove(&offer.id()).unwrap_or_default();
if suppressed {
// Our own source's echo — don't store it as the host clipboard and
// don't announce it back to the client.
return;
}
let wire = super::offer_wire_mimes(&mimes)
.into_iter()
.map(str::to_string)
.collect::<Vec<_>>();
*state.current.lock().unwrap() = Some(CurrentSelection { offer, mimes });
let _ = state.tx.send(ClipEvent::Selection { mimes: wire });
}
None => {
*state.current.lock().unwrap() = None;
if !suppressed {
let _ = state.tx.send(ClipEvent::Selection { mimes: Vec::new() });
}
}
}
}
Event::Finished => {
let _ = state.tx.send(ClipEvent::Closed);
}
// Primary selection is out of scope for the shared clipboard.
_ => {}
}
}
event_created_child!(State, ExtDataControlDeviceV1, [
ext_data_control_device_v1::EVT_DATA_OFFER_OPCODE => (ExtDataControlOfferV1, ()),
]);
}
impl Dispatch<ExtDataControlOfferV1, ()> for State {
fn event(
state: &mut Self,
offer: &ExtDataControlOfferV1,
event: ext_data_control_offer_v1::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
if let ext_data_control_offer_v1::Event::Offer { mime_type } = event {
if let Some(list) = state.pending.get_mut(&offer.id()) {
list.push(mime_type);
}
}
}
}
impl Dispatch<ExtDataControlSourceV1, ()> for State {
fn event(
state: &mut Self,
_src: &ExtDataControlSourceV1,
event: ext_data_control_source_v1::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
use ext_data_control_source_v1::Event;
match event {
// A host app pasted our (the client's) offered data.
Event::Send { mime_type, fd } => match super::wayland_to_wire(&mime_type) {
Some(wire) => {
let _ = state.tx.send(ClipEvent::Paste {
mime: wire.to_string(),
responder: PasteResponder::Fd(fd),
});
}
// We can't satisfy this format — closing the fd yields an empty paste.
None => drop(fd),
},
// Our source was superseded (a host app or another client set a new selection).
Event::Cancelled => {}
_ => {}
}
}
}
/// The host clipboard backend handle used by the session thread.
pub struct ClipboardBackend {
conn: Connection,
mgr: ExtDataControlManagerV1,
device: ExtDataControlDeviceV1,
qh: QueueHandle<State>,
current: Arc<Mutex<Option<CurrentSelection>>>,
suppress_echoes: Arc<AtomicU32>,
active_source: Mutex<Option<ExtDataControlSourceV1>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl ClipboardBackend {
/// Connect to the active session's Wayland display (env already applied by
/// `vdisplay::apply_session_env`), bind `ext_data_control`, and start the dispatch thread.
/// Returns the handle plus the event stream. Errors if the compositor lacks the protocol
/// (caller reports `BackendUnavailable`).
pub fn open() -> Result<(
ClipboardBackend,
tokio::sync::mpsc::UnboundedReceiver<ClipEvent>,
)> {
let conn = Connection::connect_to_env()
.context("connect to Wayland for clipboard (WAYLAND_DISPLAY/XDG_RUNTIME_DIR set?)")?;
let mut queue = conn.new_event_queue();
let qh = queue.handle();
let _registry = conn.display().get_registry(&qh, ());
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
let current = Arc::new(Mutex::new(None));
let suppress_echoes = Arc::new(AtomicU32::new(0));
let mut state = State {
mgr: None,
seat: None,
pending: HashMap::new(),
current: current.clone(),
suppress_echoes: suppress_echoes.clone(),
tx,
};
queue
.roundtrip(&mut state)
.context("Wayland registry roundtrip")?;
let mgr = state
.mgr
.clone()
.context("compositor lacks ext_data_control_manager_v1")?;
let seat = state
.seat
.clone()
.context("compositor advertised no wl_seat")?;
let device = mgr.get_data_device(&seat, &qh, ());
// Second roundtrip: the compositor sends the initial selection for the freshly-bound device
// (the current host clipboard), which the session announces to the client.
queue
.roundtrip(&mut state)
.context("Wayland get_data_device roundtrip")?;
let stop = Arc::new(AtomicBool::new(false));
let thread = {
let conn = conn.clone();
let stop = stop.clone();
std::thread::Builder::new()
.name("punktfunk-clipboard".into())
.spawn(move || dispatch_loop(conn, queue, state, stop))
.context("spawn clipboard dispatch thread")?
};
Ok((
ClipboardBackend {
conn,
mgr,
device,
qh,
current,
suppress_echoes,
active_source: Mutex::new(None),
stop,
thread: Some(thread),
},
rx,
))
}
/// Install a lazy source advertising a client's offered formats (wire MIMEs) as the host
/// selection. A later host-app paste fires a [`ClipEvent::Paste`]. Replaces any previous offer.
pub fn set_offer(&self, wire_mimes: &[String]) -> Result<()> {
let wl_mimes = super::wayland_offers_for(wire_mimes);
if wl_mimes.is_empty() {
return self.clear_offer();
}
let src = self.mgr.create_data_source(&self.qh, ());
for m in &wl_mimes {
src.offer(m.clone());
}
// Suppress the selection echo our own set triggers (loop prevention).
self.suppress_echoes.fetch_add(1, Ordering::SeqCst);
self.device.set_selection(Some(&src));
self.conn.flush().context("flush set_selection")?;
let mut slot = self.active_source.lock().unwrap();
if let Some(old) = slot.take() {
old.destroy();
}
*slot = Some(src);
Ok(())
}
/// Drop the host selection we own (client disabled sync / offered nothing).
pub fn clear_offer(&self) -> Result<()> {
let mut slot = self.active_source.lock().unwrap();
if let Some(old) = slot.take() {
self.suppress_echoes.fetch_add(1, Ordering::SeqCst);
self.device.set_selection(None);
old.destroy();
self.conn.flush().context("flush clear selection")?;
}
Ok(())
}
/// The current host selection's wire MIMEs (what a client offer announcement would carry), or
/// empty if the clipboard is empty. Used to answer an immediate query.
pub fn current_wire_mimes(&self) -> Vec<String> {
match self.current.lock().unwrap().as_ref() {
Some(sel) => super::offer_wire_mimes(&sel.mimes)
.into_iter()
.map(str::to_string)
.collect(),
None => Vec::new(),
}
}
/// Read one format (`wire_mime`) of the current host selection into a byte vector — a client's
/// lazy fetch. BLOCKS on the pipe until the source app finishes, so call from a blocking
/// context (e.g. `spawn_blocking`). Errors if there is no selection or the format isn't offered.
pub fn read_current(&self, wire_mime: &str) -> Result<Vec<u8>> {
let (offer, wl_mime) = {
let cur = self.current.lock().unwrap();
let sel = cur.as_ref().context("no current host selection")?;
let wl = super::pick_wayland_mime(wire_mime, &sel.mimes)
.context("format not offered by the host clipboard")?;
(sel.offer.clone(), wl)
};
let (read_fd, write_fd) = make_pipe()?;
offer.receive(wl_mime, write_fd.as_fd());
self.conn.flush().context("flush receive")?;
// Close our write end so the pipe reaches EOF once the source app closes its dup.
drop(write_fd);
let mut buf = Vec::new();
// `read_fd` is a fresh, uniquely-owned pipe read end; `File` takes sole ownership and closes
// it on drop.
let file = std::fs::File::from(read_fd);
file.take(CLIP_READ_CAP)
.read_to_end(&mut buf)
.context("read clipboard transfer")?;
Ok(buf)
}
}
impl Drop for ClipboardBackend {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
/// The dispatch thread: poll the Wayland socket with a short timeout so `stop` is honored promptly,
/// dispatching selection/paste events into `state`.
fn dispatch_loop(
conn: Connection,
mut queue: wayland_client::EventQueue<State>,
mut state: State,
stop: Arc<AtomicBool>,
) {
while !stop.load(Ordering::SeqCst) {
if queue.dispatch_pending(&mut state).is_err() {
break;
}
if conn.flush().is_err() {
break;
}
let Some(guard) = conn.prepare_read() else {
// Events are already queued; loop to dispatch them.
continue;
};
let raw_fd = guard.connection_fd().as_raw_fd();
let mut pfd = libc::pollfd {
fd: raw_fd,
events: libc::POLLIN,
revents: 0,
};
// SAFETY: `pfd` is a single valid pollfd; `poll` reads/writes exactly it for 200 ms.
let rc = unsafe { libc::poll(&mut pfd, 1, 200) };
if rc < 0 {
let err = std::io::Error::last_os_error();
drop(guard);
if err.kind() == std::io::ErrorKind::Interrupted {
continue; // EINTR — recheck stop, retry
}
break;
}
if rc == 0 {
drop(guard); // timeout — recheck stop
continue;
}
if pfd.revents & libc::POLLIN != 0 {
if guard.read().is_err() {
break;
}
} else {
drop(guard); // POLLHUP / POLLERR — connection gone
break;
}
}
let _ = state.tx.send(ClipEvent::Closed);
}
/// Create a `pipe2(O_CLOEXEC)`, returning `(read_end, write_end)` as owned fds.
fn make_pipe() -> Result<(OwnedFd, OwnedFd)> {
let mut fds = [0 as libc::c_int; 2];
// SAFETY: `pipe2` fully initializes the 2-element `fds` on success (returns 0); on failure (-1)
// we bail before reading it. Each returned fd is fresh and owned by exactly one `OwnedFd`.
let rc = unsafe { libc::pipe2(fds.as_mut_ptr(), libc::O_CLOEXEC) };
if rc < 0 {
return Err(anyhow!("pipe2 failed: {}", std::io::Error::last_os_error()));
}
// SAFETY: `fds[0]`/`fds[1]` are the fresh, uniquely-owned pipe ends from the checked `pipe2`.
let read_fd = unsafe { OwnedFd::from_raw_fd(fds[0]) };
// SAFETY: as above for the write end.
let write_fd = unsafe { OwnedFd::from_raw_fd(fds[1]) };
Ok((read_fd, write_fd))
}
/// On-glass tests against a **live** `data-control` compositor (Hyprland / Sway / KWin). `#[ignore]`d
/// — run explicitly under such a session with `wl-clipboard` present:
///
/// ```text
/// WAYLAND_DISPLAY=wayland-1 cargo test -p punktfunk-host --bin punktfunk-host \
/// -- --ignored --nocapture clipboard::wayland::live
/// ```
///
/// Each test skips (does not fail) when `open()` finds no backend — so `--ignored` on GNOME (no
/// data-control) or a headless CI runner is a clean no-op instead of a false failure.
#[cfg(test)]
mod live {
use super::*;
use std::io::Write as _;
use std::process::{Command, Stdio};
use std::time::{Duration, Instant};
/// Poll the event channel (sync `try_recv`, no runtime) until `pred` matches or `timeout`.
fn wait_event(
rx: &mut tokio::sync::mpsc::UnboundedReceiver<ClipEvent>,
timeout: Duration,
mut pred: impl FnMut(&ClipEvent) -> bool,
) -> Option<ClipEvent> {
let deadline = Instant::now() + timeout;
loop {
match rx.try_recv() {
Ok(ev) if pred(&ev) => return Some(ev),
Ok(_) => {}
Err(tokio::sync::mpsc::error::TryRecvError::Empty) => {
if Instant::now() >= deadline {
return None;
}
std::thread::sleep(Duration::from_millis(20));
}
Err(tokio::sync::mpsc::error::TryRecvError::Disconnected) => return None,
}
}
}
/// Set the compositor selection from a "host app" (`wl-copy`, which forks a server that holds it).
fn wl_copy(bytes: &[u8], mime: &str) {
let mut child = Command::new("wl-copy")
.arg("--type")
.arg(mime)
.stdin(Stdio::piped())
.spawn()
.expect("spawn wl-copy");
child
.stdin
.take()
.unwrap()
.write_all(bytes)
.expect("write to wl-copy");
let _ = child.wait(); // foreground exits; the fork keeps serving
std::thread::sleep(Duration::from_millis(150));
}
fn open_or_skip() -> Option<(
ClipboardBackend,
tokio::sync::mpsc::UnboundedReceiver<ClipEvent>,
)> {
if Command::new("wl-copy").arg("--version").output().is_err() {
eprintln!("SKIP: wl-clipboard not installed");
return None;
}
match ClipboardBackend::open() {
Ok(v) => Some(v),
Err(e) => {
eprintln!("SKIP: no data-control backend on this compositor: {e:#}");
None
}
}
}
/// Host copy → we observe a `Selection` and can `read_current` the exact bytes back — both text
/// and PNG (§3.5 format normalization end to end).
#[test]
#[ignore = "needs a live data-control compositor (WAYLAND_DISPLAY)"]
fn live_host_copy_is_readable() {
let Some((backend, mut rx)) = open_or_skip() else {
return;
};
// Text.
wl_copy(b"hello-from-host-app", "text/plain;charset=utf-8");
let ev = wait_event(&mut rx, Duration::from_secs(3), |e| {
matches!(e, ClipEvent::Selection { mimes } if mimes.iter().any(|m| m == super::super::WIRE_TEXT))
})
.expect("Selection event carrying text after wl-copy");
assert!(matches!(ev, ClipEvent::Selection { .. }));
assert_eq!(
backend.read_current(super::super::WIRE_TEXT).unwrap(),
b"hello-from-host-app"
);
// PNG (arbitrary bytes tagged image/png — data-control is format-agnostic).
let png = b"\x89PNG\r\n\x1a\n-fake-but-tagged-image/png";
wl_copy(png, "image/png");
wait_event(&mut rx, Duration::from_secs(3), |e| {
matches!(e, ClipEvent::Selection { mimes } if mimes.iter().any(|m| m == super::super::WIRE_PNG))
})
.expect("Selection event carrying image/png");
assert_eq!(backend.read_current(super::super::WIRE_PNG).unwrap(), png);
}
/// We install a client's offer as the host selection; a host app (`wl-paste`) pasting it fires a
/// `Paste` event that we fulfill with bytes, and the host app receives exactly those bytes.
#[test]
#[ignore = "needs a live data-control compositor (WAYLAND_DISPLAY)"]
fn live_set_offer_is_pasteable() {
let Some((backend, mut rx)) = open_or_skip() else {
return;
};
backend
.set_offer(&[super::super::WIRE_TEXT.to_string()])
.expect("install offer");
// A host app pastes our offered selection.
let child = Command::new("wl-paste")
.arg("-n")
.stdout(Stdio::piped())
.spawn()
.expect("spawn wl-paste");
let paste = wait_event(&mut rx, Duration::from_secs(3), |e| {
matches!(e, ClipEvent::Paste { .. })
})
.expect("Paste event after wl-paste reads our offer");
match paste {
ClipEvent::Paste { mime, responder } => {
assert_eq!(
mime,
super::super::WIRE_TEXT,
"paste requested the text format"
);
match responder {
PasteResponder::Fd(fd) => {
super::super::fulfill_paste(fd, b"served-by-punktfunk").expect("fulfill");
}
PasteResponder::Channel(_) => panic!("data-control paste must carry an fd"),
}
}
_ => unreachable!(),
}
let out = child.wait_with_output().expect("wl-paste output");
assert_eq!(out.stdout, b"served-by-punktfunk");
}
}
+664
View File
@@ -0,0 +1,664 @@
//! Host-side shared-clipboard backend for Windows (`design/clipboard-and-file-transfer.md` §4, Phase
//! 3). The Win32 clipboard is thread-affine and message-driven, so the whole backend lives on one
//! dedicated **message-loop thread** owning a hidden message-only window:
//!
//! * **host copy → client** — `AddClipboardFormatListener` delivers `WM_CLIPBOARDUPDATE`; we map the
//! available formats to wire MIMEs, cache them, and emit [`ClipEvent::Selection`]. Our own offers
//! are suppressed by the owner-check (we never forward what we ourselves put on the clipboard).
//! * **client fetch of the host clipboard** — a `Cmd::Read` reads the requested format's HGLOBAL and
//! converts it to wire bytes ([`super::winfmt`]).
//! * **client copy → host** — a `Cmd::SetOffer` installs the client's formats via OLE **delayed
//! rendering**: `SetClipboardData(fmt, NULL)`, so no bytes cross until a host app actually pastes.
//! * **host paste of client content** — the paste triggers `WM_RENDERFORMAT`; the message-loop thread
//! blocks (bounded) while the coordinator fetches the bytes from the client, then `SetClipboardData`s
//! them for the pasting app.
//!
//! The async coordinator drives the thread through a [`Cmd`] channel woken by `PostMessage(WM_APP_CMD)`
//! (`PostMessage` is the documented thread-safe way to poke a message loop). Per-window state hangs
//! off `GWLP_USERDATA`, so multiple concurrent sessions each get their own window + state.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use std::cell::RefCell;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use anyhow::Context as _;
use ::windows::core::{w, PCWSTR};
use ::windows::Win32::Foundation::{
GetLastError, GlobalFree, HANDLE, HGLOBAL, HINSTANCE, HWND, LPARAM, LRESULT, WPARAM,
};
use ::windows::Win32::System::DataExchange::{
AddClipboardFormatListener, CloseClipboard, EmptyClipboard, GetClipboardData,
GetClipboardOwner, IsClipboardFormatAvailable, OpenClipboard, RegisterClipboardFormatW,
SetClipboardData,
};
use ::windows::Win32::System::LibraryLoader::GetModuleHandleW;
use ::windows::Win32::System::Memory::{
GlobalAlloc, GlobalLock, GlobalSize, GlobalUnlock, GMEM_MOVEABLE, GMEM_ZEROINIT,
};
use ::windows::Win32::System::Ole::CF_UNICODETEXT;
use ::windows::Win32::UI::WindowsAndMessaging::{
CreateWindowExW, DefWindowProcW, DestroyWindow, DispatchMessageW, GetMessageW,
GetWindowLongPtrW, PostMessageW, PostQuitMessage, RegisterClassW, SetWindowLongPtrW,
TranslateMessage, GWLP_USERDATA, HWND_MESSAGE, MSG, WINDOW_EX_STYLE, WINDOW_STYLE, WM_APP,
WM_CLIPBOARDUPDATE, WM_DESTROY, WM_RENDERFORMAT, WNDCLASSW,
};
use super::winfmt;
use super::{ClipEvent, PasteResponder, WIRE_HTML, WIRE_PNG, WIRE_RTF, WIRE_TEXT};
/// Custom app message that wakes the pump to drain the [`Cmd`] channel.
const WM_APP_CMD: u32 = WM_APP + 1;
/// Upper bound the message-loop thread waits for the client's bytes during a `WM_RENDERFORMAT` paste.
/// The pasting app is frozen until we answer, so this caps how long a paste can hang; on expiry the
/// format is left unrendered (an empty paste) rather than blocking indefinitely.
const RENDER_TIMEOUT: Duration = Duration::from_secs(10);
/// `OpenClipboard` fails while another process transiently holds the clipboard (clipboard managers do
/// this constantly); retry briefly before giving up.
const OPEN_RETRIES: u32 = 20;
const OPEN_RETRY_DELAY: Duration = Duration::from_millis(5);
/// `RegisterClassW` returns this when the (process-global) class already exists — expected on the 2nd+
/// concurrent session, and not an error (we never unregister the class).
const ERROR_CLASS_ALREADY_EXISTS: u32 = 1410;
type ClipTx = tokio::sync::mpsc::UnboundedSender<ClipEvent>;
/// A command from the async coordinator into the message-loop thread. Delivered over a tokio channel
/// and drained on `WM_APP_CMD`.
enum Cmd {
/// Install the client's wire MIMEs as a delayed-render host selection (empty ⇒ clear).
SetOffer(Vec<String>),
/// Drop the selection we own.
Clear,
/// Read one wire format of the current host selection for a client fetch.
Read {
wire: String,
resp: tokio::sync::oneshot::Sender<anyhow::Result<Vec<u8>>>,
},
/// Tear the window + thread down.
Shutdown,
}
/// Message-loop-thread-owned state, reached from the `WndProc` via `GWLP_USERDATA`. Only the message
/// thread ever dereferences it, so the `RefCell`s are sound (no cross-thread sharing); the fields the
/// async handle also touches (`current_wire`) are behind their own `Arc<Mutex>`.
struct WinClip {
/// Backend → coordinator events.
clip_tx: ClipTx,
/// The current host selection's wire MIMEs, shared with the [`WindowsClipboard`] handle.
current_wire: Arc<Mutex<Vec<String>>>,
/// Coordinator → backend commands, drained on `WM_APP_CMD`.
cmd_rx: RefCell<tokio::sync::mpsc::UnboundedReceiver<Cmd>>,
/// Clipboard format ids we currently promise via delayed rendering (for `WM_RENDERFORMAT`).
offered: RefCell<Vec<u32>>,
fmt_html: u32,
fmt_rtf: u32,
fmt_png: u32,
/// Our own message window — used for the owner-check and clipboard opens.
own_hwnd: HWND,
}
impl WinClip {
/// `WM_CLIPBOARDUPDATE`: a host app copied (or the clipboard was cleared). Suppress our own
/// delayed-render echoes via the owner-check, else announce the new wire MIMEs.
fn on_clipboard_update(&self, hwnd: HWND) {
// SAFETY: GetClipboardOwner has no preconditions and needs no open clipboard.
let owner = unsafe { GetClipboardOwner() }.unwrap_or_default();
if owner.0 == hwnd.0 {
// Our own offer's echo (we own the clipboard) — not a host copy.
return;
}
let mimes = self.available_wire_mimes();
*self.current_wire.lock().unwrap() = mimes.clone();
let _ = self.clip_tx.send(ClipEvent::Selection { mimes });
}
/// The wire MIMEs the current clipboard advertises, in a stable order.
fn available_wire_mimes(&self) -> Vec<String> {
// SAFETY: IsClipboardFormatAvailable has no preconditions and needs no open clipboard.
let avail = |fmt: u32| unsafe { IsClipboardFormatAvailable(fmt) }.is_ok();
let mut out = Vec::new();
if avail(CF_UNICODETEXT.0 as u32) {
out.push(WIRE_TEXT.to_string());
}
if avail(self.fmt_html) {
out.push(WIRE_HTML.to_string());
}
if avail(self.fmt_rtf) {
out.push(WIRE_RTF.to_string());
}
if avail(self.fmt_png) {
out.push(WIRE_PNG.to_string());
}
out
}
/// `WM_APP_CMD`: run every queued coordinator command on this thread. Drained into a `Vec` first so
/// the `cmd_rx` borrow is released before any command runs (defensive against re-entry).
fn drain_commands(&self, hwnd: HWND) {
let mut cmds = Vec::new();
{
let mut rx = self.cmd_rx.borrow_mut();
while let Ok(c) = rx.try_recv() {
cmds.push(c);
}
}
for c in cmds {
match c {
Cmd::SetOffer(wire) => self.apply_offer(hwnd, &wire),
Cmd::Clear => self.clear(hwnd),
Cmd::Read { wire, resp } => {
let _ = resp.send(self.read(&wire));
}
Cmd::Shutdown => {
// Drop our offer first so no WM_RENDERALLFORMATS fires as the window dies (we do
// NOT want the client's content to outlive the session on the host clipboard).
self.clear(hwnd);
// SAFETY: our own live window; triggers WM_DESTROY → PostQuitMessage → pump exit.
unsafe {
let _ = DestroyWindow(hwnd);
}
return;
}
}
}
}
/// Install the client's offer as a delayed-render host selection.
fn apply_offer(&self, hwnd: HWND, wire: &[String]) {
let fmts = self.formats_for_offer(wire);
if fmts.is_empty() {
self.clear(hwnd);
return;
}
if open_clipboard_retry(hwnd).is_err() {
tracing::debug!("clipboard: OpenClipboard for set_offer failed");
return;
}
let _guard = ClipboardGuard;
// SAFETY: the clipboard is open (ClipboardGuard closes it); EmptyClipboard makes us the owner,
// then each SetClipboardData(_, None) registers a delayed-render promise for that format.
unsafe {
let _ = EmptyClipboard();
for &f in &fmts {
let _ = SetClipboardData(f, None);
}
}
*self.offered.borrow_mut() = fmts;
}
/// Drop the selection we own (empty the clipboard iff we're still its owner).
fn clear(&self, hwnd: HWND) {
let had = {
let mut o = self.offered.borrow_mut();
let was = !o.is_empty();
o.clear();
was
};
if !had {
return;
}
// SAFETY: GetClipboardOwner has no preconditions.
let owner = unsafe { GetClipboardOwner() }.unwrap_or_default();
if owner.0 != hwnd.0 {
return; // someone else took the clipboard already
}
if open_clipboard_retry(hwnd).is_err() {
return;
}
let _guard = ClipboardGuard;
// SAFETY: the clipboard is open (ClipboardGuard closes it); empty it to drop our promises.
unsafe {
let _ = EmptyClipboard();
}
}
/// Read one wire format of the current host selection (a client fetch).
fn read(&self, wire: &str) -> anyhow::Result<Vec<u8>> {
let fmt = self
.format_for_wire(wire)
.context("unsupported wire MIME")?;
// If we own the clipboard, its content is our own delayed-render offer (the client's copy),
// not a host selection — declining avoids GetClipboardData re-entering our own WM_RENDERFORMAT.
// SAFETY: GetClipboardOwner has no preconditions.
if unsafe { GetClipboardOwner() }.unwrap_or_default().0 == self.own_hwnd.0 {
anyhow::bail!("clipboard currently held by our own offer");
}
open_clipboard_retry(self.own_hwnd)?;
let _guard = ClipboardGuard;
// SAFETY: the clipboard is open (ClipboardGuard closes it). GetClipboardData hands back a
// clipboard-owned HGLOBAL (we must NOT free it); GlobalLock/Size/Unlock are balanced and we
// copy exactly GlobalSize bytes out before the lock is released.
let raw = unsafe {
let handle = GetClipboardData(fmt).context("GetClipboardData")?;
let hg = HGLOBAL(handle.0);
let p = GlobalLock(hg);
if p.is_null() {
anyhow::bail!("GlobalLock failed");
}
let n = GlobalSize(hg);
let mut buf = vec![0u8; n];
std::ptr::copy_nonoverlapping(p as *const u8, buf.as_mut_ptr(), n);
let _ = GlobalUnlock(hg);
buf
};
Ok(convert_from_win(wire, &raw))
}
/// `WM_RENDERFORMAT`: a host app is pasting a format we promised. Fetch the bytes from the client
/// (blocking this thread, bounded) and `SetClipboardData` them for the paster.
fn on_render_format(&self, fmt: u32) {
let Some(wire) = self.wire_for_format(fmt) else {
return;
};
let (tx, rx) = std::sync::mpsc::channel::<Vec<u8>>();
let ev = ClipEvent::Paste {
mime: wire.to_string(),
responder: PasteResponder::Sync(tx),
};
if self.clip_tx.send(ev).is_err() {
return; // coordinator gone
}
let bytes = match rx.recv_timeout(RENDER_TIMEOUT) {
Ok(b) => b,
Err(_) => return, // timeout / dropped → leave the format unrendered (empty paste)
};
let win_bytes = convert_to_win(wire, &bytes);
let Ok(hg) = alloc_hglobal(&win_bytes) else {
return;
};
// Do NOT OpenClipboard here — the pasting app already holds it open across WM_RENDERFORMAT.
// SAFETY: `hg` is a freshly-filled moveable HGLOBAL. On success the clipboard takes ownership
// (we must not free it); on failure ownership stays with us, so we free it.
unsafe {
if SetClipboardData(fmt, Some(HANDLE(hg.0))).is_err() {
let _ = GlobalFree(Some(hg));
}
}
}
/// The Win32 clipboard format id for a wire MIME (`None` = unsupported).
fn format_for_wire(&self, wire: &str) -> Option<u32> {
match wire {
WIRE_TEXT => Some(CF_UNICODETEXT.0 as u32),
WIRE_HTML => Some(self.fmt_html),
WIRE_RTF => Some(self.fmt_rtf),
WIRE_PNG => Some(self.fmt_png),
_ => None,
}
}
/// The wire MIME for a Win32 clipboard format id (`None` = one we don't offer).
fn wire_for_format(&self, fmt: u32) -> Option<&'static str> {
if fmt == CF_UNICODETEXT.0 as u32 {
Some(WIRE_TEXT)
} else if fmt == self.fmt_html {
Some(WIRE_HTML)
} else if fmt == self.fmt_rtf {
Some(WIRE_RTF)
} else if fmt == self.fmt_png {
Some(WIRE_PNG)
} else {
None
}
}
/// The clipboard format ids to promise for a client offer (dedup, 1:1 with the wire MIMEs — the OS
/// auto-synthesizes CF_TEXT/CF_OEMTEXT from CF_UNICODETEXT, so no manual text fan-out is needed).
fn formats_for_offer(&self, wire: &[String]) -> Vec<u32> {
let mut out = Vec::new();
for w in wire {
if let Some(f) = self.format_for_wire(w) {
if !out.contains(&f) {
out.push(f);
}
}
}
out
}
}
/// The active Windows clipboard backend handle held by [`super::HostClipboard`]. All Win32 work runs
/// on the message-loop thread; this is just the async-side control surface.
pub struct WindowsClipboard {
cmd_tx: tokio::sync::mpsc::UnboundedSender<Cmd>,
/// The message window's `HWND` as an `isize` (so the handle stays `Send`/`Sync`); rebuilt for the
/// `PostMessage` wakeups, which are documented thread-safe.
hwnd: isize,
current_wire: Arc<Mutex<Vec<String>>>,
join: Option<std::thread::JoinHandle<()>>,
}
impl WindowsClipboard {
/// Spin up the message-loop thread + hidden window and return once it has bound (or failed).
pub async fn open() -> anyhow::Result<(
WindowsClipboard,
tokio::sync::mpsc::UnboundedReceiver<ClipEvent>,
)> {
let (clip_tx, clip_rx) = tokio::sync::mpsc::unbounded_channel::<ClipEvent>();
let (cmd_tx, cmd_rx) = tokio::sync::mpsc::unbounded_channel::<Cmd>();
let current_wire = Arc::new(Mutex::new(Vec::new()));
// Register the three custom formats up front — process-global and thread-agnostic, so this is
// fine off the message thread and lets bring-up fail cleanly if the atoms can't be created.
let fmt_html = register_format(w!("HTML Format"))?;
let fmt_rtf = register_format(w!("Rich Text Format"))?;
let fmt_png = register_format(w!("PNG"))?;
let (ready_tx, ready_rx) = tokio::sync::oneshot::channel::<anyhow::Result<isize>>();
let cw = Arc::clone(&current_wire);
let join = std::thread::Builder::new()
.name("punktfunk-clipboard-win".into())
.spawn(move || pump_thread(clip_tx, cmd_rx, cw, fmt_html, fmt_rtf, fmt_png, ready_tx))
.context("spawn windows clipboard thread")?;
let hwnd = match tokio::time::timeout(Duration::from_secs(3), ready_rx).await {
Ok(Ok(Ok(h))) => h,
Ok(Ok(Err(e))) => return Err(e),
Ok(Err(_)) => anyhow::bail!("windows clipboard thread exited during bring-up"),
Err(_) => anyhow::bail!("windows clipboard bring-up timed out"),
};
Ok((
WindowsClipboard {
cmd_tx,
hwnd,
current_wire,
join: Some(join),
},
clip_rx,
))
}
/// The current host selection's wire MIMEs (empty = nothing to offer).
pub fn current_wire_mimes(&self) -> Vec<String> {
self.current_wire.lock().unwrap().clone()
}
/// Install a client's offered formats as the host selection (fire-and-forget onto the thread).
pub fn set_offer(&self, wire_mimes: &[String]) {
let _ = self.cmd_tx.send(Cmd::SetOffer(wire_mimes.to_vec()));
self.wake();
}
/// Drop the host selection we own (fire-and-forget onto the thread).
pub fn clear_offer(&self) {
let _ = self.cmd_tx.send(Cmd::Clear);
self.wake();
}
/// Read one wire format of the current host selection (a client's fetch).
pub async fn read_current(&self, wire_mime: &str) -> anyhow::Result<Vec<u8>> {
let (tx, rx) = tokio::sync::oneshot::channel();
self.cmd_tx
.send(Cmd::Read {
wire: wire_mime.to_string(),
resp: tx,
})
.map_err(|_| anyhow::anyhow!("clipboard thread gone"))?;
self.wake();
rx.await
.map_err(|_| anyhow::anyhow!("clipboard read dropped"))?
}
/// Poke the message loop so it drains the command channel.
fn wake(&self) {
// SAFETY: PostMessageW is documented thread-safe; `hwnd` is our message window (or already
// destroyed, in which case the post harmlessly fails and is ignored).
let _ = unsafe {
PostMessageW(
Some(HWND(self.hwnd as *mut core::ffi::c_void)),
WM_APP_CMD,
WPARAM(0),
LPARAM(0),
)
};
}
}
impl Drop for WindowsClipboard {
fn drop(&mut self) {
let _ = self.cmd_tx.send(Cmd::Shutdown);
self.wake();
if let Some(j) = self.join.take() {
let _ = j.join();
}
}
}
/// RAII `CloseClipboard` guard — pairs with a successful `open_clipboard_retry`, closing on scope exit
/// (including early `?`/`bail!` returns).
struct ClipboardGuard;
impl Drop for ClipboardGuard {
fn drop(&mut self) {
// SAFETY: constructed only after a successful OpenClipboard on this thread.
unsafe {
let _ = CloseClipboard();
}
}
}
/// Register (or resolve the existing id of) a custom clipboard format.
fn register_format(name: PCWSTR) -> anyhow::Result<u32> {
// SAFETY: RegisterClipboardFormatW is thread-agnostic and process-global; `name` is a static
// NUL-terminated wide literal.
let id = unsafe { RegisterClipboardFormatW(name) };
if id == 0 {
anyhow::bail!("RegisterClipboardFormatW failed");
}
Ok(id)
}
/// Allocate a moveable HGLOBAL holding `bytes` (zero-init so an empty payload is still a valid, locked
/// buffer). Ownership is transferred to the clipboard by a following `SetClipboardData`.
fn alloc_hglobal(bytes: &[u8]) -> anyhow::Result<HGLOBAL> {
// SAFETY: allocate at least one byte (GlobalLock of a 0-size block is unreliable), lock it, copy
// the payload in, unlock. Alloc/lock/unlock are balanced; on lock failure we free before erroring.
unsafe {
let hg = GlobalAlloc(GMEM_MOVEABLE | GMEM_ZEROINIT, bytes.len().max(1))
.context("GlobalAlloc")?;
let p = GlobalLock(hg);
if p.is_null() {
let _ = GlobalFree(Some(hg));
anyhow::bail!("GlobalLock failed");
}
std::ptr::copy_nonoverlapping(bytes.as_ptr(), p as *mut u8, bytes.len());
let _ = GlobalUnlock(hg);
Ok(hg)
}
}
/// `OpenClipboard(hwnd)` with a brief retry loop (another process often holds it transiently).
fn open_clipboard_retry(hwnd: HWND) -> anyhow::Result<()> {
for _ in 0..OPEN_RETRIES {
// SAFETY: OpenClipboard with our window as owner; balanced by ClipboardGuard/CloseClipboard.
if unsafe { OpenClipboard(Some(hwnd)) }.is_ok() {
return Ok(());
}
std::thread::sleep(OPEN_RETRY_DELAY);
}
anyhow::bail!("OpenClipboard failed after retries")
}
/// Convert a Win32 clipboard payload to wire bytes.
fn convert_from_win(wire: &str, raw: &[u8]) -> Vec<u8> {
match wire {
WIRE_TEXT => winfmt::text_from_utf16(raw),
WIRE_HTML => winfmt::html_from_cf(raw),
WIRE_RTF => winfmt::rtf_from_cf(raw),
_ => raw.to_vec(), // PNG + anything else: verbatim
}
}
/// Convert wire bytes to a Win32 clipboard payload.
fn convert_to_win(wire: &str, wire_bytes: &[u8]) -> Vec<u8> {
match wire {
WIRE_TEXT => winfmt::text_to_utf16(wire_bytes),
WIRE_HTML => winfmt::html_to_cf(wire_bytes),
_ => wire_bytes.to_vec(), // RTF + PNG + anything else: verbatim
}
}
/// Create the hidden message-only window (registering the class once, process-wide).
fn create_window() -> anyhow::Result<HWND> {
// SAFETY: standard window-class registration + message-only window creation; every argument is a
// valid handle / static literal, and `wndproc` matches the WNDPROC ABI.
unsafe {
let hinstance: HINSTANCE = GetModuleHandleW(PCWSTR::null())
.context("GetModuleHandleW")?
.into();
let class_name = w!("PunktfunkClipboardWindow");
let wc = WNDCLASSW {
lpfnWndProc: Some(wndproc),
hInstance: hinstance,
lpszClassName: class_name,
..Default::default()
};
if RegisterClassW(&wc) == 0 {
let code = GetLastError();
if code.0 != ERROR_CLASS_ALREADY_EXISTS {
anyhow::bail!("RegisterClassW failed: {code:?}");
}
}
let hwnd = CreateWindowExW(
WINDOW_EX_STYLE(0),
class_name,
w!(""),
WINDOW_STYLE(0),
0,
0,
0,
0,
Some(HWND_MESSAGE),
None,
Some(hinstance),
None,
)
.context("CreateWindowExW")?;
Ok(hwnd)
}
}
/// The message-loop thread body: build the window, wire up state, then pump until `WM_QUIT`.
fn pump_thread(
clip_tx: ClipTx,
cmd_rx: tokio::sync::mpsc::UnboundedReceiver<Cmd>,
current_wire: Arc<Mutex<Vec<String>>>,
fmt_html: u32,
fmt_rtf: u32,
fmt_png: u32,
ready_tx: tokio::sync::oneshot::Sender<anyhow::Result<isize>>,
) {
let hwnd = match create_window() {
Ok(h) => h,
Err(e) => {
let _ = ready_tx.send(Err(e));
return;
}
};
// A clone that outlives the boxed state, so we can announce Closed after the pump ends.
let closed_tx = clip_tx.clone();
let state = Box::new(WinClip {
clip_tx,
current_wire,
cmd_rx: RefCell::new(cmd_rx),
offered: RefCell::new(Vec::new()),
fmt_html,
fmt_rtf,
fmt_png,
own_hwnd: hwnd,
});
let ptr = Box::into_raw(state);
// SAFETY: stash the state pointer for the WndProc; the window was created on this thread and the
// pointer stays valid until we reclaim the Box after the pump exits.
unsafe {
SetWindowLongPtrW(hwnd, GWLP_USERDATA, ptr as isize);
}
// Snapshot whatever is already on the host clipboard, so the first client `enable` announces it
// (AddClipboardFormatListener only delivers *subsequent* changes).
{
// SAFETY: `ptr` is the live state we just stored; only this thread dereferences it.
let st = unsafe { &*ptr };
*st.current_wire.lock().unwrap() = st.available_wire_mimes();
}
// SAFETY: `hwnd` is our live window; start receiving WM_CLIPBOARDUPDATE.
if let Err(e) = unsafe { AddClipboardFormatListener(hwnd) } {
// SAFETY: tear down the half-built window and reclaim the leaked state box.
unsafe {
let _ = DestroyWindow(hwnd);
drop(Box::from_raw(ptr));
}
let _ = ready_tx.send(Err(
anyhow::Error::new(e).context("AddClipboardFormatListener")
));
return;
}
let _ = ready_tx.send(Ok(hwnd.0 as isize));
// SAFETY: the standard Win32 message pump. GetMessageW returns >0 for a message, 0 for WM_QUIT,
// and -1 on error — `.0 > 0` exits on both 0 and -1.
unsafe {
let mut msg = MSG::default();
while GetMessageW(&mut msg, None, 0, 0).0 > 0 {
let _ = TranslateMessage(&msg);
DispatchMessageW(&msg);
}
}
// Pump exited (window destroyed): reclaim the leaked state box. No WndProc runs after this point.
// SAFETY: `ptr` came from Box::into_raw above, is dereferenced only on this thread, and the
// message loop has ended so no further access occurs.
unsafe {
drop(Box::from_raw(ptr));
}
let _ = closed_tx.send(ClipEvent::Closed);
}
/// The window procedure. Reaches per-window state through `GWLP_USERDATA`; runs only on the message
/// thread. Registered as the class `WNDPROC` (a safe fn coerces to the `unsafe extern "system"` ABI).
extern "system" fn wndproc(hwnd: HWND, msg: u32, wparam: WPARAM, lparam: LPARAM) -> LRESULT {
// SAFETY: GWLP_USERDATA holds the `*const WinClip` stored right after window creation (0/null for
// the WM_(NC)CREATE messages that fire before that — handled by the null check below).
let ptr = unsafe { GetWindowLongPtrW(hwnd, GWLP_USERDATA) } as *const WinClip;
if ptr.is_null() {
// SAFETY: default processing before our state pointer is attached.
return unsafe { DefWindowProcW(hwnd, msg, wparam, lparam) };
}
// SAFETY: `ptr` is the live Box<WinClip> leaked in pump_thread, owned by this (the only) message
// thread and freed only after the pump exits; the WndProc is not re-entered for this window, so
// `&*ptr` is a valid shared borrow.
let st = unsafe { &*ptr };
match msg {
WM_CLIPBOARDUPDATE => {
st.on_clipboard_update(hwnd);
LRESULT(0)
}
WM_RENDERFORMAT => {
st.on_render_format(wparam.0 as u32);
LRESULT(0)
}
WM_APP_CMD => {
st.drain_commands(hwnd);
LRESULT(0)
}
WM_DESTROY => {
// SAFETY: ends the GetMessageW pump by posting WM_QUIT to this thread's queue.
unsafe {
PostQuitMessage(0);
}
LRESULT(0)
}
// SAFETY: default handling for every other message.
_ => unsafe { DefWindowProcW(hwnd, msg, wparam, lparam) },
}
}
+257
View File
@@ -0,0 +1,257 @@
//! Pure byte conversions between the Win32 clipboard formats and the portable wire MIMEs
//! (`design/clipboard-and-file-transfer.md` §3.5). Kept free of any `windows`-crate dependency so it
//! compiles on every host and its unit tests exercise the fiddly bits (CF_HTML offset math, UTF-16
//! (de)serialization) without a Windows box. The [`super::windows`] backend is the only production
//! consumer; it wraps these with the actual `GetClipboardData`/`SetClipboardData` calls.
//!
//! Format map (Win32 ↔ wire):
//! * `CF_UNICODETEXT` (UTF-16LE + NUL) ↔ `text/plain;charset=utf-8`
//! * `"HTML Format"` (CF_HTML, UTF-8 + ASCII header) ↔ `text/html`
//! * `"Rich Text Format"` (raw RTF) ↔ `text/rtf`
//! * `"PNG"` (raw PNG) ↔ `image/png` — identity, handled inline by the backend.
// ---- CF_UNICODETEXT ↔ text/plain;charset=utf-8 -----------------------------------------------
/// `CF_UNICODETEXT` HGLOBAL bytes → UTF-8 wire bytes. `raw` is the exact `GlobalSize`-length buffer;
/// it holds little-endian UTF-16 code units terminated by a single `0x0000`.
pub fn text_from_utf16(raw: &[u8]) -> Vec<u8> {
// Reinterpret each LE 2-byte pair as a UTF-16 code unit; a stray odd trailing byte (never present
// in valid data) is dropped by `chunks_exact`.
let mut units: Vec<u16> = raw
.chunks_exact(2)
.map(|c| u16::from_le_bytes([c[0], c[1]]))
.collect();
// Strip exactly one trailing NUL terminator if present (guard against eating a real code unit).
if units.last() == Some(&0) {
units.pop();
}
String::from_utf16_lossy(&units).into_bytes()
}
/// UTF-8 wire bytes → `CF_UNICODETEXT` HGLOBAL bytes (UTF-16LE + a required `0x0000` terminator).
pub fn text_to_utf16(wire: &[u8]) -> Vec<u8> {
let s = String::from_utf8_lossy(wire);
let mut out = Vec::with_capacity(wire.len() * 2 + 2);
for u in s.encode_utf16() {
out.extend_from_slice(&u.to_le_bytes());
}
out.extend_from_slice(&0u16.to_le_bytes()); // REQUIRED NUL terminator for CF_UNICODETEXT
out
}
// ---- "HTML Format" (CF_HTML) ↔ text/html -----------------------------------------------------
//
// CF_HTML is UTF-8: an ASCII `Key:Value\r\n` header carrying byte offsets, then the HTML with
// `<!--StartFragment-->`/`<!--EndFragment-->` markers. Offsets are byte counts from buffer start;
// the offsets live *inside* the header, so their digit-width feeds back into the header length. The
// spec-blessed fix (Chromium/Firefox/LibreOffice) is fixed-width 10-digit zero-padded offsets, which
// makes the header a compile-time constant and every offset a one-pass computation.
const CF_HTML_HEADER: &str = "Version:0.9\r\n\
StartHTML:0000000000\r\n\
EndHTML:0000000000\r\n\
StartFragment:0000000000\r\n\
EndFragment:0000000000\r\n";
const CF_HTML_PREFIX: &str = "<html><body>\r\n<!--StartFragment-->";
const CF_HTML_SUFFIX: &str = "<!--EndFragment-->\r\n</body></html>";
/// UTF-8 HTML fragment (wire bytes) → a `CF_HTML` buffer, NUL-terminated. The trailing NUL is the
/// conventional CF_HTML expectation (§4); `EndHTML` still points at content end, before the NUL.
pub fn html_to_cf(wire: &[u8]) -> Vec<u8> {
let fragment = String::from_utf8_lossy(wire);
let start_html = CF_HTML_HEADER.len(); // 105
let start_fragment = start_html + CF_HTML_PREFIX.len(); // 139
let end_fragment = start_fragment + fragment.len(); // byte length — fragment may be multibyte
let end_html = end_fragment + CF_HTML_SUFFIX.len();
let mut buf = Vec::with_capacity(end_html + 1);
buf.extend_from_slice(CF_HTML_HEADER.as_bytes());
buf.extend_from_slice(CF_HTML_PREFIX.as_bytes());
buf.extend_from_slice(fragment.as_bytes());
buf.extend_from_slice(CF_HTML_SUFFIX.as_bytes());
// Overwrite the four zero-padded fields in place, restricting the search to the header region so a
// fragment that happens to contain "StartHTML:" can't fool the patcher.
patch_offset(&mut buf[..start_html], b"StartHTML:", start_html);
patch_offset(&mut buf[..start_html], b"EndHTML:", end_html);
patch_offset(&mut buf[..start_html], b"StartFragment:", start_fragment);
patch_offset(&mut buf[..start_html], b"EndFragment:", end_fragment);
buf.push(0); // conventional NUL terminator
buf
}
/// A `CF_HTML` buffer → the UTF-8 HTML fragment (wire bytes). Uses the `StartFragment`/`EndFragment`
/// offsets; falls back to `StartHTML`/`EndHTML`, then to the whole buffer, if the markers are absent.
pub fn html_from_cf(raw: &[u8]) -> Vec<u8> {
let range = header_range(raw, b"StartFragment:", b"EndFragment:")
.or_else(|| header_range(raw, b"StartHTML:", b"EndHTML:"));
match range {
// Content is UTF-8 per spec; return the exact slice (drop any trailing NUL for cleanliness).
Some((start, end)) => {
let slice = &raw[start..end];
strip_trailing_nul(slice).to_vec()
}
None => strip_trailing_nul(raw).to_vec(),
}
}
/// Resolve `[start_label .. end_label]` into a validated byte range within `raw`.
fn header_range(raw: &[u8], start_label: &[u8], end_label: &[u8]) -> Option<(usize, usize)> {
let start = read_header_offset(raw, start_label)?;
let end = read_header_offset(raw, end_label)?;
if start <= end && end <= raw.len() {
Some((start, end))
} else {
None
}
}
/// Overwrite the 10 ASCII digits following `label` in `header` with `value`, zero-padded.
fn patch_offset(header: &mut [u8], label: &[u8], value: usize) {
if let Some(pos) = find(header, label) {
let at = pos + label.len();
if at + 10 <= header.len() {
let digits = format!("{value:010}");
header[at..at + 10].copy_from_slice(digits.as_bytes());
}
}
}
/// Read the decimal integer following `label:` in the ASCII header. The colon-suffixed labels only
/// match in the header, never the marker comments (`<!--StartFragment-->`) or fragment text.
fn read_header_offset(raw: &[u8], label: &[u8]) -> Option<usize> {
let mut at = find(raw, label)? + label.len();
let mut n: usize = 0;
let mut any = false;
while let Some(&b) = raw.get(at) {
if b.is_ascii_digit() {
n = n.checked_mul(10)?.checked_add((b - b'0') as usize)?;
any = true;
at += 1;
} else {
break; // stops at '\r'
}
}
any.then_some(n)
}
fn find(hay: &[u8], needle: &[u8]) -> Option<usize> {
if needle.is_empty() || needle.len() > hay.len() {
return None;
}
hay.windows(needle.len()).position(|w| w == needle)
}
// ---- "Rich Text Format" ↔ text/rtf -----------------------------------------------------------
/// `"Rich Text Format"` HGLOBAL bytes → RTF wire bytes. RTF is `{ }`-delimited; some producers append
/// a NUL past the final `}`, so strip a single trailing NUL to keep the wire payload byte-clean.
pub fn rtf_from_cf(raw: &[u8]) -> Vec<u8> {
strip_trailing_nul(raw).to_vec()
}
fn strip_trailing_nul(b: &[u8]) -> &[u8] {
match b.last() {
Some(0) => &b[..b.len() - 1],
_ => b,
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn text_round_trips_and_handles_terminator() {
// UTF-8 → UTF-16LE+NUL → UTF-8.
let wire = "héllo 🌍".as_bytes();
let cf = text_to_utf16(wire);
// Ends with a single 0x0000 terminator.
assert_eq!(&cf[cf.len() - 2..], &[0, 0]);
assert_eq!(text_from_utf16(&cf), wire);
// A CF buffer *without* a terminator still decodes (no code unit eaten).
let no_term: Vec<u8> = "hi".encode_utf16().flat_map(u16::to_le_bytes).collect();
assert_eq!(text_from_utf16(&no_term), b"hi");
// Empty text → just the terminator → empty wire.
assert_eq!(text_to_utf16(b""), vec![0, 0]);
assert_eq!(text_from_utf16(&[0, 0]), b"");
}
#[test]
fn cf_html_matches_the_spec_offsets() {
// The worked example from the format reference: fragment "Hello".
let cf = html_to_cf(b"Hello");
let s = String::from_utf8(cf.clone()).unwrap();
assert!(s.contains("StartHTML:0000000105"), "{s}");
assert!(s.contains("EndHTML:0000000178"), "{s}");
assert!(s.contains("StartFragment:0000000139"), "{s}");
assert!(s.contains("EndFragment:0000000144"), "{s}");
// The declared fragment range must slice back to exactly "Hello".
let start = read_header_offset(&cf, b"StartFragment:").unwrap();
let end = read_header_offset(&cf, b"EndFragment:").unwrap();
assert_eq!(&cf[start..end], b"Hello");
// Trailing NUL present, and EndHTML points *before* it.
assert_eq!(*cf.last().unwrap(), 0);
assert_eq!(read_header_offset(&cf, b"EndHTML:").unwrap(), cf.len() - 1);
}
#[test]
fn cf_html_round_trips_including_multibyte() {
for frag in [
"Hello",
"<b>bold</b> & <i>ital</i>",
"café ☕ <span>x</span>",
"",
] {
let cf = html_to_cf(frag.as_bytes());
assert_eq!(html_from_cf(&cf), frag.as_bytes(), "fragment {frag:?}");
}
}
#[test]
fn cf_html_extract_tolerates_foreign_producers() {
// A producer that adds SourceURL and uses Version 1.0 — offsets must still drive extraction,
// never a hardcoded 105-byte header.
let fragment = "picked";
let prefix = "<html><body><!--StartFragment-->";
let header_body = format!(
"Version:1.0\r\nStartHTML:{sh:010}\r\nEndHTML:{eh:010}\r\n\
StartFragment:{sf:010}\r\nEndFragment:{ef:010}\r\nSourceURL:https://x/\r\n",
sh = 0,
eh = 0,
sf = 0,
ef = 0,
);
// Compute real offsets against this ad-hoc layout.
let start_html = header_body.len();
let start_fragment = start_html + prefix.len();
let end_fragment = start_fragment + fragment.len();
let end_html = end_fragment + "<!--EndFragment--></body></html>".len();
let full = format!(
"Version:1.0\r\nStartHTML:{start_html:010}\r\nEndHTML:{end_html:010}\r\n\
StartFragment:{start_fragment:010}\r\nEndFragment:{end_fragment:010}\r\nSourceURL:https://x/\r\n\
{prefix}{fragment}<!--EndFragment--></body></html>"
);
assert_eq!(html_from_cf(full.as_bytes()), fragment.as_bytes());
}
#[test]
fn cf_html_extract_falls_back_without_markers() {
// No fragment markers at all → whole buffer (minus any NUL).
let mut b = b"<p>no markers</p>".to_vec();
assert_eq!(html_from_cf(&b), b"<p>no markers</p>");
b.push(0);
assert_eq!(html_from_cf(&b), b"<p>no markers</p>");
}
#[test]
fn rtf_strips_one_trailing_nul() {
assert_eq!(rtf_from_cf(br"{\rtf1 hi}"), br"{\rtf1 hi}");
assert_eq!(rtf_from_cf(b"{\\rtf1 hi}\0"), br"{\rtf1 hi}");
// Only one NUL is stripped.
assert_eq!(rtf_from_cf(b"x\0\0"), b"x\0");
}
}
+147
View File
@@ -0,0 +1,147 @@
//! Shared clipboard, host side (plan §W6 shape; `design/clipboard-and-file-transfer.md` §4).
//!
//! The wire protocol and the client half live in `punktfunk-core` (`punktfunk_core::quic` +
//! `punktfunk_core::clipboard`); this crate drives the **host's** real session clipboard through
//! the per-OS backends in [`host`] and bridges it to the QUIC clipboard plane through the
//! [`host::session`] coordinator.
//!
//! The orchestrator consumes only this portable facade — [`policy`] / [`enabled`] /
//! [`cap_advertised`], the [`ClipCoordCmd`] channel vocabulary, [`start`], and
//! [`spawn_decline_loop`] — so its control loop compiles unchanged on every host platform; the
//! platform split lives entirely behind [`start`].
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use punktfunk_core::quic::ClipOffer;
/// The per-OS backends (`ext-data-control-v1` / Mutter direct / Win32) behind one
/// `HostClipboard`, plus the backend-agnostic [`host::session`] coordinator.
#[cfg(any(target_os = "linux", target_os = "windows"))]
pub mod host;
/// Operator clipboard policy from `PUNKTFUNK_CLIPBOARD` (`design/clipboard-and-file-transfer.md`
/// §4.2): `off` (default — the whole feature is dark), `on` / `1` (text + files), `text-only` /
/// `no-files` (text/RTF/HTML/image only). Returns `None` when clipboard is off (the host neither
/// advertises the cap nor accepts fetch streams); otherwise the permitted-format
/// [`punktfunk_core::quic::CLIP_POLICY_TEXT`] / `CLIP_POLICY_FILES` bitfield.
///
/// The policy gates the advertised capability and whether the [`host::session`] coordinator
/// starts. `off` keeps the whole feature dark.
pub fn policy() -> Option<u8> {
use punktfunk_core::quic::{CLIP_POLICY_FILES, CLIP_POLICY_TEXT};
match std::env::var("PUNKTFUNK_CLIPBOARD")
.unwrap_or_default()
.trim()
.to_ascii_lowercase()
.as_str()
{
"" | "0" | "off" | "false" => None,
"text-only" | "no-files" | "text" => Some(CLIP_POLICY_TEXT),
_ => Some(CLIP_POLICY_TEXT | CLIP_POLICY_FILES), // "on" / "1" / anything truthy
}
}
/// Whether the shared clipboard is enabled at all for this host (policy not `off`).
pub fn enabled() -> bool {
policy().is_some()
}
/// Whether the host should advertise `HOST_CAP_CLIPBOARD` in the `Welcome`: the operator policy
/// enables it AND this platform has a backend (Linux data-control / Mutter, or the Win32
/// clipboard) — the client greys the toggle out otherwise. A Linux host whose compositor lacks
/// data-control still advertises it and answers a later enable with `BACKEND_UNAVAILABLE`, so the
/// client can surface *why* it's unavailable.
pub fn cap_advertised() -> bool {
enabled() && cfg!(any(target_os = "linux", target_os = "windows"))
}
/// A command from the session control loop into the host clipboard coordinator
/// ([`host::session`]). Defined here — portable — so the control loop compiles on every host
/// platform; the coordinator that consumes it exists only where a backend does.
pub enum ClipCoordCmd {
/// The client toggled sync. When enabled, the coordinator (re)announces the current host
/// clipboard; when disabled, it drops any selection it owns and stops forwarding host copies.
SetEnabled(bool),
/// The client copied: install its offered wire MIMEs as a lazy host selection (empty = clear).
RemoteOffer { seq: u32, mimes: Vec<String> },
}
/// Handle to the host clipboard coordinator, held by the session control loop.
pub struct ClipCoord {
/// Whether a real backend is live. `false` on gamescope / older GNOME / an unsupported
/// platform; the control loop then answers an enable request with
/// `CLIP_REASON_BACKEND_UNAVAILABLE` and [`spawn_decline_loop`] handles any stray fetch stream.
pub available: bool,
pub cmd_tx: tokio::sync::mpsc::UnboundedSender<ClipCoordCmd>,
/// Host-copy announcements from the coordinator → control loop → client.
pub offer_rx: tokio::sync::mpsc::UnboundedReceiver<ClipOffer>,
}
/// Open the host clipboard backend (when the operator policy allows it, this session mirrors a
/// real compositor, and the platform has a backend) and spawn its coordinator, returning a handle.
/// Otherwise the handle is inert (`available = false`, channels dropped) so the caller's control
/// loop stays platform-agnostic. `has_compositor` is false for the synthetic protocol-test source,
/// which has no display/clipboard to share — keeping it out of the real session clipboard.
pub async fn start(
conn: quinn::Connection,
clip_enabled: Arc<AtomicBool>,
has_compositor: bool,
) -> ClipCoord {
let (cmd_tx, cmd_rx) = tokio::sync::mpsc::unbounded_channel();
let (offer_tx, offer_rx) = tokio::sync::mpsc::unbounded_channel();
#[cfg(any(target_os = "linux", target_os = "windows"))]
let available = if has_compositor && enabled() {
host::session::start(conn, clip_enabled, cmd_rx, offer_tx).await
} else {
drop((conn, clip_enabled, cmd_rx, offer_tx));
false
};
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
let available = {
let _ = (conn, clip_enabled, cmd_rx, offer_tx, has_compositor);
false
};
ClipCoord {
available,
cmd_tx,
offer_rx,
}
}
/// Clipboard fetch-stream accept loop, fallback flavor (`design/clipboard-and-file-transfer.md`
/// §3.3, §4.2). When a backend is live the coordinator (spawned by [`start`]) owns `accept_bi` and
/// serves real host clipboard bytes. This is for the other case: the operator allowed the cap but
/// no backend bound (gamescope / older GNOME / a not-yet-implemented platform), so a stray or
/// hostile fetch stream is answered `CLIP_FETCH_UNAVAILABLE` instead of hanging. Exactly one
/// `accept_bi` consumer runs (this OR the coordinator). The control stream is the FIRST bi-stream
/// (already accepted at the handshake), so this loop only ever sees clipboard fetch streams; it
/// dies with the connection.
pub fn spawn_decline_loop(conn: quinn::Connection) {
tokio::spawn(async move {
use punktfunk_core::quic::{clipstream, ClipFetchHdr, CLIP_FETCH_UNAVAILABLE};
while let Ok((mut send, mut recv)) = conn.accept_bi().await {
tokio::spawn(async move {
// Validate the stream header + request; a malformed/unknown stream is dropped.
match clipstream::read_stream_header(&mut recv).await {
Ok(k) if k == clipstream::CLIP_STREAM_KIND_FETCH => {}
_ => {
let _ = send.reset(clipstream::cancelled_code());
return;
}
}
if clipstream::read_fetch(&mut recv).await.is_err() {
return;
}
let _ = clipstream::write_fetch_hdr(
&mut send,
&ClipFetchHdr {
status: CLIP_FETCH_UNAVAILABLE,
total_size: 0,
},
)
.await;
});
}
});
}
+1 -1
View File
@@ -101,7 +101,7 @@ pub fn import(
// EGL could leave an INVALID modifier to the driver's implied choice; explicit-
// modifier images can't — LINEAR is the only honest guess (debug-visible if wrong).
let modifier = if frame.modifier == DRM_FORMAT_MOD_INVALID {
tracing::debug!("dmabuf carried no explicit modifier — importing as LINEAR");
tracing::trace!("dmabuf carried no explicit modifier — importing as LINEAR");
DRM_FORMAT_MOD_LINEAR
} else {
frame.modifier
+19 -3
View File
@@ -184,6 +184,11 @@ struct StreamState {
// Hardware-path health: a failure streak (or a device with no import support at
// all) demotes the decoder to software via the shared flag — once per session.
dmabuf_demoted: bool,
/// PyroWave present has no demote rung (nothing else decodes the codec), so a
/// persistent non-device-lost present failure would warn on every frame. Latch it:
/// warn on the first failure of a streak, then stay quiet until a present succeeds.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
pyro_present_warned: bool,
hw_fails: u32,
/// The OSD's text (multi-line; rebuilt each Stats window and on a live tier cycle).
osd_text: String,
@@ -255,6 +260,8 @@ impl StreamState {
win_start: Instant::now(),
presented: PresentedWindow::default(),
dmabuf_demoted: false,
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
pyro_present_warned: false,
hw_fails: 0,
osd_text: String::new(),
last_stats: None,
@@ -542,7 +549,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
fullscreen = !fullscreen;
tracing::debug!(fullscreen, "fullscreen toggle");
if let Err(e) = window.set_fullscreen(fullscreen) {
tracing::warn!(error = %e, "fullscreen toggle");
tracing::warn!(error = %e, fullscreen, "failed to toggle fullscreen");
}
continue;
}
@@ -985,13 +992,22 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
FrameInput::PyroWave(f),
overlay_frame.as_ref(),
) {
Ok(p) => p,
Ok(p) => {
st.pyro_present_warned = false;
p
}
Err(e) => {
if device_lost(&e) {
return Err(e)
.context("GPU device lost — the session cannot continue");
}
tracing::warn!(error = %format!("{e:#}"), "pyrowave present failed");
if !st.pyro_present_warned {
st.pyro_present_warned = true;
tracing::warn!(
error = %format!("{e:#}"),
"pyrowave present failed — suppressing repeats until it recovers"
);
}
false
}
}
+5 -1
View File
@@ -17,9 +17,13 @@ crate-type = ["lib", "cdylib", "staticlib"]
[features]
default = []
# Cert-fingerprint pinning shared across the clients (the one `PinVerify` in `tls.rs`). Light:
# rustls + sha2 only, no QUIC runtime — so a lean consumer (the tray's loopback status poll) can
# pin the host cert without pulling tokio/quinn. The heavier `quic` feature builds on top of it.
tls = ["dep:rustls", "dep:sha2", "dep:rustls-pki-types"]
# Control-plane QUIC (pairing, config, reverse audio). tokio is permitted ONLY here,
# never on the per-frame hot path. Off by default so the core stays runtime-free.
quic = ["dep:quinn", "dep:tokio", "dep:rustls", "dep:rcgen", "dep:rustls-pki-types", "dep:sha2", "dep:hmac", "dep:spake2", "dep:opus"]
quic = ["tls", "dep:quinn", "dep:tokio", "dep:rcgen", "dep:hmac", "dep:spake2", "dep:opus"]
[dependencies]
reed-solomon-simd = "3.1" # GF(2^16) Leopard-RS, SIMD, O(n log n) — the wall-breaker (P2)
+406
View File
@@ -527,6 +527,10 @@ pub struct PunktfunkConnection {
/// `inner.audio_channels`; `pcm` is a fixed-capacity reusable buffer the returned pointer
/// borrows until the next PCM call (same contract as `last_audio`).
audio_pcm: std::sync::Mutex<AudioPcmState>,
/// Backs the `data`/`len` pointer of the last `punktfunk_connection_next_clipboard` event
/// (a fetched payload, an offer's format list, or a fetch-request's MIME) —
/// borrow-until-next-call, same contract as `last`.
last_clip: std::sync::Mutex<Option<Vec<u8>>>,
}
/// Lazily-initialized in-core Opus decode state. A coupled-1-stream multistream decoder is
@@ -948,6 +952,14 @@ pub const PUNKTFUNK_CODEC_AV1: u8 = 0x04;
/// (design/pyrowave-codec-plan.md §3). (Mirrors `quic::CODEC_PYROWAVE`.)
pub const PUNKTFUNK_CODEC_PYROWAVE: u8 = 0x08;
/// Host-capability bit in [`punktfunk_connection_host_caps`]: the host applies gamepad-state
/// snapshots (a capable client sends full-state snapshots instead of per-transition events).
/// (Mirrors `quic::HOST_CAP_GAMEPAD_STATE`.)
pub const PUNKTFUNK_HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// Host-capability bit in [`punktfunk_connection_host_caps`]: the host supports the shared
/// clipboard, so a client may offer the toggle. (Mirrors `quic::HOST_CAP_CLIPBOARD`.)
pub const PUNKTFUNK_HOST_CAP_CLIPBOARD: u8 = 0x02;
// Keep the ABI cap bits in lockstep with the wire constants (compile-time guard against drift).
#[cfg(feature = "quic")]
const _: () = {
@@ -958,6 +970,8 @@ const _: () = {
assert!(PUNKTFUNK_CODEC_HEVC == crate::quic::CODEC_HEVC);
assert!(PUNKTFUNK_CODEC_AV1 == crate::quic::CODEC_AV1);
assert!(PUNKTFUNK_CODEC_PYROWAVE == crate::quic::CODEC_PYROWAVE);
assert!(PUNKTFUNK_HOST_CAP_GAMEPAD_STATE == crate::quic::HOST_CAP_GAMEPAD_STATE);
assert!(PUNKTFUNK_HOST_CAP_CLIPBOARD == crate::quic::HOST_CAP_CLIPBOARD);
};
// Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift).
@@ -1552,6 +1566,7 @@ unsafe fn connect_ex_impl(
last: std::sync::Mutex::new(None),
last_audio: std::sync::Mutex::new(None),
audio_pcm: std::sync::Mutex::new(AudioPcmState::default()),
last_clip: std::sync::Mutex::new(None),
}))
}
Err(e) => {
@@ -2456,6 +2471,397 @@ pub unsafe extern "C" fn punktfunk_connection_gamepad(
})
}
// ============================================================================================
// Shared clipboard (design/clipboard-and-file-transfer.md §5.1). Additive, ABI v6. All poll/serve
// bytes ride the mTLS-pinned QUIC session; nothing here opens a new listener or port.
// ============================================================================================
/// [`PunktfunkClipEvent::kind`] — the host announced clipboard content is available
/// (`transfer_id` = the offer `seq`; `data`/`len` = a `\n`-separated `"<mime>\t<size_hint>"`
/// format list). Fetch it lazily (only on a local paste) via
/// [`punktfunk_connection_clipboard_fetch`].
pub const PUNKTFUNK_CLIP_REMOTE_OFFER: u8 = 1;
/// [`PunktfunkClipEvent::kind`] — host ack / policy / backend update (`enabled`/`policy`/`reason`
/// valid). Reflect it in the toggle UI.
pub const PUNKTFUNK_CLIP_STATE: u8 = 2;
/// [`PunktfunkClipEvent::kind`] — the host is pasting our offered data: answer with
/// [`punktfunk_connection_clipboard_serve`] (`transfer_id` = `req_id`; `seq`/`file_index` valid;
/// `data`/`len` = the requested MIME).
pub const PUNKTFUNK_CLIP_FETCH_REQUEST: u8 = 3;
/// [`PunktfunkClipEvent::kind`] — bytes for a fetch we started (`transfer_id` = `xfer_id`;
/// `data`/`len` = the payload, borrowed until the next `next_clipboard`; `last` = final chunk).
pub const PUNKTFUNK_CLIP_DATA: u8 = 4;
/// [`PunktfunkClipEvent::kind`] — a transfer was cancelled (`transfer_id` = the id).
pub const PUNKTFUNK_CLIP_CANCELLED: u8 = 5;
/// [`PunktfunkClipEvent::kind`] — a transfer failed (`transfer_id` = the id; `status` = a
/// `PunktfunkStatus` code).
pub const PUNKTFUNK_CLIP_ERROR: u8 = 6;
/// One advertised clipboard format passed to [`punktfunk_connection_clipboard_offer`].
#[cfg(feature = "quic")]
#[repr(C)]
pub struct PunktfunkClipKind {
/// NUL-terminated UTF-8 wire MIME (e.g. `text/plain;charset=utf-8`). ≤ 128 bytes on the wire.
pub mime: *const std::os::raw::c_char,
/// Best-effort size in bytes; `0` = unknown.
pub size_hint: u64,
}
/// A shared-clipboard event, filled by [`punktfunk_connection_next_clipboard`]. A flat tagged
/// struct (like `PunktfunkHidOutput`): read the fields named in the `kind`'s doc; the rest are 0.
#[cfg(feature = "quic")]
#[repr(C)]
#[derive(Clone, Copy)]
pub struct PunktfunkClipEvent {
/// One of `PUNKTFUNK_CLIP_*`.
pub kind: u8,
/// `State`: 1 = enabled, 0 = disabled.
pub enabled: u8,
/// `State`: bitfield of `quic::CLIP_POLICY_*` — what the host currently permits.
pub policy: u8,
/// `State`: one of `quic::CLIP_REASON_*`.
pub reason: u8,
/// `Data`: 1 = final chunk of this transfer.
pub last: u8,
/// Per-transfer id: the offer `seq` (RemoteOffer), the `req_id` (FetchRequest), or the
/// `xfer_id` (Data/Cancelled/Error).
pub transfer_id: u32,
/// `FetchRequest`: the offer `seq` the request is against.
pub seq: u32,
/// `FetchRequest`: file index, or `quic::CLIP_FILE_INDEX_NONE`.
pub file_index: u32,
/// `Error`: a `PunktfunkStatus` code (negative); 0 otherwise.
pub status: i32,
/// RemoteOffer/FetchRequest/Data: a pointer into a per-connection slot, valid until the next
/// `next_clipboard` call; NULL for the other kinds.
pub data: *const u8,
/// Byte length of `data` (0 when `data` is NULL).
pub len: usize,
}
/// Fill a [`PunktfunkClipEvent`] from a core event, parking any variable-length bytes in `slot`
/// (borrow-until-next-call) and pointing `data`/`len` at them.
#[cfg(feature = "quic")]
fn build_clip_event(
ev: crate::clipboard::ClipEventCore,
slot: &mut Option<Vec<u8>>,
) -> PunktfunkClipEvent {
use crate::clipboard::ClipEventCore as E;
let mut out = PunktfunkClipEvent {
kind: 0,
enabled: 0,
policy: 0,
reason: 0,
last: 0,
transfer_id: 0,
seq: 0,
file_index: 0,
status: 0,
data: std::ptr::null(),
len: 0,
};
*slot = None;
match ev {
E::RemoteOffer { seq, kinds } => {
out.kind = PUNKTFUNK_CLIP_REMOTE_OFFER;
out.transfer_id = seq;
let mut blob = String::new();
for k in &kinds {
blob.push_str(&k.mime);
blob.push('\t');
blob.push_str(&k.size_hint.to_string());
blob.push('\n');
}
*slot = Some(blob.into_bytes());
}
E::State {
enabled,
policy,
reason,
} => {
out.kind = PUNKTFUNK_CLIP_STATE;
out.enabled = enabled as u8;
out.policy = policy;
out.reason = reason;
}
E::FetchRequest {
req_id,
seq,
file_index,
mime,
} => {
out.kind = PUNKTFUNK_CLIP_FETCH_REQUEST;
out.transfer_id = req_id;
out.seq = seq;
out.file_index = file_index;
*slot = Some(mime.into_bytes());
}
E::Data {
xfer_id,
bytes,
last,
} => {
out.kind = PUNKTFUNK_CLIP_DATA;
out.transfer_id = xfer_id;
out.last = last as u8;
*slot = Some(bytes);
}
E::Cancelled { id } => {
out.kind = PUNKTFUNK_CLIP_CANCELLED;
out.transfer_id = id;
}
E::Error { id, code } => {
out.kind = PUNKTFUNK_CLIP_ERROR;
out.transfer_id = id;
out.status = code;
}
}
if let Some(v) = slot.as_ref() {
out.data = v.as_ptr();
out.len = v.len();
}
out
}
/// The host capability bitfield the session's `Welcome` carried — a bitfield of
/// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD`. A client tests
/// `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide whether to offer the shared-clipboard toggle.
/// Safe any time after connect.
///
/// # Safety
/// `c` is a valid connection handle; `caps` is writable (NULL is skipped).
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_host_caps(
c: *const PunktfunkConnection,
caps: *mut u8,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
unsafe {
if !caps.is_null() {
*caps = c.inner.host_caps();
}
}
PunktfunkStatus::Ok
})
}
/// Enable or disable the shared clipboard for this session (`design` §3.1). Opt-in: nothing is
/// announced or served until this is called with `enabled = true`. `flags` carries
/// `quic::CLIP_FLAG_FILES` (allow file transfer). The host replies with a `State` event.
///
/// # Safety
/// `c` is a valid connection handle.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_clipboard_control(
c: *const PunktfunkConnection,
enabled: bool,
flags: u8,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
match c.inner.clip_control(enabled, flags) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
}
})
}
/// Announce that the local clipboard changed — the lazy format-list offer. `seq` is a monotonic
/// per-sender counter (newest wins); `kinds`/`n` is the advertised formats (≤ 16). The bytes cross
/// only if the host later fetches.
///
/// # Safety
/// `c` is a valid connection handle; `kinds` points to `n` `PunktfunkClipKind`s (NULL allowed only
/// when `n == 0`), each `mime` a valid NUL-terminated UTF-8 string.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_clipboard_offer(
c: *const PunktfunkConnection,
seq: u32,
kinds: *const PunktfunkClipKind,
n: usize,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if kinds.is_null() && n != 0 {
return PunktfunkStatus::NullPointer;
}
let mut out = Vec::with_capacity(n);
if n != 0 {
let slice = unsafe { std::slice::from_raw_parts(kinds, n) };
for k in slice {
let mime = if k.mime.is_null() {
String::new()
} else {
match unsafe { std::ffi::CStr::from_ptr(k.mime) }.to_str() {
Ok(s) => s.to_string(),
Err(_) => return PunktfunkStatus::InvalidArg,
}
};
out.push(crate::quic::ClipKind {
mime,
size_hint: k.size_hint,
});
}
}
match c.inner.clip_offer(seq, out) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
}
})
}
/// Start pulling one format (`mime`) of the host's current offer `seq` — lazily, on a local paste.
/// `file_index` selects a file for a file transfer, or `quic::CLIP_FILE_INDEX_NONE` for a non-file
/// format. Writes the transfer id (echoed on the resulting `Data`/`Error`/`Cancelled` event) to
/// `xfer_id_out`.
///
/// # Safety
/// `c` is a valid connection handle; `mime` is a valid NUL-terminated UTF-8 string; `xfer_id_out`
/// is writable (NULL is skipped).
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_clipboard_fetch(
c: *const PunktfunkConnection,
seq: u32,
mime: *const std::os::raw::c_char,
file_index: u32,
xfer_id_out: *mut u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if mime.is_null() {
return PunktfunkStatus::NullPointer;
}
let mime = match unsafe { std::ffi::CStr::from_ptr(mime) }.to_str() {
Ok(s) => s.to_string(),
Err(_) => return PunktfunkStatus::InvalidArg,
};
match c.inner.clip_fetch(seq, mime, file_index) {
Ok(xfer_id) => {
unsafe {
if !xfer_id_out.is_null() {
*xfer_id_out = xfer_id;
}
}
PunktfunkStatus::Ok
}
Err(e) => e.status(),
}
})
}
/// Provide bytes answering a `FetchRequest` event (the host is pasting our offered data). Call
/// repeatedly to stream a large payload; `last = true` completes it. `data` may be NULL only when
/// `len == 0` (e.g. a final empty chunk). `punktfunk_connection_clipboard_cancel(req_id)` aborts.
///
/// # Safety
/// `c` is a valid connection handle; `data` points to `len` bytes (NULL allowed only when
/// `len == 0`).
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_clipboard_serve(
c: *const PunktfunkConnection,
req_id: u32,
data: *const u8,
len: usize,
last: bool,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if data.is_null() && len != 0 {
return PunktfunkStatus::NullPointer;
}
let bytes = if len == 0 {
Vec::new()
} else {
unsafe { std::slice::from_raw_parts(data, len) }.to_vec()
};
match c.inner.clip_serve(req_id, bytes, last) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
}
})
}
/// Cancel a clipboard transfer by id — either an outbound fetch (`xfer_id` from
/// [`punktfunk_connection_clipboard_fetch`]) or an inbound serve (`req_id` from a `FetchRequest`).
///
/// # Safety
/// `c` is a valid connection handle.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_clipboard_cancel(
c: *const PunktfunkConnection,
id: u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
match c.inner.clip_cancel(id) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
}
})
}
/// Pull the next shared-clipboard event into `*out`. [`PunktfunkStatus::NoFrame`] on timeout,
/// [`PunktfunkStatus::Closed`] once the session ended. A native client drains this on its own
/// thread and drives the OS pasteboard from it. The `data`/`len` pointer (when non-NULL) borrows a
/// per-connection buffer valid until the next `next_clipboard` call on this handle.
///
/// # Safety
/// `c` is a valid connection handle; `out` is writable for one `PunktfunkClipEvent`.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_next_clipboard(
c: *mut PunktfunkConnection,
out: *mut PunktfunkClipEvent,
timeout_ms: u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if out.is_null() {
return PunktfunkStatus::NullPointer;
}
match c
.inner
.next_clip(std::time::Duration::from_millis(timeout_ms as u64))
{
Ok(ev) => {
let mut slot = c.last_clip.lock().unwrap();
let out_ev = build_clip_event(ev, &mut slot);
unsafe { *out = out_ev };
PunktfunkStatus::Ok
}
Err(e) => e.status(),
}
})
}
/// The compositor backend the host actually resolved for this session (one of the
/// `PUNKTFUNK_COMPOSITOR_*` values; the `Welcome`'s echo of the [`punktfunk_connect_ex`]
/// preference). `PUNKTFUNK_COMPOSITOR_AUTO` = an older host that didn't say. Clients use it for
+162 -6
View File
@@ -12,15 +12,16 @@
//! channel. All methods are safe to call from any single embedder thread.
use crate::abr::BitrateController;
use crate::clipboard::{ClipCommand, ClipEventCore};
use crate::config::{CompositorPref, GamepadPref, Mode, Role};
use crate::error::{PunktfunkError, Result};
use crate::input::InputEvent;
use crate::packet::FLAG_PROBE;
use crate::quic::{
accept_resync, endpoint, io, wall_clock_ns, window_loss_ppm, BitrateChanged, ClockEcho,
ClockResync, ColorInfo, HdrMeta, Hello, HidOutput, LossReport, ProbeRequest, ProbeResult,
Reconfigure, Reconfigured, RequestKeyframe, ResyncStep, RfiRequest, RichInput, SetBitrate,
Start, Welcome,
accept_resync, endpoint, io, wall_clock_ns, window_loss_ppm, BitrateChanged, ClipControl,
ClipKind, ClipOffer, ClipState, ClockEcho, ClockResync, ColorInfo, HdrMeta, Hello, HidOutput,
LossReport, ProbeRequest, ProbeResult, Reconfigure, Reconfigured, RequestKeyframe, ResyncStep,
RfiRequest, RichInput, SetBitrate, Start, Welcome,
};
use crate::session::{Frame, Session};
use crate::transport::UdpTransport;
@@ -62,6 +63,12 @@ enum CtrlRequest {
/// its report tick after the first no-op clock flush — the "the clock stepped under me"
/// signal; the control task also self-triggers one every [`CLOCK_RESYNC_INTERVAL`].
ClockResync,
/// Shared-clipboard enable/disable for this session (`design/clipboard-and-file-transfer.md`
/// §3.1). Idempotent; carries the file-permission flag.
ClipControl(ClipControl),
/// Announce that the local clipboard changed — the lazy format-list offer (bytes cross later on
/// a fetch stream). Symmetric message; the host may send one too.
ClipOffer(ClipOffer),
}
/// What the worker reports to [`NativeClient::connect`] once the handshake lands: the
@@ -95,6 +102,10 @@ struct Negotiated {
audio_channels: u8,
/// The single codec the host will emit (`quic::CODEC_*`).
codec: u8,
/// The host capability bitfield ([`Welcome::host_caps`]): [`crate::quic::HOST_CAP_GAMEPAD_STATE`],
/// [`crate::quic::HOST_CAP_CLIPBOARD`]. Exposed to the embedder via
/// [`NativeClient::host_caps`] so a native client greys out unsupported toggles.
host_caps: u8,
}
/// Accumulated state of an in-flight / finished speed test. The data-plane pump mirrors the
@@ -368,6 +379,12 @@ const HDR_META_QUEUE: usize = 8;
/// harmless, it's per-frame observability, not state.
const HOST_TIMING_QUEUE: usize = 512;
/// Clipboard event plane depth (offers, host acks, fetch-requests, fetched payloads). Clipboard
/// activity is human-paced and sparse; a small ring is ample. Overflow drops the newest event
/// (try_send), same discipline as the other planes — a dropped offer heals on the next copy, and
/// a dropped fetch-request makes the serving stream time out and reset cleanly.
const CLIP_EVENT_QUEUE: usize = 32;
/// One Opus packet from the host's audio datagram stream (48 kHz stereo, 5 ms frames).
#[derive(Clone, Debug)]
pub struct AudioPacket {
@@ -480,6 +497,19 @@ pub struct NativeClient {
/// Bounded ([`CTRL_QUEUE`]) — the requests are sparse; a full queue means the control task
/// is wedged/dead, and callers treat it like a closed session.
ctrl_tx: tokio::sync::mpsc::Sender<CtrlRequest>,
/// Inbound shared-clipboard events (remote offers, host acks, fetch-requests, fetched
/// payloads), drained by [`NativeClient::next_clip`] → the C ABI poll. Fed by the control task
/// (metadata) and the clipboard task (fetch data).
clip: Mutex<Receiver<ClipEventCore>>,
/// Outbound clipboard fetch/serve/cancel commands → the worker's clipboard task
/// ([`crate::clipboard::run`]). Unbounded like `input_tx`; the commands are sparse and each
/// carries at most one paste's bytes.
clip_cmd_tx: tokio::sync::mpsc::UnboundedSender<ClipCommand>,
/// Monotonic id for outbound fetches ([`NativeClient::clip_fetch`]); stays below
/// [`crate::clipboard::INBOUND_REQ_FLAG`] so it never collides with an inbound serve `req_id`.
next_xfer_id: AtomicU32,
/// The host capability bitfield ([`Welcome::host_caps`]) — see [`NativeClient::host_caps`].
pub host_caps: u8,
/// Speed-test accumulator, shared with the data-plane pump + control task.
probe: Arc<Mutex<ProbeState>>,
shutdown: Arc<AtomicBool>,
@@ -694,6 +724,9 @@ impl NativeClient {
let (mic_tx, mic_rx) = tokio::sync::mpsc::channel::<(u32, u64, Vec<u8>)>(MIC_QUEUE);
let (rich_input_tx, rich_input_rx) = tokio::sync::mpsc::unbounded_channel::<RichInput>();
let (ctrl_tx, ctrl_rx) = tokio::sync::mpsc::channel::<CtrlRequest>(CTRL_QUEUE);
let (clip_event_tx, clip_event_rx) =
std::sync::mpsc::sync_channel::<ClipEventCore>(CLIP_EVENT_QUEUE);
let (clip_cmd_tx, clip_cmd_rx) = tokio::sync::mpsc::unbounded_channel::<ClipCommand>();
let (ready_tx, ready_rx) = std::sync::mpsc::channel::<Result<Negotiated>>();
let shutdown = Arc::new(AtomicBool::new(false));
let quit = Arc::new(AtomicBool::new(false));
@@ -761,6 +794,8 @@ impl NativeClient {
rich_input_rx,
ctrl_rx,
ctrl_tx: ctrl_tx_pump,
clip_event_tx,
clip_cmd_rx,
ready_tx,
shutdown: shutdown_w,
quit: quit_w,
@@ -795,6 +830,10 @@ impl NativeClient {
mic_tx,
rich_input_tx,
ctrl_tx,
clip: Mutex::new(clip_event_rx),
clip_cmd_tx,
next_xfer_id: AtomicU32::new(1),
host_caps: negotiated.host_caps,
probe,
shutdown,
quit,
@@ -1309,6 +1348,83 @@ impl NativeClient {
self.input_tx.send(*ev).map_err(|_| PunktfunkError::Closed)
}
/// The host capability bitfield the [`Welcome`] carried ([`crate::quic::HOST_CAP_GAMEPAD_STATE`],
/// [`crate::quic::HOST_CAP_CLIPBOARD`]). A native client tests
/// `host_caps() & HOST_CAP_CLIPBOARD` to decide whether to offer the shared-clipboard toggle.
pub fn host_caps(&self) -> u8 {
self.host_caps
}
/// Enable or disable the shared clipboard for this session (`design/clipboard-and-file-transfer.md`
/// §3.1). Opt-in: nothing is announced or served until this crosses with `enabled = true`.
/// `flags` carries [`crate::quic::CLIP_FLAG_FILES`]. Non-blocking; the host replies with a
/// `State` event ([`NativeClient::next_clip`]).
pub fn clip_control(&self, enabled: bool, flags: u8) -> Result<()> {
self.ctrl_tx
.try_send(CtrlRequest::ClipControl(ClipControl { enabled, flags }))
.map_err(|_| PunktfunkError::Closed)
}
/// Announce that the local clipboard changed — the lazy format-list offer. `seq` is a
/// monotonic per-sender counter (newest wins); `kinds` is the advertised formats (≤
/// [`crate::quic::CLIP_MAX_KINDS`]). The bytes cross only if the host later fetches.
pub fn clip_offer(&self, seq: u32, kinds: Vec<ClipKind>) -> Result<()> {
self.ctrl_tx
.try_send(CtrlRequest::ClipOffer(ClipOffer { seq, kinds }))
.map_err(|_| PunktfunkError::Closed)
}
/// Start pulling one format (`mime`) of the host's current offer `seq` — lazily, when a local
/// app pastes. `file_index` selects a file for a file transfer, or
/// [`crate::quic::CLIP_FILE_INDEX_NONE`] for a non-file format. Returns the `xfer_id` echoed on
/// the resulting `Data` / `Error` / `Cancelled` event.
pub fn clip_fetch(&self, seq: u32, mime: String, file_index: u32) -> Result<u32> {
let xfer_id = self.next_xfer_id.fetch_add(1, Ordering::Relaxed);
// Stay in the low id space (inbound serve ids carry the high bit); wrap defensively.
let xfer_id = xfer_id & !crate::clipboard::INBOUND_REQ_FLAG;
self.clip_cmd_tx
.send(ClipCommand::Fetch {
xfer_id,
seq,
file_index,
mime,
})
.map_err(|_| PunktfunkError::Closed)?;
Ok(xfer_id)
}
/// Provide bytes answering a `FetchRequest` event (the host is pasting our offered data). Call
/// repeatedly to stream a large payload; `last = true` completes it. `clip_cancel(req_id)`
/// aborts instead.
pub fn clip_serve(&self, req_id: u32, bytes: Vec<u8>, last: bool) -> Result<()> {
self.clip_cmd_tx
.send(ClipCommand::Serve {
req_id,
bytes,
last,
})
.map_err(|_| PunktfunkError::Closed)
}
/// Cancel a clipboard transfer by id — either an outbound fetch (`xfer_id` from
/// [`NativeClient::clip_fetch`]) or an inbound serve (`req_id` from a `FetchRequest` event).
pub fn clip_cancel(&self, id: u32) -> Result<()> {
self.clip_cmd_tx
.send(ClipCommand::Cancel { id })
.map_err(|_| PunktfunkError::Closed)
}
/// Pull the next shared-clipboard event (remote offer, host ack/state, fetch-request, fetched
/// data, cancel, error); same timeout/closed semantics as [`NativeClient::next_hidout`]. A
/// native client drains this on its own thread and drives the OS pasteboard from it.
pub fn next_clip(&self, timeout: Duration) -> Result<ClipEventCore> {
match self.clip.lock().unwrap().recv_timeout(timeout) {
Ok(e) => Ok(e),
Err(RecvTimeoutError::Timeout) => Err(PunktfunkError::NoFrame),
Err(RecvTimeoutError::Disconnected) => Err(PunktfunkError::Closed),
}
}
/// Queue one Opus mic frame for delivery as a 0xCB uplink datagram (the inverse of
/// [`next_audio`](Self::next_audio)). `seq`/`pts_ns` are the caller's own counters (the host
/// uses them only for diagnostics). The host decodes it into a virtual microphone source.
@@ -1408,6 +1524,8 @@ struct WorkerArgs {
rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<RichInput>,
ctrl_rx: tokio::sync::mpsc::Receiver<CtrlRequest>,
ctrl_tx: tokio::sync::mpsc::Sender<CtrlRequest>,
clip_event_tx: SyncSender<ClipEventCore>,
clip_cmd_rx: tokio::sync::mpsc::UnboundedReceiver<ClipCommand>,
ready_tx: std::sync::mpsc::Sender<Result<Negotiated>>,
shutdown: Arc<AtomicBool>,
/// Deliberate-quit flag (see [`NativeClient::quit`]): the worker closes with the quit code if set.
@@ -1466,6 +1584,8 @@ async fn worker_main(args: WorkerArgs) {
mut rich_input_rx,
mut ctrl_rx,
ctrl_tx,
clip_event_tx,
clip_cmd_rx,
ready_tx,
shutdown,
quit,
@@ -1630,6 +1750,7 @@ async fn worker_main(args: WorkerArgs) {
audio_channels: welcome.audio_channels,
codec: welcome.codec,
shard_payload: welcome.shard_payload,
host_caps: welcome.host_caps,
},
welcome.host_caps,
))
@@ -1653,7 +1774,7 @@ async fn worker_main(args: WorkerArgs) {
return;
}
};
// Copies the pump needs after `negotiated` is handed over to `connect`.
// Copies the worker needs after `negotiated` is handed over to `connect`.
let clock_rtt_ns = negotiated.clock_rtt_ns;
let resolved_bitrate_kbps = negotiated.bitrate_kbps;
let negotiated_codec = negotiated.codec;
@@ -1830,6 +1951,9 @@ async fn worker_main(args: WorkerArgs) {
let bitrate_ack = bitrate_ack.clone();
let clock_offset = clock_offset.clone();
let clock_gen = clock_gen.clone();
// The control task feeds clipboard metadata events (ClipState/ClipOffer) onto the same event
// plane the clipboard task uses for fetch data; the original tx goes to that task below.
let clip_event_tx = clip_event_tx.clone();
tokio::spawn(async move {
// Mid-stream clock re-sync (see [`ClockResync`]): a batch runs every
// CLOCK_RESYNC_INTERVAL and whenever the pump asks (CtrlRequest::ClockResync after
@@ -1859,6 +1983,8 @@ async fn worker_main(args: WorkerArgs) {
}
resync.begin(wall_clock_ns()).encode()
}
CtrlRequest::ClipControl(c) => c.encode(),
CtrlRequest::ClipOffer(o) => o.encode(),
};
if io::write_msg(&mut ctrl_send, &bytes).await.is_err() {
break;
@@ -1940,8 +2066,27 @@ async fn worker_main(args: WorkerArgs) {
}
ResyncStep::Idle => {}
}
} else if let Ok(state) = ClipState::decode(&msg) {
// Host ack / policy / backend update for the toggle UI (try_send: a
// lagging embedder drops the newest — a stale toggle heals on the next).
let _ = clip_event_tx.try_send(ClipEventCore::State {
enabled: state.enabled,
policy: state.policy,
reason: state.reason,
});
} else if let Ok(offer) = ClipOffer::decode(&msg) {
// The host copied something: surface the lazy format list; the embedder
// fetches only if a local app pastes.
let _ = clip_event_tx.try_send(ClipEventCore::RemoteOffer {
seq: offer.seq,
kinds: offer.kinds,
});
} else {
tracing::warn!("unknown control message — ignoring");
tracing::warn!(
tag = ?msg.first(),
len = msg.len(),
"unknown control message — ignoring"
);
}
}
}
@@ -2016,6 +2161,17 @@ async fn worker_main(args: WorkerArgs) {
}
});
// Clipboard task: the fetch-stream accept loop (host pulls what we offered) + outbound fetches
// (we pull what the host offered). Metadata (enable/offer/state) rides the control task above;
// only bulk bytes flow here. Dies with the connection (accept_bi errors) or when the embedder
// drops the command sender. Always spawned — a host without HOST_CAP_CLIPBOARD simply never
// opens a clip stream, and our control-plane offers hit its "unknown message" arm harmlessly.
tokio::spawn(crate::clipboard::run(
conn.clone(),
clip_event_tx,
clip_cmd_rx,
));
// Watch for connection close → stop the pump.
{
let shutdown = shutdown.clone();
+304
View File
@@ -0,0 +1,304 @@
//! Client-side shared-clipboard transport (`design/clipboard-and-file-transfer.md` §5.1).
//!
//! This is the client-core half of Phase 0: the per-session async task that runs the fetch-stream
//! **accept loop** (so the host can pull data the client offered) and drives **outbound fetches**
//! (so the client can pull what the host offered), surfacing everything to the embedder as
//! poll-style [`ClipEventCore`] events. The metadata plane — enabling sync and announcing offers —
//! rides the control stream as ordinary [`ClipControl`]/[`ClipOffer`] control messages
//! ([`crate::client`] routes those); only the bulk bytes flow through here, over the
//! [`crate::quic::clipstream`] fetch bi-streams.
//!
//! There is intentionally **no OS pasteboard code here** — that lives in the native client (macOS
//! first). The event/command seam is what the C ABI ([`crate::abi`]) exposes so a native client
//! polls offers/fetch-requests and answers with bytes.
use std::collections::HashMap;
use std::sync::mpsc::SyncSender;
use std::sync::{Arc, Mutex};
use tokio::sync::mpsc::UnboundedReceiver;
use tokio::sync::oneshot;
use crate::error::PunktfunkStatus;
use crate::quic::{
clipstream, ClipFetch, ClipFetchHdr, ClipKind, CLIP_FETCH_DENIED, CLIP_FETCH_OK,
CLIP_FETCH_STALE, CLIP_FETCH_UNAVAILABLE,
};
/// Per-fetch requester-side size cap (bytes). A holder that streams more than this is treated as a
/// cap breach and the fetch fails rather than buffering unboundedly (§7). Phase 0 uses one fixed
/// value; a future host-policy `PUNKTFUNK_CLIP_MAX_MB` tightens it per session.
pub const CLIP_FETCH_CAP: usize = 64 << 20;
/// Inbound-serve `req_id`s carry this high bit so they never collide with the client-assigned
/// outbound-fetch `xfer_id`s (which count up from 1). A single [`ClipCommand::Cancel`] `id` can
/// then be routed to the right table.
pub const INBOUND_REQ_FLAG: u32 = 0x8000_0000;
/// Overall stall bound for one outbound fetch (`design/clipboard-and-file-transfer.md` §3.4): a
/// holder that never answers fails the transfer instead of hanging it.
const FETCH_STALL_SECS: u64 = 60;
/// A clipboard event surfaced to the embedder via `NativeClient::next_clip` (→ the C ABI poll
/// `punktfunk_connection_next_clipboard`).
#[derive(Clone, Debug)]
pub enum ClipEventCore {
/// The host announced new clipboard content (the host copied). The embedder decides whether to
/// fetch it — lazily, only when a local app actually pastes.
RemoteOffer { seq: u32, kinds: Vec<ClipKind> },
/// Host ack / unsolicited policy or backend update, for the toggle UI.
State {
enabled: bool,
policy: u8,
reason: u8,
},
/// The host is pasting content the client offered: it opened a fetch stream for
/// `(mime, file_index)`. The embedder must answer with `clip_serve(req_id, …)` (or
/// `clip_cancel(req_id)`).
FetchRequest {
req_id: u32,
seq: u32,
file_index: u32,
mime: String,
},
/// Bytes for a fetch the embedder started (`xfer_id` from `clip_fetch`). Phase 0 delivers the
/// whole payload in one event (`last = true`).
Data {
xfer_id: u32,
bytes: Vec<u8>,
last: bool,
},
/// A transfer was cancelled (by either side).
Cancelled { id: u32 },
/// A transfer failed; `code` is a [`PunktfunkStatus`] value (negative).
Error { id: u32, code: i32 },
}
/// A command from the embedder (via the C ABI) into the clipboard task. `ClipControl`/`ClipOffer`
/// are *not* here — they ride the control stream as ordinary control messages.
pub enum ClipCommand {
/// Open a fetch of the remote's offered content; `xfer_id` is client-assigned and echoed back
/// on the resulting [`ClipEventCore::Data`]/`Error`/`Cancelled`.
Fetch {
xfer_id: u32,
seq: u32,
file_index: u32,
mime: String,
},
/// Provide bytes answering an inbound [`ClipEventCore::FetchRequest`] (`req_id`). Chunks
/// accumulate; `last` completes the transfer.
Serve {
req_id: u32,
bytes: Vec<u8>,
last: bool,
},
/// Cancel a transfer by id — either an outbound fetch (`xfer_id`) or an inbound serve
/// (`req_id`, high bit set).
Cancel { id: u32 },
}
type ServeWaiters = Arc<Mutex<HashMap<u32, oneshot::Sender<Option<Vec<u8>>>>>>;
/// Map a non-OK [`ClipFetchHdr::status`] to the [`PunktfunkStatus`] surfaced on an
/// [`ClipEventCore::Error`].
fn fetch_status_to_code(status: u8) -> i32 {
let s = match status {
CLIP_FETCH_STALE => PunktfunkStatus::NoFrame, // stale offer → "nothing to insert"
CLIP_FETCH_UNAVAILABLE => PunktfunkStatus::Unsupported,
CLIP_FETCH_DENIED => PunktfunkStatus::InvalidArg,
_ => PunktfunkStatus::BadPacket,
};
s as i32
}
/// The per-session clipboard task. Runs until the connection closes or the embedder drops the
/// command sender. It owns no clipboard *content* — the embedder supplies bytes on demand.
pub async fn run(
conn: quinn::Connection,
events: SyncSender<ClipEventCore>,
mut cmd_rx: UnboundedReceiver<ClipCommand>,
) {
// Inbound fetch-serve waiters: req_id -> a oneshot the serving stream task parks on until the
// embedder answers with bytes (`Some`) or denies/cancels (`None`).
let serve_waiters: ServeWaiters = Arc::new(Mutex::new(HashMap::new()));
// Accumulation buffers for chunked `clip_serve` (req_id -> bytes so far).
let mut serve_bufs: HashMap<u32, Vec<u8>> = HashMap::new();
// Outbound fetch cancel triggers: xfer_id -> a oneshot that aborts the fetch task.
let mut fetch_cancels: HashMap<u32, oneshot::Sender<()>> = HashMap::new();
let mut next_req_id: u32 = 1;
loop {
tokio::select! {
// The host opened a fetch bi-stream toward us (it is pasting our offered data).
accepted = conn.accept_bi() => {
let Ok((send, recv)) = accepted else { break }; // connection gone
let req_id = INBOUND_REQ_FLAG | next_req_id;
next_req_id = next_req_id.wrapping_add(1);
if next_req_id == 0 {
next_req_id = 1;
}
let events = events.clone();
let waiters = serve_waiters.clone();
tokio::spawn(serve_inbound(send, recv, req_id, events, waiters));
}
cmd = cmd_rx.recv() => {
let Some(cmd) = cmd else { break }; // NativeClient dropped
match cmd {
ClipCommand::Fetch { xfer_id, seq, file_index, mime } => {
let (cancel_tx, cancel_rx) = oneshot::channel();
fetch_cancels.insert(xfer_id, cancel_tx);
let conn = conn.clone();
let events = events.clone();
let req = ClipFetch { seq, file_index, mime };
tokio::spawn(run_outbound_fetch(conn, xfer_id, req, events, cancel_rx));
}
ClipCommand::Serve { req_id, bytes, last } => {
serve_bufs.entry(req_id).or_default().extend_from_slice(&bytes);
if last {
let full = serve_bufs.remove(&req_id).unwrap_or_default();
if let Some(tx) = serve_waiters.lock().unwrap().remove(&req_id) {
let _ = tx.send(Some(full));
}
}
}
ClipCommand::Cancel { id } => {
// Route to whichever table owns the id (they are disjoint by the high bit).
if let Some(tx) = fetch_cancels.remove(&id) {
let _ = tx.send(());
}
serve_bufs.remove(&id);
if let Some(tx) = serve_waiters.lock().unwrap().remove(&id) {
let _ = tx.send(None); // deny — the serving task writes UNAVAILABLE
}
}
}
}
}
}
}
/// Serve one inbound fetch stream: validate the header + request, emit a [`ClipEventCore::FetchRequest`],
/// then park until the embedder supplies bytes (or denies), and stream them back.
async fn serve_inbound(
mut send: quinn::SendStream,
mut recv: quinn::RecvStream,
req_id: u32,
events: SyncSender<ClipEventCore>,
waiters: ServeWaiters,
) {
let _ = send.set_priority(-1);
let kind = match clipstream::read_stream_header(&mut recv).await {
Ok(k) => k,
Err(_) => return,
};
if kind != clipstream::CLIP_STREAM_KIND_FETCH {
let _ = send.reset(clipstream::cancelled_code());
return;
}
let req = match clipstream::read_fetch(&mut recv).await {
Ok(r) => r,
Err(_) => return,
};
// Register the waiter before emitting the event, so an immediate `clip_serve` can't race ahead
// of the insert.
let (tx, rx) = oneshot::channel();
waiters.lock().unwrap().insert(req_id, tx);
let ev = ClipEventCore::FetchRequest {
req_id,
seq: req.seq,
file_index: req.file_index,
mime: req.mime,
};
if events.try_send(ev).is_err() {
// The embedder isn't draining events (or the session is ending): refuse cleanly.
waiters.lock().unwrap().remove(&req_id);
let _ = clipstream::write_fetch_hdr(
&mut send,
&ClipFetchHdr {
status: CLIP_FETCH_UNAVAILABLE,
total_size: 0,
},
)
.await;
return;
}
match rx.await {
Ok(Some(bytes)) => {
if clipstream::write_fetch_hdr(
&mut send,
&ClipFetchHdr {
status: CLIP_FETCH_OK,
total_size: bytes.len() as u64,
},
)
.await
.is_ok()
{
let _ = clipstream::write_data(&mut send, &bytes).await;
}
}
// Denied, cancelled, or the waiter was dropped (task ending): tell the peer it's gone.
_ => {
let _ = clipstream::write_fetch_hdr(
&mut send,
&ClipFetchHdr {
status: CLIP_FETCH_UNAVAILABLE,
total_size: 0,
},
)
.await;
}
}
}
/// Drive one outbound fetch: open the stream, read the header + data (bounded), and emit a
/// [`ClipEventCore::Data`] / `Error` / `Cancelled`.
async fn run_outbound_fetch(
conn: quinn::Connection,
xfer_id: u32,
req: ClipFetch,
events: SyncSender<ClipEventCore>,
cancel_rx: oneshot::Receiver<()>,
) {
let transfer = async {
let (send, mut recv) = clipstream::open_fetch(&conn, &req)
.await
.map_err(|_| PunktfunkStatus::Io as i32)?;
let hdr = clipstream::read_fetch_hdr(&mut recv)
.await
.map_err(|_| PunktfunkStatus::Io as i32)?;
if hdr.status != CLIP_FETCH_OK {
return Err(fetch_status_to_code(hdr.status));
}
let data = clipstream::read_data(&mut recv, CLIP_FETCH_CAP)
.await
.map_err(|_| PunktfunkStatus::Io as i32)?;
drop(send); // done — dropping the send half is a clean FIN-less close on our side
Ok(data)
};
tokio::select! {
r = transfer => match r {
Ok(data) => {
let _ = events.try_send(ClipEventCore::Data { xfer_id, bytes: data, last: true });
}
Err(code) => {
let _ = events.try_send(ClipEventCore::Error { id: xfer_id, code });
}
},
_ = cancel_rx => {
// The `transfer` future is dropped here; its streams reset on drop.
let _ = events.try_send(ClipEventCore::Cancelled { id: xfer_id });
}
// Overall stall bound (design/clipboard-and-file-transfer.md §3.4): a holder that never
// answers must not hang the transfer. Dropping `transfer` resets the streams.
_ = tokio::time::sleep(std::time::Duration::from_secs(FETCH_STALL_SECS)) => {
let _ = events.try_send(ClipEventCore::Error {
id: xfer_id,
code: PunktfunkStatus::Timeout as i32,
});
}
}
}
+36
View File
@@ -320,6 +320,42 @@ impl InputEvent {
}
}
/// One decoded GameStream (Moonlight-plane) controller event. Shared vocabulary: the host's
/// GameStream/Moonlight decode path produces these, and the platform-neutral input injectors
/// (`pf-inject`) consume them — so the type lives in `core::input`, below both, rather than in
/// either plane. The `buttons` bitmask uses the same [`gamepad`] `BTN_*` layout as the native
/// [`GamepadSnapshot`] (GameStream's `buttonFlags | buttonFlags2 << 16` is bit-identical).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum GamepadEvent {
/// Full state of one controller + the set of attached controllers.
State(GamepadFrame),
/// Sunshine arrival metadata (precedes the first State for that pad).
Arrival {
index: u8,
/// 0 unknown, 1 xbox, 2 ps, 3 nintendo.
kind: u8,
/// LI_CCAP_* bits (0x02 = rumble).
capabilities: u16,
},
}
/// Snapshot of one controller's inputs (Moonlight conventions: sticks 32768..32767 with +Y
/// up, triggers 0..255, buttons = `buttonFlags | buttonFlags2 << 16`). The decoded-frame twin of
/// [`GamepadSnapshot`] on the GameStream/Moonlight plane; see [`GamepadEvent`] for why it lives here.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct GamepadFrame {
pub index: i16,
/// Bit n set = controller n attached; a clear bit for an allocated pad means unplug.
pub active_mask: u16,
pub buttons: u32,
pub left_trigger: u8,
pub right_trigger: u8,
pub ls_x: i16,
pub ls_y: i16,
pub rs_x: i16,
pub rs_y: i16,
}
#[cfg(test)]
mod tests {
use super::*;
+12 -1
View File
@@ -30,6 +30,11 @@ mod abr;
pub mod audio;
#[cfg(feature = "quic")]
pub mod client;
/// Client-side shared-clipboard transport: the per-session task that runs the fetch-stream accept
/// loop, drives outbound fetches, and serves inbound ones — surfaced to the embedder as poll
/// events. Wire codecs live in [`quic`]; the OS pasteboard integration lives in the native client.
#[cfg(feature = "quic")]
pub mod clipboard;
pub mod config;
pub mod crypto;
pub mod error;
@@ -42,6 +47,8 @@ pub mod reanchor;
pub mod reject;
pub mod session;
pub mod stats;
#[cfg(feature = "tls")]
pub mod tls;
pub mod transport;
pub mod wol;
@@ -71,7 +78,11 @@ pub use stats::Stats;
/// application close) and the `PunktfunkStatus` 20 block itself. Additive — the close codes are
/// new application-close vocabulary an old peer simply never sends/reads, so [`WIRE_VERSION`] is
/// unchanged.
pub const ABI_VERSION: u32 = 7;
/// v8: added the shared-clipboard client surface — `punktfunk_connection_host_caps` and
/// `punktfunk_connection_clipboard_{control,offer,fetch,serve,cancel}` +
/// `punktfunk_connection_next_clipboard`. Additive; the wire grows only backward-compatible control
/// messages (0x40-0x44) and a new `Welcome::host_caps` bit, so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 8;
/// The punktfunk/1 **wire** version — what `Hello`/`Welcome` carry and hosts equality-check.
/// Deliberately its own constant: [`ABI_VERSION`] tracks the embeddable **C surface**
@@ -0,0 +1,117 @@
//! Per-transfer clipboard fetch streams (`design/clipboard-and-file-transfer.md` §3.3).
//!
//! Bulk clipboard / file bytes never ride the control stream (u16-capped) or datagrams (lossy,
//! single-packet). The **requester opens a fresh QUIC bi-stream** toward the data holder, writes a
//! small stream header + a [`ClipFetch`]; the holder replies with a [`ClipFetchHdr`] then raw data
//! chunks until FIN. One transfer per stream ⇒ natural flow control, clean cancelation
//! (`RESET_STREAM`), and no head-of-line blocking against control or other transfers.
//!
//! These helpers are the transport half only; they hold no clipboard state, so the host and the
//! client-core reuse the exact same open/accept/serve wire dance (the accept-loop that dispatches
//! by stream kind lives on each side, since the two sides own their connections differently).
use super::{io, ClipFetch, ClipFetchHdr};
/// First bytes an opener writes on a freshly-opened clipboard bi-stream: a magic keeping this
/// stream namespace disjoint from any future stream kind, plus a 1-byte kind discriminator. A
/// distinct magic means a stream opened for some other future purpose can never be misrouted here.
pub const STREAM_MAGIC: &[u8; 4] = b"PKFs";
/// Stream-kind byte: a clipboard fetch (request/response of one format). Future stream kinds
/// (e.g. a bulk file-content push) mux under the same [`STREAM_MAGIC`] with a different byte.
pub const CLIP_STREAM_KIND_FETCH: u8 = 0x01;
/// QUIC application error code used to `reset`/`stop` a clipboard fetch stream on cancel — sync
/// disabled mid-transfer, paste timed out, size cap exceeded, teardown. Distinct from the
/// connection close codes ([`super::QUIT_CLOSE_CODE`] `0x51` / [`super::APP_EXITED_CLOSE_CODE`]
/// `0x52`), the connection reject code `0x42`, and the pairing-rejection close block
/// `0x60``0x67` — stream reset codes and connection close codes are separate QUIC namespaces,
/// but the vocabularies stay disjoint on purpose so a captured code is unambiguous.
pub const CLIP_CANCELLED_CODE: u32 = 0x70;
/// Chunk size for streaming fetch data (64 KiB writes — matches the control-frame bound).
pub const CLIP_CHUNK: usize = 64 * 1024;
/// The `VarInt` form of [`CLIP_CANCELLED_CODE`], for `SendStream::reset` / `RecvStream::stop`.
pub fn cancelled_code() -> quinn::VarInt {
quinn::VarInt::from_u32(CLIP_CANCELLED_CODE)
}
/// Requester side: open a fresh bi-stream toward the holder, deprioritize it under the control
/// stream, write the stream header + the [`ClipFetch`], and hand back both halves. The send half
/// is returned so the caller can `reset`/`finish` for cancelation; the recv half is positioned to
/// read the [`ClipFetchHdr`] next (see [`read_fetch_hdr`]).
pub async fn open_fetch(
conn: &quinn::Connection,
req: &ClipFetch,
) -> std::io::Result<(quinn::SendStream, quinn::RecvStream)> {
let (mut send, recv) = conn.open_bi().await.map_err(std::io::Error::other)?;
// Yield to the control stream (default priority 0) so a large paste never head-of-line-blocks
// the input/audio/control traffic sharing this connection.
let _ = send.set_priority(-1);
// The opener MUST write before the peer's `accept_bi()` can return (quinn contract), so send
// the whole request eagerly.
let mut hdr = Vec::with_capacity(5);
hdr.extend_from_slice(STREAM_MAGIC);
hdr.push(CLIP_STREAM_KIND_FETCH);
send.write_all(&hdr).await.map_err(std::io::Error::other)?;
io::write_msg(&mut send, &req.encode()).await?;
Ok((send, recv))
}
/// Holder side, step 1: after `accept_bi()`, read and validate the 5-byte stream header. Returns
/// the kind byte (e.g. [`CLIP_STREAM_KIND_FETCH`]); an unknown magic is an error and the caller
/// should `stop` the stream.
pub async fn read_stream_header(recv: &mut quinn::RecvStream) -> std::io::Result<u8> {
let mut hdr = [0u8; 5];
recv.read_exact(&mut hdr)
.await
.map_err(std::io::Error::other)?;
if &hdr[0..4] != STREAM_MAGIC {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
"bad clip stream magic",
));
}
Ok(hdr[4])
}
/// Holder side, step 2: read the [`ClipFetch`] request that follows the header.
pub async fn read_fetch(recv: &mut quinn::RecvStream) -> std::io::Result<ClipFetch> {
let raw = io::read_msg(recv).await?;
ClipFetch::decode(&raw)
.map_err(|_| std::io::Error::new(std::io::ErrorKind::InvalidData, "bad ClipFetch"))
}
/// Holder side, step 3: send the response header (before any data chunks).
pub async fn write_fetch_hdr(
send: &mut quinn::SendStream,
hdr: &ClipFetchHdr,
) -> std::io::Result<()> {
io::write_msg(send, &hdr.encode()).await
}
/// Holder side, step 4 (only when the header was [`super::CLIP_FETCH_OK`]): stream `data` as
/// 64 KiB chunks then FIN so the requester's [`read_data`] terminates.
pub async fn write_data(send: &mut quinn::SendStream, data: &[u8]) -> std::io::Result<()> {
for chunk in data.chunks(CLIP_CHUNK) {
send.write_all(chunk).await.map_err(std::io::Error::other)?;
}
send.finish().map_err(std::io::Error::other)?;
Ok(())
}
/// Requester side: read the [`ClipFetchHdr`] the holder sends before any data chunks.
pub async fn read_fetch_hdr(recv: &mut quinn::RecvStream) -> std::io::Result<ClipFetchHdr> {
let raw = io::read_msg(recv).await?;
ClipFetchHdr::decode(&raw)
.map_err(|_| std::io::Error::new(std::io::ErrorKind::InvalidData, "bad ClipFetchHdr"))
}
/// Requester side: after an OK [`ClipFetchHdr`], drain the data chunks to a `Vec`, bounded by
/// `max_bytes` (the requester's size cap — a breach errors, and the caller resets the stream).
pub async fn read_data(recv: &mut quinn::RecvStream, max_bytes: usize) -> std::io::Result<Vec<u8>> {
recv.read_to_end(max_bytes)
.await
.map_err(std::io::Error::other)
}
+10 -77
View File
@@ -1,3 +1,6 @@
//! Shared QUIC endpoint construction (host + client) and transport tuning — keep-alive and idle
//! timeout, plus the certificate-fingerprint helpers. The cert-pinning verifier itself is the
//! one shared [`crate::tls::PinVerify`]; this module only wires it into the client endpoint.
use std::sync::{Arc, Mutex};
/// Shared QUIC transport tuning for BOTH the host and client endpoints. Keep-alive is the
@@ -130,11 +133,10 @@ pub fn peer_fingerprint(conn: &quinn::Connection) -> Option<[u8; 32]> {
certs.first().map(|c| cert_fingerprint(c.as_ref()))
}
/// SHA-256 of a certificate's DER encoding — the fingerprint clients pin.
pub fn cert_fingerprint(cert_der: &[u8]) -> [u8; 32] {
use sha2::Digest;
sha2::Sha256::digest(cert_der).into()
}
/// SHA-256 of a certificate's DER encoding — the fingerprint clients pin. The implementation is
/// [`crate::tls::cert_fingerprint`] (shared with the HTTP clients); re-exported here so callers
/// reaching it as `quic::endpoint::cert_fingerprint` keep working.
pub use crate::tls::cert_fingerprint;
/// Fingerprint of a PEM-encoded certificate (what a host logs/shows for pairing UX —
/// must match what the client's verifier computes from the DER on the wire).
@@ -178,10 +180,10 @@ pub fn client_pinned_with_identity(
let _ = rustls::crypto::ring::default_provider().install_default();
let builder = rustls::ClientConfig::builder()
.dangerous()
.with_custom_certificate_verifier(Arc::new(PinVerify {
.with_custom_certificate_verifier(Arc::new(crate::tls::PinVerify::with_observed(
pin,
observed: observed.clone(),
}));
observed.clone(),
)));
let mut rustls_cfg = match identity {
None => builder.with_no_client_auth(),
Some((cert_pem, key_pem)) => {
@@ -232,9 +234,6 @@ pub mod anyhow_result {
}
}
/// Fingerprint-pinning verifier: trust is the SHA-256 of the host's (self-signed) leaf
/// cert, not a CA chain. With no pin it accepts any cert (TOFU) but still records what
/// it saw, so the embedder can persist the fingerprint and pin it from then on.
/// Server-side client-cert verifier: accept any (self-signed) client certificate but
/// verify the handshake signature for real — possession of the presented cert's key is
/// what makes the post-handshake fingerprint ([`peer_fingerprint`]) meaningful.
@@ -294,69 +293,3 @@ impl rustls::server::danger::ClientCertVerifier for AcceptAnyClientCert {
.supported_schemes()
}
}
#[derive(Debug)]
struct PinVerify {
pin: Option<[u8; 32]>,
observed: Arc<Mutex<Option<[u8; 32]>>>,
}
impl rustls::client::danger::ServerCertVerifier for PinVerify {
fn verify_server_cert(
&self,
end_entity: &rustls::pki_types::CertificateDer<'_>,
_intermediates: &[rustls::pki_types::CertificateDer<'_>],
_server_name: &rustls::pki_types::ServerName<'_>,
_ocsp: &[u8],
_now: rustls::pki_types::UnixTime,
) -> std::result::Result<rustls::client::danger::ServerCertVerified, rustls::Error> {
let fp = cert_fingerprint(end_entity.as_ref());
*self.observed.lock().unwrap() = Some(fp);
if let Some(expected) = self.pin {
if fp != expected {
return Err(rustls::Error::InvalidCertificate(
rustls::CertificateError::ApplicationVerificationFailure,
));
}
}
Ok(rustls::client::danger::ServerCertVerified::assertion())
}
// The handshake signatures MUST be verified for real even though we pin the cert:
// CertificateVerify is what proves the peer *holds the pinned cert's private key* —
// skip it and an active MITM can replay the host's (public) certificate, match the
// pin, and complete the handshake with its own key.
fn verify_tls12_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> std::result::Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::crypto::verify_tls12_signature(
message,
cert,
dss,
&rustls::crypto::ring::default_provider().signature_verification_algorithms,
)
}
fn verify_tls13_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> std::result::Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::crypto::verify_tls13_signature(
message,
cert,
dss,
&rustls::crypto::ring::default_provider().signature_verification_algorithms,
)
}
fn supported_verify_schemes(&self) -> Vec<rustls::SignatureScheme> {
rustls::crypto::ring::default_provider()
.signature_verification_algorithms
.supported_schemes()
}
}
+2
View File
@@ -1,3 +1,5 @@
//! Length-prefixed framing for QUIC control-stream messages: a `u16` length header followed by the
//! payload, bounded at 64 KiB (control messages are tiny).
/// Read one framed message (bounded at 64 KiB — control messages are tiny).
pub async fn read_msg(recv: &mut quinn::RecvStream) -> std::io::Result<Vec<u8>> {
let mut len = [0u8; 2];
+4
View File
@@ -49,6 +49,10 @@ pub mod endpoint;
/// Async framed-message IO over a quinn stream (`u16 LE length || payload`).
pub mod io;
/// Per-transfer clipboard fetch bi-streams (`PKFs` magic + kind byte, then request/response). The
/// transport half of the shared clipboard; wire codecs are in [`msgs`], state lives per side.
pub mod clipstream;
/// SPAKE2 over Ed25519 for the pairing ceremony. The two roles use the asymmetric flow so
/// the identities are ordered; each side binds **both** certificate fingerprints as the
/// SPAKE2 identities, so the derived key only matches when client and host agree on the PIN
+317
View File
@@ -147,6 +147,14 @@ pub use crate::reject::*;
/// button/axis events; toward a host that doesn't set the bit it keeps the legacy events.
pub const HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// [`Welcome::host_caps`] bit: the host has a shared-clipboard service (a working OS backend)
/// **and** its operator policy does not hard-disable it, so the client may offer the clipboard
/// toggle. Absent (an older host, or `PUNKTFUNK_CLIPBOARD` off) ⇒ the client greys the toggle
/// out. Purely additive: nothing clipboard-related happens until a [`ClipControl`]`{ enabled:
/// true }` crosses (see `design/clipboard-and-file-transfer.md` §3.1). Packs into the existing
/// trailing `host_caps` byte — no wire-layout change.
pub const HOST_CAP_CLIPBOARD: u8 = 0x02;
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
/// advertise this.
@@ -527,6 +535,147 @@ pub const MSG_CLOCK_PROBE: u8 = 0x30;
/// Type byte of [`ClockEcho`].
pub const MSG_CLOCK_ECHO: u8 = 0x31;
// ---------------------------------------------------------------------------------------------
// Shared clipboard & file transfer (design/clipboard-and-file-transfer.md §3). The small
// metadata messages ride the control stream (0x40-0x42); the two fetch-stream messages
// (0x43-0x44) travel on a per-transfer bi-stream (see the [`super::clipstream`] helpers), never
// the control stream, so they are never dispatched by the control loops. All are typed
// (`CTL_MAGIC` + type byte), so an older peer hits its "unknown control message" arm and drops
// any it doesn't know — the whole feature is forward-safe.
// ---------------------------------------------------------------------------------------------
/// Type byte of [`ClipControl`] (client → host): enable/disable the shared clipboard for this
/// session. Idempotent; opt-in is enforced here, not just in UI.
pub const MSG_CLIP_CONTROL: u8 = 0x40;
/// Type byte of [`ClipState`] (host → client): ack + unsolicited policy/backend updates.
pub const MSG_CLIP_STATE: u8 = 0x41;
/// Type byte of [`ClipOffer`] (symmetric): the lazy announcement — format list only, no bytes.
pub const MSG_CLIP_OFFER: u8 = 0x42;
/// Type byte of [`ClipFetch`] (requester → holder, **fetch stream only**): pull one format of the
/// current offer.
pub const MSG_CLIP_FETCH: u8 = 0x43;
/// Type byte of [`ClipFetchHdr`] (holder → requester, **fetch stream only**): the fetch response
/// header that precedes the data chunks.
pub const MSG_CLIP_FETCH_HDR: u8 = 0x44;
/// [`ClipControl::flags`] bit: the client permits file kinds to be offered/fetched this session.
/// Absent ⇒ files are filtered out of offers in both directions (text/rich/image only).
pub const CLIP_FLAG_FILES: u8 = 0x01;
/// [`ClipState::policy`] bit: the host permits non-file formats (text/RTF/HTML/image). Always set
/// while enabled unless a future direction limit clears it.
pub const CLIP_POLICY_TEXT: u8 = 0x01;
/// [`ClipState::policy`] bit: the host permits file formats. Cleared by the operator `no-files`
/// / `text-only` policy so the client can grey out "Include files".
pub const CLIP_POLICY_FILES: u8 = 0x02;
/// [`ClipState::reason`]: normal ack, nothing exceptional.
pub const CLIP_REASON_OK: u8 = 0;
/// [`ClipState::reason`]: this session type has no working clipboard backend (e.g. a gamescope
/// session with no data-control global) — the client shows "not supported in this session type".
pub const CLIP_REASON_BACKEND_UNAVAILABLE: u8 = 1;
/// [`ClipState::reason`]: another client took over the single per-desktop clipboard binding; this
/// one was disabled (last `ClipControl{enabled}` wins).
pub const CLIP_REASON_TAKEN_OVER: u8 = 2;
/// [`ClipState::reason`]: the host operator policy (`PUNKTFUNK_CLIPBOARD=off`) disables clipboard.
pub const CLIP_REASON_POLICY_DISABLED: u8 = 3;
/// [`ClipState::reason`]: enabled, but the host policy forbids file transfer (`no-files` /
/// `text-only`) — surfaced so the client greys "Include files" with a footnote.
pub const CLIP_REASON_NO_FILES: u8 = 4;
/// [`ClipFetchHdr::status`]: the requested format is being served; data chunks follow until FIN.
pub const CLIP_FETCH_OK: u8 = 0;
/// [`ClipFetchHdr::status`]: the fetch named a `seq` that is no longer the holder's current offer;
/// the requester degrades the paste to "nothing inserted" rather than wrong data. No chunks follow.
pub const CLIP_FETCH_STALE: u8 = 1;
/// [`ClipFetchHdr::status`]: the format/index is not available (no backend, or it vanished). No
/// chunks follow.
pub const CLIP_FETCH_UNAVAILABLE: u8 = 2;
/// [`ClipFetchHdr::status`]: policy/cap denies this fetch (e.g. a file fetch under `no-files`). No
/// chunks follow.
pub const CLIP_FETCH_DENIED: u8 = 3;
/// Maximum number of [`ClipKind`] entries in one [`ClipOffer`] (resource cap, §7).
pub const CLIP_MAX_KINDS: usize = 16;
/// Maximum length in bytes of a [`ClipKind::mime`] string (resource cap, §7).
pub const CLIP_MAX_MIME: usize = 128;
/// [`ClipFetch::file_index`] sentinel meaning "not a file fetch" (a whole non-file format, or the
/// file *manifest* itself). Real file fetches use `0..n`.
pub const CLIP_FILE_INDEX_NONE: u32 = u32::MAX;
/// One advertised clipboard format inside a [`ClipOffer`] — a portable MIME name plus a size hint.
/// The bytes never ride here; they cross lazily on a fetch stream only when the destination pastes.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ClipKind {
/// Portable wire MIME, e.g. `text/plain;charset=utf-8`, `text/html`, `image/png`,
/// `application/x-punktfunk-files`. Each end maps it to a platform type at fetch time. ≤
/// [`CLIP_MAX_MIME`] bytes; a longer one is rejected on decode.
pub mime: String,
/// Best-effort total size of this format in bytes; `0` = unknown (a streaming provider).
pub size_hint: u64,
}
/// `client → host` ([`MSG_CLIP_CONTROL`]): flip the shared clipboard on/off for this session.
/// Sent when the user toggles the per-host pref and once at session start if it is on. **Nothing
/// clipboard-related happens on either side until an `enabled: true` arrives** — opt-in at the
/// protocol layer.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct ClipControl {
pub enabled: bool,
/// Bitfield of [`CLIP_FLAG_FILES`] (+ reserved bits for future direction limits).
pub flags: u8,
}
/// `host → client` ([`MSG_CLIP_STATE`]): acknowledge a [`ClipControl`] and push unsolicited
/// updates (policy changed, backend lost). The client surfaces `reason`/`policy` in the toggle UI
/// instead of failing silently.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct ClipState {
pub enabled: bool,
/// Bitfield of [`CLIP_POLICY_TEXT`] / [`CLIP_POLICY_FILES`] — what the host currently permits.
pub policy: u8,
/// One of the `CLIP_REASON_*` values explaining `enabled`/`policy`.
pub reason: u8,
}
/// Symmetric ([`MSG_CLIP_OFFER`], either direction): the lazy announcement. Sent when the local
/// clipboard changes; carries the **format list only** (comfortably inside the 64 KiB control
/// frame). A new offer replaces the sender's previous one; `seq` lets the holder reject stale
/// fetches (§3.4). Files are announced as one `application/x-punktfunk-files` kind — the file
/// list itself is fetched lazily, never inlined here.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ClipOffer {
/// Monotonic per sender; newest wins.
pub seq: u32,
/// ≤ [`CLIP_MAX_KINDS`] entries.
pub kinds: Vec<ClipKind>,
}
/// `requester → holder` ([`MSG_CLIP_FETCH`], **fetch stream only**): the first message on a
/// per-transfer bi-stream, naming which format (and, for files, which entry) of `seq` to pull.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ClipFetch {
/// The offer `seq` this fetch is against; the holder answers [`CLIP_FETCH_STALE`] if it is no
/// longer current.
pub seq: u32,
/// File index for a file transfer, or [`CLIP_FILE_INDEX_NONE`] for a non-file format / the
/// file manifest.
pub file_index: u32,
/// The requested wire MIME (≤ [`CLIP_MAX_MIME`] bytes).
pub mime: String,
}
/// `holder → requester` ([`MSG_CLIP_FETCH_HDR`], **fetch stream only**): the response header that
/// precedes the raw data chunks (which run until the stream's FIN). When `status` is anything
/// other than [`CLIP_FETCH_OK`] no chunks follow.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct ClipFetchHdr {
/// One of the `CLIP_FETCH_*` values.
pub status: u8,
/// Total byte count that will follow; `0` = unknown (a streaming provider — FIN ends it).
pub total_size: u64,
}
// ---------------------------------------------------------------------------------------------
// Pairing ceremony (typed control messages): instead of a session Hello, a client may open
// the control stream with PairRequest. The host shows a short PIN out-of-band (log/UI); the
@@ -1293,6 +1442,174 @@ impl ClockEcho {
}
}
/// Append one [`ClipKind`] to `b`: `mime_len u8 || mime bytes || size_hint u64 LE`.
fn put_clip_kind(b: &mut Vec<u8>, k: &ClipKind) {
let mime = k.mime.as_bytes();
let n = mime.len().min(CLIP_MAX_MIME);
b.push(n as u8);
b.extend_from_slice(&mime[..n]);
b.extend_from_slice(&k.size_hint.to_le_bytes());
}
/// Read one [`ClipKind`] at `off`, returning it and the next offset.
fn get_clip_kind(b: &[u8], off: usize) -> Result<(ClipKind, usize)> {
if off >= b.len() {
return Err(PunktfunkError::InvalidArg("truncated ClipKind"));
}
let n = b[off] as usize;
if n > CLIP_MAX_MIME {
return Err(PunktfunkError::InvalidArg("ClipKind mime too long"));
}
let mime_start = off + 1;
let size_start = mime_start + n;
if size_start + 8 > b.len() {
return Err(PunktfunkError::InvalidArg("ClipKind overruns message"));
}
let mime = String::from_utf8_lossy(&b[mime_start..size_start]).into_owned();
let size_hint = u64::from_le_bytes(b[size_start..size_start + 8].try_into().unwrap());
Ok((ClipKind { mime, size_hint }, size_start + 8))
}
impl ClipControl {
pub fn encode(&self) -> Vec<u8> {
// magic[0..4] type[4] enabled[5] flags[6]
let mut b = Vec::with_capacity(7);
b.extend_from_slice(CTL_MAGIC);
b.push(MSG_CLIP_CONTROL);
b.push(self.enabled as u8);
b.push(self.flags);
b
}
pub fn decode(b: &[u8]) -> Result<ClipControl> {
if b.len() != 7 || &b[0..4] != CTL_MAGIC || b[4] != MSG_CLIP_CONTROL {
return Err(PunktfunkError::InvalidArg("bad ClipControl"));
}
Ok(ClipControl {
enabled: b[5] != 0,
flags: b[6],
})
}
}
impl ClipState {
pub fn encode(&self) -> Vec<u8> {
// magic[0..4] type[4] enabled[5] policy[6] reason[7]
let mut b = Vec::with_capacity(8);
b.extend_from_slice(CTL_MAGIC);
b.push(MSG_CLIP_STATE);
b.push(self.enabled as u8);
b.push(self.policy);
b.push(self.reason);
b
}
pub fn decode(b: &[u8]) -> Result<ClipState> {
if b.len() != 8 || &b[0..4] != CTL_MAGIC || b[4] != MSG_CLIP_STATE {
return Err(PunktfunkError::InvalidArg("bad ClipState"));
}
Ok(ClipState {
enabled: b[5] != 0,
policy: b[6],
reason: b[7],
})
}
}
impl ClipOffer {
pub fn encode(&self) -> Vec<u8> {
// magic[0..4] type[4] seq[5..9] count[9] then `count` ClipKinds
let mut b = Vec::with_capacity(10 + self.kinds.len() * 16);
b.extend_from_slice(CTL_MAGIC);
b.push(MSG_CLIP_OFFER);
b.extend_from_slice(&self.seq.to_le_bytes());
let count = self.kinds.len().min(CLIP_MAX_KINDS);
b.push(count as u8);
for k in &self.kinds[..count] {
put_clip_kind(&mut b, k);
}
b
}
pub fn decode(b: &[u8]) -> Result<ClipOffer> {
if b.len() < 10 || &b[0..4] != CTL_MAGIC || b[4] != MSG_CLIP_OFFER {
return Err(PunktfunkError::InvalidArg("bad ClipOffer"));
}
let seq = u32::from_le_bytes(b[5..9].try_into().unwrap());
let count = b[9] as usize;
if count > CLIP_MAX_KINDS {
return Err(PunktfunkError::InvalidArg("ClipOffer too many kinds"));
}
let mut kinds = Vec::with_capacity(count);
let mut off = 10;
for _ in 0..count {
let (k, next) = get_clip_kind(b, off)?;
kinds.push(k);
off = next;
}
if off != b.len() {
return Err(PunktfunkError::InvalidArg("trailing bytes"));
}
Ok(ClipOffer { seq, kinds })
}
}
impl ClipFetch {
pub fn encode(&self) -> Vec<u8> {
// magic[0..4] type[4] seq[5..9] file_index[9..13] mime(len u8 || bytes)[13..]
let mime = self.mime.as_bytes();
let n = mime.len().min(CLIP_MAX_MIME);
let mut b = Vec::with_capacity(14 + n);
b.extend_from_slice(CTL_MAGIC);
b.push(MSG_CLIP_FETCH);
b.extend_from_slice(&self.seq.to_le_bytes());
b.extend_from_slice(&self.file_index.to_le_bytes());
b.push(n as u8);
b.extend_from_slice(&mime[..n]);
b
}
pub fn decode(b: &[u8]) -> Result<ClipFetch> {
if b.len() < 14 || &b[0..4] != CTL_MAGIC || b[4] != MSG_CLIP_FETCH {
return Err(PunktfunkError::InvalidArg("bad ClipFetch"));
}
let seq = u32::from_le_bytes(b[5..9].try_into().unwrap());
let file_index = u32::from_le_bytes(b[9..13].try_into().unwrap());
let n = b[13] as usize;
if n > CLIP_MAX_MIME || b.len() != 14 + n {
return Err(PunktfunkError::InvalidArg("bad ClipFetch mime"));
}
let mime = String::from_utf8_lossy(&b[14..14 + n]).into_owned();
Ok(ClipFetch {
seq,
file_index,
mime,
})
}
}
impl ClipFetchHdr {
pub fn encode(&self) -> Vec<u8> {
// magic[0..4] type[4] status[5] total_size[6..14]
let mut b = Vec::with_capacity(14);
b.extend_from_slice(CTL_MAGIC);
b.push(MSG_CLIP_FETCH_HDR);
b.push(self.status);
b.extend_from_slice(&self.total_size.to_le_bytes());
b
}
pub fn decode(b: &[u8]) -> Result<ClipFetchHdr> {
if b.len() != 14 || &b[0..4] != CTL_MAGIC || b[4] != MSG_CLIP_FETCH_HDR {
return Err(PunktfunkError::InvalidArg("bad ClipFetchHdr"));
}
Ok(ClipFetchHdr {
status: b[5],
total_size: u64::from_le_bytes(b[6..14].try_into().unwrap()),
})
}
}
/// Frame a message for the control stream: `u16 LE length || payload`.
pub fn frame(payload: &[u8]) -> Vec<u8> {
let mut b = Vec::with_capacity(2 + payload.len());
+2
View File
@@ -1,3 +1,5 @@
//! SPAKE2 password-authenticated key exchange for pairing: derive a shared key from the PIN and
//! confirm it against both certificate fingerprints.
use crate::error::{PunktfunkError, Result};
use hmac::{Hmac, Mac};
use spake2::{Ed25519Group, Identity, Password, Spake2};
+387
View File
@@ -1316,3 +1316,390 @@ fn fingerprint_is_sha256_of_der() {
assert_eq!(a, endpoint::cert_fingerprint(b"cert-a"));
assert_ne!(a, endpoint::cert_fingerprint(b"cert-b"));
}
// ---- Shared clipboard control + fetch-stream message codecs (0x40-0x44) -----------------------
#[test]
fn clip_control_roundtrip() {
for (enabled, flags) in [
(true, 0u8),
(false, 0),
(true, CLIP_FLAG_FILES),
(false, 0xFF),
] {
let m = ClipControl { enabled, flags };
assert_eq!(ClipControl::decode(&m.encode()).unwrap(), m);
}
// Disjoint from its host→client sibling (type byte + length) and exact length.
assert!(ClipControl::decode(
&ClipState {
enabled: true,
policy: 0,
reason: 0
}
.encode()
)
.is_err());
let bytes = ClipControl {
enabled: true,
flags: 0,
}
.encode();
assert!(ClipControl::decode(&[bytes.as_slice(), &[0]].concat()).is_err());
assert!(ClipControl::decode(&bytes[..bytes.len() - 1]).is_err());
}
#[test]
fn clip_state_roundtrip() {
let cases = [
ClipState {
enabled: true,
policy: CLIP_POLICY_TEXT | CLIP_POLICY_FILES,
reason: CLIP_REASON_OK,
},
ClipState {
enabled: false,
policy: 0,
reason: CLIP_REASON_BACKEND_UNAVAILABLE,
},
ClipState {
enabled: true,
policy: CLIP_POLICY_TEXT,
reason: CLIP_REASON_NO_FILES,
},
];
for m in cases {
assert_eq!(ClipState::decode(&m.encode()).unwrap(), m);
}
// A ClipControl must not decode as a ClipState (type byte).
assert!(ClipState::decode(
&ClipControl {
enabled: true,
flags: 0
}
.encode()
)
.is_err());
let bytes = cases[0].encode();
assert!(ClipState::decode(&bytes[..bytes.len() - 1]).is_err());
}
#[test]
fn clip_offer_roundtrip() {
// Empty offer, one kind, and a full multi-format offer (text/rich/image/files).
let cases = [
ClipOffer {
seq: 0,
kinds: vec![],
},
ClipOffer {
seq: 1,
kinds: vec![ClipKind {
mime: "text/plain;charset=utf-8".into(),
size_hint: 12,
}],
},
ClipOffer {
seq: u32::MAX,
kinds: vec![
ClipKind {
mime: "text/plain;charset=utf-8".into(),
size_hint: 0,
},
ClipKind {
mime: "text/html".into(),
size_hint: 4096,
},
ClipKind {
mime: "image/png".into(),
size_hint: 1 << 30,
},
ClipKind {
mime: "application/x-punktfunk-files".into(),
size_hint: 5_000_000_000,
},
],
},
];
for m in &cases {
assert_eq!(&ClipOffer::decode(&m.encode()).unwrap(), m);
}
// Trailing bytes are rejected (get_clip_kind consumes exactly to the end).
let mut padded = cases[1].encode();
padded.push(0);
assert!(ClipOffer::decode(&padded).is_err());
// A count byte over the cap is rejected before allocating.
let mut over = cases[0].encode();
over[9] = (CLIP_MAX_KINDS + 1) as u8;
assert!(ClipOffer::decode(&over).is_err());
// Disjoint from a same-family control message.
assert!(ClipOffer::decode(
&ClipControl {
enabled: true,
flags: 0
}
.encode()
)
.is_err());
}
#[test]
fn clip_fetch_roundtrip() {
let cases = [
ClipFetch {
seq: 1,
file_index: CLIP_FILE_INDEX_NONE,
mime: "text/plain;charset=utf-8".into(),
},
ClipFetch {
seq: 7,
file_index: 0,
mime: "application/x-punktfunk-files".into(),
},
ClipFetch {
seq: u32::MAX,
file_index: 41,
mime: String::new(),
},
];
for m in &cases {
assert_eq!(&ClipFetch::decode(&m.encode()).unwrap(), m);
}
// Trailing + truncation both rejected (exact-length mime check).
let bytes = cases[0].encode();
assert!(ClipFetch::decode(&[bytes.as_slice(), &[0]].concat()).is_err());
assert!(ClipFetch::decode(&bytes[..bytes.len() - 1]).is_err());
// A fetch-stream message must not decode as a control-stream offer, and vice-versa.
assert!(ClipOffer::decode(&cases[0].encode()).is_err());
assert!(ClipFetch::decode(
&ClipOffer {
seq: 1,
kinds: vec![]
}
.encode()
)
.is_err());
}
#[test]
fn clip_fetch_hdr_roundtrip() {
for (status, total_size) in [
(CLIP_FETCH_OK, 15u64),
(CLIP_FETCH_STALE, 0),
(CLIP_FETCH_UNAVAILABLE, 0),
(CLIP_FETCH_DENIED, 0),
(CLIP_FETCH_OK, u64::MAX),
] {
let m = ClipFetchHdr { status, total_size };
assert_eq!(ClipFetchHdr::decode(&m.encode()).unwrap(), m);
}
let bytes = ClipFetchHdr {
status: CLIP_FETCH_OK,
total_size: 1,
}
.encode();
assert!(ClipFetchHdr::decode(&[bytes.as_slice(), &[0]].concat()).is_err());
assert!(ClipFetchHdr::decode(&bytes[..bytes.len() - 1]).is_err());
}
#[test]
fn host_cap_clipboard_bit_is_distinct_and_survives_welcome() {
// The new cap packs into the existing trailing host_caps byte with no layout change.
assert_ne!(HOST_CAP_CLIPBOARD, HOST_CAP_GAMEPAD_STATE);
let mut w = Welcome {
abi_version: 1,
udp_port: 1,
mode: Mode {
width: 1920,
height: 1080,
refresh_hz: 60,
},
fec: FecConfig {
scheme: FecScheme::Gf16,
fec_percent: 0,
max_data_per_block: 1024,
},
shard_payload: 1024,
encrypt: false,
key: [0; 16],
salt: [0; 4],
frames: 0,
compositor: CompositorPref::Auto,
gamepad: GamepadPref::Auto,
bitrate_kbps: 0,
bit_depth: 8,
color: ColorInfo::SDR_BT709,
chroma_format: CHROMA_IDC_420,
audio_channels: 2,
codec: CODEC_HEVC,
host_caps: HOST_CAP_GAMEPAD_STATE | HOST_CAP_CLIPBOARD,
};
let got = Welcome::decode(&w.encode()).unwrap();
assert_eq!(got.host_caps & HOST_CAP_CLIPBOARD, HOST_CAP_CLIPBOARD);
assert_eq!(
got.host_caps & HOST_CAP_GAMEPAD_STATE,
HOST_CAP_GAMEPAD_STATE
);
// Clipboard-off host: the bit is clear, gamepad bit still set.
w.host_caps = HOST_CAP_GAMEPAD_STATE;
assert_eq!(
Welcome::decode(&w.encode()).unwrap().host_caps & HOST_CAP_CLIPBOARD,
0
);
}
// ---- In-process QUIC loopback: the real clipstream fetch transport, both success and cancel ----
mod clip_loopback {
use super::*;
use crate::quic::clipstream;
/// Stand up two loopback quinn endpoints, connect, and return
/// `(server_ep, client_ep, host_conn, client_conn)`. Both endpoints are returned so the caller
/// keeps them in scope — dropping a `quinn::Endpoint` tears down its connections.
async fn connect_pair() -> (
quinn::Endpoint,
quinn::Endpoint,
quinn::Connection,
quinn::Connection,
) {
let server = endpoint::server("127.0.0.1:0".parse().unwrap()).unwrap();
let addr = server.local_addr().unwrap();
let client = endpoint::client_insecure().unwrap();
let accept = tokio::spawn(async move {
let incoming = server.accept().await.expect("incoming connection");
let conn = incoming.await.expect("host side connects");
(server, conn)
});
let client_conn = client
.connect(addr, "punktfunk")
.unwrap()
.await
.expect("client side connects");
let (server, host_conn) = accept.await.unwrap();
(server, client, host_conn, client_conn)
}
#[tokio::test]
async fn fetch_text_transfers_then_cancel_resets() {
let (_server_ep, _client_ep, host_conn, client_conn) = connect_pair().await;
let payload = b"hello clipboard \xf0\x9f\x93\x8b".to_vec(); // text + a 4-byte emoji
let holder_payload = payload.clone();
// Holder = the host side: accept two fetch streams. Serve the first; cancel the second.
let holder = tokio::spawn(async move {
// Fetch #1 — serve the payload.
let (mut send, mut recv) = host_conn.accept_bi().await.expect("accept fetch #1");
let kind = clipstream::read_stream_header(&mut recv)
.await
.expect("stream header #1");
assert_eq!(kind, clipstream::CLIP_STREAM_KIND_FETCH);
let req = clipstream::read_fetch(&mut recv)
.await
.expect("fetch req #1");
assert_eq!(req.seq, 1);
assert_eq!(req.file_index, CLIP_FILE_INDEX_NONE);
assert_eq!(req.mime, "text/plain;charset=utf-8");
clipstream::write_fetch_hdr(
&mut send,
&ClipFetchHdr {
status: CLIP_FETCH_OK,
total_size: holder_payload.len() as u64,
},
)
.await
.expect("write hdr #1");
clipstream::write_data(&mut send, &holder_payload)
.await
.expect("write data #1");
// Fetch #2 — read the request, then cancel mid-transfer with RESET_STREAM.
let (mut send2, mut recv2) = host_conn.accept_bi().await.expect("accept fetch #2");
clipstream::read_stream_header(&mut recv2)
.await
.expect("stream header #2");
let _ = clipstream::read_fetch(&mut recv2)
.await
.expect("fetch req #2");
send2.reset(clipstream::cancelled_code()).unwrap();
host_conn // keep alive until the requester side is done
});
// Requester = the client side.
// #1: full lazy fetch of the text payload.
let req = ClipFetch {
seq: 1,
file_index: CLIP_FILE_INDEX_NONE,
mime: "text/plain;charset=utf-8".into(),
};
let (_send, mut recv) = clipstream::open_fetch(&client_conn, &req)
.await
.expect("open fetch #1");
let hdr = clipstream::read_fetch_hdr(&mut recv)
.await
.expect("read hdr #1");
assert_eq!(hdr.status, CLIP_FETCH_OK);
assert_eq!(hdr.total_size as usize, payload.len());
let got = clipstream::read_data(&mut recv, 8 << 20)
.await
.expect("read data #1");
assert_eq!(got, payload);
// #2: the holder resets the stream — the requester surfaces an error rather than hanging.
let req2 = ClipFetch {
seq: 2,
file_index: CLIP_FILE_INDEX_NONE,
mime: "text/plain;charset=utf-8".into(),
};
let (_send2, mut recv2) = clipstream::open_fetch(&client_conn, &req2)
.await
.expect("open fetch #2");
assert!(
clipstream::read_fetch_hdr(&mut recv2).await.is_err(),
"a cancelled fetch must surface as an error, not a hang"
);
let _host_conn = holder.await.unwrap();
}
#[tokio::test]
async fn read_data_enforces_size_cap() {
let (_server_ep, _client_ep, host_conn, client_conn) = connect_pair().await;
let big = vec![0xABu8; 200_000]; // > the 64 KiB chunk, and > the cap we set below
let holder_payload = big.clone();
let holder = tokio::spawn(async move {
let (mut send, mut recv) = host_conn.accept_bi().await.expect("accept");
clipstream::read_stream_header(&mut recv).await.unwrap();
let _ = clipstream::read_fetch(&mut recv).await.unwrap();
clipstream::write_fetch_hdr(
&mut send,
&ClipFetchHdr {
status: CLIP_FETCH_OK,
total_size: holder_payload.len() as u64,
},
)
.await
.unwrap();
let _ = clipstream::write_data(&mut send, &holder_payload).await;
host_conn
});
let req = ClipFetch {
seq: 1,
file_index: CLIP_FILE_INDEX_NONE,
mime: "application/octet-stream".into(),
};
let (_send, mut recv) = clipstream::open_fetch(&client_conn, &req).await.unwrap();
assert_eq!(
clipstream::read_fetch_hdr(&mut recv).await.unwrap().status,
CLIP_FETCH_OK
);
// Cap below the payload size ⇒ read_data errors instead of buffering unboundedly.
assert!(clipstream::read_data(&mut recv, 64 * 1024).await.is_err());
let _host_conn = holder.await.unwrap();
}
}
+164
View File
@@ -0,0 +1,164 @@
//! Shared TLS trust primitives for the punktfunk clients: the certificate-fingerprint hash and
//! the one canonical fingerprint-pinning [`ServerCertVerifier`](rustls::client::danger::ServerCertVerifier)
//! (`PinVerify`). Trust across the whole system is the SHA-256 of the host's self-signed leaf cert
//! (TOFU-pinned), not a CA chain — and this verifier is what the QUIC connect, the game-library
//! HTTP client, and the tray status poll all share, instead of hand-rolling it three times on a
//! trust boundary. Behind the light `tls` feature (rustls + sha2 only — no QUIC runtime), which
//! the heavier `quic` feature pulls in.
use std::sync::{Arc, Mutex};
/// SHA-256 of a certificate's DER encoding — the fingerprint clients pin. Re-exported as
/// `crate::quic::endpoint::cert_fingerprint` for callers that already reach it there.
pub fn cert_fingerprint(cert_der: &[u8]) -> [u8; 32] {
use sha2::Digest;
sha2::Sha256::digest(cert_der).into()
}
/// Fingerprint-pinning verifier: trust is the SHA-256 of the host's (self-signed) leaf cert,
/// not a CA chain.
///
/// - `pin = Some(sha256)` rejects any host whose leaf doesn't hash to `sha256`.
/// - `pin = None` accepts any leaf (trust-on-first-use) — pair with [`with_observed`](Self::with_observed)
/// to record what was seen so the embedder can persist it and pin it from then on.
///
/// The handshake signatures are ALWAYS verified for real even though the cert is pinned:
/// `CertificateVerify` is what proves the peer *holds the pinned cert's private key* — skip it and
/// an active MITM can replay the host's (public) certificate, match the pin, and complete the
/// handshake with its own key.
#[derive(Debug)]
pub struct PinVerify {
pin: Option<[u8; 32]>,
observed: Option<Arc<Mutex<Option<[u8; 32]>>>>,
}
impl PinVerify {
/// A verifier that pins `pin` (or accepts any when `None`) without recording what it saw —
/// the HTTP clients, which connect with a known pin or accept-any and never need to persist
/// the observed fingerprint.
pub fn new(pin: Option<[u8; 32]>) -> Self {
Self {
pin,
observed: None,
}
}
/// Like [`new`](Self::new) but also writes the observed leaf fingerprint into `slot` during
/// the handshake, so a trust-on-first-use caller (the QUIC connect) can read it off the slot
/// and pin it on the next connect.
pub fn with_observed(pin: Option<[u8; 32]>, slot: Arc<Mutex<Option<[u8; 32]>>>) -> Self {
Self {
pin,
observed: Some(slot),
}
}
}
impl rustls::client::danger::ServerCertVerifier for PinVerify {
fn verify_server_cert(
&self,
end_entity: &rustls::pki_types::CertificateDer<'_>,
_intermediates: &[rustls::pki_types::CertificateDer<'_>],
_server_name: &rustls::pki_types::ServerName<'_>,
_ocsp: &[u8],
_now: rustls::pki_types::UnixTime,
) -> std::result::Result<rustls::client::danger::ServerCertVerified, rustls::Error> {
// Only hash the leaf when the result depends on it: a pin to check and/or a slot to
// record into. Accept-any-without-recording (an un-pinned HTTP agent) skips it.
if self.pin.is_some() || self.observed.is_some() {
let fp = cert_fingerprint(end_entity.as_ref());
if let Some(slot) = &self.observed {
*slot.lock().unwrap() = Some(fp);
}
if let Some(expected) = self.pin {
if fp != expected {
return Err(rustls::Error::InvalidCertificate(
rustls::CertificateError::ApplicationVerificationFailure,
));
}
}
}
Ok(rustls::client::danger::ServerCertVerified::assertion())
}
fn verify_tls12_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> std::result::Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::crypto::verify_tls12_signature(
message,
cert,
dss,
&rustls::crypto::ring::default_provider().signature_verification_algorithms,
)
}
fn verify_tls13_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> std::result::Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::crypto::verify_tls13_signature(
message,
cert,
dss,
&rustls::crypto::ring::default_provider().signature_verification_algorithms,
)
}
fn supported_verify_schemes(&self) -> Vec<rustls::SignatureScheme> {
rustls::crypto::ring::default_provider()
.signature_verification_algorithms
.supported_schemes()
}
}
#[cfg(test)]
mod tests {
use super::*;
use rustls::client::danger::ServerCertVerifier;
use rustls::pki_types::{CertificateDer, ServerName, UnixTime};
/// Drive the pin check against `cert_bytes`. `verify_server_cert` only hashes the leaf, so
/// arbitrary bytes stand in for a DER certificate here.
fn verify(v: &PinVerify, cert_bytes: &[u8]) -> std::result::Result<(), rustls::Error> {
let der = CertificateDer::from(cert_bytes.to_vec());
let name = ServerName::try_from("punktfunk").unwrap();
v.verify_server_cert(
&der,
&[],
&name,
&[],
UnixTime::since_unix_epoch(std::time::Duration::ZERO),
)
.map(|_| ())
}
#[test]
fn matching_pin_accepts_and_mismatch_is_rejected() {
let cert = b"the-host-leaf-cert";
let good = cert_fingerprint(cert);
assert!(verify(&PinVerify::new(Some(good)), cert).is_ok());
let mut wrong = good;
wrong[0] ^= 0xff;
match verify(&PinVerify::new(Some(wrong)), cert) {
Err(rustls::Error::InvalidCertificate(
rustls::CertificateError::ApplicationVerificationFailure,
)) => {}
other => panic!("a pin mismatch must be rejected, got {other:?}"),
}
}
#[test]
fn no_pin_accepts_any_and_records_the_observed_fingerprint() {
let cert = b"whatever-the-host-presents";
let slot = Arc::new(Mutex::new(None));
let v = PinVerify::with_observed(None, slot.clone());
assert!(verify(&v, cert).is_ok());
assert_eq!(*slot.lock().unwrap(), Some(cert_fingerprint(cert)));
}
}
+1 -1
View File
@@ -26,7 +26,7 @@ use std::sync::atomic::{AtomicBool, Ordering};
/// Sized for 1 Gbps+: at ~1.2 Gbps on the wire an 8 MB buffer is only ~49 ms of steady state, and a
/// single multi-MB IDR keyframe (~4 MB ≈ 3300 packets) instantly fills most of it. 32 MB gives ~200 ms
/// of headroom and absorbs a keyframe burst without EAGAIN/ENOBUFS drops. (Paced sending —
/// `punktfunk1.rs::paced_submit` — spreads a big frame's overflow, so this buffer mostly absorbs the
/// `native.rs::paced_submit` — spreads a big frame's overflow, so this buffer mostly absorbs the
/// immediate microburst rather than a whole unpaced frame.)
pub(crate) const TARGET_SOCKBUF: usize = 32 * 1024 * 1024;
+2 -3
View File
@@ -4,7 +4,7 @@
//! ([`Transport::recv_batch`], ≤128/syscall into a reused ring) on Linux AND Android (which is
//! `target_os = "android"`, not `"linux"` — it needs its own bionic binding, see [`android_mmsg`])
//! — the 1 Gbps+ syscall lever (~125k → a few-k syscalls/sec at line rate). The host additionally
//! paces each frame's send across the frame interval (see `punktfunk1.rs::paced_submit`) so a real
//! paces each frame's send across the frame interval (see `native.rs::paced_submit`) so a real
//! NIC doesn't drop a line-rate burst. All three layer on this same [`Transport`] seam (scalar
//! fallbacks for loopback and the remaining targets).
@@ -230,8 +230,7 @@ mod uso {
STATE.store(if off { 2 } else { 1 }, Ordering::Relaxed);
tracing::info!(
enabled = !off,
"Windows UDP Send Offload (USO): {} (the 1 Gbps+ send lever; PUNKTFUNK_GSO=0 disables)",
if off { "off" } else { "on" }
"Windows UDP Send Offload (USO) resolved (the 1 Gbps+ send lever; PUNKTFUNK_GSO=0 disables)"
);
!off
}
+13
View File
@@ -10,6 +10,10 @@ repository.workspace = true
[dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
# Shared clipboard (design/clipboard-and-file-transfer.md §4): the per-OS session-clipboard
# backends + coordinator. Portable dep — the crate's facade (policy / ClipCoordCmd / start)
# compiles everywhere; the backends are cfg-gated inside it.
pf-clipboard = { path = "../pf-clipboard" }
# M3 native control plane (the `punktfunk/1` QUIC handshake; data plane stays native-thread UDP).
quinn = "0.11"
anyhow = "1"
@@ -56,6 +60,12 @@ tokio-rustls = "0.26"
hyper = { version = "1", features = ["server", "http1", "http2"] }
hyper-util = { version = "0.1", features = ["server", "server-auto", "tokio", "service"] }
tower = { version = "0.5", features = ["util"] }
# Stream combinators for the mgmt API's SSE event feed (`GET /api/v1/events`) — already in the
# tree transitively (axum/hyper) and as a Linux target dep; control plane only.
futures-util = "0.3"
# Webhook signing (X-Punktfunk-Signature: sha256=<hex HMAC>) for operator hooks; pairs with
# the existing sha2. Already in the lockfile transitively.
hmac = "0.12"
rusty_enet = "0.4"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
@@ -217,6 +227,9 @@ windows = { version = "0.62", features = [
# (src/windows/crash.rs); Kernel gates the CONTEXT type EXCEPTION_POINTERS embeds.
"Win32_System_Diagnostics_Debug",
"Win32_System_Kernel",
# CreateToolhelp32Snapshot/Process32*W — the conflicting-streaming-host process scan
# (src/detect/windows.rs): is Sunshine/Apollo/... running alongside us?
"Win32_System_Diagnostics_ToolHelp",
] }
# The SCM plumbing for the `service` subcommand (define_windows_service! / dispatcher / control
# handler / ServiceManager install). Wraps the Win32 service API; the supervision loop itself uses
+1 -1
View File
@@ -77,7 +77,7 @@ src/
inject/ · inject.rs input backends (libei · wlr · uinput gamepads · UHID DualSense/DS4)
audio/ · audio.rs Opus out + virtual mic (PipeWire / WASAPI)
gamestream/ Moonlight compat: nvhttp · pairing · rtsp · control · stream · gamepad · apps
punktfunk1.rs the native punktfunk/1 host (QUIC control + native-thread UDP data plane)
native.rs the native punktfunk/1 host (QUIC control + native-thread UDP data plane)
mgmt.rs · native_pairing.rs · stats_recorder.rs management API, pairing, perf capture
hdr.rs · library.rs HDR metadata; multi-store game library
linux/ · windows/ platform-confined backends
+2 -422
View File
@@ -106,220 +106,6 @@ pub fn open_virtual_mic(_channels: u32) -> Result<Box<dyn VirtualMic>> {
anyhow::bail!("virtual mic requires Linux + PipeWire or Windows + a virtual audio device")
}
/// Mic is 48 kHz stereo — matches the Opus stereo decoder and the host→client audio layout.
pub const MIC_CHANNELS: u32 = 2;
/// Bound for the shared mic frame queue (drop-newest when full): the host-lifetime queue is
/// shared across all concurrent sessions and must not grow without limit under a near-line-rate
/// flood (security-review 2026-06-28 S6). 64 × 520 ms frames ≈ 0.31.3 s of slack.
const MIC_QUEUE_CAP: usize = 64;
/// Tuning for [`MicPump`]'s open/reopen/flush behaviour — parameterized so the tests can run the
/// real pump loop in milliseconds instead of seconds.
#[derive(Clone, Copy)]
struct PumpTuning {
/// First-retry delay after a failed backend open; doubles per failure up to `backoff_cap`
/// (a persistently-absent PipeWire session / audio endpoint isn't hammered), resets on
/// success.
backoff_start: std::time::Duration,
backoff_cap: std::time::Duration,
/// Idle liveness-probe interval: with no frames flowing, the pump still notices a dead
/// backend this often and reopens — so the mic is healthy BEFORE the next session starts.
heartbeat: std::time::Duration,
/// An uplink gap longer than this discards the backend's buffered audio before pushing the
/// next frame (a recorder must never hear a stale burst from before a mute/session end).
stale_gap: std::time::Duration,
/// A backend that dies before living this long counts as a FAILED open for backoff purposes
/// (an open that succeeds but dies instantly — e.g. a flapping daemon — must not churn at
/// heartbeat rate); one that lived longer resets the backoff.
stable_after: std::time::Duration,
}
const PUMP_TUNING: PumpTuning = PumpTuning {
backoff_start: std::time::Duration::from_secs(2),
backoff_cap: std::time::Duration::from_secs(60),
heartbeat: std::time::Duration::from_secs(1),
stale_gap: std::time::Duration::from_millis(600),
stable_after: std::time::Duration::from_secs(5),
};
/// Host-lifetime virtual-microphone pump: one thread owns the [`VirtualMic`] backend + an Opus
/// decoder; sessions forward the client's Opus mic frames (0xCB) over a clonable `Send` sender,
/// the thread decodes and feeds the backend.
///
/// The rock-solid properties live HERE, not in the backends:
/// - **Eager**: the backend opens at host start (retrying with backoff), NOT on the first mic
/// frame — so the virtual mic device already exists when host apps/games launch and bind
/// their capture device (most games never re-follow a default-device change mid-run).
/// - **Self-healing**: a dead backend (PipeWire restart, Windows endpoint churn) is detected on
/// every push and on an idle heartbeat, and reopened with backoff. Sessions keep their
/// senders; nothing upstream notices.
/// - **Stale-flush**: buffered audio is discarded after an uplink gap (see [`PumpTuning`]).
///
/// Per-frame Opus DECODE errors stay non-fatal (dropped frame): the mic is shared across every
/// concurrent session, so one paired client's junk frames must not deny everyone's mic
/// (security-review 2026-06-28 S2). The thread exits when every sender is dropped (host
/// shutdown), tearing the backend down.
pub struct MicPump {
tx: std::sync::mpsc::SyncSender<Vec<u8>>,
}
impl MicPump {
/// Start the host-lifetime pump (Linux/Windows). On platforms without a virtual-mic backend
/// the thread just drains and drops frames (sessions still count the datagrams).
pub fn start() -> MicPump {
let (tx, rx) = std::sync::mpsc::sync_channel::<Vec<u8>>(MIC_QUEUE_CAP);
let spawned = std::thread::Builder::new()
.name("punktfunk-mic-pump".into())
.spawn(move || {
#[cfg(any(target_os = "linux", target_os = "windows"))]
pump_thread(rx, || open_virtual_mic(MIC_CHANNELS), PUMP_TUNING);
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
{
tracing::warn!("mic passthrough unsupported on this platform — frames dropped");
for _ in rx {}
}
});
if let Err(e) = spawned {
tracing::error!(error = %e, "mic pump thread spawn failed — mic passthrough disabled");
}
MicPump { tx }
}
/// A sender a session forwards the client's Opus mic frames to (`try_send` — never block a
/// datagram loop). Cloned per session; dropping a clone does NOT stop the pump (it holds
/// the original sender for the host life).
pub fn sender(&self) -> std::sync::mpsc::SyncSender<Vec<u8>> {
self.tx.clone()
}
}
/// Sleep for `dur` while draining (and dropping) queued frames, so a closed/reopening backend
/// never accumulates a stale backlog and senders never see a wedged queue. Returns `false` when
/// every sender is gone (host shutdown).
#[cfg_attr(not(any(target_os = "linux", target_os = "windows")), allow(dead_code))]
fn drain_sleep(rx: &std::sync::mpsc::Receiver<Vec<u8>>, dur: std::time::Duration) -> bool {
use std::sync::mpsc::RecvTimeoutError;
let deadline = std::time::Instant::now() + dur;
loop {
let left = deadline.saturating_duration_since(std::time::Instant::now());
if left.is_zero() {
return true;
}
match rx.recv_timeout(left.min(std::time::Duration::from_millis(250))) {
Ok(_) => {} // drop frames while closed
Err(RecvTimeoutError::Timeout) => {} // keep waiting
Err(RecvTimeoutError::Disconnected) => return false, // host shutdown
}
}
}
/// The pump loop. `opener` is injected so the tests can run the REAL loop against a mock
/// backend; production passes [`open_virtual_mic`].
#[cfg_attr(not(any(target_os = "linux", target_os = "windows")), allow(dead_code))]
fn pump_thread<O>(rx: std::sync::mpsc::Receiver<Vec<u8>>, opener: O, tuning: PumpTuning)
where
O: Fn() -> Result<Box<dyn VirtualMic>>,
{
use std::sync::mpsc::RecvTimeoutError;
use std::time::Instant;
let mut backoff = tuning.backoff_start;
let mut open_fails: u64 = 0;
loop {
// Open phase — eager, from thread start.
let (mic, mut decoder) = loop {
let opened = opener().and_then(|m| {
let d = opus::Decoder::new(SAMPLE_RATE, opus::Channels::Stereo)
.map_err(|e| anyhow::anyhow!("opus decoder: {e}"))?;
Ok((m, d))
});
match opened {
Ok(pair) => break pair,
Err(e) => {
// Throttle (1st, 2nd, 4th, 8th … failure): a box without a PipeWire session
// or virtual audio device would otherwise log every backoff forever.
open_fails += 1;
if open_fails.is_power_of_two() {
tracing::warn!(error = %format!("{e:#}"), attempts = open_fails,
"virtual mic unavailable — retrying with backoff");
}
if !drain_sleep(&rx, backoff) {
return;
}
backoff = (backoff * 2).min(tuning.backoff_cap);
}
}
};
tracing::info!("virtual mic ready (host-lifetime)");
// Drop anything queued while (re)opening — it predates the backend. (The backoff does
// NOT reset here: only an instance that proves stable resets it — see the death triage.)
while rx.try_recv().is_ok() {}
let opened_at = Instant::now();
// Pump phase — runs until the backend dies (break) or the host shuts down (return).
let mut decode_fails: u64 = 0;
let mut pcm = vec![0f32; 5760 * MIC_CHANNELS as usize]; // up to 120 ms scratch
let mut last_push = Instant::now();
loop {
match rx.recv_timeout(tuning.heartbeat) {
Ok(frame) => {
if frame.is_empty() {
continue; // DTX silence — the source underruns to silence on its own
}
if last_push.elapsed() > tuning.stale_gap {
mic.discard();
}
match decoder.decode_float(&frame, &mut pcm, false) {
Ok(samples_per_ch) => {
let total = (samples_per_ch * MIC_CHANNELS as usize).min(pcm.len());
if !mic.push(&pcm[..total]) {
tracing::warn!("virtual mic backend died — reopening");
break;
}
last_push = Instant::now();
decode_fails = 0;
}
Err(e) => {
// Malformed/garbage frame: drop it, keep the shared mic + decoder
// (see the struct docs). Throttled log (1, 2, 4, … fails).
decode_fails += 1;
if decode_fails.is_power_of_two() {
tracing::warn!(error = %e, fails = decode_fails,
"mic opus decode failed — dropping frame");
}
}
}
}
Err(RecvTimeoutError::Timeout) => {
if !mic.alive() {
tracing::warn!("virtual mic backend died while idle — reopening");
break;
}
}
Err(RecvTimeoutError::Disconnected) => {
tracing::debug!("mic pump stopped (host shutting down)");
return;
}
}
}
// Death triage: an instance that lived is a one-off (PipeWire/audio-engine restart) —
// reopen immediately with the backoff reset. One that died right after opening is a
// failed open in disguise (flapping daemon, endpoint racing away): back off like the
// open loop, or the pump would churn open→die→reopen at heartbeat rate.
if opened_at.elapsed() >= tuning.stable_after {
backoff = tuning.backoff_start;
open_fails = 0;
} else {
open_fails += 1;
if !drain_sleep(&rx, backoff) {
return;
}
backoff = (backoff * 2).min(tuning.backoff_cap);
}
}
}
#[cfg(target_os = "windows")]
#[path = "audio/windows/audio_control.rs"]
mod audio_control;
@@ -335,211 +121,5 @@ mod wasapi_mic;
#[path = "audio/wiring_plan.rs"]
pub(crate) mod wiring_plan;
#[cfg(test)]
mod pump_tests {
use super::*;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;
/// Mock backend: records pushes/discards, dies on command.
struct MockMic {
alive: Arc<AtomicBool>,
pushed: Arc<AtomicUsize>,
discards: Arc<AtomicUsize>,
}
impl VirtualMic for MockMic {
fn push(&self, pcm: &[f32]) -> bool {
if !self.alive.load(Ordering::Acquire) {
return false;
}
self.pushed.fetch_add(pcm.len(), Ordering::Relaxed);
true
}
fn alive(&self) -> bool {
self.alive.load(Ordering::Acquire)
}
fn discard(&self) {
self.discards.fetch_add(1, Ordering::Relaxed);
}
}
struct Harness {
tx: std::sync::mpsc::SyncSender<Vec<u8>>,
opens: Arc<AtomicUsize>,
alive: Arc<Mutex<Option<Arc<AtomicBool>>>>, // latest instance's kill switch
pushed: Arc<AtomicUsize>,
discards: Arc<AtomicUsize>,
join: std::thread::JoinHandle<()>,
}
/// Run the REAL pump loop against mock backends; `fail_first` opens fail before the first
/// success (exercises the eager retry/backoff path). `dead_on_arrival` opens every instance
/// pre-killed (exercises the rapid-death churn guard). `stable_after` mirrors the tuning
/// field (ZERO = every death counts as stable → immediate reopen, keeping tests fast).
fn start_tuned(fail_first: usize, dead_on_arrival: bool, stable_after: Duration) -> Harness {
let (tx, rx) = std::sync::mpsc::sync_channel::<Vec<u8>>(MIC_QUEUE_CAP);
let opens = Arc::new(AtomicUsize::new(0));
let alive = Arc::new(Mutex::new(None::<Arc<AtomicBool>>));
let pushed = Arc::new(AtomicUsize::new(0));
let discards = Arc::new(AtomicUsize::new(0));
let (opens2, alive2, pushed2, discards2) = (
opens.clone(),
alive.clone(),
pushed.clone(),
discards.clone(),
);
let tuning = PumpTuning {
backoff_start: Duration::from_millis(10),
backoff_cap: Duration::from_millis(40),
heartbeat: Duration::from_millis(20),
stale_gap: Duration::from_millis(80),
stable_after,
};
let join = std::thread::spawn(move || {
pump_thread(
rx,
move || {
let n = opens2.fetch_add(1, Ordering::SeqCst);
if n < fail_first {
anyhow::bail!("backend not up yet (simulated)");
}
let a = Arc::new(AtomicBool::new(!dead_on_arrival));
*alive2.lock().unwrap() = Some(a.clone());
Ok(Box::new(MockMic {
alive: a,
pushed: pushed2.clone(),
discards: discards2.clone(),
}) as Box<dyn VirtualMic>)
},
tuning,
)
});
Harness {
tx,
opens,
alive,
pushed,
discards,
join,
}
}
fn start(fail_first: usize) -> Harness {
start_tuned(fail_first, false, Duration::ZERO)
}
fn wait_until(what: &str, mut cond: impl FnMut() -> bool) {
for _ in 0..200 {
if cond() {
return;
}
std::thread::sleep(Duration::from_millis(10));
}
panic!("timed out waiting for: {what}");
}
fn opus_frame() -> Vec<u8> {
let mut enc = opus::Encoder::new(48_000, opus::Channels::Stereo, opus::Application::Voip)
.expect("opus encoder");
let pcm = [0.1f32; 960 * 2]; // 20 ms stereo
let mut out = vec![0u8; 4000];
let n = enc.encode_float(&pcm, &mut out).expect("encode");
out.truncate(n);
out
}
/// Eager: the backend opens (after transient failures) with NO frame ever sent.
#[test]
fn opens_eagerly_with_backoff() {
let h = start(3);
wait_until("eager open after 3 failures", || {
h.opens.load(Ordering::SeqCst) >= 4 && h.alive.lock().unwrap().is_some()
});
drop(h.tx);
h.join.join().unwrap();
}
/// Frames flow: opus in → PCM pushed to the backend.
#[test]
fn decodes_and_pushes() {
let h = start(0);
wait_until("open", || h.alive.lock().unwrap().is_some());
h.tx.send(opus_frame()).unwrap();
wait_until("pcm pushed", || h.pushed.load(Ordering::SeqCst) > 0);
drop(h.tx);
h.join.join().unwrap();
}
/// A dead backend is noticed WHILE IDLE (heartbeat) and reopened without any traffic.
#[test]
fn reopens_after_idle_death() {
let h = start(0);
wait_until("first open", || h.opens.load(Ordering::SeqCst) >= 1);
wait_until("instance", || h.alive.lock().unwrap().is_some());
h.alive
.lock()
.unwrap()
.as_ref()
.unwrap()
.store(false, Ordering::Release); // kill it
wait_until("reopen after idle death", || {
h.opens.load(Ordering::SeqCst) >= 2
});
drop(h.tx);
h.join.join().unwrap();
}
/// A death detected on push (frame flowing) also reopens, and the frame after reopen flows.
#[test]
fn reopens_after_push_death() {
let h = start(0);
wait_until("instance", || h.alive.lock().unwrap().is_some());
h.alive
.lock()
.unwrap()
.as_ref()
.unwrap()
.store(false, Ordering::Release);
h.tx.send(opus_frame()).unwrap(); // push sees death → reopen
wait_until("reopen", || h.opens.load(Ordering::SeqCst) >= 2);
h.tx.send(opus_frame()).unwrap();
wait_until("pcm after reopen", || h.pushed.load(Ordering::SeqCst) > 0);
drop(h.tx);
h.join.join().unwrap();
}
/// Instances that die immediately after opening must be retried with BACKOFF, not at
/// heartbeat rate — a flapping backend (daemon up but dropping us instantly) would
/// otherwise churn open→die→reopen every heartbeat forever.
#[test]
fn rapid_death_backs_off() {
// Every instance is dead on arrival; stability threshold high so each death counts
// as a failed open. Without the guard: ~1 reopen per heartbeat (20 ms) ≈ 25 opens in
// 500 ms. With backoff 10→20→40 (cap): ≈ 7.
let h = start_tuned(0, true, Duration::from_secs(10));
std::thread::sleep(Duration::from_millis(500));
let opens = h.opens.load(Ordering::SeqCst);
assert!(opens >= 2, "must keep retrying (got {opens})");
assert!(
opens <= 15,
"must back off, not churn per heartbeat (got {opens})"
);
drop(h.tx);
h.join.join().unwrap();
}
/// An uplink gap discards buffered-stale audio before the next frame plays.
#[test]
fn discards_after_gap() {
let h = start(0);
wait_until("instance", || h.alive.lock().unwrap().is_some());
h.tx.send(opus_frame()).unwrap();
wait_until("first push", || h.pushed.load(Ordering::SeqCst) > 0);
std::thread::sleep(Duration::from_millis(150)); // > stale_gap
h.tx.send(opus_frame()).unwrap();
wait_until("discard on gap", || h.discards.load(Ordering::SeqCst) >= 1);
drop(h.tx);
h.join.join().unwrap();
}
}
mod mic_pump;
pub use mic_pump::MicPump;
+6 -3
View File
@@ -156,7 +156,10 @@ impl PwMicSource {
.name("punktfunk-pw-mic".into())
.spawn(move || {
if let Err(e) = mic_pw_thread(pcm_rx, quit_rx, channels, flush_t, ready_tx) {
tracing::error!(error = %format!("{e:#}"), "pipewire virtual-mic thread failed");
// Reaching here is always a setup/open failure (once the mainloop runs it exits
// Ok) — and it was already reported to the pump via the ready handshake, which
// owns the throttled operator-facing warn. Keep only a debug breadcrumb.
tracing::debug!(error = %format!("{e:#}"), "pipewire virtual-mic setup failed — pump will back off and retry");
}
// Whether a clean quit or a daemon death: this instance is done — the pump reopens.
alive_t.store(false, Ordering::Release);
@@ -322,7 +325,7 @@ fn mic_pw_thread(
.state_changed({
let mainloop = mainloop.clone();
move |_s, _ud, old, new| {
tracing::info!(?old, ?new, "pipewire virtual-mic stream state");
tracing::debug!(?old, ?new, "pipewire virtual-mic stream state");
// A stream error is unrecoverable for this instance — exit so the pump reopens.
if matches!(new, pw::stream::StreamState::Error(_)) {
mainloop.quit();
@@ -522,7 +525,7 @@ fn pw_thread(
let _listener = stream
.add_local_listener_with_user_data(tx)
.state_changed(|_s, _ud, old, new| {
tracing::info!(?old, ?new, "pipewire audio stream state");
tracing::debug!(?old, ?new, "pipewire audio stream state");
})
.param_changed(|_stream, _tx, id, param| {
let Some(param) = param else { return };
+432
View File
@@ -0,0 +1,432 @@
//! Host-lifetime virtual-microphone pump: one thread owns the [`VirtualMic`] backend plus an Opus
//! decoder, self-heals across backend deaths, and feeds decoded client-mic PCM into the source so
//! the client's microphone reaches host apps. Split out of the `audio` facade (§2.1 — a
//! self-contained stateful subsystem does not belong in the trait facade); the [`VirtualMic`]
//! trait, its factory ([`open_virtual_mic`](super::open_virtual_mic)) and the audio-plane sample
//! rate stay in `super`.
use super::{VirtualMic, SAMPLE_RATE};
use anyhow::Result;
/// Mic is 48 kHz stereo — matches the Opus stereo decoder and the host→client audio layout.
pub const MIC_CHANNELS: u32 = 2;
/// Bound for the shared mic frame queue (drop-newest when full): the host-lifetime queue is
/// shared across all concurrent sessions and must not grow without limit under a near-line-rate
/// flood (security-review 2026-06-28 S6). 64 × 520 ms frames ≈ 0.31.3 s of slack.
const MIC_QUEUE_CAP: usize = 64;
/// Tuning for [`MicPump`]'s open/reopen/flush behaviour — parameterized so the tests can run the
/// real pump loop in milliseconds instead of seconds.
#[derive(Clone, Copy)]
struct PumpTuning {
/// First-retry delay after a failed backend open; doubles per failure up to `backoff_cap`
/// (a persistently-absent PipeWire session / audio endpoint isn't hammered), resets on
/// success.
backoff_start: std::time::Duration,
backoff_cap: std::time::Duration,
/// Idle liveness-probe interval: with no frames flowing, the pump still notices a dead
/// backend this often and reopens — so the mic is healthy BEFORE the next session starts.
heartbeat: std::time::Duration,
/// An uplink gap longer than this discards the backend's buffered audio before pushing the
/// next frame (a recorder must never hear a stale burst from before a mute/session end).
stale_gap: std::time::Duration,
/// A backend that dies before living this long counts as a FAILED open for backoff purposes
/// (an open that succeeds but dies instantly — e.g. a flapping daemon — must not churn at
/// heartbeat rate); one that lived longer resets the backoff.
stable_after: std::time::Duration,
}
const PUMP_TUNING: PumpTuning = PumpTuning {
backoff_start: std::time::Duration::from_secs(2),
backoff_cap: std::time::Duration::from_secs(60),
heartbeat: std::time::Duration::from_secs(1),
stale_gap: std::time::Duration::from_millis(600),
stable_after: std::time::Duration::from_secs(5),
};
/// Host-lifetime virtual-microphone pump: one thread owns the [`VirtualMic`] backend + an Opus
/// decoder; sessions forward the client's Opus mic frames (0xCB) over a clonable `Send` sender,
/// the thread decodes and feeds the backend.
///
/// The rock-solid properties live HERE, not in the backends:
/// - **Eager**: the backend opens at host start (retrying with backoff), NOT on the first mic
/// frame — so the virtual mic device already exists when host apps/games launch and bind
/// their capture device (most games never re-follow a default-device change mid-run).
/// - **Self-healing**: a dead backend (PipeWire restart, Windows endpoint churn) is detected on
/// every push and on an idle heartbeat, and reopened with backoff. Sessions keep their
/// senders; nothing upstream notices.
/// - **Stale-flush**: buffered audio is discarded after an uplink gap (see [`PumpTuning`]).
///
/// Per-frame Opus DECODE errors stay non-fatal (dropped frame): the mic is shared across every
/// concurrent session, so one paired client's junk frames must not deny everyone's mic
/// (security-review 2026-06-28 S2). The thread exits when every sender is dropped (host
/// shutdown), tearing the backend down.
pub struct MicPump {
tx: std::sync::mpsc::SyncSender<Vec<u8>>,
}
impl MicPump {
/// Start the host-lifetime pump (Linux/Windows). On platforms without a virtual-mic backend
/// the thread just drains and drops frames (sessions still count the datagrams).
pub fn start() -> MicPump {
let (tx, rx) = std::sync::mpsc::sync_channel::<Vec<u8>>(MIC_QUEUE_CAP);
let spawned = std::thread::Builder::new()
.name("punktfunk-mic-pump".into())
.spawn(move || {
#[cfg(any(target_os = "linux", target_os = "windows"))]
pump_thread(rx, || super::open_virtual_mic(MIC_CHANNELS), PUMP_TUNING);
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
{
tracing::warn!("mic passthrough unsupported on this platform — frames dropped");
for _ in rx {}
}
});
if let Err(e) = spawned {
tracing::error!(error = %e, "mic pump thread spawn failed — mic passthrough disabled");
}
MicPump { tx }
}
/// A sender a session forwards the client's Opus mic frames to (`try_send` — never block a
/// datagram loop). Cloned per session; dropping a clone does NOT stop the pump (it holds
/// the original sender for the host life).
pub fn sender(&self) -> std::sync::mpsc::SyncSender<Vec<u8>> {
self.tx.clone()
}
}
/// Sleep for `dur` while draining (and dropping) queued frames, so a closed/reopening backend
/// never accumulates a stale backlog and senders never see a wedged queue. Returns `false` when
/// every sender is gone (host shutdown).
#[cfg_attr(not(any(target_os = "linux", target_os = "windows")), allow(dead_code))]
fn drain_sleep(rx: &std::sync::mpsc::Receiver<Vec<u8>>, dur: std::time::Duration) -> bool {
use std::sync::mpsc::RecvTimeoutError;
let deadline = std::time::Instant::now() + dur;
loop {
let left = deadline.saturating_duration_since(std::time::Instant::now());
if left.is_zero() {
return true;
}
match rx.recv_timeout(left.min(std::time::Duration::from_millis(250))) {
Ok(_) => {} // drop frames while closed
Err(RecvTimeoutError::Timeout) => {} // keep waiting
Err(RecvTimeoutError::Disconnected) => return false, // host shutdown
}
}
}
/// The pump loop. `opener` is injected so the tests can run the REAL loop against a mock
/// backend; production passes [`open_virtual_mic`](super::open_virtual_mic).
#[cfg_attr(not(any(target_os = "linux", target_os = "windows")), allow(dead_code))]
fn pump_thread<O>(rx: std::sync::mpsc::Receiver<Vec<u8>>, opener: O, tuning: PumpTuning)
where
O: Fn() -> Result<Box<dyn VirtualMic>>,
{
use std::sync::mpsc::RecvTimeoutError;
use std::time::Instant;
let mut backoff = tuning.backoff_start;
let mut open_fails: u64 = 0;
loop {
// Open phase — eager, from thread start.
let (mic, mut decoder) = loop {
let opened = opener().and_then(|m| {
let d = opus::Decoder::new(SAMPLE_RATE, opus::Channels::Stereo)
.map_err(|e| anyhow::anyhow!("opus decoder: {e}"))?;
Ok((m, d))
});
match opened {
Ok(pair) => break pair,
Err(e) => {
// Throttle (1st, 2nd, 4th, 8th … failure): a box without a PipeWire session
// or virtual audio device would otherwise log every backoff forever.
open_fails += 1;
if open_fails.is_power_of_two() {
tracing::warn!(error = %format!("{e:#}"), attempts = open_fails,
"virtual mic unavailable — retrying with backoff");
}
if !drain_sleep(&rx, backoff) {
return;
}
backoff = (backoff * 2).min(tuning.backoff_cap);
}
}
};
tracing::info!("virtual mic ready (host-lifetime)");
// Drop anything queued while (re)opening — it predates the backend. (The backoff does
// NOT reset here: only an instance that proves stable resets it — see the death triage.)
while rx.try_recv().is_ok() {}
let opened_at = Instant::now();
// Pump phase — runs until the backend dies (break) or the host shuts down (return).
let mut decode_fails: u64 = 0;
let mut pcm = vec![0f32; 5760 * MIC_CHANNELS as usize]; // up to 120 ms scratch
let mut last_push = Instant::now();
loop {
match rx.recv_timeout(tuning.heartbeat) {
Ok(frame) => {
if frame.is_empty() {
continue; // DTX silence — the source underruns to silence on its own
}
if last_push.elapsed() > tuning.stale_gap {
mic.discard();
}
match decoder.decode_float(&frame, &mut pcm, false) {
Ok(samples_per_ch) => {
let total = (samples_per_ch * MIC_CHANNELS as usize).min(pcm.len());
if !mic.push(&pcm[..total]) {
tracing::warn!("virtual mic backend died — reopening");
break;
}
last_push = Instant::now();
decode_fails = 0;
}
Err(e) => {
// Malformed/garbage frame: drop it, keep the shared mic + decoder
// (see the struct docs). Throttled log (1, 2, 4, … fails).
decode_fails += 1;
if decode_fails.is_power_of_two() {
tracing::warn!(error = %e, fails = decode_fails,
"mic opus decode failed — dropping frame");
}
}
}
}
Err(RecvTimeoutError::Timeout) => {
if !mic.alive() {
tracing::warn!("virtual mic backend died while idle — reopening");
break;
}
}
Err(RecvTimeoutError::Disconnected) => {
tracing::debug!("mic pump stopped (host shutting down)");
return;
}
}
}
// Death triage: an instance that lived is a one-off (PipeWire/audio-engine restart) —
// reopen immediately with the backoff reset. One that died right after opening is a
// failed open in disguise (flapping daemon, endpoint racing away): back off like the
// open loop, or the pump would churn open→die→reopen at heartbeat rate.
if opened_at.elapsed() >= tuning.stable_after {
backoff = tuning.backoff_start;
open_fails = 0;
} else {
open_fails += 1;
if !drain_sleep(&rx, backoff) {
return;
}
backoff = (backoff * 2).min(tuning.backoff_cap);
}
}
}
#[cfg(test)]
mod pump_tests {
use super::*;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;
/// Mock backend: records pushes/discards, dies on command.
struct MockMic {
alive: Arc<AtomicBool>,
pushed: Arc<AtomicUsize>,
discards: Arc<AtomicUsize>,
}
impl VirtualMic for MockMic {
fn push(&self, pcm: &[f32]) -> bool {
if !self.alive.load(Ordering::Acquire) {
return false;
}
self.pushed.fetch_add(pcm.len(), Ordering::Relaxed);
true
}
fn alive(&self) -> bool {
self.alive.load(Ordering::Acquire)
}
fn discard(&self) {
self.discards.fetch_add(1, Ordering::Relaxed);
}
}
struct Harness {
tx: std::sync::mpsc::SyncSender<Vec<u8>>,
opens: Arc<AtomicUsize>,
alive: Arc<Mutex<Option<Arc<AtomicBool>>>>, // latest instance's kill switch
pushed: Arc<AtomicUsize>,
discards: Arc<AtomicUsize>,
join: std::thread::JoinHandle<()>,
}
/// Run the REAL pump loop against mock backends; `fail_first` opens fail before the first
/// success (exercises the eager retry/backoff path). `dead_on_arrival` opens every instance
/// pre-killed (exercises the rapid-death churn guard). `stable_after` mirrors the tuning
/// field (ZERO = every death counts as stable → immediate reopen, keeping tests fast).
fn start_tuned(fail_first: usize, dead_on_arrival: bool, stable_after: Duration) -> Harness {
let (tx, rx) = std::sync::mpsc::sync_channel::<Vec<u8>>(MIC_QUEUE_CAP);
let opens = Arc::new(AtomicUsize::new(0));
let alive = Arc::new(Mutex::new(None::<Arc<AtomicBool>>));
let pushed = Arc::new(AtomicUsize::new(0));
let discards = Arc::new(AtomicUsize::new(0));
let (opens2, alive2, pushed2, discards2) = (
opens.clone(),
alive.clone(),
pushed.clone(),
discards.clone(),
);
let tuning = PumpTuning {
backoff_start: Duration::from_millis(10),
backoff_cap: Duration::from_millis(40),
heartbeat: Duration::from_millis(20),
stale_gap: Duration::from_millis(80),
stable_after,
};
let join = std::thread::spawn(move || {
pump_thread(
rx,
move || {
let n = opens2.fetch_add(1, Ordering::SeqCst);
if n < fail_first {
anyhow::bail!("backend not up yet (simulated)");
}
let a = Arc::new(AtomicBool::new(!dead_on_arrival));
*alive2.lock().unwrap() = Some(a.clone());
Ok(Box::new(MockMic {
alive: a,
pushed: pushed2.clone(),
discards: discards2.clone(),
}) as Box<dyn VirtualMic>)
},
tuning,
)
});
Harness {
tx,
opens,
alive,
pushed,
discards,
join,
}
}
fn start(fail_first: usize) -> Harness {
start_tuned(fail_first, false, Duration::ZERO)
}
fn wait_until(what: &str, mut cond: impl FnMut() -> bool) {
for _ in 0..200 {
if cond() {
return;
}
std::thread::sleep(Duration::from_millis(10));
}
panic!("timed out waiting for: {what}");
}
fn opus_frame() -> Vec<u8> {
let mut enc = opus::Encoder::new(48_000, opus::Channels::Stereo, opus::Application::Voip)
.expect("opus encoder");
let pcm = [0.1f32; 960 * 2]; // 20 ms stereo
let mut out = vec![0u8; 4000];
let n = enc.encode_float(&pcm, &mut out).expect("encode");
out.truncate(n);
out
}
/// Eager: the backend opens (after transient failures) with NO frame ever sent.
#[test]
fn opens_eagerly_with_backoff() {
let h = start(3);
wait_until("eager open after 3 failures", || {
h.opens.load(Ordering::SeqCst) >= 4 && h.alive.lock().unwrap().is_some()
});
drop(h.tx);
h.join.join().unwrap();
}
/// Frames flow: opus in → PCM pushed to the backend.
#[test]
fn decodes_and_pushes() {
let h = start(0);
wait_until("open", || h.alive.lock().unwrap().is_some());
h.tx.send(opus_frame()).unwrap();
wait_until("pcm pushed", || h.pushed.load(Ordering::SeqCst) > 0);
drop(h.tx);
h.join.join().unwrap();
}
/// A dead backend is noticed WHILE IDLE (heartbeat) and reopened without any traffic.
#[test]
fn reopens_after_idle_death() {
let h = start(0);
wait_until("first open", || h.opens.load(Ordering::SeqCst) >= 1);
wait_until("instance", || h.alive.lock().unwrap().is_some());
h.alive
.lock()
.unwrap()
.as_ref()
.unwrap()
.store(false, Ordering::Release); // kill it
wait_until("reopen after idle death", || {
h.opens.load(Ordering::SeqCst) >= 2
});
drop(h.tx);
h.join.join().unwrap();
}
/// A death detected on push (frame flowing) also reopens, and the frame after reopen flows.
#[test]
fn reopens_after_push_death() {
let h = start(0);
wait_until("instance", || h.alive.lock().unwrap().is_some());
h.alive
.lock()
.unwrap()
.as_ref()
.unwrap()
.store(false, Ordering::Release);
h.tx.send(opus_frame()).unwrap(); // push sees death → reopen
wait_until("reopen", || h.opens.load(Ordering::SeqCst) >= 2);
h.tx.send(opus_frame()).unwrap();
wait_until("pcm after reopen", || h.pushed.load(Ordering::SeqCst) > 0);
drop(h.tx);
h.join.join().unwrap();
}
/// Instances that die immediately after opening must be retried with BACKOFF, not at
/// heartbeat rate — a flapping backend (daemon up but dropping us instantly) would
/// otherwise churn open→die→reopen every heartbeat forever.
#[test]
fn rapid_death_backs_off() {
// Every instance is dead on arrival; stability threshold high so each death counts
// as a failed open. Without the guard: ~1 reopen per heartbeat (20 ms) ≈ 25 opens in
// 500 ms. With backoff 10→20→40 (cap): ≈ 7.
let h = start_tuned(0, true, Duration::from_secs(10));
std::thread::sleep(Duration::from_millis(500));
let opens = h.opens.load(Ordering::SeqCst);
assert!(opens >= 2, "must keep retrying (got {opens})");
assert!(
opens <= 15,
"must back off, not churn per heartbeat (got {opens})"
);
drop(h.tx);
h.join.join().unwrap();
}
/// An uplink gap discards buffered-stale audio before the next frame plays.
#[test]
fn discards_after_gap() {
let h = start(0);
wait_until("instance", || h.alive.lock().unwrap().is_some());
h.tx.send(opus_frame()).unwrap();
wait_until("first push", || h.pushed.load(Ordering::SeqCst) > 0);
std::thread::sleep(Duration::from_millis(150)); // > stale_gap
h.tx.send(opus_frame()).unwrap();
wait_until("discard on gap", || h.discards.load(Ordering::SeqCst) >= 1);
drop(h.tx);
h.join.join().unwrap();
}
}
@@ -39,7 +39,7 @@ impl WasapiLoopbackCapturer {
.name("punktfunk-wasapi-audio".into())
.spawn(move || {
if let Err(e) = capture_thread(tx, stop_t, ready_tx, channels) {
tracing::error!(error = format!("{e:#}"), "wasapi loopback thread failed");
tracing::error!(error = %format!("{e:#}"), "wasapi loopback thread failed");
}
})
.context("spawn wasapi audio thread")?;
+19 -35
View File
@@ -81,12 +81,14 @@ pub struct OutputFormat {
impl OutputFormat {
/// Resolve the output format for an entry point that doesn't build a full [`SessionPlan`]
/// (`crate::session_plan`) — the GameStream + spike paths: `gpu` from the resolved encode backend,
/// `hdr` as given. The native punktfunk/1 path uses `SessionPlan::output_format()` instead (it already
/// resolved the encoder), so neither path makes a capturer re-derive it.
pub fn resolve(hdr: bool) -> Self {
/// (`crate::session_plan`) — the GameStream + spike paths. `gpu` is the encoder's GPU-residency,
/// resolved by the caller via [`crate::encode::resolved_backend_is_gpu`] and passed **in** (capture
/// never re-derives the backend — the one-way capture→encode edge, plan §2.4 / §W4); `hdr` as given.
/// The native punktfunk/1 path uses `SessionPlan::output_format()` instead (it already resolved the
/// encoder), so neither path makes a capturer re-derive it.
pub fn resolve(hdr: bool, gpu: bool) -> Self {
OutputFormat {
gpu: gpu_encode(),
gpu,
hdr,
// The GameStream + spike paths are always 4:2:0 (4:4:4 is punktfunk/1-native only).
chroma_444: false,
@@ -94,26 +96,6 @@ impl OutputFormat {
}
}
/// True if the resolved encode backend produces GPU frames (anything but the software encoder). The single
/// source for [`OutputFormat::resolve`]'s `gpu`; on Linux always true (the portal/VAAPI/CUDA path is GPU).
#[cfg(target_os = "windows")]
pub(crate) fn gpu_encode() -> bool {
!matches!(
crate::encode::windows_resolved_backend(),
crate::encode::WindowsBackend::Software
)
}
#[cfg(not(target_os = "windows"))]
pub(crate) fn gpu_encode() -> bool {
// The GPU-less software encoder (openh264) needs CPU-staged RGB frames; every other Linux
// backend (NVENC/CUDA, VAAPI) is GPU-resident. Mirrors `session_plan::resolve_encoder`, for the
// GameStream/spike entry points that use `OutputFormat::resolve` instead of a full `SessionPlan`.
!matches!(
crate::config::config().encoder_pref.as_str(),
"software" | "sw" | "openh264"
)
}
/// A mouse-cursor overlay to composite onto a frame at encode time (cursor-as-metadata). Rides on
/// [`CapturedFrame::cursor`] for the GPU zero-copy payloads (Cuda/Dmabuf), whose pixels never touch
/// the CPU — the encoder blends this small bitmap into its owned surface (Vulkan CSC image / CUDA
@@ -453,25 +435,27 @@ pub fn capture_virtual_output(
/// Whether the active capturer can deliver a full-chroma (RGB) source for a 4:4:4 HEVC encode. The
/// negotiator gates 4:4:4 on this so the host honestly downgrades to 4:2:0 when the capturer can only
/// produce subsampled frames. Linux (the portal capturer feeding CPU RGB → `yuv444p`) can; the Windows
/// IDD-push path delivers subsampled NV12/P010 today, so full-chroma capture there is a follow-up.
/// produce subsampled frames. `encoder_ingests_rgb_444` is the encoder half of the gate, resolved by
/// the caller ([`crate::encode::resolved_backend_ingests_rgb_444`]) and passed **in** so capture never
/// re-derives the backend (the one-way capture→encode edge, plan §2.4 / §W4). Linux (the portal capturer
/// feeding CPU RGB → `yuv444p`) can regardless; the Windows IDD-push path delivers subsampled NV12/P010
/// today, so full-chroma capture there rides entirely on the encoder gate.
#[cfg(target_os = "linux")]
pub(crate) fn capturer_supports_444() -> bool {
pub(crate) fn capturer_supports_444(_encoder_ingests_rgb_444: bool) -> bool {
true
}
#[cfg(target_os = "windows")]
pub(crate) fn capturer_supports_444() -> bool {
// IDD-push delivers full-chroma BGRA for an SDR 4:4:4 session (skipping the NV12
// VideoConverter) — but only the direct-NVENC backend ingests RGB and CSCs it to 4:4:4
// (measured on-glass: true full chroma, matrix follows the configured VUI), so gate on it
// (AMF can't 4:4:4 at all; the QSV/ffmpeg path has no RGB-input 4:4:4 wiring). An HDR
pub(crate) fn capturer_supports_444(encoder_ingests_rgb_444: bool) -> bool {
// IDD-push delivers full-chroma BGRA for an SDR 4:4:4 session (skipping the NV12 VideoConverter),
// but only a backend that ingests RGB and CSCs it to 4:4:4 itself can use it — today just
// direct-NVENC (AMF can't 4:4:4 at all; the QSV/ffmpeg path has no RGB-input 4:4:4 wiring). An HDR
// display can't be known here (the virtual display's mode settles after the Welcome); that
// combination downgrades at capture time — the capturer emits P010 and the encoder's caps
// cross-check reports the 4:2:0 truth (the in-band SPS keeps the client correct either way).
crate::encode::windows_resolved_backend() == crate::encode::WindowsBackend::Nvenc
encoder_ingests_rgb_444
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub(crate) fn capturer_supports_444() -> bool {
pub(crate) fn capturer_supports_444(_encoder_ingests_rgb_444: bool) -> bool {
false
}
+16 -22
View File
@@ -650,8 +650,6 @@ mod pipewire {
/// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves,
/// so the GPU encoder re-uploads its cursor texture only on change.
serial: u64,
/// One-shot guard for the "cursor present but this frame is zero-copy" notice.
warned_zerocopy: bool,
}
impl CursorState {
@@ -1174,22 +1172,6 @@ mod pipewire {
if ud.broken.load(Ordering::Relaxed) {
return;
}
// Cursor-as-metadata only reaches the frame on the CPU de-pad path below (a small
// straight-alpha blit). The zero-copy paths hand a GPU-resident buffer straight to the
// encoder, so the cached cursor can't be composited here — that needs a GPU blit in the
// encoder (follow-up). Note it once, so a gamescope host (zero-copy by default) shows in
// the logs that the metadata IS arriving even while the overlay isn't drawn yet.
if ud.cursor.visible
&& !ud.cursor.warned_zerocopy
&& (ud.importer.is_some() || ud.vaapi_passthrough)
{
ud.cursor.warned_zerocopy = true;
tracing::warn!(
"cursor metadata received, but frames are delivered zero-copy (GPU-resident) — \
the cursor overlay is composited only on the CPU capture path today; GPU-path \
compositing (Vulkan/CUDA/VAAPI encode) is a follow-up"
);
}
// SAFETY: `spa_buf` is the `*mut spa_buffer` of the PipeWire buffer we dequeued and still hold for
// this `.process` callback (not requeued until after `consume_frame` returns), so it is live. The
// block null-checks `spa_buf`, requires `n_datas != 0`, and null-checks the `datas` array pointer
@@ -1241,7 +1223,7 @@ mod pipewire {
std::sync::atomic::AtomicBool::new(true);
if F2.swap(false, Ordering::Relaxed) {
tracing::warn!(
error = %format!("{e}"),
error = %e,
"dmabuf EXPORT_SYNC_FILE failed — no implicit-fence sync; NVIDIA \
zero-copy may show stale frames (no producer explicit sync)"
);
@@ -1675,7 +1657,7 @@ mod pipewire {
sample = ?&modifiers[..modifiers.len().min(6)],
"zero-copy: advertising EGL-importable dmabuf modifiers"
);
} else if backend_is_vaapi && crate::capture::gpu_encode() {
} else if backend_is_vaapi && crate::encode::resolved_backend_is_gpu() {
// A VAAPI session on the CPU path pays three full-frame CPU touches (mmap de-pad +
// swscale RGB→NV12 + surface upload) — make the silent fallback visible.
tracing::warn!(
@@ -1915,7 +1897,15 @@ mod pipewire {
unsafe { stream.queue_raw_buffer(newest) };
}));
if outcome.is_err() {
tracing::error!("panic in pipewire process callback — frame dropped");
// In the per-frame `.process` callback: a deterministic panic (e.g. a bad
// format) would fire this every frame, so power-of-two throttle it — enough to
// surface the fault without evicting the whole log ring.
static PANICS: std::sync::atomic::AtomicU64 =
std::sync::atomic::AtomicU64::new(0);
let n = PANICS.fetch_add(1, Ordering::Relaxed) + 1;
if n.is_power_of_two() {
tracing::error!(count = n, "panic in pipewire process callback — frame dropped");
}
}
})
.register()
@@ -1930,7 +1920,11 @@ mod pipewire {
// Request raw video in any encoder-mappable layout, any size/framerate.
let obj = if let Some((fw, fh)) = fixed_pod {
tracing::info!(fw, fh, "PW DEBUG: offering fixed BGRx pod");
tracing::info!(
fw,
fh,
"pipewire: offering a fixed BGRx format pod (PUNKTFUNK_PW_FIXED_POD)"
);
pw::spa::pod::object!(
pw::spa::utils::SpaTypes::ObjectParamFormat,
pw::spa::param::ParamType::EnumFormat,
@@ -299,7 +299,7 @@ pub(crate) fn install_gpu_pref_hook() {
// 100% DuplicateOutput1 E_ACCESSDENIED is diagnosable instead of silent.
match SetProcessDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) {
Ok(()) => tracing::info!("DPI awareness set: PER_MONITOR_AWARE_V2"),
Err(e) => tracing::warn!(error = %format!("{e:?}"),
Err(e) => tracing::warn!(error = ?e,
"SetProcessDpiAwarenessContext failed (already set?) — DuplicateOutput1 may E_ACCESSDENIED"),
}
// 0=UNAWARE 1=SYSTEM 2=PER_MONITOR(_V2). DuplicateOutput1 needs 2.
@@ -193,10 +193,20 @@ impl Drop for KeyedMutexGuard<'_> {
}
}
/// Nudge DWM into composing THE TARGET virtual display. DWM presents a display only when something
/// DIRTIES it — an idle desktop never does, so a freshly-attached ring (session open, or a
/// mid-session ring recreate) can sit at E_PENDING with no first frame even though everything is
/// healthy. pf-vdisplay implements no hardware-cursor plane, so a cursor move is composited into
/// LAST-RESORT fallback: nudge DWM into composing THE TARGET virtual display. DWM presents a
/// display only when something DIRTIES it — an idle desktop never does, so a freshly-attached ring
/// (session open, or a mid-session ring recreate) can sit at E_PENDING with no first frame even
/// though everything is healthy.
///
/// The PRIMARY first-frame mechanism is the driver's `FrameStash` (frame_transport.rs): the driver
/// retains the last composed frame and republishes it into every freshly-attached ring, so with a
/// stash-capable driver the first frame lands milliseconds after the channel delivery and this kick
/// never fires. It remains for pre-stash drivers and for the empty-stash cold start (a monitor that
/// has NEVER composed — normally the activation compose covers that). Synthetic input is inherently
/// unreliable — blocked on the secure desktop, defeated by a fullscreen game's ClipCursor, and
/// user-visible in the sibling-display case — which is exactly why it was demoted to fallback.
///
/// pf-vdisplay implements no hardware-cursor plane, so a cursor move is composited into
/// the frame — a guaranteed real present onto the IDD swap-chain (empirically what
/// `punktfunk-probe --input-test` always relied on).
///
@@ -313,384 +323,15 @@ pub(crate) unsafe fn verify_is_wudfhost(process: HANDLE, wudf_pid: u32, what: &s
Ok(())
}
/// The sealed channel's handle-duplication broker (`design/idd-push-security.md`): the frame objects
/// are unnamed, so the ONLY way the driver can reach them is handles this broker duplicates into its
/// WUDFHost process and delivers — as bare handle VALUES — over the SYSTEM-only control device
/// (`IOCTL_SET_FRAME_CHANNEL`). Ownership is a strict hand-off: on IOCTL success the DRIVER owns the
/// duplicates (it closes them); on any failure [`Self::send`] reaps every duplicate it already made
/// (`DUPLICATE_CLOSE_SOURCE`), so a half-delivered channel never leaks handles in WUDFHost.
struct ChannelBroker {
/// `PROCESS_DUP_HANDLE | SYNCHRONIZE` handle to the driver's WUDFHost (pid from the ADD reply;
/// `ProcessSharingDisabled` makes that process exclusively pf-vdisplay's). `SYNCHRONIZE` lets the
/// handle double as the driver-death probe ([`Self::driver_alive`]).
process: OwnedHandle,
/// The WUDFHost pid `process` refers to (diagnostics for the driver-death bail).
wudf_pid: u32,
/// The pf-vdisplay control device — owned by the `VirtualDisplayManager`, never closed for the
/// process lifetime (a dead one is retired, kept alive), so holding the bare `HANDLE` is sound.
control: HANDLE,
}
impl ChannelBroker {
/// Open the duplication target. Fails when the driver predates the sealed channel (`wudf_pid == 0`
/// can't survive the v2 version handshake, but guard anyway) or the WUDFHost is gone (device
/// restart mid-open) — either way the caller fails the capture open cleanly.
///
/// `wudf_pid` comes from the driver's ADD reply, so before we duplicate whole-desktop frame handles
/// INTO it we VERIFY it is a genuine system WUDFHost ([`verify_is_wudfhost`]). Without that check a
/// spoofed devnode (same interface GUID) could name an arbitrary process and receive the frames; a
/// fully-compromised REAL pf_vdisplay driver is already a frame endpoint, so this specifically closes
/// the reachable-without-owning-the-driver case (`design/idd-push-security.md` §hardening).
fn open(wudf_pid: u32) -> Result<Self> {
if wudf_pid == 0 {
bail!("driver reported no WUDFHost pid for the frame channel");
}
let control = crate::vdisplay::manager::control_device_handle().context(
"pf-vdisplay control device not open (monitor not created via the manager?)",
)?;
// SAFETY: plain FFI; `wudf_pid` is a copy. The handle (checked by `?`) is owned solely here and
// moved into the `OwnedHandle` (single owner, closes on drop); `verify_is_wudfhost` borrows it
// for the duration of the synchronous check and forms no lasting alias.
let process = unsafe {
let h = OpenProcess(
PROCESS_DUP_HANDLE | PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_SYNCHRONIZE,
false,
wudf_pid,
)
.context("OpenProcess(PROCESS_DUP_HANDLE) on the driver's WUDFHost")?;
let process = OwnedHandle::from_raw_handle(h.0 as _);
verify_is_wudfhost(HANDLE(process.as_raw_handle()), wudf_pid, "frame-channel")?;
process
};
Ok(Self {
process,
wudf_pid,
control,
})
}
/// Whether the driver's WUDFHost is still alive. The pinned process handle doubles as the
/// liveness probe (`SYNCHRONIZE` requested at open): signaled ⇔ the process exited. This is the
/// definitive "driver died mid-session" signal — at the ring, a dead driver and an idle desktop
/// are indistinguishable (both simply stop publishing).
fn driver_alive(&self) -> bool {
// SAFETY: `process` is the live `OwnedHandle` this broker owns (borrowed for this synchronous
// call); a 0 ms wait only reads the handle's signaled state.
unsafe { WaitForSingleObject(HANDLE(self.process.as_raw_handle()), 0) != WAIT_OBJECT_0 }
}
/// Duplicate `h` into the WUDFHost handle table, returning the handle VALUE valid there (and only
/// there — the value is meaningless in any other process). `access = Some(rights)` grants the
/// driver's handle exactly those rights (least privilege — see [`SECTION_MAP_RW`]);
/// `access = None` copies the source handle's access (`DUPLICATE_SAME_ACCESS`), used only where the
/// source is already scoped (the DXGI shared-texture handles, minted by `CreateSharedHandle` with
/// just `DXGI_SHARED_RESOURCE_READ|WRITE`).
///
/// # Safety
/// `h` must be a live handle of the current process.
unsafe fn dup_into(&self, h: HANDLE, access: Option<u32>) -> Result<u64> {
let mut out = HANDLE::default();
let (desired, options) = match access {
Some(rights) => (rights, DUPLICATE_HANDLE_OPTIONS(0)),
None => (0, DUPLICATE_SAME_ACCESS),
};
// SAFETY: `h` is live per the contract; `self.process` is the live PROCESS_DUP_HANDLE target;
// `&mut out` is a valid out-param. Either an explicit least-privilege access mask (options == 0)
// or `DUPLICATE_SAME_ACCESS` (desired ignored) — never both.
unsafe {
DuplicateHandle(
GetCurrentProcess(),
h,
HANDLE(self.process.as_raw_handle()),
&mut out,
desired,
false,
options,
)
}
.context("DuplicateHandle into the driver's WUDFHost")?;
Ok(out.0 as usize as u64)
}
/// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE`
/// with no target closes the source handle regardless of the (ignored) result.
fn close_remote(&self, value: u64) {
if value == 0 {
return;
}
// SAFETY: `self.process` is the live duplication target and `value` is a handle value THIS
// broker just created in that process's table (callers only pass back `dup_into` results the
// driver never received); closing it there cannot touch any other process's handles.
unsafe {
let _ = DuplicateHandle(
HANDLE(self.process.as_raw_handle()),
HANDLE(value as usize as *mut core::ffi::c_void),
HANDLE::default(),
std::ptr::null_mut(),
0,
false,
DUPLICATE_CLOSE_SOURCE,
);
}
}
/// Duplicate the whole ring (header + event + every slot texture) into WUDFHost and deliver the
/// values via `IOCTL_SET_FRAME_CHANNEL`. All-or-nothing: on any failure every duplicate already
/// made is reaped remotely and an error returns (the caller fails the open / logs the recreate).
/// The ownership contract with the driver is adopt-on-success only — it closes the handles iff the
/// IOCTL succeeded, we reap them iff it didn't, so no value is ever closed twice.
///
/// # Safety
/// `header` and `event` must be live handles of the current process (the capturer's own section +
/// event, borrowed for this synchronous call).
unsafe fn send(
&self,
target_id: u32,
generation: u32,
header: HANDLE,
event: HANDLE,
slots: &[HostSlot],
) -> Result<()> {
debug_assert!(slots.len() <= control::RING_LEN_USIZE);
let mut req = control::SetFrameChannelRequest {
target_id,
generation,
ring_len: slots.len() as u32,
_pad: 0,
header_handle: 0,
event_handle: 0,
texture_handles: [0; control::RING_LEN_USIZE],
};
// SAFETY: `header`/`event` are live per this fn's contract; each slot's `shared` is the live
// `OwnedHandle` the slot keeps for exactly this purpose.
let result = unsafe { self.duplicate_and_deliver(&mut req, header, event, slots) };
if result.is_err() {
// The driver never adopted the delivery — reap every remote duplicate so nothing lingers.
self.close_remote(req.header_handle);
self.close_remote(req.event_handle);
for v in req.texture_handles {
self.close_remote(v);
}
}
result
}
/// The fallible middle of [`Self::send`]: fill `req` with fresh duplicates, then issue the IOCTL.
/// Split out so `send` can reap whatever landed in `req` when any step errors.
///
/// # Safety
/// As [`Self::send`].
unsafe fn duplicate_and_deliver(
&self,
req: &mut control::SetFrameChannelRequest,
header: HANDLE,
event: HANDLE,
slots: &[HostSlot],
) -> Result<()> {
// SAFETY: forwarded from the caller's contract — `header`/`event`/each `slot.shared` are live
// handles of this process, and `self.control` is the manager's control handle, never closed for
// the process lifetime (`send_frame_channel`'s precondition).
unsafe {
// Least privilege per handle: the header maps read/write, the event is only signalled, and
// the textures keep their already-scoped `CreateSharedHandle` access (see `dup_into`).
req.header_handle = self.dup_into(header, Some(SECTION_MAP_RW))?;
req.event_handle = self.dup_into(event, Some(EVENT_MODIFY_STATE))?;
for (k, s) in slots.iter().enumerate() {
req.texture_handles[k] = self.dup_into(HANDLE(s.shared.as_raw_handle()), None)?;
}
crate::vdisplay::pf_vdisplay::send_frame_channel(self.control, req)
}
}
}
/// Creates + owns the shared ring; yields the driver's frames as [`FramePayload::D3d11`].
/// The display descriptor the capture loop follows: live HDR state + active resolution of the
/// virtual target.
#[derive(Clone, Copy, PartialEq, Eq)]
struct DisplayDescriptor {
hdr: bool,
width: u32,
height: u32,
}
/// Off-thread poller for [`DisplayDescriptor`]. The CCD queries behind it (`QueryDisplayConfig`,
/// twice per sample) serialize on the session-global display-configuration lock, which display-
/// topology events and third-party display-poller software (the SteelSeries-GG class) can hold
/// for tens-to-hundreds of milliseconds at a time. Polled inline — the old design — that stall
/// landed ON the capture/encode thread: a periodic frame hitch on an otherwise healthy host, and
/// invisible in any log. Now a dedicated thread samples every [`Self::INTERVAL`] and publishes a
/// snapshot; the capture thread's per-frame cost is one uncontended mutex read, and a slow CCD
/// sample is *measured and logged* instead of silently stalling the stream.
///
/// Failure policy is last-known-good, per field: a transient CCD failure — including the target
/// briefly missing from the active-path list during a topology re-probe — keeps the previous
/// value instead of reading as `hdr = false` (the old behavior, which on an HDR session turned
/// every blip into TWO ring recreates: false, then true again a poll later). `seq` bumps only
/// when at least one query succeeded, so the consumer's debounce counts real observations, never
/// failures.
struct DescriptorPoller {
/// Latest merged sample + its sequence number; the poller holds the lock only to copy it.
snap: Arc<Mutex<(DisplayDescriptor, u64)>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl DescriptorPoller {
/// Poll cadence — the old inline throttle. With the consumer's two-strikes debounce on top, a
/// real "Use HDR" flip or mode-set is acted on within ~2 samples (≈ ½ s).
const INTERVAL: Duration = Duration::from_millis(250);
/// A sample slower than this means something is sitting on the display-config lock (topology
/// churn / display-poller software) — the disturbance class behind periodic virtual-display
/// stream hitches. Logged (rate-limited) so an affected host self-diagnoses.
const SLOW: Duration = Duration::from_millis(50);
fn spawn(target_id: u32, initial: DisplayDescriptor) -> Self {
let snap = Arc::new(Mutex::new((initial, 0u64)));
let stop = Arc::new(AtomicBool::new(false));
let (snap_t, stop_t) = (snap.clone(), stop.clone());
let thread = std::thread::Builder::new()
.name("pf-idd-desc-poll".into())
.spawn(move || {
let mut last = initial;
let mut seq = 0u64;
let mut last_slow_log: Option<Instant> = None;
while !stop_t.load(Ordering::Relaxed) {
let t = Instant::now();
// SAFETY: both are read-only CCD queries taking only a copy of the plain `u32`
// target id (see their own SAFETY docs); nothing is borrowed across the calls.
let (hdr, res) = unsafe {
(
crate::win_display::advanced_color_enabled(target_id),
crate::win_display::active_resolution(target_id),
)
};
let took = t.elapsed();
if took >= Self::SLOW
&& last_slow_log.is_none_or(|t| t.elapsed() >= Duration::from_secs(10))
{
last_slow_log = Some(Instant::now());
tracing::warn!(
took_ms = took.as_millis() as u64,
target_id,
"slow display-descriptor poll — something is holding the Windows \
display-config lock (topology churn / display-poller software); on \
a host with periodic stream hitches, correlate this cadence"
);
}
if hdr.is_some() || res.is_some() {
if let Some(hdr) = hdr {
last.hdr = hdr;
}
if let Some((width, height)) = res {
last.width = width;
last.height = height;
}
seq += 1;
*snap_t.lock().unwrap() = (last, seq);
}
// Park (not sleep) so `drop` wakes the thread immediately via `unpark`.
std::thread::park_timeout(Self::INTERVAL);
}
})
.map_err(|e| {
// Degraded, not fatal: the session streams, it just never follows a mid-session
// HDR flip / mode-set (seq stays 0 → the consumer sees no changes).
tracing::error!(error = %e, "IDD push: descriptor-poller thread failed to spawn");
})
.ok();
Self { snap, stop, thread }
}
/// The latest sample (lock held only for the copy — the poller writes at 4 Hz).
fn snapshot(&self) -> (DisplayDescriptor, u64) {
*self.snap.lock().unwrap()
}
}
impl Drop for DescriptorPoller {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(t) = self.thread.take() {
t.thread().unpark();
let _ = t.join();
}
}
}
/// A detected capture stall: a multi-hundred-ms hole in DWM's frame delivery that opened while the
/// desktop was actively composing right beforehand (see [`StallWatch`]).
struct Stall {
/// How long the hole lasted (last fresh frame → the frame that ended it).
gap: Duration,
/// `Some(mean period)` when this stall completes a metronomic cycle (see
/// [`crate::metronome::Metronome`]).
metronomic: Option<Duration>,
}
/// Capture-stall watch — the "sole virtual display" stutter diagnostic (field reports: Exclusive
/// topology = periodic double-jolt, Extend = smooth, i.e. the disturbance lives in the display/present
/// path BELOW capture and only while no physical output is active).
///
/// On a damage-driven capture an idle desktop legitimately goes quiet (no damage → no frames), so a
/// gap only counts as a stall when the [`Self::RECENT`] frames before it all arrived within
/// [`Self::ACTIVE_SPAN`] — sustained ≥ ~20 fps flow (a game or video), not a blinking caret or a
/// mouse twitch. Each stall feeds a [`crate::metronome::Metronome`], so periodic stalls self-diagnose
/// in the log WITHOUT needing any client keyframe request — discriminating "DWM stopped composing"
/// from encode/network causes that the recovery-cadence detector covers. Pure logic — unit-tested
/// below; the caller does the logging.
struct StallWatch {
/// The last [`Self::RECENT`] fresh-frame instants (pre-gap history for the activity gate).
recent: std::collections::VecDeque<Instant>,
cadence: crate::metronome::Metronome,
}
impl StallWatch {
/// Frames of pre-gap history that must be tight for flow to count as active. Stalls are thus
/// naturally spaced ≥ RECENT frame times apart — no extra log rate limit needed.
const RECENT: usize = 8;
/// The RECENT pre-gap frames must all fit in this span (8 frames in 400 ms ≈ ≥ 20 fps flow —
/// loose enough for a 30 fps-capped game, tight enough to reject idle-desktop damage).
const ACTIVE_SPAN: Duration = Duration::from_millis(400);
/// The smallest hole that counts as a stall (~9 missed frames at 60 Hz) — well below the
/// reported 300700 ms freezes, above encode/present jitter.
const STALL_MIN: Duration = Duration::from_millis(150);
fn new() -> Self {
Self {
recent: std::collections::VecDeque::with_capacity(Self::RECENT + 1),
cadence: crate::metronome::Metronome::new(),
}
}
/// Forget the flow history (a ring recreate's gap is self-inflicted, not a DWM stall — without
/// the reset the first post-recreate frame would read as one).
fn reset(&mut self) {
self.recent.clear();
}
/// Record a fresh driver frame at `now`; `Some` exactly when it ended a stall.
fn note_fresh(&mut self, now: Instant) -> Option<Stall> {
let was_active = self.recent.len() == Self::RECENT
&& self
.recent
.back()
.zip(self.recent.front())
.is_some_and(|(b, f)| b.duration_since(*f) <= Self::ACTIVE_SPAN);
let gap = self.recent.back().map(|last| now.duration_since(*last));
self.recent.push_back(now);
if self.recent.len() > Self::RECENT {
self.recent.pop_front();
}
let gap = gap?;
if !was_active || gap < Self::STALL_MIN {
return None;
}
Some(Stall {
gap,
metronomic: self.cadence.note(now),
})
}
}
#[path = "idd_push/channel.rs"]
mod channel;
#[path = "idd_push/descriptor.rs"]
mod descriptor;
#[path = "idd_push/stall.rs"]
mod stall;
use channel::ChannelBroker;
use descriptor::{DescriptorPoller, DisplayDescriptor};
use stall::StallWatch;
pub struct IddPushCapturer {
device: ID3D11Device,
@@ -753,6 +394,13 @@ pub struct IddPushCapturer {
/// during active flow and warns when they turn metronomic — the sole-virtual-display
/// periodic-stutter diagnostic.
stall_watch: StallWatch,
/// Stall↔OS-event correlation counters for the metronomic warn: how many stalls this session,
/// and how many had a coinciding [`crate::display_events`] event in their gap window — the
/// discriminator between "Windows re-enumerates a monitor each cycle" (devnode churn the
/// `pnp_disable_monitors` axis suppresses) and "the disturbance is below the OS" (GPU driver
/// servicing a standby sink / display-poller software).
stalls_seen: u32,
stalls_with_os_events: u32,
/// Host-owned ROTATING output ring NVENC encodes (one YUV texture per slot). Rotating it per frame
/// is the precondition for pipelining the encode loop: while NVENC encodes frame N's texture on the
/// ASIC, frame N+1's convert writes a DIFFERENT texture — the two overlap. Format = `out_format()`:
@@ -886,6 +534,9 @@ impl IddPushCapturer {
want_444: bool,
keepalive: Box<dyn Send>,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime.
crate::display_events::spawn_once();
match Self::open_inner(target, preferred, client_10bit, want_444) {
Ok(mut me) => {
me._keepalive = keepalive;
@@ -1015,6 +666,19 @@ impl IddPushCapturer {
// there is no "last known" yet; the descriptor poller corrects a wrong guess mid-session.
let display_hdr = enabled_hdr
|| crate::win_display::advanced_color_enabled(target.target_id).unwrap_or(false);
// Downgrade point D (design/hdr-10bit-default-and-av1.md item 2d): the session was
// NEGOTIATED 10-bit (the client was told HDR in the Welcome), but the virtual display
// could not enable advanced color — the ring sizes SDR and the encoder will emit 8-bit
// BT.709, so the client's label overstates the stream until the descriptor poller sees
// HDR come on. Loud, because every frame of this session is affected.
if client_10bit && !display_hdr {
tracing::error!(
target = target.target_id,
"IDD push: 10-bit HDR was negotiated but enabling advanced color on the \
virtual display FAILED encoding 8-bit SDR while the client was told HDR \
(check the display driver / Windows HDR support on this box)"
);
}
let ring_fmt = if display_hdr {
DXGI_FORMAT_R16G16B16A16_FLOAT
} else {
@@ -1146,6 +810,8 @@ impl IddPushCapturer {
last_liveness: Instant::now(),
last_kick: Instant::now(),
stall_watch: StallWatch::new(),
stalls_seen: 0,
stalls_with_os_events: 0,
out_ring: Vec::new(),
out_idx: 0,
video_conv: None,
@@ -1173,9 +839,12 @@ impl IddPushCapturer {
/// Requiring the first frame — not just the attach — catches the *reconnect-into-a-broken-state* case:
/// a fullscreen game can leave the virtual display in a format/size that the driver's `publish()` guard
/// rejects, so the driver ATTACHES but silently drops every frame; without this the host sails past
/// `open()` and only dies on `next_frame`'s 20 s deadline (the "reconnect = black + audio" symptom). At
/// session open the OS activates the virtual display → DWM composites it → a frame arrives within ~1 s,
/// so this does not false-fail a normal (even idle) open; no frame within the window = genuinely broken.
/// `open()` and only dies on `next_frame`'s 20 s deadline (the "reconnect = black + audio" symptom).
/// A stash-capable driver republishes its retained desktop frame the moment it attaches (the
/// first-frame guarantee — `FrameStash`, driver frame_transport.rs), so the normal case clears this
/// gate in milliseconds even on an idle desktop; failing that, at session open the OS activates the
/// virtual display → DWM composites it → a frame arrives within ~1 s, plus the compose-kick fallback
/// below — no frame within the window = genuinely broken.
fn wait_for_attach(&self) -> Result<()> {
// Symmetric host-side binding sanity (proto v3 §3.2): OUR header must still name OUR
// monitor. The stamp is ours and nothing legitimate rewrites it, so a mismatch means a
@@ -1192,11 +861,15 @@ impl IddPushCapturer {
);
}
let deadline = Instant::now() + Duration::from_secs(4);
// Compose-kick schedule: DWM only presents a display something DIRTIED, so on an idle
// desktop a perfectly healthy attach sees no first frame (E_PENDING forever) and this gate
// used to fail the session — the "idle desktop → no frames" gotcha (a real client escaped
// it only because its own input soon dirtied the desktop; a headless probe never did).
// Give the natural post-activate compose a moment, then nudge.
// First-frame expectation: a stash-capable driver republishes its retained desktop frame
// the moment it attaches (`FrameStash`, frame_transport.rs), so on a healthy pairing the
// gate below clears in milliseconds even on a perfectly idle desktop. The compose-kick
// schedule is the FALLBACK for pre-stash drivers / an empty stash (a display that has
// never composed): DWM only presents a display something DIRTIED, so on an idle desktop
// an attach would otherwise sit at E_PENDING forever and fail this gate — the
// "idle desktop → no frames" gotcha. Give the natural post-activate compose (and the
// stash republish) a moment, then nudge; log when we do, so field logs show whether the
// stash path is working.
let mut next_kick = Instant::now() + Duration::from_millis(600);
loop {
// SAFETY: `self.header` points into the live shared-header mapping this capturer owns (sized
@@ -1249,6 +922,14 @@ impl IddPushCapturer {
return Ok(());
}
if Instant::now() >= next_kick {
// Reaching a kick at all means the driver did NOT republish a retained frame
// (pre-stash driver, or a never-composed display) — worth a line in the field log.
tracing::debug!(
target_id = self.target_id,
driver_status = st,
"IDD push: no first frame after attach delivery — falling back to a synthetic \
compose kick (stash-capable drivers republish instantly; old driver?)"
);
kick_dwm_compose(self.target_id);
next_kick = Instant::now() + Duration::from_millis(800);
}
@@ -1546,12 +1227,19 @@ impl IddPushCapturer {
}
// Same idle-desktop stall as the open-time attach gate: after a mid-session ring
// recreate (HDR flip / mode change) an idle desktop composes nothing, so the fresh ring
// never sees a frame and the 3 s recover-or-drop above kills a healthy session. Nudge
// DWM (rate-limited) once the natural post-recreate compose has had its chance.
// never sees a frame and the 3 s recover-or-drop above kills a healthy session. A
// stash-capable driver republishes its retained frame at the re-attach, so this kick
// is the legacy-driver fallback here too. Nudge DWM (rate-limited) once the natural
// post-recreate compose (and the stash republish) has had its chance.
if since.elapsed() > Duration::from_millis(600)
&& self.last_kick.elapsed() > Duration::from_millis(800)
{
self.last_kick = Instant::now();
tracing::debug!(
target_id = self.target_id,
"IDD push: no frame after ring recreate — falling back to a synthetic compose \
kick (stash-capable drivers republish at re-attach; old driver?)"
);
kick_dwm_compose(self.target_id);
}
}
@@ -1646,24 +1334,70 @@ impl IddPushCapturer {
// doesn't read as a DWM stall.
self.stall_watch.reset();
} else if let Some(stall) = self.stall_watch.note_fresh(now) {
// OS display events inside the gap (plus a lead-in margin: the event that CAUSED the
// hole lands just before DWM stops delivering) — the attribution that turns "DWM
// stopped composing" into "…because Windows re-enumerated SAMSUNG on HDMI".
let window = stall.gap + Duration::from_millis(300);
let events = now
.checked_sub(window)
.map(|from| crate::display_events::events_between(from, now))
.unwrap_or_default();
self.stalls_seen = self.stalls_seen.saturating_add(1);
if !events.is_empty() {
self.stalls_with_os_events = self.stalls_with_os_events.saturating_add(1);
}
// debug (not warn): a single hole also happens when content legitimately pauses;
// the reportable signal is the metronomic cycle below. Mounjay-class triage runs
// at debug level, and the web-console debug ring captures these.
tracing::debug!(
gap_ms = stall.gap.as_millis() as u64,
os_display_events = %crate::display_events::summarize(&events),
"IDD-push capture stall — the desktop was composing at speed, then DWM \
delivered no frame for the gap; the present path stalled below capture"
);
if let Some(period) = stall.metronomic {
let suspects = crate::display_events::connected_inactive_externals();
let suspects = if suspects.is_empty() {
"none".to_string()
} else {
suspects.join(", ")
};
let correlated = format!("{}/{}", self.stalls_with_os_events, self.stalls_seen);
// Half-or-more of the stalls carrying a coinciding OS event = the reaction
// cascade is OS-visible; otherwise the disturbance never surfaces above the
// driver. Different classes, different cures — say which one this box has.
if self.stalls_with_os_events * 2 >= self.stalls_seen {
tracing::warn!(
period_s = format!("{:.2}", period.as_secs_f64()),
"capture stalls are METRONOMIC — DWM stops composing the virtual display \
on a stable period, i.e. a periodic display-path disturbance BELOW \
capture (DWM present clock / GPU driver / display-poller software). \
Correlate with 'slow display-descriptor poll'; if that never fires, the \
disturbance is outside punktfunk try display topology=primary or \
extend (keep a physical output active), or a different refresh rate"
os_correlated = correlated,
connected_inactive = %suspects,
"capture stalls are METRONOMIC and coincide with Windows monitor \
hot-plug/re-enumeration events a connected display (or its \
cable/switch/AVR) re-probes the link on a timer and Windows re-reacts \
each time. Cures, best-first: that display's OSD 'auto input \
scan/detect' OFF (and on TVs: instant-on/quick-start + CEC off), \
unplug its cable at the GPU, an HPD-holding adapter/dummy plug, or \
keep it active while streaming; the pnp_disable_monitors policy axis \
suppresses the Windows-side reaction (see connected_inactive for the \
suspects)"
);
} else {
tracing::warn!(
period_s = format!("{:.2}", period.as_secs_f64()),
os_correlated = correlated,
connected_inactive = %suspects,
"capture stalls are METRONOMIC with NO coinciding OS display event — \
the disturbance is BELOW Windows: the GPU driver servicing a \
connected-but-asleep sink (standby HPD/DDC/link probing), \
display-poller software (the SteelSeries-GG/SignalRGB class \
correlate 'slow display-descriptor poll' lines), or the DWM present \
clock (try a different refresh rate). If connected_inactive lists a \
display, its standby probing is the prime suspect: unplug it at the \
GPU, disable its OSD auto input scan (TVs: instant-on/quick-start + \
CEC off), use an HPD-holding adapter/dummy, or keep it active while \
streaming"
);
}
}
}
self.last_fresh = now; // feeds the driver-death watch
@@ -1820,6 +1554,7 @@ impl Drop for IddPushCapturer {
#[cfg(test)]
mod tests {
use super::stall::Stall;
use super::*;
/// Feed a [`StallWatch`] fresh frames at the given offsets (ms from a common origin) and
@@ -0,0 +1,198 @@
//! The sealed frame channel's handle-duplication broker (plan §W4, carved out of the IDD-push
//! capturer): duplicates the unnamed shared header / ring / event handles into the driver's WUDFHost
//! and delivers them as bare handle values over the SYSTEM-only control device.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
/// The sealed channel's handle-duplication broker (`design/idd-push-security.md`): the frame objects
/// are unnamed, so the ONLY way the driver can reach them is handles this broker duplicates into its
/// WUDFHost process and delivers — as bare handle VALUES — over the SYSTEM-only control device
/// (`IOCTL_SET_FRAME_CHANNEL`). Ownership is a strict hand-off: on IOCTL success the DRIVER owns the
/// duplicates (it closes them); on any failure [`Self::send`] reaps every duplicate it already made
/// (`DUPLICATE_CLOSE_SOURCE`), so a half-delivered channel never leaks handles in WUDFHost.
pub(super) struct ChannelBroker {
/// `PROCESS_DUP_HANDLE | SYNCHRONIZE` handle to the driver's WUDFHost (pid from the ADD reply;
/// `ProcessSharingDisabled` makes that process exclusively pf-vdisplay's). `SYNCHRONIZE` lets the
/// handle double as the driver-death probe ([`Self::driver_alive`]).
process: OwnedHandle,
/// The WUDFHost pid `process` refers to (diagnostics for the driver-death bail).
pub(super) wudf_pid: u32,
/// The pf-vdisplay control device — owned by the `VirtualDisplayManager`, never closed for the
/// process lifetime (a dead one is retired, kept alive), so holding the bare `HANDLE` is sound.
control: HANDLE,
}
impl ChannelBroker {
/// Open the duplication target. Fails when the driver predates the sealed channel (`wudf_pid == 0`
/// can't survive the v2 version handshake, but guard anyway) or the WUDFHost is gone (device
/// restart mid-open) — either way the caller fails the capture open cleanly.
///
/// `wudf_pid` comes from the driver's ADD reply, so before we duplicate whole-desktop frame handles
/// INTO it we VERIFY it is a genuine system WUDFHost ([`verify_is_wudfhost`]). Without that check a
/// spoofed devnode (same interface GUID) could name an arbitrary process and receive the frames; a
/// fully-compromised REAL pf_vdisplay driver is already a frame endpoint, so this specifically closes
/// the reachable-without-owning-the-driver case (`design/idd-push-security.md` §hardening).
pub(super) fn open(wudf_pid: u32) -> Result<Self> {
if wudf_pid == 0 {
bail!("driver reported no WUDFHost pid for the frame channel");
}
let control = crate::vdisplay::manager::control_device_handle().context(
"pf-vdisplay control device not open (monitor not created via the manager?)",
)?;
// SAFETY: plain FFI; `wudf_pid` is a copy. The handle (checked by `?`) is owned solely here and
// moved into the `OwnedHandle` (single owner, closes on drop); `verify_is_wudfhost` borrows it
// for the duration of the synchronous check and forms no lasting alias.
let process = unsafe {
let h = OpenProcess(
PROCESS_DUP_HANDLE | PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_SYNCHRONIZE,
false,
wudf_pid,
)
.context("OpenProcess(PROCESS_DUP_HANDLE) on the driver's WUDFHost")?;
let process = OwnedHandle::from_raw_handle(h.0 as _);
verify_is_wudfhost(HANDLE(process.as_raw_handle()), wudf_pid, "frame-channel")?;
process
};
Ok(Self {
process,
wudf_pid,
control,
})
}
/// Whether the driver's WUDFHost is still alive. The pinned process handle doubles as the
/// liveness probe (`SYNCHRONIZE` requested at open): signaled ⇔ the process exited. This is the
/// definitive "driver died mid-session" signal — at the ring, a dead driver and an idle desktop
/// are indistinguishable (both simply stop publishing).
pub(super) fn driver_alive(&self) -> bool {
// SAFETY: `process` is the live `OwnedHandle` this broker owns (borrowed for this synchronous
// call); a 0 ms wait only reads the handle's signaled state.
unsafe { WaitForSingleObject(HANDLE(self.process.as_raw_handle()), 0) != WAIT_OBJECT_0 }
}
/// Duplicate `h` into the WUDFHost handle table, returning the handle VALUE valid there (and only
/// there — the value is meaningless in any other process). `access = Some(rights)` grants the
/// driver's handle exactly those rights (least privilege — see [`SECTION_MAP_RW`]);
/// `access = None` copies the source handle's access (`DUPLICATE_SAME_ACCESS`), used only where the
/// source is already scoped (the DXGI shared-texture handles, minted by `CreateSharedHandle` with
/// just `DXGI_SHARED_RESOURCE_READ|WRITE`).
///
/// # Safety
/// `h` must be a live handle of the current process.
unsafe fn dup_into(&self, h: HANDLE, access: Option<u32>) -> Result<u64> {
let mut out = HANDLE::default();
let (desired, options) = match access {
Some(rights) => (rights, DUPLICATE_HANDLE_OPTIONS(0)),
None => (0, DUPLICATE_SAME_ACCESS),
};
// SAFETY: `h` is live per the contract; `self.process` is the live PROCESS_DUP_HANDLE target;
// `&mut out` is a valid out-param. Either an explicit least-privilege access mask (options == 0)
// or `DUPLICATE_SAME_ACCESS` (desired ignored) — never both.
unsafe {
DuplicateHandle(
GetCurrentProcess(),
h,
HANDLE(self.process.as_raw_handle()),
&mut out,
desired,
false,
options,
)
}
.context("DuplicateHandle into the driver's WUDFHost")?;
Ok(out.0 as usize as u64)
}
/// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE`
/// with no target closes the source handle regardless of the (ignored) result.
fn close_remote(&self, value: u64) {
if value == 0 {
return;
}
// SAFETY: `self.process` is the live duplication target and `value` is a handle value THIS
// broker just created in that process's table (callers only pass back `dup_into` results the
// driver never received); closing it there cannot touch any other process's handles.
unsafe {
let _ = DuplicateHandle(
HANDLE(self.process.as_raw_handle()),
HANDLE(value as usize as *mut core::ffi::c_void),
HANDLE::default(),
std::ptr::null_mut(),
0,
false,
DUPLICATE_CLOSE_SOURCE,
);
}
}
/// Duplicate the whole ring (header + event + every slot texture) into WUDFHost and deliver the
/// values via `IOCTL_SET_FRAME_CHANNEL`. All-or-nothing: on any failure every duplicate already
/// made is reaped remotely and an error returns (the caller fails the open / logs the recreate).
/// The ownership contract with the driver is adopt-on-success only — it closes the handles iff the
/// IOCTL succeeded, we reap them iff it didn't, so no value is ever closed twice.
///
/// # Safety
/// `header` and `event` must be live handles of the current process (the capturer's own section +
/// event, borrowed for this synchronous call).
pub(super) unsafe fn send(
&self,
target_id: u32,
generation: u32,
header: HANDLE,
event: HANDLE,
slots: &[HostSlot],
) -> Result<()> {
debug_assert!(slots.len() <= control::RING_LEN_USIZE);
let mut req = control::SetFrameChannelRequest {
target_id,
generation,
ring_len: slots.len() as u32,
_pad: 0,
header_handle: 0,
event_handle: 0,
texture_handles: [0; control::RING_LEN_USIZE],
};
// SAFETY: `header`/`event` are live per this fn's contract; each slot's `shared` is the live
// `OwnedHandle` the slot keeps for exactly this purpose.
let result = unsafe { self.duplicate_and_deliver(&mut req, header, event, slots) };
if result.is_err() {
// The driver never adopted the delivery — reap every remote duplicate so nothing lingers.
self.close_remote(req.header_handle);
self.close_remote(req.event_handle);
for v in req.texture_handles {
self.close_remote(v);
}
}
result
}
/// The fallible middle of [`Self::send`]: fill `req` with fresh duplicates, then issue the IOCTL.
/// Split out so `send` can reap whatever landed in `req` when any step errors.
///
/// # Safety
/// As [`Self::send`].
unsafe fn duplicate_and_deliver(
&self,
req: &mut control::SetFrameChannelRequest,
header: HANDLE,
event: HANDLE,
slots: &[HostSlot],
) -> Result<()> {
// SAFETY: forwarded from the caller's contract — `header`/`event`/each `slot.shared` are live
// handles of this process, and `self.control` is the manager's control handle, never closed for
// the process lifetime (`send_frame_channel`'s precondition).
unsafe {
// Least privilege per handle: the header maps read/write, the event is only signalled, and
// the textures keep their already-scoped `CreateSharedHandle` access (see `dup_into`).
req.header_handle = self.dup_into(header, Some(SECTION_MAP_RW))?;
req.event_handle = self.dup_into(event, Some(EVENT_MODIFY_STATE))?;
for (k, s) in slots.iter().enumerate() {
req.texture_handles[k] = self.dup_into(HANDLE(s.shared.as_raw_handle()), None)?;
}
crate::vdisplay::pf_vdisplay::send_frame_channel(self.control, req)
}
}
}

Some files were not shown because too many files have changed in this diff Show More