Compare commits

..
Author SHA1 Message Date
enricobuehlerandClaude Opus 4.8 ec219763a6 fix(ci/release): make _release_notes_path POSIX-sh safe (deb/decky attach runs under dash)
ci / web (push) Successful in 55s
apple / swift (push) Successful in 1m26s
ci / docs-site (push) Successful in 56s
android-screenshots / screenshots (push) Successful in 3m20s
ci / bench (push) Successful in 6m47s
release / apple (push) Successful in 10m49s
android / android (push) Successful in 15m8s
decky / build-publish (push) Successful in 26s
apple / screenshots (push) Successful in 6m40s
deb / build-publish (push) Successful in 9m36s
deb / build-publish-host (push) Successful in 9m39s
flatpak / build-publish (push) Successful in 6m33s
arch / build-publish (push) Successful in 15m57s
ci / rust (push) Successful in 26m4s
docker / deploy-docs (push) Successful in 16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
web-screenshots / screenshots (push) Successful in 3m10s
windows-host / package (push) Successful in 10m33s
linux-client-screenshots / screenshots (push) Successful in 7m51s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m12s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m3s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m18s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m57s
The v0.19.0 deb + decky release-attach steps source gitea-release.sh under `sh` (dash) and
died with "Bad substitution" at _release_notes_path's `${BASH_SOURCE[0]:-$0}` — a bash array
subscript dash rejects. So those legs never attached their assets to the release (the bash
legs — apple, android, rpm — were fine, which is why the body still seeded and the DMG/.ipa
attached). CI always sources this from the repo root, so resolve the notes as
docs/releases/<tag>.md relative to CWD and drop $BASH_SOURCE entirely. Verified under dash.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 19:03:01 +02:00
enricobuehlerandClaude Opus 4.8 0dce1cd116 chore(release): bump workspace version to 0.19.0
audit / bun-audit (push) Successful in 11s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 54s
apple / swift (push) Successful in 1m20s
audit / cargo-audit (push) Successful in 2m13s
ci / bench (push) Successful in 6m52s
android-screenshots / screenshots (push) Successful in 3m58s
release / apple (push) Successful in 11m11s
deb / build-publish (push) Failing after 8m56s
decky / build-publish (push) Failing after 24s
deb / build-publish-host (push) Failing after 9m15s
android / android (push) Successful in 13m6s
ci / rust (push) Successful in 25m44s
arch / build-publish (push) Successful in 13m37s
docker / deploy-docs (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
apple / screenshots (push) Successful in 6m32s
web-screenshots / screenshots (push) Successful in 3m2s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 2m43s
flatpak / build-publish (push) Failing after 8m20s
linux-client-screenshots / screenshots (push) Successful in 7m46s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m45s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m4s
windows-host / package (push) Successful in 15m54s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m26s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m49s
Headline: the whole pen/tablet input stack (P0–P5) — a pressure-sensitive stylus plane
across core wire, Linux, GameStream/Moonlight, Windows, iPad Pencil and Android. Plus
touch-injection fixes, cursor-DPI scaling, three Windows-host display/cursor fixes, a
gamepad same-PID conflict fix, and the Android mouse/keyboard (TV) regressions.

Release notes authored in-repo at docs/releases/v0.19.0.md (first release on the new flow;
CI seeds the release body from it). cargo update --workspace syncs the lock (versions only).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:26:49 +02:00
enricobuehlerandClaude Opus 4.8 c3c0115a82 fix(core/abi): bump ABI_VERSION to 13 for punktfunk_connection_send_pen
The pen P0 work added a new embeddable C entry point (punktfunk_connection_send_pen)
but left ABI_VERSION at 12 — the value 0.18.0 shipped. Per the project's convention
(0.18.0 bumped to 12 for the additive cursor channel), additive C-surface growth bumps
the ABI so embedders can gate on the new symbol. Regenerated include/punktfunk_core.h
via cbindgen (cargo build -p punktfunk-core); WIRE_VERSION is unchanged (pen is
capability-gated on HOST_CAP_PEN).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:23:38 +02:00
enricobuehlerandClaude Opus 4.8 00c8c12a0f feat(ci/apple): export the iOS .ipa to the release + run artifacts
The iOS leg archives with development signing and re-signs for App Store distribution at
export time, but the existing export uses destination=upload (straight to TestFlight) and
leaves no .ipa on disk. Add an export step that re-exports the SAME archive with
destination=export to produce an App Store distribution-signed .ipa, then:
  * attach it as the `punktfunk-ios-ipa` run artifact on every build (upload-artifact@v3,
    since Gitea's backend rejects @v4), and
  * attach it to the unified Gitea release on vX.Y.Z tags (alongside the DMG), via the
    shared ensure_release/upsert_asset helpers.

Same gate as the archive; warn+skip (never fails the best-effort iOS leg) if the archive is
absent. Note: an App Store-signed .ipa installs only via TestFlight/App Store, not by direct
sideload — it is a release/archival artifact.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:18:45 +02:00
enricobuehlerandClaude Fable 5 988b5742ec fix(gamestream): tolerate NaN pressureOrDistance — VoidLink's finger touches were dropped whole
apple / swift (push) Successful in 1m18s
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m6s
apple / screenshots (push) Successful in 6m11s
ci / bench (push) Successful in 5m34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
decky / build-publish (push) Successful in 27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
windows-host / package (push) Successful in 16m57s
android / android (push) Successful in 15m58s
docker / deploy-docs (push) Successful in 24s
arch / build-publish (push) Successful in 17m3s
deb / build-publish (push) Successful in 9m26s
deb / build-publish-host (push) Successful in 10m5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m39s
ci / rust (push) Successful in 25m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m31s
The hex dumps from the on-glass session show textbook SS_TOUCH packets with
pressureOrDistance = 0x7fc00000 (NaN): VoidLink encodes 'unknown finger
pressure' as NaN (iPad fingers have no force sensor), and the anti-forgery
finiteness gate rejected the entire packet — silently disabling the touch
plane while pen (real Pencil force, always finite) worked. Coordinates keep
the strict gate (they feed injector scaling); pressureOrDistance now
sanitizes non-finite to 0.0, the spec's own 'unknown', which the pen path
already maps to full-scale ink on contact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 f938174d86 debug(gamestream): hex-dump the first few undecodable pointer packets
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 40611cd54d debug(gamestream): surface the pointer plane's silent failure modes
An on-glass 'touch does nothing' currently leaves NO trace: log the first
SS_TOUCH per session at info (proves the client sends + we decode), and warn
when a packet carrying a pointer magic fails the body parse (layout mismatch)
instead of vanishing into the unknown-magic drop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 98b97d7d76 fix(android/input): stop mouse back/forward leaking to nav + IME popping on hardware typing
apple / swift (push) Successful in 1m25s
apple / screenshots (push) Successful in 7m4s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m14s
android / android (push) Successful in 11m54s
arch / build-publish (push) Successful in 11m58s
ci / bench (push) Successful in 5m36s
decky / build-publish (push) Successful in 24s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
deb / build-publish (push) Successful in 11m52s
docker / deploy-docs (push) Successful in 11s
deb / build-publish-host (push) Successful in 10m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m15s
ci / rust (push) Successful in 26m1s
Two Android TV M&K regressions from the c90343c2 overhaul, both user-reported:

1. Mouse back/forward buttons doubled as Android navigation while streaming.
   The BUTTON_BACK/FORWARD motion edges already go over the wire as X1/X2, but
   Android's input reader ALSO synthesizes KEYCODE_BACK/FORWARD key events from
   those buttons unconditionally (stamped SOURCE_MOUSE, no FLAG_FALLBACK) — and
   the guard only swallowed the FLAG_FALLBACK variant, so the duplicate reached
   the BackHandler and yanked the user out of the stream. Swallow mouse-sourced
   BACK/FORWARD too; a remote/keyboard BACK is never mouse-sourced and still
   exits.

2. The soft keyboard popped up on physical-keyboard typing. KeyCaptureView (the
   always-focused IME/capture anchor) unconditionally claimed to be a text
   editor with a live editable InputConnection, so the TV IME counted input as
   active and showed its UI the moment hardware keys arrived. Gate the editor
   identity on imeShown (the user actually summoning the keyboard via gesture /
   remote toggle), with restartInput on both edges so the gate is re-read, and
   an onKeyPreIme BACK hook so an IME self-dismissal can't leave the gate
   stale.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:07:18 +02:00
enricobuehlerandClaude Fable 5 51792536d1 fix(pf-win-display): escalate CCD isolate to a keep-only supplied config on 0x57
apple / swift (push) Successful in 1m30s
ci / web (push) Successful in 1m8s
apple / screenshots (push) Successful in 6m59s
ci / docs-site (push) Successful in 1m6s
android / android (push) Successful in 14m35s
arch / build-publish (push) Successful in 14m20s
decky / build-publish (push) Successful in 21s
ci / bench (push) Successful in 6m4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 18s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 18s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
deb / build-publish (push) Successful in 9m41s
ci / rust (push) Successful in 20m14s
deb / build-publish-host (push) Successful in 11m30s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m42s
docker / deploy-docs (push) Successful in 12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m2s
Field bug (Steam Deck client, exclusive topology): every isolate retry failed
SetDisplayConfig 0x57 with the physical panel left lit, while iPhone sessions
worked — their virtual monitor's connected-set already had a sole-display
topology persisted, so Windows deactivated the physical on hotplug and the
isolate was a no-op re-commit (deactivated 0). The Deck's monitor identity had
no such entry, forcing the live-deactivate path: supplying the doomed path
(inactive, modes unpinned) plus its orphaned mode entries is rejected 0x57 by
some driver/OS validation combos, so the retry loop could never converge — the
prior mode-index unpin (5e088af7) was necessary but not sufficient on this box.

Attempt 1 keeps the shipped shape; once a survivor is seen, later attempts
supply ONLY the keep paths with the mode table rebuilt to just their referenced
entries (indexes remapped, clone-shared entries deduped). The same host applies
exactly this keep-only shape rc=0 whenever the DB has pre-isolated the virtual
display, so the escalated shape is field-validated on the failing machine. The
final attempt also drops SDC_FORCE_MODE_ENUMERATION — a real path removal
drives COMMIT_MODES on its own.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:50:44 +02:00
enricobuehlerandClaude Fable 5 bf2c4456bb fix(inject/windows): compact wire touch ids into slots — Moonlight native touch injected nothing
apple / swift (push) Successful in 1m28s
ci / web (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m12s
release / apple (push) Successful in 9m24s
ci / bench (push) Successful in 6m28s
android / android (push) Successful in 11m59s
decky / build-publish (push) Successful in 22s
arch / build-publish (push) Successful in 12m30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 15s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
apple / screenshots (push) Successful in 6m53s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 19s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 17s
deb / build-publish (push) Successful in 9m36s
deb / build-publish-host (push) Successful in 11m24s
ci / rust (push) Successful in 25m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m53s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m59s
docker / deploy-docs (push) Successful in 25s
On-glass report (iPad Moonlight vs the Windows host): pen inked, native touch
did nothing. Cause: POINTER_INFO.pointerId was the client's raw wire id —
Moonlight sends arbitrary large pointerIds, and synthetic-pointer injection
rejects them (Apollo compacts ids into dense slots for exactly this reason);
the failure was logged at trace, i.e. invisibly. Wire ids now map to the
lowest free slot for the contact's lifetime, and the FIRST injection failure
of a device logs at WARN (pen too) so an inert input plane is never silent
again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Fable 5 c0f792ee8d feat(android): pen P5 — stylus capture onto the pen plane
The Android leg of design/pen-tablet-input.md §7: against a HOST_CAP_PEN host,
stylus/eraser pointers split out of BOTH touch models (passthrough + gesture)
into StylusStream — state-full samples with pressure, AXIS_TILT, azimuth from
AXIS_ORIENTATION (Android's 0 = away-from-user IS wire north), AXIS_DISTANCE
hover, both stylus barrel buttons, the eraser tool, and historical (coalesced)
samples batched oldest-first. Kotlin heartbeats ≤100ms per the wire contract.
JNI: nativeHostSupportsPen + nativeSendPen (flat 10-float stride, sentinels
<0). No barrel-roll axis exists on Android — roll stays unknown here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Fable 5 c8aa3ed48b feat(apple/ios): pen P4 — Apple Pencil (Pro) capture onto the stylus plane
The flagship leg of design/pen-tablet-input.md §7: against a HOST_CAP_PEN host
the Pencil splits out of the finger path (independent of touch-input mode) into
PencilStream — state-full samples with force→pressure, altitude→polar tilt,
azimuth (+90° to the wire's north-clockwise convention), Pencil Pro rollAngle→
barrel roll (17.5+), coalesced touches batched ≤8/send for full 240Hz fidelity,
and Pencil hover (zOffset>0 distinguishes it from trackpad hover) as in-range
samples with distance. Squeeze holds barrel 1, double-tap clicks barrel 2 (the
Pencil has no hardware buttons/eraser — these are how host apps get those
affordances). Implements the ≤100ms heartbeat wire contract (80ms timer) so a
stationary held stroke survives the host's 200ms dead-client failsafe. Toward a
pen-less host nothing changes (Pencil stays a finger).

Also fixes hostSupportsCursor testing the STALE 0x04 bit — HOST_CAP_CURSOR
moved to 0x08 when TEXT_INPUT claimed 0x04; the old test would mistake a
text-input host (e.g. Windows) for a cursor grant and double-render the pointer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Opus 4.8 15d09a8c10 feat(ci/release): author release notes in-repo + announce stable releases to Discord
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m25s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
ci / bench (push) Successful in 6m55s
apple / screenshots (push) Successful in 6m47s
deb / build-publish (push) Successful in 11m23s
android / android (push) Successful in 12m36s
arch / build-publish (push) Successful in 12m46s
docker / deploy-docs (push) Successful in 23s
deb / build-publish-host (push) Successful in 12m32s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m9s
ci / rust (push) Successful in 26m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m30s
Release notes now live in the repo at docs/releases/vX.Y.Z.md, authored during the
version bump (before the tag). ensure_release (gitea-release.sh + .ps1 twin) seeds the
Gitea release body from that file in the create POST, so the release is born WITH its
notes instead of being created empty and PATCHed afterward. canary/rc have no file ->
empty body, unchanged.

New manual `announce` workflow (workflow_dispatch, tag input) re-syncs the notes file
over the live release and posts a violet embed (title, notes lead-in, release-page link)
to the Discord #releases channel via the DISCORD_RELEASE_WEBHOOK secret. Stable-only:
a -rc tag is refused unless allow_prerelease=true. Pressing "go" is the quality gate, so
a half-built or failed release is never announced.

docs/releases/README.md documents the ritual; TEMPLATE.md is the skeleton.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 16:43:18 +02:00
enricobuehlerandClaude Opus 4.8 6cbc3eec8b fix(sdl/cursor): scale the forwarded pointer to the video fit
apple / swift (push) Successful in 1m30s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m3s
android / android (push) Canceled after 1m27s
apple / screenshots (push) Canceled after 1m39s
arch / build-publish (push) Canceled after 1m29s
ci / rust (push) Canceled after 1m29s
ci / bench (push) Canceled after 1m17s
deb / build-publish (push) Canceled after 1m15s
deb / build-publish-host (push) Canceled after 27s
decky / build-publish (push) Canceled after 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 15s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 1m18s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 8s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 18s
flatpak / build-publish (push) Canceled after 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / deploy-docs (push) Canceled after 0s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 17s
Parity with the Apple client (35c2c145): the SDL/Linux presenter built the
host-forwarded cursor as an OS cursor at its native bitmap size, so the
on-screen pointer tracked the HOST's display scaling — a 4K/high-DPI virtual
display forwards a 96 px bitmap that then rendered huge on the client.

SDL cursors are fixed-size from their surface (no draw-time scaling), so
cache the shapes RAW and resample each by the live video-fit factor
(min(window_px / mode) — the same aspect-fit `finger_to_content` uses) at
install time, rebuilding when the serial OR the fit changes (a resize). The
resample area-averages on premultiplied alpha so a transparent pixel's
colour can't bleed into the fringe. The pointer now stays sized to the
streamed desktop at any host scaling.

Not yet eyeballed on a Deck/Linux box — SDL's HiDPI cursor sizing is
version-dependent; validate on glass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 16:28:34 +02:00
enricobuehlerandClaude Fable 5 45c9799aa6 feat(inject/windows): pen P3 — PT_PEN + PT_TOUCH synthetic pointer injection
apple / swift (push) Successful in 1m26s
ci / web (push) Successful in 59s
ci / docs-site (push) Successful in 54s
release / apple (push) Successful in 9m40s
arch / build-publish (push) Successful in 12m4s
ci / bench (push) Successful in 7m5s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 22s
apple / screenshots (push) Successful in 6m50s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
android / android (push) Successful in 14m47s
ci / rust (push) Canceled after 13m52s
deb / build-publish (push) Canceled after 7m26s
deb / build-publish-host (push) Canceled after 5m38s
flatpak / build-publish (push) Canceled after 1m55s
docker / deploy-docs (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 1m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 1m59s
windows-host / package (push) Canceled after 8m32s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 0s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
The Windows leg of design/pen-tablet-input.md §6, following Apollo's recipe:
a per-session PT_PEN device (pressure rescaled 0..1024, polar tilt→tiltX/Y,
barrel roll on rotation — Windows Ink renders Pencil Pro roll natively, barrel
button, eraser via INVERTED/ERASER flags, hover) with a 40ms refresh thread
against the ~100ms synthetic-pointer staleness auto-lift, plus a PT_TOUCH
device closing the historical SendInput wire-touch no-op (full active-contact
frames, per-id DOWN/UP edges, self-healing lost-DOWN synthesis). pen_supported
now probes PT_PEN creation on Windows (1809+), which lights up HOST_CAP_PEN
and the GameStream featureFlags there — Moonlight iPad + Pencil can ink on a
Windows host. Frame grouping mirrors the Linux uinput backend.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 b192825869 feat(gamestream): pen P2 — SS_PEN/SS_TOUCH ingest, featureFlags advertised on capable hosts
The Moonlight leg of design/pen-tablet-input.md §4: DESCRIBE now advertises
SS_FF_PEN_TOUCH_EVENTS wherever pen injection exists (Linux+uinput, same gate
as HOST_CAP_PEN — elsewhere the flag stays 0 and Moonlight keeps client-side
mouse emulation exactly as today). SS_PEN packets merge over last-known state
into state-full PenSamples (BUTTON_ONLY per spec carries no position; unknown
contact pressure 0.0 inks at full scale; UP keeps proximity only for clients
that demonstrated hover) and drive the same PenTracker→VirtualPen chain as the
native plane — Moonlight iOS + Apple Pencil now exercises the full injection
path end to end. SS_TOUCH forwards as wire touch on a synthetic 65535² surface
with CANCEL_ALL replayed per tracked contact. No stroke timeout on this plane:
ENet is ordered/reliable and Moonlight doesn't heartbeat a stationary pen.
Packet layouts pinned against moonlight-common-c Input.h/Limelight.h upstream.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 aca5aa9993 feat(host+inject): pen P1 — per-session uinput virtual tablet, HOST_CAP_PEN live
The Linux injection leg of design/pen-tablet-input.md: 0xCC/0x05 pen batches now
route through the per-session PenTracker into a lazily-created 'Punktfunk Pen'
uinput tablet (BTN_TOOL_PEN/RUBBER, pressure, tilt-from-polar, ABS_Z barrel
roll, hover distance, INPUT_PROP_DIRECT) — compositors pick it up via libinput
and hand apps zwp_tablet_v2 with full fidelity. The host now advertises
HOST_CAP_PEN when /dev/uinput is accessible (PUNKTFUNK_PEN=0 kill-switch);
transitions group into SYN frames so proximity-enter carries its position.
Stroke failsafe: clients heartbeat ≤100ms while in range (documented wire
contract — capture APIs are silent for a stationary pen); 200ms of silence
force-releases. 'punktfunk-host pen-test' draws a pressure-ramped sine stroke
through the real tracker→uinput chain, no client needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 248e1cbf08 fix(pf-vdisplay/driver): report cursor_excluded adapter-wide — the declare's exclusion is not per-target
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m20s
windows-drivers / probe-and-proto (push) Successful in 1m2s
decky / build-publish (push) Successful in 28s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 21s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
windows-drivers / driver-build (push) Successful in 2m23s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m5s
ci / bench (push) Successful in 6m3s
apple / screenshots (push) Successful in 6m41s
deb / build-publish (push) Successful in 12m20s
arch / build-publish (push) Successful in 12m47s
deb / build-publish-host (push) Successful in 13m7s
android / android (push) Successful in 16m14s
windows-host / package (push) Successful in 15m53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m8s
ci / rust (push) Successful in 26m6s
docker / deploy-docs (push) Canceled after 25s
Field report (iPad via GameStream, 2026-07-23): cursor missing from the
stream. The session's fresh target 257 reported cursor_excluded=false
(DECLARED_TARGETS is keyed per target and only 259 had declared), so the
host skipped forced-composite — but DWM's pointer exclusion after an
IddCx hardware-cursor declare reaches EVERY later monitor of the adapter,
not just the declaring target (proven on-glass: declare on 259 under
WUDFHost 2192, then 257's frames streamed pointer-free with CURSOR_SHOWING
set and SM_MOUSEPRESENT true). GameStream/Moonlight clients cannot draw a
forwarded cursor, so those sessions were silently cursor-less.

ADD replies now report any-declare-this-WUDFHost-life (the state's real
scope — it dies with the adapter reset, same as before); per-target ids
stay only as the dbglog audit trail. Channel-less sessions on a declared
adapter then hit the existing forced-composite gate (GDI poller + blend)
and carry the pointer in-video. Host code unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 15:58:54 +02:00
enricobuehlerandClaude Opus 4.8 35c2c1450e fix(apple/cursor): scale the forwarded pointer to the video fit
apple / swift (push) Successful in 1m24s
release / apple (push) Successful in 10m11s
apple / screenshots (push) Successful in 6m42s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m9s
ci / bench (push) Successful in 5m48s
decky / build-publish (push) Successful in 25s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 20s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 18s
android / android (push) Successful in 16m24s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
deb / build-publish (push) Successful in 10m11s
arch / build-publish (push) Successful in 12m18s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7m5s
docker / deploy-docs (push) Successful in 11s
deb / build-publish-host (push) Successful in 11m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m4s
ci / rust (push) Successful in 18m39s
The Mac client drew the host-forwarded cursor bitmap at its native pixel
size (pixel-as-points), so the on-screen pointer tracked the HOST's display
scaling: a 4K/high-DPI virtual display renders a 96 px pointer, which then
appeared huge on the client — and on retina every pointer was already
2x-inflated and upscaled-blurry, since the backing scale was ignored.

Cache the shape raw and rebuild the NSCursor at the live video-fit factor
(fit.width / mode.width — the same AVMakeRect fit the input mapping uses)
on each apply, re-fitting on resize / retina moves. The pointer now renders
at its true size relative to the streamed desktop at any host scaling:
crisp and 1:1 with the video at 100%, and proportional (not ballooning)
when the host sits at 200-300% — including a deliberately raised scale.

iOS/tvOS (UIPointerInteraction) and Android (no forwarded-bitmap path)
render the pointer differently and need no change; the SDL/Linux client has
the same native-size issue and will follow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 14:14:41 +02:00
enricobuehlerandClaude Fable 5 326d6e17c8 fix(windows/capture): stand the IddCx hardware cursor down while the secure desktop is up
apple / swift (push) Successful in 1m22s
apple / screenshots (push) Successful in 6m41s
ci / web (push) Successful in 1m7s
ci / docs-site (push) Successful in 1m18s
android / android (push) Successful in 12m44s
arch / build-publish (push) Successful in 12m36s
decky / build-publish (push) Successful in 26s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 21s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 20s
ci / bench (push) Successful in 6m14s
windows-host / package (push) Successful in 9m46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 21s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
deb / build-publish (push) Successful in 9m38s
deb / build-publish-host (push) Successful in 9m58s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7m0s
ci / rust (push) Successful in 24m1s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m7s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m55s
docker / deploy-docs (push) Successful in 12s
0.18.0 regression: UAC consent and Winlogon (lock/logon) stopped appearing
in streams. The cursor channel's IddCx hardware-cursor declare — re-issued
by the driver on every swap-chain assign — keeps the path out of the OS's
software-cursor mode, which is the only mode the secure desktop renders
through; DWM then never presents UAC/Winlogon into our swap-chain and the
stream repeats the last normal-desktop frame for the whole interaction.

The GDI cursor poller now classifies the input desktop on its reattach
cadence (UOI_NAME != Default = secure, 250 ms cadence instead of 2 s), and
the capturer edge-triggers the existing-but-unused proto-v6
IOCTL_SET_CURSOR_FORWARD flip: OFF at secure entry (the driver stops its
per-assign re-declare; the host facade forces the same-mode re-commit that
actualises the software-cursor default, under the vdisplay manager lock via
the new force_recommit()), back ON at dismissal for channel sessions.
Channel-session open resets the driver's persisted desired state to ON so
a session that died mid-secure-desktop can't leave the next one adopting
UNdeclared (the §8.6 cross-session composite trap); orderly teardown does
the same. The flip closure is built for every Windows session — a
channel-less session can reuse a driver monitor whose earlier-session
cursor worker still re-declares, and NOT_FOUND from never-declared targets
is ignored.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 14:13:15 +02:00
enricobuehlerandClaude Opus 4.8 588a7077e8 fix(host/gamepad): only degrade a virtual Steam pad on a same-PID conflict
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 1m0s
apple / swift (push) Successful in 1m26s
decky / build-publish (push) Successful in 32s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 23s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 20s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 26s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 16s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
ci / bench (push) Successful in 6m10s
apple / screenshots (push) Successful in 6m38s
deb / build-publish (push) Successful in 9m26s
android / android (push) Successful in 12m27s
deb / build-publish-host (push) Successful in 13m55s
windows-host / package (push) Successful in 16m20s
arch / build-publish (push) Successful in 17m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m2s
ci / rust (push) Successful in 27m20s
docker / deploy-docs (push) Successful in 26s
The Steam-conflict gate matched any physical Valve `28DE` HID device, so a
Steam Machine with a physical Steam Controller 2 (`28DE:1302`) plugged in
degraded a client's requested virtual Steam Deck (`28DE:1205`) to a DualSense
— the reporter's controller passthrough came up as the wrong pad.

Steam Input drives *distinct* Steam controllers side by side; only two
*identical* `28DE` devices confuse it (the Deck-as-host case: built-in
`28DE:1205` + a second virtual Deck, observed live on Bazzite). Narrow the
gate to key on the exact VID+PID instead of the `28DE` vendor alone:

- `steam_backend_product()` maps each virtual Steam backend to the PID it
  enumerates as (Deck 1205, classic SC 1102, SC2 1302, SC2 Puck 1304).
- `physical_steam_controller_present()` -> `physical_steam_product_present(pid)`
  matches `:28DE:{PID}`, keeping the FVPF-serial / `vhci_hcd` / `/virtual/`
  exclusions so our own virtual pads never count.
- `degrade_steam_on_conflict` degrades only on a same-PID duplicate and logs
  the conflicting identity. `PUNKTFUNK_STEAM_FORCE=1` override unchanged.

A physical SC2 no longer blocks a virtual Deck (and vice versa); a genuine
same-identity duplicate (physical Deck + virtual Deck, or physical SC2 +
SC2 passthrough) still degrades. Adds a unit test for the PID mapping.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 14:11:44 +02:00
enricobuehlerandClaude Fable 5 4714235fe6 feat(core): stylus wire P0 — state-full RICH_PEN batches, PenTracker, HOST_CAP_PEN
apple / swift (push) Successful in 1m24s
release / apple (push) Successful in 10m3s
android / android (push) Successful in 13m31s
arch / build-publish (push) Successful in 13m39s
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m12s
windows-host / package (push) Successful in 16m40s
apple / screenshots (push) Successful in 6m38s
ci / bench (push) Successful in 5m26s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m9s
ci / rust (push) Successful in 20m46s
decky / build-publish (push) Successful in 30s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m15s
deb / build-publish (push) Successful in 8m50s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 45s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m6s
deb / build-publish-host (push) Successful in 9m52s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m44s
docker / deploy-docs (push) Successful in 28s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 7m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m45s
flatpak / build-publish (push) Failing after 8m6s
The pen plane from design/pen-tablet-input.md, protocol side only (the P1 uinput
tablet injector will consume it): 0xCC kind 0x05 carries batches of state-full
PenSamples (pressure, polar tilt + azimuth, barrel roll, hover distance, eraser
tool, barrel buttons); PenTracker diffs samples into injector transitions so a
lost datagram self-heals; HOST_CAP_PEN (0x10) gates sending and stays
unadvertised until a real backend exists. C ABI: PunktfunkPenSample +
punktfunk_connection_send_pen, documented in docs/embedding-the-c-abi.md §8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 12:58:31 +02:00
enricobuehlerandClaude Opus 4.8 1a6deeb781 feat(host/library): scan the user's non-Steam shortcuts into the game library
apple / swift (push) Successful in 1m21s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 57s
apple / screenshots (push) Successful in 6m41s
ci / bench (push) Successful in 6m22s
ci / rust (push) Successful in 24m42s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
decky / build-publish (push) Successful in 28s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
android-screenshots / screenshots (push) Successful in 3m10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m41s
windows-host / package (push) Successful in 9m53s
linux-client-screenshots / screenshots (push) Successful in 7m11s
deb / build-publish (push) Successful in 11m1s
docker / deploy-docs (push) Successful in 24s
deb / build-publish-host (push) Successful in 11m55s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m31s
arch / build-publish (push) Successful in 12m57s
web-screenshots / screenshots (push) Successful in 3m12s
android / android (push) Successful in 13m46s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m34s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m0s
release / apple (push) Successful in 10m32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m45s
flatpak / build-publish (push) Successful in 5m51s
Steam's "Add a Non-Steam Game to My Library" entries carry no `appmanifest`, so
`scan_manifests` can't see them and they never appear in the streamed library.
Read each Steam account's binary `userdata/<id>/config/shortcuts.vdf` directly
(a binary KeyValues parse) to surface the user's own custom entries, plus their
grid artwork from `userdata/<id>/config/grid`. Best-effort: an unreadable or
absent shortcuts.vdf contributes nothing, and hidden shortcuts / duplicate
appids are dropped. Verified: clippy -D warnings + compile green on Linux.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 00:45:25 +02:00
enricobuehlerandClaude Opus 4.8 8c9baf3214 fix(decky): recreate the library shortcut when its cached appId is stale
The visible "Punktfunk" shortcut's Steam appId is cached in SteamUI
localStorage, which lives in Steam's CEF profile — so it survives a
plugin uninstall/reinstall. Deleting the shortcut left the key dangling;
on the next mount ensure*Shortcut recalled the dead appId, took the reuse
branch, and ran SetShortcut* against a non-existent id (silent no-ops).
AddShortcut was never reached, so the entry never came back. The same
stale-reuse also made the hidden stream shortcut RunGame a dead gameid.

Verify the remembered appId still maps to a live shortcut via
appStore.GetAppOverviewByAppID before reusing it; a confident miss falls
through to AddShortcut. When appStore is unavailable, assume it exists so
a false negative can't spawn a duplicate library entry.

Also add a "Recreate library shortcut" recovery button to the QAM About
section (recreateShortcuts): drops any dead cached keys and re-ensures.
Covers deleting the shortcut without reinstalling, where no mount fires
the self-heal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 00:41:32 +02:00
enricobuehlerandClaude Fable 5 d83eeedde4 ci(flatpak): drop the use-vc TCP-DNS hack — the runner-side cache made it the failure
audit / bun-audit (push) Successful in 11s
ci / web (push) Successful in 46s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m21s
audit / cargo-audit (push) Successful in 2m20s
decky / build-publish (push) Successful in 19s
ci / bench (push) Successful in 6m53s
apple / screenshots (push) Successful in 6m28s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 1m0s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6m27s
windows-drivers / probe-and-proto (push) Successful in 16s
windows-drivers / driver-build (push) Successful in 1m27s
deb / build-publish-host (push) Successful in 9m37s
android / android (push) Successful in 11m56s
docker / deploy-docs (push) Successful in 24s
deb / build-publish (push) Successful in 11m31s
arch / build-publish (push) Successful in 15m34s
windows-host / package (push) Successful in 11m29s
ci / rust (push) Successful in 24m30s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m37s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m28s
flatpak / build-publish (push) Failing after 8m18s
Every flatpak Tooling failure on 2026-07-22 (4 runs, incl. two AFTER the
runner fleet got its local dnsmasq cache) died in flatpak remote-add with an
instant 'Could not resolve hostname' ×10 while dnf in the SAME container
resolved fine. The one thing only this leg does is force glibc onto TCP DNS
(use-vc) — a 2026-07-11 mitigation for the embedded resolver dropping UDP
under fleet load. With job containers now resolving against the on-box
dnsmasq cache (daemon.json dns → docker0), UDP is reliable again and the TCP
path through the same chain is the flaky one under concurrency. Remove the
hack; keep the nss-resolve drop and retry.sh as the backstop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 00:27:52 +02:00
enricobuehlerandClaude Fable 5 786f6ad28f fix(pf-driver-proto): catch the layout tests up with the cursor-channel proto bump
The cursor-channel sweep grew AddReply by the cursor_excluded u32 tail (24
bytes, legacy prefix 20) and bumped PROTOCOL_VERSION to 6, but left two lib
tests behind: the AddReply byte-layout roundtrip still built the legacy
initializer (a compile error that turned the rust CI leg red on main), and
the version-coherence assertion still pinned v5 (hidden behind that compile
error). Cover the new tail bytes + the legacy-size constant, and widen the
compat-window note to v4–v6 (floor stays 3).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 00:22:34 +02:00
enricobuehlerandClaude Fable 5 1e9957d97b chore(web): clear the 2026-07-22 JS advisory wave — overrides + lock re-resolve
bun-audit went red mid-day on a fresh advisory wave (13 findings across two
snapshots of a moving DB): node-tar DoS trio+1 (critical), sharp/libvips CVEs,
fast-uri host confusion, undici ×7, brace-expansion/linkify-it/js-yaml/postcss
DoS-class, immutable, dompurify. All transitive pins in the management-console
tree; none ship in the streaming stack.

Every fix version is in-range for its consumers, so: pin overrides for the
single-major packages (tar/dompurify/linkify-it/sharp/fast-uri/immutable/
undici/postcss/js-yaml — bun ignores range-scoped override keys, so plain keys)
and a full lockfile re-resolve, which also lifts brace-expansion 2.1.1→2.1.2
in place while its 5.x line (minimatch@10) stays untouched. : No
vulnerabilities found.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 00:20:44 +02:00
65 changed files with 5819 additions and 731 deletions
+39
View File
@@ -0,0 +1,39 @@
# Announce a stable release to the Discord #releases channel.
#
# This is the deliberate "go" step for a release. Release notes live in the repo at
# docs/releases/<tag>.md and are seeded into the Gitea release body at creation by the build
# workflows (scripts/ci/gitea-release.sh), so the release is never noteless. Once every
# platform's CI is green for a tag, dispatch this workflow with that tag: it re-asserts the notes
# file over the live release and posts a formatted embed to #releases.
#
# Manual on purpose — pressing "go" is the quality gate that says "all platforms built, notes are
# final, tell the community." It is NOT wired to the tag push, so a half-built or failed release
# is never announced. Stable-only: a -rc/pre-release tag is refused unless allow_prerelease=true.
#
# Requires the repo secret DISCORD_RELEASE_WEBHOOK (the #releases channel webhook URL); GITEA auth
# reuses REGISTRY_TOKEN like the other release workflows.
name: announce
on:
workflow_dispatch:
inputs:
tag:
description: "Release tag to announce (e.g. v0.18.0)"
required: true
allow_prerelease:
description: "Announce even if the tag is a pre-release (-rc)"
required: false
default: "false"
jobs:
announce:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- name: Post release announcement to Discord
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
DISCORD_RELEASE_WEBHOOK: ${{ secrets.DISCORD_RELEASE_WEBHOOK }}
ALLOW_PRERELEASE: ${{ inputs.allow_prerelease }}
run: bash scripts/ci/discord-announce.sh "${{ inputs.tag }}"
+12 -25
View File
@@ -73,33 +73,20 @@ jobs:
# sufficient — the Tooling step's dnf install pulls a systemd package upgrade whose RPM
# trigger re-runs authselect and regenerates this file, undoing the fix. It's reapplied
# there, right before the first `flatpak` network call.
- name: Fix container DNS (drop nss-resolve, resolve over TCP)
- name: Fix container DNS (drop nss-resolve)
run: |
sed -i 's/resolve \[!UNAVAIL=return\] //' /etc/nsswitch.conf
# Resolve over TCP instead of UDP. The documented root cause of the flathub
# bootstrap failures (investigated 2026-07-11, see the Tooling step) is this box's
# Docker embedded resolver at 127.0.0.11 DROPPING UDP lookups while the shared
# runner fleet is saturated — a datagram nobody retransmits, so the lookup just
# times out. The answer then was to widen retry.sh's budget to 10 attempts (~9 min),
# which is enough to outlast a main push's ~8-workflow fan-out but NOT a TAG push's
# 13: v0.15.0 (twice) and v0.16.0 each burned all 10 attempts and failed the job,
# each needing a manual re-run.
#
# `use-vc` makes glibc use TCP, where the kernel retransmits and the query cannot be
# silently lost under load. Same resolver, same search path — only the transport
# changes, so internal names (git.unom.io) resolve exactly as before; deliberately
# NO extra nameservers, which would risk answering an internal name from a public
# resolver. Docker's embedded DNS serves TCP on 127.0.0.11:53 as well as UDP.
# retry.sh stays as the backstop for genuine upstream blips.
#
# Non-fatal: Docker bind-mounts /etc/resolv.conf and can present it read-only, and a
# DNS tuning that cannot be applied must not be what fails the release build — that
# would trade an occasional re-run for a hard stop. Falling back to UDP just restores
# today's behaviour, which retry.sh already covers.
if ! grep -q '^options .*use-vc' /etc/resolv.conf 2>/dev/null; then
echo 'options use-vc timeout:3 attempts:3' >> /etc/resolv.conf \
|| echo "::warning::could not set use-vc (read-only resolv.conf?); staying on UDP"
fi
# History: this step used to ALSO force glibc onto TCP DNS (`options use-vc`) because
# the runner fleet's Docker embedded resolver dropped UDP lookups under parallel-job
# load (investigated 2026-07-11; v0.15.0/v0.16.0 each burned retry.sh's whole budget).
# That root cause is now fixed at the infra level (2026-07-22): the runner host runs a
# local dnsmasq cache on the docker bridge and daemon.json points every job container
# at it, so lookups terminate on-box instead of crossing the saturated uplink — the
# UDP path is reliable again. The TCP path through the same chain proved FLAKY under
# fleet concurrency (flatpak remote-add failed 10/10 with instant NXDOMAIN while dnf
# in the same container resolved fine), so `use-vc` flipped from mitigation to sole
# cause of this leg's failures — removed. retry.sh (10×) stays as the backstop for
# genuine upstream blips.
cat /etc/resolv.conf || true
# fedora:43 has no node, but actions/checkout (a JS action) needs it. A plain `run:` step
+70
View File
@@ -393,6 +393,76 @@ jobs:
-authenticationKeyID "${{ secrets.ASC_API_KEY_ID }}" \
-authenticationKeyIssuerID "${{ secrets.ASC_API_ISSUER_ID }}"
- name: iOS — export .ipa (Gitea release + run artifact)
# The TestFlight step above uploads straight to App Store Connect (destination=upload) and
# leaves NO .ipa on disk. Re-export the SAME archive with destination=export to get an
# App Store distribution-signed .ipa for the Gitea release + the run artifacts. Same gate as
# that archive; a warn+skip (never fails the best-effort iOS leg) if the archive is absent,
# e.g. a workflow_dispatch with testflight=false. NOTE: an App Store-signed .ipa installs
# only via TestFlight/App Store, not by direct sideload — it's a release/archival artifact.
if: gitea.event_name != 'workflow_dispatch' || inputs.testflight == 'true'
id: ios_ipa
run: |
ARCHIVE="$RUNNER_TEMP/Punktfunk-ios.xcarchive"
if [ ! -d "$ARCHIVE" ]; then
echo "::warning::iOS archive not found — skipping .ipa export"
exit 0
fi
PROFILE="Punktfunk iOS App Store Distribution"
WIDGET_PROFILE="Punktfunk iOS Widgets App Store Distribution"
# destination=export writes the .ipa to -exportPath; otherwise identical manual signing to
# the upload plist (both profiles, Apple Distribution). No ASC key needed — no network.
cat > "$RUNNER_TEMP/export-appstore-ipa.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>method</key><string>app-store-connect</string>
<key>destination</key><string>export</string>
<key>teamID</key><string>$TEAM_ID</string>
<key>signingStyle</key><string>manual</string>
<key>signingCertificate</key><string>Apple Distribution</string>
<key>provisioningProfiles</key>
<dict>
<key>io.unom.punktfunk</key><string>$PROFILE</string>
<key>io.unom.punktfunk.widgets</key><string>$WIDGET_PROFILE</string>
</dict>
</dict>
</plist>
EOF
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild -exportArchive \
-archivePath "$ARCHIVE" \
-exportOptionsPlist "$RUNNER_TEMP/export-appstore-ipa.plist" \
-exportPath "$RUNNER_TEMP/export-ipa"
SRC=$(ls "$RUNNER_TEMP/export-ipa/"*.ipa 2>/dev/null | head -1)
[ -n "$SRC" ] || { echo "::warning::no .ipa was produced by export"; exit 0; }
mkdir -p "$GITHUB_WORKSPACE/dist"
IPA="$GITHUB_WORKSPACE/dist/Punktfunk-$VERSION.ipa"
mv "$SRC" "$IPA"
echo "IPA=$IPA" >> "$GITHUB_ENV"
echo "ipa=dist/Punktfunk-$VERSION.ipa" >> "$GITHUB_OUTPUT"
echo "exported $IPA"
- name: Attach .ipa to the workflow run
if: steps.ios_ipa.outputs.ipa != ''
# v3, not v4: Gitea's artifact backend identifies as GHES, which upload-artifact@v4 refuses
# (same reason as android.yml / apple.yml). Download is a zip of the .ipa.
uses: actions/upload-artifact@v3
with:
name: punktfunk-ios-ipa
path: ${{ steps.ios_ipa.outputs.ipa }}
if-no-files-found: warn
retention-days: 30
- name: Attach .ipa to the Gitea release (stable tags only)
if: startsWith(gitea.ref, 'refs/tags/v') && steps.ios_ipa.outputs.ipa != ''
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
. scripts/ci/gitea-release.sh
RID=$(ensure_release "$GITHUB_REF_NAME" "$GITHUB_REF_NAME" auto)
upsert_asset "$RID" "$IPA" "Punktfunk-$VERSION.ipa"
- name: tvOS — archive + upload to TestFlight
# Canary + stable, the same track as iOS/macOS — the tvOS xcframework slice is now built
# on every apple push (above), so this matches the iOS step's gate exactly.
Generated
+27 -27
View File
@@ -2194,7 +2194,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.18.0"
version = "0.19.0"
[[package]]
name = "lazy_static"
@@ -2299,7 +2299,7 @@ dependencies = [
[[package]]
name = "libvpl-sys"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"bindgen",
"cmake",
@@ -2334,7 +2334,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"punktfunk-core",
]
@@ -2823,7 +2823,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-capture"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2844,7 +2844,7 @@ dependencies = [
[[package]]
name = "pf-client-core"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"ash",
@@ -2868,7 +2868,7 @@ dependencies = [
[[package]]
name = "pf-clipboard"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2886,7 +2886,7 @@ dependencies = [
[[package]]
name = "pf-console-ui"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"ash",
@@ -2907,7 +2907,7 @@ dependencies = [
[[package]]
name = "pf-encode"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"ash",
@@ -2931,7 +2931,7 @@ dependencies = [
[[package]]
name = "pf-ffvk"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"ash",
"bindgen",
@@ -2940,7 +2940,7 @@ dependencies = [
[[package]]
name = "pf-frame"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"libc",
@@ -2952,7 +2952,7 @@ dependencies = [
[[package]]
name = "pf-gpu"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"pf-host-config",
@@ -2966,11 +2966,11 @@ dependencies = [
[[package]]
name = "pf-host-config"
version = "0.18.0"
version = "0.19.0"
[[package]]
name = "pf-inject"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2998,14 +2998,14 @@ dependencies = [
[[package]]
name = "pf-paths"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"ash",
@@ -3020,7 +3020,7 @@ dependencies = [
[[package]]
name = "pf-vdisplay"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"ashpd",
@@ -3050,7 +3050,7 @@ dependencies = [
[[package]]
name = "pf-win-display"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"pf-paths",
@@ -3062,7 +3062,7 @@ dependencies = [
[[package]]
name = "pf-zerocopy"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"ash",
@@ -3269,7 +3269,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"android_logger",
"jni",
@@ -3285,7 +3285,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"async-channel",
@@ -3301,7 +3301,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-session"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"pf-client-core",
@@ -3316,7 +3316,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"async-channel",
"ffmpeg-next",
@@ -3335,7 +3335,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"aes-gcm",
"bytes",
@@ -3367,7 +3367,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"aes",
"aes-gcm",
@@ -3451,7 +3451,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3465,7 +3465,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"anyhow",
"ksni",
@@ -3488,7 +3488,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]]
name = "pyrowave-sys"
version = "0.18.0"
version = "0.19.0"
dependencies = [
"bindgen",
"cmake",
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package]
version = "0.18.0"
version = "0.19.0"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
@@ -355,11 +355,19 @@ class MainActivity : ComponentActivity() {
return true
}
when (event.keyCode) {
// A mouse back/forward button whose BUTTON_* press went unconsumed makes the
// framework synthesize a FALLBACK BACK — the button already went over the wire
// as X1/X2, and it must never yank the user out of the stream.
KeyEvent.KEYCODE_BACK ->
if (event.flags and KeyEvent.FLAG_FALLBACK != 0) return true
// A mouse's back/forward buttons already go over the wire as X1/X2 via their
// BUTTON_* motion edges — but Android ALSO delivers them as key events: the input
// reader synthesizes KEYCODE_BACK/FORWARD (stamped SOURCE_MOUSE) unconditionally,
// and a view-level FALLBACK BACK appears when the BUTTON_* press goes unconsumed.
// Swallow every such duplicate or it doubles as Android navigation and yanks the
// user out of the stream. A remote/keyboard BACK is never mouse-sourced, so it
// still falls through to the BackHandler and exits.
KeyEvent.KEYCODE_BACK, KeyEvent.KEYCODE_FORWARD ->
if (event.isFromSource(InputDevice.SOURCE_MOUSE) ||
event.flags and KeyEvent.FLAG_FALLBACK != 0
) {
return true
}
// Leave these to the system even while streaming.
// (BACK above → BackHandler leaves the stream.)
KeyEvent.KEYCODE_VOLUME_UP,
@@ -13,6 +13,7 @@ import android.net.wifi.WifiManager
import android.os.Build
import android.text.InputType
import android.util.Log
import android.view.KeyEvent
import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.View
@@ -471,12 +472,22 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no
// keyboard gesture: its fingers belong to the host verbatim (a swipe there may BE a
// host-OS gesture), so intercepting three fingers would corrupt real multi-touch.
// Stylus lane (design/pen-tablet-input.md §7): against a HOST_CAP_PEN host a stylus
// splits out of BOTH touch models onto the pen plane; its heartbeat coroutine keeps a
// stationary held stroke alive (and its cancellation lifts everything on teardown).
val stylus = remember(handle) {
if (NativeBridge.nativeHostSupportsPen(handle)) StylusStream(handle) else null
}
if (stylus != null) {
LaunchedEffect(stylus) { stylus.heartbeatLoop() }
}
Box(
Modifier.fillMaxSize().pointerInput(handle, touchMode) {
when (touchMode) {
TouchMode.TOUCH -> streamTouchPassthrough(handle)
TouchMode.TOUCH -> streamTouchPassthrough(handle, stylus)
else -> streamTouchInput(
handle,
stylus,
trackpad = touchMode == TouchMode.TRACKPAD,
invertScroll = initialSettings.invertScroll,
onCycleStats = { statsVerbosity = statsVerbosity.next() },
@@ -549,9 +560,15 @@ private class KeyCaptureView(context: Context) : View(context) {
var imeShown = false
private set
override fun onCheckIsTextEditor(): Boolean = true
override fun onCheckIsTextEditor(): Boolean = imeShown
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection {
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection? {
// Only an editor while the user has SUMMONED the keyboard (gesture / remote toggle).
// This view holds focus for the whole stream (it's the capture anchor), and with an
// always-live editable connection the IME counts input as active on it — TV IMEs then
// pop their UI the moment a PHYSICAL keyboard key arrives. With no connection, hardware
// typing stays on the raw dispatchKeyEvent → Keymap → wire path and no keyboard appears.
if (!imeShown) return null
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or
EditorInfo.IME_FLAG_NO_FULLSCREEN or EditorInfo.IME_FLAG_NO_ENTER_ACTION
return if (textHandle != 0L) {
@@ -570,11 +587,29 @@ private class KeyCaptureView(context: Context) : View(context) {
imeShown = show
if (show) {
requestFocus()
// The view may already be focused from a null-connection state — restart so the
// framework re-queries onCreateInputConnection with the gate now open.
imm.restartInput(this)
imm.showSoftInput(this, 0)
} else {
imm.hideSoftInputFromWindow(windowToken, 0)
imm.restartInput(this) // gate closed — drop the editable connection
}
}
/**
* BACK while the summoned keyboard is up: the IME consumes it pre-IME to dismiss itself, so
* [setImeVisible] never hears about it — sync the gate here or a stale `imeShown` leaves the
* editable connection live and physical typing re-pops the keyboard.
*/
override fun onKeyPreIme(keyCode: Int, event: KeyEvent): Boolean {
if (keyCode == KeyEvent.KEYCODE_BACK && imeShown && event.action == KeyEvent.ACTION_UP) {
imeShown = false
(context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager)
?.restartInput(this)
}
return super.onKeyPreIme(keyCode, event)
}
}
/**
@@ -0,0 +1,195 @@
package io.unom.punktfunk
import android.view.MotionEvent
import androidx.compose.ui.ExperimentalComposeUiApi
import androidx.compose.ui.input.pointer.PointerEvent
import androidx.compose.ui.input.pointer.PointerType
import androidx.compose.ui.unit.IntSize
import io.unom.punktfunk.kit.NativeBridge
import kotlinx.coroutines.delay
// Wire PEN_* state bits (punktfunk_core::quic::pen; mirrored, asserted by the Rust shim's docs).
private const val PEN_IN_RANGE = 1f
private const val PEN_TOUCHING = 2f
private const val PEN_BARREL1 = 4f
private const val PEN_BARREL2 = 8f
private const val STRIDE = 10
private const val MAX_SAMPLES = 8
/**
* Android stylus → the state-full pen plane (design/pen-tablet-input.md §7): pressure, tilt
* (`AXIS_TILT`, radians from the surface normal), azimuth (`AXIS_ORIENTATION` — Android's 0 =
* "pointed away from the user" IS the wire's north, no offset needed), hover with
* `AXIS_DISTANCE`, the eraser tool, both stylus barrel buttons, and historical (coalesced)
* samples batched oldest-first for full capture-rate fidelity. Android has no barrel-roll
* axis — roll stays unknown on this client.
*
* Both touch loops call [intercept] first; stylus/eraser pointers are consumed here (against a
* pen-capable host) and never reach the finger paths, independent of the touch-input mode.
* [heartbeatLoop] implements the ≤100 ms keepalive wire contract: a stationary held stylus is
* silent in Android's input pipeline, and the host force-releases a stroke after 200 ms
* without samples.
*/
internal class StylusStream(private val handle: Long) {
private var inRange = false
private var touching = false
private var sawHover = false
private val last = FloatArray(STRIDE)
private val batch = FloatArray(MAX_SAMPLES * STRIDE)
init {
idle(last)
}
/**
* Consume the event's stylus pointers into pen samples. Returns true when this event
* carried any (the caller's finger/gesture handling must then skip those changes).
*/
@OptIn(ExperimentalComposeUiApi::class)
fun intercept(ev: PointerEvent, size: IntSize): Boolean {
val stylusChanges = ev.changes.filter {
it.type == PointerType.Stylus || it.type == PointerType.Eraser
}
if (stylusChanges.isEmpty()) return false
stylusChanges.forEach { it.consume() }
val me = ev.motionEvent ?: return true
if (size.width <= 0 || size.height <= 0) return true
// At most one stylus exists — find its pointer index by tool type.
val idx = (0 until me.pointerCount).firstOrNull {
me.getToolType(it) == MotionEvent.TOOL_TYPE_STYLUS ||
me.getToolType(it) == MotionEvent.TOOL_TYPE_ERASER
} ?: return true
when (me.actionMasked) {
MotionEvent.ACTION_DOWN, MotionEvent.ACTION_POINTER_DOWN,
MotionEvent.ACTION_MOVE,
-> {
touching = true
inRange = true
emitSamples(me, idx, size)
}
MotionEvent.ACTION_HOVER_ENTER, MotionEvent.ACTION_HOVER_MOVE -> {
sawHover = true
inRange = true
touching = false
emitSamples(me, idx, size)
}
MotionEvent.ACTION_UP, MotionEvent.ACTION_POINTER_UP -> {
touching = false
// Hover-capable hardware keeps proximity (HOVER_EXIT owns the leave);
// anything else leaves range on lift — the host never parks a phantom pen.
inRange = sawHover
emitSamples(me, idx, size)
}
MotionEvent.ACTION_HOVER_EXIT, MotionEvent.ACTION_CANCEL -> release()
else -> {}
}
return true
}
/** Session/composition teardown: leave range so the host lifts anything still inked. */
fun reset() {
if (inRange || touching) release()
sawHover = false
}
/** The ≤100 ms keepalive (80 ms leaves headroom for one lost datagram). Runs until
* cancelled; resends the last state-full sample while the pen is in range. */
suspend fun heartbeatLoop() {
try {
while (true) {
delay(80)
if (inRange || touching) {
last[9] = 0f // dt
NativeBridge.nativeSendPen(handle, last, 1)
}
}
} finally {
reset()
}
}
private fun release() {
touching = false
inRange = false
last[0] = 0f // state: out of range
last[4] = 0f // pressure
NativeBridge.nativeSendPen(handle, last, 1)
}
/** Historical (coalesced) samples oldest-first, then the current one — a single batch. */
private fun emitSamples(me: MotionEvent, idx: Int, size: IntSize) {
val history = minOf(me.historySize, MAX_SAMPLES - 1)
var count = 0
var prevT = if (history > 0) me.getHistoricalEventTime(0) else me.eventTime
for (h in (me.historySize - history) until me.historySize) {
val t = me.getHistoricalEventTime(h)
fill(
batch, count * STRIDE, size,
x = me.getHistoricalX(idx, h), y = me.getHistoricalY(idx, h),
pressure = me.getHistoricalPressure(idx, h),
tiltRad = me.getHistoricalAxisValue(MotionEvent.AXIS_TILT, idx, h),
orientRad = me.getHistoricalAxisValue(MotionEvent.AXIS_ORIENTATION, idx, h),
distance = me.getHistoricalAxisValue(MotionEvent.AXIS_DISTANCE, idx, h),
buttons = me.buttonState, tool = me.getToolType(idx),
dtUs = ((t - prevT) * 1000).coerceIn(0, 65535).toFloat(),
)
prevT = t
count++
}
fill(
batch, count * STRIDE, size,
x = me.getX(idx), y = me.getY(idx), pressure = me.getPressure(idx),
tiltRad = me.getAxisValue(MotionEvent.AXIS_TILT, idx),
orientRad = me.getAxisValue(MotionEvent.AXIS_ORIENTATION, idx),
distance = me.getAxisValue(MotionEvent.AXIS_DISTANCE, idx),
buttons = me.buttonState, tool = me.getToolType(idx),
dtUs = ((me.eventTime - prevT) * 1000).coerceIn(0, 65535).toFloat(),
)
count++
batch.copyInto(last, 0, (count - 1) * STRIDE, count * STRIDE)
NativeBridge.nativeSendPen(handle, batch, count)
}
private fun fill(
out: FloatArray,
off: Int,
size: IntSize,
x: Float,
y: Float,
pressure: Float,
tiltRad: Float,
orientRad: Float,
distance: Float,
buttons: Int,
tool: Int,
dtUs: Float,
) {
var state = 0f
if (inRange || touching) state += PEN_IN_RANGE
if (touching) state += PEN_TOUCHING
if (buttons and MotionEvent.BUTTON_STYLUS_PRIMARY != 0) state += PEN_BARREL1
if (buttons and MotionEvent.BUTTON_STYLUS_SECONDARY != 0) state += PEN_BARREL2
out[off + 0] = state
out[off + 1] = if (tool == MotionEvent.TOOL_TYPE_ERASER) 1f else 0f
out[off + 2] = (x / (size.width - 1).coerceAtLeast(1)).coerceIn(0f, 1f)
out[off + 3] = (y / (size.height - 1).coerceAtLeast(1)).coerceIn(0f, 1f)
out[off + 4] = if (touching) pressure.coerceIn(0f, 1f) else 0f
// AXIS_DISTANCE units are device-arbitrary; 0..1 covers real hardware, and 0 while
// hovering legitimately means "at the hover floor".
out[off + 5] = if (touching) 0f else distance.coerceIn(0f, 1f)
out[off + 6] = Math.toDegrees(tiltRad.toDouble()).toFloat().coerceIn(0f, 90f)
// AXIS_ORIENTATION: 0 = pointed away from the user (= wire north), clockwise, −π..π.
out[off + 7] = ((Math.toDegrees(orientRad.toDouble()) + 360.0) % 360.0).toFloat()
out[off + 8] = -1f // no barrel-roll axis on Android
out[off + 9] = dtUs
}
private fun idle(out: FloatArray) {
out.fill(0f)
out[5] = -1f // distance unknown
out[6] = -1f // tilt unknown
out[7] = -1f // azimuth unknown
out[8] = -1f // roll unknown
}
}
@@ -1,9 +1,11 @@
package io.unom.punktfunk
import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.foundation.gestures.awaitFirstDown
import androidx.compose.ui.input.pointer.AwaitPointerEventScope
import androidx.compose.ui.input.pointer.PointerId
import androidx.compose.ui.input.pointer.PointerInputChange
import androidx.compose.ui.input.pointer.PointerInputScope
import androidx.compose.ui.input.pointer.PointerType
import androidx.compose.ui.input.pointer.changedToDownIgnoreConsumed
import androidx.compose.ui.input.pointer.changedToUpIgnoreConsumed
import androidx.compose.ui.input.pointer.positionChanged
@@ -56,7 +58,26 @@ private const val ACCEL_MAX = 3.0f
* normalizes and maps into the output). On teardown (stream leaves composition) every still-held
* contact is lifted so nothing stays stuck on the host.
*/
internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
/** Whether this change belongs to the stylus lane (only when a pen-capable host is live). */
private fun isStylus(c: PointerInputChange, stylus: StylusStream?): Boolean =
stylus != null && (c.type == PointerType.Stylus || c.type == PointerType.Eraser)
/** [awaitFirstDown] with the stylus lane split out: pen events feed [stylus] and never start a
* mouse/touch gesture. Toward a pen-less host ([stylus] == null) a stylus stays a finger. */
private suspend fun AwaitPointerEventScope.awaitFirstFingerDown(
stylus: StylusStream?,
): PointerInputChange {
while (true) {
val ev = awaitPointerEvent()
stylus?.intercept(ev, size)
val down = ev.changes.firstOrNull {
it.changedToDownIgnoreConsumed() && !isStylus(it, stylus)
}
if (down != null) return down
}
}
internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long, stylus: StylusStream?) {
val ids = mutableMapOf<PointerId, Int>()
fun alloc(p: PointerId): Int {
var id = 0
@@ -68,10 +89,12 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
awaitPointerEventScope {
while (true) {
val ev = awaitPointerEvent()
stylus?.intercept(ev, size)
val sw = size.width
val sh = size.height
if (sw <= 0 || sh <= 0) continue
for (c in ev.changes) {
if (isStylus(c, stylus)) continue // the pen plane owns it
val x = c.position.x.roundToInt().coerceIn(0, sw - 1)
val y = c.position.y.roundToInt().coerceIn(0, sh - 1)
when {
@@ -98,6 +121,7 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
internal suspend fun PointerInputScope.streamTouchInput(
handle: Long,
stylus: StylusStream?,
trackpad: Boolean,
invertScroll: Boolean,
onCycleStats: () -> Unit,
@@ -120,7 +144,7 @@ internal suspend fun PointerInputScope.streamTouchInput(
)
}
awaitEachGesture {
val down = awaitFirstDown(requireUnconsumed = false)
val down = awaitFirstFingerDown(stylus)
val startX = down.position.x
val startY = down.position.y
// A touch landing just after a quick tap nearby = tap-and-drag: hold the left
@@ -157,7 +181,8 @@ internal suspend fun PointerInputScope.streamTouchInput(
while (true) {
val ev = awaitPointerEvent()
val pressed = ev.changes.filter { it.pressed }
stylus?.intercept(ev, size)
val pressed = ev.changes.filter { it.pressed && !isStylus(it, stylus) }
if (pressed.isEmpty()) {
upTime = ev.changes.firstOrNull()?.uptimeMillis ?: upTime
break
@@ -287,6 +287,22 @@ object NativeBridge {
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
/**
* Whether the host advertised full-fidelity stylus injection (`HOST_CAP_PEN`) — the gate
* for splitting stylus pointers out of the touch path onto the pen plane. False on `0`.
*/
external fun nativeHostSupportsPen(handle: Long): Boolean
/**
* One stylus batch of STATE-FULL samples (the pen plane; design/pen-tablet-input.md §7):
* [count] × 10 floats, oldest first — `[state, tool, x, y, pressure, distance, tilt_deg,
* azimuth_deg, roll_deg, dt_us]`. `state` = the wire in-range/touching/barrel bits; `tool`
* 0=pen 1=eraser; x/y/pressure/distance normalized 0..1; distance/tilt/azimuth/roll < 0 =
* unknown. Send only when [nativeHostSupportsPen]; repeat the last sample ≤100 ms while the
* pen is in range (the host force-releases a silent stroke after 200 ms).
*/
external fun nativeSendPen(handle: Long, samples: FloatArray, count: Int)
/**
* Whether the host advertised committed-text injection (`HOST_CAP_TEXT_INPUT`) — its inject
* backend can type Unicode text directly. Picks the real IME `InputConnection` (autocorrect,
+92 -2
View File
@@ -6,11 +6,14 @@
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::{JByteBuffer, JObject, JString};
use jni::objects::{JByteBuffer, JFloatArray, JObject, JString};
use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX, HOST_CAP_TEXT_INPUT};
use punktfunk_core::quic::{
PenSample, PenTool, RichInput, HID_REPORT_MAX, HOST_CAP_PEN, HOST_CAP_TEXT_INPUT,
PEN_ANGLE_UNKNOWN, PEN_BATCH_MAX, PEN_DISTANCE_UNKNOWN, PEN_TILT_UNKNOWN,
};
use super::SessionHandle;
@@ -162,6 +165,93 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSu
u8::from(h.client.host_caps() & HOST_CAP_TEXT_INPUT != 0)
}
/// `NativeBridge.nativeHostSupportsPen(handle)` — the host advertised `HOST_CAP_PEN`, so the
/// Kotlin side splits stylus pointers out of the touch path onto the pen plane
/// (design/pen-tablet-input.md §7). `0` handle → false.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeHostSupportsPen(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; host_caps is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
u8::from(h.client.host_caps() & HOST_CAP_PEN != 0)
}
/// Floats per sample in the `nativeSendPen` flat array.
const PEN_JNI_STRIDE: usize = 10;
/// `NativeBridge.nativeSendPen(handle, samples, count)` — one stylus batch of STATE-FULL
/// samples, `count` × [`PEN_JNI_STRIDE`] floats, oldest first:
/// `[state, tool, x, y, pressure, distance, tilt_deg, azimuth_deg, roll_deg, dt_us]`.
/// `state` = the wire `PEN_*` bits; `tool` 0=pen 1=eraser; `x`/`y`/`pressure`/`distance`
/// normalized 0..1; `distance`/`tilt_deg`/`azimuth_deg`/`roll_deg` < 0 = unknown. Call only
/// against a [`nativeHostSupportsPen`] host; the client heartbeats the last sample ≤100 ms
/// while in range (Kotlin side — see `StylusStream`).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPen(
env: JNIEnv,
_this: JObject,
handle: jlong,
samples: JFloatArray,
count: jint,
) {
if handle == 0 || count <= 0 {
return;
}
let count = (count as usize).min(PEN_BATCH_MAX);
let mut buf = [0f32; PEN_BATCH_MAX * PEN_JNI_STRIDE];
let flat = &mut buf[..count * PEN_JNI_STRIDE];
if env.get_float_array_region(&samples, 0, flat).is_err() {
return; // short array — a bridge bug, never worth a crash on the input path
}
let mut batch = [PenSample::default(); PEN_BATCH_MAX];
for (slot, s) in batch.iter_mut().zip(flat.chunks_exact(PEN_JNI_STRIDE)) {
if !s[2].is_finite() || !s[3].is_finite() {
return; // never forward a NaN coordinate
}
*slot = PenSample {
state: s[0] as u8,
tool: if s[1] as u8 == 1 {
PenTool::Eraser
} else {
PenTool::Pen
},
x: s[2].clamp(0.0, 1.0),
y: s[3].clamp(0.0, 1.0),
pressure: (s[4].clamp(0.0, 1.0) * 65535.0) as u16,
distance: if s[5] < 0.0 {
PEN_DISTANCE_UNKNOWN
} else {
(s[5].clamp(0.0, 1.0) * 65534.0) as u16
},
tilt_deg: if s[6] < 0.0 {
PEN_TILT_UNKNOWN
} else {
(s[6].clamp(0.0, 90.0)) as u8
},
azimuth_deg: if s[7] < 0.0 {
PEN_ANGLE_UNKNOWN
} else {
(s[7] as u16) % 360
},
roll_deg: if s[8] < 0.0 {
PEN_ANGLE_UNKNOWN
} else {
(s[8] as u16) % 360
},
dt_us: s[9].clamp(0.0, 65535.0) as u16,
};
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; send_pen is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_pen(&batch[..count]);
}
/// `NativeBridge.nativeSendText(handle, text)` — committed IME text, one `TextInput` event per
/// Unicode scalar (`code` = the scalar; multi-char commits are consecutive events in order).
/// Control characters are skipped — Enter/Backspace/Tab ride the VK key path. Call only when
@@ -387,8 +387,16 @@ public final class PunktfunkConnection {
/// The host answered `HOST_CAP_CURSOR`: it stopped compositing the pointer and forwards
/// shape/state on the cursor planes the client MUST draw the cursor locally.
/// `0x08` the bit moved when `HOST_CAP_TEXT_INPUT` claimed `0x04` on main; testing the
/// old bit would mistake a text-input-capable host (e.g. Windows) for a cursor grant.
public var hostSupportsCursor: Bool {
hostCaps & 0x04 != 0
hostCaps & 0x08 != 0
}
/// The host injects full-fidelity stylus input (`HOST_CAP_PEN`) the gate for splitting
/// Apple Pencil out of the touch path onto the pen plane (``sendPen(_:)``).
public var hostSupportsPen: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_PEN) != 0
}
/// One forwarded host-cursor shape (the cursor channel, ABI v11): straight-alpha RGBA,
@@ -1133,6 +1141,19 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_send_input(h, &ev)
}
/// Send one stylus sample batch ( `PUNKTFUNK_PEN_BATCH_MAX`, oldest first) on the pen
/// plane. Gate on ``hostSupportsPen`` the core refuses toward a host without the cap.
/// Thread-safe; silently dropped after close (input is lossy by design).
public func sendPen(_ samples: [PunktfunkPenSample]) {
guard !samples.isEmpty else { return }
abiLock.lock()
defer { abiLock.unlock() }
guard let h = handle, !closeRequested else { return }
samples.withUnsafeBufferPointer { buf in
_ = punktfunk_connection_send_pen(h, buf.baseAddress, UInt32(buf.count))
}
}
/// Signal a **deliberate** user-initiated quit before ``close()``: the connection closes with
/// `QUIT_CLOSE_CODE` (81) so the host tears the session down immediately instead of holding the
/// keep-alive linger for a reconnect. Call only from an explicit "Disconnect" action NOT from a
@@ -0,0 +1,268 @@
// Apple Pencil state-full wire pen samples (design/pen-tablet-input.md §7).
//
// Every sample carries the COMPLETE pen state (in-range/touching/buttons + all axes) the
// host diffs consecutive samples and synthesizes down/up/button transitions itself, so a lost
// datagram self-heals and this file never sends edge events. Three sources feed one stream:
// UITouch contacts (with coalesced samples for full 240 Hz fidelity), the hover gesture
// (zOffset > 0 distinguishes a hovering Pencil from a trackpad pointer), and
// UIPencilInteraction (squeeze held barrel 1, double-tap a momentary barrel 2
// Apple Pencil has no hardware eraser end or barrel buttons, so these mappings are how
// host-side apps get their stylus button/eraser affordances).
//
// HEARTBEAT (wire contract see `PunktfunkPenSample` in punktfunk_core.h): while the pen is
// in range or touching, the last sample repeats every 100 ms even when nothing changed.
// UIKit is silent for a stationary Pencil, and the host force-releases the stroke after
// 200 ms without samples (its dead-client failsafe) the timer keeps a held stroke alive.
#if os(iOS)
import PunktfunkCore
import UIKit
final class PencilStream: NSObject, UIPencilInteractionDelegate {
enum Phase { case down, move, up, cancel }
/// One assembled batch ( `PUNKTFUNK_PEN_BATCH_MAX` samples) ready for the connection.
var send: (([PunktfunkPenSample]) -> Void)?
/// View-space point normalized [0,1] video coordinates (the letterbox mapping the
/// touch path already uses). nil until a mode is negotiated samples are dropped then.
var videoNorm: ((CGPoint) -> (Float, Float)?)?
private var inRange = false
private var touching = false
/// Squeeze held (mapped to wire BARREL1).
private var squeezeHeld = false
/// Whether this device/Pencil pair has demonstrated hover decides what a lift means:
/// hover-capable hardware keeps proximity (the hover recognizer owns the exit), anything
/// else leaves range on lift so the host never parks a phantom hovering pen.
private var sawHover = false
/// A hover gesture is live right now (routes ended-state hover callbacks to us even when
/// the recognizer's final zOffset reads 0).
private(set) var hoverActive = false
private var last = PencilStream.idleSample()
private var heartbeat: Timer?
// MARK: - Contact path (UITouch, `.pencil` only)
func touches(_ touches: Set<UITouch>, event: UIEvent?, phase: Phase, in view: UIView) {
// At most one Pencil exists; a set with several is UIKit batching phases of the same
// stylus the last one carries the freshest state.
guard let touch = touches.max(by: { $0.timestamp < $1.timestamp }) else { return }
switch phase {
case .down, .move:
touching = true
inRange = true
// Coalesced samples restore the Pencil's full capture rate (UIKit delivers at
// display cadence); oldest first, `dt_us` preserving their spacing.
let raw = event?.coalescedTouches(for: touch) ?? [touch]
var batch: [PunktfunkPenSample] = []
var prevTs: TimeInterval?
for t in raw.suffix(Int(PUNKTFUNK_PEN_BATCH_MAX)) {
guard let s = contactSample(t, in: view, prevTs: prevTs) else { continue }
prevTs = t.timestamp
batch.append(s)
}
emit(batch)
case .up:
touching = false
// Hover-capable hardware: lift back to hover, the recognizer exits range later.
// Otherwise a lift IS the range exit (mirror of the host's GameStream heuristic).
inRange = sawHover
var s = last
s.pressure = 0
s.state = stateBits()
if let posSample = contactSample(touch, in: view, prevTs: nil) {
s.x = posSample.x
s.y = posSample.y
}
emit([s])
case .cancel:
release()
}
}
// MARK: - Hover path (forwarded from the view's hover recognizer)
/// Returns whether the event was consumed as Pencil hover; `false` hands it back to the
/// pointer path. A hovering Pencil reports `zOffset > 0`; trackpad/mouse hover is 0.
func maybeHover(_ r: UIHoverGestureRecognizer, in view: UIView) -> Bool {
switch r.state {
case .began, .changed:
guard r.zOffset > 0 || hoverActive else { return false }
hoverActive = true
sawHover = true
inRange = true
touching = false
guard let (x, y) = videoNorm?(r.location(in: view)) else { return true }
var s = PencilStream.idleSample()
s.state = stateBits()
s.x = x
s.y = y
s.distance = UInt16((r.zOffset.clamped(to: 0...1) * 65534).rounded())
s.tilt_deg = Self.tiltDeg(altitude: r.altitudeAngle)
s.azimuth_deg = Self.azimuthDeg(r.azimuthAngle(in: view))
if #available(iOS 17.5, *) { s.roll_deg = Self.rollDeg(r.rollAngle) }
emit([s])
return true
case .ended, .cancelled, .failed:
guard hoverActive else { return false }
hoverActive = false
if !touching { release() }
return true
default:
return hoverActive
}
}
// MARK: - UIPencilInteractionDelegate (squeeze barrel 1 held, tap barrel 2 click)
@available(iOS 17.5, *)
func pencilInteraction(
_ interaction: UIPencilInteraction, didReceiveSqueeze squeeze: UIPencilInteraction.Squeeze
) {
switch squeeze.phase {
case .began:
squeezeHeld = true
case .ended, .cancelled:
squeezeHeld = false
default:
return
}
guard inRange || touching else { return }
var s = last
s.state = stateBits()
emit([s])
}
func pencilInteractionDidTap(_ interaction: UIPencilInteraction) {
guard inRange || touching else { return }
// A momentary barrel-2 click: press + release as two state-full samples in ONE batch
// the host's tracker emits the button press and release in order.
var press = last
press.state = stateBits() | UInt8(PUNKTFUNK_PEN_BARREL2)
var releaseS = last
releaseS.state = stateBits()
emit([press, releaseS])
}
// MARK: - Lifecycle
/// Session stop / view teardown: leave range so the host lifts anything held.
func reset() {
if inRange || touching { release() }
sawHover = false
hoverActive = false
squeezeHeld = false
}
private func release() {
touching = false
inRange = false
var s = last
s.pressure = 0
s.state = 0
emit([s])
}
// MARK: - Sample assembly
private func contactSample(
_ t: UITouch, in view: UIView, prevTs: TimeInterval?
) -> PunktfunkPenSample? {
guard let (x, y) = videoNorm?(t.location(in: view)) else { return nil }
var s = PencilStream.idleSample()
s.state = stateBits()
s.x = x
s.y = y
// maximumPossibleForce is 0 until the system knows the stylus full force then
// (binary-stylus semantics, matching the host's unknown-pressure rule).
let maxForce = t.maximumPossibleForce
s.pressure =
maxForce > 0
? UInt16((Double(t.force / maxForce).clamped(to: 0...1) * 65535).rounded())
: UInt16.max
s.distance = 0
s.tilt_deg = Self.tiltDeg(altitude: t.altitudeAngle)
s.azimuth_deg = Self.azimuthDeg(t.azimuthAngle(in: view))
if #available(iOS 17.5, *) { s.roll_deg = Self.rollDeg(t.rollAngle) }
if let prevTs {
s.dt_us = UInt16(((t.timestamp - prevTs) * 1_000_000).clamped(to: 0...65535))
}
return s
}
private func stateBits() -> UInt8 {
var bits: UInt8 = 0
if inRange || touching { bits |= UInt8(PUNKTFUNK_PEN_IN_RANGE) }
if touching { bits |= UInt8(PUNKTFUNK_PEN_TOUCHING) }
if squeezeHeld { bits |= UInt8(PUNKTFUNK_PEN_BARREL1) }
return bits
}
private func emit(_ batch: [PunktfunkPenSample]) {
guard !batch.isEmpty else { return }
last = batch[batch.count - 1]
last.dt_us = 0
send?(batch)
armHeartbeat()
}
/// The 100 ms keepalive while in range (see the file header). 80 ms leaves headroom
/// under the host's 200 ms failsafe even with one lost datagram.
private func armHeartbeat() {
heartbeat?.invalidate()
guard inRange || touching else {
heartbeat = nil
return
}
heartbeat = Timer.scheduledTimer(withTimeInterval: 0.08, repeats: true) {
[weak self] _ in
guard let self, self.inRange || self.touching else {
self?.heartbeat?.invalidate()
self?.heartbeat = nil
return
}
self.send?([self.last])
}
}
// MARK: - Angle conversions
/// Altitude (π/2 = perpendicular) wire tilt-from-normal in degrees, 0...90.
private static func tiltDeg(altitude: CGFloat) -> UInt8 {
UInt8((90 - altitude * 180 / .pi).rounded().clamped(to: 0...90))
}
/// Apple azimuth (0 along the view's +x axis, clockwise, y-down) wire azimuth
/// (0 = north/up on screen, clockwise): +90° offset.
private static func azimuthDeg(_ apple: CGFloat) -> UInt16 {
let deg = (apple * 180 / .pi + 90).truncatingRemainder(dividingBy: 360)
return UInt16((deg + 360).truncatingRemainder(dividingBy: 360).rounded()) % 360
}
/// Pencil Pro roll (radians, π...π) wire barrel roll 0...359°.
private static func rollDeg(_ roll: CGFloat) -> UInt16 {
let deg = (roll * 180 / .pi).truncatingRemainder(dividingBy: 360)
return UInt16(((deg + 360).truncatingRemainder(dividingBy: 360)).rounded()) % 360
}
/// All-unknown baseline: sentinel angles/distance, tool = pen (the eraser is host-side
/// state driven by the squeeze/tap mappings, not a hardware end).
private static func idleSample() -> PunktfunkPenSample {
PunktfunkPenSample(
x: 0, y: 0, pressure: 0,
distance: UInt16(PUNKTFUNK_PEN_DISTANCE_UNKNOWN),
azimuth_deg: UInt16(PUNKTFUNK_PEN_ANGLE_UNKNOWN),
roll_deg: UInt16(PUNKTFUNK_PEN_ANGLE_UNKNOWN),
dt_us: 0, state: 0,
tool: UInt8(PUNKTFUNK_PEN_TOOL_PEN),
tilt_deg: UInt8(PUNKTFUNK_PEN_TILT_UNKNOWN),
_reserved: (0, 0, 0))
}
}
extension Comparable {
fileprivate func clamped(to range: ClosedRange<Self>) -> Self {
min(max(self, range.lowerBound), range.upperBound)
}
}
#endif
@@ -223,7 +223,19 @@ public final class StreamLayerView: NSView {
/// when the Welcome carried `HOST_CAP_CURSOR` (only sessions that advertised the client
/// cap get it). Shapes cache by serial; state is latest-wins. Main-thread only.
private var cursorChannelActive = false
private var hostCursors: [UInt32: NSCursor] = [:]
/// A forwarded host cursor shape, cached RAW (not as a finished `NSCursor`) so the pointer can be
/// (re)built at the CURRENT video-fit scale see `scaledCursor`. The host forwards the bitmap in
/// host FRAMEBUFFER pixels, whose size tracks the host's display scaling (32 px at 100%, 96 px at
/// 300% DPI); scaling by the video fit keeps the pointer sized to the streamed desktop at any host
/// scaling instead of ballooning on a high-DPI host.
private struct HostCursorShape {
let cg: CGImage
let width: Int
let height: Int
let hotX: Int
let hotY: Int
}
private var hostCursors: [UInt32: HostCursorShape] = [:]
private var cursorState: PunktfunkConnection.CursorStateEvent?
/// Last `CursorRenderMode.clientDraws` told to the host (the §8 mid-stream render flip);
/// nil = nothing sent yet. Edge-detected by [`reconcileCursorRender`] from the live mouse
@@ -500,8 +512,8 @@ public final class StreamLayerView: NSView {
// video); hidden host pointer (or no shape yet) = invisible. Without the channel,
// M1 behavior: invisible local cursor, the composited host cursor is the visible one.
if cursorChannelActive, let st = cursorState, st.visible,
let host = hostCursors[st.serial] {
addCursorRect(bounds, cursor: host)
let shape = hostCursors[st.serial] {
addCursorRect(bounds, cursor: scaledCursor(shape))
} else {
addCursorRect(bounds, cursor: Self.invisibleCursor)
}
@@ -570,12 +582,12 @@ public final class StreamLayerView: NSView {
}
private func applyCursorShape(_ ev: PunktfunkConnection.CursorShapeEvent) {
guard let cursor = Self.makeCursor(ev) else {
guard let shape = Self.makeShape(ev) else {
streamInputLog.warning("cursor shape rejected (\(ev.width)x\(ev.height)) — keeping the previous cursor")
return
}
if hostCursors.count >= 64 { hostCursors.removeAll() } // degenerate host: reset
hostCursors[ev.serial] = cursor
hostCursors[ev.serial] = shape
if cursorState?.serial == ev.serial {
window?.invalidateCursorRects(for: self)
}
@@ -597,8 +609,10 @@ public final class StreamLayerView: NSView {
_ = (lastHint, hintOverride)
}
/// Build an `NSCursor` from a forwarded straight-alpha RGBA shape.
private static func makeCursor(_ ev: PunktfunkConnection.CursorShapeEvent) -> NSCursor? {
/// Decode a forwarded straight-alpha RGBA shape into a CGImage + hotspot. The on-screen SIZE is
/// NOT baked in here it is applied per-use in `scaledCursor` from the live video-fit scale, so
/// the same shape re-fits across window resizes / retina moves without a re-forward.
private static func makeShape(_ ev: PunktfunkConnection.CursorShapeEvent) -> HostCursorShape? {
let (w, h) = (ev.width, ev.height)
guard w > 0, h > 0, ev.rgba.count >= w * h * 4,
let provider = CGDataProvider(data: ev.rgba as CFData),
@@ -609,10 +623,40 @@ public final class StreamLayerView: NSView {
provider: provider, decode: nil, shouldInterpolate: false,
intent: .defaultIntent)
else { return nil }
let image = NSImage(cgImage: cg, size: NSSize(width: w, height: h))
return NSCursor(
image: image,
hotSpot: NSPoint(x: min(ev.hotX, w - 1), y: min(ev.hotY, h - 1)))
return HostCursorShape(
cg: cg, width: w, height: h,
hotX: min(ev.hotX, w - 1), hotY: min(ev.hotY, h - 1))
}
/// Points-per-host-pixel: the exact factor the video frame is aspect-fit into the view (the same
/// `AVMakeRect` fit `hostPoint`/`cgScreenPoint` use). The host forwards the pointer bitmap in host
/// framebuffer pixels the mode we drive is in the client's BACKING pixels, so on retina this is
/// ~1/backingScale and the pointer lands at its TRUE size relative to the streamed desktop
/// (crisp, 1:1 with the video) rather than the 2×-inflated pixel-as-points it used to be. Because
/// the bitmap grows with the host's display scaling (96 px at 300% DPI), scaling by this is what
/// keeps a high-DPI host from forwarding a giant pointer. Falls back to 1 before the first
/// mode/layout.
private func cursorFitScale() -> CGFloat {
guard let connection else { return 1 }
let mode = connection.currentMode()
guard mode.width > 0, mode.height > 0, bounds.width > 0, bounds.height > 0 else { return 1 }
let fit = AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)), insideRect: bounds)
guard fit.width > 0 else { return 1 }
return fit.width / CGFloat(mode.width)
}
/// Build the `NSCursor` for a cached shape at the CURRENT video-fit scale (see `cursorFitScale`).
/// Both the image size and the hotspot scale together so the click point stays true.
private func scaledCursor(_ shape: HostCursorShape) -> NSCursor {
let scale = cursorFitScale()
let sw = max(1, (CGFloat(shape.width) * scale).rounded())
let sh = max(1, (CGFloat(shape.height) * scale).rounded())
let image = NSImage(cgImage: shape.cg, size: NSSize(width: sw, height: sh))
let hot = NSPoint(
x: min(CGFloat(shape.hotX) * scale, sw - 1),
y: min(CGFloat(shape.hotY) * scale, sh - 1))
return NSCursor(image: image, hotSpot: hot)
}
/// Host video px CG GLOBAL screen coordinates (top-left origin, the
@@ -908,6 +952,11 @@ public final class StreamLayerView: NSView {
matchFollower?.noteSize(
widthPx: Int(px.width.rounded()), heightPx: Int(px.height.rounded()))
}
// The video-fit scale just changed (resize / retina move); rebuild the worn host pointer at
// the new scale so it tracks the video instead of freezing at its build-time size.
if captured, desktopMouse, cursorChannelActive {
window?.invalidateCursorRects(for: self)
}
}
public override func viewDidChangeBackingProperties() {
@@ -333,6 +333,13 @@ public final class StreamViewController: StreamViewControllerBase {
guard self?.captureEnabled == true else { return }
connection?.send(event)
}
// Apple Pencil the stylus plane, only against a pen-capable host (elsewhere the
// Pencil stays a finger, exactly as before). Same trust gate as touch.
streamView.penEnabled = connection.hostSupportsPen
streamView.onPenBatch = { [weak self, weak connection] batch in
guard self?.captureEnabled == true else { return }
connection?.sendPen(batch)
}
// Indirect pointer (mouse/trackpad) WITHOUT a lock absolute cursor + buttons + scroll.
// While the scene is pointer-LOCKED the GCMouse path owns motion AND buttons AND scroll, so
// the whole UIKit indirect path is gated off here (`gcMouseForwarding`). The trackpad and a
@@ -499,6 +506,8 @@ public final class StreamViewController: StreamViewControllerBase {
// onTouchEvent can still deliver the button-up.
streamView.resetTouchInput()
streamView.onTouchEvent = nil
streamView.onPenBatch = nil // after reset the pen's leave-range sample rides it
streamView.penEnabled = false
streamView.onPointerMoveAbs = nil
streamView.onPointerButton = nil
streamView.onScroll = nil
@@ -692,6 +701,12 @@ final class StreamLayerUIView: UIView {
/// Direct fingers / Pencil wire events: real touches in passthrough mode, or the
/// touch-driven mouse events (`TouchMouse`) in the trackpad/pointer modes.
var onTouchEvent: ((PunktfunkInputEvent) -> Void)?
/// Apple Pencil state-full pen sample batches (the stylus plane). Active only while
/// `penEnabled`; without it the Pencil stays on the finger path exactly as before.
var onPenBatch: (([PunktfunkPenSample]) -> Void)?
/// The host advertised `HOST_CAP_PEN`, so Pencil input splits out of the finger path onto
/// the pen plane independent of the touch-input mode (drawing must not depend on it).
var penEnabled = false
/// Indirect pointer (mouse/trackpad with no lock) absolute cursor moves.
var onPointerMoveAbs: ((HostPoint) -> Void)?
/// Indirect-pointer buttons (GameStream ids: 1=left 3=right); `down` = press.
@@ -715,10 +730,21 @@ final class StreamLayerUIView: UIView {
/// The finger route latched at gesture start a Settings change mid-gesture applies to
/// the NEXT touch, so one gesture never splits across input models.
private var fingerRoute: TouchInputMode?
/// The Apple Pencil pipeline (contacts + hover + squeeze/tap pen samples).
private lazy var pencil: PencilStream = {
let stream = PencilStream()
stream.send = { [weak self] batch in self?.onPenBatch?(batch) }
stream.videoNorm = { [weak self] point in
guard let h = self?.hostPoint(from: point) else { return nil }
return (Float(h.x) / Float(max(h.w - 1, 1)), Float(h.y) / Float(max(h.h - 1, 1)))
}
return stream
}()
/// Release anything the touch-driven mouse holds and forget gesture state session stop.
func resetTouchInput() {
touchMouse.reset()
pencil.reset() // leaves range the host lifts anything still inked
fingerRoute = nil
setSoftKeyboardVisible(false) // a stream that's gone takes its keyboard with it
}
@@ -755,6 +781,11 @@ final class StreamLayerUIView: UIView {
scrollPan.allowedScrollTypesMask = .all
scrollPan.allowedTouchTypes = []
addGestureRecognizer(scrollPan)
// Pencil squeeze / double-tap the pen plane's barrel buttons (no-op while
// `penEnabled` is false PencilStream ignores interactions out of range).
let pencilInteraction = UIPencilInteraction()
pencilInteraction.delegate = pencil
addInteraction(pencilInteraction)
#endif
backgroundColor = .black
}
@@ -779,17 +810,32 @@ final class StreamLayerUIView: UIView {
private enum TouchKind { case down, move, up, cancel }
/// Split a touch batch by kind: an INDIRECT POINTER (mouse/trackpad with no lock) drives
/// the host cursor as an absolute mouse; everything else (direct finger, Pencil) is a host
/// touch. Mixed batches are possible, so partition rather than branch on the first touch.
/// the host cursor as an absolute mouse; a Pencil goes to the pen plane when the host
/// supports it; everything else (direct finger and the Pencil toward a pen-less host)
/// is a host touch. Mixed batches are possible, so partition rather than branch on the
/// first touch.
private func route(_ touches: Set<UITouch>, event: UIEvent?, kind: TouchKind) {
var fingers: Set<UITouch> = []
var pencilTouches: Set<UITouch> = []
for touch in touches {
if touch.type == .indirectPointer {
handleIndirectPointer(touch, event: event, kind: kind)
} else if penEnabled, touch.type == .pencil {
pencilTouches.insert(touch)
} else {
fingers.insert(touch)
}
}
if !pencilTouches.isEmpty {
let phase: PencilStream.Phase =
switch kind {
case .down: .down
case .move: .move
case .up: .up
case .cancel: .cancel
}
pencil.touches(pencilTouches, event: event, phase: phase, in: self)
}
if !fingers.isEmpty { forwardFingers(fingers, kind: kind) }
}
@@ -861,8 +907,11 @@ final class StreamLayerUIView: UIView {
}
}
/// Button-less mouse/trackpad movement (no lock) absolute cursor move.
/// Button-less mouse/trackpad movement (no lock) absolute cursor move unless it is a
/// hovering PENCIL (`zOffset > 0`) on a pen-capable host, which becomes in-range pen
/// samples (hover preview with distance/tilt/azimuth) instead of a cursor move.
@objc private func handleHover(_ recognizer: UIHoverGestureRecognizer) {
if penEnabled, pencil.maybeHover(recognizer, in: self) { return }
switch recognizer.state {
case .began, .changed:
if let h = hostPoint(from: recognizer.location(in: self)) { onPointerMoveAbs?(h) }
+33 -3
View File
@@ -10,9 +10,17 @@ import {
showModal,
staticClasses,
} from "@decky/ui";
import { definePlugin, routerHook } from "@decky/api";
import { definePlugin, routerHook, toaster } from "@decky/api";
import { FC } from "react";
import { FaDownload, FaLock, FaLockOpen, FaPlay, FaSyncAlt, FaTv } from "react-icons/fa";
import {
FaDownload,
FaLock,
FaLockOpen,
FaPlay,
FaPlus,
FaSyncAlt,
FaTv,
} from "react-icons/fa";
import { PluginErrorBoundary } from "./boundary";
import {
applyUpdate,
@@ -31,7 +39,19 @@ import {
import { streamPin } from "./library";
import { PunktfunkRoute, ROUTE } from "./page";
import { PairModal } from "./pair";
import { ensureGamepadUiShortcut } from "./steam";
import { ensureGamepadUiShortcut, recreateShortcuts } from "./steam";
// Recovery action for "the Punktfunk library entry vanished" — recreates the visible shortcut.
// Deleting the shortcut (optionally + reinstalling the plugin) leaves a stale appId in Steam's
// CEF localStorage that self-heal fixes on the next mount, but this gives an in-session button
// that works even without a reload. Always ends in a toast so the tap has feedback.
async function recreatePunktfunkShortcut(): Promise<void> {
const appId = await recreateShortcuts();
toaster.toast({
title: "Punktfunk",
body: appId != null ? "Shortcut restored to your library" : "Couldn't create the shortcut",
});
}
// ----------------------------------------------------------------------------------------
// QAM panel — quick status + entry into the full page + one-tap stream for known hosts
@@ -190,6 +210,16 @@ const QamPanel: FC = () => {
{checking ? "Checking…" : "Check for updates"}
</ButtonItem>
</PanelSectionRow>
<PanelSectionRow>
<ButtonItem
layout="below"
description="Missing the Punktfunk entry in your library? This puts it back."
onClick={() => void recreatePunktfunkShortcut()}
>
<FaPlus style={{ marginRight: "0.5em" }} />
Recreate library shortcut
</ButtonItem>
</PanelSectionRow>
</PanelSection>
</>
);
+54 -2
View File
@@ -55,6 +55,29 @@ declare const collectionStore:
| { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void }
| undefined;
// SteamUI's appStore indexes every registered app/shortcut by appId; a remembered appId whose
// overview is gone was deleted out from under us (the user removed the library entry). We must
// verify this because the remembered appId lives in Steam's CEF localStorage — which survives a
// plugin UNINSTALL/REINSTALL — so a manually-deleted shortcut otherwise leaves a dangling appId
// that the reuse path below silently repoints (SetShortcut* on a dead id is a no-op), and the
// entry never comes back.
declare const appStore:
| { GetAppOverviewByAppID?: (appId: number) => unknown | null }
| undefined;
/** True if a remembered appId still maps to a live Steam shortcut. When appStore is unavailable
* we can't tell, so assume it exists better to keep reusing than risk a duplicate library
* entry from a false "missing". A confident null means the shortcut was deleted recreate. */
function shortcutStillExists(appId: number): boolean {
try {
const get = appStore?.GetAppOverviewByAppID;
if (!get) return true; // no way to verify — preserve the reuse path
return get(appId) != null;
} catch {
return true;
}
}
/** Set a shortcut's library visibility (best-effort, deferred the overview registers a moment
* after AddShortcut). Hides the stateful stream shortcut; keeps the gamepad-UI one visible. */
function setShortcutHidden(appId: number, hidden: boolean): void {
@@ -199,8 +222,10 @@ async function ensureStreamShortcut(): Promise<{ appId: number; runner: string }
const startDir = info.runner.replace(/\/[^/]*$/, ""); // the plugin's bin/ dir
void ensureControllerConfig(); // fire-and-forget — never blocks the launch
// Reuse the remembered shortcut only if it still exists — a stale appId (shortcut deleted, key
// outlived it across a reinstall) must fall through to AddShortcut, not be silently repointed.
const remembered = recall(STORAGE_KEY_STREAM);
if (remembered != null) {
if (remembered != null && shortcutStillExists(remembered)) {
SteamClient.Apps.SetShortcutExe(remembered, SHELL);
SteamClient.Apps.SetShortcutStartDir(remembered, startDir);
SteamClient.Apps.SetShortcutName(remembered, SHORTCUT_NAME);
@@ -236,8 +261,11 @@ export async function ensureGamepadUiShortcut(): Promise<number | null> {
// home). %command% expands to the shortcut exe (/bin/sh); the wrapper rides behind as an arg.
const launchOpts = `PF_BROWSE=1 %command% "${info.runner}"`;
// Reuse the remembered entry only if it still exists; a stale appId (deleted shortcut whose
// localStorage key survived a plugin reinstall) falls through to AddShortcut so the visible
// library entry actually comes back instead of repointing a dead id.
let appId = recall(STORAGE_KEY_UI);
if (appId != null) {
if (appId != null && shortcutStillExists(appId)) {
SteamClient.Apps.SetShortcutExe(appId, SHELL);
SteamClient.Apps.SetShortcutStartDir(appId, startDir);
SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME);
@@ -256,6 +284,30 @@ export async function ensureGamepadUiShortcut(): Promise<number | null> {
}
}
/**
* Force the visible "Punktfunk" library entry back into existence the recovery button for
* "my shortcut disappeared". Drops any remembered appId that no longer maps to a live shortcut
* (so it can't shadow a fresh AddShortcut), then re-ensures. Safe to press anytime: a shortcut
* that still exists is left in place (no duplicate); a missing one is recreated. Covers the case
* self-heal-on-mount can't deleting the shortcut WITHOUT reinstalling (no mount no ensure).
* Returns the (new or existing) visible appId, or null on failure.
*/
export async function recreateShortcuts(): Promise<number | null> {
for (const key of [STORAGE_KEY_STREAM, STORAGE_KEY_UI]) {
const id = recall(key);
if (id != null && !shortcutStillExists(id)) {
try {
localStorage.removeItem(artKey(id)); // stale art marker for the dead appId
localStorage.removeItem(key);
} catch {
/* ignore */
}
}
}
// Recreate the visible entry now; the hidden stream shortcut re-registers lazily on next launch.
return ensureGamepadUiShortcut();
}
/** Launch the stateless gamepad-UI shortcut (console home) from the plugin, e.g. a QAM button. */
export async function launchGamepadUi(): Promise<void> {
const appId = await ensureGamepadUiShortcut();
+12
View File
@@ -405,6 +405,16 @@ pub type CursorChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetCursorChannelRequest) -> Result<()> + Send + Sync,
>;
/// The mid-stream cursor-render flip (`IOCTL_SET_CURSOR_FORWARD`, proto v6) as a host-facade
/// closure — same contract as [`CursorChannelSender`]. `bool` = declare the IddCx hardware
/// cursor (`true`) or stand it down (`false`; the host facade additionally forces the same-mode
/// re-commit that actualises the OS's software-cursor default). The capturer drives this from
/// its secure-desktop watch: UAC/Winlogon render only through the software-cursor path, so a
/// path pinned to the hardware cursor never presents them (the 0.18.0 secure-desktop
/// regression).
#[cfg(target_os = "windows")]
pub type CursorForwardSender = std::sync::Arc<dyn Fn(bool) -> Result<()> + Send + Sync>;
// One-time PipeWire library init, shared by the video (portal) and audio capture threads.
#[cfg(target_os = "linux")]
pub mod pwinit;
@@ -491,6 +501,7 @@ pub fn open_idd_push(
keepalive: Box<dyn Send>,
sender: FrameChannelSender,
cursor_sender: Option<CursorChannelSender>,
cursor_forward: Option<CursorForwardSender>,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open(
target,
@@ -501,6 +512,7 @@ pub fn open_idd_push(
keepalive,
sender,
cursor_sender,
cursor_forward,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
+102 -1
View File
@@ -408,6 +408,16 @@ pub struct IddPushCapturer {
/// shared slot) destroys the driver's cursor worker — the section here survives, so the
/// channel is re-delivered on ring recreates.
cursor_sender: Option<crate::CursorChannelSender>,
/// The cursor-render flip sender (`IOCTL_SET_CURSOR_FORWARD`) — the secure-desktop guard's
/// actuator. UAC/Winlogon render only through the OS's software-cursor path (its default on
/// every mode commit); with our hardware cursor declared (and re-declared on every
/// swap-chain assign) that path never comes back, and the secure desktop never presents —
/// the stream freezes on the last normal-desktop frame for the whole UAC/lock interaction.
/// [`Self::poll_secure_desktop`] flips the declare off/on at the secure-desktop edges.
cursor_forward: Option<crate::CursorForwardSender>,
/// The secure-desktop guard's edge state: `true` = the poller reports a secure input
/// desktop and the declare is currently stood down.
secure_active: bool,
/// The CAPTURE mouse model is active — the HOST composites the pointer into the frame
/// (see cursor_blend.rs for why DWM cannot: a declared IddCx hardware cursor is forever).
composite_cursor: bool,
@@ -657,7 +667,6 @@ impl IddPushCapturer {
/// instead of tearing the display down (audit §5.1 — no more 20 s black bail). "Failure" includes the
/// driver not attaching to the ring within a few seconds (e.g. a hybrid-GPU render mismatch).
#[allow(clippy::too_many_arguments)]
#[allow(clippy::too_many_arguments)]
pub fn open(
target: WinCaptureTarget,
preferred: Option<(u32, u32, u32)>,
@@ -667,6 +676,7 @@ impl IddPushCapturer {
keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime.
@@ -679,6 +689,7 @@ impl IddPushCapturer {
pyrowave,
sender,
cursor_sender,
cursor_forward,
) {
Ok(mut me) => {
me._keepalive = keepalive;
@@ -697,6 +708,7 @@ impl IddPushCapturer {
pyrowave: bool,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
// selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor
@@ -720,6 +732,7 @@ impl IddPushCapturer {
luid,
sender.clone(),
cursor_sender.clone(),
cursor_forward.clone(),
) {
Ok(me) => Ok(me),
Err(e) => {
@@ -754,6 +767,7 @@ impl IddPushCapturer {
drv,
sender,
cursor_sender,
cursor_forward,
)
.context("IDD-push rebind to the driver's reported render adapter")
}
@@ -770,6 +784,7 @@ impl IddPushCapturer {
luid: LUID,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> Result<Self> {
let (pw, ph, _hz) = preferred
.context("IDD push needs the negotiated mode (WxH) to size the shared ring")?;
@@ -1046,6 +1061,17 @@ impl IddPushCapturer {
.unwrap_or((0, 0, i32::MAX, i32::MAX));
cursor_poll::CursorPoller::spawn(target.target_id, rect)
});
// Heal the driver's persisted cursor-forward state: a session that died on the
// secure desktop (client drops at the lock screen — the common case) leaves the
// per-target desired state `false`, and the NEXT session's channel delivery would
// adopt UNdeclared (the exact cross-session composite trap of §8.6). A fresh
// session always starts declared; the secure-desktop guard re-disables if the
// secure desktop is (still) up, via its first `poll_secure_desktop` edge.
if let (Some(_), Some(fwd)) = (cursor_shared.as_ref(), cursor_forward.as_ref()) {
if let Err(e) = fwd(true) {
tracing::debug!("cursor-forward reset at open failed (pre-v6 driver?): {e:#}");
}
}
tracing::info!(
target_id = target.target_id,
@@ -1108,6 +1134,8 @@ impl IddPushCapturer {
cursor_shared,
cursor_poll,
cursor_sender,
cursor_forward,
secure_active: false,
composite_cursor: composite_forced,
composite_forced,
cursor_blend: None,
@@ -1883,8 +1911,71 @@ impl IddPushCapturer {
Some((tex, srv))
}
/// The secure-desktop guard (the 0.18.0 UAC/Winlogon regression). UAC consent and Winlogon
/// live on the SECURE desktop, which the OS renders through the software-cursor path — its
/// per-mode-commit default. With this session's IddCx hardware cursor declared (and
/// re-declared by the driver on every swap-chain assign), that path never materialises, the
/// secure desktop never presents into our swap-chain, and the stream freezes on the last
/// normal-desktop frame for the entire UAC/lock interaction. On the poller's secure edge:
/// stand the declare down (`SET_CURSOR_FORWARD` off — the driver stops its per-assign
/// re-declare — plus the host facade's forced same-mode re-commit that actualises the
/// software cursor); on dismissal, re-declare. Runs on the capture/encode thread every tick
/// (it must keep running while frames are stalled — that is exactly the state it exits).
fn poll_secure_desktop(&mut self) {
let Some(fwd) = self.cursor_forward.as_ref() else {
return;
};
// Sessions with a declare possibly in play: the channel session that declared it, and
// the forced-composite session whose (reused) driver monitor may still run an earlier
// session's cursor worker. A plain session on a clean target has no poller — no guard.
if self.cursor_shared.is_none() && !self.composite_forced {
return;
}
let secure = self
.cursor_poll
.as_ref()
.is_some_and(|p| p.secure_desktop());
if secure == self.secure_active {
return;
}
self.secure_active = secure;
if secure {
tracing::info!(
target_id = self.target_id,
"secure desktop (UAC/Winlogon) active — standing the IddCx hardware-cursor \
declare down so the OS software-cursor path can render it"
);
if let Err(e) = fwd(false) {
tracing::warn!(
"secure-desktop cursor-forward stand-down failed (secure content may stay \
invisible this session): {e:#}"
);
}
} else {
tracing::info!(
target_id = self.target_id,
"secure desktop dismissed — restoring the cursor render model"
);
// Re-declare only for the session that RUNS the cursor channel; a forced-composite
// session never wanted the declare (leaving the driver's desired state off also
// stops a reused worker's per-assign re-declares for good — the next channel
// session's open-time reset re-arms it).
if self.cursor_shared.is_some() {
if let Err(e) = fwd(true) {
tracing::warn!(
"secure-desktop cursor-forward re-enable failed (client-drawn cursor \
may double with a composited one): {e:#}"
);
}
}
}
}
fn try_consume(&mut self) -> Result<Option<CapturedFrame>> {
self.log_driver_status_once();
// The secure-desktop guard first: while UAC/Winlogon is up there may be NO fresh frames
// at all — this edge is what brings them back.
self.poll_secure_desktop();
// Follow the display: a "Use HDR" flip recreates the ring at the matching format.
self.poll_display_hdr();
// Recover-or-drop (GB1): if a descriptor change triggered a recreate but no fresh frame has resumed
@@ -2486,6 +2577,16 @@ fn warn_444_hdr_downgrade_once() {
impl Drop for IddPushCapturer {
fn drop(&mut self) {
// A channel session ending while the secure-desktop guard is engaged must not leave the
// driver's per-target desired state off — the next session's channel delivery would
// adopt UNdeclared and silently run the composite model (§8.6's cross-session trap).
// The open-time reset also covers this (host-crash case); this is the orderly-teardown
// belt.
if self.secure_active && self.cursor_shared.is_some() {
if let Some(fwd) = self.cursor_forward.as_ref() {
let _ = fwd(true);
}
}
self.slots.clear();
// The shared header section (`MappedSection`), the frame-ready `event` (`OwnedHandle`) and the
// broker's WUDFHost process handle free themselves via RAII (unmap view, then close handle) —
@@ -29,8 +29,8 @@ use windows::Win32::Graphics::Gdi::{
BI_RGB, DIB_RGB_COLORS, HBITMAP, HDC,
};
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, OpenInputDesktop, SetThreadDesktop, DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS,
HDESK,
CloseDesktop, GetUserObjectInformationW, OpenInputDesktop, SetThreadDesktop,
DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS, HDESK, UOI_NAME,
};
use windows::Win32::UI::HiDpi::{
SetThreadDpiAwarenessContext, DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2,
@@ -63,6 +63,11 @@ struct Shape {
pub(super) struct CursorPoller {
slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>>,
stop: Arc<AtomicBool>,
/// The input desktop is a SECURE desktop (Winlogon — UAC consent / lock / logon). Classified
/// on every reattach; the capturer polls it to stand the IddCx hardware-cursor declare down
/// while the secure desktop needs the software-cursor path to render (see
/// `IddPushCapturer::poll_secure_desktop`).
secure: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
@@ -75,7 +80,11 @@ impl CursorPoller {
const INTERVAL: Duration = Duration::from_millis(4);
/// Unconditional input-desktop reattach cadence — catches secure-desktop (UAC/lock) switches
/// without a failure signal (`GetCursorInfo` on a stale desktop *succeeds* with stale data).
const REATTACH: Duration = Duration::from_secs(2);
/// 250 ms, not the original 2 s: the reattach now also feeds [`Self::secure_desktop`], which
/// gates when the secure desktop becomes VISIBLE in the stream (the hardware-cursor
/// stand-down) — a 2 s freeze at every UAC prompt is user-visible, ~4 `OpenInputDesktop`
/// syscalls/s are not.
const REATTACH: Duration = Duration::from_millis(250);
/// Spawn the poller for the virtual display `target_id`. `rect` = the target's desktop rect
/// (`source_desktop_rect` order: x, y, w, h) — cursor positions are desktop-global; the
@@ -84,15 +93,21 @@ impl CursorPoller {
pub(super) fn spawn(target_id: u32, rect: (i32, i32, i32, i32)) -> Self {
let slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>> = Arc::new(Mutex::new(None));
let stop = Arc::new(AtomicBool::new(false));
let (slot_t, stop_t) = (slot.clone(), stop.clone());
let secure = Arc::new(AtomicBool::new(false));
let (slot_t, stop_t, secure_t) = (slot.clone(), stop.clone(), secure.clone());
let thread = std::thread::Builder::new()
.name("pf-cursor-poll".into())
.spawn(move || run(target_id, rect, &slot_t, &stop_t))
.spawn(move || run(target_id, rect, &slot_t, &stop_t, &secure_t))
.ok();
if thread.is_none() {
tracing::warn!("cursor poller thread spawn failed — cursor falls back to driver shm");
}
Self { slot, stop, thread }
Self {
slot,
stop,
secure,
thread,
}
}
/// The latest overlay snapshot (`None` until the first successful shape rasterisation).
@@ -100,6 +115,12 @@ impl CursorPoller {
self.slot.lock().unwrap_or_else(|p| p.into_inner()).clone()
}
/// Whether the input desktop is currently a SECURE desktop (UAC consent / Winlogon lock or
/// logon). Latched by the poll thread on its reattach cadence (≤ [`Self::REATTACH`] stale).
pub(super) fn secure_desktop(&self) -> bool {
self.secure.load(Ordering::Relaxed)
}
/// Whether the worker thread is (still) alive — `false` degrades the capturer to the shm read.
pub(super) fn alive(&self) -> bool {
self.thread.as_ref().is_some_and(|t| !t.is_finished())
@@ -121,6 +142,7 @@ fn run(
rect: (i32, i32, i32, i32),
slot: &Mutex<Option<pf_frame::CursorOverlay>>,
stop: &AtomicBool,
secure: &AtomicBool,
) {
// Physical-pixel coordinates on this thread regardless of the process's DPI awareness:
// `rect` comes from CCD (always physical), and a DPI-virtualized `GetCursorInfo` position
@@ -130,7 +152,8 @@ fn run(
let _ = unsafe { SetThreadDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) };
let mut desktop = DesktopBinding::default();
desktop.reattach(); // best-effort: already on winsta0\default if this fails
// best-effort: already on winsta0\default if this fails
publish_secure(secure, desktop.reattach());
let mut last_attach = Instant::now();
let mut shape: Option<Shape> = None;
@@ -143,7 +166,7 @@ fn run(
std::thread::sleep(CursorPoller::INTERVAL);
if last_attach.elapsed() >= CursorPoller::REATTACH {
last_attach = Instant::now();
desktop.reattach();
publish_secure(secure, desktop.reattach());
}
let mut ci = CURSORINFO {
@@ -155,7 +178,7 @@ fn run(
if unsafe { GetCursorInfo(&mut ci) }.is_err() {
// Desktop went away under us (secure-desktop switch mid-call) — rebind and retry
// next tick; the slot keeps its last snapshot meanwhile.
desktop.reattach();
publish_secure(secure, desktop.reattach());
last_attach = Instant::now();
continue;
}
@@ -218,13 +241,25 @@ fn run(
}
}
/// Store a reattach's secure-desktop verdict (`None` = classification unavailable — keep the
/// previous state rather than flapping the capturer's hardware-cursor stand-down).
fn publish_secure(secure: &AtomicBool, verdict: Option<bool>) {
if let Some(s) = verdict {
secure.store(s, Ordering::Relaxed);
}
}
/// The thread's owned input-desktop handle — the [`SendInputInjector`] reattach model
/// (`pf-inject` sendinput.rs): keep the current binding, swap on demand, close exactly once.
#[derive(Default)]
struct DesktopBinding(Option<HDESK>);
impl DesktopBinding {
fn reattach(&mut self) {
/// Rebind to the CURRENT input desktop. Returns whether that desktop is a SECURE one
/// (`UOI_NAME` != "Default": "Winlogon" during UAC consent / lock / logon) — `None` when the
/// input desktop could not be opened, in which case the binding (and the caller's secure
/// state) stays put.
fn reattach(&mut self) -> Option<bool> {
const GENERIC_ALL: u32 = 0x1000_0000;
// SAFETY: `OpenInputDesktop`/`SetThreadDesktop`/`CloseDesktop` take only by-value args.
// `OpenInputDesktop` yields an owned `HDESK` only on `Ok`; it is either installed (and the
@@ -238,6 +273,7 @@ impl DesktopBinding {
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
) {
Ok(h) => {
let secure = desktop_is_secure(h);
if SetThreadDesktop(h).is_ok() {
if let Some(old) = self.0.replace(h) {
let _ = CloseDesktop(old);
@@ -245,13 +281,40 @@ impl DesktopBinding {
} else {
let _ = CloseDesktop(h);
}
Some(secure)
}
Err(_) => { /* not privileged for this desktop; stay put */ }
Err(_) => None, // not privileged for this desktop; stay put
}
}
}
}
/// `UOI_NAME` of `h` != "Default" — i.e. the input desktop is Winlogon (UAC consent / lock /
/// logon) or a screen-saver desktop, both of which need the OS's software-cursor render path.
/// Unnameable desktops read as NOT secure: the only in-contract failure is a too-small buffer,
/// and misreading secure-as-normal merely keeps today's behavior for a beat.
fn desktop_is_secure(h: HDESK) -> bool {
let mut name = [0u16; 64]; // "Default"/"Winlogon"/"Screen-saver" all fit with room to spare
let mut needed = 0u32;
// SAFETY: `h` is the live desktop handle the caller just opened; `name`/`needed` are live
// out-params sized exactly as passed; the call writes at most `nlength` bytes.
let ok = unsafe {
GetUserObjectInformationW(
windows::Win32::Foundation::HANDLE(h.0),
UOI_NAME,
Some(name.as_mut_ptr().cast()),
(name.len() * 2) as u32,
Some(&mut needed),
)
};
if ok.is_err() {
return false;
}
let len = name.iter().position(|&c| c == 0).unwrap_or(name.len());
let name = String::from_utf16_lossy(&name[..len]);
!name.eq_ignore_ascii_case("Default")
}
impl Drop for DesktopBinding {
fn drop(&mut self) {
if let Some(h) = self.0.take() {
+19 -12
View File
@@ -80,12 +80,13 @@ pub const fn interface_guid_fields() -> (u32, u16, u16, [u8; 8]) {
/// and the desktop model gets exclusion + forwarding. Nothing existing changed; against a v5
/// driver the unknown IOCTL fails and the host logs + keeps the declared-at-ADD behavior.
/// v6 tail ext (no bump, the `AddRequest` luminance-tail discipline):
/// [`control::AddReply::cursor_excluded`] — the driver reports whether the ADDed monitor's OS
/// target already carries a hardware-cursor declare from an earlier session. A declare is
/// IRREVOCABLE for the target's life (remote-desktop-sweep §8.6, proven on-glass): DWM never
/// composites the software cursor back into that target's frames, and the sticky state survives
/// monitor REMOVE→ADD because the host hands every client a STABLE target id. The host uses the
/// flag to self-composite the pointer (GDI poller + blend) in sessions that never negotiate the
/// [`control::AddReply::cursor_excluded`] — the driver reports whether its ADAPTER already
/// carries a hardware-cursor declare from an earlier session. A declare is IRREVOCABLE
/// (remote-desktop-sweep §8.6, proven on-glass) and its exclusion reaches EVERY later monitor of
/// the adapter, not just the declaring target (on-glass 2026-07-23: a declare on one target left
/// a different client's fresh target cursor-less): DWM never composites the software cursor back
/// into any of the adapter's frames until the adapter resets. The host uses the flag to
/// self-composite the pointer (GDI poller + blend) in sessions that never negotiate the
/// cursor channel — without it those sessions are silently cursor-less. Both skews degrade
/// cleanly: an old driver writes only the 20-byte reply prefix (host reads `0` = unknown/clean),
/// an old host retrieves a 20-byte buffer (driver writes just the prefix).
@@ -217,9 +218,10 @@ pub mod control {
/// `DuplicateHandle`, then [`IOCTL_SET_FRAME_CHANNEL`]). Reported per-ADD, not per-open, so a
/// WUDFHost restart between sessions can never leave the host duplicating into a dead process.
pub wudf_pid: u32,
/// Non-zero = this monitor's OS target already carries an IRREVOCABLE hardware-cursor
/// declare from an earlier session (remote-desktop-sweep §8.6): DWM excludes the pointer
/// from every frame on this target, forever, and a session without the cursor channel must
/// Non-zero = the ADAPTER already carries an IRREVOCABLE hardware-cursor declare from an
/// earlier session (remote-desktop-sweep §8.6; reach is adapter-wide, not per-target —
/// on-glass 2026-07-23): DWM excludes the pointer from every frame on every monitor until
/// the adapter resets, and a session without the cursor channel must
/// self-composite (GDI poller + blend) or stream a cursor-less desktop. Appended after
/// [`ADD_REPLY_LEGACY_SIZE`] under the same dual-size discipline as the `AddRequest`
/// luminance tail: an un-upgraded driver writes only the legacy prefix (the host's
@@ -1278,14 +1280,19 @@ mod tests {
target_id: 262,
resolved_monitor_id: 7,
wudf_pid: 4242,
cursor_excluded: 1,
};
let rbytes = bytemuck::bytes_of(&reply);
assert_eq!(rbytes.len(), 20);
assert_eq!(rbytes.len(), 24);
assert_eq!(*bytemuck::from_bytes::<control::AddReply>(rbytes), reply);
// resolved_monitor_id occupies the old `_reserved` slot at offset 12 — byte-compatible.
assert_eq!(rbytes[12..16], 7u32.to_le_bytes());
// The v2 duplication-target pid trails at offset 16.
assert_eq!(rbytes[16..20], 4242u32.to_le_bytes());
// The cursor-excluded tail rides after the legacy boundary; an un-upgraded driver writes
// only the prefix, so a zero-filled tail reads as "unknown/clean" (see the field docs).
assert_eq!(rbytes[20..24], 1u32.to_le_bytes());
assert_eq!(control::ADD_REPLY_LEGACY_SIZE, 20);
}
#[test]
@@ -1330,8 +1337,8 @@ mod tests {
req
);
assert_eq!(bytes[8..12], 2560u32.to_le_bytes());
// The compat window: v4/v5 are additive over v3, so the host floor stays at 3.
assert_eq!(PROTOCOL_VERSION, 5);
// The compat window: v4v6 are additive over v3, so the host floor stays at 3.
assert_eq!(PROTOCOL_VERSION, 6);
assert_eq!(MIN_DRIVER_PROTOCOL_VERSION, 3);
}
+6 -4
View File
@@ -32,10 +32,12 @@ pub struct WinCaptureTarget {
/// duplicates the sealed frame channel's handles INTO (`idd_push::ChannelBroker`). `0` = unknown
/// (a pre-v2 pairing can't occur — the version handshake is hard — so this only guards misuse).
pub wudf_pid: u32,
/// The ADD reply flagged this target as carrying an IRREVOCABLE IddCx hardware-cursor declare
/// from an earlier session (remote-desktop-sweep §8.6): DWM excludes the pointer from its
/// frames forever, so a session WITHOUT the cursor channel must self-composite (the IDD-push
/// capturer's forced-composite gate) or the streamed desktop has no cursor at all.
/// The ADD reply flagged the ADAPTER as carrying an IRREVOCABLE IddCx hardware-cursor declare
/// from an earlier session (remote-desktop-sweep §8.6; reach is adapter-wide, not per-target —
/// on-glass 2026-07-23, a GameStream session's fresh target streamed cursor-less): DWM
/// excludes the pointer from its frames until adapter reset, so a session WITHOUT the cursor
/// channel must self-composite (the IDD-push capturer's forced-composite gate) or the
/// streamed desktop has no cursor at all.
pub cursor_excluded: bool,
}
+5
View File
@@ -61,5 +61,10 @@ windows = { version = "0.62", features = [
"Win32_System_StationsAndDesktops",
"Win32_System_Threading",
"Win32_UI_Input_KeyboardAndMouse",
# Synthetic pointer devices (PT_PEN / PT_TOUCH) — the pen/touch injectors. The device
# create/destroy + POINTER_TYPE_INFO live under Controls in this metadata layout;
# InjectSyntheticPointerInput + POINTER_*_INFO under Input_Pointer.
"Win32_UI_Controls",
"Win32_UI_Input_Pointer",
"Win32_UI_WindowsAndMessaging",
] }
+396
View File
@@ -0,0 +1,396 @@
//! Virtual tablet ("Punktfunk Pen"): a uinput stylus device carrying the pen plane's full
//! fidelity — pressure, tilt, barrel roll, hover distance, eraser, barrel buttons
//! (design/pen-tablet-input.md §5).
//!
//! Deliberately a **uinput device, not a compositor-protocol citizen**: no virtual-tablet
//! protocol exists in EI, the RemoteDesktop portal, KWin `fake_input`, or wlroots — while
//! every compositor consumes evdev tablets via libinput and forwards them to apps over
//! `zwp_tablet_v2` with nothing to configure. udev's `input_id` builtin classifies the device
//! from its capabilities (`BTN_TOOL_PEN` + `ABS_X/Y` ⇒ `ID_INPUT_TABLET`), so Krita/GIMP/
//! Xournal++ see a real pen. Output mapping is the compositor's own tablet-mapping default
//! (single output ⇒ correct; multi-monitor pinning is the documented follow-up, which is why
//! the device carries a stable, distinctive identity to key rules on).
//!
//! The consumer feeds it [`PenTransition`]s straight from the core's
//! [`PenTracker`](punktfunk_core::quic::PenTracker); this file only translates transitions to
//! evdev events and groups them into SYN frames so proximity-enter carries its position in the
//! same frame (libinput would otherwise report an entry at a stale point).
//!
//! ioctl numbers/struct layouts mirror `gamepad.rs` (verified against the same kernel
//! generation); each backend file stays self-contained by convention.
use anyhow::{bail, Result};
use punktfunk_core::quic::{PenSample, PenTool, PenTransition, PEN_BARREL1, PEN_BARREL2};
use std::os::fd::{AsRawFd, OwnedFd};
// ioctls (x86_64).
const UI_DEV_CREATE: libc::c_ulong = 0x5501;
const UI_DEV_DESTROY: libc::c_ulong = 0x5502;
const UI_DEV_SETUP: libc::c_ulong = 0x405c_5503;
const UI_ABS_SETUP: libc::c_ulong = 0x401c_5504;
const UI_SET_EVBIT: libc::c_ulong = 0x4004_5564;
const UI_SET_KEYBIT: libc::c_ulong = 0x4004_5565;
const UI_SET_PROPBIT: libc::c_ulong = 0x4004_556e;
// input-event-codes.h subset.
const EV_SYN: u16 = 0x00;
const EV_KEY: u16 = 0x01;
const EV_ABS: u16 = 0x03;
const SYN_REPORT: u16 = 0;
const ABS_X: u16 = 0x00;
const ABS_Y: u16 = 0x01;
/// Barrel roll rides ABS_Z (the Wacom Art-Pen rotation convention); libinput normalizes the
/// declared min..max onto its 0..360° rotation axis.
const ABS_Z: u16 = 0x02;
const ABS_PRESSURE: u16 = 0x18;
const ABS_DISTANCE: u16 = 0x19;
const ABS_TILT_X: u16 = 0x1a;
const ABS_TILT_Y: u16 = 0x1b;
const BTN_TOOL_PEN: u16 = 0x140;
const BTN_TOOL_RUBBER: u16 = 0x141;
const BTN_TOUCH: u16 = 0x14a;
const BTN_STYLUS: u16 = 0x14b;
const BTN_STYLUS2: u16 = 0x14c;
/// The pen writes on the display it is mapped to (a "screen tablet"), not a desk pad —
/// libinput then maps the full ABS range onto the output rect, exactly the wire's normalized
/// coordinate contract.
const INPUT_PROP_DIRECT: libc::c_int = 0x01;
/// Full-scale wire pressure (u16) → the declared 0..4095 axis.
const PRESSURE_SHIFT: u32 = 4;
/// Wire hover distance (u16, 0xFFFF = unknown) → the declared 0..1023 axis.
const DISTANCE_SHIFT: u32 = 6;
const ABS_RANGE: f32 = 65535.0;
#[repr(C)]
struct InputId {
bustype: u16,
vendor: u16,
product: u16,
version: u16,
}
#[repr(C)]
struct UinputSetup {
id: InputId,
name: [u8; 80],
ff_effects_max: u32,
}
#[repr(C)]
#[derive(Default, Clone, Copy)]
struct AbsInfo {
value: i32,
minimum: i32,
maximum: i32,
fuzz: i32,
flat: i32,
resolution: i32,
}
#[repr(C)]
struct UinputAbsSetup {
code: u16,
_pad: u16,
absinfo: AbsInfo,
}
#[repr(C)]
#[derive(Clone, Copy)]
struct InputEventRaw {
time: libc::timeval,
type_: u16,
code: u16,
value: i32,
}
fn ioctl_int(fd: i32, req: libc::c_ulong, arg: libc::c_int, what: &str) -> Result<()> {
// SAFETY: every caller passes a UI_SET_*/UI_DEV_* request whose argument the kernel reads
// as a plain int; `fd` is a live uinput fd owned by the caller. No memory is handed over.
if unsafe { libc::ioctl(fd, req, arg) } < 0 {
bail!("{what}: {}", std::io::Error::last_os_error());
}
Ok(())
}
fn ioctl_ptr<T>(fd: i32, req: libc::c_ulong, arg: *mut T, what: &str) -> Result<()> {
// SAFETY: every caller passes a pointer to a live, initialized `#[repr(C)]` struct matching
// the request's expected layout (UI_DEV_SETUP/UI_ABS_SETUP); the kernel reads it during the
// call and retains nothing.
if unsafe { libc::ioctl(fd, req, arg) } < 0 {
bail!("{what}: {}", std::io::Error::last_os_error());
}
Ok(())
}
/// The active tool's evdev key, one in proximity at a time (Wacom semantics — the core's
/// [`PenTracker`](punktfunk_core::quic::PenTracker) already re-enters proximity on a tool
/// switch, so this only tracks which key to release on `ProximityOut`).
fn tool_key(tool: PenTool) -> u16 {
match tool {
PenTool::Eraser => BTN_TOOL_RUBBER,
// Unknown = a newer client's future tool — nearest ink-capable behavior is the pen.
PenTool::Pen | PenTool::Unknown => BTN_TOOL_PEN,
}
}
/// One per-session virtual tablet. Created lazily on the first pen batch (a session that never
/// draws never creates a device), destroyed with the session (Drop → `UI_DEV_DESTROY`).
pub struct VirtualPen {
fd: OwnedFd,
/// The tool key currently held in proximity (release target for `ProximityOut`).
tool: u16,
/// Whether the current SYN frame already carries a Motion — the frame-split trigger for
/// consecutive samples in one batch.
frame_has_motion: bool,
/// Whether the current SYN frame has any unflushed events.
frame_dirty: bool,
}
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
use std::os::fd::FromRawFd;
// SAFETY: `c"/dev/uinput"` is a 'static NUL-terminated C string literal; `open` reads it
// as a path, returns a fresh fd (or -1) and retains nothing.
let raw = unsafe {
libc::open(
c"/dev/uinput".as_ptr(),
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if raw < 0 {
bail!(
"open /dev/uinput: {} (install the udev rule granting the 'input' group access \
see scripts/60-punktfunk.rules and add the user to the 'input' group)",
std::io::Error::last_os_error()
);
}
// SAFETY: `raw >= 0` here, a freshly-opened fd owned nowhere else; `OwnedFd` becomes the
// unique owner and closes it exactly once on drop.
let fd = unsafe { OwnedFd::from_raw_fd(raw) };
ioctl_int(raw, UI_SET_EVBIT, EV_KEY as i32, "UI_SET_EVBIT(EV_KEY)")?;
ioctl_int(raw, UI_SET_EVBIT, EV_ABS as i32, "UI_SET_EVBIT(EV_ABS)")?;
for key in [
BTN_TOOL_PEN,
BTN_TOOL_RUBBER,
BTN_TOUCH,
BTN_STYLUS,
BTN_STYLUS2,
] {
ioctl_int(raw, UI_SET_KEYBIT, key as i32, "UI_SET_KEYBIT")?;
}
ioctl_int(
raw,
UI_SET_PROPBIT,
INPUT_PROP_DIRECT,
"UI_SET_PROPBIT(DIRECT)",
)?;
// Position spans the full u16 range; `resolution` (units/mm) only feeds libinput's mm
// math (nothing pen-relevant), but tablets without one trip its missing-resolution
// fixup — 100 declares a plausible ~655 mm drawing surface.
let pos = AbsInfo {
minimum: 0,
maximum: 65535,
resolution: 100,
..Default::default()
};
// Tilt in degrees from vertical, per evdev convention; resolution = units/radian (57
// ⇔ 1 unit = 1°, what the Wacom driver declares).
let tilt = AbsInfo {
minimum: -90,
maximum: 90,
resolution: 57,
..Default::default()
};
for (code, info) in [
(ABS_X, pos),
(ABS_Y, pos),
(
ABS_PRESSURE,
AbsInfo {
minimum: 0,
maximum: 4095,
..Default::default()
},
),
(
ABS_DISTANCE,
AbsInfo {
minimum: 0,
maximum: 1023,
..Default::default()
},
),
(ABS_TILT_X, tilt),
(ABS_TILT_Y, tilt),
(
// Barrel roll: libinput maps the declared range linearly onto 0..360°.
ABS_Z,
AbsInfo {
minimum: 0,
maximum: 359,
..Default::default()
},
),
] {
let mut a = UinputAbsSetup {
code,
_pad: 0,
absinfo: info,
};
ioctl_ptr(raw, UI_ABS_SETUP, &mut a, "UI_ABS_SETUP")?;
}
// A stable, distinctive identity (pid.codes open-source VID) so compositor
// tablet-mapping rules — GNOME's per-`vendor:product` gsettings path, sway
// `map_to_output` — can target exactly this device.
let mut setup = UinputSetup {
id: InputId {
bustype: 0x0006, // BUS_VIRTUAL
vendor: 0x1209,
product: 0x5046, // "PF"
version: 1,
},
name: [0; 80],
ff_effects_max: 0,
};
let name = b"Punktfunk Pen";
setup.name[..name.len()].copy_from_slice(name);
ioctl_ptr(raw, UI_DEV_SETUP, &mut setup, "UI_DEV_SETUP")?;
ioctl_int(raw, UI_DEV_CREATE, 0, "UI_DEV_CREATE")?;
tracing::info!("virtual tablet created (Punktfunk Pen, uinput)");
Ok(VirtualPen {
fd,
tool: BTN_TOOL_PEN,
frame_has_motion: false,
frame_dirty: false,
})
}
fn emit(&self, type_: u16, code: u16, value: i32) {
let ev = InputEventRaw {
time: libc::timeval {
tv_sec: 0,
tv_usec: 0,
},
type_,
code,
value,
};
// SAFETY: `ev` is a live local `#[repr(C)]` all-integer struct (no padding: timeval=16 +
// u16 + u16 + i32 = 24), so every byte is initialized; the slice spans exactly `ev`'s
// bytes and is used immediately below with no concurrent mutation.
let bytes = unsafe {
std::slice::from_raw_parts(
&ev as *const _ as *const u8,
std::mem::size_of::<InputEventRaw>(),
)
};
// Best-effort like the gamepad path: a full kernel queue drops the event; pen samples
// are state-full, so the next frame re-syncs axes (and the tracker re-syncs state).
// SAFETY: `self.fd` stays open for the synchronous call; `write` only reads
// `bytes.len()` bytes from the still-live local and retains nothing.
let _ = unsafe {
libc::write(
self.fd.as_raw_fd(),
bytes.as_ptr() as *const libc::c_void,
bytes.len(),
)
};
}
fn flush(&mut self) {
if self.frame_dirty {
self.emit(EV_SYN, SYN_REPORT, 0);
self.frame_dirty = false;
self.frame_has_motion = false;
}
}
fn motion(&mut self, s: &PenSample) {
self.emit(EV_ABS, ABS_X, (s.x * ABS_RANGE) as i32);
self.emit(EV_ABS, ABS_Y, (s.y * ABS_RANGE) as i32);
self.emit(EV_ABS, ABS_PRESSURE, (s.pressure >> PRESSURE_SHIFT) as i32);
if s.distance != punktfunk_core::quic::PEN_DISTANCE_UNKNOWN {
self.emit(EV_ABS, ABS_DISTANCE, (s.distance >> DISTANCE_SHIFT) as i32);
}
// Polar → tiltX/tiltY needs both angles; azimuth clockwise from north, so east (90°)
// tilts +X and south (180°, toward the user) tilts +Y — the evdev/W3C signs.
if s.tilt_deg != punktfunk_core::quic::PEN_TILT_UNKNOWN
&& s.azimuth_deg != punktfunk_core::quic::PEN_ANGLE_UNKNOWN
{
let az = (s.azimuth_deg as f32).to_radians();
let tilt = s.tilt_deg as f32;
self.emit(EV_ABS, ABS_TILT_X, (tilt * az.sin()).round() as i32);
self.emit(EV_ABS, ABS_TILT_Y, (-tilt * az.cos()).round() as i32);
}
if s.roll_deg != punktfunk_core::quic::PEN_ANGLE_UNKNOWN {
self.emit(EV_ABS, ABS_Z, (s.roll_deg % 360) as i32);
}
self.frame_dirty = true;
self.frame_has_motion = true;
}
/// Apply one decoded batch's transitions (the core tracker's output, in its documented
/// order), grouping them into SYN frames: a frame closes before a `ProximityIn` (an entry
/// is a new instant — and must carry its own position, not inherit the stale frame) and
/// before a second `Motion` (consecutive samples are consecutive instants), plus a final
/// close. So `[ProxIn, Motion, TipDown]` lands as ONE frame — libinput reports the entry
/// already at the right point with contact — while a drag batch's `[Motion, Motion]`
/// stays two.
pub fn apply_batch(&mut self, transitions: &[PenTransition]) {
for t in transitions {
match t {
PenTransition::ProximityIn { tool } => {
self.flush();
self.tool = tool_key(*tool);
self.emit(EV_KEY, self.tool, 1);
self.frame_dirty = true;
}
PenTransition::Motion { sample } => {
if self.frame_has_motion {
self.flush();
}
self.motion(sample);
}
PenTransition::TipDown => {
self.emit(EV_KEY, BTN_TOUCH, 1);
self.frame_dirty = true;
}
PenTransition::ButtonsChanged { pressed, released } => {
for (bit, key) in [(PEN_BARREL1, BTN_STYLUS), (PEN_BARREL2, BTN_STYLUS2)] {
if pressed & bit != 0 {
self.emit(EV_KEY, key, 1);
self.frame_dirty = true;
}
if released & bit != 0 {
self.emit(EV_KEY, key, 0);
self.frame_dirty = true;
}
}
}
PenTransition::TipUp => {
self.emit(EV_KEY, BTN_TOUCH, 0);
self.emit(EV_ABS, ABS_PRESSURE, 0);
self.frame_dirty = true;
}
PenTransition::ProximityOut => {
self.emit(EV_KEY, self.tool, 0);
self.frame_dirty = true;
}
}
}
self.flush();
}
}
impl Drop for VirtualPen {
fn drop(&mut self) {
// SAFETY: `self.fd` is still open (OwnedFd closes only after this body returns);
// UI_DEV_DESTROY takes no pointer argument. Errors are moot on teardown.
let _ = unsafe { libc::ioctl(self.fd.as_raw_fd(), UI_DEV_DESTROY, 0) };
}
}
@@ -0,0 +1,556 @@
//! Windows synthetic-pointer injection (design/pen-tablet-input.md §6): a per-session `PT_PEN`
//! device carrying the pen plane's full fidelity — pressure (rescaled to Windows' 0..1024),
//! polar tilt → tiltX/tiltY, barrel roll on `rotation` (0..359 — Windows Ink renders Pencil
//! Pro roll natively), barrel button, eraser (`PEN_FLAG_INVERTED`/`ERASER`), hover — plus a
//! `PT_TOUCH` device that closes the long-standing SendInput touch no-op for wire touches.
//!
//! Both follow Apollo's proven recipe (`design/apollo-comparison.md`): synthetic pointer state
//! goes STALE if not re-injected (~100 ms), so each device runs a small refresh thread that
//! re-asserts the last frame every [`REFRESH_MS`] while the pen is in range / contacts are
//! held — a stationary stylus must not hover-out (and a held finger must not auto-lift) just
//! because no new samples arrived. `CreateSyntheticPointerDevice` needs Win10 1809+;
//! [`crate::pen_supported`] probes it, so older hosts simply never advertise pen.
//!
//! Frame grouping mirrors the Linux uinput backend: a proximity-enter is injected together
//! with its position (never at a stale point), tip edges get their own DOWN/UP frames, and a
//! range-leave is a final frame without `INRANGE`.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::{Context, Result};
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{
PenSample, PenTool, PenTransition, PEN_ANGLE_UNKNOWN, PEN_BARREL1, PEN_TILT_UNKNOWN,
};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use windows::Win32::Foundation::POINT;
use windows::Win32::UI::Controls::{
CreateSyntheticPointerDevice, DestroySyntheticPointerDevice, HSYNTHETICPOINTERDEVICE,
POINTER_FEEDBACK_DEFAULT, POINTER_TYPE_INFO, POINTER_TYPE_INFO_0,
};
use windows::Win32::UI::Input::Pointer::{
InjectSyntheticPointerInput, POINTER_FLAGS, POINTER_FLAG_DOWN, POINTER_FLAG_FIRSTBUTTON,
POINTER_FLAG_INCONTACT, POINTER_FLAG_INRANGE, POINTER_FLAG_NEW, POINTER_FLAG_UP,
POINTER_FLAG_UPDATE, POINTER_INFO, POINTER_PEN_INFO, POINTER_TOUCH_INFO,
};
use windows::Win32::UI::WindowsAndMessaging::{
GetSystemMetrics, PEN_FLAG_BARREL, PEN_FLAG_ERASER, PEN_FLAG_INVERTED, PEN_FLAG_NONE,
PEN_MASK_PRESSURE, PEN_MASK_ROTATION, PEN_MASK_TILT_X, PEN_MASK_TILT_Y, PT_PEN, PT_TOUCH,
SM_CXVIRTUALSCREEN, SM_CYVIRTUALSCREEN, SM_XVIRTUALSCREEN, SM_YVIRTUALSCREEN, TOUCH_FLAG_NONE,
TOUCH_MASK_NONE,
};
/// Re-inject cadence while state is held (in-range pen / touching contacts). Apollo uses
/// 50 ms against the ~100 ms staleness window; 40 ms keeps two refreshes inside it.
const REFRESH_MS: u64 = 40;
/// Windows pen pressure is 0..1024 (vs the wire's full-scale u16).
const WIN_PEN_PRESSURE_MAX: u32 = 1024;
/// Map a normalized [0,1] coordinate pair onto virtual-desktop pixels — the same surface the
/// SendInput absolute mouse targets, so pen, touch, and pointer all land identically.
fn to_screen(x: f32, y: f32) -> POINT {
// SAFETY: `GetSystemMetrics` takes a constant index and reads global metrics; no memory in.
let (vx, vy, vw, vh) = unsafe {
(
GetSystemMetrics(SM_XVIRTUALSCREEN),
GetSystemMetrics(SM_YVIRTUALSCREEN),
GetSystemMetrics(SM_CXVIRTUALSCREEN).max(1),
GetSystemMetrics(SM_CYVIRTUALSCREEN).max(1),
)
};
POINT {
x: vx + (x.clamp(0.0, 1.0) * (vw - 1) as f32) as i32,
y: vy + (y.clamp(0.0, 1.0) * (vh - 1) as f32) as i32,
}
}
/// An owned `HSYNTHETICPOINTERDEVICE` (destroyed exactly once on drop).
struct Device(HSYNTHETICPOINTERDEVICE);
// SAFETY: the handle is a plain kernel object identifier with no thread affinity —
// `InjectSyntheticPointerInput`/`DestroySyntheticPointerDevice` are documented callable from
// any thread; ownership transfer/sharing does not alias memory.
unsafe impl Send for Device {}
impl Drop for Device {
fn drop(&mut self) {
// SAFETY: `self.0` is the device this wrapper uniquely owns; destroyed once here.
unsafe { DestroySyntheticPointerDevice(self.0) };
}
}
/// Probe: can this Windows build create a synthetic pen device (Win10 1809+)?
pub fn synthetic_pen_available() -> bool {
// SAFETY: FFI create with by-value args; on success the returned handle is destroyed
// immediately by the `Device` wrapper, exactly once.
match unsafe { CreateSyntheticPointerDevice(PT_PEN, 1, POINTER_FEEDBACK_DEFAULT) } {
Ok(h) => {
drop(Device(h));
true
}
Err(_) => false,
}
}
/// The tracked pen state a refresh tick re-asserts.
#[derive(Default)]
struct PenState {
in_range: bool,
touching: bool,
barrel: bool,
eraser: bool,
x: f32,
y: f32,
pressure: u16,
tilt_deg: u8,
azimuth_deg: u16,
roll_deg: u16,
}
struct PenShared {
dev: Device,
state: PenState,
/// First injection failure logs at WARN (see `TouchShared::fail_warned`).
fail_warned: bool,
}
/// One per-session virtual pen (the Windows sibling of the Linux uinput tablet — same
/// [`PenTransition`] consumer API). Wire barrel button 2 has no Windows pen equivalent
/// (one barrel + eraser is the platform model) and is ignored here.
pub struct VirtualPen {
shared: Arc<Mutex<PenShared>>,
stop: Arc<AtomicBool>,
refresher: Option<std::thread::JoinHandle<()>>,
// Batch-local frame grouping (single-threaded within apply_batch).
edge_down: bool,
edge_up: bool,
is_new: bool,
frame_dirty: bool,
frame_has_motion: bool,
}
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
// SAFETY: FFI create with by-value args; the handle's sole owner becomes `Device`.
let dev = unsafe { CreateSyntheticPointerDevice(PT_PEN, 1, POINTER_FEEDBACK_DEFAULT) }
.context("CreateSyntheticPointerDevice(PT_PEN) — needs Windows 10 1809+")?;
let shared = Arc::new(Mutex::new(PenShared {
dev: Device(dev),
state: PenState::default(),
fail_warned: false,
}));
let stop = Arc::new(AtomicBool::new(false));
// The staleness guard: re-assert the last frame while in range so a stationary pen
// (native plane: between 100 ms heartbeats; GameStream: indefinitely) never hovers out.
let refresher = {
let shared = Arc::clone(&shared);
let stop = Arc::clone(&stop);
std::thread::Builder::new()
.name("pf-pen-refresh".into())
.spawn(move || {
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(std::time::Duration::from_millis(REFRESH_MS));
let s = &mut *shared.lock().unwrap();
if s.state.in_range {
inject_pen(s, POINTER_FLAG_UPDATE, false);
}
}
})
.context("spawn pen refresh thread")?
};
tracing::info!("virtual pen created (Windows synthetic pointer, PT_PEN)");
Ok(VirtualPen {
shared,
stop,
refresher: Some(refresher),
edge_down: false,
edge_up: false,
is_new: false,
frame_dirty: false,
frame_has_motion: false,
})
}
/// Apply one batch of tracker transitions — same grouping contract as the Linux backend:
/// `[ProxIn, Motion, TipDown]` is ONE frame (entry lands at its position, in contact),
/// consecutive `Motion`s split, tip edges own their frames, range-leave is a final
/// no-`INRANGE` frame.
pub fn apply_batch(&mut self, transitions: &[PenTransition]) {
for t in transitions {
match t {
PenTransition::ProximityIn { tool } => {
self.flush();
let mut s = self.shared.lock().unwrap();
s.state.in_range = true;
s.state.eraser = *tool == PenTool::Eraser;
drop(s);
self.is_new = true;
self.frame_dirty = true;
}
PenTransition::Motion { sample } => {
if self.frame_has_motion {
self.flush();
}
self.set_axes(sample);
self.frame_dirty = true;
self.frame_has_motion = true;
}
PenTransition::TipDown => {
self.shared.lock().unwrap().state.touching = true;
self.edge_down = true;
self.frame_dirty = true;
}
PenTransition::ButtonsChanged { pressed, released } => {
let mut s = self.shared.lock().unwrap();
if pressed & PEN_BARREL1 != 0 {
s.state.barrel = true;
}
if released & PEN_BARREL1 != 0 {
s.state.barrel = false;
}
drop(s);
self.frame_dirty = true;
}
PenTransition::TipUp => {
self.shared.lock().unwrap().state.touching = false;
self.edge_up = true;
self.frame_dirty = true;
self.flush(); // UP owns its frame (still INRANGE)
}
PenTransition::ProximityOut => {
self.shared.lock().unwrap().state.in_range = false;
self.frame_dirty = true;
self.flush(); // final frame without INRANGE
}
}
}
self.flush();
}
fn set_axes(&mut self, s: &PenSample) {
let mut sh = self.shared.lock().unwrap();
sh.state.x = s.x;
sh.state.y = s.y;
sh.state.pressure = s.pressure;
sh.state.tilt_deg = s.tilt_deg;
sh.state.azimuth_deg = s.azimuth_deg;
sh.state.roll_deg = s.roll_deg;
}
fn flush(&mut self) {
if !self.frame_dirty {
return;
}
let edge = if self.edge_down {
POINTER_FLAG_DOWN
} else if self.edge_up {
POINTER_FLAG_UP
} else {
POINTER_FLAG_UPDATE
};
inject_pen(&mut self.shared.lock().unwrap(), edge, self.is_new);
self.edge_down = false;
self.edge_up = false;
self.is_new = false;
self.frame_dirty = false;
self.frame_has_motion = false;
}
}
impl Drop for VirtualPen {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(h) = self.refresher.take() {
let _ = h.join();
}
// The device itself dies with `shared` (Device::drop) — Windows releases any held
// in-range/contact state when the synthetic device is destroyed.
}
}
/// Build + inject one pen frame from tracked state. `edge` is DOWN/UP/UPDATE;
/// `INRANGE`/`INCONTACT` derive from the state itself.
fn inject_pen(sh: &mut PenShared, edge: POINTER_FLAGS, is_new: bool) {
let st = &sh.state;
let mut flags = edge;
if st.in_range {
flags |= POINTER_FLAG_INRANGE;
}
if st.touching {
flags |= POINTER_FLAG_INCONTACT | POINTER_FLAG_FIRSTBUTTON;
}
if is_new {
flags |= POINTER_FLAG_NEW;
}
let mut pen_flags = PEN_FLAG_NONE;
if st.barrel {
pen_flags |= PEN_FLAG_BARREL;
}
if st.eraser {
pen_flags |= PEN_FLAG_INVERTED;
if st.touching {
pen_flags |= PEN_FLAG_ERASER;
}
}
let mut pen_mask = PEN_MASK_PRESSURE;
// Contact needs a nonzero pressure to ink; hover reports 0 (Windows convention).
let pressure = if st.touching {
((st.pressure as u32 * WIN_PEN_PRESSURE_MAX) / u16::MAX as u32).max(1)
} else {
0
};
let (mut tilt_x, mut tilt_y) = (0i32, 0i32);
if st.tilt_deg != PEN_TILT_UNKNOWN && st.azimuth_deg != PEN_ANGLE_UNKNOWN {
let az = (st.azimuth_deg as f32).to_radians();
let tilt = st.tilt_deg as f32;
tilt_x = (tilt * az.sin()).round() as i32;
tilt_y = (-tilt * az.cos()).round() as i32;
pen_mask |= PEN_MASK_TILT_X | PEN_MASK_TILT_Y;
}
let mut rotation = 0u32;
if st.roll_deg != PEN_ANGLE_UNKNOWN {
rotation = (st.roll_deg % 360) as u32;
pen_mask |= PEN_MASK_ROTATION;
}
let pt = to_screen(st.x, st.y);
let info = POINTER_TYPE_INFO {
r#type: PT_PEN,
Anonymous: POINTER_TYPE_INFO_0 {
penInfo: POINTER_PEN_INFO {
pointerInfo: POINTER_INFO {
pointerType: PT_PEN,
pointerId: 0,
pointerFlags: flags,
ptPixelLocation: pt,
ptPixelLocationRaw: pt,
..Default::default()
},
penFlags: pen_flags,
penMask: pen_mask,
pressure,
rotation,
tiltX: tilt_x,
tiltY: tilt_y,
},
},
};
// SAFETY: `sh.dev.0` is the live device this wrapper owns; the one-element array is a live
// stack value the call only reads. Best-effort like every injector write — a transient
// failure (desktop switch) is healed by the next refresh tick re-asserting state.
if let Err(e) = unsafe { InjectSyntheticPointerInput(sh.dev.0, &[info]) } {
if !sh.fail_warned {
sh.fail_warned = true;
tracing::warn!(error = %e, "pen inject failed");
} else {
tracing::trace!(error = %e, "pen inject failed (transient)");
}
} else {
sh.fail_warned = false;
}
}
/// One live wire-touch contact. `slot` is the SMALL, DENSE pointer id handed to Windows —
/// synthetic-pointer injection rejects arbitrary large ids, and clients (Moonlight's
/// `pointerId` especially) send exactly those, so wire ids compact into the lowest free slot
/// for the contact's lifetime (Apollo's slot-contiguity rule; the on-glass symptom of passing
/// wire ids through was pen working while every touch silently failed to inject).
#[derive(Clone, Copy)]
struct Contact {
id: u32,
slot: u32,
x: f32,
y: f32,
}
/// Lowest slot not held by a live contact.
fn free_slot(contacts: &[Contact]) -> u32 {
let mut slot = 0u32;
while contacts.iter().any(|c| c.slot == slot) {
slot += 1;
}
slot
}
/// Windows can inject at most this many simultaneous synthetic touch contacts.
const MAX_CONTACTS: usize = 10;
struct TouchShared {
dev: Device,
contacts: Vec<Contact>,
/// First injection failure logs at WARN (an on-glass "touch does nothing" must be
/// visible in the host log); repeats stay at trace.
fail_warned: bool,
}
/// The `PT_TOUCH` device servicing wire `TouchDown/Move/Up` events — closes the SendInput
/// touch no-op. Contacts are keyed by the wire's finger id; every frame re-injects the FULL
/// active set (the synthetic-pointer contract), and a refresh thread re-asserts held contacts
/// against the ~100 ms staleness auto-lift.
pub struct SyntheticTouch {
shared: Arc<Mutex<TouchShared>>,
stop: Arc<AtomicBool>,
refresher: Option<std::thread::JoinHandle<()>>,
}
impl SyntheticTouch {
pub fn create() -> Result<SyntheticTouch> {
// SAFETY: FFI create with by-value args; the handle's sole owner becomes `Device`.
let dev = unsafe {
CreateSyntheticPointerDevice(PT_TOUCH, MAX_CONTACTS as u32, POINTER_FEEDBACK_DEFAULT)
}
.context("CreateSyntheticPointerDevice(PT_TOUCH) — needs Windows 10 1809+")?;
let shared = Arc::new(Mutex::new(TouchShared {
dev: Device(dev),
contacts: Vec::new(),
fail_warned: false,
}));
let stop = Arc::new(AtomicBool::new(false));
let refresher = {
let shared = Arc::clone(&shared);
let stop = Arc::clone(&stop);
std::thread::Builder::new()
.name("pf-touch-refresh".into())
.spawn(move || {
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(std::time::Duration::from_millis(REFRESH_MS));
let s = &mut *shared.lock().unwrap();
if !s.contacts.is_empty() {
inject_touch_frame(s, None);
}
}
})
.context("spawn touch refresh thread")?
};
tracing::info!("virtual touchscreen created (Windows synthetic pointer, PT_TOUCH)");
Ok(SyntheticTouch {
shared,
stop,
refresher: Some(refresher),
})
}
/// Apply one wire touch event (`code` = finger id, pixel x/y against the
/// `flags = (w << 16) | h` reference extent, exactly like `MouseMoveAbs`).
pub fn apply(&mut self, ev: &InputEvent) {
let (w, h) = ((ev.flags >> 16) as f32, (ev.flags & 0xFFFF) as f32);
if (w < 1.0 || h < 1.0) && ev.kind != InputKind::TouchUp {
return; // the documented zero-extent drop, as for MouseMoveAbs
}
let (x, y) = (ev.x as f32 / w.max(1.0), ev.y as f32 / h.max(1.0));
let s = &mut *self.shared.lock().unwrap();
match ev.kind {
InputKind::TouchDown => {
match s.contacts.iter().position(|c| c.id == ev.code) {
Some(i) => (s.contacts[i].x, s.contacts[i].y) = (x, y),
None if s.contacts.len() < MAX_CONTACTS => {
let slot = free_slot(&s.contacts);
s.contacts.push(Contact {
id: ev.code,
slot,
x,
y,
})
}
None => return, // beyond the platform max — drop, never evict a live finger
}
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_DOWN)));
}
InputKind::TouchMove => {
match s.contacts.iter().position(|c| c.id == ev.code) {
Some(i) => {
(s.contacts[i].x, s.contacts[i].y) = (x, y);
inject_touch_frame(s, None);
}
// A move for an unknown id (its DOWN was dropped/lost): synthesize the
// contact so the stroke self-heals, like the pen plane does.
None if s.contacts.len() < MAX_CONTACTS => {
let slot = free_slot(&s.contacts);
s.contacts.push(Contact {
id: ev.code,
slot,
x,
y,
});
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_DOWN)));
}
None => {}
}
}
InputKind::TouchUp => {
let Some(idx) = s.contacts.iter().position(|c| c.id == ev.code) else {
return;
};
// The UP frame still carries the lifting contact (with UP flags), then it
// leaves the active set.
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_UP)));
s.contacts.remove(idx);
}
_ => {}
}
}
}
impl Drop for SyntheticTouch {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(h) = self.refresher.take() {
let _ = h.join();
}
}
}
/// Inject the full active-contact frame; `edge` marks one contact's DOWN/UP transition (by
/// WIRE id — everyone else is a held UPDATE). Windows sees the compacted `slot` ids only.
fn inject_touch_frame(sh: &mut TouchShared, edge: Option<(u32, POINTER_FLAGS)>) {
let contacts = &sh.contacts;
if contacts.is_empty() {
return;
}
let mut frame: Vec<POINTER_TYPE_INFO> = Vec::with_capacity(contacts.len());
for c in contacts {
let mut flags = POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT | POINTER_FLAG_UPDATE;
if let Some((id, e)) = edge {
if id == c.id {
if e == POINTER_FLAG_DOWN {
flags = POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT | POINTER_FLAG_DOWN;
} else if e == POINTER_FLAG_UP {
flags = POINTER_FLAG_UP; // contact + range end together
}
}
}
let pt = to_screen(c.x, c.y);
frame.push(POINTER_TYPE_INFO {
r#type: PT_TOUCH,
Anonymous: POINTER_TYPE_INFO_0 {
touchInfo: POINTER_TOUCH_INFO {
pointerInfo: POINTER_INFO {
pointerType: PT_TOUCH,
pointerId: c.slot,
pointerFlags: flags,
ptPixelLocation: pt,
ptPixelLocationRaw: pt,
..Default::default()
},
touchFlags: TOUCH_FLAG_NONE,
touchMask: TOUCH_MASK_NONE,
..Default::default()
},
},
});
}
// SAFETY: `sh.dev.0` is the live owned device; `frame` is a live Vec the call only reads.
// Best-effort — a transient failure heals on the next event/refresh re-assertion.
if let Err(e) = unsafe { InjectSyntheticPointerInput(sh.dev.0, &frame) } {
if !sh.fail_warned {
sh.fail_warned = true;
tracing::warn!(error = %e, contacts = frame.len(), "touch inject failed");
} else {
tracing::trace!(error = %e, "touch inject failed (transient)");
}
} else {
sh.fail_warned = false;
}
}
@@ -46,6 +46,11 @@ const XBUTTON2: u32 = 0x0002;
pub struct SendInputInjector {
desktop: Option<HDESK>,
/// PT_TOUCH synthetic device, created on the first wire-touch event (a session that never
/// touches never creates one). `None` after a failed create (pre-1809) — touch then stays
/// the historical no-op.
touch: Option<crate::pen::SyntheticTouch>,
touch_failed: bool,
}
// SAFETY: `SendInputInjector` holds only an `Option<HDESK>` (a desktop handle). The host creates
@@ -58,7 +63,11 @@ unsafe impl Send for SendInputInjector {}
impl SendInputInjector {
pub fn open() -> Result<Self> {
let mut me = Self { desktop: None };
let mut me = Self {
desktop: None,
touch: None,
touch_failed: false,
};
me.reattach_input_desktop(); // best-effort
tracing::info!("SendInput injector ready (Win32 KeyboardAndMouse)");
Ok(me)
@@ -324,15 +333,33 @@ impl InputInjector for SendInputInjector {
}
self.send(&inputs)
}
// Gamepad goes through the XUSB backend. Touch: no SendInput equivalent -> no-op.
// Gamepad goes through the XUSB backend.
InputKind::GamepadButton
| InputKind::GamepadAxis
| InputKind::GamepadState
| InputKind::GamepadRemove
| InputKind::GamepadArrival
| InputKind::TouchDown
| InputKind::TouchMove
| InputKind::TouchUp => Ok(()),
| InputKind::GamepadArrival => Ok(()),
// Wire touch → the PT_TOUCH synthetic pointer device (design/pen-tablet-input.md
// §6; closes the historical SendInput no-op). Lazily created — a session that never
// touches never creates one; a pre-1809 create failure latches back to the no-op.
InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp => {
if self.touch.is_none() && !self.touch_failed {
match crate::pen::SyntheticTouch::create() {
Ok(t) => self.touch = Some(t),
Err(e) => {
self.touch_failed = true;
tracing::warn!(
error = %format!("{e:#}"),
"touch: synthetic pointer unavailable — wire touch stays a no-op"
);
}
}
}
if let Some(t) = self.touch.as_mut() {
t.apply(event);
}
Ok(())
}
}
}
}
+68
View File
@@ -197,6 +197,50 @@ pub fn text_input_supported() -> bool {
false
}
/// Whether this host can inject full-fidelity stylus input (`HOST_CAP_PEN` —
/// design/pen-tablet-input.md): Linux only, via the [`pen::VirtualPen`] uinput tablet, so the
/// probe is "can we open /dev/uinput" (the same permission the virtual gamepads need) plus the
/// `PUNKTFUNK_PEN=0` operator kill-switch. Consulted at Welcome time; clients without the bit
/// keep folding pen into touch/pointer. Windows PT_PEN synthetic pointers are the design's P3.
#[cfg(target_os = "linux")]
pub fn pen_supported() -> bool {
if std::env::var("PUNKTFUNK_PEN").as_deref() == Ok("0") {
return false;
}
// SAFETY: 'static NUL-terminated path literal; `open` returns a fresh fd (or -1) and
// retains nothing.
let fd = unsafe {
libc::open(
c"/dev/uinput".as_ptr(),
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if fd < 0 {
return false;
}
// SAFETY: `fd >= 0` is the fd opened above, owned by no one else; closed exactly once here.
unsafe { libc::close(fd) };
true
}
/// Windows: pen (and touch) inject via synthetic pointer devices — available on Win10 1809+,
/// probed by actually creating (and immediately destroying) a PT_PEN device. Same
/// `PUNKTFUNK_PEN=0` kill-switch as Linux. The probe result also stands in for PT_TOUCH
/// (both APIs arrived together in 1809).
#[cfg(target_os = "windows")]
pub fn pen_supported() -> bool {
if std::env::var("PUNKTFUNK_PEN").as_deref() == Ok("0") {
return false;
}
pen::synthetic_pen_available()
}
/// See the Linux/Windows variants — no pen injection elsewhere.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn pen_supported() -> bool {
false
}
#[path = "inject/service.rs"]
mod service;
pub use service::InjectorService;
@@ -362,6 +406,30 @@ pub mod gamepad {
pub fn pump_rumble(&mut self, _send: impl FnMut(u16, u16, u16)) {}
}
}
/// Linux: the "Punktfunk Pen" uinput virtual tablet (design/pen-tablet-input.md §5) — the
/// per-session stylus device the native pen plane injects through.
#[cfg(target_os = "linux")]
#[path = "inject/linux/pen.rs"]
pub mod pen;
/// Windows: PT_PEN/PT_TOUCH synthetic pointer devices (design/pen-tablet-input.md §6).
/// `pen::VirtualPen` here is the PT_PEN device; `pen::SyntheticTouch` backs the SendInput
/// injector's wire-touch path.
#[cfg(target_os = "windows")]
#[path = "inject/windows/pointer_windows.rs"]
pub mod pen;
/// Stub — pen injection needs the Linux uinput tablet or Windows synthetic pointers;
/// `pen_supported()` is false here, so no host advertises the cap and no batches arrive.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub mod pen {
use anyhow::{bail, Result};
pub struct VirtualPen;
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
bail!("no pen injection backend on this platform")
}
pub fn apply_batch(&mut self, _transitions: &[punktfunk_core::quic::PenTransition]) {}
}
}
#[cfg(target_os = "linux")]
#[path = "inject/linux/kwin_fake_input.rs"]
mod kwin_fake_input;
+141 -34
View File
@@ -5,10 +5,19 @@
//! negotiated it (`HOST_CAP_CURSOR` in the Welcome — the host stopped compositing then) and
//! only applied while the DESKTOP mouse model is engaged: under capture the pointer is
//! relative-locked (SDL hides it) and games draw their own cursor in-frame.
//!
//! The host sends the bitmap in host-FRAMEBUFFER pixels, whose size tracks the host virtual
//! display's DPI scaling (32 px at 100%, 96 px at 300%). Drawn 1:1 it balloons on a high-DPI
//! host; instead we scale it by the SAME aspect-fit factor the video is drawn at
//! (`min(window_px/mode)`), so the pointer stays sized to the streamed desktop at any host
//! scaling. SDL cursors are fixed-size from their surface (no draw-time scaling), so we cache
//! shapes RAW and resample per install — rebuilding when the serial OR the fit changes.
use punktfunk_core::client::NativeClient;
use punktfunk_core::quic::{CursorState, HOST_CAP_CURSOR};
use sdl3::mouse::{Cursor, MouseUtil, SystemCursor};
use sdl3::pixels::PixelFormat;
use sdl3::surface::Surface;
use std::collections::HashMap;
use std::time::Duration;
@@ -17,14 +26,27 @@ use std::time::Duration;
/// on the reliable stream via the serial-miss path).
const SHAPE_CACHE_MAX: usize = 64;
/// A forwarded cursor shape held RAW (host-framebuffer-pixel bytes + hotspot), so it can be
/// rebuilt into a scaled OS cursor whenever the video-fit changes (a window resize). Caching a
/// fixed-size `Cursor` instead would freeze the pointer at its build-time size.
struct RawShape {
rgba: Vec<u8>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
}
pub struct CursorChannel {
/// The Welcome carried `HOST_CAP_CURSOR` — the host forwards instead of compositing.
negotiated: bool,
/// Serial → built OS cursor. An SDL `Cursor` must outlive its `set()`, so the cache owns
/// every shape ever applied this session (bounded by [`SHAPE_CACHE_MAX`]).
shapes: HashMap<u32, Cursor>,
/// The serial currently installed via `Cursor::set` (`None` = default/system cursor).
installed: Option<u32>,
/// Serial → raw forwarded shape (bounded by [`SHAPE_CACHE_MAX`]).
shapes: HashMap<u32, RawShape>,
/// The serial + fit scale the currently-installed OS cursor was built at (`None` =
/// default/system cursor). A change in EITHER forces a rebuild.
installed: Option<(u32, f32)>,
/// Keeps the installed `Cursor` alive — SDL requires it to outlive its `set()`.
installed_cursor: Option<Cursor>,
/// Latest `0xD0` state (latest-wins across a drained batch).
state: Option<CursorState>,
}
@@ -39,6 +61,7 @@ impl CursorChannel {
negotiated,
shapes: HashMap::new(),
installed: None,
installed_cursor: None,
state: None,
}
}
@@ -57,8 +80,16 @@ impl CursorChannel {
/// Drain the two planes and apply the newest state — once per run-loop iteration.
/// `desktop_active` = the desktop mouse model is engaged (captured + desktop): only then
/// do we own the local cursor's shape/visibility; under capture SDL's relative mode owns
/// it, and released the system cursor must look normal.
pub fn pump(&mut self, connector: &NativeClient, mouse: &MouseUtil, desktop_active: bool) {
/// it, and released the system cursor must look normal. `fit_scale` is host-framebuffer
/// pixels → window pixels (the aspect-fit factor the video is drawn at); the shape is
/// resampled by it so the pointer matches the streamed desktop at any host DPI.
pub fn pump(
&mut self,
connector: &NativeClient,
mouse: &MouseUtil,
desktop_active: bool,
fit_scale: f32,
) {
if !self.negotiated {
return;
}
@@ -68,30 +99,25 @@ impl CursorChannel {
self.shapes.clear();
self.installed = None;
}
let mut data = shape.rgba;
let built = sdl3::surface::Surface::from_data(
&mut data,
shape.w as u32,
shape.h as u32,
shape.w as u32 * 4,
sdl3::pixels::PixelFormat::RGBA32,
)
.map_err(|e| e.to_string())
.and_then(|surf| {
Cursor::from_surface(&surf, shape.hot_x as i32, shape.hot_y as i32)
.map_err(|e| e.to_string())
});
match built {
Ok(cursor) => {
let (w, h) = (shape.w as u32, shape.h as u32);
if w == 0 || h == 0 || shape.rgba.len() < (w * h * 4) as usize {
tracing::warn!(w, h, "cursor shape malformed — ignored");
continue;
}
// A re-sent serial replaces its entry; force re-install if it's current.
if self.installed == Some(shape.serial) {
if matches!(self.installed, Some((s, _)) if s == shape.serial) {
self.installed = None;
}
self.shapes.insert(shape.serial, cursor);
}
Err(e) => tracing::warn!(error = %e, w = shape.w, h = shape.h,
"cursor shape rejected by SDL — keeping the previous cursor"),
}
self.shapes.insert(
shape.serial,
RawShape {
rgba: shape.rgba,
w,
h,
hot_x: shape.hot_x as u32,
hot_y: shape.hot_y as u32,
},
);
}
while let Ok(st) = connector.next_cursor_state(Duration::ZERO) {
self.state = Some(st); // latest wins
@@ -101,17 +127,25 @@ impl CursorChannel {
// Capture mode / released: hand the cursor back to the system default so a
// released pointer over the window doesn't wear the host's shape.
if self.installed.take().is_some() {
Cursor::from_system(SystemCursor::Arrow)
.map(|c| c.set())
.ok();
if let Ok(c) = Cursor::from_system(SystemCursor::Arrow) {
c.set();
self.installed_cursor = Some(c); // keep it alive past set()
}
}
return;
}
let Some(st) = self.state else { return };
if st.visible() && self.installed != Some(st.serial) {
if let Some(cursor) = self.shapes.get(&st.serial) {
if st.visible() && self.installed != Some((st.serial, fit_scale)) {
if let Some(shape) = self.shapes.get(&st.serial) {
match build_scaled_cursor(shape, fit_scale) {
Ok(cursor) => {
cursor.set();
self.installed = Some(st.serial);
self.installed = Some((st.serial, fit_scale));
self.installed_cursor = Some(cursor); // outlive set()
}
Err(e) => tracing::warn!(error = %e, w = shape.w, h = shape.h,
"cursor shape rejected by SDL — keeping the previous cursor"),
}
}
// Serial miss: the (reliable) shape hasn't landed yet — keep the previous
// cursor for the RTT rather than flashing default.
@@ -123,3 +157,76 @@ impl CursorChannel {
}
}
}
/// Resample a raw shape by `fit_scale` and build an SDL color cursor from it. The hotspot scales
/// with the bitmap so the click point stays true. `fit_scale <= 0` (or a degenerate result) is
/// clamped so we always hand SDL a ≥1×1 surface.
fn build_scaled_cursor(shape: &RawShape, fit_scale: f32) -> Result<Cursor, String> {
let scale = if fit_scale.is_finite() && fit_scale > 0.0 {
fit_scale
} else {
1.0
};
let dw = ((shape.w as f32 * scale).round() as u32).max(1);
let dh = ((shape.h as f32 * scale).round() as u32).max(1);
let hot_x = ((shape.hot_x as f32 * scale).round() as u32).min(dw - 1) as i32;
let hot_y = ((shape.hot_y as f32 * scale).round() as u32).min(dh - 1) as i32;
if dw == shape.w && dh == shape.h {
// 1:1 fit (mode == window) — no resample, feed the bytes straight through.
let mut data = shape.rgba.clone();
let surf = Surface::from_data(
&mut data,
shape.w,
shape.h,
shape.w * 4,
PixelFormat::RGBA32,
)
.map_err(|e| e.to_string())?;
return Cursor::from_surface(&surf, hot_x, hot_y).map_err(|e| e.to_string());
}
let mut scaled = resample_rgba(&shape.rgba, shape.w, shape.h, dw, dh);
let surf = Surface::from_data(&mut scaled, dw, dh, dw * 4, PixelFormat::RGBA32)
.map_err(|e| e.to_string())?;
Cursor::from_surface(&surf, hot_x, hot_y).map_err(|e| e.to_string())
}
/// Area-average resample of a straight-alpha RGBA bitmap `(sw×sh) → (dw×dh)`. Averaging is done on
/// PREMULTIPLIED colour (weighting each source texel by its alpha) so transparent-pixel colour
/// can't bleed into the fringe, then un-premultiplied back to straight alpha. Handles both down-
/// and up-scale; for a cursor the common case is a downscale (high-DPI host → smaller pointer).
fn resample_rgba(src: &[u8], sw: u32, sh: u32, dw: u32, dh: u32) -> Vec<u8> {
let mut out = vec![0u8; (dw * dh * 4) as usize];
let fx = sw as f32 / dw as f32;
let fy = sh as f32 / dh as f32;
for dy in 0..dh {
let sy0 = (dy as f32 * fy).floor() as u32;
let sy1 = (((dy + 1) as f32 * fy).ceil() as u32).clamp(sy0 + 1, sh);
for dx in 0..dw {
let sx0 = (dx as f32 * fx).floor() as u32;
let sx1 = (((dx + 1) as f32 * fx).ceil() as u32).clamp(sx0 + 1, sw);
let (mut r, mut g, mut b, mut a_sum, mut n) = (0f32, 0f32, 0f32, 0f32, 0f32);
for sy in sy0..sy1 {
for sx in sx0..sx1 {
let i = ((sy * sw + sx) * 4) as usize;
let a = src[i + 3] as f32 / 255.0;
r += src[i] as f32 * a;
g += src[i + 1] as f32 * a;
b += src[i + 2] as f32 * a;
a_sum += a;
n += 1.0;
}
}
let di = ((dy * dw + dx) * 4) as usize;
if a_sum > 0.0 {
out[di] = (r / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 1] = (g / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 2] = (b / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 3] = (a_sum / n * 255.0).round().clamp(0.0, 255.0) as u8;
}
// else fully transparent — already zero-filled.
}
}
out
}
+9 -1
View File
@@ -773,12 +773,20 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// Cursor channel (M2): drain forwarded shape/state and drive the local OS cursor —
// only meaningful in the desktop mouse model (capture's relative lock hides it).
if let Some(st) = stream.as_mut() {
// Host-framebuffer px → window px: the aspect-fit factor the video is drawn at
// (same `min(surface/content)` as `finger_to_content`). The forwarded pointer is
// resampled by it so a high-DPI host's oversized bitmap lands sized to the streamed
// desktop rather than ballooning. 1:1 until the first frame gives `last_video`.
let fit_scale = st.last_video.map_or(1.0, |(vw, vh)| {
let (pw, ph) = window.size_in_pixels();
(pw as f32 / vw.max(1) as f32).min(ph as f32 / vh.max(1) as f32)
});
if let (Some(chan), Some(c)) = (st.cursor_chan.as_mut(), st.connector.as_ref()) {
let desktop_active = st
.capture
.as_ref()
.is_some_and(|cap| cap.captured() && cap.desktop());
chan.pump(c, &mouse, desktop_active);
chan.pump(c, &mouse, desktop_active, fit_scale);
// §8 mid-stream render flip: tell the host who renders the pointer whenever
// the local model changes. Desktop-active = we draw it (host excludes +
// forwards); anything else — the capture model OR a released pointer — the
@@ -300,6 +300,21 @@ pub fn control_device_handle() -> Option<HANDLE> {
VDM.get().and_then(VirtualDisplayManager::device_handle)
}
/// Re-commit the CURRENT display config under the manager `state` lock (the sole-topology-mutator
/// contract of [`force_mode_reenumeration`]). The secure-desktop guard's actuator: the OS only
/// reverts a path to its software-cursor default ON a mode commit, so standing the hardware-cursor
/// declare down (`IOCTL_SET_CURSOR_FORWARD` off) needs this nudge for UAC/Winlogon to actually
/// render. `false` (no-op) before the first backend open — no monitors exist to re-commit for.
pub fn force_recommit() -> bool {
let Some(m) = VDM.get() else {
return false;
};
let _guard = m.state.lock().unwrap();
// SAFETY: `force_mode_reenumeration`'s contract is "call under the manager `state` lock";
// held above. The call reads + re-applies the current CCD config over owned locals.
unsafe { pf_win_display::win_display::force_mode_reenumeration() }
}
/// Best-effort "is this WUDFHost pid still alive?" — the monitor-liveness probe for the JOIN path.
/// `OpenProcess` failing (pid reaped) or the process being signaled ⇒ dead. Pid reuse could
/// theoretically alias a fresh process and read "alive"; the joining session then just retries into
+79 -1
View File
@@ -928,6 +928,26 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
// never calls ASSIGN_SWAPCHAIN, so the driver receives no frames. SDC_FORCE_MODE_ENUMERATION
// forces the re-commit; SAVE_TO_DATABASE only in the sole-path case (matches prior behavior —
// don't permanently rewrite the user's multi-display layout; the teardown restore handles it).
let rc = if others > 0 && attempt >= 2 {
// ESCALATION (attempt 2+): supply ONLY the keep paths. Field-reported (AMD +
// pf-vdisplay): carrying the doomed path in the array — inactive, modes unpinned —
// gets the whole config rejected 0x57 on EVERY retry, so the loop alone never
// converged; the same host applies the keep-only shape rc=0 whenever the topology
// database has already made the virtual display sole. The final attempt also drops
// SDC_FORCE_MODE_ENUMERATION in case the driver rejects it combined with a real
// topology change — an actual path removal drives COMMIT_MODES on its own, so the
// re-commit rationale doesn't need the flag here.
let (kp, km) = keep_only_supplied(&paths, &modes);
let mut esc = SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES;
if attempt < 4 {
esc |= SDC_FORCE_MODE_ENUMERATION;
}
tracing::info!(
"display isolate (CCD): escalating to a keep-only supplied config (attempt {attempt}/4, paths {}→{}, modes {}→{})",
paths.len(), kp.len(), modes.len(), km.len()
);
SetDisplayConfig(Some(kp.as_slice()), Some(km.as_slice()), esc)
} else {
let mut flags = SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
@@ -935,7 +955,8 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
if others == 0 {
flags |= SDC_SAVE_TO_DATABASE;
}
let rc = SetDisplayConfig(Some(paths.as_slice()), Some(modes.as_slice()), flags);
SetDisplayConfig(Some(paths.as_slice()), Some(modes.as_slice()), flags)
};
// A failed apply must be VISIBLE even when the verification below passes vacuously (nothing
// else was active to deactivate — the lid-closed laptop case, where the success INFO used to
// swallow rc=0x5): the re-commit above is load-bearing (COMMIT_MODES → ASSIGN_SWAPCHAIN),
@@ -962,6 +983,63 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
Some(saved)
}
/// Build the ESCALATED supplied config for [`isolate_displays_ccd`]: ONLY the paths still flagged
/// ACTIVE (the keep set — the caller already cleared ACTIVE on every doomed path), with the mode
/// table rebuilt to just the entries those paths reference (indexes remapped). Docs-wise the
/// dropped inactive entries were declared ignored anyway ("Only the paths within this array that
/// have the DISPLAYCONFIG_PATH_ACTIVE flag set are set"), so this shape asks for the identical
/// topology — minus the array contents some driver/OS validation combos reject with 0x57.
unsafe fn keep_only_supplied(
paths: &[DISPLAYCONFIG_PATH_INFO],
modes: &[DISPLAYCONFIG_MODE_INFO],
) -> (Vec<DISPLAYCONFIG_PATH_INFO>, Vec<DISPLAYCONFIG_MODE_INFO>) {
let mut out_paths = Vec::new();
let mut out_modes = Vec::new();
// old mode index → new. Shared entries dedup through here: a clone-style pair references ONE
// source mode, and the docs require each source/target mode to appear in the table only once.
let mut remap = std::collections::HashMap::new();
for p in paths {
if p.flags & DISPLAYCONFIG_PATH_ACTIVE == 0 {
continue;
}
let mut q = *p;
q.sourceInfo.Anonymous.modeInfoIdx = remap_mode_idx(
q.sourceInfo.Anonymous.modeInfoIdx,
modes,
&mut out_modes,
&mut remap,
);
q.targetInfo.Anonymous.modeInfoIdx = remap_mode_idx(
q.targetInfo.Anonymous.modeInfoIdx,
modes,
&mut out_modes,
&mut remap,
);
out_paths.push(q);
}
(out_paths, out_modes)
}
/// Move `modes[old]` into `out` (once — `remap` dedups) and return its new index. INVALID and
/// out-of-range indexes stay INVALID — `SDC_ALLOW_CHANGES` lets best-mode logic fill the gap.
fn remap_mode_idx(
old: u32,
modes: &[DISPLAYCONFIG_MODE_INFO],
out: &mut Vec<DISPLAYCONFIG_MODE_INFO>,
remap: &mut std::collections::HashMap<u32, u32>,
) -> u32 {
if old == DISPLAYCONFIG_PATH_MODE_IDX_INVALID {
return old;
}
let Some(m) = modes.get(old as usize) else {
return DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
};
*remap.entry(old).or_insert_with(|| {
out.push(*m);
(out.len() - 1) as u32
})
}
/// The desktop-space rectangle `(x, y, w, h)` of `target_id`'s SOURCE — where this display's
/// region lives in the desktop coordinate space. `None` while the target isn't an active path.
/// Used by the IDD-push compose kick to dirty THE TARGET display: with parallel displays the
+152 -3
View File
@@ -870,6 +870,94 @@ impl PunktfunkRichInputEx {
}
}
/// [`PunktfunkPenSample::state`] bit: the pen hovers in range (implied by `TOUCHING`).
pub const PUNKTFUNK_PEN_IN_RANGE: u8 = 0x01;
/// [`PunktfunkPenSample::state`] bit: the tip is in contact.
pub const PUNKTFUNK_PEN_TOUCHING: u8 = 0x02;
/// [`PunktfunkPenSample::state`] bit: primary barrel button (or squeeze mapping) held.
pub const PUNKTFUNK_PEN_BARREL1: u8 = 0x04;
/// [`PunktfunkPenSample::state`] bit: secondary barrel button (or double-tap mapping) held.
pub const PUNKTFUNK_PEN_BARREL2: u8 = 0x08;
/// [`PunktfunkPenSample::tool`]: the pen tip.
pub const PUNKTFUNK_PEN_TOOL_PEN: u8 = 0;
/// [`PunktfunkPenSample::tool`]: the eraser (a client-side mode — Apple Pencil has no
/// hardware eraser end; the squeeze/double-tap mapping usually drives this).
pub const PUNKTFUNK_PEN_TOOL_ERASER: u8 = 1;
/// Most samples one [`punktfunk_connection_send_pen`] call accepts (one wire batch).
pub const PUNKTFUNK_PEN_BATCH_MAX: u32 = 8;
/// [`PunktfunkPenSample::tilt_deg`] sentinel: no tilt reading.
pub const PUNKTFUNK_PEN_TILT_UNKNOWN: u8 = 0xFF;
/// [`PunktfunkPenSample::azimuth_deg`] / `roll_deg` sentinel: no reading.
pub const PUNKTFUNK_PEN_ANGLE_UNKNOWN: u16 = 0xFFFF;
/// [`PunktfunkPenSample::distance`] sentinel: no hover-distance reading.
pub const PUNKTFUNK_PEN_DISTANCE_UNKNOWN: u16 = 0xFFFF;
/// One complete stylus state at one instant ([`punktfunk_connection_send_pen`];
/// design/pen-tablet-input.md). STATE-FULL, never an edge event: fill every field on every
/// sample (unknown axes take their `*_UNKNOWN` sentinel) — the host diffs consecutive samples
/// and synthesizes down/up/button transitions itself, which is what makes a lost datagram
/// self-heal. `x`/`y` are normalized `0.0..=1.0` in VIDEO-FRAME space (map your letterbox
/// before filling, exactly like wire touches).
#[cfg(feature = "quic")]
#[repr(C)]
#[derive(Clone, Copy)]
pub struct PunktfunkPenSample {
/// Normalized `0.0..=1.0` across the video frame. Must be finite.
pub x: f32,
/// Normalized `0.0..=1.0` across the video frame. Must be finite.
pub y: f32,
/// Tip force, `0..=65535` full scale (`0` while hovering).
pub pressure: u16,
/// Hover distance `0..=65534` (0 = at the hover floor), or `PUNKTFUNK_PEN_DISTANCE_UNKNOWN`.
pub distance: u16,
/// Tilt azimuth, degrees `0..=359` clockwise from north, or `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
pub azimuth_deg: u16,
/// Barrel roll (Apple Pencil Pro `rollAngle`), degrees `0..=359`, or
/// `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
pub roll_deg: u16,
/// µs since the previous sample in the same call (`0` for the first) — the coalesced
/// capture spacing.
pub dt_us: u16,
/// Bitfield of `PUNKTFUNK_PEN_*` state bits. Unknown bits are rejected (`InvalidArg`).
pub state: u8,
/// `PUNKTFUNK_PEN_TOOL_PEN` or `PUNKTFUNK_PEN_TOOL_ERASER`.
pub tool: u8,
/// Tilt from the surface normal, degrees `0..=90`, or `PUNKTFUNK_PEN_TILT_UNKNOWN`.
pub tilt_deg: u8,
/// Set to 0.
pub _reserved: [u8; 3],
}
#[cfg(feature = "quic")]
impl PunktfunkPenSample {
/// `None` = invalid field (non-finite coordinate, unknown state bit, unknown tool) —
/// embedder input is validated strictly, unlike the loss-tolerant wire decode.
fn to_sample(self) -> Option<crate::quic::PenSample> {
use crate::quic as q;
let known = q::PEN_IN_RANGE | q::PEN_TOUCHING | q::PEN_BARREL1 | q::PEN_BARREL2;
if !self.x.is_finite() || !self.y.is_finite() || self.state & !known != 0 {
return None;
}
let tool = match self.tool {
PUNKTFUNK_PEN_TOOL_PEN => q::PenTool::Pen,
PUNKTFUNK_PEN_TOOL_ERASER => q::PenTool::Eraser,
_ => return None,
};
Some(q::PenSample {
state: self.state,
tool,
x: self.x,
y: self.y,
pressure: self.pressure,
distance: self.distance,
tilt_deg: self.tilt_deg,
azimuth_deg: self.azimuth_deg,
roll_deg: self.roll_deg,
dt_us: self.dt_us,
})
}
}
/// Read an optional NUL-terminated UTF-8 string parameter; `Err` = invalid pointer/UTF-8.
#[cfg(feature = "quic")]
unsafe fn opt_cstr<'a>(p: *const std::os::raw::c_char) -> std::result::Result<Option<&'a str>, ()> {
@@ -988,6 +1076,12 @@ pub const PUNKTFUNK_HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// Host-capability bit in [`punktfunk_connection_host_caps`]: the host supports the shared
/// clipboard, so a client may offer the toggle. (Mirrors `quic::HOST_CAP_CLIPBOARD`.)
pub const PUNKTFUNK_HOST_CAP_CLIPBOARD: u8 = 0x02;
/// Host-capability bit in [`punktfunk_connection_host_caps`]: the host injects full-fidelity
/// stylus input, so a capable client splits pen contacts out of its touch path and sends them
/// via [`punktfunk_connection_send_pen`]; without the bit that call returns `Unsupported` and
/// the client keeps its pen-as-touch fallback. (Mirrors `quic::HOST_CAP_PEN`;
/// design/pen-tablet-input.md.)
pub const PUNKTFUNK_HOST_CAP_PEN: u8 = 0x10;
// Keep the ABI cap bits in lockstep with the wire constants (compile-time guard against drift).
#[cfg(feature = "quic")]
@@ -1001,6 +1095,15 @@ const _: () = {
assert!(PUNKTFUNK_CODEC_PYROWAVE == crate::quic::CODEC_PYROWAVE);
assert!(PUNKTFUNK_HOST_CAP_GAMEPAD_STATE == crate::quic::HOST_CAP_GAMEPAD_STATE);
assert!(PUNKTFUNK_HOST_CAP_CLIPBOARD == crate::quic::HOST_CAP_CLIPBOARD);
assert!(PUNKTFUNK_HOST_CAP_PEN == crate::quic::HOST_CAP_PEN);
assert!(PUNKTFUNK_PEN_IN_RANGE == crate::quic::PEN_IN_RANGE);
assert!(PUNKTFUNK_PEN_TOUCHING == crate::quic::PEN_TOUCHING);
assert!(PUNKTFUNK_PEN_BARREL1 == crate::quic::PEN_BARREL1);
assert!(PUNKTFUNK_PEN_BARREL2 == crate::quic::PEN_BARREL2);
assert!(PUNKTFUNK_PEN_BATCH_MAX as usize == crate::quic::PEN_BATCH_MAX);
assert!(PUNKTFUNK_PEN_TILT_UNKNOWN == crate::quic::PEN_TILT_UNKNOWN);
assert!(PUNKTFUNK_PEN_ANGLE_UNKNOWN == crate::quic::PEN_ANGLE_UNKNOWN);
assert!(PUNKTFUNK_PEN_DISTANCE_UNKNOWN == crate::quic::PEN_DISTANCE_UNKNOWN);
};
// Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift).
@@ -2763,6 +2866,51 @@ pub unsafe extern "C" fn punktfunk_connection_send_rich_input2(
})
}
/// Send one stylus sample batch — `count` (`1..=PUNKTFUNK_PEN_BATCH_MAX`) state-full
/// [`PunktfunkPenSample`]s, oldest first (a capture callback's coalesced samples) — as one
/// `0xCC/0x05` pen datagram (non-blocking enqueue; design/pen-tablet-input.md). Split longer
/// runs into consecutive calls. Gate on `punktfunk_connection_host_caps() &
/// PUNKTFUNK_HOST_CAP_PEN`: toward a host without the bit this returns
/// [`PunktfunkStatus::Unsupported`] — keep the pen-as-touch fallback there.
/// [`PunktfunkStatus::InvalidArg`] on a bad count or a bad sample (non-finite coordinate,
/// unknown state bit / tool).
///
/// # Safety
/// `c` is a valid connection handle; `samples` is null or points to `count` valid
/// [`PunktfunkPenSample`]s.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_send_pen(
c: *mut PunktfunkConnection,
samples: *const PunktfunkPenSample,
count: u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if samples.is_null() {
return PunktfunkStatus::NullPointer;
}
if count == 0 || count > PUNKTFUNK_PEN_BATCH_MAX {
return PunktfunkStatus::InvalidArg;
}
let raw = unsafe { std::slice::from_raw_parts(samples, count as usize) };
let mut batch = [crate::quic::PenSample::default(); crate::quic::PEN_BATCH_MAX];
for (slot, s) in batch.iter_mut().zip(raw) {
match s.to_sample() {
Some(v) => *slot = v,
None => return PunktfunkStatus::InvalidArg,
}
}
match c.inner.send_pen(&batch[..count as usize]) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
}
})
}
/// The currently active session mode — the Welcome's, until an accepted
/// [`punktfunk_connection_request_mode`] switches it. Safe any time after connect.
///
@@ -2977,9 +3125,10 @@ fn build_clip_event(
}
/// The host capability bitfield the session's `Welcome` carried — a bitfield of
/// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD`. A client tests
/// `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide whether to offer the shared-clipboard toggle.
/// Safe any time after connect.
/// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD` /
/// `PUNKTFUNK_HOST_CAP_PEN`. A client tests `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide
/// whether to offer the shared-clipboard toggle, `caps & PUNKTFUNK_HOST_CAP_PEN` before
/// sending stylus batches. Safe any time after connect.
///
/// # Safety
/// `c` is a valid connection handle; `caps` is writable (NULL is skipped).
+44 -5
View File
@@ -20,7 +20,7 @@ use crate::quic::{
RfiRequest, RichInput,
};
use crate::session::Frame;
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU32, AtomicU64, Ordering};
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU16, AtomicU32, AtomicU64, Ordering};
use std::sync::mpsc::{Receiver, RecvTimeoutError};
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
@@ -104,8 +104,11 @@ pub struct NativeClient {
/// Bounded ([`MIC_QUEUE`]): a wedged worker drops fresh frames (logged) instead of queueing
/// audio-latency (and memory) without limit — mic is best-effort end to end.
mic_tx: tokio::sync::mpsc::Sender<(u32, u64, Vec<u8>)>,
/// Outbound rich input (DualSense touchpad / motion) → 0xCC datagrams by the worker.
rich_input_tx: tokio::sync::mpsc::UnboundedSender<RichInput>,
/// Outbound 0xCC rich-input plane, PRE-ENCODED datagrams: [`RichInput`] touchpad/motion
/// (encoded in [`NativeClient::send_rich_input`]) and stylus [`crate::quic::PenBatch`]es
/// (encoded in [`NativeClient::send_pen`]) share the channel — the worker's task just
/// forwards bytes, so a new 0xCC kind never touches the pump.
rich_input_tx: tokio::sync::mpsc::UnboundedSender<Vec<u8>>,
/// Outbound control-stream requests (mode switch, speed test) → the worker's control task.
/// Bounded ([`CTRL_QUEUE`]) — the requests are sparse; a full queue means the control task
/// is wedged/dead, and callers treat it like a closed session.
@@ -121,6 +124,9 @@ pub struct NativeClient {
/// Monotonic id for outbound fetches ([`NativeClient::clip_fetch`]); stays below
/// [`crate::clipboard::INBOUND_REQ_FLAG`] so it never collides with an inbound serve `req_id`.
next_xfer_id: AtomicU32,
/// Wrapping per-connection [`crate::quic::PenBatch::seq`] counter, stamped by
/// [`NativeClient::send_pen`] (the host's reorder gate compares it).
pen_seq: AtomicU16,
/// The host capability bitfield ([`crate::quic::Welcome::host_caps`]) — see
/// [`NativeClient::host_caps`].
pub host_caps: u8,
@@ -344,7 +350,7 @@ impl NativeClient {
std::sync::mpsc::sync_channel::<crate::quic::HostTiming>(HOST_TIMING_QUEUE);
let (input_tx, input_rx) = tokio::sync::mpsc::unbounded_channel::<InputEvent>();
let (mic_tx, mic_rx) = tokio::sync::mpsc::channel::<(u32, u64, Vec<u8>)>(MIC_QUEUE);
let (rich_input_tx, rich_input_rx) = tokio::sync::mpsc::unbounded_channel::<RichInput>();
let (rich_input_tx, rich_input_rx) = tokio::sync::mpsc::unbounded_channel::<Vec<u8>>();
let (ctrl_tx, ctrl_rx) = tokio::sync::mpsc::channel::<CtrlRequest>(CTRL_QUEUE);
let (clip_event_tx, clip_event_rx) =
std::sync::mpsc::sync_channel::<ClipEventCore>(CLIP_EVENT_QUEUE);
@@ -473,6 +479,7 @@ impl NativeClient {
clip: Mutex::new(clip_event_rx),
clip_cmd_tx,
next_xfer_id: AtomicU32::new(1),
pen_seq: AtomicU16::new(0),
host_caps: negotiated.host_caps,
probe,
shutdown,
@@ -1073,7 +1080,39 @@ impl NativeClient {
/// loss like every datagram. No-op unless the host runs the DualSense gamepad backend.
pub fn send_rich_input(&self, rich: RichInput) -> Result<()> {
self.rich_input_tx
.send(rich)
.send(rich.encode())
.map_err(|_| PunktfunkError::Closed)
}
/// Queue one stylus sample batch for delivery as a `0xCC/0x05` pen datagram
/// (design/pen-tablet-input.md). `samples` are state-full and oldest-first (a capture
/// callback's coalesced samples), at most [`crate::quic::PEN_BATCH_MAX`] per call — split
/// longer runs into consecutive calls so the stamped wrapping `seq` keeps them ordered.
/// Best-effort like every datagram: a lost batch self-heals on the next one (the samples
/// carry full state, the host diffs — see [`crate::quic::PenTracker`]).
///
/// **Heartbeat contract**: while the pen is in range or touching, repeat the last sample
/// at least every ~100 ms even when nothing changed (capture APIs are silent for a
/// stationary pen) — the host force-releases the stroke after
/// [`crate::quic::PEN_TOUCH_TIMEOUT_MS`] of silence as its dead-client failsafe.
///
/// Requires the host to have advertised [`crate::quic::HOST_CAP_PEN`]; toward an older
/// host this returns `Unsupported` (embedders keep their pen-as-touch fallback instead of
/// spraying 240 Hz datagrams the host drops unread).
pub fn send_pen(&self, samples: &[crate::quic::PenSample]) -> Result<()> {
if self.host_caps & crate::quic::HOST_CAP_PEN == 0 {
return Err(PunktfunkError::Unsupported(
"host did not advertise HOST_CAP_PEN",
));
}
if samples.is_empty() || samples.len() > crate::quic::PEN_BATCH_MAX {
return Err(PunktfunkError::InvalidArg(
"pen batch must hold 1..=PEN_BATCH_MAX samples",
));
}
let seq = self.pen_seq.fetch_add(1, Ordering::Relaxed);
self.rich_input_tx
.send(crate::quic::PenBatch::new(seq, samples).encode())
.map_err(|_| PunktfunkError::Closed)
}
+4 -3
View File
@@ -99,11 +99,12 @@ pub(super) async fn run_pump(args: WorkerArgs) {
}
});
// Rich-input task: embedder DualSense touchpad / motion → 0xCC uplink datagrams.
// Rich-input task: pre-encoded 0xCC uplink datagrams (DualSense touchpad / motion, pen
// batches — encoded at the NativeClient surface so new plane kinds never touch the pump).
let rich_conn = conn.clone();
tokio::spawn(async move {
while let Some(rich) = rich_input_rx.recv().await {
let _ = rich_conn.send_datagram(rich.encode().into());
while let Some(d) = rich_input_rx.recv().await {
let _ = rich_conn.send_datagram(d.into());
}
});
+3 -2
View File
@@ -5,7 +5,7 @@ use crate::clipboard::{ClipCommand, ClipEventCore};
use crate::config::{CompositorPref, GamepadPref, Mode};
use crate::error::Result;
use crate::input::InputEvent;
use crate::quic::{HdrMeta, HidOutput, RichInput};
use crate::quic::{HdrMeta, HidOutput};
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU64};
use std::sync::mpsc::SyncSender;
use std::sync::{Arc, Mutex};
@@ -43,7 +43,8 @@ pub(crate) struct WorkerArgs {
pub(crate) cursor_state_tx: SyncSender<crate::quic::CursorState>,
pub(crate) input_rx: tokio::sync::mpsc::UnboundedReceiver<InputEvent>,
pub(crate) mic_rx: tokio::sync::mpsc::Receiver<(u32, u64, Vec<u8>)>,
pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<RichInput>,
/// Pre-encoded 0xCC datagrams (rich input AND pen batches — see `NativeClient.rich_input_tx`).
pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<Vec<u8>>,
pub(crate) ctrl_rx: tokio::sync::mpsc::Receiver<CtrlRequest>,
pub(crate) ctrl_tx: tokio::sync::mpsc::Sender<CtrlRequest>,
/// Inbound clipboard event plane feed — the control task pushes ClipState/ClipOffer, the
+4 -1
View File
@@ -106,7 +106,10 @@ pub use stats::Stats;
/// (design/remote-desktop-sweep.md §8): the client's mouse-model chord tells the host who
/// renders the pointer. Additive; rides the existing control stream (a new message TYPE, which
/// pre-§8 hosts ignore), so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 12;
/// v13: added `punktfunk_connection_send_pen` — the stylus wire plane
/// (design/pen-tablet-input.md): a client sends `RICH_PEN` sample batches once the host
/// advertises `HOST_CAP_PEN`. Additive and capability-gated, so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 13;
/// The punktfunk/1 **wire** version — what `Hello`/`Welcome` carry and hosts equality-check.
/// Deliberately its own constant: [`ABI_VERSION`] tracks the embeddable **C surface**
+12
View File
@@ -100,6 +100,18 @@ pub const CLIENT_CAP_CURSOR: u8 = 0x01;
/// [`HOST_CAP_TEXT_INPUT`], `0x01`/`0x02` are gamepad-state / clipboard.
pub const HOST_CAP_CURSOR: u8 = 0x08;
/// [`Welcome::host_caps`] bit: the host injects full-fidelity stylus input — it routes
/// [`PenBatch`](super::pen::PenBatch) `0xCC/0x05` datagrams (pressure, tilt, azimuth, barrel
/// roll, hover, eraser, barrel buttons) through the [`PenTracker`](super::pen::PenTracker)
/// into a virtual tablet device (design/pen-tablet-input.md). A capable client (Apple Pencil,
/// Android stylus) then splits pen contacts out of its finger/touch path and sends pen
/// batches; absent the bit it keeps folding the pen into touch/pointer like today, and
/// [`NativeClient::send_pen`](crate::client::NativeClient::send_pen) refuses to send. The
/// wire ships ahead of the backend (P0): no host sets this bit until the P1 injector lands —
/// which is exactly why the gate exists. `0x10` — `0x08` is [`HOST_CAP_CURSOR`], `0x04` is
/// [`HOST_CAP_TEXT_INPUT`], `0x01`/`0x02` are gamepad-state / clipboard.
pub const HOST_CAP_PEN: u8 = 0x10;
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
/// advertise this.
+8 -1
View File
@@ -162,6 +162,12 @@ pub(super) const RICH_TOUCHPAD: u8 = 0x01;
pub(super) const RICH_MOTION: u8 = 0x02;
pub(super) const RICH_TOUCHPAD_EX: u8 = 0x03;
pub(super) const RICH_HID_REPORT: u8 = 0x04;
/// Claimed by the stylus plane ([`PenBatch`](super::pen::PenBatch)), which is NOT a
/// [`RichInput`] variant: rich input is pad-indexed controller state consumed by the gamepad
/// backends, while a pen batch routes to the (P1) tablet injector — so it gets its own decoder
/// and [`RichInput::decode`] keeps returning `None` here (= the documented unknown-kind drop
/// on a pre-pen host). Registered in this list so 0xCC kind bytes stay unique.
pub(super) const RICH_PEN: u8 = 0x05;
/// Longest raw HID report a [`RichInput::HidReport`] / [`HidOutput::HidRaw`] can carry — the
/// 64-byte interrupt/feature report size every Valve controller uses (Triton input reports are
@@ -171,7 +177,8 @@ pub const HID_REPORT_MAX: usize = 64;
/// A rich client→host controller input beyond the fixed [`InputEvent`](crate::input::InputEvent):
/// the DualSense touchpad and motion sensors. `pad` is the gamepad index. Wire form is
/// `[0xCC][kind][fields…]` — variable-length and kind-tagged (forward-compatible: an unknown
/// kind decodes to `None` and is dropped).
/// kind decodes to `None` and is dropped). Kind `0x05` on this plane is the stylus batch
/// ([`PenBatch`](super::pen::PenBatch)) with its own decoder — see [`RICH_PEN`].
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum RichInput {
/// One touchpad contact. `x`/`y` are normalized `0..=65535` in SCREEN convention —
+3
View File
@@ -26,6 +26,7 @@
//! positional Hello/Welcome/Start codecs, `caps` the capability/codec-negotiation
//! vocabulary, `control` the typed control + clipboard messages, `pairing` the pairing
//! message codecs with [`pake`] the SPAKE2 itself, `datagram` the 0xC90xCF plane codecs,
//! `pen` the stylus batch (0xCC kind 0x05) + host stroke tracker,
//! [`io`] framed stream IO, `clock` skew estimation + mid-stream re-sync, [`endpoint`] the
//! quinn constructors, [`clipstream`] the per-transfer clipboard fetch streams. Every item
//! is re-exported here, so all existing `crate::quic::X` paths compile unchanged; each
@@ -46,6 +47,7 @@ mod control;
mod datagram;
mod handshake;
mod pairing;
mod pen;
/// quinn endpoint constructors. Host: self-signed identity (fresh, or persisted PEMs via
/// [`endpoint::server_with_identity`]). Client: fingerprint pinning / TOFU via
@@ -72,6 +74,7 @@ pub use control::*;
pub use datagram::*;
pub use handshake::*;
pub use pairing::*;
pub use pen::*;
// Typed rejection close codes + [`RejectReason`] live in `crate::reject` (ungated — the
// error enum references them even in `quic`-less builds) and are re-exported here so the
+691
View File
@@ -0,0 +1,691 @@
//! The stylus plane: full-fidelity pen input on the 0xCC rich-input datagram
//! (design/pen-tablet-input.md).
//!
//! A pen datagram is a batch of **state-full samples**: every sample carries the *complete*
//! pen state — in-range/touching/buttons/tool plus all axes — never an edge ("down"/"up")
//! event. The host diffs each sample against its own tracked state ([`PenTracker`]) and
//! synthesizes the transitions, so a lost datagram self-heals on the next sample: a dropped
//! "first contact" batch becomes a tip-down when the next in-contact sample arrives, and a
//! dropped lift heals on the next hover/out-of-range sample (with the
//! [`PEN_TOUCH_TIMEOUT_MS`] failsafe for a client that dies mid-stroke). That is what makes
//! the lossy datagram plane sound for a 240 Hz stylus without any reliable-delivery
//! machinery — the same idempotent-snapshot argument as [`GamepadSnapshot`]
//! (crate::input::GamepadSnapshot) and [`RichInput::HidReport`](super::RichInput).
//!
//! Batches are ordered by a wrapping `u16` sequence number and dropped **whole** when stale
//! ([`pen_seq_newer`]) — applying a stale state-full sample would rewind the stroke.
//!
//! Clients send this only after the host advertised [`HOST_CAP_PEN`](super::HOST_CAP_PEN);
//! a pre-pen host drops the unknown 0xCC kind by the plane's documented forward-compat rule.
use super::datagram::{RICH_INPUT_MAGIC, RICH_PEN};
/// [`PenSample::state`] bit: the pen is in the hover range of the surface. Implied by
/// [`PEN_TOUCHING`] (decode normalizes, so a client that only sets TOUCHING still produces a
/// coherent contact).
pub const PEN_IN_RANGE: u8 = 0x01;
/// [`PenSample::state`] bit: the tip is in contact with the surface.
pub const PEN_TOUCHING: u8 = 0x02;
/// [`PenSample::state`] bit: the primary barrel button (or the client's squeeze mapping) is held.
pub const PEN_BARREL1: u8 = 0x04;
/// [`PenSample::state`] bit: the secondary barrel button (or the client's double-tap mapping)
/// is held.
pub const PEN_BARREL2: u8 = 0x08;
/// [`PenSample::state`] bit, RESERVED: a predicted (not yet observed) sample. Never sent v1;
/// receivers MUST ignore samples carrying it until a capability negotiates otherwise
/// (design/pen-tablet-input.md §8).
pub const PEN_PREDICTED: u8 = 0x80;
/// The button subset of [`PenSample::state`].
const PEN_BUTTONS_MASK: u8 = PEN_BARREL1 | PEN_BARREL2;
/// [`PenSample::tilt_deg`] sentinel: the client has no tilt sensor / no reading.
pub const PEN_TILT_UNKNOWN: u8 = 0xFF;
/// [`PenSample::azimuth_deg`] / [`PenSample::roll_deg`] sentinel: no reading.
pub const PEN_ANGLE_UNKNOWN: u16 = 0xFFFF;
/// [`PenSample::distance`] sentinel: no hover-distance reading.
pub const PEN_DISTANCE_UNKNOWN: u16 = 0xFFFF;
/// Most samples one [`PenBatch`] can carry. Sized for coalesced capture at video-frame cadence
/// (240 Hz pen ÷ 30 fps = 8); a client producing more splits into consecutive batches.
pub const PEN_BATCH_MAX: usize = 8;
/// Wire length of one encoded [`PenSample`].
pub const PEN_SAMPLE_WIRE_LEN: usize = 21;
/// `[0xCC][0x05][flags][count][u16 seq LE]` — bytes before the first sample.
const PEN_HEADER_LEN: usize = 6;
/// Host-side failsafe (design/pen-tablet-input.md §2): a tracker still in range after this
/// many ms without a sample force-releases ([`PenTracker::force_release`]) — a client that
/// died mid-stroke must not leave the host's virtual pen inked-down forever. This makes the
/// **client heartbeat a wire contract**: capture APIs only fire on change, so a stationary
/// pen is naturally silent — senders MUST repeat the last sample at least every ~100 ms while
/// the pen is in range or touching (it re-decodes as pure Motion, harmless), keeping a live
/// stationary stroke two heartbeats clear of the deadline.
pub const PEN_TOUCH_TIMEOUT_MS: u32 = 200;
/// Which end of the stylus (or which mapped mode) a sample describes. A tool *switch* while in
/// range is a physical re-entry — [`PenTracker`] emits a full release + re-proximity, matching
/// how a real tablet treats each tool as its own proximity session.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
#[repr(u8)]
pub enum PenTool {
#[default]
Pen = 0,
Eraser = 1,
/// An unrecognized wire value (a future tool from a newer client) — injectors treat it as
/// [`PenTool::Pen`]. Inside a proximity session it inherits the session's tool instead of
/// forcing a spurious re-entry.
Unknown = 0xFF,
}
impl PenTool {
fn from_u8(v: u8) -> PenTool {
match v {
0 => PenTool::Pen,
1 => PenTool::Eraser,
_ => PenTool::Unknown,
}
}
}
/// One complete stylus state at one instant. All axes ride every sample (state-full — see the
/// module doc); unknown axes carry their sentinel, never 0.
///
/// `x`/`y` are normalized `0.0..=1.0` in **video-frame space** — the client maps its letterbox
/// / viewport before sending (exactly as its wire touches already do), so the host scales
/// straight to the streamed output. f32 keeps sub-pixel precision at any resolution.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct PenSample {
/// Bitfield of `PEN_*` state bits ([`PEN_IN_RANGE`] … [`PEN_PREDICTED`]).
pub state: u8,
/// Active tool. Meaningful while in range; [`PenTool::Unknown`] inherits (see [`PenTool`]).
pub tool: PenTool,
/// Normalized `0.0..=1.0` across the video frame (decode clamps; see [`PenBatch::decode`]).
pub x: f32,
/// Normalized `0.0..=1.0` across the video frame.
pub y: f32,
/// Tip force, `0..=65535` full scale, `0` while hovering. Injectors rescale (Windows pens
/// are 0..1024, uinput declares its own range) — full u16 keeps every source's precision.
pub pressure: u16,
/// Hover distance, `0..=65534` normalized (0 = touching the hover floor), or
/// [`PEN_DISTANCE_UNKNOWN`].
pub distance: u16,
/// Tilt from the surface normal, degrees `0..=90`, or [`PEN_TILT_UNKNOWN`]. Polar form —
/// what Apple capture and the GameStream wire both produce; injectors needing tiltX/tiltY
/// convert (design/pen-tablet-input.md §2).
pub tilt_deg: u8,
/// Tilt azimuth, degrees `0..=359` clockwise from north, or [`PEN_ANGLE_UNKNOWN`].
pub azimuth_deg: u16,
/// Barrel roll (Apple Pencil Pro `rollAngle`), degrees `0..=359`, or [`PEN_ANGLE_UNKNOWN`].
pub roll_deg: u16,
/// µs since the previous sample in the same batch (`0` for the first) — preserves the
/// coalesced capture spacing for injectors/consumers that pace.
pub dt_us: u16,
}
impl Default for PenSample {
fn default() -> PenSample {
PenSample {
state: 0,
tool: PenTool::Pen,
x: 0.0,
y: 0.0,
pressure: 0,
distance: PEN_DISTANCE_UNKNOWN,
tilt_deg: PEN_TILT_UNKNOWN,
azimuth_deg: PEN_ANGLE_UNKNOWN,
roll_deg: PEN_ANGLE_UNKNOWN,
dt_us: 0,
}
}
}
impl PenSample {
fn encode_into(&self, out: &mut Vec<u8>) {
out.push(self.state);
out.push(self.tool as u8);
out.extend_from_slice(&self.x.to_le_bytes());
out.extend_from_slice(&self.y.to_le_bytes());
out.extend_from_slice(&self.pressure.to_le_bytes());
out.extend_from_slice(&self.distance.to_le_bytes());
out.push(self.tilt_deg);
out.extend_from_slice(&self.azimuth_deg.to_le_bytes());
out.extend_from_slice(&self.roll_deg.to_le_bytes());
out.extend_from_slice(&self.dt_us.to_le_bytes());
}
/// Decode one sample from exactly [`PEN_SAMPLE_WIRE_LEN`] bytes (caller bounds-checks).
/// `None` on a non-finite coordinate — an attacker-forged NaN/∞ must never reach an
/// injector's pixel scaling. Finite out-of-range coordinates clamp to `0.0..=1.0` (a
/// stroke drifting a hair past the letterbox edge is real input, not corruption).
fn decode(b: &[u8]) -> Option<PenSample> {
let f32at = |o: usize| f32::from_le_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]]);
let u16at = |o: usize| u16::from_le_bytes([b[o], b[o + 1]]);
let (x, y) = (f32at(2), f32at(6));
if !x.is_finite() || !y.is_finite() {
return None;
}
Some(PenSample {
state: b[0],
tool: PenTool::from_u8(b[1]),
x: x.clamp(0.0, 1.0),
y: y.clamp(0.0, 1.0),
pressure: u16at(10),
distance: u16at(12),
tilt_deg: b[14],
azimuth_deg: u16at(15),
roll_deg: u16at(17),
dt_us: u16at(19),
})
}
}
/// One pen datagram: `[0xCC][0x05][flags][count][u16 seq LE]` + `count` ×
/// [`PEN_SAMPLE_WIRE_LEN`]-byte samples, oldest first. `flags` is reserved (sent 0, ignored on
/// decode — semantic changes take a new 0xCC kind, never a flag reinterpretation). `seq` is the
/// sender's wrapping batch counter, the reorder gate ([`pen_seq_newer`]).
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct PenBatch {
pub seq: u16,
count: u8,
samples: [PenSample; PEN_BATCH_MAX],
}
impl PenBatch {
/// Build a batch from up to [`PEN_BATCH_MAX`] samples (a longer slice truncates — senders
/// with more coalesced samples split into consecutive batches so nothing is lost).
pub fn new(seq: u16, samples: &[PenSample]) -> PenBatch {
let count = samples.len().min(PEN_BATCH_MAX);
let mut buf = [PenSample::default(); PEN_BATCH_MAX];
buf[..count].copy_from_slice(&samples[..count]);
PenBatch {
seq,
count: count as u8,
samples: buf,
}
}
/// The batch's samples, oldest first.
pub fn samples(&self) -> &[PenSample] {
&self.samples[..self.count as usize]
}
pub fn encode(&self) -> Vec<u8> {
let n = self.count as usize;
let mut out = Vec::with_capacity(PEN_HEADER_LEN + n * PEN_SAMPLE_WIRE_LEN);
out.extend_from_slice(&[RICH_INPUT_MAGIC, RICH_PEN, 0, self.count]);
out.extend_from_slice(&self.seq.to_le_bytes());
for s in self.samples() {
s.encode_into(&mut out);
}
out
}
/// Parse a pen datagram. `None` on bad tag/kind, an empty batch, or a forged coordinate
/// (see [`PenSample::decode`]). Every read is bounded: `count` clamps to the declared
/// value, the fixed maximum, AND what the buffer actually holds — a torn datagram yields
/// the complete samples that arrived, never an over-read (the
/// [`RichInput::HidReport`](super::RichInput) truncation contract).
pub fn decode(b: &[u8]) -> Option<PenBatch> {
if b.len() < PEN_HEADER_LEN || b[0] != RICH_INPUT_MAGIC || b[1] != RICH_PEN {
return None;
}
let count = (b[3] as usize)
.min(PEN_BATCH_MAX)
.min((b.len() - PEN_HEADER_LEN) / PEN_SAMPLE_WIRE_LEN);
if count == 0 {
return None;
}
let mut samples = [PenSample::default(); PEN_BATCH_MAX];
for (i, slot) in samples.iter_mut().enumerate().take(count) {
let o = PEN_HEADER_LEN + i * PEN_SAMPLE_WIRE_LEN;
*slot = PenSample::decode(&b[o..o + PEN_SAMPLE_WIRE_LEN])?;
}
Some(PenBatch {
seq: u16::from_le_bytes([b[4], b[5]]),
count: count as u8,
samples,
})
}
}
/// The batch reorder gate: is `new` strictly newer than `last` on the wrapping u16 circle?
/// `None` (nothing applied yet) always passes. The u16 analog of
/// [`GamepadSnapshot::seq_newer`](crate::input::GamepadSnapshot::seq_newer): newer ⇔ the
/// forward distance is `1..=0x7FFF`, so reordered stale batches drop and a wrap (65535 → 0)
/// still counts as newer.
pub fn pen_seq_newer(new: u16, last: Option<u16>) -> bool {
match last {
None => true,
Some(last) => (new.wrapping_sub(last) as i16) > 0,
}
}
/// One synthesized stroke transition, in the order [`PenTracker`] emits them for a sample:
/// `ProximityIn?` → `Motion` → `TipDown?` → `ButtonsChanged?` → `TipUp?` → `ProximityOut?`.
/// `Motion` precedes `TipDown` so the contact lands where the sample says; a release
/// ([`PenTracker::force_release`] or an out-of-range sample) orders
/// `ButtonsChanged?` → `TipUp?` → `ProximityOut` so nothing is left held.
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum PenTransition {
/// The pen entered hover range. Injectors map [`PenTool::Unknown`] to a plain pen.
ProximityIn { tool: PenTool },
/// Position + all axes moved to this sample's values (emitted for every in-range sample).
Motion { sample: PenSample },
/// The tip made contact.
TipDown,
/// Barrel buttons changed: `pressed` / `released` are disjoint `PEN_BARREL*` subsets.
ButtonsChanged { pressed: u8, released: u8 },
/// The tip lifted.
TipUp,
/// The pen left hover range.
ProximityOut,
}
/// The host-side stroke state machine (one per session): diffs state-full [`PenSample`]s
/// against tracked state and appends the synthesized [`PenTransition`]s. Pure and clock-free —
/// the owner arms its own [`PEN_TOUCH_TIMEOUT_MS`] timer over [`PenTracker::is_active`] and
/// calls [`PenTracker::force_release`] when it fires (and on session teardown).
#[derive(Debug, Default)]
pub struct PenTracker {
last_seq: Option<u16>,
in_range: bool,
touching: bool,
buttons: u8,
tool: PenTool,
}
impl PenTracker {
/// Apply one decoded batch, appending transitions to `out` (callers reuse the buffer). A
/// stale batch ([`pen_seq_newer`]) is dropped whole with no transitions. Samples carrying
/// the reserved [`PEN_PREDICTED`] bit are skipped (never injected — module doc).
pub fn apply(&mut self, batch: &PenBatch, out: &mut Vec<PenTransition>) {
if !pen_seq_newer(batch.seq, self.last_seq) {
return;
}
self.last_seq = Some(batch.seq);
for s in batch.samples() {
if s.state & PEN_PREDICTED != 0 {
continue;
}
self.apply_sample(s, out);
}
}
/// The tracker holds live state a dead client could leave stuck (in range, or mid-stroke).
pub fn is_active(&self) -> bool {
self.in_range || self.touching
}
/// Release everything held (buttons → tip → proximity) — the [`PEN_TOUCH_TIMEOUT_MS`]
/// failsafe and session teardown. Keeps the seq gate armed so a late stale datagram from
/// the dead stroke cannot re-apply after the release.
pub fn force_release(&mut self, out: &mut Vec<PenTransition>) {
if self.buttons != 0 {
out.push(PenTransition::ButtonsChanged {
pressed: 0,
released: self.buttons,
});
self.buttons = 0;
}
if self.touching {
out.push(PenTransition::TipUp);
self.touching = false;
}
if self.in_range {
out.push(PenTransition::ProximityOut);
self.in_range = false;
}
}
fn apply_sample(&mut self, s: &PenSample, out: &mut Vec<PenTransition>) {
let touching = s.state & PEN_TOUCHING != 0;
// Touching implies in-range (a contact IS a proximity) — normalize here once so every
// consumer sees coherent states.
let in_range = touching || s.state & PEN_IN_RANGE != 0;
if !in_range {
self.force_release(out);
return;
}
// Unknown inherits the session's tool (a newer client's future tool must not thrash
// proximity); outside a session it grounds to the default.
let tool = match s.tool {
PenTool::Unknown if self.in_range => self.tool,
t => t,
};
// A tool switch mid-session is a physical re-entry (see [`PenTool`]).
if self.in_range && tool != self.tool {
self.force_release(out);
}
if !self.in_range {
out.push(PenTransition::ProximityIn { tool });
self.in_range = true;
}
self.tool = tool;
out.push(PenTransition::Motion { sample: *s });
if touching && !self.touching {
out.push(PenTransition::TipDown);
self.touching = true;
}
let buttons = s.state & PEN_BUTTONS_MASK;
if buttons != self.buttons {
out.push(PenTransition::ButtonsChanged {
pressed: buttons & !self.buttons,
released: self.buttons & !buttons,
});
self.buttons = buttons;
}
if !touching && self.touching {
out.push(PenTransition::TipUp);
self.touching = false;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::quic::RichInput;
fn hover(x: f32, y: f32) -> PenSample {
PenSample {
state: PEN_IN_RANGE,
x,
y,
distance: 300,
..Default::default()
}
}
fn touch(x: f32, y: f32, pressure: u16) -> PenSample {
PenSample {
state: PEN_IN_RANGE | PEN_TOUCHING,
x,
y,
pressure,
distance: 0,
tilt_deg: 35,
azimuth_deg: 180,
roll_deg: 90,
..Default::default()
}
}
#[test]
fn pen_batch_roundtrip_and_truncation() {
let samples = [hover(0.25, 0.5), touch(0.26, 0.5, 40000), {
let mut s = touch(0.27, 0.51, 42000);
s.state |= PEN_BARREL1;
s.dt_us = 4167;
s
}];
let b = PenBatch::new(7, &samples);
let d = b.encode();
assert_eq!(d[0], RICH_INPUT_MAGIC);
assert_eq!(d.len(), 6 + 3 * PEN_SAMPLE_WIRE_LEN);
let back = PenBatch::decode(&d).unwrap();
assert_eq!(back.seq, 7);
assert_eq!(back.samples(), &samples);
// A torn datagram yields exactly the complete samples that arrived — never an
// over-read, never a partial sample.
let torn = PenBatch::decode(&d[..6 + 2 * PEN_SAMPLE_WIRE_LEN + 5]).unwrap();
assert_eq!(torn.samples(), &samples[..2]);
// Header-only / empty batches and short buffers are rejected whole.
assert!(PenBatch::decode(&d[..PEN_HEADER_LEN]).is_none());
assert!(PenBatch::decode(&PenBatch::new(0, &[]).encode()).is_none());
// Wrong tag / wrong kind are None before any read.
let mut bad = d.clone();
bad[0] = 0xC8;
assert!(PenBatch::decode(&bad).is_none());
let mut bad = d;
bad[1] = 0x01; // RICH_TOUCHPAD
assert!(PenBatch::decode(&bad).is_none());
}
#[test]
fn pen_batch_oversize_truncates_and_flags_reserved() {
// 10 samples truncate to PEN_BATCH_MAX on construction (senders split instead).
let many: Vec<PenSample> = (0..10).map(|i| hover(i as f32 / 10.0, 0.5)).collect();
let b = PenBatch::new(1, &many);
assert_eq!(b.samples().len(), PEN_BATCH_MAX);
// A declared count larger than the payload clamps to what arrived.
let mut d = b.encode();
d[3] = 200;
assert_eq!(PenBatch::decode(&d).unwrap().samples().len(), PEN_BATCH_MAX);
// A nonzero reserved flags byte still parses (receivers MUST ignore).
d[2] = 0xAA;
assert!(PenBatch::decode(&d).is_some());
}
#[test]
fn pen_batch_rejects_forged_floats_and_clamps_stragglers() {
// NaN / ∞ coordinates kill the whole batch — nothing legitimate produces them.
for forged in [f32::NAN, f32::INFINITY, f32::NEG_INFINITY] {
let mut s = touch(0.5, 0.5, 1);
s.x = forged;
assert!(PenBatch::decode(&PenBatch::new(0, &[s]).encode()).is_none());
}
// A finite coordinate a hair outside the letterbox clamps instead (real input).
let mut s = hover(0.5, 0.5);
s.x = -0.01;
s.y = 1.25;
let back = PenBatch::decode(&PenBatch::new(0, &[s]).encode()).unwrap();
assert_eq!((back.samples()[0].x, back.samples()[0].y), (0.0, 1.0));
}
#[test]
fn pen_plane_is_disjoint_from_rich_input() {
// A pen datagram shares the 0xCC tag but is NOT a RichInput: a pre-pen host takes the
// documented unknown-kind drop, and the pen decoder rejects every RichInput kind.
let d = PenBatch::new(3, &[touch(0.5, 0.5, 100)]).encode();
assert!(RichInput::decode(&d).is_none());
let rich = RichInput::Touchpad {
pad: 0,
finger: 0,
active: true,
x: 1,
y: 2,
}
.encode();
assert!(PenBatch::decode(&rich).is_none());
}
#[test]
fn seq_gate_wraps_and_drops_stale() {
assert!(pen_seq_newer(0, None));
assert!(pen_seq_newer(6, Some(5)));
assert!(!pen_seq_newer(5, Some(5)));
assert!(!pen_seq_newer(4, Some(5)));
assert!(pen_seq_newer(2, Some(0xFFFE))); // wrap
assert!(!pen_seq_newer(0xFFFE, Some(2))); // stale across the wrap
}
/// Drives a tracker and returns the transitions of one batch.
fn run(t: &mut PenTracker, seq: u16, samples: &[PenSample]) -> Vec<PenTransition> {
let mut out = Vec::new();
t.apply(&PenBatch::new(seq, samples), &mut out);
out
}
#[test]
fn tracker_full_stroke_lifecycle() {
let mut t = PenTracker::default();
// Hover in → ProximityIn + Motion.
let h = hover(0.2, 0.2);
assert_eq!(
run(&mut t, 0, &[h]),
vec![
PenTransition::ProximityIn { tool: PenTool::Pen },
PenTransition::Motion { sample: h },
]
);
// Contact: Motion precedes TipDown so ink lands at the sample's position.
let c = touch(0.21, 0.2, 30000);
assert_eq!(
run(&mut t, 1, &[c]),
vec![PenTransition::Motion { sample: c }, PenTransition::TipDown]
);
assert!(t.is_active());
// Drag: motion only.
let m = touch(0.3, 0.25, 45000);
assert_eq!(
run(&mut t, 2, &[m]),
vec![PenTransition::Motion { sample: m }]
);
// Lift back to hover, then leave range: buttons(none) → TipUp, then ProximityOut.
let l = hover(0.3, 0.25);
assert_eq!(
run(&mut t, 3, &[l]),
vec![PenTransition::Motion { sample: l }, PenTransition::TipUp]
);
let gone = PenSample::default(); // state 0 = out of range
assert_eq!(run(&mut t, 4, &[gone]), vec![PenTransition::ProximityOut]);
assert!(!t.is_active());
}
#[test]
fn tracker_self_heals_lost_transitions() {
let mut t = PenTracker::default();
// The hover batch AND the tip-down batch were lost: the first surviving mid-stroke
// sample synthesizes the whole entry.
let m = touch(0.5, 0.5, 20000);
assert_eq!(
run(&mut t, 10, &[m]),
vec![
PenTransition::ProximityIn { tool: PenTool::Pen },
PenTransition::Motion { sample: m },
PenTransition::TipDown,
]
);
// The lift batch was lost; the next out-of-range sample heals it fully.
assert_eq!(
run(&mut t, 11, &[PenSample::default()]),
vec![PenTransition::TipUp, PenTransition::ProximityOut]
);
}
#[test]
fn tracker_drops_stale_batches_whole() {
let mut t = PenTracker::default();
let c = touch(0.5, 0.5, 100);
assert!(!run(&mut t, 5, &[c]).is_empty());
// A reordered older batch (a hover from before the contact) must not rewind the stroke.
assert!(run(&mut t, 4, &[hover(0.4, 0.4)]).is_empty());
assert!(t.is_active());
}
#[test]
fn tracker_buttons_and_eraser_reentry() {
let mut t = PenTracker::default();
let mut held = touch(0.5, 0.5, 100);
held.state |= PEN_BARREL1;
// Buttons apply after TipDown on entry…
assert_eq!(
run(&mut t, 0, &[held]),
vec![
PenTransition::ProximityIn { tool: PenTool::Pen },
PenTransition::Motion { sample: held },
PenTransition::TipDown,
PenTransition::ButtonsChanged {
pressed: PEN_BARREL1,
released: 0
},
]
);
// …swap BARREL1 → BARREL2 in one sample: one delta, both directions.
let mut swapped = held;
swapped.state = (swapped.state & !PEN_BARREL1) | PEN_BARREL2;
assert_eq!(
run(&mut t, 1, &[swapped]),
vec![
PenTransition::Motion { sample: swapped },
PenTransition::ButtonsChanged {
pressed: PEN_BARREL2,
released: PEN_BARREL1
},
]
);
// Tool switch to eraser mid-contact = full release + re-entry as the eraser, with the
// held button released first so nothing sticks across tools.
let mut erase = touch(0.5, 0.5, 200);
erase.tool = PenTool::Eraser;
assert_eq!(
run(&mut t, 2, &[erase]),
vec![
PenTransition::ButtonsChanged {
pressed: 0,
released: PEN_BARREL2
},
PenTransition::TipUp,
PenTransition::ProximityOut,
PenTransition::ProximityIn {
tool: PenTool::Eraser
},
PenTransition::Motion { sample: erase },
PenTransition::TipDown,
]
);
// Unknown tool inside the session inherits (no re-entry thrash from a newer client).
let mut unk = touch(0.51, 0.5, 210);
unk.tool = PenTool::Unknown;
assert_eq!(
run(&mut t, 3, &[unk]),
vec![PenTransition::Motion { sample: unk }]
);
}
#[test]
fn tracker_force_release_and_late_stale_datagram() {
let mut t = PenTracker::default();
let mut held = touch(0.5, 0.5, 100);
held.state |= PEN_BARREL2;
run(&mut t, 100, &[held]);
// The 200 ms failsafe / teardown: buttons → tip → proximity, all released.
let mut out = Vec::new();
t.force_release(&mut out);
assert_eq!(
out,
vec![
PenTransition::ButtonsChanged {
pressed: 0,
released: PEN_BARREL2
},
PenTransition::TipUp,
PenTransition::ProximityOut,
]
);
assert!(!t.is_active());
// Idempotent.
let mut out = Vec::new();
t.force_release(&mut out);
assert!(out.is_empty());
// The seq gate survives the release: a late datagram from the dead stroke is stale.
assert!(run(&mut t, 99, &[held]).is_empty());
assert!(!t.is_active());
// But the stroke after it proceeds normally.
assert!(!run(&mut t, 101, &[held]).is_empty());
assert!(t.is_active());
}
#[test]
fn tracker_skips_reserved_predicted_samples() {
let mut t = PenTracker::default();
let mut p = touch(0.5, 0.5, 100);
p.state |= PEN_PREDICTED;
// A v1 host must never inject a predicted sample — and skipping must not corrupt
// tracking for the real samples around it.
let real = touch(0.6, 0.5, 120);
let out = run(&mut t, 0, &[p, real]);
assert_eq!(
out,
vec![
PenTransition::ProximityIn { tool: PenTool::Pen },
PenTransition::Motion { sample: real },
PenTransition::TipDown,
]
);
}
}
+31
View File
@@ -175,6 +175,36 @@ pub fn capture_virtual_output(
},
) as pf_capture::CursorChannelSender
});
// The secure-desktop guard's actuator (`IOCTL_SET_CURSOR_FORWARD`): the capturer flips the
// driver's hardware-cursor declare off while UAC/Winlogon is up (the secure desktop renders
// only through the OS's software-cursor path) and back on at dismissal. The stand-down needs
// the same-mode re-commit that actualises the software-cursor default — driven here because
// topology commits belong under the vdisplay manager's lock, which pf-capture cannot take.
// Built for EVERY session (not just `want.hw_cursor`): a channel-less session can reuse a
// driver monitor whose cursor worker (an earlier session's) is still live and re-declaring —
// the flip is the only way to stop it; on a never-declared target the driver answers
// NOT_FOUND, which the capturer logs and ignores.
let target_id = target.target_id;
let cursor_forward: Option<pf_capture::CursorForwardSender> = Some({
std::sync::Arc::new(move |enable: bool| {
let req = pf_driver_proto::control::SetCursorForwardRequest {
target_id,
enable: enable as u32,
};
// SAFETY: `control_raw` is the pf-vdisplay control handle resolved above; it is
// never closed for the process lifetime (`send_cursor_forward`'s precondition).
unsafe {
crate::vdisplay::driver::send_cursor_forward(
windows::Win32::Foundation::HANDLE(control_raw as *mut core::ffi::c_void),
&req,
)?;
}
if !enable {
crate::vdisplay::manager::force_recommit();
}
Ok(())
}) as pf_capture::CursorForwardSender
});
pf_capture::open_idd_push(
target,
pref,
@@ -184,6 +214,7 @@ pub fn capture_virtual_output(
keep,
sender,
cursor_sender,
cursor_forward,
)
.map_err(|(e, _keep)| e.context("IDD-push capture open (no fallback)"))
}
+69
View File
@@ -8,6 +8,75 @@
use anyhow::Context;
use anyhow::Result;
/// Draw a scripted stylus stroke through the REAL pen chain — wire-shaped samples → the core
/// [`PenTracker`](punktfunk_core::quic::PenTracker) → the "Punktfunk Pen" uinput tablet — so
/// full-fidelity pen injection is validated without any client (design/pen-tablet-input.md P1):
/// hover in from the left, tip down, a sine stroke across the mapped output with a pressure
/// ramp + tilt sweep, tip up, hover out. Observe in Krita/GIMP with a pressure brush, or
/// `sudo libinput debug-events` (expect `TABLET_TOOL_PROXIMITY/TIP/AXIS`).
#[cfg(target_os = "linux")]
pub fn pen_test() -> Result<()> {
use punktfunk_core::quic::{
PenBatch, PenSample, PenTracker, PenTransition, PEN_IN_RANGE, PEN_TOUCHING,
};
use std::time::Duration;
let mut dev = crate::inject::pen::VirtualPen::create()?;
let mut tracker = PenTracker::default();
let mut out: Vec<PenTransition> = Vec::new();
// Compositors need a beat to enumerate the new evdev node before events count.
std::thread::sleep(Duration::from_secs(2));
let mut seq = 0u16;
let mut send = |tracker: &mut PenTracker, out: &mut Vec<PenTransition>, s: PenSample| {
out.clear();
tracker.apply(&PenBatch::new(seq, &[s]), out);
seq = seq.wrapping_add(1);
dev.apply_batch(out);
};
tracing::info!("pen-test: hover in, then a 3 s pressure-ramped sine stroke");
let hover = |x: f32| PenSample {
state: PEN_IN_RANGE,
x,
y: 0.5,
distance: 300,
..Default::default()
};
for i in 0..20 {
send(&mut tracker, &mut out, hover(0.05 + i as f32 * 0.005));
std::thread::sleep(Duration::from_millis(10));
}
const STEPS: u32 = 360;
for i in 0..=STEPS {
let t = i as f32 / STEPS as f32;
send(
&mut tracker,
&mut out,
PenSample {
state: PEN_IN_RANGE | PEN_TOUCHING,
x: 0.15 + 0.7 * t,
y: 0.5 + 0.2 * (t * std::f32::consts::TAU * 2.0).sin(),
// Ramp 10 % → 100 % so a pressure brush visibly widens along the stroke.
pressure: (6553.0 + 58982.0 * t) as u16,
distance: 0,
tilt_deg: 25 + (20.0 * t) as u8,
azimuth_deg: ((90.0 + 180.0 * t) as u16) % 360,
roll_deg: ((360.0 * t) as u16) % 360,
..Default::default()
},
);
std::thread::sleep(Duration::from_millis(8));
}
for i in 0..10 {
send(&mut tracker, &mut out, hover(0.85 + i as f32 * 0.005));
std::thread::sleep(Duration::from_millis(10));
}
send(&mut tracker, &mut out, PenSample::default()); // state 0 = out of range
tracing::info!("pen-test: done (stroke drawn, pen out of range) — device destroyed on exit");
Ok(())
}
/// Inject a scripted mouse + keyboard pattern through the session's input backend (libei on
/// KWin/GNOME, wlr on Sway). Lets us validate input injection without a Moonlight client.
#[cfg(target_os = "linux")]
@@ -71,6 +71,9 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
// from `seq`, so a per-message-type counter would reuse (key, nonce) pairs across
// message types in the host direction.
let mut pads = GamepadManager::new();
// Pen/touch translator (SS_PEN/SS_TOUCH → virtual tablet / wire touch). Sent only
// by clients that saw our SS_FF_PEN_TOUCH_EVENTS feature flag (rtsp.rs).
let mut pointer = super::pen::GsPointer::new();
let mut host_seq: u32 = 0;
// One-shot latch for the HDR-mode control message (0x010e); re-armed on Disconnect.
let mut hdr_sent = false;
@@ -90,8 +93,10 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
peer = None;
// Re-arm the HDR-mode signal for the next connection.
hdr_sent = false;
// Unplug the session's virtual pads.
// Unplug the session's virtual pads + tablet (destroying the
// uinput pen releases any held tool/tip kernel-side).
pads = GamepadManager::new();
pointer = super::pen::GsPointer::new();
}
Event::Receive {
channel_id, packet, ..
@@ -104,6 +109,7 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
&mut decrypt_fails,
&inj_tx,
&mut pads,
&mut pointer,
);
}
},
@@ -196,6 +202,7 @@ fn decode_rfi_range(pt: &[u8]) -> Option<(i64, i64)> {
/// Handle one received control packet: decrypt it (learning the GCM scheme on the first one),
/// decode any input event, and inject it into the host session.
#[allow(clippy::too_many_arguments)]
fn on_receive(
state: &AppState,
_channel_id: u8,
@@ -204,6 +211,7 @@ fn on_receive(
decrypt_fails: &mut u64,
inj_tx: &Sender<InputEvent>,
pads: &mut GamepadManager,
pointer: &mut super::pen::GsPointer,
) {
let Some(key) = state.launch.lock().unwrap().map(|s| s.gcm_key) else {
return; // control traffic before /launch — no key yet
@@ -274,6 +282,34 @@ fn on_receive(
return;
}
// Pen/touch extension events (Moonlight sends them only after seeing our feature flag):
// pen drives this session's virtual tablet; touch forwards as ordinary wire touches.
if let Some(p) = super::input::decode_pointer(&pt) {
pointer.apply(&p, |ev| {
let _ = inj_tx.send(ev);
});
return;
} else if super::input::is_pointer_magic(&pt) {
// A pointer magic that failed the body parse — a layout mismatch against this
// client, exactly what an on-glass "touch/pen does nothing" needs surfaced. The
// first few dump their bytes so the mismatch is diagnosable from the log alone.
static HEX_DUMPS: std::sync::atomic::AtomicU32 = std::sync::atomic::AtomicU32::new(0);
if HEX_DUMPS.fetch_add(1, std::sync::atomic::Ordering::Relaxed) < 5 {
let hex: String = pt.iter().map(|b| format!("{b:02x}")).collect();
tracing::warn!(
len = pt.len(),
bytes = %hex,
"gamestream: SS_TOUCH/SS_PEN packet failed to decode (malformed/unexpected layout)"
);
} else {
tracing::warn!(
len = pt.len(),
"gamestream: SS_TOUCH/SS_PEN packet failed to decode (malformed/unexpected layout)"
);
}
return;
}
let events = super::input::decode(&pt);
if events.is_empty() {
return; // keepalive / QoS / unhandled input kind
@@ -25,6 +25,8 @@ const MAGIC_MOUSE_BTN_UP: u32 = 0x09;
const MAGIC_SCROLL_GEN5: u32 = 0x0A;
const MAGIC_UTF8: u32 = 0x17;
const MAGIC_HSCROLL: u32 = 0x5500_0001;
const MAGIC_SS_TOUCH: u32 = 0x5500_0002;
const MAGIC_SS_PEN: u32 = 0x5500_0003;
/// `code` value marking a [`InputKind::MouseScroll`] as horizontal (vs `0` = vertical).
pub const SCROLL_HORIZONTAL: u32 = 1;
@@ -111,6 +113,106 @@ fn decode_input_packet(p: &[u8]) -> Option<InputEvent> {
})
}
/// One decoded `SS_PEN_PACKET` body (moonlight-common-c `Input.h`; all fields little-endian,
/// coordinates/pressure as normalized floats). Semantics — `pressure_or_distance` is pressure
/// (0..1, 0 = unknown) for contact events and hover distance (1 = farthest) for hover;
/// `rotation` is the tilt azimuth (0..360, `0xFFFF` unknown); `tilt` is degrees from the
/// surface normal (0..90, `0xFF` unknown) — are interpreted in [`super::pen`].
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct SsPen {
pub event_type: u8,
pub tool: u8,
pub buttons: u8,
pub x: f32,
pub y: f32,
pub pressure_or_distance: f32,
pub rotation: u16,
pub tilt: u8,
}
/// One decoded `SS_TOUCH_PACKET` body (same conventions as [`SsPen`]; contact-area fields are
/// not carried — the wire touch kinds have nowhere to put them yet).
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct SsTouch {
pub event_type: u8,
pub rotation: u16,
pub pointer_id: u32,
pub x: f32,
pub y: f32,
pub pressure_or_distance: f32,
}
/// A Sunshine-extension pointer event (sent only after we advertise
/// `SS_FF_PEN_TOUCH_EVENTS` — see [`super::rtsp`]).
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum SsPointer {
Pen(SsPen),
Touch(SsTouch),
}
/// Whether this control plaintext carries a pointer magic ([`decode_pointer`]'s domain) —
/// lets the caller tell "malformed pointer packet" (worth a warn) apart from "some other
/// message" when `decode_pointer` returns `None`.
pub fn is_pointer_magic(plaintext: &[u8]) -> bool {
plaintext.len() >= 12
&& u16::from_le_bytes([plaintext[0], plaintext[1]]) == INPUT_DATA_TYPE
&& matches!(
u32::from_le_bytes([plaintext[8], plaintext[9], plaintext[10], plaintext[11]]),
MAGIC_SS_TOUCH | MAGIC_SS_PEN
)
}
/// Decode a control plaintext into a pen/touch pointer event, or `None` for every other
/// message (the caller then falls through to [`decode`]). Bounds- and sanity-checked like the
/// rest of the plane: short bodies and non-finite floats (a forged NaN must never reach the
/// injectors' scaling) drop the packet whole.
pub fn decode_pointer(plaintext: &[u8]) -> Option<SsPointer> {
if plaintext.len() < 12 || u16::from_le_bytes([plaintext[0], plaintext[1]]) != INPUT_DATA_TYPE {
return None;
}
let p = &plaintext[4..];
let magic = u32::from_le_bytes([p[4], p[5], p[6], p[7]]);
let b = &p[8..];
// Coordinates must be finite (they feed the injectors' scaling); a forged NaN drops the
// packet whole.
let f32at = |o: usize| -> Option<f32> {
let v = f32::from_le_bytes([*b.get(o)?, *b.get(o + 1)?, *b.get(o + 2)?, *b.get(o + 3)?]);
v.is_finite().then_some(v)
};
// pressureOrDistance is different: real clients ship NaN there to mean "unknown" (iPad
// fingers have no force sensor — observed live from VoidLink, which NaN'd every touch and
// a strict gate silently killed the whole plane). The spec's own unknown value is 0.0, so
// non-finite sanitizes to that instead of poisoning the packet.
let f32_pressure = |o: usize| -> Option<f32> {
let v = f32::from_le_bytes([*b.get(o)?, *b.get(o + 1)?, *b.get(o + 2)?, *b.get(o + 3)?]);
Some(if v.is_finite() { v } else { 0.0 })
};
match magic {
// eventType, zero[1], rotation u16, pointerId u32, x, y, pressureOrDistance, areas.
MAGIC_SS_TOUCH => Some(SsPointer::Touch(SsTouch {
event_type: *b.first()?,
rotation: u16::from_le_bytes([*b.get(2)?, *b.get(3)?]),
pointer_id: u32::from_le_bytes([*b.get(4)?, *b.get(5)?, *b.get(6)?, *b.get(7)?]),
x: f32at(8)?,
y: f32at(12)?,
pressure_or_distance: f32_pressure(16)?,
})),
// eventType, toolType, penButtons, zero[1], x, y, pressureOrDistance, rotation u16,
// tilt, zero2[1], areas.
MAGIC_SS_PEN => Some(SsPointer::Pen(SsPen {
event_type: *b.first()?,
tool: *b.get(1)?,
buttons: *b.get(2)?,
x: f32at(4)?,
y: f32at(8)?,
pressure_or_distance: f32_pressure(12)?,
rotation: u16::from_le_bytes([*b.get(16)?, *b.get(17)?]),
tilt: *b.get(18)?,
})),
_ => None,
}
}
fn ev(kind: InputKind, code: u32, x: i32, y: i32, flags: u32) -> InputEvent {
InputEvent {
kind,
@@ -176,6 +278,78 @@ mod tests {
assert!(decode(&bad).is_empty());
}
#[test]
fn decodes_ss_pen_and_touch_golden_bytes() {
// SS_PEN body per Input.h: DOWN, pen tool, primary button, x=0.5 y=0.25,
// pressure=0.75, rotation=180, tilt=45, then contact areas (present but ignored).
let mut body = vec![0x01, 0x01, 0x01, 0x00];
for f in [0.5f32, 0.25, 0.75] {
body.extend_from_slice(&f.to_le_bytes());
}
body.extend_from_slice(&180u16.to_le_bytes());
body.extend_from_slice(&[45, 0x00]);
for f in [0.0f32, 0.0] {
body.extend_from_slice(&f.to_le_bytes());
}
let pt = wrap(0x5500_0003, &body);
assert_eq!(
decode_pointer(&pt),
Some(SsPointer::Pen(SsPen {
event_type: 0x01,
tool: 0x01,
buttons: 0x01,
x: 0.5,
y: 0.25,
pressure_or_distance: 0.75,
rotation: 180,
tilt: 45,
}))
);
// A pen packet is invisible to the classic decoder (no misparse as mouse/key).
assert!(decode(&pt).is_empty());
// SS_TOUCH body: MOVE, rotation unknown, pointerId 42, x=1.0 y=0.0, pressure 1.0.
let mut body = vec![0x03, 0x00];
body.extend_from_slice(&0xFFFFu16.to_le_bytes());
body.extend_from_slice(&42u32.to_le_bytes());
for f in [1.0f32, 0.0, 1.0, 0.0, 0.0] {
body.extend_from_slice(&f.to_le_bytes());
}
let pt = wrap(0x5500_0002, &body);
assert_eq!(
decode_pointer(&pt),
Some(SsPointer::Touch(SsTouch {
event_type: 0x03,
rotation: 0xFFFF,
pointer_id: 42,
x: 1.0,
y: 0.0,
pressure_or_distance: 1.0,
}))
);
// Truncated bodies and forged NaN coordinates drop the packet whole.
assert_eq!(decode_pointer(&pt[..pt.len() - 18]), None);
let mut nan = body.clone();
nan[8..12].copy_from_slice(&f32::NAN.to_le_bytes());
assert_eq!(decode_pointer(&wrap(0x5500_0002, &nan)), None);
// …but a NaN pressureOrDistance sanitizes to 0.0 ("unknown") instead of killing the
// packet — a REAL client convention, observed live: VoidLink NaN's the pressure of
// every iPad finger touch (no force sensor), and the strict gate silently disabled
// the whole touch plane.
let mut nan_pod = body.clone();
nan_pod[16..20].copy_from_slice(&f32::NAN.to_le_bytes());
match decode_pointer(&wrap(0x5500_0002, &nan_pod)) {
Some(SsPointer::Touch(t)) => assert_eq!(t.pressure_or_distance, 0.0),
other => panic!("NaN pressure must decode with pod=0.0, got {other:?}"),
}
// Non-pointer magics fall through to the classic decoder.
assert_eq!(
decode_pointer(&wrap(MAGIC_MOUSE_REL_GEN5, &[0, 0, 0, 0])),
None
);
}
#[test]
fn ignores_non_input_type() {
let mut pt = vec![0x00, 0x02]; // type 0x0200 (keepalive)
@@ -18,6 +18,8 @@ mod input;
mod mdns;
mod nvhttp;
mod pairing;
/// Moonlight `SS_PEN`/`SS_TOUCH` → the native pen model / wire touch (design/pen-tablet-input.md §4).
mod pen;
mod rtsp;
mod serverinfo;
mod stream;
+371
View File
@@ -0,0 +1,371 @@
//! Translate Moonlight's edge-based `SS_PEN` / `SS_TOUCH` events (design/pen-tablet-input.md
//! §4) into punktfunk's state-full pen model and wire-touch events.
//!
//! Pen: each packet becomes one complete [`PenSample`] (packet fields merged over the last
//! known state, since e.g. `BUTTON_ONLY` carries no position by spec), fed through the same
//! [`PenTracker`] → [`VirtualPen`] chain the native plane uses — so Moonlight iOS with an
//! Apple Pencil exercises the exact injection path our own clients will.
//!
//! No stroke timeout here, deliberately: the native plane's 200 ms failsafe exists because
//! datagrams are lossy and clients heartbeat; Moonlight's control stream is ordered/reliable
//! ENet and its clients do NOT heartbeat a stationary pen — a timeout would lift a held
//! stroke. Lost-client cleanup is the ENet disconnect (the control loop re-news this
//! translator, and destroying the uinput device releases everything kernel-side).
//!
//! Touch: forwarded as the ordinary wire touch kinds (`TouchDown/Move/Up`, normalized onto a
//! synthetic 65535² surface) through the injector service — pressure/area are dropped v1
//! (the wire touch has no field for them; `RICH_TOUCH` is the design's §8 upgrade).
use super::input::{SsPen, SsPointer, SsTouch};
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{
PenSample, PenTool, PenTracker, PenTransition, PEN_ANGLE_UNKNOWN, PEN_BARREL1, PEN_BARREL2,
PEN_IN_RANGE, PEN_TILT_UNKNOWN, PEN_TOUCHING,
};
// moonlight-common-c Limelight.h event/tool/button vocabulary.
const LI_TOUCH_EVENT_HOVER: u8 = 0x00;
const LI_TOUCH_EVENT_DOWN: u8 = 0x01;
const LI_TOUCH_EVENT_UP: u8 = 0x02;
const LI_TOUCH_EVENT_MOVE: u8 = 0x03;
const LI_TOUCH_EVENT_CANCEL: u8 = 0x04;
const LI_TOUCH_EVENT_BUTTON_ONLY: u8 = 0x05;
const LI_TOUCH_EVENT_HOVER_LEAVE: u8 = 0x06;
const LI_TOUCH_EVENT_CANCEL_ALL: u8 = 0x07;
const LI_TOOL_TYPE_PEN: u8 = 0x01;
const LI_TOOL_TYPE_ERASER: u8 = 0x02;
const LI_PEN_BUTTON_PRIMARY: u8 = 0x01;
const LI_PEN_BUTTON_SECONDARY: u8 = 0x02;
const LI_ROT_UNKNOWN: u16 = 0xFFFF;
const LI_TILT_UNKNOWN: u8 = 0xFF;
/// Synthetic reference extent for forwarded touches: coordinates arrive normalized, so map
/// them onto a full-range surface and let the injector rescale to its output (the
/// `MouseMoveAbs`/touch `flags = (w << 16) | h` contract).
const TOUCH_SURFACE: u32 = 65535;
/// Most concurrently-tracked touch pointers (for `CANCEL_ALL` replay). A flood of distinct
/// ids can't grow memory — untracked ids still forward down/up verbatim, they just miss a
/// later `CANCEL_ALL` (which a real client never has more than ~10 contacts for anyway).
const MAX_TOUCH_IDS: usize = 32;
/// Per-GameStream-session pen + touch translator. Owned by the control loop next to the
/// gamepad manager; re-created on client disconnect (the dropped [`VirtualPen`] destroys the
/// uinput device, which releases any held tool/tip kernel-side).
pub struct GsPointer {
tracker: PenTracker,
dev: Option<crate::inject::pen::VirtualPen>,
create_failed: bool,
seq: u16,
out: Vec<PenTransition>,
/// Last complete pen state — the merge base for packets that omit fields
/// (`BUTTON_ONLY` ignores x/y/pressure/tilt/rotation by spec).
last: PenSample,
/// This client sends hover events, so an `UP` means "back to hover" rather than "gone" —
/// clients without hover support get proximity-out on lift instead of a pen parked
/// forever at the last point.
saw_hover: bool,
/// Active forwarded touch ids, for `CANCEL_ALL` replay as per-id ups.
touch_ids: Vec<u32>,
/// Whether this session's first SS_TOUCH was logged (the one observable breadcrumb an
/// on-glass "touch does nothing" report needs — every later stage logs its own failure).
touch_seen: bool,
}
impl GsPointer {
pub fn new() -> GsPointer {
GsPointer {
tracker: PenTracker::default(),
dev: None,
create_failed: false,
seq: 0,
out: Vec::new(),
last: PenSample::default(),
saw_hover: false,
touch_ids: Vec::new(),
touch_seen: false,
}
}
/// Apply one decoded pointer packet. Touch events forward through `sink` (the injector
/// channel); pen events drive this session's virtual tablet.
pub fn apply(&mut self, p: &SsPointer, sink: impl FnMut(InputEvent)) {
match p {
SsPointer::Pen(pen) => self.apply_pen(pen),
SsPointer::Touch(touch) => self.apply_touch(touch, sink),
}
}
fn apply_pen(&mut self, p: &SsPen) {
let Some(sample) = self.pen_sample(p) else {
return;
};
self.last = sample;
if self.dev.is_none() && !self.create_failed {
match crate::inject::pen::VirtualPen::create() {
Ok(d) => self.dev = Some(d),
Err(e) => {
self.create_failed = true;
tracing::warn!(
error = %format!("{e:#}"),
"gamestream pen: virtual tablet creation failed — dropping pen input"
);
}
}
}
self.out.clear();
let batch = punktfunk_core::quic::PenBatch::new(self.seq, &[sample]);
self.seq = self.seq.wrapping_add(1);
self.tracker.apply(&batch, &mut self.out);
if let Some(dev) = self.dev.as_mut() {
dev.apply_batch(&self.out);
}
}
/// Merge one edge-based packet over the last known state into a complete sample.
/// `None` = nothing to apply (a `BUTTON_ONLY` before any positioned event).
fn pen_sample(&mut self, p: &SsPen) -> Option<PenSample> {
let buttons = (if p.buttons & LI_PEN_BUTTON_PRIMARY != 0 {
PEN_BARREL1
} else {
0
}) | (if p.buttons & LI_PEN_BUTTON_SECONDARY != 0 {
PEN_BARREL2
} else {
0
});
let tool = match p.tool {
LI_TOOL_TYPE_PEN => PenTool::Pen,
LI_TOOL_TYPE_ERASER => PenTool::Eraser,
_ => PenTool::Unknown,
};
// BUTTON_ONLY ignores x/y/pressure/rotation/tilt by spec — reuse the last state.
if p.event_type == LI_TOUCH_EVENT_BUTTON_ONLY {
if !self.tracker.is_active() {
return None;
}
let mut s = self.last;
s.state = (s.state & !(PEN_BARREL1 | PEN_BARREL2)) | buttons;
return Some(s);
}
let mut s = PenSample {
state: buttons,
tool,
x: p.x.clamp(0.0, 1.0),
y: p.y.clamp(0.0, 1.0),
dt_us: 0,
roll_deg: PEN_ANGLE_UNKNOWN, // the GameStream wire has no barrel-roll axis
azimuth_deg: if p.rotation == LI_ROT_UNKNOWN {
PEN_ANGLE_UNKNOWN
} else {
p.rotation % 360
},
tilt_deg: if p.tilt == LI_TILT_UNKNOWN {
PEN_TILT_UNKNOWN
} else {
p.tilt.min(90)
},
..PenSample::default()
};
match p.event_type {
LI_TOUCH_EVENT_DOWN | LI_TOUCH_EVENT_MOVE => {
s.state |= PEN_IN_RANGE | PEN_TOUCHING;
// pressureOrDistance is pressure (0..1) in contact — and 0.0 means UNKNOWN
// there, which must still ink: binary-stylus clients get full pressure.
s.pressure = if p.pressure_or_distance <= 0.0 {
u16::MAX
} else {
(p.pressure_or_distance.clamp(0.0, 1.0) * 65535.0) as u16
};
s.distance = 0;
}
LI_TOUCH_EVENT_HOVER => {
self.saw_hover = true;
s.state |= PEN_IN_RANGE;
// Hovering: pressureOrDistance is distance, 1.0 = farthest.
s.distance = (p.pressure_or_distance.clamp(0.0, 1.0) * 65534.0) as u16;
}
LI_TOUCH_EVENT_UP => {
// A hover-capable client keeps proximity after lift (its HOVER/HOVER_LEAVE
// events own the exit); one that never hovers would otherwise park the pen
// in proximity forever, so lift = leave for those.
if self.saw_hover {
s.state |= PEN_IN_RANGE;
}
}
LI_TOUCH_EVENT_HOVER_LEAVE | LI_TOUCH_EVENT_CANCEL | LI_TOUCH_EVENT_CANCEL_ALL => {
s.state &= !(PEN_BARREL1 | PEN_BARREL2); // out of range releases buttons too
}
_ => return None, // unknown future event type — drop, never guess
}
Some(s)
}
fn apply_touch(&mut self, t: &SsTouch, mut sink: impl FnMut(InputEvent)) {
if !self.touch_seen {
self.touch_seen = true;
tracing::info!(
event_type = t.event_type,
"gamestream: touch plane active (first SS_TOUCH from this client)"
);
}
let ev = |kind: InputKind, id: u32, x: f32, y: f32| InputEvent {
kind,
_pad: [0; 3],
code: id,
x: (x.clamp(0.0, 1.0) * TOUCH_SURFACE as f32) as i32,
y: (y.clamp(0.0, 1.0) * TOUCH_SURFACE as f32) as i32,
flags: (TOUCH_SURFACE << 16) | TOUCH_SURFACE,
};
match t.event_type {
LI_TOUCH_EVENT_DOWN => {
if !self.touch_ids.contains(&t.pointer_id) && self.touch_ids.len() < MAX_TOUCH_IDS {
self.touch_ids.push(t.pointer_id);
}
sink(ev(InputKind::TouchDown, t.pointer_id, t.x, t.y));
}
LI_TOUCH_EVENT_MOVE => sink(ev(InputKind::TouchMove, t.pointer_id, t.x, t.y)),
LI_TOUCH_EVENT_UP | LI_TOUCH_EVENT_CANCEL => {
self.touch_ids.retain(|&id| id != t.pointer_id);
sink(ev(InputKind::TouchUp, t.pointer_id, t.x, t.y));
}
LI_TOUCH_EVENT_CANCEL_ALL => {
for id in self.touch_ids.drain(..) {
sink(ev(InputKind::TouchUp, id, 0.0, 0.0));
}
}
// Touch hover has no wire representation (and BUTTON_ONLY is pen-only in
// practice) — nothing to forward.
_ => {}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn pen(event_type: u8, x: f32, y: f32, pod: f32) -> SsPen {
SsPen {
event_type,
tool: LI_TOOL_TYPE_PEN,
buttons: 0,
x,
y,
pressure_or_distance: pod,
rotation: 270,
tilt: 40,
}
}
#[test]
fn pen_down_move_up_maps_to_state_full_samples() {
let mut g = GsPointer::new();
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_DOWN, 0.25, 0.5, 0.5))
.unwrap();
assert_eq!(s.state, PEN_IN_RANGE | PEN_TOUCHING);
assert_eq!(s.pressure, 32767);
assert_eq!((s.tilt_deg, s.azimuth_deg), (40, 270));
assert_eq!(s.roll_deg, PEN_ANGLE_UNKNOWN);
// Unknown contact pressure (0.0) must still ink — binary-stylus full scale.
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_MOVE, 0.3, 0.5, 0.0))
.unwrap();
assert_eq!(s.pressure, u16::MAX);
// No hover ever seen: UP = fully out of range (no parked proximity).
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_UP, 0.3, 0.5, 0.0))
.unwrap();
assert_eq!(s.state, 0);
}
#[test]
fn hover_capable_client_keeps_proximity_on_lift() {
let mut g = GsPointer::new();
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_HOVER, 0.2, 0.2, 0.5))
.unwrap();
assert_eq!(s.state, PEN_IN_RANGE);
assert_eq!(s.distance, 32767);
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_UP, 0.2, 0.2, 0.0))
.unwrap();
assert_eq!(s.state, PEN_IN_RANGE);
// HOVER_LEAVE exits.
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_HOVER_LEAVE, 0.2, 0.2, 0.0))
.unwrap();
assert_eq!(s.state, 0);
}
#[test]
fn button_only_merges_over_last_state() {
let mut g = GsPointer::new();
// BUTTON_ONLY before any positioned event: nothing to merge over.
let mut b = pen(LI_TOUCH_EVENT_BUTTON_ONLY, 0.0, 0.0, 0.0);
b.buttons = LI_PEN_BUTTON_PRIMARY;
assert!(g.pen_sample(&b).is_none());
// After a DOWN is applied (through the real tracker path minus the device), the
// button packet reuses the held position/pressure.
g.apply_pen(&pen(LI_TOUCH_EVENT_DOWN, 0.4, 0.6, 0.8));
let s = g.pen_sample(&b).unwrap();
assert_eq!(s.state & (PEN_BARREL1 | PEN_BARREL2), PEN_BARREL1);
assert_eq!(s.state & PEN_TOUCHING, PEN_TOUCHING);
assert!((s.x - 0.4).abs() < 1e-6);
assert_eq!(s.pressure, (0.8f32 * 65535.0) as u16);
}
#[test]
fn eraser_and_unknown_tools_map() {
let mut g = GsPointer::new();
let mut p = pen(LI_TOUCH_EVENT_DOWN, 0.5, 0.5, 1.0);
p.tool = LI_TOOL_TYPE_ERASER;
assert_eq!(g.pen_sample(&p).unwrap().tool, PenTool::Eraser);
p.tool = 0x7F;
assert_eq!(g.pen_sample(&p).unwrap().tool, PenTool::Unknown);
// Unknown sentinel angles pass through as our sentinels.
p.rotation = LI_ROT_UNKNOWN;
p.tilt = LI_TILT_UNKNOWN;
let s = g.pen_sample(&p).unwrap();
assert_eq!(s.azimuth_deg, PEN_ANGLE_UNKNOWN);
assert_eq!(s.tilt_deg, PEN_TILT_UNKNOWN);
}
#[test]
fn touch_forwards_wire_touch_and_cancel_all_replays_ups() {
let mut g = GsPointer::new();
let mut got: Vec<InputEvent> = Vec::new();
let t = |event_type, pointer_id, x: f32| SsTouch {
event_type,
rotation: 0,
pointer_id,
x,
y: 0.5,
pressure_or_distance: 0.5,
};
g.apply_touch(&t(LI_TOUCH_EVENT_DOWN, 7, 0.5), |e| got.push(e));
g.apply_touch(&t(LI_TOUCH_EVENT_DOWN, 9, 0.25), |e| got.push(e));
g.apply_touch(&t(LI_TOUCH_EVENT_MOVE, 7, 0.75), |e| got.push(e));
assert_eq!(got.len(), 3);
assert_eq!(got[0].kind, InputKind::TouchDown);
assert_eq!(got[0].code, 7);
assert_eq!(got[0].x, 32767);
assert_eq!(got[0].flags, (65535 << 16) | 65535);
assert_eq!(got[2].kind, InputKind::TouchMove);
assert_eq!(got[2].x, 49151);
// CANCEL_ALL lifts every tracked contact.
got.clear();
g.apply_touch(&t(LI_TOUCH_EVENT_CANCEL_ALL, 0, 0.0), |e| got.push(e));
assert_eq!(got.len(), 2);
assert!(got.iter().all(|e| e.kind == InputKind::TouchUp));
let mut ids: Vec<u32> = got.iter().map(|e| e.code).collect();
ids.sort_unstable();
assert_eq!(ids, vec![7, 9]);
// And the set is empty now — a second CANCEL_ALL is a no-op.
got.clear();
g.apply_touch(&t(LI_TOUCH_EVENT_CANCEL_ALL, 0, 0.0), |e| got.push(e));
assert!(got.is_empty());
}
}
+15 -1
View File
@@ -309,12 +309,26 @@ fn handle_request(req: &Request, state: &AppState, peer: Option<SocketAddr>) ->
}
}
/// moonlight-common-c `LI_FF_PEN_TOUCH_EVENTS`: with this featureFlags bit set, Moonlight
/// clients send native `SS_PEN`/`SS_TOUCH` events (an iPad's Apple Pencil included) instead
/// of synthesizing mouse input client-side.
const SS_FF_PEN_TOUCH_EVENTS: u32 = 0x01;
/// Host capability SDP returned by DESCRIBE. Advertises HEVC + AV1 and no encryption
/// (plaintext streams for now; P1.5 adds the negotiated AES paths).
fn describe_sdp() -> String {
// Pen/touch events are advertised only where we can actually inject them (Linux with
// uinput — the same gate as HOST_CAP_PEN; design/pen-tablet-input.md §4). Elsewhere the
// flag stays 0 so Moonlight keeps its client-side mouse emulation for touch, exactly as
// today. PUNKTFUNK_PEN=0 clears it (the operator kill-switch inside pen_supported).
let feature_flags: u32 = if crate::inject::pen_supported() {
SS_FF_PEN_TOUCH_EVENTS
} else {
0
};
// Line-oriented a=key:value, matching what moonlight-common-c scans for.
let mut lines: Vec<String> = vec![
"a=x-ss-general.featureFlags:0".into(),
format!("a=x-ss-general.featureFlags:{feature_flags}"),
"a=x-ss-general.encryptionSupported:0".into(),
"a=x-ss-general.encryptionRequested:0".into(),
"sprop-parameter-sets=AAAAAU".into(), // HEVC capability indicator
+438 -8
View File
@@ -6,7 +6,9 @@ use super::*;
/// Reads the **local** Steam install — no Steam Web API key, no network. Installed titles come
/// from `steamapps/appmanifest_<appid>.acf`; extra library folders from
/// `steamapps/libraryfolders.vdf`; artwork from the public Steam CDN by appid.
/// `steamapps/libraryfolders.vdf`; the user's own non-Steam shortcuts ("Add a Non-Steam Game to My
/// Library") from each account's binary `userdata/<id>/config/shortcuts.vdf`; artwork from the
/// public Steam CDN by appid, or the user's per-account `grid/` overrides (all a shortcut ever has).
pub struct SteamProvider;
impl LibraryProvider for SteamProvider {
@@ -21,7 +23,7 @@ impl LibraryProvider for SteamProvider {
by_appid.entry(appid).or_insert(name); // first library wins; dedups shared appids
}
}
by_appid
let mut games: Vec<GameEntry> = by_appid
.into_iter()
.filter(|(appid, name)| !is_steam_tool(*appid, name))
.map(|(appid, title)| GameEntry {
@@ -35,7 +37,11 @@ impl LibraryProvider for SteamProvider {
value: appid.to_string(),
}),
})
.collect()
.collect();
// Non-Steam shortcuts have no `appmanifest` — [`scan_manifests`] can't see them, so the
// user's own custom entries are gathered separately from `shortcuts.vdf`.
games.extend(steam_shortcuts());
games
}
}
@@ -54,17 +60,26 @@ fn steam_art(appid: u32) -> Artwork {
}
/// Resolve one Steam cover-art kind to bytes: the host's own local Steam cache first (exact — it's
/// literally what the user's Steam client already shows for this title), the legacy flat CDN URL
/// as a fallback. `None` when neither has it (the client then falls through to its next art
/// candidate). Blocking (disk + network) — call off the async runtime.
/// literally what the user's Steam client already shows for this title), then the user's per-account
/// `grid/` overrides (the *only* art a non-Steam shortcut ever has), then the legacy flat CDN URL.
/// `None` when none has it (the client then falls through to its next art candidate). Blocking
/// (disk + network) — call off the async runtime.
pub fn steam_art_bytes(appid: u32, kind: ArtKind) -> Option<(Vec<u8>, String)> {
steam_local_art_bytes(appid, kind).or_else(|| {
if let Some(local) =
steam_local_art_bytes(appid, kind).or_else(|| steam_grid_art_bytes(appid, kind))
{
return Some(local);
}
// A non-Steam shortcut's appid has the high bit set (see [`shortcut_appid`]) and is never a real
// store appid, so the CDN would only 404 — skip the wasted request and fall through cleanly.
if appid & 0x8000_0000 != 0 {
return None;
}
let url = format!(
"https://cdn.cloudflare.steamstatic.com/steam/apps/{appid}/{}",
kind.cdn_filename()
);
fetch_image(&url)
})
}
/// Cap on a local librarycache file we'll read into memory — generous for a Steam-quality JPEG/PNG
@@ -113,6 +128,59 @@ fn find_local_art_file(root: &Path, appid: u32, kind: ArtKind) -> Option<PathBuf
None
}
/// Artwork a user set in Steam itself for a **non-Steam shortcut** (or a `grid/` override for a real
/// title), stored per-account under `userdata/<id>/config/grid/`, keyed by the same 32-bit appid the
/// shortcut carries. Tried before the CDN — a shortcut has no CDN art at all, so this is where its
/// poster lives.
fn steam_grid_art_bytes(appid: u32, kind: ArtKind) -> Option<(Vec<u8>, String)> {
for root in steam_roots() {
let Ok(users) = std::fs::read_dir(root.join("userdata")) else {
continue;
};
for user in users.flatten() {
let grid = user.path().join("config").join("grid");
if let Some(path) = find_grid_art_file(&grid, appid, kind) {
if let Ok(bytes) = std::fs::read(&path) {
let ctype = if path.extension().is_some_and(|e| e == "png") {
"image/png"
} else {
"image/jpeg"
};
return Some((bytes, ctype.to_string()));
}
}
}
}
None
}
/// The `grid/` filenames Steam stores this art kind under for appid `<A>`, tried in order (PNG then
/// JPG — Steam accepts either). Pure path logic, so it's unit-testable against a directory fixture.
fn find_grid_art_file(grid: &Path, appid: u32, kind: ArtKind) -> Option<PathBuf> {
for name in grid_filenames(kind, appid) {
let path = grid.join(&name);
if let Ok(meta) = std::fs::metadata(&path) {
if meta.len() > 0 && meta.len() <= LOCAL_ART_MAX_BYTES {
return Some(path);
}
}
}
None
}
/// The `userdata/<id>/config/grid/` basenames Steam names each art kind under for appid `<A>`:
/// portrait `<A>p`, hero `<A>_hero`, logo `<A>_logo`, and the wide capsule `<A>` — each as `.png`
/// then `.jpg`.
fn grid_filenames(kind: ArtKind, appid: u32) -> Vec<String> {
let both = |base: String| vec![format!("{base}.png"), format!("{base}.jpg")];
match kind {
ArtKind::Portrait => both(format!("{appid}p")),
ArtKind::Hero => both(format!("{appid}_hero")),
ArtKind::Logo => both(format!("{appid}_logo")),
ArtKind::Header => both(format!("{appid}")),
}
}
/// Candidate Steam roots (classic, Flatpak, Deck) that actually exist, canonicalized + deduped.
#[cfg(not(target_os = "windows"))]
fn steam_roots() -> Vec<PathBuf> {
@@ -243,6 +311,226 @@ fn is_steam_tool(appid: u32, name: &str) -> bool {
|| n.contains("steamvr")
}
/// One non-Steam shortcut ("Add a Non-Steam Game to My Library"), as read from `shortcuts.vdf`.
struct Shortcut {
/// The 32-bit shortcut appid Steam assigns it (high bit set — see [`shortcut_appid`]). Keys the
/// entry id and its `grid/` artwork; the 64-bit launch id derives from it ([`shortcut_gameid`]).
appid: u32,
/// Display name (`AppName`).
name: String,
/// Whether Steam has this shortcut hidden from the library (`IsHidden`) — we honor that.
hidden: bool,
}
/// Every non-Steam shortcut across all Steam accounts on this host, as launchable [`GameEntry`]s.
/// These carry no `appmanifest`, so [`scan_manifests`] never sees them — this is the only path that
/// surfaces a user's custom Steam entries. Best-effort: an unreadable/absent `shortcuts.vdf`
/// contributes nothing; hidden shortcuts and duplicate appids are dropped.
fn steam_shortcuts() -> Vec<GameEntry> {
let mut seen = HashSet::new();
let mut out = Vec::new();
for path in shortcuts_files() {
let Ok(bytes) = std::fs::read(&path) else {
continue;
};
for sc in parse_shortcuts(&bytes) {
if seen.insert(sc.appid) {
if let Some(entry) = shortcut_entry(sc) {
out.push(entry);
}
}
}
}
out
}
/// Map one parsed [`Shortcut`] to a library entry, or `None` if Steam has it hidden. Launch reuses
/// the `steam_appid` recipe (`steam steam://rungameid/<id>`) — the value is the 64-bit shortcut
/// game id, not the 32-bit appid, because the plain appid won't launch a non-Steam shortcut.
fn shortcut_entry(sc: Shortcut) -> Option<GameEntry> {
if sc.hidden {
return None;
}
Some(GameEntry {
provider: None,
id: format!("steam:{}", sc.appid),
store: "steam".into(),
title: sc.name,
art: steam_art(sc.appid),
launch: Some(LaunchSpec {
kind: "steam_appid".into(),
value: shortcut_gameid(sc.appid).to_string(),
}),
})
}
/// Every `userdata/<id>/config/shortcuts.vdf` under each Steam root — one file per Steam account
/// that has signed in on this host.
fn shortcuts_files() -> Vec<PathBuf> {
let mut files = Vec::new();
for root in steam_roots() {
let Ok(users) = std::fs::read_dir(root.join("userdata")) else {
continue;
};
for user in users.flatten() {
let path = user.path().join("config").join("shortcuts.vdf");
if path.is_file() {
files.push(path);
}
}
}
files
}
/// The 64-bit game id `steam://rungameid/` needs to launch a non-Steam shortcut: high dword = the
/// 32-bit shortcut appid, low dword = the shortcut marker `0x0200_0000`. (Handing `rungameid` the
/// bare 32-bit appid does not launch a shortcut — it must be this composed id.)
fn shortcut_gameid(appid: u32) -> u64 {
((appid as u64) << 32) | 0x0200_0000
}
/// The 32-bit appid Steam derives for a shortcut from its target+name — `crc32(exe + name)` with the
/// high bit set. Only used when `shortcuts.vdf` omits the stored `appid` (very old Steam); modern
/// Steam writes it, and we prefer the stored value.
fn shortcut_appid(exe: &str, name: &str) -> u32 {
crc32(format!("{exe}{name}").as_bytes()) | 0x8000_0000
}
/// Standard reflected (IEEE) CRC-32 — a few short strings' worth per scan, so a table-free bitwise
/// loop is plenty. Matches what Steam uses to hash a shortcut's `exe + name`.
fn crc32(data: &[u8]) -> u32 {
let mut crc: u32 = 0xFFFF_FFFF;
for &byte in data {
crc ^= byte as u32;
for _ in 0..8 {
let mask = (crc & 1).wrapping_neg();
crc = (crc >> 1) ^ (0xEDB8_8320 & mask);
}
}
!crc
}
/// Parse a **binary** `shortcuts.vdf` into its shortcuts. The format is Steam's binary KeyValues: a
/// 1-byte type tag (`0x00` nested map, `0x01` string, `0x02` int32), a NUL-terminated key, then a
/// type-specific payload; `0x08` closes the current map. The whole file is one `shortcuts` map whose
/// children (keyed `"0"`, `"1"`, …) are the individual shortcuts. Lenient and panic-free: a
/// truncated file or an unrecognized tag stops the walk and returns whatever parsed so far.
fn parse_shortcuts(buf: &[u8]) -> Vec<Shortcut> {
let mut out = Vec::new();
let mut pos = 0usize;
// Enter the top-level map (`<0x00> "shortcuts" <NUL>`); tolerate any key name.
if buf.first() != Some(&0x00) {
return out;
}
pos += 1;
if read_cstr(buf, &mut pos).is_none() {
return out;
}
// Each child is a map describing one shortcut, until the map-closing `0x08`.
while let Some(&tag) = buf.get(pos) {
pos += 1;
if tag != 0x00 {
break; // `0x08` (end of shortcuts) or anything unexpected
}
if read_cstr(buf, &mut pos).is_none() {
break; // the index key ("0", "1", …)
}
match parse_one_shortcut(buf, &mut pos) {
Some(sc) => out.push(sc),
None => break,
}
}
out
}
/// Parse one shortcut's fields (positioned just after its index key) up to the map-closing `0x08`,
/// pulling the ones we surface. `None` on a truncated/garbled entry.
fn parse_one_shortcut(buf: &[u8], pos: &mut usize) -> Option<Shortcut> {
let mut appid: Option<u32> = None;
let mut name = String::new();
let mut exe = String::new();
let mut hidden = false;
loop {
let tag = *buf.get(*pos)?;
*pos += 1;
if tag == 0x08 {
break; // end of this shortcut
}
let key = read_cstr(buf, pos)?.to_ascii_lowercase();
match tag {
0x00 => skip_map(buf, pos)?, // nested map (e.g. `tags`) — not needed
0x01 => {
let val = read_cstr(buf, pos)?;
match key.as_str() {
"appname" => name = val,
"exe" => exe = val,
_ => {}
}
}
0x02 => {
let val = read_i32(buf, pos)?;
match key.as_str() {
"appid" => appid = Some(val as u32),
"ishidden" => hidden = val != 0,
_ => {}
}
}
0x07 => *pos += 8, // uint64 — skip
_ => return None, // unknown tag: payload size unknown, can't continue safely
}
}
if name.trim().is_empty() {
return None; // nothing worth showing
}
// Prefer the stored appid; fall back to Steam's derivation when it's absent (0 / missing).
let appid = appid
.filter(|a| *a != 0)
.unwrap_or_else(|| shortcut_appid(&exe, &name));
Some(Shortcut {
appid,
name,
hidden,
})
}
/// Skip a nested map's contents (positioned just after its key) up to and including its `0x08`.
fn skip_map(buf: &[u8], pos: &mut usize) -> Option<()> {
loop {
let tag = *buf.get(*pos)?;
*pos += 1;
if tag == 0x08 {
return Some(());
}
read_cstr(buf, pos)?; // key
match tag {
0x00 => skip_map(buf, pos)?,
0x01 => {
read_cstr(buf, pos)?;
}
0x02 => *pos += 4,
0x07 => *pos += 8,
_ => return None,
}
}
}
/// Read a NUL-terminated UTF-8 string (lossy) starting at `pos`, advancing past the terminator.
/// `None` if the buffer ends before a NUL.
fn read_cstr(buf: &[u8], pos: &mut usize) -> Option<String> {
let start = *pos;
let end = buf.get(start..)?.iter().position(|&b| b == 0)? + start;
let s = String::from_utf8_lossy(&buf[start..end]).into_owned();
*pos = end + 1;
Some(s)
}
/// Read a little-endian int32 at `pos`, advancing 4 bytes. `None` if fewer than 4 bytes remain.
fn read_i32(buf: &[u8], pos: &mut usize) -> Option<i32> {
let bytes: [u8; 4] = buf.get(*pos..*pos + 4)?.try_into().ok()?;
*pos += 4;
Some(i32::from_le_bytes(bytes))
}
#[cfg(test)]
mod tests {
use super::*;
@@ -338,4 +626,146 @@ mod tests {
let found = find_local_art_file(dir.path(), 570, ArtKind::Portrait).unwrap();
assert_eq!(found, cache.join("library_600x900_2x.jpg"));
}
// --- Non-Steam shortcuts (custom Steam entries) ---
/// Build one binary-VDF field for a test `shortcuts.vdf`.
fn field_str(key: &str, val: &str) -> Vec<u8> {
let mut v = vec![0x01u8];
v.extend_from_slice(key.as_bytes());
v.push(0);
v.extend_from_slice(val.as_bytes());
v.push(0);
v
}
fn field_i32(key: &str, val: i32) -> Vec<u8> {
let mut v = vec![0x02u8];
v.extend_from_slice(key.as_bytes());
v.push(0);
v.extend_from_slice(&val.to_le_bytes());
v
}
fn map_open(key: &str) -> Vec<u8> {
let mut v = vec![0x00u8];
v.extend_from_slice(key.as_bytes());
v.push(0);
v
}
#[test]
fn parse_shortcuts_reads_entries_honors_hidden_and_key_case() {
let mut buf = Vec::new();
buf.extend(map_open("shortcuts"));
// Entry 0: a normal shortcut, with a nested `tags` map to exercise skip_map, and mixed-case
// keys (Steam has shipped both `AppName` and `appname`). appid stored as a negative i32.
buf.extend(map_open("0"));
buf.extend(field_i32("appid", -1838178284)); // == 2456789012 as u32
buf.extend(field_str("AppName", "My Emulator"));
buf.extend(field_str("Exe", "\"/usr/bin/foo\""));
buf.extend(field_i32("IsHidden", 0));
buf.extend(map_open("tags"));
buf.extend(field_str("0", "emulator"));
buf.push(0x08); // end tags
buf.push(0x08); // end entry 0
// Entry 1: hidden, lowercase key variant.
buf.extend(map_open("1"));
buf.extend(field_str("appname", "Hidden Game"));
buf.extend(field_i32("ishidden", 1));
buf.push(0x08); // end entry 1
buf.push(0x08); // end shortcuts
let scs = parse_shortcuts(&buf);
assert_eq!(scs.len(), 2);
assert_eq!(scs[0].appid, 2_456_789_012);
assert_eq!(scs[0].name, "My Emulator");
assert!(!scs[0].hidden);
assert_eq!(scs[1].name, "Hidden Game");
assert!(scs[1].hidden);
// A hidden shortcut is dropped from the surfaced library; a visible one launches via its
// 64-bit game id (not the bare appid).
assert!(shortcut_entry(scs.into_iter().nth(1).unwrap()).is_none());
}
#[test]
fn parse_shortcuts_is_lenient() {
assert!(parse_shortcuts(b"").is_empty()); // not even a top-level map
assert!(parse_shortcuts(b"{not binary vdf}").is_empty());
// A truncated entry (buffer ends mid-int) yields what parsed cleanly before it — here, none.
let mut buf = Vec::new();
buf.extend(map_open("shortcuts"));
buf.extend(map_open("0"));
buf.extend_from_slice(b"\x02appid\x00\x01\x02"); // 2 of 4 int bytes, then EOF
assert!(parse_shortcuts(&buf).is_empty());
}
#[test]
fn shortcut_entry_launches_via_rungameid() {
let sc = Shortcut {
appid: 2_456_789_012,
name: "My Emulator".into(),
hidden: false,
};
let entry = shortcut_entry(sc).unwrap();
assert_eq!(entry.id, "steam:2456789012");
assert_eq!(entry.store, "steam");
let launch = entry.launch.unwrap();
assert_eq!(launch.kind, "steam_appid");
// Value is the 64-bit game id — digits only, so it passes the shared appid guard.
assert_eq!(launch.value, shortcut_gameid(2_456_789_012).to_string());
assert!(launch.value.bytes().all(|b| b.is_ascii_digit()));
}
#[test]
fn shortcut_gameid_composes_appid_and_marker() {
let id = shortcut_gameid(0x8000_0000);
assert_eq!(id >> 32, 0x8000_0000); // high dword is the appid
assert_eq!(id & 0xFFFF_FFFF, 0x0200_0000); // low dword is the shortcut marker
}
#[test]
fn crc32_matches_the_known_check_value_and_derives_a_high_bit_appid() {
assert_eq!(crc32(b"123456789"), 0xCBF4_3926); // IEEE CRC-32 check value
// A derived shortcut appid always has the high bit set (so it never collides with a real
// store appid, and its CDN art fetch is skipped).
assert_ne!(
shortcut_appid("\"/usr/bin/foo\"", "My Emulator") & 0x8000_0000,
0
);
}
#[test]
fn grid_filenames_follow_steams_naming() {
assert_eq!(
grid_filenames(ArtKind::Portrait, 42),
vec!["42p.png", "42p.jpg"]
);
assert_eq!(
grid_filenames(ArtKind::Hero, 42),
vec!["42_hero.png", "42_hero.jpg"]
);
assert_eq!(
grid_filenames(ArtKind::Logo, 42),
vec!["42_logo.png", "42_logo.jpg"]
);
assert_eq!(
grid_filenames(ArtKind::Header, 42),
vec!["42.png", "42.jpg"]
);
}
#[test]
fn find_grid_art_file_matches_the_userdata_grid_layout() {
let grid = tempfile::tempdir().unwrap();
std::fs::write(grid.path().join("2456789012p.jpg"), b"poster").unwrap();
let found = find_grid_art_file(grid.path(), 2_456_789_012, ArtKind::Portrait).unwrap();
assert_eq!(found, grid.path().join("2456789012p.jpg"));
// Missing kinds and a zero-byte file both miss cleanly.
assert_eq!(
find_grid_art_file(grid.path(), 2_456_789_012, ArtKind::Hero),
None
);
std::fs::write(grid.path().join("42p.png"), b"").unwrap();
assert_eq!(find_grid_art_file(grid.path(), 42, ArtKind::Portrait), None);
}
}
+5
View File
@@ -307,6 +307,11 @@ fn real_main() -> Result<()> {
// Standalone input-injection smoke test (no client needed): open the session's input
// backend and inject a scripted mouse/keyboard pattern. Watch a focused app / `wev`.
Some("input-test") => devtest::input_test(),
// Standalone stylus smoke test (no client needed): create the "Punktfunk Pen" virtual
// tablet and draw a pressure-ramped stroke through the real tracker→uinput chain.
// Watch in Krita/GIMP (pressure brush) or `libinput debug-events`.
#[cfg(target_os = "linux")]
Some("pen-test") => devtest::pen_test(),
// Zero-copy FFI/GPU probe: init the EGL importer + CUDA context (no capture needed).
#[cfg(target_os = "linux")]
Some("zerocopy-probe") => zerocopy::probe(),
+8
View File
@@ -1066,6 +1066,14 @@ async fn serve_session(
if rich_tx.send(ClientInput::Rich(rich)).is_err() {
break;
}
} else if let Some(pen) = punktfunk_core::quic::PenBatch::decode(&d) {
// 0xCC kind 0x05 — the stylus plane (RichInput::decode returns None for it by
// design; see punktfunk_core::quic::pen). Routed to the same input thread,
// which owns the per-session tracker + virtual tablet.
rich_count += 1;
if rich_tx.send(ClientInput::Pen(pen)).is_err() {
break;
}
} else if let Some(mut ev) = InputEvent::decode(&d) {
input_count += 1;
// Wire hygiene: KEY_FLAG_SEMANTIC_VK is an in-process tag (GameStream ingest
+72 -31
View File
@@ -3,7 +3,7 @@
//! the live platform) to a backend this host can actually build, applying the runtime UHID /
//! Steam-conflict degrades. Pure selection ([`pick_gamepad`]) is separated from the env/logging
//! shell ([`resolve_gamepad`]) and the per-pad variant ([`resolve_pad_kind`]); the platform degrades
//! (`degrade_if_no_uhid`, `physical_steam_controller_present`, `degrade_steam_on_conflict`) are
//! (`degrade_if_no_uhid`, `physical_steam_product_present`, `degrade_steam_on_conflict`) are
//! cfg-split linux/other and MUST be re-verified on Windows when touched (Linux clippy can't see the
//! non-linux copies).
@@ -128,31 +128,53 @@ fn degrade_if_no_uhid(chosen: GamepadPref) -> GamepadPref {
chosen
}
/// True if a **physical** Valve Steam controller (`28DE`) is already attached. The host's own Steam
/// Input is then managing a `28DE` device, and presenting a second (virtual) one makes Steam juggle
/// two Decks — confirmed conflict-prone on a Deck-as-host (the physical `28DE:1205` + Steam's
/// `28DE:11FF` XInput output pad are both live). HID device dirs are named `BUS:VID:PID.INST`
/// (uppercase); a UHID virtual device resolves through `/devices/virtual/…`, a real one does not.
///
/// Punktfunk's OWN virtual Decks must never count: the usbip/gadget transports present a real USB
/// device (vhci resolves through `vhci_hcd`, NOT `/devices/virtual/`), so a just-ended session's
/// pad still detaching — or a concurrent session's live one — read as "physical" and degraded
/// every back-to-back Deck session to DualSense (observed live on Bazzite 2026-07-04). Ours are
/// recognizable by the `FVPF…` serial ([`steam_proto::deck_serial`]) in `HID_UNIQ`, with the
/// vhci path as belt and braces.
/// The Valve product id (`28DE:xxxx`) a virtual Steam backend enumerates as, or `None` for a
/// non-Steam backend. This is the identity the conflict gate compares against the *physical* Valve
/// devices attached to the host: only a genuine duplicate (same VID **and** PID) confuses Steam
/// Input, so the gate keys on the PID, not on the `28DE` vendor alone.
#[cfg(target_os = "linux")]
fn physical_steam_controller_present() -> bool {
fn steam_backend_product(pref: GamepadPref) -> Option<u16> {
match pref {
GamepadPref::SteamDeck => Some(0x1205), // built-in Deck controller identity
GamepadPref::SteamController => Some(0x1102), // classic wired Steam Controller
GamepadPref::SteamController2 => Some(0x1302), // Steam Controller 2 (Triton), wired
GamepadPref::SteamController2Puck => Some(0x1304), // Steam Controller 2 via its Puck dongle
_ => None,
}
}
/// True if a **physical** Valve device with this exact product id (`28DE:{product}`) is already
/// attached. The host's own Steam Input is then managing that identity, and presenting a *second,
/// identical* virtual one makes Steam juggle two copies of the same Deck — confirmed conflict-prone
/// on a Deck-as-host (the physical `28DE:1205` + Steam's `28DE:11FF` XInput output pad are both
/// live).
///
/// Crucially this keys on the PID, not just the `28DE` vendor: a *different* Valve product is not a
/// conflict. A physical Steam Controller 2 (`28DE:1302`) must NOT block a virtual Steam Deck
/// (`28DE:1205`) — Steam Input drives distinct Steam controllers side by side, so the wanted pad
/// has to pass through unharmed (this over-broad vendor match degraded such sessions to DualSense).
///
/// HID device dirs are named `BUS:VID:PID.INST` (uppercase); a UHID virtual device resolves through
/// `/devices/virtual/…`, a real one does not. Punktfunk's OWN virtual pads must never count: the
/// usbip/gadget transports present a real USB device (vhci resolves through `vhci_hcd`, NOT
/// `/devices/virtual/`), so a just-ended session's pad still detaching — or a concurrent session's
/// live one — read as "physical" and degraded every back-to-back Deck session to DualSense
/// (observed live on Bazzite 2026-07-04). Ours are recognizable by the `FVPF…` serial
/// ([`steam_proto::deck_serial`]) in `HID_UNIQ`, with the vhci path as belt and braces.
#[cfg(target_os = "linux")]
fn physical_steam_product_present(product: u16) -> bool {
let needle = format!(":28DE:{product:04X}");
let Ok(entries) = std::fs::read_dir("/sys/bus/hid/devices") else {
return false;
};
entries.flatten().any(|e| {
if !e.file_name().to_string_lossy().contains(":28DE:") {
if !e.file_name().to_string_lossy().contains(&needle) {
return false;
}
if std::fs::read_to_string(e.path().join("uevent"))
.is_ok_and(|u| u.lines().any(|l| l.starts_with("HID_UNIQ=FVPF")))
{
return false; // one of our own virtual Decks
return false; // one of our own virtual pads
}
match std::fs::read_link(e.path()) {
Ok(target) => {
@@ -164,28 +186,30 @@ fn physical_steam_controller_present() -> bool {
})
}
/// Gate a virtual Steam pad off when a physical Steam controller is attached (§ conflict). Degrade to
/// DualSense (then the uhid ladder), which Steam treats as an ordinary, distinct pad. Override with
/// `PUNKTFUNK_STEAM_FORCE=1` when the host has no competing Steam Input (e.g. a remote-only box).
/// Gate a virtual Steam pad off when a physical Steam controller *of the same identity* is attached
/// (§ conflict). Degrade to DualSense (then the uhid ladder), which Steam treats as an ordinary,
/// distinct pad. Override with `PUNKTFUNK_STEAM_FORCE=1` when the host has no competing Steam Input
/// (e.g. a remote-only box).
///
/// Only a same-PID duplicate degrades: a physical Steam Controller 2 no longer blocks a virtual
/// Steam Deck (and vice versa), so a Steam Machine with an SC2 plugged in streams the pad the client
/// actually asked for.
#[cfg(target_os = "linux")]
fn degrade_steam_on_conflict(chosen: GamepadPref) -> GamepadPref {
if !matches!(
chosen,
GamepadPref::SteamDeck
| GamepadPref::SteamController
| GamepadPref::SteamController2
| GamepadPref::SteamController2Puck
) {
return chosen;
}
let Some(product) = steam_backend_product(chosen) else {
return chosen; // not a virtual Steam (28DE) pad — nothing to gate
};
let forced = std::env::var("PUNKTFUNK_STEAM_FORCE")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(false);
if !forced && physical_steam_controller_present() {
if !forced && physical_steam_product_present(product) {
let conflict = format!("28DE:{product:04X}");
tracing::warn!(
wanted = chosen.as_str(),
"a physical Steam controller is attached — the host's Steam Input would manage two 28DE \
devices; falling back to DualSense (set PUNKTFUNK_STEAM_FORCE=1 to override)"
conflict = conflict.as_str(),
"a physical Steam controller of the same identity is attached — the host's Steam Input \
would manage two identical 28DE devices; falling back to DualSense (set \
PUNKTFUNK_STEAM_FORCE=1 to override)"
);
return degrade_if_no_uhid(GamepadPref::DualSense);
}
@@ -383,4 +407,21 @@ mod tests {
Xbox360
);
}
// The conflict gate keys on the exact Valve PID, so a physical Steam Controller 2 (`28DE:1302`)
// never blocks a virtual Steam Deck (`28DE:1205`) — only a same-identity duplicate degrades.
#[cfg(target_os = "linux")]
#[test]
fn steam_backend_product_ids() {
use super::steam_backend_product;
use GamepadPref::*;
assert_eq!(steam_backend_product(SteamDeck), Some(0x1205));
assert_eq!(steam_backend_product(SteamController), Some(0x1102));
assert_eq!(steam_backend_product(SteamController2), Some(0x1302));
assert_eq!(steam_backend_product(SteamController2Puck), Some(0x1304));
// Non-Steam backends carry no Valve identity, so the gate never fires for them.
assert_eq!(steam_backend_product(DualSense), None);
assert_eq!(steam_backend_product(Xbox360), None);
assert_eq!(steam_backend_product(SwitchPro), None);
}
}
@@ -494,6 +494,15 @@ pub(super) async fn negotiate(
punktfunk_core::quic::HOST_CAP_CURSOR
} else {
0
}
// Full-fidelity stylus (0xCC/0x05 pen batches → the per-session uinput tablet):
// Linux with /dev/uinput access, minus the PUNKTFUNK_PEN=0 kill-switch. Clients
// without the bit keep folding pen into touch/pointer (and NativeClient::send_pen
// refuses toward us if we don't set it).
| if crate::inject::pen_supported() {
punktfunk_core::quic::HOST_CAP_PEN
} else {
0
},
// The negotiated session AEAD (resolved above) + its 32-byte key toward a ChaCha
// client; toward everyone else cipher 0 keeps the Welcome byte-identical to the
+100 -1
View File
@@ -523,6 +523,87 @@ pub(super) enum ClientInput {
Event(InputEvent),
/// The 0xCC plane: touchpad contacts + motion samples.
Rich(punktfunk_core::quic::RichInput),
/// The 0xCC/0x05 stylus plane: state-full pen sample batches, diffed into a per-session
/// virtual tablet (design/pen-tablet-input.md).
Pen(punktfunk_core::quic::PenBatch),
}
/// The per-session stylus lane: the core [`PenTracker`](punktfunk_core::quic::PenTracker)
/// diffs state-full batches into transitions, applied to a lazily-created uinput tablet
/// ([`crate::inject::pen::VirtualPen`]) — a session that never draws never creates a device,
/// and the device dies with the session (kernel removes the tablet, apps see the pen unplug).
struct PenSession {
tracker: punktfunk_core::quic::PenTracker,
dev: Option<crate::inject::pen::VirtualPen>,
/// Creation failed once — don't retry per batch (240 Hz of ioctl churn + log spam); the
/// tracker still consumes batches so its state stays coherent.
create_failed: bool,
last_rx: std::time::Instant,
/// Reused transition buffer (a batch yields at most a few).
out: Vec<punktfunk_core::quic::PenTransition>,
}
impl PenSession {
fn new() -> PenSession {
PenSession {
tracker: punktfunk_core::quic::PenTracker::default(),
dev: None,
create_failed: false,
last_rx: std::time::Instant::now(),
out: Vec::new(),
}
}
fn apply(&mut self, batch: &punktfunk_core::quic::PenBatch) {
self.last_rx = std::time::Instant::now();
if self.dev.is_none() && !self.create_failed {
match crate::inject::pen::VirtualPen::create() {
Ok(d) => self.dev = Some(d),
Err(e) => {
// Shouldn't happen when the Welcome advertised HOST_CAP_PEN off the same
// uinput probe — but permissions can change between then and first ink.
self.create_failed = true;
tracing::warn!(
error = %format!("{e:#}"),
"pen: virtual tablet creation failed — dropping pen input this session"
);
}
}
}
self.out.clear();
self.tracker.apply(batch, &mut self.out);
if let Some(dev) = self.dev.as_mut() {
dev.apply_batch(&self.out);
}
}
/// The dead-client failsafe ([`PEN_TOUCH_TIMEOUT_MS`](punktfunk_core::quic::PEN_TOUCH_TIMEOUT_MS)):
/// clients repeat the last sample (≤100 ms) while the pen is in range — a stationary
/// touching pen included — so silence past the timeout means the client is gone, and the
/// host must not leave the stroke inked-down. Called every input-loop wake; the loop caps
/// its recv timeout at 100 ms while the pen is active so this actually gets to run.
fn check_timeout(&mut self) {
if self.tracker.is_active()
&& self.last_rx.elapsed().as_millis()
>= punktfunk_core::quic::PEN_TOUCH_TIMEOUT_MS as u128
{
tracing::debug!("pen: sample stream went silent — force-releasing the stroke");
self.release_all();
}
}
/// Lift buttons/tip/proximity in order (a no-op when idle). Session end + the timeout.
fn release_all(&mut self) {
self.out.clear();
self.tracker.force_release(&mut self.out);
if let Some(dev) = self.dev.as_mut() {
dev.apply_batch(&self.out);
}
}
fn active(&self) -> bool {
self.tracker.is_active()
}
}
/// Default TTL stamped on a non-zero rumble envelope (0xCA v2): how long the client renders the
@@ -640,8 +721,17 @@ pub(super) fn input_thread(
const MAX_HELD: usize = 256;
let mut held_buttons: std::collections::HashSet<u32> = std::collections::HashSet::new();
let mut held_keys: std::collections::HashSet<u32> = std::collections::HashSet::new();
let mut pen = PenSession::new();
loop {
match rx.recv_timeout(pads.feedback_poll_interval()) {
// While a pen is in range/touching, wake at least every 100 ms so the stroke failsafe
// (PenSession::check_timeout) has real granularity against its 200 ms deadline.
let poll = if pen.active() {
pads.feedback_poll_interval()
.min(std::time::Duration::from_millis(100))
} else {
pads.feedback_poll_interval()
};
match rx.recv_timeout(poll) {
// Rich input (touchpad / motion) is applied the moment it arrives; the single channel
// wakes for gyro samples instead of making them wait out the feedback poll interval.
Ok(ClientInput::Rich(rich)) => {
@@ -673,6 +763,9 @@ pub(super) fn input_thread(
}
pads.apply_rich(rich);
}
// Stylus batches apply on arrival like rich input — the tracker synthesizes the
// transitions, the lazily-created virtual tablet renders them.
Ok(ClientInput::Pen(batch)) => pen.apply(&batch),
Ok(ClientInput::Event(ev)) => match ev.kind {
InputKind::GamepadButton | InputKind::GamepadAxis => {
// A bad index / unknown axis just doesn't update a pad — fall through (no
@@ -774,6 +867,9 @@ pub(super) fn input_thread(
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {}
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => break,
}
// Pen stroke failsafe: a silent sample stream past the deadline force-releases (see
// PenSession::check_timeout — clients heartbeat while the pen is down).
pen.check_timeout();
// Service feedback every iteration (≤1 ms for Triton, ≤4 ms otherwise; games block on
// EVIOCSFF, and HID handshakes must be answered promptly). Rumble → the universal 0xCA
// plane; rich/raw HID feedback → 0xCD.
@@ -863,6 +959,9 @@ pub(super) fn input_thread(
}
}
}
// Pen: lift anything still inked (buttons → tip → proximity), then the device dies with
// this thread (VirtualPen::drop → UI_DEV_DESTROY; apps see the tablet unplug).
pen.release_all();
// Session ended (client gone). Release anything still held through the host-lifetime injector —
// its EIS connection (and any implicit grab Mutter holds for our pressed button) outlives this
// session, so without this a button pressed at disconnect stays latched and breaks clicks for
+36
View File
@@ -437,6 +437,42 @@ punktfunk_connection_send_input(c, &ev);
- `punktfunk_connection_send_rich_input2(c, &richEx)` — the forward-compatible superset (Steam
trackpads, signed coords, pressure); set `struct_size = sizeof(PunktfunkRichInputEx)`.
### Stylus / pen
Full-fidelity pen (pressure, tilt, azimuth, barrel roll, hover, eraser, barrel buttons) has its
own plane. **Gate on the capability first** — only send when
`punktfunk_connection_host_caps(c, &caps)` shows `PUNKTFUNK_HOST_CAP_PEN`; without it the call
returns `UNSUPPORTED` and you keep your pen-as-touch fallback (what every client does today).
Samples are **state-full**: every sample carries the complete pen state (in-range / touching /
buttons in `state`, plus all axes — unknown axes take their `PUNKTFUNK_PEN_*_UNKNOWN` sentinel,
never 0). There are no down/up events to send; the host diffs consecutive samples and
synthesizes the transitions, so a lost datagram self-heals on the next one. Batch a capture
callback's coalesced samples (oldest first, `dt_us` = spacing) into one call, at most
`PUNKTFUNK_PEN_BATCH_MAX` per call — split longer runs into consecutive calls.
```c
// Example: an in-contact Apple Pencil sample, mapped into video-frame space
PunktfunkPenSample s; memset(&s, 0, sizeof s);
s.state = PUNKTFUNK_PEN_IN_RANGE | PUNKTFUNK_PEN_TOUCHING;
s.tool = PUNKTFUNK_PEN_TOOL_PEN;
s.x = 0.5f; s.y = 0.5f; // normalized 0..1 across the video frame
s.pressure = (uint16_t)(force_norm * 65535); // 0 while hovering
s.tilt_deg = 90 - altitude_deg; // polar tilt-from-normal
s.azimuth_deg = azimuth_deg; // 0..359, clockwise from north
s.roll_deg = PUNKTFUNK_PEN_ANGLE_UNKNOWN; // Pencil Pro: rollAngle in degrees
s.distance = PUNKTFUNK_PEN_DISTANCE_UNKNOWN;
punktfunk_connection_send_pen(c, &s, 1);
```
While hovering, send `PUNKTFUNK_PEN_IN_RANGE` samples (with `distance` if you have it); when the
pen leaves range, send one final sample with `state = 0` so the host releases proximity.
**Heartbeat**: while the pen is in range or touching, repeat the last sample at least every
~100 ms even when nothing changed — pen capture APIs are silent for a stationary pen, and the
host force-releases the stroke after 200 ms without samples (its dead-client failsafe). Run a
timer alongside your touch callbacks.
---
## 9. Feedback planes (rumble, HID, HDR)
+45
View File
@@ -0,0 +1,45 @@
# Release notes
One file per **stable** release: `docs/releases/vX.Y.Z.md`. Its contents become the Gitea release
body **verbatim** and are the source of the Discord `#releases` announcement.
## Why this exists
Releases used to be created by CI with an **empty body**; the notes were pasted in by hand
afterward. That left a window where the release — and anything announcing it — carried no notes.
Now the notes are authored **before the tag is pushed**, as part of the version bump, so the
release is born complete and the announcement always has something to say.
## The flow
1. **Write the notes.** Add `docs/releases/vX.Y.Z.md` in the same commit (or PR) as the version
bump. Copy `TEMPLATE.md` and fill it in. This file is the single source of truth for the body.
2. **Tag & push.** `git tag -a vX.Y.Z … && git push origin vX.Y.Z` fans out to the build
workflows. Whichever one wins the create race seeds the release body from this file
(`scripts/ci/gitea-release.sh``ensure_release`, and its PowerShell twin). The release page
shows the notes immediately.
3. **Wait for green.** Let every platform's CI finish and go green.
4. **Announce.** Dispatch the `announce` workflow (`.gitea/workflows/announce.yml`) with the tag.
It re-asserts this file over the live release (so any late edit wins) and posts an embed to
Discord `#releases`. Pressing "go" is the quality gate — a half-built release is never
announced. Stable-only; a `-rc` tag is refused unless `allow_prerelease=true`.
Editing the notes after the tag is fine: update this file, then re-run step 4 (or PATCH the body
via the API) — the announce step always re-syncs from the file, so the file stays authoritative
even across a tag re-point.
Canary / `-rc` builds have **no** file here on purpose: they get no curated body and are not
announced.
## Format
Match the house style (see any recent `vX.Y.Z.md`):
- Open with a **wire-compatibility** line — *"Wire-compatible with X.Y.x — existing pairings and
clients keep working."* — plus a one-sentence fallback/negotiation note. This lead-in (all text
before the first `##` header) is what the Discord embed shows, so make it a real summary.
- Then `## Section` headers grouping the changes, with **bold lead-in** bullets.
- Be concrete: env vars, ABI/protocol versions, on-glass-verified hardware, platform scope.
The short annotated-**tag** message stays separate and short (a headline + a paragraph); it is the
tag object's message, not this file.
+20
View File
@@ -0,0 +1,20 @@
Wire-compatible with X.Y.x — existing pairings and clients keep working. <one sentence on how
older/newer clients negotiate or fall back, so nobody fears updating.>
<Optional: one or two sentences naming the headline change of this release — this whole lead-in
(everything above the first `##`) is what the Discord #releases embed shows, so make it read as a
standalone summary.>
## Highlights
- **<Lead-in>.** <What changed and why it matters, concretely.>
- **<Lead-in>.** <…>
## Fixes
- **<Lead-in>.** <…>
## Platform notes
- **<Platform>.** <Anything platform-specific: env vars, ABI/protocol versions, hardware that was
verified on-glass.>
+47
View File
@@ -0,0 +1,47 @@
Wire-compatible with 0.18.x — existing pairings and clients keep working. The new stylus plane is capability-negotiated (`HOST_CAP_PEN`) and rides an additive datagram that older peers never send and never have to parse, so a 0.18 host and a 0.19 client (or the reverse) pair exactly as before and simply fall back to pen-as-touch. WIRE_VERSION stays **2**; the embeddable C ABI moves to **13** for one new entry point (`punktfunk_connection_send_pen`); the Windows display-driver protocol is unchanged at **6** (compat floor **3**).
## Pen, stylus & tablet — a whole new input plane
Punktfunk streams now carry a first-class, pressure-sensitive stylus, end to end and on every platform. It is its own wire plane — not mouse events in disguise — so pressure, tilt, hover and barrel buttons survive the trip from glass to host.
- **The wire (P0).** A state-full `RICH_PEN` datagram carries batches of up to 8 samples — sub-pixel `f32` position, `u16` pressure, hover distance, tilt + azimuth, an eraser tool, and two barrel buttons — ordered by a wrapping sequence number and dropped whole when stale, so a late batch never rewrites the present. A `PenTracker` coalesces the stream and `HOST_CAP_PEN` advertises the plane; toward a host without the bit the client keeps its pen-as-touch fallback.
- **Linux host (P1).** A per-session uinput virtual graphics tablet injects the full stylus — pressure and tilt included — and tears down with the session. `HOST_CAP_PEN` goes live wherever uinput is available.
- **GameStream / Moonlight (P2).** The host ingests Moonlight's `SS_PEN`/`SS_TOUCH` pointer packets and advertises the matching `featureFlags`, so a capable Moonlight client sends real stylus onto the same plane.
- **Windows host (P3).** `PT_PEN` + `PT_TOUCH` synthetic pointer injection drives the Windows pen stack directly — pressure and barrel buttons reach applications as genuine pen input.
- **iPad (P4).** Apple Pencil (including Pencil Pro) capture maps onto the stylus plane — pressure and tilt from the panel, with the cursor-capability fix so the pencil and the pointer coexist.
- **Android (P5).** Active-stylus capture (S Pen and friends) forwards onto the pen plane with pressure and tilt.
## Touch injection, fixed
- **Windows native touch actually injects now.** `PT_TOUCH` pointer ids must be contiguous injector slots, but the host was passing the raw wire touch ids straight through — Windows rejected them **silently** and Moonlight-native multitouch landed nothing. Wire ids are now compacted into slots, and the first rejection is surfaced with a warning instead of vanishing.
- **NaN pressure no longer drops a finger.** VoidLink's finger touches arrive with a NaN `pressureOrDistance`; the GameStream path discarded the whole packet. It now tolerates NaN and injects the touch.
## Cursor & pointer polish
- **The forwarded cursor is scaled to the video fit** on the SDL/Linux and Apple/macOS clients. A high-DPI host pointer was drawn against the client's own backing scale and came out roughly 2× too large; it now renders true-size against the streamed video.
## Windows host fixes
- **No more `0x57` on display-config isolate.** When a supplied CCD path is doomed (a Steam Deck's live-deactivate always is), the isolate escalates to a keep-only supplied config instead of failing the whole apply with `0x57`.
- **Cursor exclusion is reported adapter-wide.** The IddCx declare's cursor exclusion is not a per-target property; the virtual-display driver now reports `cursor_excluded` across the adapter.
- **Secure-desktop cursor stood down.** The IddCx hardware cursor is held down while the Windows secure desktop (UAC / Winlogon) is up, ending the duplicated/stuck pointer there.
## Gamepad
- **A virtual Steam Deck pad no longer degrades to a DualSense because of an unrelated Steam controller.** To stop Steam Input from double-driving two *identical* controllers, the host downgrades a requested virtual Steam pad to a DualSense when a matching physical Valve controller is present — but the gate matched *any* `28DE` (Valve) device. So plugging a physical Steam Controller 2 (`28DE:1302`) into the host dropped a client's virtual Steam Deck (`28DE:1205`) to the wrong pad — the passthrough came up as a DualSense. The gate now keys on the exact VID+PID: distinct Steam controllers coexist (Steam Input drives them side by side fine), and only a genuine same-identity duplicate — a physical Deck alongside a virtual Deck — still degrades.
## Android input — mouse & keyboard regressions
Two regressions from 0.18.0's mouse-&-keyboard overhaul, felt most on Android TV boxes:
- **Mouse back/forward stays in the stream.** A mouse's back/forward buttons were synthesized by the reader as `SOURCE_MOUSE` key events and leaked into Android navigation (yanking you out of the stream on a TV); they're now swallowed while streaming.
- **Hardware typing no longer pops the IME.** The soft keyboard is gated on `imeShown`, so typing on a physical keyboard against a text-input host doesn't summon the on-screen IME.
## Release & CI
- **Release notes now ship with the release.** Notes are authored in-repo at `docs/releases/vX.Y.Z.md` and seeded into the release body at creation, and stable releases announce to the Discord `#releases` channel. This file is the first of them.
- **iOS `.ipa` attached.** The iOS build exports an App Store-signed `.ipa` to the unified release and to the run artifacts (archival/TestFlight, not a direct sideload).
## Versions
WIRE_VERSION **2** (unchanged — 0.18.x hosts and clients interoperate) · C ABI **13** (adds `punktfunk_connection_send_pen`; embedders recompile) · Windows display driver protocol **6**, compat floor **3** (unchanged — pen injects through the Windows pen stack, not the display driver). Windows drivers ship separately, as always.
+180 -4
View File
@@ -49,7 +49,10 @@
// (design/remote-desktop-sweep.md §8): the client's mouse-model chord tells the host who
// renders the pointer. Additive; rides the existing control stream (a new message TYPE, which
// pre-§8 hosts ignore), so [`WIRE_VERSION`] is unchanged.
#define ABI_VERSION 12
// v13: added `punktfunk_connection_send_pen` — the stylus wire plane
// (design/pen-tablet-input.md): a client sends `RICH_PEN` sample batches once the host
// advertises `HOST_CAP_PEN`. Additive and capability-gated, so [`WIRE_VERSION`] is unchanged.
#define ABI_VERSION 13
// The punktfunk/1 **wire** version — what `Hello`/`Welcome` carry and hosts equality-check.
// Deliberately its own constant: [`ABI_VERSION`] tracks the embeddable **C surface**
@@ -88,6 +91,37 @@
// `punktfunk_connection_send_rich_input2` (added with client capture).
#define PUNKTFUNK_RICH_TOUCHPAD_EX 3
// [`PunktfunkPenSample::state`] bit: the pen hovers in range (implied by `TOUCHING`).
#define PUNKTFUNK_PEN_IN_RANGE 1
// [`PunktfunkPenSample::state`] bit: the tip is in contact.
#define PUNKTFUNK_PEN_TOUCHING 2
// [`PunktfunkPenSample::state`] bit: primary barrel button (or squeeze mapping) held.
#define PUNKTFUNK_PEN_BARREL1 4
// [`PunktfunkPenSample::state`] bit: secondary barrel button (or double-tap mapping) held.
#define PUNKTFUNK_PEN_BARREL2 8
// [`PunktfunkPenSample::tool`]: the pen tip.
#define PUNKTFUNK_PEN_TOOL_PEN 0
// [`PunktfunkPenSample::tool`]: the eraser (a client-side mode — Apple Pencil has no
// hardware eraser end; the squeeze/double-tap mapping usually drives this).
#define PUNKTFUNK_PEN_TOOL_ERASER 1
// Most samples one [`punktfunk_connection_send_pen`] call accepts (one wire batch).
#define PUNKTFUNK_PEN_BATCH_MAX 8
// [`PunktfunkPenSample::tilt_deg`] sentinel: no tilt reading.
#define PUNKTFUNK_PEN_TILT_UNKNOWN 255
// [`PunktfunkPenSample::azimuth_deg`] / `roll_deg` sentinel: no reading.
#define PUNKTFUNK_PEN_ANGLE_UNKNOWN 65535
// [`PunktfunkPenSample::distance`] sentinel: no hover-distance reading.
#define PUNKTFUNK_PEN_DISTANCE_UNKNOWN 65535
// Compositor preference for [`punktfunk_connect_ex`] (`compositor` arg). `AUTO` lets the host
// pick (auto-detect from its running desktop); a concrete value is honored only if that backend
// is available on the host right now, else the host falls back to auto-detect. The resolved
@@ -219,6 +253,13 @@
// clipboard, so a client may offer the toggle. (Mirrors `quic::HOST_CAP_CLIPBOARD`.)
#define PUNKTFUNK_HOST_CAP_CLIPBOARD 2
// Host-capability bit in [`punktfunk_connection_host_caps`]: the host injects full-fidelity
// stylus input, so a capable client splits pen contacts out of its touch path and sends them
// via [`punktfunk_connection_send_pen`]; without the bit that call returns `Unsupported` and
// the client keeps its pen-as-touch fallback. (Mirrors `quic::HOST_CAP_PEN`;
// design/pen-tablet-input.md.)
#define PUNKTFUNK_HOST_CAP_PEN 16
// [`punktfunk_connect_ex9`] `client_caps` bit: render the host cursor locally (the cursor
// channel, `design/remote-desktop-sweep.md` M2).
#define PUNKTFUNK_CLIENT_CAP_CURSOR 1
@@ -541,6 +582,20 @@
#define HOST_CAP_CURSOR 8
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host injects full-fidelity stylus input — it routes
// [`PenBatch`](super::pen::PenBatch) `0xCC/0x05` datagrams (pressure, tilt, azimuth, barrel
// roll, hover, eraser, barrel buttons) through the [`PenTracker`](super::pen::PenTracker)
// into a virtual tablet device (design/pen-tablet-input.md). A capable client (Apple Pencil,
// Android stylus) then splits pen contacts out of its finger/touch path and sends pen
// batches; absent the bit it keeps folding the pen into touch/pointer like today, and
// [`NativeClient::send_pen`](crate::client::NativeClient::send_pen) refuses to send. The
// wire ships ahead of the backend (P0): no host sets this bit until the P1 injector lands —
// which is exactly why the gate exists. `0x10` — `0x08` is [`HOST_CAP_CURSOR`], `0x04` is
// [`HOST_CAP_TEXT_INPUT`], `0x01`/`0x02` are gamepad-state / clipboard.
#define HOST_CAP_PEN 16
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
@@ -985,6 +1040,73 @@
#define MSG_PAIR_RESULT 19
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::state`] bit: the pen is in the hover range of the surface. Implied by
// [`PEN_TOUCHING`] (decode normalizes, so a client that only sets TOUCHING still produces a
// coherent contact).
#define PEN_IN_RANGE 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::state`] bit: the tip is in contact with the surface.
#define PEN_TOUCHING 2
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::state`] bit: the primary barrel button (or the client's squeeze mapping) is held.
#define PEN_BARREL1 4
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::state`] bit: the secondary barrel button (or the client's double-tap mapping)
// is held.
#define PEN_BARREL2 8
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::state`] bit, RESERVED: a predicted (not yet observed) sample. Never sent v1;
// receivers MUST ignore samples carrying it until a capability negotiates otherwise
// (design/pen-tablet-input.md §8).
#define PEN_PREDICTED 128
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::tilt_deg`] sentinel: the client has no tilt sensor / no reading.
#define PEN_TILT_UNKNOWN 255
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::azimuth_deg`] / [`PenSample::roll_deg`] sentinel: no reading.
#define PEN_ANGLE_UNKNOWN 65535
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::distance`] sentinel: no hover-distance reading.
#define PEN_DISTANCE_UNKNOWN 65535
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Most samples one [`PenBatch`] can carry. Sized for coalesced capture at video-frame cadence
// (240 Hz pen ÷ 30 fps = 8); a client producing more splits into consecutive batches.
#define PEN_BATCH_MAX 8
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Wire length of one encoded [`PenSample`].
#define PEN_SAMPLE_WIRE_LEN 21
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Host-side failsafe (design/pen-tablet-input.md §2): a tracker still in range after this
// many ms without a sample force-releases ([`PenTracker::force_release`]) — a client that
// died mid-stroke must not leave the host's virtual pen inked-down forever. This makes the
// **client heartbeat a wire contract**: capture APIs only fire on change, so a stationary
// pen is naturally silent — senders MUST repeat the last sample at least every ~100 ms while
// the pen is in range or touching (it re-decodes as pure Motion, harmless), keeping a live
// stationary stroke two heartbeats clear of the deadline.
#define PEN_TOUCH_TIMEOUT_MS 200
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Stream-kind byte: a clipboard fetch (request/response of one format). Future stream kinds
// (e.g. a bulk file-content push) mux under the same [`STREAM_MAGIC`] with a different byte.
@@ -1489,6 +1611,41 @@ typedef struct {
} PunktfunkRichInputEx;
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// One complete stylus state at one instant ([`punktfunk_connection_send_pen`];
// design/pen-tablet-input.md). STATE-FULL, never an edge event: fill every field on every
// sample (unknown axes take their `*_UNKNOWN` sentinel) — the host diffs consecutive samples
// and synthesizes down/up/button transitions itself, which is what makes a lost datagram
// self-heal. `x`/`y` are normalized `0.0..=1.0` in VIDEO-FRAME space (map your letterbox
// before filling, exactly like wire touches).
typedef struct {
// Normalized `0.0..=1.0` across the video frame. Must be finite.
float x;
// Normalized `0.0..=1.0` across the video frame. Must be finite.
float y;
// Tip force, `0..=65535` full scale (`0` while hovering).
uint16_t pressure;
// Hover distance `0..=65534` (0 = at the hover floor), or `PUNKTFUNK_PEN_DISTANCE_UNKNOWN`.
uint16_t distance;
// Tilt azimuth, degrees `0..=359` clockwise from north, or `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
uint16_t azimuth_deg;
// Barrel roll (Apple Pencil Pro `rollAngle`), degrees `0..=359`, or
// `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
uint16_t roll_deg;
// µs since the previous sample in the same call (`0` for the first) — the coalesced
// capture spacing.
uint16_t dt_us;
// Bitfield of `PUNKTFUNK_PEN_*` state bits. Unknown bits are rejected (`InvalidArg`).
uint8_t state;
// `PUNKTFUNK_PEN_TOOL_PEN` or `PUNKTFUNK_PEN_TOOL_ERASER`.
uint8_t tool;
// Tilt from the surface normal, degrees `0..=90`, or `PUNKTFUNK_PEN_TILT_UNKNOWN`.
uint8_t tilt_deg;
// Set to 0.
uint8_t _reserved[3];
} PunktfunkPenSample;
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// One advertised clipboard format passed to [`punktfunk_connection_clipboard_offer`].
typedef struct {
@@ -2310,6 +2467,24 @@ PunktfunkStatus punktfunk_connection_send_rich_input2(PunktfunkConnection *c,
const PunktfunkRichInputEx *rich);
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Send one stylus sample batch — `count` (`1..=PUNKTFUNK_PEN_BATCH_MAX`) state-full
// [`PunktfunkPenSample`]s, oldest first (a capture callback's coalesced samples) — as one
// `0xCC/0x05` pen datagram (non-blocking enqueue; design/pen-tablet-input.md). Split longer
// runs into consecutive calls. Gate on `punktfunk_connection_host_caps() &
// PUNKTFUNK_HOST_CAP_PEN`: toward a host without the bit this returns
// [`PunktfunkStatus::Unsupported`] — keep the pen-as-touch fallback there.
// [`PunktfunkStatus::InvalidArg`] on a bad count or a bad sample (non-finite coordinate,
// unknown state bit / tool).
//
// # Safety
// `c` is a valid connection handle; `samples` is null or points to `count` valid
// [`PunktfunkPenSample`]s.
PunktfunkStatus punktfunk_connection_send_pen(PunktfunkConnection *c,
const PunktfunkPenSample *samples,
uint32_t count);
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// The currently active session mode — the Welcome's, until an accepted
// [`punktfunk_connection_request_mode`] switches it. Safe any time after connect.
@@ -2335,9 +2510,10 @@ PunktfunkStatus punktfunk_connection_gamepad(const PunktfunkConnection *c, uint3
#if defined(PUNKTFUNK_FEATURE_QUIC)
// The host capability bitfield the session's `Welcome` carried — a bitfield of
// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD`. A client tests
// `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide whether to offer the shared-clipboard toggle.
// Safe any time after connect.
// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD` /
// `PUNKTFUNK_HOST_CAP_PEN`. A client tests `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide
// whether to offer the shared-clipboard toggle, `caps & PUNKTFUNK_HOST_CAP_PEN` before
// sending stylus batches. Safe any time after connect.
//
// # Safety
// `c` is a valid connection handle; `caps` is writable (NULL is skipped).
@@ -165,9 +165,11 @@ unsafe fn add(request: WDFREQUEST) {
// This WUDFHost's pid — where the host duplicates the sealed frame channel's handles INTO
// (`ProcessSharingDisabled`: this process is exclusively ours and dies with the device).
wudf_pid: std::process::id(),
// The target already carries an irrevocable hardware-cursor declare from an earlier
// session — a channel-less session must self-composite the pointer (§8.6 gap).
cursor_excluded: crate::monitor::target_declared(target_id) as u32,
// The ADAPTER already carries an irrevocable hardware-cursor declare from an earlier
// session — DWM's exclusion reaches every later monitor, not just the declaring target
// (on-glass 2026-07-23: declare on 259 left a fresh GameStream 257 cursor-less) — so a
// channel-less session must self-composite the pointer (§8.6 gap, adapter-wide).
cursor_excluded: crate::monitor::any_declared() as u32,
};
// Dual-size reply (the `cursor_excluded` tail ext): an un-upgraded host retrieves only the
// legacy 20-byte buffer — write the prefix it asked for instead of failing its ADD.
@@ -200,15 +200,18 @@ fn cursor_forward_desired(target_id: u32) -> bool {
}
/// OS target ids on which `IddCxMonitorSetupHardwareCursor` ever SUCCEEDED. A declare is
/// IRREVOCABLE for the target's life (no un-declare DDI; empty-caps re-setup is rejected; even a
/// successful same-mode re-commit never brings DWM's composited pointer back — all proven
/// on-glass, remote-desktop-sweep §8.6) and it survives monitor REMOVE→ADD because the host hands
/// each client a STABLE target id. Reported back on every ADD
/// ([`AddReply::cursor_excluded`](pf_driver_proto::control::AddReply)) so a session that never
/// negotiates the cursor channel knows it must self-composite the pointer instead of streaming a
/// cursor-less desktop. Never cleared: the state's true scope is this WUDFHost's life
/// (`ProcessSharingDisabled` — the process, and with it the adapter and every sticky declare,
/// dies on adapter reset), which is exactly when this static resets too. Bounded: ≤ 16 ids.
/// IRREVOCABLE (no un-declare DDI; empty-caps re-setup is rejected; even a successful same-mode
/// re-commit never brings DWM's composited pointer back — all proven on-glass,
/// remote-desktop-sweep §8.6) and its EXCLUSION REACH IS THE WHOLE ADAPTER, not the declaring
/// target: a declare on one target leaves every LATER monitor's frames pointer-free too, even a
/// fresh never-declared target id (proven on-glass 2026-07-23: declare on 259, then a GameStream
/// session's fresh 257 streamed a cursor-less desktop). ADD replies therefore report
/// [`AddReply::cursor_excluded`](pf_driver_proto::control::AddReply) from [`any_declared`] —
/// a session that never negotiates the cursor channel must self-composite the pointer instead
/// of streaming a cursor-less desktop. Never cleared: the state's true scope is this WUDFHost's
/// life (`ProcessSharingDisabled` — the process, and with it the adapter and every sticky
/// declare, dies on adapter reset), which is exactly when this static resets too. Per-target
/// ids are kept purely for the dbglog audit trail. Bounded: ≤ 16 ids.
static DECLARED_TARGETS: Mutex<Vec<u32>> = Mutex::new(Vec::new());
/// Record a SUCCESSFUL hardware-cursor declare on `target_id` (see [`DECLARED_TARGETS`]).
@@ -220,12 +223,13 @@ fn mark_declared(target_id: u32) {
}
}
/// True if `target_id` ever had a hardware cursor declared (see [`DECLARED_TARGETS`]).
pub fn target_declared(target_id: u32) -> bool {
DECLARED_TARGETS
/// True if ANY target ever had a hardware cursor declared in this WUDFHost's life — the
/// adapter-wide exclusion reach (see [`DECLARED_TARGETS`]).
pub fn any_declared() -> bool {
!DECLARED_TARGETS
.lock()
.unwrap_or_else(|e| e.into_inner())
.contains(&target_id)
.is_empty()
}
/// Record a departing monitor's advertised list for its id ([`MODE_HISTORY`]).
+114
View File
@@ -0,0 +1,114 @@
#!/usr/bin/env bash
# Announce a stable Punktfunk release to the Discord #releases channel.
#
# Model: release notes live in the repo at docs/releases/<tag>.md (authored during the version
# bump, BEFORE the tag is pushed — see docs/releases/README.md). CI seeds the Gitea release body
# from that file at creation, so the release is never noteless. This script is the deliberate
# "go" step (workflow_dispatch, .gitea/workflows/announce.yml): once every platform's CI is green
# it re-asserts the notes file over the live release, then posts a formatted embed to Discord.
#
# Usage: scripts/ci/discord-announce.sh <tag> # e.g. v0.18.0
# Env:
# GITHUB_SERVER_URL https://git.unom.io (Gitea Actions sets it)
# GITHUB_REPOSITORY unom/punktfunk (Gitea Actions sets it)
# GITEA_TOKEN PAT with repo write scope (from secrets.REGISTRY_TOKEN)
# DISCORD_RELEASE_WEBHOOK the #releases channel webhook URL (from secrets)
# ALLOW_PRERELEASE "1" to permit announcing a -rc/pre-release tag (default: refuse)
#
# Requires: curl + python3 (both present on the ubuntu-24.04 runner).
set -euo pipefail
TAG="${1:?usage: discord-announce.sh <tag>}"
: "${GITHUB_SERVER_URL:?}" "${GITHUB_REPOSITORY:?}" "${GITEA_TOKEN:?}" "${DISCORD_RELEASE_WEBHOOK:?}"
# Stable-only guard: a `-` in the tag marks a pre-release (v0.2.0-rc1). Refuse unless explicitly
# allowed, so an rc dispatched by accident never pings the community.
case "$TAG" in
*-*)
case "$(printf '%s' "${ALLOW_PRERELEASE:-0}" | tr '[:upper:]' '[:lower:]')" in
1|true|yes) : ;;
*) echo "discord-announce: '$TAG' looks like a pre-release; set ALLOW_PRERELEASE=1 to override" >&2
exit 1 ;;
esac ;;
esac
_here="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" && pwd)"
API="$GITHUB_SERVER_URL/api/v1/repos/$GITHUB_REPOSITORY"
# Resolve the release object created by the build workflows (id, html_url, published_at).
release_json="$(curl -fsS "$API/releases/tags/$TAG" -H "Authorization: token $GITEA_TOKEN")"
RID="$(printf '%s' "$release_json" | python3 -c 'import json,sys;print(json.load(sys.stdin).get("id",""))')"
[ -n "$RID" ] || { echo "discord-announce: no release found for tag '$TAG' — has CI created it yet?" >&2; exit 1; }
# Re-assert the notes file over the live release (source of truth), covering an edit after the
# release object was first created. No-op if docs/releases/<tag>.md is absent.
# shellcheck source=scripts/ci/gitea-release.sh
. "$_here/gitea-release.sh"
apply_release_notes "$RID" "$TAG"
# Build and post the Discord embed. The description is the notes' lead-in (everything before the
# first `## ` section header), trimmed to a readable length, with a link to the full release page
# for the sections + downloads. Falls back to the release's own body if there is no notes file.
NOTES_PATH="$(_release_notes_path "$TAG" || true)"
payload_file="$(mktemp)"
trap 'rm -f "$payload_file"' EXIT
RELEASE_JSON="$release_json" NOTES_PATH="$NOTES_PATH" TAG="$TAG" python3 - "$payload_file" <<'PY'
import json, os, sys
rel = json.loads(os.environ["RELEASE_JSON"])
tag = os.environ["TAG"]
version = tag[1:] if tag.startswith("v") else tag
url = rel.get("html_url") or ""
published = rel.get("published_at") or rel.get("created_at") or ""
notes_path = os.environ.get("NOTES_PATH") or ""
body = ""
if notes_path and os.path.isfile(notes_path):
with open(notes_path, encoding="utf-8") as f:
body = f.read()
else:
body = rel.get("body") or ""
# Lead-in = text before the first `## ` section header (the intro paragraphs); fall back to the
# whole body when the notes open straight into a section.
lead = body
idx = body.find("\n## ")
if idx != -1:
lead = body[:idx]
lead = lead.strip()
if not lead:
lead = body.strip()
LIMIT = 1800
if len(lead) > LIMIT:
cut = lead.rfind("\n\n", 0, LIMIT)
if cut < LIMIT // 2:
cut = lead.rfind(" ", 0, LIMIT)
if cut <= 0:
cut = LIMIT
lead = lead[:cut].rstrip() + " …"
description = lead
if url:
description = (description + "\n\n" if description else "") + \
f"**[⬇️ Download & full release notes →]({url})**"
embed = {
"title": f"Punktfunk {version}",
"color": 0x6C5BF3, # --pf-brand deep violet
"description": description,
"footer": {"text": "Punktfunk • git.unom.io"},
}
if url:
embed["url"] = url
if published:
embed["timestamp"] = published
with open(sys.argv[1], "w", encoding="utf-8") as f:
json.dump({"embeds": [embed]}, f)
PY
curl -fsS -o /dev/null -X POST "$DISCORD_RELEASE_WEBHOOK" \
-H 'Content-Type: application/json' --data-binary @"$payload_file"
echo "discord-announce: posted Punktfunk $TAG to #releases (release $RID)"
+5
View File
@@ -40,6 +40,11 @@ function Ensure-GiteaRelease {
$api = _GiteaApi; $h = _GiteaHeaders
$payload = @{ tag_name = $Tag; name = $Name; prerelease = $pre }
if ($TargetCommitish) { $payload.target_commitish = $TargetCommitish }
# Seed the body from docs/releases/<Tag>.md (the source of truth) so a Windows job winning the
# create race is born WITH its notes, exactly like the bash twin. Missing file (canary/rc) -> no
# body key. $PSScriptRoot is scripts/ci; walk up two levels to the repo root.
$notes = Join-Path (Split-Path (Split-Path $PSScriptRoot -Parent) -Parent) "docs/releases/$Tag.md"
if (Test-Path $notes) { $payload.body = Get-Content -Raw -Encoding utf8 $notes }
try {
$r = Invoke-RestMethod -Method Post -Uri "$api/releases" -Headers $h `
-ContentType 'application/json' -Body ($payload | ConvertTo-Json -Compress)
+55 -6
View File
@@ -35,6 +35,22 @@ for a in json.load(sys.stdin):
}
_urlencode() { python3 -c 'import urllib.parse,sys;print(urllib.parse.quote(sys.argv[1],safe=""))' "$1"; }
# _release_notes_path TAG
# Print the path of the in-repo release notes for TAG (docs/releases/<TAG>.md) IFF it exists,
# else print nothing. This file is the single source of truth for a stable release's body
# (authored as part of the version bump, before the tag is pushed — see docs/releases/README.md),
# so the Gitea release is born WITH its notes instead of being PATCHed noteless-then-late.
# canary/rc tags have no such file, which is intended (they get no curated body).
#
# Resolved relative to CWD: every caller sources this as `. scripts/ci/gitea-release.sh` from the
# repo root, so the notes are always docs/releases/<tag>.md from here. Do NOT use ${BASH_SOURCE[0]}
# — the deb + decky attach steps run under POSIX sh (dash), where an array subscript is a fatal
# "Bad substitution" (it silently broke the v0.19.0 deb/decky release-attach).
_release_notes_path() {
local notes="docs/releases/$1.md"
[ -f "$notes" ] && printf '%s' "$notes"
}
# ensure_release TAG NAME PRERELEASE [TARGET_COMMITISH]
# Idempotently create (or fetch) the release for TAG; prints its numeric id on stdout.
# PRERELEASE is "true", "false", or "auto" — auto marks it a prerelease iff TAG carries a
@@ -49,12 +65,25 @@ ensure_release() {
case "$tag" in *-*) prerelease=true ;; *) prerelease=false ;; esac
fi
api="$(_gitea_api)"
if [ -n "$target" ]; then
body=$(printf '{"tag_name":"%s","name":"%s","prerelease":%s,"target_commitish":"%s"}' \
"$tag" "$name" "$prerelease" "$target")
else
body=$(printf '{"tag_name":"%s","name":"%s","prerelease":%s}' "$tag" "$name" "$prerelease")
fi
# Build the create payload with python3 so the (multi-line, quote-bearing) release body from
# docs/releases/<tag>.md is JSON-escaped correctly. Whichever workflow wins the create race thus
# sets the body ATOMICALLY at creation; the losers 409 and fall through to the fetch-by-tag path
# below (which never touches the body). No notes file (canary/rc) -> no body key -> empty body.
local notes; notes="$(_release_notes_path "$tag")"
body=$(TAG="$tag" NAME="$name" PRERELEASE="$prerelease" TARGET="$target" NOTES_FILE="$notes" \
python3 - <<'PY'
import json, os
d = {"tag_name": os.environ["TAG"], "name": os.environ["NAME"],
"prerelease": os.environ["PRERELEASE"] == "true"}
if os.environ.get("TARGET"):
d["target_commitish"] = os.environ["TARGET"]
nf = os.environ.get("NOTES_FILE") or ""
if nf:
with open(nf, encoding="utf-8") as f:
d["body"] = f.read()
print(json.dumps(d))
PY
)
# Try to create. On any failure (almost always "release already exists"), fall back to
# fetching it by tag. Either path MUST yield an id, or we error loudly — so a 401/scope
# problem can't masquerade as a successful no-op.
@@ -93,3 +122,23 @@ upsert_asset() {
-F "attachment=@$file"
echo "gitea-release: uploaded '$name' -> release $rid"
}
# apply_release_notes RELEASE_ID TAG
# Force the release body to match docs/releases/<TAG>.md (the source of truth), if that file
# exists — a no-op otherwise. PATCHes ONLY the body, so name/prerelease/assets are preserved
# (Gitea has no partial-update footgun here; sending just {"body":...} leaves everything else).
# ensure_release already seeds the body at creation; this exists so the announce step can
# re-assert the file over the live release right before publishing — covering the case where the
# notes file was edited after the release object was first created (e.g. a tag re-point).
apply_release_notes() {
local rid="${1:?release id}" tag="${2:?tag}" api notes payload
notes="$(_release_notes_path "$tag")"
[ -n "$notes" ] || { echo "gitea-release: no docs/releases/$tag.md — leaving body as-is"; return 0; }
api="$(_gitea_api)"
payload=$(NOTES_FILE="$notes" python3 -c \
'import json,os;print(json.dumps({"body":open(os.environ["NOTES_FILE"],encoding="utf-8").read()}))')
curl -fsS -o /dev/null -X PATCH "$api/releases/$rid" \
-H "Authorization: token ${GITEA_TOKEN:?}" -H 'Content-Type: application/json' \
-d "$payload"
echo "gitea-release: synced release $rid body from $notes"
}
+424 -412
View File
File diff suppressed because it is too large Load Diff
+25 -21
View File
@@ -2,7 +2,7 @@
"name": "punktfunk-web",
"private": true,
"type": "module",
"description": "punktfunk management console TanStack Start + React Query (orval) + @unom/ui + Paraglide i18n",
"description": "punktfunk management console \u2014 TanStack Start + React Query (orval) + @unom/ui + Paraglide i18n",
"scripts": {
"prepare": "bun run codegen",
"codegen": "orval --config orval.config.ts && paraglide-js compile --project ./project.inlang --outdir ./src/paraglide && node tools/check-i18n.mjs",
@@ -19,48 +19,52 @@
"screenshots:build": "bun run build-storybook && node tools/screenshots.mjs"
},
"dependencies": {
"@fontsource-variable/geist": "^5.2.9",
"@tanstack/react-query": "^5.101.2",
"@tanstack/react-router": "^1.170.17",
"@tanstack/react-start": "^1.168.27",
"@fontsource-variable/geist": "^5.3.0",
"@tanstack/react-query": "^5.101.4",
"@tanstack/react-router": "^1.170.18",
"@tanstack/react-start": "^1.168.32",
"@unom/style": "^0.4.4",
"@unom/ui": "^0.8.16",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"lucide-react": "^0.469.0",
"motion": "^12.42.2",
"radix-ui": "^1.6.2",
"react": "^19.2.7",
"react-dom": "^19.2.7",
"recharts": "^3.9.2",
"radix-ui": "^1.6.4",
"react": "^19.2.8",
"react-dom": "^19.2.8",
"recharts": "^3.10.0",
"tailwind-merge": "^2.6.1",
"zod": "^4.4.3"
},
"devDependencies": {
"@biomejs/biome": "^2.5.2",
"@inlang/paraglide-js": "^2.20.2",
"@biomejs/biome": "^2.5.5",
"@inlang/paraglide-js": "^2.22.0",
"@inlang/plugin-message-format": "^4.4.0",
"@storybook/react-vite": "^10.4.6",
"@tailwindcss/vite": "^4.3.2",
"@storybook/react-vite": "^10.5.3",
"@tailwindcss/vite": "^4.3.3",
"@tanstack/nitro-v2-vite-plugin": "^1.155.0",
"@types/node": "^22.20.0",
"@types/node": "^22.20.1",
"@types/react": "^19.2.17",
"@types/react-dom": "^19.2.3",
"@vitejs/plugin-react": "^5.2.0",
"orval": "^8.20.0",
"orval": "^8.22.0",
"playwright": "^1.61.1",
"storybook": "^10.4.6",
"tailwindcss": "^4.3.2",
"storybook": "^10.5.3",
"tailwindcss": "^4.3.3",
"tw-animate-css": "^1.4.0",
"typescript": "^5.9.3",
"vite": "^7.3.6",
"vite-tsconfig-paths": "^5.1.4"
},
"overrides": {
"tar": "^7.5.21",
"dompurify": "^3.4.12",
"linkify-it": "^5.0.2",
"sharp": "^0.35.3",
"fast-uri": "^3.1.4",
"immutable": "^4.3.9",
"undici": "^7.28.0",
"dompurify": "^3.4.11",
"postcss": "^8.5.16",
"esbuild": "^0.28.1",
"js-yaml": "^4.2.0"
"postcss": "^8.5.10",
"js-yaml": "^4.3.0"
}
}