43 Commits

Author SHA1 Message Date
enricobuehler 4306d4f914 fix(windows/gamestream): create the virtual display on Windows (Moonlight black screen)
apple / swift (push) Successful in 1m6s
ci / rust (push) Successful in 5m17s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 57s
release / apple (push) Successful in 8m42s
ci / bench (push) Successful in 4m55s
windows-host / package (push) Successful in 6m24s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m23s
apple / screenshots (push) Successful in 5m46s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m13s
android-screenshots / screenshots (push) Successful in 53s
android / android (push) Successful in 3m27s
decky / build-publish (push) Successful in 15s
deb / build-publish (push) Successful in 3m34s
linux-client-screenshots / screenshots (push) Successful in 2m17s
flatpak / build-publish (push) Successful in 4m6s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m57s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m44s
docker / deploy-docs (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
web-screenshots / screenshots (push) Successful in 2m28s
The GameStream video path (open_gs_virtual_source) ran the Linux compositor-
detection state machine on every platform. On Windows detect_active_session()
returns None and vdisplay::detect() bails ("could not detect compositor ...
XDG_CURRENT_DESKTOP=''"), killing the video thread right after RTSP PLAY — so a
Moonlight client paired, negotiated, then black-screened and dropped.

The native punktfunk/1 path already guards this (resolve_compositor returns a
placeholder Compositor on Windows, since vdisplay::open ignores the compositor
arg there and always uses the pf-vdisplay IddCx backend). Mirror that guard in
the GameStream path: short-circuit to a placeholder on Windows, keep the Linux
session detection (apply_session_env/apply_input_env) under cfg(not(windows)).

Validated live: Moonlight -> this box now creates the pf-vdisplay virtual
monitor, attaches the IDD-push ring, and NVENC streams 5120x1440@240.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 12:11:31 +02:00
enricobuehler 915f11a712 fix(apple/tvOS): guard EDR HDR APIs unavailable on tvOS
apple / swift (push) Successful in 1m12s
release / apple (push) Successful in 9m11s
apple / screenshots (push) Successful in 4m49s
ci / web (push) Successful in 1m1s
ci / docs-site (push) Successful in 1m19s
ci / rust (push) Successful in 4m45s
deb / build-publish (push) Successful in 3m11s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 6m51s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m55s
docker / deploy-docs (push) Successful in 5s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m49s
android / android (push) Successful in 3m10s
The tvOS archive failed compiling PunktfunkKit: a recent presenter HDR change
dropped the `#if os(macOS)` guard around the EDR calls and applied them "on all
platforms", but `wantsExtendedDynamicRangeContent`, `CAEDRMetadata`, and
`CAMetalLayer.edrMetadata` are all explicitly unavailable on tvOS.

Wrap the EDR usage (and the makeEDR helper, whose return type is the unavailable
CAEDRMetadata) in `#if !os(tvOS)`. macOS + iOS keep the reference-white-anchored
EDR path unchanged; tvOS now sets only the rgba16Float pixel format + itur_2100_PQ
colour space and lets its compositor tone-map from those. The 0xCE grade is still
cached on tvOS (harmless), it just can't be pushed to the layer there.

tvOS Simulator build: BUILD SUCCEEDED (PunktfunkKit Swift compile, the step that
failed). macOS build + test green (49 tests); iOS compiles clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 11:12:33 +02:00
enricobuehler fc35ea8c31 fix(windows): replace em dash with ASCII hyphen in install-vbcable.ps1
apple / swift (push) Successful in 1m9s
windows-host / package (push) Successful in 7m33s
android / android (push) Failing after 30s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m17s
apple / screenshots (push) Successful in 5m38s
ci / rust (push) Successful in 4m44s
ci / bench (push) Successful in 4m38s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
deb / build-publish (push) Successful in 3m20s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m4s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m14s
docker / deploy-docs (push) Successful in 17s
PS 5.1 mis-parses non-ASCII characters on non-UTF-8 locales; the
locale-safety gate CI check rejects any installer-run script containing
bytes above 0x7F.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 10:59:14 +02:00
enricobuehler 1e9a15699c fix(apple/iOS): capture all attached mice; gate UIKit pointer path under lock
apple / swift (push) Successful in 1m6s
ci / web (push) Has been cancelled
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7s
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
release / apple (push) Successful in 7m30s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m39s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m43s
docker / deploy-docs (push) Successful in 18s
The iPad pointer lock engaged but a Magic Keyboard trackpad went dead the
moment a second pointer (a Universal Control "V-UC Automouse") was connected —
on-device PUNKTFUNK_INPUT_DEBUG logs showed only ONE GCMouse attached (whichever
was GCMouse.current), so the other device's motion handler was never installed.

InputCapture.start() now attaches a handler to EVERY GCMouse.mice(), not just
GCMouse.current, so a trackpad and a second mouse both drive (each GCMouse
delivers its own deltas through its own handler). New arrivals still come via the
GCMouseDidConnect observer.

Also gate the WHOLE UIKit indirect-pointer path (motion, buttons AND scroll) on
!gcMouseForwarding, not just motion+scroll: under pointer lock GCMouse owns
buttons too, and the trackpad/mouse also emit UIKit indirect-pointer events
pinned at the locked position — without the gate a click double-sent (GCMouse +
UIKit). The two paths are now exact mirrors on `gcMouseForwarding` (== locked).

Removes the investigation-only diagnostics (attachedMiceSummary/hasGCMouse, the
per-event UIKit pointer/scroll logs, the GCMouse attach/became-current logs);
the pre-existing `pointer lock isLocked=… captured=…` debug line is restored.

iOS compiles against the SDK; macOS swift build + test green (49 tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 10:56:37 +02:00
enricobuehler 6c2942ee45 fix(fmt): remove extra blank line in dxgi.rs
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
windows-host / package (push) Failing after 11s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 08:56:14 +00:00
enricobuehler 188b26b584 fix build
windows-drivers / probe-and-proto (push) Successful in 27s
ci / rust (push) Failing after 1m8s
apple / swift (push) Successful in 1m8s
windows-drivers / driver-build (push) Successful in 1m14s
windows-host / package (push) Failing after 12s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m8s
android / android (push) Successful in 3m23s
apple / screenshots (push) Successful in 5m24s
deb / build-publish (push) Successful in 3m22s
decky / build-publish (push) Successful in 25s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 46s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
ci / bench (push) Successful in 5m1s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m11s
docker / deploy-docs (push) Successful in 8s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m49s
2026-06-30 10:12:45 +02:00
enricobuehler 83ee53290e feat(windows-host): mic passthrough — auto-wire audio devices + bundle VB-CABLE
The Windows virtual mic worked only with manual Sound-settings fiddling: on a
headless host (no real audio output) BOTH the desktop-audio loopback and the
virtual mic must run on virtual cables, and on DIFFERENT ones or the loopback
re-captures the injected mic (echo). The Steam pair gives only one usable cable
(Steam Streaming Speakers loopback is silent — validated), so the mic + loopback
collided and echoed, and when the default playback happened to be the mic device
the anti-echo guard reported the mic "unavailable".

Host now auto-wires the devices at startup (audio/windows/audio_control.rs,
ensure_wired_once, hooked from open_audio_capture/open_virtual_mic): default
playback = a loopback-capable render that is NOT a cable and NOT the dead Steam
Speakers (real output > Steam Streaming Microphone); default recording = the mic
capture (VB-Cable "CABLE Output" preferred). Uses a hand-rolled IPolicyConfig
vtable (the only way to set a default endpoint; not in windows/wasapi crates).
Opt out with PUNKTFUNK_KEEP_DEFAULT. wasapi_mic candidates now prefer "cable
input". Validated live: from a deliberately-wrong start (playback=CABLE Input)
the host corrected both default endpoints at the OS level.

A Windows audio endpoint can only be created by a kernel-mode driver (no UMDF
path — ACX is KMDF-only), so we cannot self-sign our own like the UMDF gamepad/
display drivers. Instead the installer bundles + silently installs the official
base VB-CABLE (VB-Audio donationware, vendor-signed → loads with no test-signing,
redistributed under VB-Audio's bundling grant): install-vbcable.ps1 (seed the
VB-Audio cert into TrustedPublisher, run -i -h) + an installaudiocable task,
gated on -VbCableDir/$env:VBCABLE_DIR (the package binary is not in the repo).
Attribution in packaging/windows/licenses/VB-CABLE-NOTICE.txt. .iss compiles
with the path enabled.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 09:09:26 +02:00
enricobuehler 0f798d62b6 feat(windows-host): pf-vdisplay — fix the ADD/REMOVE wedge + per-client display-config persistence
Two phases of pf-vdisplay (IddCx virtual display) lifecycle work, both validated on-glass on the RTX box.

Phase 1 — fix the long-standing IOCTL_ADD 0x80070490 (ERROR_NOT_FOUND) wedge that ghost-monitor
slot-budget exhaustion produced under ADD/REMOVE churn (the reset-script/reboot recurring failure).
Validated: 43 reconnect-churn cycles, 0 wedges, monitor-node count flat at 1.
  * driver: on IddCxMonitorArrival failure, tear the created-but-not-arrived monitor down with
    WdfObjectDelete + reclaim its id — the asymmetric-with-the-create-failure-path leak that exhausted
    the 16-monitor MaxMonitorsSupported budget; recover MONITOR_MODES from lock poisoning instead of
    failing closed (defensive; the driver builds panic=abort).
  * host: collapse the build-retry churn — hold ONE monitor lease across all build attempts and preempt
    only on Lingering (not Active), so a cold start does 1 ADD not 8; reap not-present "punktfunk"
    monitor PDOs on startup (the reset-script step-2 logic, in-process) and self-heal a detected
    0x80070490 by reaping + retrying ADD; force-preempt a stuck-Active prior monitor on the
    begin_idd_setup timeout (the safety net the Lingering-only preempt would otherwise drop).

Phase 2 — give each client (keyed by its cert FINGERPRINT) a STABLE virtual-monitor id (1..=15) so
Windows reapplies that client's saved per-monitor config (DPI SCALING) across reconnects, and two
clients never share/bleed config. Validated: distinct clients -> distinct ids (1, 2); the driver
honors the host's id (echoed resolved == preferred).
  * proto: rename AddRequest._reserved -> preferred_monitor_id (offset 20) and AddReply._reserved ->
    resolved_monitor_id (offset 12) — byte-compatible (offset asserts), NO PROTOCOL_VERSION bump, so a
    pre-Phase-2 driver degrades gracefully to auto-id (the host detects it via the resolved echo).
  * driver: create_monitor honors a host-supplied preferred id via resolve_id (range 1..=15, never
    collides with a live monitor) and seeds the EDID serial + IddCx ConnectorIndex + ContainerId from it.
  * host: a persisted LRU fingerprint->id map (%ProgramData%\punktfunk\pf-vdisplay-identity.json),
    threaded to add_monitor via a set_client_identity no-op trait method (Linux/GameStream unaffected).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 09:09:26 +02:00
enricobuehler 080c55dbf7 refactor(host/windows): collapse Windows capture to IDD-push only
apple / swift (push) Successful in 1m5s
ci / rust (push) Failing after 1m29s
windows-host / package (push) Failing after 1m11s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m4s
android / android (push) Successful in 3m35s
apple / screenshots (push) Successful in 5m30s
deb / build-publish (push) Successful in 3m18s
decky / build-publish (push) Successful in 27s
ci / bench (push) Successful in 4m39s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m38s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m23s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 52s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m24s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m7s
docker / deploy-docs (push) Failing after 12m53s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
Remove DXGI Desktop Duplication (DuplCapturer), Windows.Graphics.Capture
(WgcCapturer), the two-process SYSTEM+helper relay (virtual_stream_relay /
HelperRelay / DesktopWatcher / composed_flip), and the five source files that
implemented them. IDD direct-push is now the sole Windows capture path; the
session topology is always SingleProcess.

Deleted files: wgc.rs, wgc_relay.rs, desktop_watch.rs, composed_flip.rs,
windows/wgc_helper.rs (+ wgc-helper subcommand in main.rs).

dxgi.rs is kept but carved to shared GPU primitives only (make_device,
HdrP010Converter, VideoConverter, install_gpu_pref_hook, WinCaptureTarget,
pack_luid) — ~2237 lines of DDA-only code removed; imports cleaned.

capture.rs: IDD-push open failure fails the session cleanly (no fallback).
Adds capturer_supports_444() — returns false on Windows (IDD-push 4:4:4 is a
follow-up), replacing the stale single_process gate in 4:4:4 negotiation.
session_plan.rs: CaptureBackend{Dda,Wgc} and SessionTopology::TwoProcessRelay
removed. config.rs: no_helper/force_helper/no_wgc/capture_backend/secure_dda
removed. merged_env_block relocated from wgc_relay to windows/interactive.rs.

Linux cargo check clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 06:46:52 +00:00
enricobuehler 1c04e77293 feat(apple): Improve presenter
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
android-screenshots / screenshots (push) Successful in 2m16s
deb / build-publish (push) Successful in 3m26s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
windows-host / package (push) Successful in 6m48s
release / apple (push) Successful in 7m45s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m22s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m37s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
android / android (push) Successful in 9m35s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m32s
linux-client-screenshots / screenshots (push) Successful in 2m31s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m53s
web-screenshots / screenshots (push) Successful in 2m32s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m37s
flatpak / build-publish (push) Failing after 3m47s
docker / deploy-docs (push) Failing after 1m9s
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
feat(apple): add cursor capture on iPad
2026-06-30 01:31:48 +02:00
enricobuehler e2d4c40167 feat(android): HDR toggle, video-feed stats, landscape lock
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 5m14s
ci / web (push) Successful in 49s
apple / screenshots (push) Successful in 5m19s
ci / docs-site (push) Successful in 1m0s
ci / bench (push) Successful in 4m36s
ci / rust (push) Successful in 13m31s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 40s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
deb / build-publish (push) Successful in 10m9s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m50s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m19s
docker / deploy-docs (push) Failing after 1m4s
- HDR toggle in Settings → Display. Persisted (hdr_enabled, default on); the
  host is advertised HDR only when the toggle is on AND the panel can present
  HDR10 (displaySupportsHdr), so SDR panels never get PQ they'd mis-tone-map.
  The toggle is disabled/greyed on non-HDR displays (ToggleRow gained `enabled`).
- Extend the stats HUD with a video-feed line, e.g.
  "HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0". nativeVideoStats now returns 14
  doubles (appends bitDepth, CICP primaries/transfer, chroma_format_idc from the
  negotiated Welcome); older/shorter layouts just omit the line.
- Lock the stream to landscape while streaming (SENSOR_LANDSCAPE), restoring the
  prior orientation on exit. The activity declares configChanges=orientation, so
  it re-lays-out in place with no stream restart; harmless no-op on TV.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 22:16:04 +02:00
enricobuehler 580b1ea7a7 feat(host/steam): shippable usbip/vhci_hcd virtual Deck + client leave-shortcuts
apple / screenshots (push) Has been cancelled
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
audit / cargo-audit (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
Steam Deck pass-through (design/steam-deck-passthrough-plan.md), code-complete +
all CI checks green on Linux + adversarially reviewed; on-glass validation pending:

- usbip/`vhci_hcd` virtual Deck transport (inject/linux/steam_usbip.rs) for
  non-SteamOS hosts (Bazzite/generic) — presents a real interface-2 USB Deck so
  Steam Input promotes it. In-process vhci attach (loopback OP_REQ_IMPORT handshake
  → sysfs attach) with a bounded `usbip`-CLI fallback; detach on drop.
- Backed by a vendored, libusb-free trim of the `usbip` crate
  (crates/punktfunk-host/vendor/usbip-sim, MIT + NOTICE; host/cdc/hid + rusb/nusb
  removed; interrupt-IN paced by bInterval).
- Selection ladder raw_gadget (SteamOS fast-path) → usbip (universal) → UHID,
  with PUNKTFUNK_STEAM_USBIP / PUNKTFUNK_USBIP_ATTACH knobs.
- Shared Deck descriptors + the 0x83/0xAE feature contract + a Steam-accepted
  serial consolidated into steam_proto.rs; the raw_gadget backend reuses them.
- Linux client leave-shortcuts: Ctrl+Alt+Shift+D + holding the escape chord
  (L1+R1+Start+Select) >=1.5s end the session (short press still exits
  fullscreen); the chord state resets across sessions.

Also bundles in-progress work already staged in the tree:
- host(kwin): xdg-output logical-geometry mapping so the KWin fake_input backend
  places absolute coordinates correctly under display scaling.
- docs: design/README index entries + design/controller-only-mode.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 831b37b4b7 build: exclude the usbip-poc from the workspace (standalone PoC, pulls libusb)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 4f0b4aa68f docs(steam): production plan for Deck client pass-through + shippable usbip host
Write design/steam-deck-passthrough-plan.md — the build plan to ship exact Steam
Deck pass-through from the Linux client (incl. the Steam + QAM buttons) plus a
virtual Deck on any Linux host. Key validated facts captured so the next session
doesn't re-investigate:

- Client capture is ALREADY correct: SDL3 maps Steam->Guide, QAM->Misc1; the
  client forwards BTN_GUIDE/BTN_MISC1; the host maps them to btn::STEAM/btn::QAM.
  Only precondition: Steam Input disabled on the client (the Decky UX).
- Shippable host transport = usbip + vhci_hcd (in-tree + signed everywhere, no
  module build, no MOK) — PROVEN on Bazzite: Steam promotes the usbip interface-2
  Deck (XInput slot + X-Box pad), identical to raw_gadget on SteamOS.
- Build steps: refactor steam_gadget.rs into shared Deck-logic + a transport
  trait; add the usbip transport (vendor-trim the usbip crate to drop rusb/libusb,
  in-process vhci attach); transport-select raw_gadget->usbip->UHID/DualSense;
  client leave-shortcut (controller chord + Ctrl+Alt+Shift+D); serial polish.

Also checks in the working usbip Deck PoC (packaging/linux/steam-deck-gadget/
usbip-poc/) for the next session to build on. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 963c406f33 feat(host/steam): default the gadget Deck on for SteamOS (glass-confirmed)
The virtual Steam Deck is validated glass-to-glass on a Deck: it appears as a
distinct second Steam controller, a held A drives Steam's overlay ("Resume
Game"), and a button press registers in a real game (confirmed in-game).

gadget_preferred() now defaults ON for SteamOS hosts (/etc/os-release ID=steamos
or ID_LIKE), OFF elsewhere where the universal UHID path stays the default;
PUNKTFUNK_STEAM_GADGET=1/0 forces it. A Deck-as-host with a physical Deck never
reaches this path — resolve_gamepad's conflict gate degrades SteamDeck → DualSense
first, so the two-Deck case never happens in production (it was only a test-rig
confound on the dev Deck).

The feature is complete: a virtual Steam Deck that Steam Input recognizes +
promotes, churn-free, with input flowing to games. Workspace clippy/fmt/test
green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 7ab8acaf55 feat(host/steam): harden the gadget feature contract — fixes the evdev churn
The virtual Deck's gamepad evdev was churning (destroyed + recreated) because
Steam kept re-probing: GetControllerInfo reads HID feature reports, and the gadget
served zeros for them. Captured the real contract off a physical Deck
(packaging/linux/steam-deck-gadget/get_deck_attrs.c, hidraw HIDIOCGFEATURE — usbmon
truncates to 32B) and implemented it in steam_gadget.rs::feature_reply:

- 0x83 GET_ATTRIBUTES_VALUES: [83, 2d, 9×(attr-id, u32-LE)] — product id 0x1205, a
  per-instance unit serial (0x0a/0x04, so a gadget never collides with a real Deck
  or another gadget), and the capability attrs (0x09=0x2e, 0x0b=0x0fa0, rest 0).
- 0xAE GET_STRING_ATTRIBUTE: [ae, len, attr, ascii] — serial (attr 1) / board
  serial (attr 0).
- other commands (0x87 settings): echo the last write.

Validated on the Deck: 1 connect / 0 disconnect / 1 gamepad evdev (was constant
churn), Steam activates the gadget cleanly (no GetControllerInfo failed, no zombie)
and emits its X-Box 360 pad. usbmon on the gadget's bus confirms our state reports
(pressed button at byte 8) are delivered on the interrupt-IN and consumed by
hid-steam — so with M1/M2's byte-8→BTN_SOUTH decode the input chain is proven
end-to-end. Remaining: a foreground-game confirmation of Steam Input's XInput
mapping, then default the gadget on for SteamOS.

Workspace clippy/fmt/test green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler c8e19396e4 feat(host/steam): raw_gadget Deck host backend (Steam-Input path, opt-in)
Port the proven raw_gadget virtual Deck to a Rust host gamepad backend, the
SteamOS-only transport that gets Steam Input to actually promote the Deck.

- inject/linux/steam_gadget.rs (new): SteamDeckGadget — a userspace raw_gadget
  emulator of the real 3-interface USB Deck (mouse=0/keyboard=1/controller=2,
  28DE:1205) on a dummy_hcd loopback UDC, descriptors captured from a physical
  Deck, answering every control transfer incl. the HID feature reports. Driven by
  the same steam_proto::serialize_deck_state as the UHID pad; rumble feedback via
  parse_steam_output. The raw_gadget UAPI is funneled through 4 documented ioctl
  wrappers (the crate denies undocumented unsafe).
- inject/linux/steam_controller.rs: the manager pad is now a DeckTransport enum
  (Uhid | Gadget); ensure() prefers the gadget when PUNKTFUNK_STEAM_GADGET=1
  (best-effort modprobe dummy_hcd+raw_gadget), gracefully falling back to the
  universal UHID SteamDeckPad. write/pump/heartbeat dispatch through the enum.

Validated on a real Deck via a static musl harness that #[path]-includes the
module: enumerates, hid-steam binds + reads our serial + creates the Steam Deck +
Motion Sensors evdevs — identical to the C PoC. Caught a real portability bug:
raw_gadget's no-arg ioctls (RUN/CONFIGURE/EP0_STALL) reject a non-zero `value`
with EINVAL, and on musl an omitted ioctl vararg is a garbage register — so they
must pass an explicit 0.

Opt-in (default off) while the Steam GetControllerInfo feature contract is
hardened (to stop the gamepad-evdev churn). Workspace clippy/fmt/test green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 78020cd66c docs(steam): gadget input-flow status — reports delivered + format-validated
On the Deck, a pressa build shows hid-steam polls our interface-2 interrupt-IN
endpoint and our 64-byte state reports are delivered ("STREAM: first input report
delivered"). The report format is already validated (M1 serializer on-box + M2's
EVIOCGKEY/EVIOCGABS test on the same hid-steam decode). The "Steam Deck" gamepad
evdev forms but is transient (hid-steam recreates it as gamepad_mode toggles —
Steam keeps re-probing because the PoC serves the serial but not Steam's full
GetControllerInfo attribute set, on a heavily-churned test Deck), so a stable live
EVIOCGKEY catch of the held A wasn't obtained. Delivery + format proven; the
evdev transience is a feature-report-completeness gap the host backend resolves.
Doc §11. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 8870e85233 feat(steam): raw_gadget virtual Deck — full Steam Input recognition (proven on Deck)
The interface-2 wall is climbed. packaging/linux/steam-deck-gadget/deck_raw_gadget.c
is a raw_gadget userspace emulator of a real 3-interface USB Steam Deck (28DE:1205,
mouse=0/keyboard=1/controller=2) on a dummy_hcd loopback UDC, with descriptors
captured verbatim from a physical Deck and full HID feature-report handling.

Live on a real Deck (SteamOS 3.8.11): hid-steam reads our serial (PFDECK000),
creates the Steam Deck + Motion Sensors evdevs, and Steam Input PROMOTES it —
controller.txt "Interface: 2 ... device opened ... reserving XInput slot 1" +
"input: Microsoft X-Box 360 pad 1". Stable (1 connect, 0 disconnects, no zombie);
the kernel Steam Deck evdev is then grabbed by Steam Input which exposes its own
X-Box pad, exactly like a real Deck. First time a virtual Deck is fully Steam-Input
promoted (UHID can't — it has no USB interface number, so Steam filters it).

Also includes the configfs f_hid variant (configfs_gadget_up/down.sh) — the minimal
reproducer that proved interface 2 makes Steam open+XInput-reserve the device, but
f_hid can't serve feature reports so Steam dropped it as a zombie.

Gotchas documented in the README: 7-byte vs 9-byte endpoint descriptor, no-data OUT
controls acked via zero-length EP0_READ (not WRITE, else error -110), streamer must
not start before SET_CONFIGURATION is acked. SteamOS-host only (needs dummy_hcd +
raw_gadget). Recognition proven; feeding real client reports + a host backend is next.
Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler a81f1304cd docs(steam): gadget PoC — interface 2 PROVEN (Steam opens + XInput-reserves it)
On the Deck (which ships dummy_hcd + raw_gadget + configfs f_hid), a pure-shell
configfs gadget stood up a real 3-interface USB Deck (kbd=0/mouse=1/controller=2,
28de:1205) on a dummy_hcd loopback UDC. hid-steam bound all 3 interfaces, and
crucially Steam PROMOTED the interface-2 controller: "Local Device Found ...
Interface: 2 ... Steam controller device opened for index 14 ... Steam Controller
reserving XInput slot 1" — exactly where the interface -1 UHID Deck was filtered.

It then failed only at feature-report exchange (f_hid can't serve HID GET_REPORT:
"steam_send_report: error -32", "couldn't get controller details ... zombie
controller"), and no gamepad evdev formed for the same reason. So interface 2 is
necessary AND sufficient for Steam to open+XInput-reserve the Deck; the remaining
piece is serving feature/output reports, which raw_gadget can (full control,
like UHID). Next: a raw_gadget 3-interface Deck emulator. Doc §11. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler c75f39fd8e docs(steam): VALIDATED — virtual DualSense IS Steam-Input recognized; isolates the Deck wall to interface 2
Definitive hardware test (Bazzite running Steam): a virtual DualSense (UHID,
054c:0ce6, Interface: -1) is FULLY promoted by Steam — controller.txt logs
"Local Device Found 054c 0ce6 DualSense Wireless Controller", then "Controller
using HIDAPI driver vid=0x054c pid=0x0ce6" and loads configset_controller_ps5.vdf
(our calibration/pairing/firmware feature blobs read back). The SAME Interface:
-1 that the Deck is rejected at is accepted for the DualSense.

So the wall is specifically the Deck's MULTI-INTERFACE requirement (Steam must
pick interface 2 among kbd/mouse/controller), NOT a UHID limitation. The
DualSense path delivers real Steam Input (gyro + touchpad + glyphs + bindings)
for a streamed Deck/SC client; it loses only Deck glyphs, the 2nd trackpad, and
the 4 back grips as distinct Steam-Input paddles (M5 folds them to buttons).

Full Deck-identity Steam Input would need interface 2 -> a USB gadget (dummy_hcd
+ configfs HID, controller on interface 2). Feasible but heavy/non-portable:
dummy_hcd isn't built on Bazzite/Deck/dev-box, so it'd be a per-kernel build +
(on immutable SteamOS/Bazzite) a package-layer + reboot per host.

Doc-only (design §11). Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 37c3e2bed2 docs(steam): criterion-4 RESOLVED — the interface-2 ceiling; recommend dropping M7
Hardware finding (a SteamOS Deck @ .253 + a Bazzite host @ .41, both running
Steam, via a minimal C UHID probe on Bazzite): a UHID virtual Steam Deck binds
the kernel hid-steam and creates the evdevs (so kernel-evdev + SDL-hidapi
consumers see the full grips/trackpads/IMU surface), but Steam Input will NOT
manage it. Steam's controller.txt enumerates it ("Local Device Found, 28de 1205,
Product Punktfunk Steam Deck") but logs Interface: -1 and never promotes it (no
28de:11ff XInput pad). The physical Deck on the same logs is Interface: 2 — a
real Deck is a 3-interface USB device (kbd 0 / mouse 1 / controller 2) and Steam
binds the controller on interface 2; a single UHID device has no USB interface
number, so Steam reads -1 and filters it out. (The feared 0x83/0xA1 attribute
probes never fired — it's an interface filter, not a probe-reject.)

Consequences (design §11):
- The virtual Deck's value is non-Steam / SDL games on Linux (grips + trackpads
  + gyro via evdev / SDL HIDAPI), NOT Steam Input.
- The virtual DualSense stays the Steam-Input path everywhere (Steam recognizes
  a single-interface DualSense); M5's paddle-fold carries the back grips.
- M7 (a Windows UMDF virtual Deck) is NOT recommended: same interface filter,
  and Windows has no kernel-hid-steam evdev fallback, so nothing would consume
  it; the existing Windows virtual DualSense already covers that case.
- M0-M6 is not wasted: the protocol/wire + client capture feed the DualSense
  path too, and the virtual Deck is the best option for non-Steam Linux games.

Doc-only (design/steam-controller-deck-support.md): added §11, updated the status
+ pending-validation. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 4f40fa3cb7 feat(host/steam): M6 — Steam-pad conflict gate (hardware-validated)
Don't present a virtual Steam (28DE) pad on a host that already has a physical
Steam controller — the host's own Steam Input would then manage two Decks and
confuse player assignment.

- physical_steam_controller_present(): scans /sys/bus/hid/devices for a 28DE HID
  device on a real (non-/virtual/) path.
- degrade_steam_on_conflict() in resolve_gamepad: a resolved SteamDeck /
  SteamController with a physical Steam controller attached degrades to DualSense
  (then the M5 uhid ladder); PUNKTFUNK_STEAM_FORCE=1 overrides (e.g. a remote-only
  box with no competing Steam Input).

Validated on real hardware (a SteamOS Steam Deck @ .253 + a Bazzite host @ .41,
both running Steam):
- Conflict confirmed: the Deck-as-host already has its physical 28DE:1205 AND
  Steam's 28DE:11FF XInput output pad live; a 2nd virtual 28DE = two Decks.
- Bind robustness: the virtual Deck binds hid-steam on a SECOND kernel (Bazzite
  6.17.7, vs the dev box 7.0) and the kernel accepted our serial (the M1 fix).
- Criterion-4 (running-Steam recognition) PARTIAL: a userspace consumer (Steam/
  SDL) engaged the virtual Deck (opened the hidraw, ran the lizard-disable +
  settings sequence the kernel's Deck path skips) but emitted NO 28DE:11FF XInput
  pad on the desktop — so Steam recognizes it enough to manage lizard mode but did
  not promote it to a managed XInput controller (likely needs a Big-Picture/game
  context, or a richer device; the 0x83/0xA1 attribute probes never fired, so it
  wasn't a probe-reject either).
- The heuristic itself checks TRUE on the Deck, FALSE on Bazzite.

Workspace clippy/fmt/test green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 486a292845 feat(host/steam): M5 — fallback remap, motion rescale, degrade ladder
Keep the rich Steam inputs from silently dropping when the resolved backend
isn't the virtual hid-steam device, and fix a cross-device motion-scale bug.

- inject/proto/steam_remap.rs (new, pure + unit-tested):
  * motion_wire_to_deck — the wire carries DualSense-convention units (20 LSB/
    deg.s gyro, 10000 LSB/g accel — what every client capture emits), but the
    Deck's hid-steam report wants 16 LSB/deg.s + 16384 LSB/g. The Deck backend
    now rescales (gyro x16/20, accel x16384/10000): a real Deck<->Deck gyro/
    accel correctness fix (the DualSense/DS4 backends consume the wire 1:1).
  * fold_paddles + RemapConfig (PUNKTFUNK_STEAM_REMAP=paddles=drop|stickclicks|
    shoulders, default drop) — the DualSense + DS4 managers fold a client's back
    grips onto standard buttons rather than dropping them (those pads have no
    back-button HID slot; the uinput Xbox pad already exposes them as Elite
    paddles BTN_TRIGGER_HAPPY5-8).

- resolve_gamepad: a runtime degrade ladder — a UHID backend (DualSense / DS4 /
  Steam Deck) on a host where /dev/uhid isn't writable now falls back to the
  uinput Xbox 360 pad instead of a dead controller (the device-create would
  just fail). Separate from pick_gamepad's compile-time platform check, so the
  existing pick_gamepad tests are untouched.

- Delete the throwaway M0/M1 spike (src/bin/steam_uhid_spike.rs) — M2's
  #[ignore]d backend test subsumes its validation, and removing it frees
  steam_proto to reference steam_remap cleanly.

On-box backend test still green; workspace clippy/fmt/test green (incl. the new
steam_remap tests). Deferred as optional RemapConfig growth: gyro->mouse /
trackpad->stick synthesis on an Xbox target (no slot — documented drop today).
Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler d8c254281e feat(steam): M4 complete — C-ABI send path, Decky UX, Apple/Android parity
Finish the client side of the Steam Controller / Steam Deck pipeline.

- C-ABI (core abi.rs): PunktfunkRichInputEx — a size-prefixed superset of
  PunktfunkRichInput that can express the second trackpad (surface), a distinct
  click vs touch, signed coords + pressure — plus
  punktfunk_connection_send_rich_input2 (the struct_size ABI-skew-guard
  precedent). The only way a C client (Apple/embedders) can emit a TouchpadEx;
  the legacy struct + send_rich_input stay byte-for-byte. punktfunk_core.h
  regenerated.

- Decky (clients/decky): a "Steam Deck" gamepad type in Settings + an unmissable
  Disable-Steam-Input instruction shown when it's selected (in Game Mode Steam
  Input holds 0x1205, so the SDL HIDAPI Steam driver can't open the Deck's
  controls until the user disables Steam Input for the shortcut). Plus a
  best-effort, feature-detected disableSteamInputForShortcut() in launchStream —
  never blocks/throws; the manual toggle is the documented source of truth.

- Apple parity (PunktfunkConnection.swift): GamepadType.steamController/steamDeck
  (wire 5/6) + name parsing, so the resolved type round-trips. Capture is blocked
  (GameController never surfaces a 0x28DE HID device).

- Android parity (Gamepad.kt): PREF_STEAMCONTROLLER/STEAMDECK + the Valve 0x28DE
  PIDs in prefFor(). Rich-input capture stays out of scope (no rich-input plane
  yet) — standard buttons/sticks resolve to the host's Steam Deck pad.

Rust workspace clippy/fmt/test green; Decky src/ typechecks clean (only a
pre-existing @decky/api dep resolution error remains); Swift/Kotlin compile on
their CI. The full pipeline is now BUILT; what remains is validation that needs
hardware we don't have (a running Steam on the host, a live Deck client, the
Moonlight paddle regression). Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler ae71e4628d feat(clients/steam): M4 — desktop SDL clients capture the rich Steam inputs
The Linux + Windows native clients (clients/{linux,windows}/src/gamepad.rs) now
capture and send the Steam Controller / Steam Deck rich inputs, so a real Deck
(off Steam Input) or a Steam Controller on a desktop client drives the host's
virtual hid-steam pad end-to-end:

- Set SDL's HIDAPI Steam hints (SDL_JOYSTICK_HIDAPI_STEAMDECK / _STEAM) before
  init so SDL opens Valve devices directly (paddles + both trackpads + gyro as
  first-class SDL gamepad inputs).
- Detect the Deck/SC by VID/PID (0x28DE + 0x1205 / 0x1102 / 0x1142) ->
  GamepadPref::SteamDeck (there is no SDL gamepad type for it), so the host
  builds the virtual Deck with the right identity.
- Map the SDL paddle + Misc1 buttons -> BTN_PADDLE1..4 / BTN_MISC1 (a free win
  for Xbox Elite paddles too).
- Route a SECOND touchpad -> RichInput::TouchpadEx (SDL touchpad 0 = left ->
  surface 1, 1 = right -> surface 2, signed coords); a single touchpad keeps the
  legacy Touchpad. New forward_touch() helper centralizes the choice.
- Track held touchpad contacts per (surface, finger) and lift them on pad
  switch/detach so a contact held at that moment can't stick.
- Sensor (gyro/accel) capture was already generic across pad types.

Linux client builds + clippy clean; the Windows client is a near-verbatim
mirror (windows CI compiles it). On a Deck in Game Mode, Steam Input still holds
the device — the user disables Steam Input for the client (the Decky UX, next);
on a desktop client (or a Deck with Steam Input off) the hints just work.

Remaining M4: Decky Disable-Steam-Input UX, Apple/Android parity, and the C-ABI
PunktfunkRichInputEx + send_rich_input2 (Apple/embedder send path). Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 01c55aed38 feat(proto/steam): M3 — rich Steam wire (back buttons + 2nd trackpad)
Carry the rich Steam Controller / Steam Deck inputs end-to-end on the wire —
strictly additive + forward-compatible (unknown kinds/bits drop on old peers).

Core (punktfunk-core):
- input.rs: BTN_PADDLE1..4 + BTN_MISC1 in Moonlight's buttonFlags2<<16 namespace
  (so the GameStream paddle path and native grips share one host injector map;
  Steam L4/L5/R4/R5 reuse the four Xbox-Elite paddle slots).
- quic.rs: RichInput::TouchpadEx (kind 0x03 — surface 0/1/2, touch+click, signed
  coords, pressure; the second trackpad the single Touchpad can't express) and
  HidOutput::TrackpadHaptic (kind 0x04 — the SC voice-coil pulse). Round-tripped.
- abi.rs: PUNKTFUNK_GAMEPAD_STEAMDECK=6 / _STEAMCONTROLLER=5, the paddle bits,
  RICH_TOUCHPAD_EX / HIDOUT_TRACKPAD_HAPTIC constants. from_hid packs
  TrackpadHaptic into the existing which + effect[0..6] — the legacy structs do
  NOT grow (guarded by new size_of==20/19 asserts); GamepadPref lockstep +
  paddle-bit lockstep asserts extended. include/punktfunk_core.h regenerated.

Host (punktfunk-host):
- steam_proto::from_gamepad maps the wire paddles -> the four Deck grips + QAM;
  apply_rich routes TouchpadEx left/right -> the matching pad.
- every DualSense/DS4 manager (Linux + Windows) gained a TouchpadEx arm
  (surface 0/2 -> its one touchpad; surface 1 ignored) so the variant compiles
  everywhere and a Steam client streaming to a DS host keeps its right pad.
- the xpad BUTTON_MAP finally consumes the GameStream paddle bits
  (BTN_TRIGGER_HAPPY5-8) — Sunshine/Moonlight paddle clients were silently
  no-op'd before (design §5.6).
- Android feedback: drop TrackpadHaptic (no coils; rumble rides 0xCA).

Validated on-box: the ignored backend test now drives the full wire path —
from_gamepad (BTN_A + the L4 grip) + apply_rich (a left-pad TouchpadEx) reach the
evdev as BTN_A + ABS_HAT0X=-8000. Wire round-trips + paddle/TouchpadEx mapping
unit-tested. Workspace clippy/fmt/test green. Not pushed.

Deferred to M4: the C-ABI PunktfunkRichInputEx + send_rich_input2 (only the
Apple/embedder *send* path needs it; the host decodes TouchpadEx today).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 95308d352b feat(host/steam): M2 — virtual Steam Deck as a wired PadBackend (Linux)
Make the virtual hid-steam device a selectable per-session host gamepad,
end-to-end on Linux: PUNKTFUNK_GAMEPAD=steamdeck now builds a
SteamControllerManager that creates a /dev/uhid 28DE:1205 Deck, enters
gamepad_mode, and feeds the byte-exact Deck report (M1).

- inject/linux/steam_controller.rs: SteamControllerManager / SteamDeckPad,
  mirroring dualsense.rs (open/create2, GET/SET_REPORT pump, heartbeat, RAII
  destroy). Two Steam-specific quirks beyond the DualSense path:
    * gamepad_mode entry — best-effort `lizard_mode=0` via sysfs, plus a b9.6
      creation pulse (MODE_ENTER) so steam_do_deck_input_event stops
      early-returning, plus an anti-toggle guard (MENU_HOLD_CAP) so a long
      in-game Start-hold can't flip gamepad_mode back off.
    * UHID_SET_REPORT answered err=0 (DualSense omits it; the kernel stalls
      ~5s/cmd otherwise); the 0xEB rumble report parsed onto the 0xCA plane.
- core config.rs: GamepadPref::SteamDeck (wire byte 6) + SteamController
  (byte 5, reserved — folds to Xbox360 until its backend lands); from_u8 /
  from_name / as_str. Forward-compatible (unknown byte -> Auto); the C-ABI
  PUNKTFUNK_GAMEPAD_* constants stay M3, so no generated-header drift.
- punktfunk1.rs: PadBackend::SteamDeck variant + select / handle / apply_rich
  / pump / heartbeat arms; pick_gamepad Linux arm.

On-box: an #[ignore]d backend test (backend_binds_and_input_flows) drives the
real SteamDeckPad — it binds hid-steam (gamepad + IMU evdevs), enters gamepad
mode, BTN_A reaches the evdev, and the device tears down on drop. Workspace
clippy/fmt/test green. Not pushed. Next: M3 (protocol/ABI wire) + M4 (client
capture).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 9ff7d41bfe feat(host/steam): M1 — byte-exact Deck input serializer, on-box validated
Flesh out inject/proto/steam_proto.rs into the full Steam Deck HID contract,
transcribed verbatim from the kernel steam_do_deck_input_event /
steam_do_deck_sensors_event and validated field-for-field against kernel 7.0:

- SteamState: the u64 button map (bytes 8..16), sticks/triggers/trackpads/IMU
  stored as raw little-endian report values; serialize_deck_state is a pure,
  byte-exact memcpy into the 64-byte unnumbered frame.
- from_gamepad (XInput frame -> Deck buttons/sticks/triggers) + apply_rich
  (RichInput touchpad -> right pad, motion -> IMU).
- parse_steam_output: the 0xEB ID_TRIGGER_RUMBLE_CMD feedback -> (low, high)
  for the universal rumble plane.
- serial_reply fixed: prepend the report-id-0 byte the kernel strips
  (steam_recv_report does memcpy(data, buf+1, ...)); M0's reply lacked it, so
  the kernel fell back to the "XXXXXXXXXX" serial.
- SteamModel (Deck now; classic Controller later), command/feature IDs.

The spike is repurposed as the M1 validator: it pulses the b9.6 mode-switch to
enter gamepad_mode (steam_do_deck_input_event early-returns under the default
lizard_mode otherwise), then holds a known test pattern. Reading both evdevs via
EVIOCGABS/EVIOCGKEY, every field matched: ABS_X/Y/RX/RY (incl. the kernel
Y-negation), both triggers, the touched right-pad HAT1X/Y, the IMU accel/gyro
(with ABS_Z/RZ negations), and the 6 expected buttons incl. the L4/R5 grips.

5 unit tests + workspace clippy/fmt/test green. Next: M2 (SteamControllerManager
UHID backend + PadBackend wiring). Not pushed — pipeline not yet shippable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 2b47d8cc28 feat(host/steam): M0 — virtual hid-steam UHID device binds + parses (Linux)
Greenfield virtual Steam Deck controller, the Steam analogue of the shipped
virtual DualSense. Proves the kernel hid-steam driver binds a /dev/uhid
28DE:1205 device, registers it as a real Steam Deck, and parses our input
reports — the go/no-go gate for the full Steam Controller/Deck pipeline.

- inject/proto/steam_proto.rs: keeper module — the vendor HID descriptor (one
  feature report, the sole thing steam_is_valve_interface() checks), the
  command/feature IDs, serialize_deck_state, and the serial GET_REPORT reply.
  Unit-tested.
- src/bin/steam_uhid_spike.rs: throwaway M0 spike (Linux-only) — opens
  /dev/uhid, creates the device, services the handshake including
  UHID_SET_REPORT (which the DualSense backend omits and which hid-steam
  stalls ~5s/cmd without), and heartbeats a neutral report.
- design/steam-controller-deck-support.md: full design + M0–M7 plan; the two
  walls (Steam Input capture ownership; virtual-Steam recognition) and the
  fidelity ceiling. Status: M0 GREEN.

On-box (headless Ubuntu 26.04, kernel 7.0, no Steam): journalctl -k shows
hid-steam binding the device (rebind off hid-generic), "Steam Controller
connected", and the kernel creating BOTH a "Steam Deck" gamepad evdev and a
"Steam Deck Motion Sensors" IMU evdev (INPUT_PROP_ACCELEROMETER). A
layout-agnostic mash-probe drove 23 distinct BTN_* codes through
hid-steam -> evdev, proving the input-report parse path. M1 line-checks the
exact per-bit report layout against the lab kernel.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 7cd9364c9e style(host): rustfmt the #9/#13 pairing edits
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 3e498cd40d docs(security): #9/#13 fixed, S7 rationale corrected (red-team follow-up)
17/18 now fixed. A red-team of the three accepted findings showed #9 and #13
rested on a circular premise (each was the other's "safe fallback") and S7's
written rationale was wrong (signing exercises the same modexp Marvin targets).
#9/#13 closed; S7 accept retained for the corrected reasons + amplifier hardened.
See f0574a5, f6c9576.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 60de506f66 fix(host/gamestream): correct the rsa-Marvin (S7) rationale + cap pairing signatures
Red-team found the .cargo/audit.toml justification for RUSTSEC-2023-0071 was
materially wrong: it claimed "Marvin targets decryption, so the vulnerable path
isn't exercised" — but the advisory is a variable-time modexp of the secret
exponent, which RSA *signing* (signing_key.sign) also runs. The accept is still
correct, for the RIGHT reasons (no decryption/padding oracle; the signed
serversecret is host-random not attacker-chosen; signing is operator-PIN-gated;
GameStream is off by default and the native QUIC plane uses rustls, not rsa;
Moonlight mandates RSA-2048 so the GameStream key can't move off it). Rewrite
the rationale accordingly.

Also shut the timing-sample amplifier the review surfaced: the pairing session
was never marked after phase 3, so a peer past phase 1 could loop phase2/phase3
to harvest many RSA signing-time samples. Sign exactly once per ceremony
(reject a repeated serverchallengeresp).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 2865368771 fix(host/pairing): close native-pairing DoS findings #9 + #13 (red-team follow-up)
The accepts for #9 (PIN-window burn) and #13 (knock-queue flood) rested on a
circular premise — each cited the other as the safe fallback — and a re-review
showed one LAN attacker could defeat BOTH, denying all onboarding. Close them:

- #13 per-source-IP cap on the pending-knock queue (MAX_PENDING_PER_IP) so one
  host can't fill/evict the 32-slot queue (QUIC validates the source address);
  and eviction now NEVER drops a live *parked* knock (a held-open connection
  awaiting operator approval), so a cert-rotating flood can't evict the genuine
  device being onboarded. This makes the delegated-approval path genuinely
  flood-resistant — restoring the validity of #9's "use delegated approval on
  hostile LANs" fallback.

- #9 fingerprint-bindable PIN window: `NativePairing::arm_for(ttl, Some(fp))`
  binds the window to one operator-selected device; `pin_for_attempt` returns
  `BoundToOther` for any other fingerprint, which the QUIC pair path rejects
  WITHOUT consuming the window — so an unpaired peer can neither pair nor BURN a
  window armed for a specific device (it can't forge the bound fingerprint). The
  mgmt `POST /native/pair/arm` gains an optional `fingerprint` (from a pending
  knock); unbound arming keeps the legacy any-device behavior (trusted-LAN).
  (Web-console "pair this pending device with a PIN" UX is a follow-up; the
  flood-resistant knock path above is the immediate hostile-LAN onboarding path.)

+ regression tests (armed_pin_is_fingerprint_bindable,
  pending_per_ip_cap_and_parked_protection); api/openapi.json regenerated.
110 host tests + clippy + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 6e2e946bc9 docs(security): #5 fixed + on-box validated (RTX box, 2026-06-29)
15/18 now fixed; no finding remains open and actionable. SDDL scoped to
SYSTEM+LocalService, validated live (6943-frame DualSense+IDD session works;
non-SYSTEM OpenFileMapping now ACCESS_DENIED). See e59fa60.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler b5f02000d6 fix(host/security): scope Windows shared-section SDDL to SYSTEM+LocalService (close #5)
The gamepad host<->UMDF-driver shared sections (Global\pfds-shm-*, pfxusb-shm-*)
and the IDD-push frame ring/event (Global\pfvd-*) were created with
`D:(A;;GA;;;WD)` — GENERIC_ALL to **Everyone** — on the assumption the driver's
WUDFHost ran under a restricted token needing broad access. So any local
unprivileged user could OpenFileMapping the section to inject controller input,
tamper the trusted HID channel, or read captured screen frames
(security-review 2026-06-28 #5).

On-box validation (RTX box, 2026-06-29) disproved the restricted-token premise:
the WUDFHost token is NT AUTHORITY\LocalService (S-1-5-19), SYSTEM integrity,
with ZERO restricted SIDs. So the section only needs SYSTEM (the host creates +
writes it) and LocalService (the driver opens it). Scope both SDDL sites to
`D:(A;;GA;;;SY)(A;;GA;;;LS)`; rename the now-misnamed `permissive_sa` ->
`shared_object_sa`; correct the stale "restricted-token / Everyone" docs.

Validated live: a full DualSense + 1280x720x60 session — 6943 frames received,
HID output round-tripped, device status OK (pf_dualsense + pf_vdisplay WUDFHosts
both LocalService open the scoped sections fine), while OpenFileMapping from a
non-SYSTEM admin session now returns ACCESS_DENIED (was a granted handle under
WD). Host-only change (the SDDL is set when the host CREATES the section);
drivers unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler fe562f0562 chore(apple): rename app display name to "Punktfunk"
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
release / apple (push) Has been cancelled
android / android (push) Has been cancelled
ci / rust (push) Has been cancelled
decky / build-publish (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
CFBundleDisplayName was "Punktfunkempfänger" across all targets/configs; the
in-app title is already "Punktfunk", so make the home-screen name match. Built
iOS app resolves CFBundleDisplayName = "Punktfunk".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:34:16 +02:00
enricobuehler 4e00037a89 feat(apple): stage-2 default + pixel-perfect, decode robustness, UI/rumble polish
apple / swift (push) Successful in 1m4s
android / android (push) Successful in 4m33s
ci / rust (push) Successful in 5m4s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 59s
deb / build-publish (push) Successful in 3m12s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
release / apple (push) Successful in 8m30s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m45s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m48s
apple / screenshots (push) Successful in 5m43s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m24s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m46s
Stream reliability
- Default to the stage-2 presenter (VTDecompressionSession + CAMetalLayer): it detects
  and recovers a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard
  on a lost HEVC reference frame with no app-side recovery (confirmed Apple limitation).
  Stage 1 is now a DEBUG-only presenter toggle, plus the automatic no-Metal fallback.
- Stage-2 pixel-perfect: render the drawable at the decoded size (shader stays 1:1 =
  identity) and let the layer's contentsGravity scale via the system compositor — the
  same path stage-1's videoGravity used — instead of scaling in-shader.
- Loss recovery in both pumps is now a persistent awaitingIDR want, retried until an IDR
  actually lands, so a keyframe request swallowed by the throttle can't strand a frozen
  frame; 100 ms keyframe throttle to match the Android path.
- Fix "Publishing changes from within view updates": defer the HostStore writes out of
  the .onChange(of: model.phase) callback.
- Move AVAudioSession setActive/setCategory off the main thread (async on a shared serial
  queue) to stop the UI-stall warning.

Controllers
- Rumble: capped-exponential backoff when the gamecontrollerd.haptics XPC breaks (-4811)
  so a transient server interruption self-heals instead of cascading; playsHapticsOnly so
  a controller engine doesn't join the always-active streaming audio session.
- Host cards: iPad pointer "magnet" hover effect; iPhone press scale + light haptic.

UI / design
- Ship Geist (SIL OFL 1.1) as the app font (bundled OTFs + registration), with the
  license surfaced in Acknowledgements.
- Restructure iOS/iPadOS Settings into a category NavigationSplitView; resolution wheel
  with custom-resolution entry; 10-bit HDR toggle in Display.
- Industrial host-card redesign (left-aligned, bold, brand monogram tiles).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:26:10 +02:00
enricobuehler 46b9aa8cf0 fix(windows-host): IDD activation — resolve-first, force-EXTEND only as fallback
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 4m23s
ci / rust (push) Successful in 5m9s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 57s
apple / screenshots (push) Successful in 5m33s
windows-host / package (push) Successful in 8m46s
deb / build-publish (push) Successful in 3m13s
decky / build-publish (push) Successful in 27s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m51s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m15s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m19s
docker / deploy-docs (push) Successful in 21s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m44s
force_extend_topology() was added before the resolve loop to de-clone a fresh IDD on
integrated-screen boxes (laptops), but its bare SDC_TOPOLOGY_EXTEND preset is
ACCESS_DENIED from the Session-0 service context on a HEADLESS box and broke the IDD
auto-activation there: resolve_gdi_name stayed None -> "not an active display path" ->
black screen. That regressed the headless/primary platform (live RTX box).

Revert to the proven e2c9bfd flow: resolve FIRST (Windows auto-activates the IDD as its
own extended path), and force-EXTEND only as the FALLBACK when resolve returns None (the
integrated-screen clone case, observed live to leave resolve None). The success path is
byte-identical to e2c9bfd (resolve -> set_active_mode -> isolate_displays_ccd).

Validated live: the headless RTX box streams again (probe: frames flow, driver attaches
to the ring, host/driver render LUIDs match).

Reviewed multi-agent + adversarial: no regression on the validated headless path or the
observed Optimus-laptop clone path (a cloned IddCx target resolves to None there, so the
is_none() fallback fires + de-clones). Known theoretical caveat, documented inline and
unobserved for IddCx but untested across GPU/driver/OS: a CCD clone that manifests as a
shared-source ACTIVE path would resolve to Some and bypass the is_none() gate. Follow-up:
widen the gate (a target_is_cloned helper) once an integrated-screen box is available to
validate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 16:15:48 +02:00
enricobuehler 372b27540b fix(apple): render Acknowledgements notices in lazy chunks
apple / swift (push) Successful in 1m11s
android / android (push) Successful in 4m19s
ci / web (push) Successful in 51s
ci / rust (push) Successful in 5m14s
ci / docs-site (push) Successful in 58s
release / apple (push) Successful in 7m54s
deb / build-publish (push) Successful in 4m0s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
ci / bench (push) Successful in 5m19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
apple / screenshots (push) Successful in 5m37s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m26s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m35s
docker / deploy-docs (push) Successful in 18s
THIRD-PARTY-NOTICES.txt is ~885 KB / 16k lines; rendering it in a single
SwiftUI Text overshot the text-rendering height limit — it laid out for ages
and drew blank below the cutoff (only the small punktfunk licenses above it
showed). Split the notices into ~80 line-chunks (<=200 lines / <=18 KB each,
computed once as Licenses.thirdPartyNoticesChunks) and render them in a
top-level LazyVStack so only on-screen chunks lay out and no chunk is tall
enough to clip. Chunking is lossless — rejoining the chunks reproduces the
original byte-for-byte, so no notice text is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:34:19 +02:00
enricobuehler db4d15bf8b fix(apple): stop the iOS/iPadOS Add Host sheet from scrolling
apple / swift (push) Successful in 1m5s
android / android (push) Successful in 4m15s
ci / rust (push) Successful in 4m56s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 58s
deb / build-publish (push) Successful in 3m11s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
release / apple (push) Successful in 8m32s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m40s
apple / screenshots (push) Successful in 5m44s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m29s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m31s
docker / deploy-docs (push) Successful in 17s
The add-host content is a SwiftUI Form (backed by a scrollable list), so it
bounced/scrolled inside the fixed .height(320) detent even though the three
rows + action button fit exactly. Lock it with .scrollDisabled(true) on iOS
(covers iPadOS); macOS (fixed-size panel) and tvOS (custom rows, no Form) are
untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:12:08 +02:00
enricobuehler 8e24ea9ed7 fix(ci): archive Apple release builds with Automatic signing
The in-app OSS license screens (7591425) added a `resources:` array to the
PunktfunkKit SwiftPM target, which makes SwiftPM emit a resource-bundle target
(PunktfunkKit_PunktfunkKit). A resource bundle is a product type that cannot
carry a provisioning profile, so the explicit PROVISIONING_PROFILE_SPECIFIER
each release.yml archive step set — global on macOS, sdk-scoped on iOS/tvOS —
now lands on it and fails the archive ("does not support provisioning profiles")
on all three platforms. (Before that commit there was no resource bundle, so the
profile was harmless.)

Switch all three archive steps to CODE_SIGN_STYLE=Automatic (development):
Automatic signing assigns a profile only to the app target and leaves the
resource bundle (and the macOS-host SwiftPM macro plugins) alone, and bakes the
sandbox entitlements in. No -allowProvisioningUpdates, so it stays offline and
never cloud-signs (the App-Manager ASC key can't). DISTRIBUTION signing is
unchanged — still manual, in the -exportArchive step (which maps the profile to
io.unom.punktfunk only). Drops the now-unneeded manual signing xcconfigs.

Requires the runner to have a development provisioning profile for
io.unom.punktfunk on each platform (now installed for macOS/iOS/tvOS).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:12:08 +02:00
enricobuehler 73c0125843 fix(mgmt): regenerate api/openapi.json for 0.3.0
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 4m16s
ci / rust (push) Successful in 5m5s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 58s
apple / screenshots (push) Successful in 5m17s
deb / build-publish (push) Successful in 3m11s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / bench (push) Successful in 4m43s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 26s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m28s
docker / deploy-docs (push) Successful in 17s
The OpenAPI 'info.version' tracks CARGO_PKG_VERSION; the 0.3.0 bump made the
checked-in spec stale (the openapi_document_is_complete_and_checked_in test).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 07:54:30 +00:00
143 changed files with 11591 additions and 5884 deletions
+20 -6
View File
@@ -13,11 +13,25 @@
[advisories] [advisories]
ignore = [ ignore = [
# rsa "Marvin Attack" a timing sidechannel in RSA *decryption* (PKCS#1 v1.5 padding oracle). # rsa "Marvin Attack" (RUSTSEC-2023-0071): a timing side-channel in the rsa crate's variable-time
# There is NO fixed rsa release (the constant-time rewrite is still unreleased upstream), and rsa # modular exponentiation of the SECRET exponent. IMPORTANT — this affects the RSA private-key op in
# is required for GameStream/Moonlight pairing. Crucially, the host uses rsa ONLY for PKCS#1 v1.5 # general, INCLUDING signing (m^d mod n), which the host DOES perform (gamestream/pairing.rs
# SIGNING / VERIFYING (gamestream/cert.rs + gamestream/pairing.rs: SigningKey / VerifyingKey / # `signing_key.sign(&serversecret)`). It is NOT, as an earlier version of this note wrongly claimed,
# Signer / Verifier) — it never performs RSA decryption, which is the operation Marvin targets. # limited to decryption — so "the vulnerable path isn't exercised" is false; signing exercises it.
# So the vulnerable code path is not exercised. Revisit if a fixed rsa ships or we add RSA decrypt. # We accept it because the attack is not practically reachable here, NOT because the path is unused:
# * No RSA decryption / PKCS#1v1.5 padding oracle exists anywhere (every `decrypt` in the tree is
# AES/AES-GCM), so the classic Bleichenbacher/Marvin chosen-ciphertext oracle is absent.
# * The only signed message (`serversecret`) is HOST-generated random, never attacker-chosen — so
# there's no adaptive chosen-input probing (the lever remote RSA-timing key recovery needs); and
# signing is gated behind the operator-entered pairing PIN, ONE signature per ceremony (a
# repeated phase-3 is rejected — gamestream/pairing.rs — to deny a passive timing-sample harvester).
# * GameStream is OFF by default (bare `serve` is native-only); the secure native QUIC plane uses
# rustls' constant-time backend, NOT the rsa crate. RSA is touched only on the opt-in,
# trusted-LAN GameStream/Moonlight pairing handshake. Moonlight mandates RSA-2048, so the
# GameStream identity cannot move to Ed25519/ECDSA (only the native identity could, and it
# already avoids the rsa crate).
# There is NO fixed rsa release (the constant-time rewrite is still unreleased upstream). Revisit if:
# a constant-time rsa ships (then drop this), the host ever signs an attacker-chosen message with
# this key, or any RSA decryption / key-transport using the private key is added.
"RUSTSEC-2023-0071", "RUSTSEC-2023-0071",
] ]
+40 -48
View File
@@ -207,10 +207,20 @@ jobs:
# (Config/Punktfunk-macOS.entitlements) — mandatory for the Mac App Store. # (Config/Punktfunk-macOS.entitlements) — mandatory for the Mac App Store.
continue-on-error: true continue-on-error: true
run: | run: |
# Separate archive from the Developer ID one above: App Store needs a profile-signed # Separate archive from the Developer ID one above: App Store needs a signed, entitled
# archive (manual signing), not the unsigned-then-codesign DMG path. Same App-Manager # archive that -exportArchive can re-sign for distribution, not the unsigned-then-codesign
# ASC-key constraint as iOS/tvOS — MANUAL signing, NOT -allowProvisioningUpdates # DMG path. Archive with AUTOMATIC signing (development). Why not a manually-specified
# (cloud signing the key can't do). Quit Xcode so it can't prune the dropped profile. # profile (as this step used to do): the in-app license screens added a SwiftPM resource
# bundle (PunktfunkKit_PunktfunkKit), and a resource bundle is a product type that cannot
# carry a provisioning profile — a global PROVISIONING_PROFILE_SPECIFIER (here) or an
# sdk-scoped one (iOS/tvOS) lands on it and fails the archive ("does not support
# provisioning profiles"). Automatic signing assigns a profile only to the app and leaves
# the resource bundle (and the macOS-host macro plugins) alone, and bakes the sandbox
# entitlements in. No -allowProvisioningUpdates → it stays OFFLINE and never cloud-signs
# (the App-Manager ASC key can't), so the runner must have a macOS *development* profile
# for io.unom.punktfunk installed. DISTRIBUTION signing happens in the export step below
# (manual, via the plist). Quit Xcode so it can't prune the manually-installed App Store
# distribution profile that export needs.
osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true
pkill -x Xcode 2>/dev/null || true pkill -x Xcode 2>/dev/null || true
PROFILE="Punktfunk macOS App Store Distribution" PROFILE="Punktfunk macOS App Store Distribution"
@@ -218,11 +228,10 @@ jobs:
-project "$PROJECT" -scheme Punktfunk \ -project "$PROJECT" -scheme Punktfunk \
-destination 'generic/platform=macOS' \ -destination 'generic/platform=macOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-macos-appstore.xcarchive" \ -archivePath "$RUNNER_TEMP/Punktfunk-macos-appstore.xcarchive" \
-skipMacroValidation -skipPackagePluginValidation \
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \ MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \
CODE_SIGN_STYLE=Manual \ CODE_SIGN_STYLE=Automatic \
CODE_SIGN_IDENTITY="Apple Distribution" \ DEVELOPMENT_TEAM="$TEAM_ID"
DEVELOPMENT_TEAM="$TEAM_ID" \
PROVISIONING_PROFILE_SPECIFIER="$PROFILE"
cat > "$RUNNER_TEMP/export-macos-appstore.plist" <<EOF cat > "$RUNNER_TEMP/export-macos-appstore.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
@@ -252,35 +261,27 @@ jobs:
# Best-effort until the App Store Connect app record for io.unom.punktfunk exists. # Best-effort until the App Store Connect app record for io.unom.punktfunk exists.
continue-on-error: true continue-on-error: true
run: | run: |
# MANUAL App Store signing: the local (valid) Apple Distribution identity + the App # Archive with AUTOMATIC signing (development) — see the macOS App Store step for the full
# Store provisioning profile. NOT -allowProvisioningUpdates — with an App-Manager-role # rationale. The SwiftPM resource bundle (PunktfunkKit_PunktfunkKit, added with the in-app
# ASC key that forces Xcode's CLOUD-managed signing, which the role can't do ("Cloud # license screens) builds for iphoneos, so even the sdk-scoped PROVISIONING_PROFILE_SPECIFIER
# signing permission error"). The profile must be installed on the runner under # this step used to set matched it and failed the archive ("does not support provisioning
# ~/Library/Developer/Xcode/UserData/Provisioning Profiles/ (install it once with # profiles"). Automatic signing profiles only the app and leaves the resource bundle (and
# Xcode.app quit, or it prunes the manually-dropped distribution profile). # the macOS-host macro plugins) alone. No -allowProvisioningUpdates → OFFLINE, never
# A running Xcode.app prunes unrecognized profiles from that dir — quit it so the App # cloud-signs (the App-Manager ASC key can't), so the runner needs an iOS *development*
# Store profile survives this build; headless xcodebuild doesn't need the GUI app. # profile for io.unom.punktfunk installed. DISTRIBUTION signing is the export step below
# (manual, via the plist). A running Xcode.app prunes unrecognized profiles — quit it so the
# manually-installed App Store distribution profile survives for export.
osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true
pkill -x Xcode 2>/dev/null || true pkill -x Xcode 2>/dev/null || true
PROFILE="Punktfunk iOS App Store Distribution" PROFILE="Punktfunk iOS App Store Distribution"
# Scope signing to the iOS device SDK via an xcconfig — see the tvOS step below for the
# full rationale. A global (CLI) profile specifier would also be forced onto the shared
# macOS-host SwiftPM macro plugins, which reject it and fail the archive; [sdk=iphoneos*]
# in an xcconfig lands it on the app/framework slices only.
SIGN_XCCONFIG="$RUNNER_TEMP/sign-ios.xcconfig"
cat > "$SIGN_XCCONFIG" <<XCCONF
CODE_SIGN_STYLE = Manual
DEVELOPMENT_TEAM = $TEAM_ID
CODE_SIGN_IDENTITY[sdk=iphoneos*] = Apple Distribution
PROVISIONING_PROFILE_SPECIFIER[sdk=iphoneos*] = $PROFILE
XCCONF
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild archive \ DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild archive \
-project "$PROJECT" -scheme Punktfunk-iOS \ -project "$PROJECT" -scheme Punktfunk-iOS \
-destination 'generic/platform=iOS' \ -destination 'generic/platform=iOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-ios.xcarchive" \ -archivePath "$RUNNER_TEMP/Punktfunk-ios.xcarchive" \
-skipMacroValidation -skipPackagePluginValidation \ -skipMacroValidation -skipPackagePluginValidation \
-xcconfig "$SIGN_XCCONFIG" \ MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" CODE_SIGN_STYLE=Automatic \
DEVELOPMENT_TEAM="$TEAM_ID"
cat > "$RUNNER_TEMP/export-appstore.plist" <<EOF cat > "$RUNNER_TEMP/export-appstore.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
@@ -312,33 +313,24 @@ jobs:
# on the runner (xcodebuild -downloadPlatform tvOS). # on the runner (xcodebuild -downloadPlatform tvOS).
continue-on-error: true continue-on-error: true
run: | run: |
# Same manual App Store signing as iOS (the App-Manager ASC key can't cloud-sign). # Archive with AUTOMATIC signing (development) — see the macOS App Store step. The SwiftPM
# resource bundle (PunktfunkKit_PunktfunkKit) builds for appletvos and rejected the
# sdk-scoped profile this step used to set; Automatic signing profiles only the app and
# leaves the resource bundle + the macOS-host macro plugins (OnceMacro/SwizzlingMacro/
# AssociationMacro) alone. No -allowProvisioningUpdates → OFFLINE, never cloud-signs (the
# App-Manager ASC key can't), so the runner needs a tvOS *development* profile for
# io.unom.punktfunk installed. DISTRIBUTION signing is the export step below (manual, plist).
osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true
pkill -x Xcode 2>/dev/null || true pkill -x Xcode 2>/dev/null || true
PROFILE="Punktfunk tvOS App Store Distribution" PROFILE="Punktfunk tvOS App Store Distribution"
# Scope signing to the tvOS device SDK via an xcconfig. A global (CLI) profile specifier
# hits EVERY target, including the shared SwiftPM macro plugins (OnceMacro/SwizzlingMacro/
# AssociationMacro) which build for the macOS host and reject a provisioning profile
# ("<macro> does not support provisioning profiles"), failing the archive. Conditionals
# work only in an xcconfig (xcodebuild mis-parses a CLI "SETTING[sdk=..]=val"), and a
# command-line -xcconfig outranks target settings, so [sdk=appletvos*] puts the profile on
# the app/framework slices only — the macosx-host macros get nothing. (The macOS archive
# above is immune: its host-SDK macros are CODE_SIGNING_ALLOWED=NO, so a global specifier
# is ignored there.)
SIGN_XCCONFIG="$RUNNER_TEMP/sign-tvos.xcconfig"
cat > "$SIGN_XCCONFIG" <<XCCONF
CODE_SIGN_STYLE = Manual
DEVELOPMENT_TEAM = $TEAM_ID
CODE_SIGN_IDENTITY[sdk=appletvos*] = Apple Distribution
PROVISIONING_PROFILE_SPECIFIER[sdk=appletvos*] = $PROFILE
XCCONF
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild archive \ DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild archive \
-project "$PROJECT" -scheme Punktfunk-tvOS \ -project "$PROJECT" -scheme Punktfunk-tvOS \
-destination 'generic/platform=tvOS' \ -destination 'generic/platform=tvOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-tvos.xcarchive" \ -archivePath "$RUNNER_TEMP/Punktfunk-tvos.xcarchive" \
-skipMacroValidation -skipPackagePluginValidation \ -skipMacroValidation -skipPackagePluginValidation \
-xcconfig "$SIGN_XCCONFIG" \ MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" CODE_SIGN_STYLE=Automatic \
DEVELOPMENT_TEAM="$TEAM_ID"
cat > "$RUNNER_TEMP/export-tvos.plist" <<EOF cat > "$RUNNER_TEMP/export-tvos.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+19 -5
View File
@@ -144,11 +144,25 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
`test-loopback.sh` (Swift client vs synthetic punktfunk1-hosts on loopback — runs on macOS; `test-loopback.sh` (Swift client vs synthetic punktfunk1-hosts on loopback — runs on macOS;
includes the pairing ceremony + `--require-pairing` gate), includes the pairing ceremony + `--require-pairing` gate),
`RemoteFirstLightTests` (full pipeline over the LAN). See `RemoteFirstLightTests` (full pipeline over the LAN). See
[`clients/apple/README.md`](clients/apple/README.md). **Stage 2 presenter** [`clients/apple/README.md`](clients/apple/README.md). **Stage 2 presenter is now the DEFAULT**
(`VTDecompressionSession` + `CAMetalLayer`) is built and live-validated on glass behind the opt-in (stage-1 is the Metal-unavailable / DEBUG fallback): explicit `VTDecompressionSession` decode →
`punktfunk.presenter` flag (~11 ms p50 capture→present), to become the default after a few `CAMetalLayer`, presented from the hosting view's **main-runloop `CADisplayLink`** (`renderTick` pops
resolution/HDR checks. Next: make stage 2 the default, glass-to-glass numbers via the newest ready frame per vsync; macOS `displaySyncEnabled = false` is the real fullscreen-judder fix,
`tools/latency-probe`, iOS/iPadOS/tvOS variants. ~11 ms p50). *(An off-main `CAMetalDisplayLink` and an off-main blocking-render present thread were
both tried and reverted — both measured slower on macOS and iPad.)* **HDR fixed**
(`design/apple-stage2-presenter.md`): the "too bright" bug was a missing reference-white anchor — the
fix keeps the PQ-passthrough shader and adds `CAEDRMetadata.hdr10(…, opticalOutputScale: 203)` +
`wantsExtendedDynamicRangeContent` on the layer (on all platforms; the old `#if os(macOS)` guard left
iOS/tvOS EDR half-engaged), routing the 0xCE mastering metadata to the layer (via `setHdrMeta`) instead
of a never-composited source buffer. **Mid-session SDR↔HDR** is handled: `render` reconciles the layer
per-frame from the decoded `frame.isHDR` (per-mode pixel format `bgra8`/`rgba16Float`), so a game
entering HDR mid-stream just reconfigures (last 0xCE grade cached + re-applied; pump drains 0xCE
unconditionally). **4:4:4 added**: decode format is a 2×2 `(chroma, HDR)` matrix
(`420v/x420/444v/x444`, all biplanar so the shaders are unchanged), advertised (`VIDEO_CAP_444`) only
behind a **hardware-required `VTDecompressionSession` probe** (`Stage444Probe`, validated on M3) with a
Settings opt-out + a bounded pump backstop for an undecodable 4:4:4 session. *HDR brightness + 4:4:4
still need on-glass validation (Windows-HDR / `PUNKTFUNK_444` host).* Next: glass-to-glass numbers via
`tools/latency-probe`.
**Linux stage 1 done, first light 2026-06-12** (`clients/linux`, binary **Linux stage 1 done, first light 2026-06-12** (`clients/linux`, binary
`punktfunk-client`): GTK4/libadwaita shell linking `punktfunk-core` directly (no C ABI; `punktfunk-client`): GTK4/libadwaita shell linking `punktfunk-core` directly (no C ABI;
`NativeClient` is now `Sync` — mutexed plane receivers), mDNS host list, TOFU + SPAKE2 `NativeClient` is now `Sync` — mutexed plane receivers), mDNS host list, TOFU + SPAKE2
Generated
+24
View File
@@ -2331,6 +2331,17 @@ version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441" checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441"
[[package]]
name = "num-derive"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed3955f1a9c7c0c15e092f9c887db08b1fc683305fdf6eb6684f22555355e202"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]] [[package]]
name = "num-integer" name = "num-integer"
version = "0.1.46" version = "0.1.46"
@@ -2839,12 +2850,14 @@ dependencies = [
"tracing", "tracing",
"tracing-subscriber", "tracing-subscriber",
"ureq", "ureq",
"usbip-sim",
"utoipa", "utoipa",
"utoipa-axum", "utoipa-axum",
"utoipa-scalar", "utoipa-scalar",
"wasapi", "wasapi",
"wayland-backend", "wayland-backend",
"wayland-client", "wayland-client",
"wayland-protocols",
"wayland-protocols-misc", "wayland-protocols-misc",
"wayland-protocols-wlr", "wayland-protocols-wlr",
"wayland-scanner", "wayland-scanner",
@@ -4236,6 +4249,17 @@ dependencies = [
"serde", "serde",
] ]
[[package]]
name = "usbip-sim"
version = "0.8.0"
dependencies = [
"log",
"num-derive",
"num-traits",
"serde",
"tokio",
]
[[package]] [[package]]
name = "utf8_iter" name = "utf8_iter"
version = "1.0.4" version = "1.0.4"
+3
View File
@@ -3,6 +3,7 @@ resolver = "2"
members = [ members = [
"crates/punktfunk-core", "crates/punktfunk-core",
"crates/punktfunk-host", "crates/punktfunk-host",
"crates/punktfunk-host/vendor/usbip-sim",
"crates/pf-driver-proto", "crates/pf-driver-proto",
"clients/probe", "clients/probe",
"clients/linux", "clients/linux",
@@ -11,6 +12,8 @@ members = [
"tools/latency-probe", "tools/latency-probe",
"tools/loss-harness", "tools/loss-harness",
] ]
# Standalone PoC (built on its own; pulls usbip/tokio/libusb we don't want in the workspace).
exclude = ["packaging/linux/steam-deck-gadget/usbip-poc"]
[workspace.package] [workspace.package]
version = "0.3.0" version = "0.3.0"
+9 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0", "name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0" "identifier": "MIT OR Apache-2.0"
}, },
"version": "0.0.1" "version": "0.3.0"
}, },
"paths": { "paths": {
"/api/v1/clients": { "/api/v1/clients": {
@@ -1354,6 +1354,14 @@
"type": "object", "type": "object",
"description": "Arm-native-pairing request body.", "description": "Arm-native-pairing request body.",
"properties": { "properties": {
"fingerprint": {
"type": [
"string",
"null"
],
"description": "Optional: bind the window to ONE device fingerprint (hex SHA-256, e.g. from a pending knock).\nWhen set, only a pairing attempt from that fingerprint consumes the window — so an unpaired\nLAN peer can neither pair nor burn a window armed for a specific device (security-review #9).\nOmit for an unbound window (any device may use the PIN — trusted-LAN only).",
"example": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
},
"ttl_secs": { "ttl_secs": {
"type": [ "type": [
"integer", "integer",
@@ -175,9 +175,9 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
status = "Connecting to $targetHost:$targetPort" status = "Connecting to $targetHost:$targetPort"
discovery.stop() // free the Wi-Fi radio before the stream session discovery.stop() // free the Wi-Fi radio before the stream session
scope.launch { scope.launch {
// Advertise HDR only when this device's display can present it (else the host sends a // Advertise HDR only when the user enabled it AND this device's display can present it
// proper SDR stream rather than PQ the panel would mis-tone-map). // (else the host sends a proper SDR stream rather than PQ the panel would mis-tone-map).
val hdrEnabled = displaySupportsHdr(context) val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
// "Automatic" resolves to a concrete pad type from the connected controller's VID/PID // "Automatic" resolves to a concrete pad type from the connected controller's VID/PID
// (Android exposes no controller-type enum) — parity with the Linux/Apple clients. An // (Android exposes no controller-type enum) — parity with the Linux/Apple clients. An
// explicit choice is passed through unchanged. // explicit choice is passed through unchanged.
@@ -224,7 +224,7 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
status = null status = null
discovery.stop() // free the Wi-Fi radio before the (parked) stream session discovery.stop() // free the Wi-Fi radio before the (parked) stream session
scope.launch { scope.launch {
val hdrEnabled = displaySupportsHdr(context) val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
val gamepadPref = Gamepad.resolvePref(settings.gamepad) val gamepadPref = Gamepad.resolvePref(settings.gamepad)
// Pin the advertised fingerprint for a discovered host (defence against an impostor while // Pin the advertised fingerprint for a discovered host (defence against an impostor while
// we wait); a manually-typed host has none, so trust-on-first-use. // we wait); a manually-typed host has none, so trust-on-first-use.
@@ -14,6 +14,13 @@ data class Settings(
val height: Int = 0, val height: Int = 0,
val hz: Int = 0, val hz: Int = 0,
val bitrateKbps: Int = 0, val bitrateKbps: Int = 0,
/**
* Advertise HDR (10-bit BT.2020 PQ) to the host. Default on, but only *effective* on a panel that
* can actually present HDR10 (see [displaySupportsHdr]) — on an SDR display HDR is never
* advertised regardless, so the host sends a proper 8-bit BT.709 stream rather than PQ the panel
* would mis-tone-map. Turning this off forces SDR even on a capable panel.
*/
val hdrEnabled: Boolean = true,
val compositor: Int = 0, val compositor: Int = 0,
val gamepad: Int = 0, val gamepad: Int = 0,
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it /** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
@@ -40,6 +47,7 @@ class SettingsStore(context: Context) {
height = prefs.getInt(K_H, 0), height = prefs.getInt(K_H, 0),
hz = prefs.getInt(K_HZ, 0), hz = prefs.getInt(K_HZ, 0),
bitrateKbps = prefs.getInt(K_BITRATE, 0), bitrateKbps = prefs.getInt(K_BITRATE, 0),
hdrEnabled = prefs.getBoolean(K_HDR, true),
compositor = prefs.getInt(K_COMPOSITOR, 0), compositor = prefs.getInt(K_COMPOSITOR, 0),
gamepad = prefs.getInt(K_GAMEPAD, 0), gamepad = prefs.getInt(K_GAMEPAD, 0),
audioChannels = prefs.getInt(K_AUDIO_CH, 2), audioChannels = prefs.getInt(K_AUDIO_CH, 2),
@@ -54,6 +62,7 @@ class SettingsStore(context: Context) {
.putInt(K_H, s.height) .putInt(K_H, s.height)
.putInt(K_HZ, s.hz) .putInt(K_HZ, s.hz)
.putInt(K_BITRATE, s.bitrateKbps) .putInt(K_BITRATE, s.bitrateKbps)
.putBoolean(K_HDR, s.hdrEnabled)
.putInt(K_COMPOSITOR, s.compositor) .putInt(K_COMPOSITOR, s.compositor)
.putInt(K_GAMEPAD, s.gamepad) .putInt(K_GAMEPAD, s.gamepad)
.putInt(K_AUDIO_CH, s.audioChannels) .putInt(K_AUDIO_CH, s.audioChannels)
@@ -68,6 +77,7 @@ class SettingsStore(context: Context) {
const val K_H = "height" const val K_H = "height"
const val K_HZ = "hz" const val K_HZ = "hz"
const val K_BITRATE = "bitrate_kbps" const val K_BITRATE = "bitrate_kbps"
const val K_HDR = "hdr_enabled"
const val K_COMPOSITOR = "compositor" const val K_COMPOSITOR = "compositor"
const val K_GAMEPAD = "gamepad" const val K_GAMEPAD = "gamepad"
const val K_AUDIO_CH = "audio_channels" const val K_AUDIO_CH = "audio_channels"
@@ -94,6 +94,22 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
options = BITRATE_OPTIONS, options = BITRATE_OPTIONS,
selected = s.bitrateKbps, selected = s.bitrateKbps,
) { kbps -> update(s.copy(bitrateKbps = kbps)) } ) { kbps -> update(s.copy(bitrateKbps = kbps)) }
// HDR is only meaningful on a panel that can present HDR10; on an SDR display the toggle
// is disabled (and HDR is never advertised regardless) so the host doesn't send PQ the
// panel would mis-tone-map. The capability is fixed for the device, so read it once.
val hdrCapable = remember { displaySupportsHdr(context) }
ToggleRow(
title = "HDR",
subtitle = if (hdrCapable) {
"Stream 10-bit HDR (BT.2020 PQ) when the host supports it"
} else {
"This display can't present HDR10 — streams stay SDR"
},
checked = s.hdrEnabled && hdrCapable,
enabled = hdrCapable,
onCheckedChange = { on -> update(s.copy(hdrEnabled = on)) },
)
} }
SettingsGroup("Host") { SettingsGroup("Host") {
@@ -181,24 +197,31 @@ private fun SettingsGroup(title: String, content: @Composable ColumnScope.() ->
} }
} }
/** A title + subtitle on the left, a Switch on the right. */ /** A title + subtitle on the left, a Switch on the right. [enabled] greys out the whole row. */
@Composable @Composable
private fun ToggleRow( private fun ToggleRow(
title: String, title: String,
subtitle: String, subtitle: String,
checked: Boolean, checked: Boolean,
onCheckedChange: (Boolean) -> Unit, onCheckedChange: (Boolean) -> Unit,
enabled: Boolean = true,
) { ) {
// Dim the labels when disabled so the row reads as inactive (the Switch dims itself).
val labelAlpha = if (enabled) 1f else 0.38f
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) { Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Column(Modifier.weight(1f)) { Column(Modifier.weight(1f)) {
Text(title, style = MaterialTheme.typography.bodyLarge) Text(
title,
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurface.copy(alpha = labelAlpha),
)
Text( Text(
subtitle, subtitle,
style = MaterialTheme.typography.bodySmall, style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant, color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = labelAlpha),
) )
} }
Switch(checked = checked, onCheckedChange = onCheckedChange) Switch(checked = checked, onCheckedChange = onCheckedChange, enabled = enabled)
} }
} }
@@ -1,6 +1,7 @@
package io.unom.punktfunk package io.unom.punktfunk
import android.Manifest import android.Manifest
import android.content.pm.ActivityInfo
import android.content.pm.PackageManager import android.content.pm.PackageManager
import android.view.SurfaceHolder import android.view.SurfaceHolder
import android.view.SurfaceView import android.view.SurfaceView
@@ -102,6 +103,13 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
it.systemBarsBehavior = WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE it.systemBarsBehavior = WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
it.hide(WindowInsetsCompat.Type.systemBars()) it.hide(WindowInsetsCompat.Type.systemBars())
} }
// Lock to landscape while streaming — the host streams a landscape desktop, so pin the device
// there (either landscape direction is fine) and stop it rotating to portrait mid-session. The
// activity declares configChanges=orientation, so this re-lays out the surface in place without
// recreating the activity (no stream restart). On TV (fixed landscape) it's a harmless no-op.
// The prior request is captured and restored on the way out.
val priorOrientation = activity?.requestedOrientation
activity?.requestedOrientation = ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE
activity?.streamHandle = handle // route hardware keys to this session activity?.streamHandle = handle // route hardware keys to this session
activity?.axisMapper = Gamepad.AxisMapper(handle) // route joystick axes activity?.axisMapper = Gamepad.AxisMapper(handle) // route joystick axes
// Host→client feedback (rumble + DualSense lightbar/LEDs); poll threads stopped before close. // Host→client feedback (rumble + DualSense lightbar/LEDs); poll threads stopped before close.
@@ -114,6 +122,9 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
activity?.streamHandle = 0L activity?.streamHandle = 0L
controller?.show(WindowInsetsCompat.Type.systemBars()) controller?.show(WindowInsetsCompat.Type.systemBars())
window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON) window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
// Release the landscape lock so the rest of the app follows the device/system again.
activity?.requestedOrientation =
priorOrientation ?: ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED
// Leaving the stream: stop the mic + audio + decode threads and tear down the session. // Leaving the stream: stop the mic + audio + decode threads and tear down the session.
NativeBridge.nativeStopMic(handle) NativeBridge.nativeStopMic(handle)
NativeBridge.nativeStopAudio(handle) NativeBridge.nativeStopAudio(handle)
@@ -314,9 +325,11 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
} }
/** /**
* The live stats overlay — mirrors the Apple client's HUD. Reads the 10-double layout from * The live stats overlay — mirrors the Apple client's HUD. Reads the 14-double layout from
* [NativeBridge.nativeVideoStats]: * [NativeBridge.nativeVideoStats]:
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skew, w, h, hz, dropped]`. * `[fps, mbps, latP50Ms, latP95Ms, latValid, skew, w, h, hz, dropped, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc]`. The trailing four (present on a current native lib) describe the
* negotiated video feed and render as a codec/depth/colour/chroma line; older layouts just omit it.
*/ */
@Composable @Composable
internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) { internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
@@ -338,6 +351,14 @@ internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
fontFamily = FontFamily.Monospace, fontFamily = FontFamily.Monospace,
fontSize = 12.sp, fontSize = 12.sp,
) )
videoFeedLine(s)?.let { feed ->
Text(
feed,
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
if (latValid) { if (latValid) {
val tag = if (skew) "" else " (same-host)" val tag = if (skew) "" else " (same-host)"
Text( Text(
@@ -357,3 +378,31 @@ internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
} }
} }
} }
/**
* Format the negotiated video-feed descriptor from the trailing four stats doubles
* `[bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`, e.g.
* `HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0`. Returns `null` on a pre-video-feed layout (< 14 doubles)
* so the overlay simply omits the line. The codes are CICP / H.273: transfer 16 = PQ, 18 = HLG (else
* SDR); primaries 9 = BT.2020, 1 = BT.709; chroma_format_idc 1 = 4:2:0, 2 = 4:2:2, 3 = 4:4:4. The
* Android decoder is always HEVC (`video/hevc`).
*/
private fun videoFeedLine(s: DoubleArray): String? {
if (s.size < 14) return null
val bitDepth = s[10].toInt()
val primaries = s[11].toInt()
val transfer = s[12].toInt()
val chromaIdc = s[13].toInt()
val depthLabel = if (bitDepth > 0) "$bitDepth-bit" else "8-bit"
val (dynamicRange, colorSpace) = when (transfer) {
16 -> "HDR" to "BT.2020 PQ"
18 -> "HDR" to "BT.2020 HLG"
else -> "SDR" to if (primaries == 9) "BT.2020" else "BT.709"
}
val chromaLabel = when (chromaIdc) {
3 -> "4:4:4"
2 -> "4:2:2"
else -> "4:2:0"
}
return "HEVC · $depthLabel · $dynamicRange ($colorSpace) · $chromaLabel"
}
@@ -186,9 +186,11 @@ internal fun StreamScene() {
Brush.linearGradient(listOf(Color(0xFF2A1E5C), Color(0xFF0E1B3D), Color(0xFF06122B))), Brush.linearGradient(listOf(Color(0xFF2A1E5C), Color(0xFF0E1B3D), Color(0xFF06122B))),
), ),
) { ) {
// [fps, mbps, latP50, latP95, latValid, skew, w, h, hz, dropped] // [fps, mbps, latP50, latP95, latValid, skew, w, h, hz, dropped,
// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc] — the last four = a 10-bit
// BT.2020 PQ (HDR) 4:2:0 feed, so the HUD renders its video-feed line.
StatsOverlay( StatsOverlay(
doubleArrayOf(238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 0.0), doubleArrayOf(238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 0.0, 10.0, 9.0, 16.0, 1.0),
Modifier.align(Alignment.TopStart).padding(12.dp), Modifier.align(Alignment.TopStart).padding(12.dp),
) )
} }
@@ -50,15 +50,25 @@ object Gamepad {
const val PREF_DUALSENSE = 2 const val PREF_DUALSENSE = 2
const val PREF_XBOXONE = 3 const val PREF_XBOXONE = 3
const val PREF_DUALSHOCK4 = 4 const val PREF_DUALSHOCK4 = 4
const val PREF_STEAMCONTROLLER = 5
const val PREF_STEAMDECK = 6
// USB vendor ids of the controllers we can identify by VID/PID. // USB vendor ids of the controllers we can identify by VID/PID.
private const val VID_SONY = 0x054C private const val VID_SONY = 0x054C
private const val VID_MICROSOFT = 0x045E private const val VID_MICROSOFT = 0x045E
private const val VID_VALVE = 0x28DE
// Sony product ids. DualSense (PS5) and DualShock 4 (PS4) map to distinct host pad types. // Sony product ids. DualSense (PS5) and DualShock 4 (PS4) map to distinct host pad types.
private val PID_DUALSENSE = setOf(0x0CE6, 0x0DF2) private val PID_DUALSENSE = setOf(0x0CE6, 0x0DF2)
private val PID_DUALSHOCK4 = setOf(0x05C4, 0x09CC) private val PID_DUALSHOCK4 = setOf(0x05C4, 0x09CC)
// Valve: Steam Deck built-in controller (0x1205); classic Steam Controller wired (0x1102) /
// dongle (0x1142). The host builds the virtual hid-steam pad; rich-input capture (paddles /
// trackpads / gyro) is out of scope on Android (no rich-input plane yet), so only the standard
// buttons + sticks reach the host for now — parity with the desktop type resolution.
private val PID_STEAMDECK = setOf(0x1205)
private val PID_STEAMCONTROLLER = setOf(0x1102, 0x1142)
// Microsoft Xbox One / Series product ids (wired + the common Bluetooth/dongle revisions). All // Microsoft Xbox One / Series product ids (wired + the common Bluetooth/dongle revisions). All
// behave like Xbox 360 on the host minus the glyph identity, so they share one pref byte. // behave like Xbox 360 on the host minus the glyph identity, so they share one pref byte.
private val PID_XBOXONE = setOf( private val PID_XBOXONE = setOf(
@@ -82,6 +92,8 @@ object Gamepad {
vid == VID_SONY && pid in PID_DUALSENSE -> PREF_DUALSENSE vid == VID_SONY && pid in PID_DUALSENSE -> PREF_DUALSENSE
vid == VID_SONY && pid in PID_DUALSHOCK4 -> PREF_DUALSHOCK4 vid == VID_SONY && pid in PID_DUALSHOCK4 -> PREF_DUALSHOCK4
vid == VID_MICROSOFT && pid in PID_XBOXONE -> PREF_XBOXONE vid == VID_MICROSOFT && pid in PID_XBOXONE -> PREF_XBOXONE
vid == VID_VALVE && pid in PID_STEAMDECK -> PREF_STEAMDECK
vid == VID_VALVE && pid in PID_STEAMCONTROLLER -> PREF_STEAMCONTROLLER
else -> PREF_XBOX360 else -> PREF_XBOX360
} }
} }
@@ -103,9 +103,12 @@ object NativeBridge {
/** /**
* Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs. * Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs.
* Returns 10 doubles: * Returns 14 doubles:
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped]` * `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped,
* (the two flags are 1.0/0.0). Poll ~1 Hz; each call resets the measurement window. * bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`
* (the two flags are 1.0/0.0; the trailing four describe the negotiated video feed — bit depth
* 8/10, CICP primaries/transfer, and the HEVC chroma_format_idc 1=4:2:0 / 3=4:4:4). Poll ~1 Hz;
* each call resets the measurement window.
*/ */
external fun nativeVideoStats(handle: Long): DoubleArray? external fun nativeVideoStats(handle: Long): DoubleArray?
+5
View File
@@ -114,6 +114,11 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextHidout(
out[2..n].copy_from_slice(&effect); out[2..n].copy_from_slice(&effect);
n n
} }
HidOutput::TrackpadHaptic { .. } => {
// Steam Controller trackpad-coil haptics — no Android equivalent; drop it (motor
// rumble already rides the universal 0xCA plane).
return -1;
}
}; };
n as jint n as jint
}) })
+17 -6
View File
@@ -409,11 +409,13 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
} }
/// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD. /// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD.
/// Returns 10 doubles /// Returns 14 doubles
/// `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped]` /// `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped,
/// (the two flags are 1.0/0.0), or `null` when no decode thread is running. Poll ~1 Hz from the UI; /// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`
/// each call resets the measurement window. Not android-gated — pure `jni` + connector reads, so it /// (the two flags are 1.0/0.0; the trailing four describe the negotiated video feed — see below), or
/// links on the host build too (Kotlin only ever calls it on device). /// `null` when no decode thread is running. Poll ~1 Hz from the UI; each call resets the measurement
/// window. Not android-gated — pure `jni` + connector reads, so it links on the host build too
/// (Kotlin only ever calls it on device).
#[no_mangle] #[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats( pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
env: JNIEnv, env: JNIEnv,
@@ -431,7 +433,8 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
None => return std::ptr::null_mut(), // not streaming → no stats None => return std::ptr::null_mut(), // not streaming → no stats
}; };
let mode = h.client.mode(); let mode = h.client.mode();
let buf: [f64; 10] = [ let color = h.client.color;
let buf: [f64; 14] = [
snap.fps, snap.fps,
snap.mbps, snap.mbps,
snap.lat_p50_ms, snap.lat_p50_ms,
@@ -442,6 +445,14 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
mode.height as f64, mode.height as f64,
mode.refresh_hz as f64, mode.refresh_hz as f64,
h.client.frames_dropped() as f64, h.client.frames_dropped() as f64,
// Video-feed properties the host resolved at the handshake (Welcome): encode bit depth
// (8 / 10), the CICP colour primaries + transfer code points (Kotlin maps these to a
// colour-space / HDR label — transfer 16 = PQ, 18 = HLG ⇒ HDR), and the HEVC
// chroma_format_idc (1 = 4:2:0, 3 = 4:4:4). Static for the session unless renegotiated.
h.client.bit_depth as f64,
color.primaries as f64,
color.transfer as f64,
h.client.chroma_format as f64,
]; ];
let arr = match env.new_double_array(buf.len() as jsize) { let arr = match env.new_double_array(buf.len() as jsize) {
Ok(a) => a, Ok(a) => a,
+3
View File
@@ -24,6 +24,9 @@ let package = Package(
.copy("Resources/THIRD-PARTY-NOTICES.txt"), .copy("Resources/THIRD-PARTY-NOTICES.txt"),
.copy("Resources/LICENSE-MIT.txt"), .copy("Resources/LICENSE-MIT.txt"),
.copy("Resources/LICENSE-APACHE.txt"), .copy("Resources/LICENSE-APACHE.txt"),
// Geist (SIL OFL 1.1) the brand typeface, shared with punktfunk-website.
// Registered with Core Text at first use; see BrandFont.swift.
.copy("Resources/Fonts"),
], ],
linkerSettings: [ linkerSettings: [
// Rust staticlib system deps. // Rust staticlib system deps.
@@ -364,7 +364,7 @@
ENABLE_HARDENED_RUNTIME = YES; ENABLE_HARDENED_RUNTIME = YES;
GENERATE_INFOPLIST_FILE = YES; GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist; INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger"; INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities"; INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSHumanReadableCopyright = ""; INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input."; INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
@@ -398,7 +398,7 @@
ENABLE_HARDENED_RUNTIME = YES; ENABLE_HARDENED_RUNTIME = YES;
GENERATE_INFOPLIST_FILE = YES; GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist; INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger"; INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities"; INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSHumanReadableCopyright = ""; INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input."; INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
@@ -429,7 +429,7 @@
CURRENT_PROJECT_VERSION = 1; CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES; GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist; INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger"; INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities"; INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input."; INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone."; INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -468,7 +468,7 @@
CURRENT_PROJECT_VERSION = 1; CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES; GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist; INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger"; INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities"; INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input."; INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone."; INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -506,7 +506,7 @@
CURRENT_PROJECT_VERSION = 1; CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES; GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist; INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger"; INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities"; INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input."; INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES; INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -536,7 +536,7 @@
CURRENT_PROJECT_VERSION = 1; CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES; GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist; INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger"; INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities"; INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input."; INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES; INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -10,12 +10,17 @@ struct AcknowledgementsView: View {
var body: some View { var body: some View {
ScrollView { ScrollView {
// Top-level LazyVStack so the third-party-notices chunks (Licenses.thirdPartyNoticesChunks,
// ~885 KB total) load lazily as they scroll into view a single Text that large overshoots
// the text-rendering height limit (blank below the limit + very slow). spacing 0 keeps the
// notice chunks visually continuous; the header block carries its own spacing + bottom pad.
LazyVStack(alignment: .leading, spacing: 0) {
VStack(alignment: .leading, spacing: 18) { VStack(alignment: .leading, spacing: 18) {
Text("punktfunk") Text("punktfunk")
.font(.title2).bold() .font(.geist(22, .bold, relativeTo: .title2))
if let version { if let version {
Text("Version \(version)") Text("Version \(version)")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
Text(Licenses.appLicense) Text(Licenses.appLicense)
@@ -24,19 +29,41 @@ struct AcknowledgementsView: View {
Divider() Divider()
Text("Bundled font")
.font(.geist(17, .semibold, relativeTo: .headline))
Text("punktfunk ships the Geist typeface (Geist Sans), "
+ "© The Geist Project Authors / Vercel, used under the SIL Open Font "
+ "License 1.1.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
if !Licenses.fontLicense.isEmpty {
Text(Licenses.fontLicense)
.font(.caption2.monospaced())
.modifier(SelectableText())
}
Divider()
Text("Third-party software") Text("Third-party software")
.font(.headline) .font(.geist(17, .semibold, relativeTo: .headline))
Text( Text(
"punktfunk uses the open-source components below, each under its own license. " "punktfunk uses the open-source components below, each under its own license. "
+ "On some platforms FFmpeg is additionally bundled under the LGPL v2.1+ " + "On some platforms FFmpeg is additionally bundled under the LGPL v2.1+ "
+ "(dynamically linked, replaceable)." + "(dynamically linked, replaceable)."
) )
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
Text(Licenses.thirdPartyNotices) }
.frame(maxWidth: .infinity, alignment: .leading)
.padding(.bottom, 18)
ForEach(Licenses.thirdPartyNoticesChunks.indices, id: \.self) { i in
Text(Licenses.thirdPartyNoticesChunks[i])
.font(.caption2.monospaced()) .font(.caption2.monospaced())
.frame(maxWidth: .infinity, alignment: .leading)
.modifier(SelectableText()) .modifier(SelectableText())
} }
}
.frame(maxWidth: 900, alignment: .leading) .frame(maxWidth: 900, alignment: .leading)
.frame(maxWidth: .infinity, alignment: .leading) .frame(maxWidth: .infinity, alignment: .leading)
.padding() .padding()
@@ -80,6 +80,11 @@ struct AddHostSheet: View {
} }
#if !os(tvOS) #if !os(tvOS)
.formStyle(.grouped) .formStyle(.grouped)
#endif
#if os(iOS)
// The detent below is sized to fit all 3 rows + the action button exactly, so the
// Form must NOT scroll/bounce inside it lock it. (iOS 16+; safe at iOS 17.)
.scrollDisabled(true)
#endif #endif
#if os(macOS) #if os(macOS)
// macOS: UNCHANGED Cancel + Spacer + Add in an HStack, both wired to the // macOS: UNCHANGED Cancel + Spacer + Add in an HStack, both wired to the
@@ -120,8 +125,8 @@ struct AddHostSheet: View {
// Form + the full-width action row, instead of the half-screen .medium it used to rest // Form + the full-width action row, instead of the half-screen .medium it used to rest
// at. A single fixed detent is enough: the system keeps the content above the keyboard // at. A single fixed detent is enough: the system keeps the content above the keyboard
// when Address/Port is focused, and on iPadOS this renders as a short bottom sheet (not a // when Address/Port is focused, and on iPadOS this renders as a short bottom sheet (not a
// centered formSheet card). If Dynamic Type grows the rows past this height the Form just // centered formSheet card). The Form itself is .scrollDisabled (above) so it can't
// scrolls inside the detent nothing is clipped. (.height(_:) is iOS 16+, safe at iOS 17.) // bounce/scroll inside this fixed detent. (.height(_:) is iOS 16+, safe at iOS 17.)
.presentationDetents([.height(320)]) .presentationDetents([.height(320)])
.presentationDragIndicator(.visible) .presentationDragIndicator(.visible)
#endif #endif
@@ -0,0 +1,39 @@
// App-wide brand chrome. SwiftUI has no single switch to put a custom font on every navigation
// title, so the iOS large/inline nav titles are themed through UINavigationBar's appearance proxy
// (set once at launch). Backgrounds are left at the system defaults transparent at the scroll
// edge (the large title floats on the content), blurred once scrolled so only the typeface
// changes: Geist, matching the cards and the website.
#if os(iOS)
import PunktfunkKit
import UIKit
enum BrandTheme {
static func apply() {
BrandFont.registerIfNeeded()
let scrollEdge = UINavigationBarAppearance()
scrollEdge.configureWithTransparentBackground()
applyFonts(to: scrollEdge)
let standard = UINavigationBarAppearance()
standard.configureWithDefaultBackground()
applyFonts(to: standard)
let proxy = UINavigationBar.appearance()
proxy.scrollEdgeAppearance = scrollEdge
proxy.standardAppearance = standard
proxy.compactAppearance = standard
}
/// Override only the title fonts; leave colors/backgrounds at the configured defaults.
private static func applyFonts(to appearance: UINavigationBarAppearance) {
if let large = UIFont(name: "Geist-Bold", size: 34) {
appearance.largeTitleTextAttributes[.font] = large
}
if let inline = UIFont(name: "Geist-SemiBold", size: 17) {
appearance.titleTextAttributes[.font] = inline
}
}
}
#endif
@@ -28,6 +28,7 @@ struct ContentView: View {
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0 @AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0 @AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2 @AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2
@AppStorage(DefaultsKey.hdrEnabled) private var hdrEnabled = true
@AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true @AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.hudEnabled) private var hudEnabled = true @AppStorage(DefaultsKey.hudEnabled) private var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) private var hudPlacement = HUDPlacement.topTrailing.rawValue @AppStorage(DefaultsKey.hudPlacement) private var hudPlacement = HUDPlacement.topTrailing.rawValue
@@ -68,15 +69,19 @@ struct ContentView: View {
// A session actually started remember it on the card ("Connected ago" // A session actually started remember it on the card ("Connected ago"
// plus the accent ring on the most recent host). // plus the accent ring on the most recent host).
guard let host = model.activeHost else { break } guard let host = model.activeHost else { break }
store.markConnected(host.id)
// Delegated approval just succeeded: the operator let this device in, so pin the // Delegated approval just succeeded: the operator let this device in, so pin the
// host's observed fingerprint and remember it as paired future connects are then // host's observed fingerprint and remember it as paired future connects are then
// silent (rule 1), exactly like after a PIN/TOFU success. Dismisses the wait prompt. // silent (rule 1), exactly like after a PIN/TOFU success. Dismisses the wait prompt.
if awaitingApproval?.host.id == host.id { let approvedFingerprint = awaitingApproval?.host.id == host.id
if let fp = model.connection?.hostFingerprint { ? model.connection?.hostFingerprint : nil
store.pin(host.id, fingerprint: fp) if awaitingApproval?.host.id == host.id { awaitingApproval = nil }
} // Persist on the next runloop tick: HostStore is an ObservableObject, and mutating
awaitingApproval = nil // its @Published from inside .onChange (a view-update callback) trips SwiftUI's
// "Publishing changes from within view updates". A one-tick delay is imperceptible.
let store = store
DispatchQueue.main.async {
store.markConnected(host.id)
if let approvedFingerprint { store.pin(host.id, fingerprint: approvedFingerprint) }
} }
case .idle: case .idle:
// The delegated-approval connect failed, timed out, or was cancelled drop the // The delegated-approval connect failed, timed out, or was cancelled drop the
@@ -333,6 +338,7 @@ struct ContentView: View {
rawValue: UInt32(clamping: gamepadType)) ?? .auto), rawValue: UInt32(clamping: gamepadType)) ?? .auto),
bitrateKbps: UInt32(clamping: bitrateKbps), bitrateKbps: UInt32(clamping: bitrateKbps),
audioChannels: UInt8(clamping: audioChannels), audioChannels: UInt8(clamping: audioChannels),
hdrEnabled: hdrEnabled,
launchID: launchID, launchID: launchID,
allowTofu: allowTofu, allowTofu: allowTofu,
requestAccess: requestAccess) requestAccess: requestAccess)
@@ -475,6 +481,7 @@ struct ContentView: View {
gamepad: pad, gamepad: pad,
bitrateKbps: bitrate, bitrateKbps: bitrate,
audioChannels: UInt8(clamping: audioChannels), audioChannels: UInt8(clamping: audioChannels),
hdrEnabled: hdrEnabled,
autoTrust: true) autoTrust: true)
} }
} }
@@ -54,7 +54,7 @@ struct ControllerTestView: View {
var body: some View { var body: some View {
VStack(spacing: 0) { VStack(spacing: 0) {
HStack { HStack {
Text("Test Controller").font(.headline) Text("Test Controller").font(.geist(17, .semibold, relativeTo: .headline))
Spacer() Spacer()
Button("Done") { dismiss() }.keyboardShortcut(.cancelAction) Button("Done") { dismiss() }.keyboardShortcut(.cancelAction)
} }
@@ -99,8 +99,8 @@ struct ControllerTestView: View {
.font(.title2) .font(.title2)
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
VStack(alignment: .leading, spacing: 2) { VStack(alignment: .leading, spacing: 2) {
Text(c.name).font(.headline) Text(c.name).font(.geist(17, .semibold, relativeTo: .headline))
Text(c.productCategory).font(.caption).foregroundStyle(.secondary) Text(c.productCategory).font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
} }
Spacer() Spacer()
} }
@@ -209,7 +209,7 @@ struct ControllerTestView: View {
) -> some View { ) -> some View {
VStack(alignment: .leading, spacing: 4) { VStack(alignment: .leading, spacing: 4) {
Text("Touchpad\(tp.button.isPressed ? " — click" : "")") Text("Touchpad\(tp.button.isPressed ? " — click" : "")")
.font(.caption2).foregroundStyle(.secondary) .font(.geist(11, relativeTo: .caption2)).foregroundStyle(.secondary)
ZStack { ZStack {
RoundedRectangle(cornerRadius: 8).stroke(Color.secondary.opacity(0.3)) RoundedRectangle(cornerRadius: 8).stroke(Color.secondary.opacity(0.3))
fingerDot(tp.primary, color: .accentColor) fingerDot(tp.primary, color: .accentColor)
@@ -230,7 +230,7 @@ struct ControllerTestView: View {
private func motionReadout(_ m: GCMotion) -> some View { private func motionReadout(_ m: GCMotion) -> some View {
let a = Self.totalAccel(m) let a = Self.totalAccel(m)
return VStack(alignment: .leading, spacing: 2) { return VStack(alignment: .leading, spacing: 2) {
Text("Motion").font(.caption2).foregroundStyle(.secondary) Text("Motion").font(.geist(11, relativeTo: .caption2)).foregroundStyle(.secondary)
Text(String(format: "gyro %+.2f %+.2f %+.2f", Text(String(format: "gyro %+.2f %+.2f %+.2f",
m.rotationRate.x, m.rotationRate.y, m.rotationRate.z)) m.rotationRate.x, m.rotationRate.y, m.rotationRate.z))
.font(.caption2.monospaced()) .font(.caption2.monospaced())
@@ -254,11 +254,11 @@ struct ControllerTestView: View {
Toggle("Heavy motor (left)", isOn: $heavyOn) Toggle("Heavy motor (left)", isOn: $heavyOn)
Toggle("Light motor (right)", isOn: $lightOn) Toggle("Light motor (right)", isOn: $lightOn)
Label("Backend: \(tester.rumbleBackend)", systemImage: "waveform") Label("Backend: \(tester.rumbleBackend)", systemImage: "waveform")
.font(.caption).foregroundStyle(.secondary) .font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
Text("Toggle a motor to feel it. The host maps a game's low/high-frequency " Text("Toggle a motor to feel it. The host maps a game's low/high-frequency "
+ "rumble onto these two. A DualSense is driven over raw HID (CoreHaptics " + "rumble onto these two. A DualSense is driven over raw HID (CoreHaptics "
+ "can't reach its motors on macOS).") + "can't reach its motors on macOS).")
.font(.caption).foregroundStyle(.secondary) .font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
} }
.onChange(of: heavyOn) { _, _ in applyRumble() } .onChange(of: heavyOn) { _, _ in applyRumble() }
.onChange(of: lightOn) { _, _ in applyRumble() } .onChange(of: lightOn) { _, _ in applyRumble() }
@@ -289,11 +289,11 @@ struct ControllerTestView: View {
} }
} }
Text("Pick an effect, then pull L2/R2 to feel the resistance.") Text("Pick an effect, then pull L2/R2 to feel the resistance.")
.font(.caption).foregroundStyle(.secondary) .font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
} }
} else { } else {
Text("Adaptive triggers need a DualSense.") Text("Adaptive triggers need a DualSense.")
.font(.caption).foregroundStyle(.secondary) .font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
} }
} }
} }
@@ -348,7 +348,7 @@ struct ControllerTestView: View {
_ title: String, @ViewBuilder _ content: () -> Content _ title: String, @ViewBuilder _ content: () -> Content
) -> some View { ) -> some View {
VStack(alignment: .leading, spacing: 10) { VStack(alignment: .leading, spacing: 10) {
Text(title).font(.subheadline.weight(.semibold)) Text(title).font(.geist(15, .semibold, relativeTo: .subheadline))
content() content()
} }
.frame(maxWidth: .infinity, alignment: .leading) .frame(maxWidth: .infinity, alignment: .leading)
@@ -127,14 +127,13 @@ struct HomeView: View {
AddHostSheet { store.add($0) } AddHostSheet { store.add($0) }
} }
#if os(iOS) #if os(iOS)
// SettingsView owns its own NavigationSplitView (sidebar + detail) and Done button, so it
// is presented directly wrapping it in a NavigationStack here would nest a split view in
// a stack (double title bars). `settingsSheetSizing()` widens the sheet on iPad for the
// two-column layout.
.sheet(isPresented: $showSettings) { .sheet(isPresented: $showSettings) {
NavigationStack {
SettingsView() SettingsView()
.navigationTitle("Settings") .settingsSheetSizing()
.toolbar {
Button("Done") { showSettings = false }
}
}
} }
#endif #endif
#endif #endif
@@ -172,7 +171,7 @@ struct HomeView: View {
private var discoveredSection: some View { private var discoveredSection: some View {
VStack(alignment: .leading, spacing: 10) { VStack(alignment: .leading, spacing: 10) {
Label("On this network", systemImage: "antenna.radiowaves.left.and.right") Label("On this network", systemImage: "antenna.radiowaves.left.and.right")
.font(.headline) .font(.geist(15, .semibold, relativeTo: .headline))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
.padding(.horizontal) .padding(.horizontal)
LazyVGrid(columns: gridColumns, spacing: gridSpacing) { LazyVGrid(columns: gridColumns, spacing: gridSpacing) {
@@ -249,8 +248,10 @@ struct HomeView: View {
/// the width so the cards stay edge-aligned with the title and bars sized touch-first: one /// the width so the cards stay edge-aligned with the title and bars sized touch-first: one
/// column on iPhone portrait, 34 generous cards on iPad. /// column on iPhone portrait, 34 generous cards on iPad.
private var gridColumns: [GridItem] { private var gridColumns: [GridItem] {
// Wider than before: the monogram card is a horizontal module (tile + address line), so
// it needs room for a monospaced "IP:port" without truncating.
#if os(macOS) #if os(macOS)
[GridItem(.adaptive(minimum: 180, maximum: 240), spacing: 16)] [GridItem(.adaptive(minimum: 250, maximum: 320), spacing: 16)]
#elseif os(tvOS) #elseif os(tvOS)
[GridItem(.adaptive(minimum: 320), spacing: 48)] [GridItem(.adaptive(minimum: 320), spacing: 48)]
#else #else
@@ -1,26 +1,75 @@
// The host grid's cards: a saved host (tap to connect, context menu) and an mDNS-discovered // The host grid's cards: a saved host (tap to connect, context menu) and an mDNS-discovered
// host (tap to save + connect). Both share the same platform-tuned sizing. // host (tap to save + connect). Both share the "monogram module" look a squared brand-purple
// monogram tile + a left-aligned bold Geist name over monospaced technical metadata
// (address, status), framed by a hairline panel border. Industrial, not soft.
import PunktfunkKit import PunktfunkKit
import SwiftUI import SwiftUI
/// Shared host-card sizing touch-first on iOS, compact on macOS/tvOS. /// Shared host-card sizing touch-first on iOS, compact on macOS, roomy on tvOS.
private struct CardMetrics { private struct CardMetrics {
let iconSize: CGFloat let tile: CGFloat // monogram tile side
let iconBox: CGFloat let monogram: CGFloat // monogram letter point size
let cardPadding: CGFloat let name: CGFloat // host-name point size
let nameFont: Font let meta: CGFloat // address (mono) point size
let status: CGFloat // status-label (mono) point size
let padding: CGFloat
let spacing: CGFloat // tile text gap
let radius: CGFloat
static var current: CardMetrics { static var current: CardMetrics {
#if os(iOS) #if os(iOS)
CardMetrics(iconSize: 56, iconBox: 76, cardPadding: 28, nameFont: .title3.weight(.semibold)) CardMetrics(tile: 54, monogram: 26, name: 19, meta: 13, status: 11,
padding: 16, spacing: 14, radius: 12)
#elseif os(tvOS)
CardMetrics(tile: 64, monogram: 32, name: 24, meta: 16, status: 14,
padding: 18, spacing: 18, radius: 14)
#else #else
CardMetrics(iconSize: 42, iconBox: 56, cardPadding: 18, nameFont: .headline) CardMetrics(tile: 44, monogram: 21, name: 15, meta: 12, status: 10.5,
padding: 13, spacing: 12, radius: 10)
#endif #endif
} }
} }
/// A saved host. The accent ring marks the most-recently-connected one; the context menu /// First letter of a host name, uppercased the monogram glyph. Falls back to a bullet.
private func monogram(_ name: String) -> String {
guard let first = name.trimmingCharacters(in: .whitespacesAndNewlines).first else { return "" }
return String(first).uppercased()
}
/// The squared monogram tile. `filled` = a solid brand-purple chip (saved hosts); otherwise a
/// tinted outline (discovered hosts). Shows a spinner in place of the glyph while connecting.
private func monogramTile(_ letter: String, m: CardMetrics, connecting: Bool, filled: Bool) -> some View {
let shape = RoundedRectangle(cornerRadius: m.radius - 3, style: .continuous)
return ZStack {
shape.fill(filled
? AnyShapeStyle(LinearGradient(
colors: [Color.brand, Color.brand.opacity(0.72)],
startPoint: .top, endPoint: .bottom))
: AnyShapeStyle(Color.brand.opacity(0.14)))
if connecting {
ProgressView().tint(filled ? .white : Color.brand)
} else {
// Fixed size (not Dynamic Type): the glyph is pinned inside a fixed tile, so it must
// not scale up and spill out at large accessibility text sizes. minimumScaleFactor +
// the clip below are belt-and-suspenders for an unusually wide glyph.
Text(letter)
.font(.geistFixed(m.monogram, .bold))
.minimumScaleFactor(0.5)
.lineLimit(1)
.foregroundStyle(filled ? Color.white : Color.brand)
}
}
.frame(width: m.tile, height: m.tile)
.clipShape(shape)
.overlay {
if !filled {
shape.strokeBorder(Color.brand.opacity(0.45), lineWidth: 1)
}
}
}
/// A saved host. A left accent bar marks the most-recently-connected one; the context menu
/// pairs / speed-tests / forgets / removes. Disabled while a session is busy. /// pairs / speed-tests / forgets / removes. Disabled while a session is busy.
struct HostCardView: View { struct HostCardView: View {
let host: StoredHost let host: StoredHost
@@ -41,66 +90,44 @@ struct HostCardView: View {
var body: some View { var body: some View {
let m = CardMetrics.current let m = CardMetrics.current
return Button(action: onConnect) { return Button(action: onConnect) {
VStack(spacing: 10) { HStack(spacing: m.spacing) {
ZStack { monogramTile(monogram(host.displayName), m: m, connecting: isConnecting, filled: true)
Image(systemName: "play.display") VStack(alignment: .leading, spacing: 4) {
.font(.system(size: m.iconSize, weight: .light))
.foregroundStyle(.tint)
.opacity(isConnecting ? 0.3 : 1)
if isConnecting {
ProgressView()
}
}
.frame(height: m.iconBox)
VStack(spacing: 2) {
HStack(spacing: 6) {
// Presence dot: green = advertising on the LAN now; grey = not seen.
Circle()
.fill(isOnline ? Color.green : Color.secondary.opacity(0.35))
.frame(width: 7, height: 7)
.accessibilityLabel(isOnline ? "Online" : "Offline")
Text(host.displayName) Text(host.displayName)
.font(m.nameFont) .font(.geist(m.name, .bold, relativeTo: .title3))
.foregroundStyle(.primary)
.lineLimit(1) .lineLimit(1)
}
HStack(spacing: 4) {
if host.pinnedSHA256 != nil {
Image(systemName: "lock.fill")
.font(.system(size: 9))
.foregroundStyle(.secondary)
}
Text("\(host.address):\(String(host.port))") Text("\(host.address):\(String(host.port))")
.font(.caption) .font(.geist(m.meta, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
.lineLimit(1) .lineLimit(1)
statusRow(m)
} }
if let last = host.lastConnected { Spacer(minLength: 0)
Text("Connected \(last, format: .relative(presentation: .named))")
.font(.caption2)
.foregroundStyle(.tertiary)
.lineLimit(1)
} }
} .padding(m.padding)
} .frame(maxWidth: .infinity, alignment: .leading)
.frame(maxWidth: .infinity)
.padding(.vertical, m.cardPadding)
.padding(.horizontal, 12)
#if !os(tvOS) #if !os(tvOS)
// tvOS: the .card button style owns platter + focus motion extra chrome // tvOS: the .card button style owns platter + focus motion; extra chrome mutes it.
// inside it mutes the grow/tilt. Material + accent ring are for pointer UIs. // Elsewhere: a flat material panel with a hairline border (industrial, not a soft blob),
// Deliberately .regularMaterial, not Liquid Glass: HIG keeps glass off content // and a brand accent bar down the leading edge for the most-recent host.
// tiles (it flattens hierarchy over an opaque grid) see GlassStyle.swift. .background(.regularMaterial)
.background(.regularMaterial, in: RoundedRectangle(cornerRadius: 14)) .overlay(alignment: .leading) {
.overlay {
if isMostRecent { if isMostRecent {
RoundedRectangle(cornerRadius: 14) Rectangle().fill(Color.brand).frame(width: 3)
.strokeBorder(Color.accentColor.opacity(0.35), lineWidth: 1.5)
} }
} }
.clipShape(RoundedRectangle(cornerRadius: m.radius, style: .continuous))
.overlay {
RoundedRectangle(cornerRadius: m.radius, style: .continuous)
.strokeBorder(.quaternary, lineWidth: 1)
}
#endif #endif
} }
#if os(tvOS) #if os(tvOS)
.buttonStyle(.card) .buttonStyle(.card)
#elseif os(iOS)
.buttonStyle(HostCardButtonStyle(cornerRadius: m.radius))
#else #else
.buttonStyle(.plain) .buttonStyle(.plain)
#endif #endif
@@ -119,10 +146,31 @@ struct HostCardView: View {
Button("Remove", role: .destructive, action: onRemove) Button("Remove", role: .destructive, action: onRemove)
} }
} }
/// Technical status line: a square presence pip + monospaced ONLINE/OFFLINE, and PAIRED when a
/// certificate is pinned (the lock state, spelled out).
@ViewBuilder private func statusRow(_ m: CardMetrics) -> some View {
HStack(spacing: 6) {
RoundedRectangle(cornerRadius: 1.5)
.fill(isOnline ? Color.green : Color.secondary.opacity(0.4))
.frame(width: 6, height: 6)
// The state is spelled out in the adjacent text, so the pip is decorative
// otherwise VoiceOver reads the status twice ("Online, ONLINE ").
.accessibilityHidden(true)
Text(isOnline ? "ONLINE" : "OFFLINE")
if host.pinnedSHA256 != nil {
Text("· PAIRED")
}
}
.font(.geist(m.status, .medium, relativeTo: .caption2))
.tracking(0.8)
.foregroundStyle(.secondary)
.lineLimit(1)
}
} }
/// A host found on the LAN but not yet saved. A dashed ring distinguishes it from saved cards; /// A host found on the LAN but not yet saved. A tinted-outline monogram + dashed panel border
/// tapping saves it and connects (or pairs, if the host requires it). /// distinguish it from saved cards; tapping saves it and connects (or pairs, if required).
struct DiscoveredCardView: View { struct DiscoveredCardView: View {
let discovered: DiscoveredHost let discovered: DiscoveredHost
let isBusy: Bool let isBusy: Bool
@@ -131,47 +179,77 @@ struct DiscoveredCardView: View {
var body: some View { var body: some View {
let m = CardMetrics.current let m = CardMetrics.current
return Button(action: onConnect) { return Button(action: onConnect) {
VStack(spacing: 10) { HStack(spacing: m.spacing) {
Image(systemName: "play.display") monogramTile(monogram(discovered.name), m: m, connecting: false, filled: false)
.font(.system(size: m.iconSize, weight: .light)) VStack(alignment: .leading, spacing: 4) {
.foregroundStyle(.tint)
.frame(height: m.iconBox)
VStack(spacing: 2) {
Text(discovered.name) Text(discovered.name)
.font(m.nameFont) .font(.geist(m.name, .bold, relativeTo: .title3))
.foregroundStyle(.primary)
.lineLimit(1) .lineLimit(1)
HStack(spacing: 4) {
Image(systemName: discovered.requiresPairing ? "lock.fill" : "wifi")
.font(.system(size: 9))
.foregroundStyle(.secondary)
Text("\(discovered.host):\(String(discovered.port))") Text("\(discovered.host):\(String(discovered.port))")
.font(.caption) .font(.geist(m.meta, relativeTo: .caption))
.foregroundStyle(.secondary)
.lineLimit(1)
HStack(spacing: 6) {
Image(systemName: discovered.requiresPairing
? "lock.fill" : "antenna.radiowaves.left.and.right")
.font(.system(size: m.status))
.accessibilityHidden(true) // decorative; the adjacent text says the state
Text(discovered.requiresPairing ? "PAIRING REQUIRED" : "DISCOVERED")
}
.font(.geist(m.status, .medium, relativeTo: .caption2))
.tracking(0.8)
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
.lineLimit(1) .lineLimit(1)
} }
Text(discovered.requiresPairing ? "Pairing required" : "Discovered") Spacer(minLength: 0)
.font(.caption2)
.foregroundStyle(.tertiary)
} }
} .padding(m.padding)
.frame(maxWidth: .infinity) .frame(maxWidth: .infinity, alignment: .leading)
.padding(.vertical, m.cardPadding)
.padding(.horizontal, 12)
#if !os(tvOS) #if !os(tvOS)
.background(.regularMaterial, in: RoundedRectangle(cornerRadius: 14)) .background(.regularMaterial)
.clipShape(RoundedRectangle(cornerRadius: m.radius, style: .continuous))
.overlay { .overlay {
RoundedRectangle(cornerRadius: 14) RoundedRectangle(cornerRadius: m.radius, style: .continuous)
.strokeBorder( .strokeBorder(
Color.secondary.opacity(0.25), Color.secondary.opacity(0.3),
style: StrokeStyle(lineWidth: 1, dash: [4, 3])) style: StrokeStyle(lineWidth: 1, dash: [4, 3]))
} }
#endif #endif
} }
#if os(tvOS) #if os(tvOS)
.buttonStyle(.card) .buttonStyle(.card)
#elseif os(iOS)
.buttonStyle(HostCardButtonStyle(cornerRadius: m.radius))
#else #else
.buttonStyle(.plain) .buttonStyle(.plain)
#endif #endif
.disabled(isBusy) .disabled(isBusy)
} }
} }
#if os(iOS)
/// The iOS host-card press/hover treatment, one style for both idioms:
/// - iPhone: a subtle scale-down on press + a light impact haptic on press-down. (`hoverEffect` is
/// inert without a pointer.)
/// - iPad: the system pointer "magnet" the cursor morphs into a highlight that conforms to the
/// card's rounded rect on hover. (`sensoryFeedback` is inert without a Taptic Engine, and the
/// press scale doubles as click feedback.)
struct HostCardButtonStyle: ButtonStyle {
var cornerRadius: CGFloat
func makeBody(configuration: Configuration) -> some View {
configuration.label
.scaleEffect(configuration.isPressed ? 0.96 : 1)
.animation(.spring(response: 0.3, dampingFraction: 0.65), value: configuration.isPressed)
// Conform the pointer highlight to the card's rounded rect, not its square bounds.
.contentShape(.hoverEffect, RoundedRectangle(cornerRadius: cornerRadius, style: .continuous))
.hoverEffect(.highlight)
// Light tap on press-down (nil on release so it fires once, on touch). No haptic
// hardware on iPad silently ignored there.
.sensoryFeedback(trigger: configuration.isPressed) { _, pressed in
pressed ? .impact(weight: .light) : nil
}
}
}
#endif
@@ -146,7 +146,7 @@ private struct GameCard: View {
.clipShape(RoundedRectangle(cornerRadius: 10, style: .continuous)) .clipShape(RoundedRectangle(cornerRadius: 10, style: .continuous))
.overlay(alignment: .topLeading) { storeBadge } .overlay(alignment: .topLeading) { storeBadge }
Text(game.title) Text(game.title)
.font(.caption) .font(.geist(12, relativeTo: .caption))
.lineLimit(2) .lineLimit(2)
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
@@ -154,7 +154,7 @@ private struct GameCard: View {
private var storeBadge: some View { private var storeBadge: some View {
Text(game.isCustom ? "Custom" : "Steam") Text(game.isCustom ? "Custom" : "Steam")
.font(.caption2.weight(.semibold)) .font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 6) .padding(.horizontal, 6)
.padding(.vertical, 3) .padding(.vertical, 3)
.background(.ultraThinMaterial, in: Capsule()) .background(.ultraThinMaterial, in: Capsule())
@@ -193,7 +193,7 @@ private struct PosterImage: View {
ZStack { ZStack {
Rectangle().fill(.quaternary) Rectangle().fill(.quaternary)
Text(title) Text(title)
.font(.headline) .font(.geist(17, .semibold, relativeTo: .headline))
.multilineTextAlignment(.center) .multilineTextAlignment(.center)
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
.padding(8) .padding(8)
@@ -48,7 +48,7 @@ struct PairSheet: View {
+ "(http://<host>:3000 → Pairing). " + "(http://<host>:3000 → Pairing). "
+ "Pairing verifies both sides at once — no fingerprint comparison " + "Pairing verifies both sides at once — no fingerprint comparison "
+ "needed.") + "needed.")
.font(.callout) .font(.geist(16, relativeTo: .callout))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
.multilineTextAlignment(.center) .multilineTextAlignment(.center)
TVFieldRow( TVFieldRow(
@@ -59,7 +59,7 @@ struct PairSheet: View {
) { editing = .clientName } ) { editing = .clientName }
if let errorText { if let errorText {
Text(errorText) Text(errorText)
.font(.callout) .font(.geist(16, relativeTo: .callout))
.foregroundStyle(.red) .foregroundStyle(.red)
} }
HStack(spacing: 32) { HStack(spacing: 32) {
@@ -121,13 +121,13 @@ struct PairSheet: View {
+ "(http://<host>:3000 → Pairing). " + "(http://<host>:3000 → Pairing). "
+ "Pairing verifies both sides at once — no fingerprint " + "Pairing verifies both sides at once — no fingerprint "
+ "comparison needed.") + "comparison needed.")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
if let errorText { if let errorText {
Section { Section {
Text(errorText) Text(errorText)
.font(.callout) .font(.geist(16, relativeTo: .callout))
.foregroundStyle(.red) .foregroundStyle(.red)
} }
} }
@@ -12,8 +12,19 @@ struct PunktfunkClientApp: App {
@NSApplicationDelegateAdaptor(AppDelegate.self) private var appDelegate @NSApplicationDelegateAdaptor(AppDelegate.self) private var appDelegate
#endif #endif
init() {
#if os(iOS)
// Put Geist on the navigation titles before any bar is built.
BrandTheme.apply()
#endif
}
var body: some Scene { var body: some Scene {
WindowGroup("Punktfunk") { WindowGroup("Punktfunk") {
// Pin the whole app's tint to the brand purple explicitly the asset-catalog accent
// resolution is environment/timing-sensitive and can fall back to system blue. Wraps the
// screenshot harness too, so captured screens are on-brand.
Group {
#if DEBUG #if DEBUG
// PUNKTFUNK_SHOT_SCENE=<name> show that single mock-populated screen full-bleed for // PUNKTFUNK_SHOT_SCENE=<name> show that single mock-populated screen full-bleed for
// the App Store screenshot capture (tools/screenshots.sh). Normal launch otherwise; // the App Store screenshot capture (tools/screenshots.sh). Normal launch otherwise;
@@ -27,6 +38,11 @@ struct PunktfunkClientApp: App {
ContentView() ContentView()
#endif #endif
} }
.tint(.brand)
// Geist Sans is the app's typeface. This sets the default for unstyled text and the
// form row labels; views that pick an explicit size/weight use `.geist()` directly.
.font(.geist(17, relativeTo: .body))
}
// The Stream menu (Disconnect D, Show/Hide Statistics S) a real menu bar on // The Stream menu (Disconnect D, Show/Hide Statistics S) a real menu bar on
// macOS, hardware-keyboard shortcuts on iPad. tvOS has neither. // macOS, hardware-keyboard shortcuts on iPad. tvOS has neither.
#if !os(tvOS) #if !os(tvOS)
@@ -34,7 +50,10 @@ struct PunktfunkClientApp: App {
#endif #endif
#if os(macOS) #if os(macOS)
Settings { Settings {
// A separate scene `.tint` does not cross scene boundaries, so re-apply the brand
// tint here or the Preferences window falls back to the (unreliable) asset accent.
SettingsView() SettingsView()
.tint(.brand)
} }
#endif #endif
} }
@@ -103,11 +103,11 @@ private struct ShotSettings: View {
.shadow(radius: 40, y: 16) .shadow(radius: 40, y: 16)
} }
#elseif os(iOS) #elseif os(iOS)
NavigationStack { // SettingsView owns its NavigationSplitView (sidebar + detail) and Done button, so it is
SettingsView() // rendered directly a wrapping NavigationStack would nest a split view in a stack. Open
.navigationTitle("Settings") // on General so the shot lands on real controls (iPad: sidebar + General detail; iPhone:
.navigationBarTitleDisplayMode(.inline) // the General page) instead of the bare category list.
} SettingsView(initialCategory: .general)
#else #else
NavigationStack { SettingsView() } NavigationStack { SettingsView() }
#endif #endif
@@ -175,10 +175,10 @@ private struct ShotHUD: View {
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
#if os(macOS) #if os(macOS)
Text("⌘⎋ releases the mouse") Text("⌘⎋ releases the mouse")
.font(.caption2).foregroundStyle(.secondary) .font(.geist(11, relativeTo: .caption2)).foregroundStyle(.secondary)
#elseif os(tvOS) #elseif os(tvOS)
Text("Press Menu to disconnect") Text("Press Menu to disconnect")
.font(.caption).foregroundStyle(.secondary) .font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
#endif #endif
} }
.padding(10) .padding(10)
@@ -259,7 +259,7 @@ private struct ShotDesktopFrame: View {
HStack(spacing: 8) { HStack(spacing: 8) {
Image(systemName: "gamecontroller.fill") Image(systemName: "gamecontroller.fill")
Text("Streaming from Battlestation") Text("Streaming from Battlestation")
.font(.system(.callout, weight: .semibold)) .font(.geist(16, .semibold, relativeTo: .callout))
} }
.padding(.horizontal, 14).padding(.vertical, 9) .padding(.horizontal, 14).padding(.vertical, 9)
.glassBackground(Capsule()) .glassBackground(Capsule())
@@ -129,6 +129,8 @@ final class SessionModel: ObservableObject {
#endif #endif
}() }()
let hdrCapable = hdrEnabled && displayHDR let hdrCapable = hdrEnabled && displayHDR
// 4:4:4 opt-out (default on); the hardware-decode probe below is the real gate.
let want444 = (UserDefaults.standard.object(forKey: DefaultsKey.enable444) as? Bool) ?? true
Task.detached(priority: .userInitiated) { Task.detached(priority: .userInitiated) {
// PunktfunkConnection.init blocks on the QUIC handshake keep it off the main // PunktfunkConnection.init blocks on the QUIC handshake keep it off the main
// actor. The persistent identity is presented on every connect so a paired // actor. The persistent identity is presented on every connect so a paired
@@ -138,9 +140,21 @@ final class SessionModel: ObservableObject {
// Advertise 10-bit + HDR10 when enabled: the host upgrades to a BT.2020 PQ Main10 stream // Advertise 10-bit + HDR10 when enabled: the host upgrades to a BT.2020 PQ Main10 stream
// only for actual HDR content (its own gate); the VideoToolbox/Metal present path is // only for actual HDR content (its own gate); the VideoToolbox/Metal present path is
// HDR-capable (P010 + itur_2100_PQ + EDR). 0 keeps the 8-bit BT.709 SDR stream. // HDR-capable (P010 + itur_2100_PQ + EDR). 0 keeps the 8-bit BT.709 SDR stream.
let videoCaps: UInt8 = hdrCapable var videoCaps: UInt8 = hdrCapable
? (PunktfunkConnection.videoCap10Bit | PunktfunkConnection.videoCapHDR) ? (PunktfunkConnection.videoCap10Bit | PunktfunkConnection.videoCapHDR)
: 0 : 0
// Advertise full-chroma 4:4:4 only when allowed AND this device can HARDWARE-decode it
// (software 4:4:4 is too slow for real-time). The host content-gates depth, so an
// HDR-advertised session can still receive an 8-bit 4:4:4 stream (SDR content) require
// BOTH depths there. Otherwise a no-op (the host emits 4:4:4 only if it too opted in);
// `chromaFormat` on the connection reflects what was actually resolved.
let canDecode444 =
hdrCapable
? (Stage444Probe.hwDecode444_8bit && Stage444Probe.hwDecode444_10bit)
: Stage444Probe.hwDecode444_8bit
if want444, canDecode444 {
videoCaps |= PunktfunkConnection.videoCap444
}
let result = Result { try PunktfunkConnection( let result = Result { try PunktfunkConnection(
host: host.address, port: host.port, host: host.address, port: host.port,
width: width, height: height, refreshHz: hz, width: width, height: height, refreshHz: hz,
@@ -1,10 +1,12 @@
// App settings. The host creates a native virtual output at exactly the chosen size/refresh // App settings. The host creates a native virtual output at exactly the chosen size/refresh
// there is no scaling anywhere in the pipeline. // there is no scaling anywhere in the pipeline.
// //
// Navigation differs per platform: macOS uses a tabbed preferences window (the sections had // Navigation differs per platform, but all three group the same categories (General, Display,
// outgrown one scrolling pane); iOS uses a single grouped Form; tvOS uses a focus-native // Audio, Controllers, Advanced, About): macOS uses a tabbed preferences window; iOS/iPadOS uses
// pushed-picker layout. The individual sections (`streamModeSection`, `audioSection`, ) are // an adaptive NavigationSplitView a category sidebar + detail pane on iPad, auto-collapsing to
// shared across all three so a setting is defined exactly once. // a hierarchical push list on iPhone (the system Settings idiom on each); tvOS uses a
// focus-native pushed-picker layout. The individual sections (`streamModeSection`,
// `audioSection`, ) are shared across all three so a setting is defined exactly once.
#if os(macOS) #if os(macOS)
import AppKit import AppKit
@@ -21,7 +23,9 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.compositor) private var compositor = 0 @AppStorage(DefaultsKey.compositor) private var compositor = 0
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0 @AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0 @AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.presenter) private var presenter = "stage1" @AppStorage(DefaultsKey.presenter) private var presenter = "stage2"
@AppStorage(DefaultsKey.hdrEnabled) private var hdrEnabled = true
@AppStorage(DefaultsKey.enable444) private var enable444 = true
@AppStorage(DefaultsKey.libraryEnabled) private var libraryEnabled = false @AppStorage(DefaultsKey.libraryEnabled) private var libraryEnabled = false
@AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true @AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.micEnabled) private var micEnabled = true @AppStorage(DefaultsKey.micEnabled) private var micEnabled = true
@@ -32,6 +36,22 @@ struct SettingsView: View {
#if DEBUG && !os(tvOS) #if DEBUG && !os(tvOS)
@State private var showControllerTest = false @State private var showControllerTest = false
#endif #endif
#if os(iOS)
@AppStorage(DefaultsKey.pointerCapture) private var pointerCapture = true
// The sidebar selection drives the detail pane on iPad and the pushed sub-page on iPhone.
// Width class decides the initial value: nil on iPhone (show the category list first),
// General on iPad (a two-column layout should never open with an empty detail).
@Environment(\.horizontalSizeClass) private var horizontalSizeClass
@State private var settingsSelection: SettingsCategory?
// Tracked so the detail can show its own Done whenever the sidebar (and its Done) is off screen
// not just on iPhone, but on any iPad layout that collapses the sidebar to an overlay. Starts
// .doubleColumn so iPad reliably opens with the sidebar (and its Done) visible.
@State private var columnVisibility: NavigationSplitViewVisibility = .doubleColumn
// Sticky once the wheel lands on "Custom", so editing a width/height that briefly equals a
// preset doesn't snap the wheel back off Custom. A stored non-preset value reads as custom even
// when this is false (see `isCustomResolution`), so it survives relaunches without persisting.
@State private var customMode = false
#endif
#if os(macOS) #if os(macOS)
@AppStorage(DefaultsKey.speakerUID) private var speakerUID = "" @AppStorage(DefaultsKey.speakerUID) private var speakerUID = ""
@AppStorage(DefaultsKey.micUID) private var micUID = "" @AppStorage(DefaultsKey.micUID) private var micUID = ""
@@ -39,6 +59,15 @@ struct SettingsView: View {
@State private var inputDevices: [AudioDevice] = [] @State private var inputDevices: [AudioDevice] = []
#endif #endif
#if os(iOS)
/// `initialCategory` is nil in the app (the list opens un-selected on iPhone; iPad lands on
/// General via `onAppear`). The screenshot harness passes an explicit category so the captured
/// shot opens on a real settings page (a populated detail) rather than the bare category list.
init(initialCategory: SettingsCategory? = nil) {
_settingsSelection = State(initialValue: initialCategory)
}
#endif
var body: some View { var body: some View {
#if os(tvOS) #if os(tvOS)
// Native tv pattern: no inline text entry (typing numbers with a remote is // Native tv pattern: no inline text entry (typing numbers with a remote is
@@ -66,6 +95,7 @@ struct SettingsView: View {
Form { Form {
presenterSection presenterSection
hdrSection
windowSection windowSection
statisticsSection statisticsSection
} }
@@ -106,29 +136,116 @@ struct SettingsView: View {
} }
#endif #endif
// MARK: - iOS: one grouped Form // MARK: - iOS / iPadOS: adaptive split view
#if os(iOS) #if os(iOS)
private var iosBody: some View { private var iosBody: some View {
Form { NavigationSplitView(columnVisibility: $columnVisibility) {
streamModeSection List(selection: $settingsSelection) {
audioSection ForEach(SettingsCategory.allCases) { category in
compositorSection // On iPhone the split view collapses to a push list, but a selection List
presenterSection // draws no disclosure indicator of its own add one in compact width for the
statisticsSection // expected drill-in affordance. On iPad the selected row highlights instead, so
experimentalSection // the chevron is omitted there.
controllersSection HStack {
Section { Label(category.title, systemImage: category.symbol)
NavigationLink("Acknowledgements") { AcknowledgementsView() } if horizontalSizeClass == .compact {
Spacer()
Image(systemName: "chevron.forward")
.font(.footnote.weight(.semibold))
.foregroundStyle(.tertiary)
// Purely a drill-in affordance the row's button trait already
// conveys "opens"; keep it out of the VoiceOver announcement.
.accessibilityHidden(true)
}
}
.tag(category)
}
}
.navigationTitle("Settings")
.toolbar {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
}
}
} detail: {
// NavigationSplitView hosts the detail in its own navigation context (its title bar),
// so no inner NavigationStack that would double the bar on iPad. On iPhone the split
// view collapses to one stack and pushes this when a row is tapped. `?? .general` only
// backs the brief pre-selection window; the list never auto-pushes on a nil selection.
settingsDetail(settingsSelection ?? .general)
// Keep a Done on the detail whenever the sidebar (and its Done) isn't on screen: the
// iPhone push, or any iPad layout that collapsed the sidebar to an overlay. When the
// sidebar is showing, its Done is the only one so this stays hidden to avoid two.
.toolbar {
if horizontalSizeClass == .compact || columnVisibility == .detailOnly {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
}
}
} }
} }
.formStyle(.grouped)
.onAppear { .onAppear {
if horizontalSizeClass == .regular, settingsSelection == nil {
settingsSelection = .general
}
gamepads.refresh() gamepads.refresh()
gamepads.startDiscovery() gamepads.startDiscovery()
} }
// A regularregular launch sets the default above; this catches a compactregular change
// (e.g. an iPad leaving narrow split-screen multitasking) so the detail pane fills in.
.onChange(of: horizontalSizeClass) { _, newValue in
if newValue == .regular, settingsSelection == nil {
settingsSelection = .general
}
}
.onDisappear { gamepads.stopDiscovery() } .onDisappear { gamepads.stopDiscovery() }
} }
@ViewBuilder
private func settingsDetail(_ category: SettingsCategory) -> some View {
switch category {
case .general:
Form {
streamModeSection
pointerSection
compositorSection
}
.formStyle(.grouped)
.navigationTitle("General")
.navigationBarTitleDisplayMode(.inline)
case .display:
Form {
presenterSection
hdrSection
statisticsSection
}
.formStyle(.grouped)
.navigationTitle("Display")
.navigationBarTitleDisplayMode(.inline)
case .audio:
Form { audioSection }
.formStyle(.grouped)
.navigationTitle("Audio")
.navigationBarTitleDisplayMode(.inline)
case .controllers:
Form { controllersSection }
.formStyle(.grouped)
.navigationTitle("Controllers")
.navigationBarTitleDisplayMode(.inline)
case .advanced:
Form { experimentalSection }
.formStyle(.grouped)
.navigationTitle("Advanced")
.navigationBarTitleDisplayMode(.inline)
case .about:
// Already a full scrollable view that sets its own "Acknowledgements" title; pin the
// display mode inline to match the five sibling detail pages (it would otherwise inherit
// the large title from the "Settings" sidebar root).
AcknowledgementsView()
.navigationBarTitleDisplayMode(.inline)
}
}
#endif #endif
// MARK: - tvOS // MARK: - tvOS
@@ -156,6 +273,10 @@ struct SettingsView: View {
Binding(get: { hudEnabled ? "on" : "off" }, set: { hudEnabled = $0 == "on" }) Binding(get: { hudEnabled ? "on" : "off" }, set: { hudEnabled = $0 == "on" })
} }
private var hdrEnabledTag: Binding<String> {
Binding(get: { hdrEnabled ? "on" : "off" }, set: { hdrEnabled = $0 == "on" })
}
private var tvBody: some View { private var tvBody: some View {
let currentTag = "\(width)x\(height)x\(hz)" let currentTag = "\(width)x\(height)x\(hz)"
let bounds = UIScreen.main.nativeBounds let bounds = UIScreen.main.nativeBounds
@@ -186,20 +307,25 @@ struct SettingsView: View {
selection: $audioChannels) selection: $audioChannels)
if bitrateKbps > 1_000_000 { if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill") Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange) .foregroundStyle(.orange)
.multilineTextAlignment(.center) .multilineTextAlignment(.center)
} }
TVSelectionRow( TVSelectionRow(
title: "Compositor", options: compositors, selection: $compositor) title: "Compositor", options: compositors, selection: $compositor)
#if DEBUG
TVSelectionRow( TVSelectionRow(
title: "Presenter", title: "Presenter (debug)",
options: [("Stage 1 (default)", "stage1"), ("Stage 2 (experimental)", "stage2")], options: [("Stage 2 (default)", "stage2"), ("Stage 1 (debug)", "stage1")],
selection: $presenter) selection: $presenter)
#endif
TVSelectionRow(
title: "10-bit HDR",
options: [("On", "on"), ("Off", "off")], selection: hdrEnabledTag)
Text("The host creates a virtual output at exactly this mode — native " Text("The host creates a virtual output at exactly this mode — native "
+ "resolution, no scaling. \(Self.bitrateFooter) A specific compositor " + "resolution, no scaling. \(Self.bitrateFooter) A specific compositor "
+ "is honored only if available on the host.") + "is honored only if available on the host.")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
.multilineTextAlignment(.center) .multilineTextAlignment(.center)
.padding(.top, 8) .padding(.top, 8)
@@ -219,7 +345,7 @@ struct SettingsView: View {
TVSelectionRow( TVSelectionRow(
title: "Controller type", options: Self.padTypes, selection: $gamepadType) title: "Controller type", options: Self.padTypes, selection: $gamepadType)
Text(Self.controllersFooter) Text(Self.controllersFooter)
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
.multilineTextAlignment(.center) .multilineTextAlignment(.center)
.padding(.top, 8) .padding(.top, 8)
@@ -243,6 +369,63 @@ struct SettingsView: View {
@ViewBuilder private var streamModeSection: some View { @ViewBuilder private var streamModeSection: some View {
Section { Section {
#if os(iOS)
// Touch-first: a rotating wheel of common resolutions (this device's own mode first) and
// a segmented refresh-rate control the same family as the Clock/Timer pickers. The host
// renders a virtual output at exactly the chosen mode, so these are real pixel sizes. The
// last wheel row, "Custom", reveals width/height/refresh fields for an arbitrary mode.
VStack(alignment: .leading, spacing: 4) {
Text("Resolution")
.font(.geist(15, relativeTo: .subheadline))
.foregroundStyle(.secondary)
Picker("Resolution", selection: resolutionSelection) {
ForEach(resolutionChoices, id: \.tag) { choice in
Text(choice.label).tag(choice.tag)
}
}
.labelsHidden()
.pickerStyle(.wheel)
.frame(maxHeight: 140)
}
if isCustomResolution {
// Arbitrary entry: type the exact width × height (and refresh) the host should drive.
HStack {
TextField("Width", value: $width, format: .number.grouping(.never))
.keyboardType(.numberPad)
Text("×")
TextField("Height", value: $height, format: .number.grouping(.never))
.labelsHidden()
.keyboardType(.numberPad)
}
// A row built from an HStack of TextFields otherwise insets its bottom separator to
// the inner content, clipping the hairline under "Width"; pin it to the cell edge.
.alignmentGuide(.listRowSeparatorLeading) { _ in 0 }
LabeledContent("Refresh rate") {
TextField("Hz", value: $hz, format: .number.grouping(.never))
.keyboardType(.numberPad)
.multilineTextAlignment(.trailing)
}
} else if refreshChoices.count > 1 {
VStack(alignment: .leading, spacing: 6) {
Text("Refresh rate")
.font(.geist(15, relativeTo: .subheadline))
.foregroundStyle(.secondary)
Picker("Refresh rate", selection: $hz) {
ForEach(refreshChoices, id: \.self) { rate in
Text("\(rate) Hz").tag(rate)
}
}
.labelsHidden()
.pickerStyle(.segmented)
}
} else {
// A device with a single supported rate (e.g. 60 Hz) has nothing to pick.
LabeledContent("Refresh rate") {
Text("\(hz) Hz").foregroundStyle(.secondary)
}
}
Button("Use this display's mode") { fillFromMainScreen() }
#elseif os(macOS)
HStack { HStack {
TextField("Resolution", value: $width, format: .number.grouping(.never)) TextField("Resolution", value: $width, format: .number.grouping(.never))
Text("×") Text("×")
@@ -253,6 +436,7 @@ struct SettingsView: View {
LabeledContent("") { LabeledContent("") {
Button("Use this display's mode") { fillFromMainScreen() } Button("Use this display's mode") { fillFromMainScreen() }
} }
#endif
#if !os(tvOS) #if !os(tvOS)
Toggle("Automatic bitrate", isOn: automaticBitrate) Toggle("Automatic bitrate", isOn: automaticBitrate)
if bitrateKbps != 0 { if bitrateKbps != 0 {
@@ -267,7 +451,7 @@ struct SettingsView: View {
} }
if bitrateKbps > 1_000_000 { if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill") Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange) .foregroundStyle(.orange)
} }
} }
@@ -277,11 +461,85 @@ struct SettingsView: View {
} footer: { } footer: {
Text("The host creates a virtual output at exactly this mode — " Text("The host creates a virtual output at exactly this mode — "
+ "native resolution, no scaling. \(Self.bitrateFooter)") + "native resolution, no scaling. \(Self.bitrateFooter)")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
} }
#if os(iOS)
// MARK: - Stream mode (iOS wheel)
/// Sentinel wheel tag for the "Custom" row. Real tags are "WxH" (digits + "x"), so this can't
/// collide with a resolution.
private static let customResolutionTag = "custom"
/// 16:9 then ultrawide presets; the device's native mode is prepended at runtime.
private static let resolutionPresets: [(name: String, w: Int, h: Int)] = [
("720p", 1280, 720),
("1080p", 1920, 1080),
("1440p", 2560, 1440),
("4K", 3840, 2160),
("Ultrawide 1080p", 2560, 1080),
("Ultrawide 1440p", 3440, 1440),
("Super ultrawide", 5120, 1440),
]
/// The non-custom wheel rows: this device's native mode first, then the presets, deduped by
/// dimensions (native wins a tie).
private var resolutionModes: [(name: String, w: Int, h: Int)] {
let bounds = UIScreen.main.nativeBounds // portrait-oriented pixels
let native = (w: Int(max(bounds.width, bounds.height)), h: Int(min(bounds.width, bounds.height)))
let all = [(name: "This device", w: native.w, h: native.h)] + Self.resolutionPresets
var seen = Set<String>()
return all.filter { seen.insert("\($0.w)x\($0.h)").inserted }
}
/// Wheel rows: the resolution modes, then a "Custom" row that reveals the numeric fields.
private var resolutionChoices: [(label: String, tag: String)] {
resolutionModes.map { (label: "\($0.name) · \($0.w) × \($0.h)", tag: "\($0.w)x\($0.h)") }
+ [(label: "Custom…", tag: Self.customResolutionTag)]
}
private var presetResolutionTags: Set<String> {
Set(resolutionModes.map { "\($0.w)x\($0.h)" })
}
/// True when the editable custom fields should show: the wheel is parked on "Custom" (sticky),
/// or the stored size simply isn't one of the presets (e.g. a value synced from a Mac) so a
/// non-preset mode stays editable across relaunches without a persisted flag.
private var isCustomResolution: Bool {
customMode || !presetResolutionTags.contains("\(width)x\(height)")
}
/// The wheel works in "WxH" tags so one selection drives both width and height; the custom
/// sentinel toggles `customMode` instead of writing a size.
private var resolutionSelection: Binding<String> {
Binding(
get: { isCustomResolution ? Self.customResolutionTag : "\(width)x\(height)" },
set: { tag in
if tag == Self.customResolutionTag {
customMode = true
return
}
customMode = false
let parts = tag.split(separator: "x").compactMap { Int($0) }
guard parts.count == 2 else { return }
width = parts[0]
height = parts[1]
})
}
/// Refresh rates the device can actually display (no point asking the host to render frames the
/// screen can't show), plus any stored custom value so it stays selectable.
private var refreshChoices: [Int] {
let maxHz = UIScreen.main.maximumFramesPerSecond
var rates = [60, 120, 240].filter { $0 <= maxHz }
if rates.isEmpty { rates = [maxHz] }
if !rates.contains(hz) { rates.append(hz) }
return rates.sorted()
}
#endif
@ViewBuilder private var audioSection: some View { @ViewBuilder private var audioSection: some View {
Section { Section {
Picker("Audio channels", selection: $audioChannels) { Picker("Audio channels", selection: $audioChannels) {
@@ -321,11 +579,35 @@ struct SettingsView: View {
Text("Host audio plays through the speaker; the microphone feeds the " Text("Host audio plays through the speaker; the microphone feeds the "
+ "host's virtual mic. System default follows macOS device changes. " + "host's virtual mic. System default follows macOS device changes. "
+ "Applies from the next session.") + "Applies from the next session.")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
} }
#if os(iOS)
/// iPad-only pointer-capture toggle: lock the mouse/trackpad for relative movement (games) vs
/// forward an absolute cursor position (desktop). Empty on iPhone (no hardware-pointer lock
/// the mouse path there is always the absolute fallback).
@ViewBuilder private var pointerSection: some View {
if UIDevice.current.userInterfaceIdiom == .pad {
Section {
Toggle("Capture pointer for games", isOn: $pointerCapture)
} header: {
Text("Pointer")
} footer: {
Text("With a mouse or trackpad connected, lock the pointer and send relative "
+ "movement — the expected behavior for games (mouse-look). Turn this off for "
+ "desktop use to keep the pointer free and send its absolute position instead. "
+ "The lock needs the stream full-screen and frontmost; it falls back to the "
+ "absolute pointer automatically (Stage Manager, Slide Over). Finger touch is "
+ "unaffected. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
}
#endif
@ViewBuilder private var compositorSection: some View { @ViewBuilder private var compositorSection: some View {
Section { Section {
Picker("Compositor", selection: $compositor) { Picker("Compositor", selection: $compositor) {
@@ -341,7 +623,7 @@ struct SettingsView: View {
Text("Which compositor drives the virtual output on the host. A specific " Text("Which compositor drives the virtual output on the host. A specific "
+ "choice is honored only if that backend is available there — " + "choice is honored only if that backend is available there — "
+ "otherwise the host falls back to auto-detection.") + "otherwise the host falls back to auto-detection.")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
} }
@@ -355,26 +637,50 @@ struct SettingsView: View {
} footer: { } footer: {
Text("Take the window fullscreen when a session starts and restore it on the host " Text("Take the window fullscreen when a session starts and restore it on the host "
+ "list, so only the stream is fullscreen — not the picker.") + "list, so only the stream is fullscreen — not the picker.")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
#endif #endif
} }
// Stage-2 (Metal/VTDecompressionSession) is the default and only user-visible presenter it
// recovers from a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard on a
// lost HEVC reference. Stage-1 is kept reachable as a DEBUG-only override for diagnostics, like
// the controller test. Empty in release builds (no presenter UI; stage-2 always).
@ViewBuilder private var presenterSection: some View { @ViewBuilder private var presenterSection: some View {
#if DEBUG
Section { Section {
Picker("Presenter", selection: $presenter) { Picker("Presenter", selection: $presenter) {
Text("Stage 1 (default)").tag("stage1") Text("Stage 2 (default)").tag("stage2")
Text("Stage 2 (experimental)").tag("stage2") Text("Stage 1 (debug)").tag("stage1")
} }
} header: { } header: {
Text("Video presenter") Text("Video presenter · debug")
} footer: { } footer: {
Text("Stage 1 feeds compressed video to the system display layer (known-good). " Text("Stage 2 (default) decodes explicitly and presents through Metal with a display "
+ "Stage 2 decodes explicitly and presents through Metal with a display " + "link — it adds a capture→present (glass-to-glass) latency line in the HUD and "
+ "link — it adds a capture→present (glass-to-glass) latency line in the HUD " + "self-recovers from decode stalls. Stage 1 feeds compressed video straight to the "
+ "and shortens the present tail. Applies from the next session.") + "system display layer; it freezes on a lost HEVC reference frame, so it's a debug "
.font(.caption) + "fallback only. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
@ViewBuilder private var hdrSection: some View {
Section {
Toggle("10-bit HDR", isOn: $hdrEnabled)
Toggle("Full chroma (4:4:4)", isOn: $enable444)
} header: {
Text("Video quality")
} footer: {
Text("HDR requests a 10-bit BT.2020 PQ (HDR10) stream — it only engages when the host is "
+ "sending HDR content AND this display supports HDR. 4:4:4 requests full chroma "
+ "(sharper text/UI, more bandwidth) — it only engages when this device can "
+ "hardware-decode it AND the host opted in. Otherwise the stream stays 8-bit "
+ "4:2:0 SDR. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
} }
@@ -392,7 +698,7 @@ struct SettingsView: View {
Text("Statistics") Text("Statistics")
} footer: { } footer: {
Text(Self.statisticsFooter) Text(Self.statisticsFooter)
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
} }
@@ -407,7 +713,7 @@ struct SettingsView: View {
+ "(Steam + custom) via the host's management API; tap a title to launch it. " + "(Steam + custom) via the host's management API; tap a title to launch it. "
+ "The host must expose that API on the LAN with a token " + "The host must expose that API on the LAN with a token "
+ "(serve --mgmt-bind 0.0.0.0 --mgmt-token …).") + "(serve --mgmt-bind 0.0.0.0 --mgmt-token …).")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
} }
@@ -441,7 +747,7 @@ struct SettingsView: View {
Text("Controllers") Text("Controllers")
} footer: { } footer: {
Text(Self.controllersFooter) Text(Self.controllersFooter)
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
} }
@@ -593,13 +899,13 @@ struct SettingsView: View {
} }
} }
} }
.font(.caption2) .font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
} }
Spacer() Spacer()
if gamepads.active?.id == controller.id { if gamepads.active?.id == controller.id {
Text("In use") Text("In use")
.font(.caption2.weight(.semibold)) .font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 8) .padding(.horizontal, 8)
.padding(.vertical, 3) .padding(.vertical, 3)
.background(Capsule().fill(.green.opacity(0.2))) .background(Capsule().fill(.green.opacity(0.2)))
@@ -621,6 +927,10 @@ struct SettingsView: View {
width = Int(max(bounds.width, bounds.height)) width = Int(max(bounds.width, bounds.height))
height = Int(min(bounds.width, bounds.height)) height = Int(min(bounds.width, bounds.height))
hz = UIScreen.main.maximumFramesPerSecond hz = UIScreen.main.maximumFramesPerSecond
#if os(iOS)
// The native mode is the "This device" wheel row, so leave Custom mode if it was on.
customMode = false
#endif
#endif #endif
} }
} }
@@ -631,3 +941,52 @@ extension Double {
Swift.min(Swift.max(self, lo), hi) Swift.min(Swift.max(self, lo), hi)
} }
} }
#if os(iOS)
/// The settings groups, mirroring the macOS preference tabs. On iPad each is a sidebar row that
/// drives the detail pane; on iPhone the same list collapses to pushed sub-pages. Internal (not
/// private) so the screenshot harness can open SettingsView on a specific category.
enum SettingsCategory: String, CaseIterable, Identifiable {
case general, display, audio, controllers, advanced, about
var id: Self { self }
var title: String {
switch self {
case .general: return "General"
case .display: return "Display"
case .audio: return "Audio"
case .controllers: return "Controllers"
case .advanced: return "Advanced"
case .about: return "About"
}
}
var symbol: String {
switch self {
case .general: return "gearshape"
case .display: return "display"
case .audio: return "speaker.wave.2"
case .controllers: return "gamecontroller"
case .advanced: return "slider.horizontal.3"
case .about: return "info.circle"
}
}
}
extension View {
/// Present the settings sheet large on iPad so the NavigationSplitView has room for its
/// sidebar + detail a default form sheet is too narrow and the split view would collapse to
/// the iPhone push list. No-op on iPhone (the standard sheet is already right) and on iOS 17
/// (no `presentationSizing` it falls back to the default sheet, which still degrades cleanly
/// to the push list).
@ViewBuilder
func settingsSheetSizing() -> some View {
if UIDevice.current.userInterfaceIdiom == .pad, #available(iOS 18, *) {
presentationSizing(.page)
} else {
self
}
}
}
#endif
@@ -52,7 +52,7 @@ struct SpeedTestSheet: View {
var body: some View { var body: some View {
VStack(spacing: 20) { VStack(spacing: 20) {
Label("Speed test — \(host.displayName)", systemImage: "gauge.with.needle") Label("Speed test — \(host.displayName)", systemImage: "gauge.with.needle")
.font(.headline) .font(.geist(17, .semibold, relativeTo: .headline))
.foregroundStyle(.tint) .foregroundStyle(.tint)
switch phase { switch phase {
@@ -73,7 +73,7 @@ struct SpeedTestSheet: View {
resultView(result) resultView(result)
case .failed(let message): case .failed(let message):
Text(message) Text(message)
.font(.callout) .font(.geist(16, relativeTo: .callout))
.foregroundStyle(.red) .foregroundStyle(.red)
.multilineTextAlignment(.center) .multilineTextAlignment(.center)
} }
@@ -149,13 +149,13 @@ struct SpeedTestSheet: View {
if let rec = Self.recommendedKbps(result) { if let rec = Self.recommendedKbps(result) {
Text("Recommended bitrate: \(Self.mbpsLabel(kbps: rec)) " Text("Recommended bitrate: \(Self.mbpsLabel(kbps: rec)) "
+ "(~70% of measured, headroom for encoder bursts).") + "(~70% of measured, headroom for encoder bursts).")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
.multilineTextAlignment(.center) .multilineTextAlignment(.center)
} else { } else {
Text("Too little data made it through to recommend a bitrate — " Text("Too little data made it through to recommend a bitrate — "
+ "check the network and retry.") + "check the network and retry.")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
.multilineTextAlignment(.center) .multilineTextAlignment(.center)
} }
@@ -69,19 +69,19 @@ struct StreamHUDView: View {
Text(model.mouseCaptured Text(model.mouseCaptured
? "⌘⎋ releases the mouse" ? "⌘⎋ releases the mouse"
: "Click the stream to capture input") : "Click the stream to capture input")
.font(.caption2) .font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
// The client-side cursor (C) draws the local cursor over the stream instead of // The client-side cursor (C) draws the local cursor over the stream instead of
// capturing it the only accurate cursor for gamescope, whose capture has none. // capturing it the only accurate cursor for gamescope, whose capture has none.
Text("⌘⇧C toggles the on-screen cursor") Text("⌘⇧C toggles the on-screen cursor")
.font(.caption2) .font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
#elseif os(iOS) #elseif os(iOS)
// Touch always plays directly; (hardware keyboard) toggles kb/mouse. // Touch always plays directly; (hardware keyboard) toggles kb/mouse.
Text(model.mouseCaptured Text(model.mouseCaptured
? "⌘⎋ releases keyboard & mouse" ? "⌘⎋ releases keyboard & mouse"
: "⌘⎋ captures keyboard & mouse") : "⌘⎋ captures keyboard & mouse")
.font(.caption2) .font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
#endif #endif
#if os(tvOS) #if os(tvOS)
@@ -89,13 +89,13 @@ struct StreamHUDView: View {
// A press (the focus engine consumes it before the host sees it). Disconnect is // A press (the focus engine consumes it before the host sees it). Disconnect is
// the Siri Remote's Menu button (.onExitCommand on the stream) just hint it. // the Siri Remote's Menu button (.onExitCommand on the stream) just hint it.
Text("Press Menu to disconnect") Text("Press Menu to disconnect")
.font(.caption) .font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
#else #else
// D lives on the app's Stream menu (so it still works when the HUD is hidden); // D lives on the app's Stream menu (so it still works when the HUD is hidden);
// this button is the in-overlay, click-to-disconnect affordance. // this button is the in-overlay, click-to-disconnect affordance.
Button("Disconnect (⌘D)") { model.disconnect() } Button("Disconnect (⌘D)") { model.disconnect() }
.font(.caption) .font(.geist(12, relativeTo: .caption))
#endif #endif
} }
.padding(10) .padding(10)
@@ -3,6 +3,7 @@
// or drops this and runs the PIN pairing ceremony instead. // or drops this and runs the PIN pairing ceremony instead.
import Foundation import Foundation
import PunktfunkKit
import SwiftUI import SwiftUI
struct TrustCardView: View { struct TrustCardView: View {
@@ -18,11 +19,11 @@ struct TrustCardView: View {
.font(.system(size: 36, weight: .light)) .font(.system(size: 36, weight: .light))
.foregroundStyle(.tint) .foregroundStyle(.tint)
Text("Verify \(hostName)") Text("Verify \(hostName)")
.font(.title3.weight(.semibold)) .font(.geist(20, .semibold, relativeTo: .title3))
Text("First connection. Compare this fingerprint with the one " Text("First connection. Compare this fingerprint with the one "
+ "punktfunk-host logged at startup (\u{201C}clients pin this " + "punktfunk-host logged at startup (\u{201C}clients pin this "
+ "fingerprint\u{201D}):") + "fingerprint\u{201D}):")
.font(.callout) .font(.geist(16, relativeTo: .callout))
.foregroundStyle(.secondary) .foregroundStyle(.secondary)
.multilineTextAlignment(.center) .multilineTextAlignment(.center)
Text(Self.format(fingerprint: fingerprint)) Text(Self.format(fingerprint: fingerprint))
@@ -58,7 +59,7 @@ struct TrustCardView: View {
#else #else
.buttonStyle(.borderless) .buttonStyle(.borderless)
#endif #endif
.font(.callout) .font(.geist(16, relativeTo: .callout))
} }
.padding(28) .padding(28)
.frame(maxWidth: 440) .frame(maxWidth: 440)
@@ -0,0 +1,101 @@
// Geist the punktfunk brand typeface (the same family the website ships). Bundled as static
// OTF weights in this kit's resources and registered with Core Text at first use, so it works
// identically in the Xcode app and the `swift run` dev shell (Bundle.module resolves to the
// package resource bundle in both). Geist Sans carries titles/UI; Geist Mono carries the technical
// readouts host addresses, status labels, the stream-stats HUD for the industrial look.
//
// Licensed under the SIL Open Font License 1.1 (Resources/Fonts/Geist-OFL.txt).
import CoreText
import SwiftUI
#if canImport(UIKit)
import UIKit
#elseif canImport(AppKit)
import AppKit
#endif
public enum BrandFont {
public enum Weight {
case regular, medium, semibold, bold
}
/// PostScript names of the bundled faces (verified from each OTF's name table). Geist Sans only
/// Geist Mono is intentionally not shipped; the app's typeface is Geist Sans throughout.
private static let sansFaces = ["Geist-Regular", "Geist-Medium", "Geist-SemiBold", "Geist-Bold"]
/// Registered exactly once per process a static `let` initializer is run lazily and is
/// guaranteed thread-safe + run-at-most-once by the runtime.
private static let registered: Void = {
for face in sansFaces {
guard let url = Bundle.module.url(
forResource: face, withExtension: "otf", subdirectory: "Fonts") else {
#if DEBUG
print("BrandFont: bundled face \(face).otf not found — text will fall back to system")
#endif
continue
}
var error: Unmanaged<CFError>?
if !CTFontManagerRegisterFontsForURL(url as CFURL, .process, &error) {
#if DEBUG
let message = error?.takeRetainedValue().localizedDescription ?? "unknown error"
print("BrandFont: failed to register \(face): \(message)")
#endif
}
}
}()
/// Force registration before the first `Font.custom` lookup. Cheap to call repeatedly.
public static func registerIfNeeded() { _ = registered }
fileprivate static func sansFace(_ weight: Weight) -> String {
switch weight {
case .regular: return "Geist-Regular"
case .medium: return "Geist-Medium"
case .semibold: return "Geist-SemiBold"
case .bold: return "Geist-Bold"
}
}
}
public extension Color {
/// The punktfunk brand purple (the app-icon lens / website `--brand`). Defined explicitly,
/// independent of the asset-catalog accent `Color.accentColor` resolution is environment- and
/// timing-sensitive (it can fall back to system blue), and the brand mark must never drift.
/// Light: #6656F2, Dark: #8678F5 (the lighter violet reads better on dark surfaces).
static let brand: Color = {
#if canImport(UIKit)
return Color(UIColor { traits in
traits.userInterfaceStyle == .dark
? UIColor(red: 0x86 / 255, green: 0x78 / 255, blue: 0xF5 / 255, alpha: 1)
: UIColor(red: 0x66 / 255, green: 0x56 / 255, blue: 0xF2 / 255, alpha: 1)
})
#elseif canImport(AppKit)
return Color(NSColor(name: nil) { appearance in
appearance.bestMatch(from: [.aqua, .darkAqua]) == .darkAqua
? NSColor(red: 0x86 / 255, green: 0x78 / 255, blue: 0xF5 / 255, alpha: 1)
: NSColor(red: 0x66 / 255, green: 0x56 / 255, blue: 0xF2 / 255, alpha: 1)
})
#else
// Non-Apple fallback: the light brand value, so all branches agree on a canonical color.
return Color(red: 0x66 / 255, green: 0x56 / 255, blue: 0xF2 / 255)
#endif
}()
}
public extension Font {
/// Geist Sans at an explicit point size, scaling with Dynamic Type relative to `textStyle`.
static func geist(
_ size: CGFloat, _ weight: BrandFont.Weight = .regular,
relativeTo textStyle: TextStyle = .body
) -> Font {
BrandFont.registerIfNeeded()
return .custom(BrandFont.sansFace(weight), size: size, relativeTo: textStyle)
}
/// Geist Sans at a FIXED point size that does not scale with Dynamic Type for glyphs pinned
/// inside a fixed-size container (e.g. the monogram tile), where a scaled letter would overflow.
static func geistFixed(_ size: CGFloat, _ weight: BrandFont.Weight = .regular) -> Font {
BrandFont.registerIfNeeded()
return .custom(BrandFont.sansFace(weight), fixedSize: size)
}
}
@@ -22,9 +22,22 @@ public enum DefaultsKey {
public static let speakerUID = "punktfunk.speakerUID" public static let speakerUID = "punktfunk.speakerUID"
public static let micUID = "punktfunk.micUID" public static let micUID = "punktfunk.micUID"
public static let presenter = "punktfunk.presenter" public static let presenter = "punktfunk.presenter"
/// Request a 10-bit BT.2020 PQ (HDR10) stream. On by default; only takes effect when the host
/// has HDR content AND this display supports HDR otherwise the stream stays 8-bit SDR.
public static let hdrEnabled = "punktfunk.hdrEnabled"
/// Request a full-chroma 4:4:4 stream when this device can HARDWARE-decode it (`Stage444Probe`).
/// On by default; only takes effect when the host also opted in to 4:4:4 (otherwise the stream
/// stays 4:2:0). Sharper text/UI at the cost of more bandwidth.
public static let enable444 = "punktfunk.enable444"
public static let hosts = "punktfunk.hosts" public static let hosts = "punktfunk.hosts"
/// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never". /// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never".
public static let cursorMode = "punktfunk.cursorMode" public static let cursorMode = "punktfunk.cursorMode"
/// iPad: capture the mouse/trackpad pointer (pointer lock relative movement) for games,
/// rather than forwarding an absolute cursor position. On by default. Only meaningful on iPad
/// with a hardware mouse/trackpad; the system grants the lock only to a full-screen, frontmost
/// scene and silently falls back to the absolute pointer when it can't (Stage Manager / Slide
/// Over). Read by `StreamViewController.prefersPointerLocked`.
public static let pointerCapture = "punktfunk.pointerCapture"
/// Experimental: show the host's game library (browsed over the management API). Off by default. /// Experimental: show the host's game library (browsed over the management API). Off by default.
public static let libraryEnabled = "punktfunk.libraryEnabled" public static let libraryEnabled = "punktfunk.libraryEnabled"
/// macOS: take the window fullscreen while streaming and restore it on the host list. On by default. /// macOS: take the window fullscreen while streaming and restore it on the host list. On by default.
@@ -68,6 +68,14 @@ private final class RumbleRenderer: @unchecked Sendable {
private var broken = false private var broken = false
/// Last logged active/silent state for a one-line transition log, not per-event spam. /// Last logged active/silent state for a one-line transition log, not per-event spam.
private var wasActive = false private var wasActive = false
// Backoff after an engine failure. A broken `gamecontrollerd.haptics` XPC connection (CoreHaptics
// -4811 "server connection broke") fails EVERY rebuild until the service relaunches and that
// break fires neither stoppedHandler nor resetHandler, so without a cooldown the next rumble
// update immediately rebuilds into the same dead connection, flooding the log and never
// recovering. Delay the next setup() growing 0.5124 s on repeated failure and clear it
// the moment a player runs cleanly (or the controller changes).
private var retryAfter = Date.distantPast
private var consecutiveFailures = 0
/// CHHapticEvent sharpness = actuator frequency. A DualSense's voice-coil motors need a /// CHHapticEvent sharpness = actuator frequency. A DualSense's voice-coil motors need a
/// defined frequency to move at all an intensity-only event (no sharpness) left them /// defined frequency to move at all an intensity-only event (no sharpness) left them
@@ -91,6 +99,8 @@ private final class RumbleRenderer: @unchecked Sendable {
self.closeHID() self.closeHID()
self.controller = c self.controller = c
self.broken = false self.broken = false
self.consecutiveFailures = 0
self.retryAfter = .distantPast
_ = self.openHIDIfDualSense(c) _ = self.openHIDIfDualSense(c)
onBackend?(self.backendNote(for: c)) onBackend?(self.backendNote(for: c))
} }
@@ -108,7 +118,7 @@ private final class RumbleRenderer: @unchecked Sendable {
// other pad (and for a DualSense whose HID device could not be opened). // other pad (and for a DualSense whose HID device could not be opened).
if self.hidRumble(low: lowAmp, high: highAmp) { return } if self.hidRumble(low: lowAmp, high: highAmp) { return }
guard !self.broken else { return } guard !self.broken else { return }
if active, self.low == nil, self.high == nil { if active, self.low == nil, self.high == nil, Date() >= self.retryAfter {
self.setup() self.setup()
} }
let ok: Bool let ok: Bool
@@ -124,8 +134,15 @@ private final class RumbleRenderer: @unchecked Sendable {
} }
// Rebuild on the next nonzero amplitude if an engine errored and tear down OUTSIDE // Rebuild on the next nonzero amplitude if an engine errored and tear down OUTSIDE
// the `inout` accesses above, so teardown() never mutates a motor that a `drive` call // the `inout` accesses above, so teardown() never mutates a motor that a `drive` call
// still holds an exclusive reference to. // still holds an exclusive reference to. Back off so a broken XPC isn't re-hit every
if !ok { self.teardown() } // update; once a player is actually running the path has recovered, so clear the backoff.
if !ok {
self.teardown()
self.scheduleRetryBackoff()
} else if self.low?.player != nil || self.high?.player != nil {
self.consecutiveFailures = 0
self.retryAfter = .distantPast
}
} }
} }
@@ -157,14 +174,29 @@ private final class RumbleRenderer: @unchecked Sendable {
low = makeMotor(haptics, .default) low = makeMotor(haptics, .default)
} }
if low == nil, high == nil { if low == nil, high == nil {
// Haptics present but no engine could be built right now (server busy / a transient // Haptics present but no engine could be built right now (server busy / XPC broken). Do
// error). Do NOT latch broken the next nonzero amplitude retries setup(). // NOT latch broken back off and the next nonzero amplitude past the cooldown retries.
log.warning("rumble: haptics present but engine setup failed — will retry on next rumble") log.warning("rumble: haptics present but engine setup failed — backing off, will retry")
scheduleRetryBackoff()
} }
} }
/// Push the next engine-build attempt out after a failure (capped exponential backoff), so a
/// broken `gamecontrollerd.haptics` connection gets time to relaunch instead of being re-hit on
/// every rumble update.
private func scheduleRetryBackoff() {
consecutiveFailures += 1
let shift = min(consecutiveFailures - 1, 4)
retryAfter = Date().addingTimeInterval(min(0.5 * Double(1 << shift), 4))
}
private func makeMotor(_ haptics: GCDeviceHaptics, _ locality: GCHapticsLocality) -> Motor? { private func makeMotor(_ haptics: GCDeviceHaptics, _ locality: GCHapticsLocality) -> Motor? {
guard let engine = haptics.createEngine(withLocality: locality) else { return nil } guard let engine = haptics.createEngine(withLocality: locality) else { return nil }
// A controller's motors carry no audio, so keep this engine OUT of the app's audio session
// (the default is to join it). Streaming keeps an AVAudioSession active the whole time;
// letting a haptics-only engine join it is a needless coupling that can get its
// gamecontrollerd XPC connection interrupted (the repeated -4811 server-connection breaks).
engine.playsHapticsOnly = true
// The haptic server can stop or reset the engine out from under us app backgrounding, an // The haptic server can stop or reset the engine out from under us app backgrounding, an
// audio-session interruption (a call, Siri, another audio app), or a server crash. Left // audio-session interruption (a call, Siri, another audio app), or a server crash. Left
// unhandled the players go dead and every later rumble throws, latching rumble off for the // unhandled the players go dead and every later rumble throws, latching rumble off for the
@@ -338,29 +370,32 @@ public final class GamepadFeedback {
// Hidout traffic (lightbar / player LEDs / triggers) only exists on a PlayStation-pad // Hidout traffic (lightbar / player LEDs / triggers) only exists on a PlayStation-pad
// session a DualSense or a DualShock 4 (lightbar only). Block briefly on it there and // session a DualSense or a DualShock 4 (lightbar only). Block briefly on it there and
// let rumble own the wait elsewhere; on an Xbox session it stays nonblocking. // let rumble own the wait elsewhere; on an Xbox session it stays nonblocking.
let hasHidout = connection.resolvedGamepad == .dualSense
|| connection.resolvedGamepad == .dualShock4
let hidTimeout: UInt32 = hasHidout ? 10 : 0
let thread = Thread { [connection, flag, drainDone, weak self] in let thread = Thread { [connection, flag, drainDone, weak self] in
while !flag.isStopped { while !flag.isStopped {
do { do {
if let r = try connection.nextRumble(timeoutMs: 10), r.pad == 0 { // Poll the feedback planes NON-BLOCKING. A blocking poll (timeoutMs > 0) holds
// the connection's shared feedback lock for its whole wait; the video pump drains
// HDR mastering metadata (nextHdrMeta) on the SAME lock every frame, so a blocking
// poll here starved it and throttled HDR to ~1 fps (SDR, which never drains HDR
// meta, was unaffected). Pacing with a short sleep OUTSIDE the lock (below) keeps
// rumble/HID latency low while leaving the lock free between polls.
if let r = try connection.nextRumble(timeoutMs: 0), r.pad == 0 {
self?.rumble.apply(low: r.low, high: r.high) self?.rumble.apply(low: r.low, high: r.high)
} }
// Drain a BOUNDED burst of hidout events: only the first poll waits, // Drain a BOUNDED burst of hidout events so sustained 0xCD traffic (a game writing
// and the cap + stop check keep sustained 0xCD traffic (a game writing // per-frame LED/trigger reports) can't spin here or block stop() past one cycle.
// per-frame LED/trigger reports) from starving the rumble poll above
// or blocking stop() past one cycle.
var burst = 0 var burst = 0
while burst < 64, !flag.isStopped, while burst < 64, !flag.isStopped,
let ev = try connection.nextHidOutput( let ev = try connection.nextHidOutput(timeoutMs: 0) {
timeoutMs: burst == 0 ? hidTimeout : 0) {
self?.render(ev) self?.render(ev)
burst += 1 burst += 1
} }
} catch { } catch {
break // .closed (or fatal) the session is over break // .closed (or fatal) the session is over
} }
// ~8 ms poll cadence (125 Hz), slept OUTSIDE the feedback lock low rumble/HID
// latency without holding the lock the HDR-meta drain needs.
if !flag.isStopped { Thread.sleep(forTimeInterval: 0.008) }
} }
drainDone.signal() drainDone.signal()
} }
@@ -160,7 +160,13 @@ public final class InputCapture {
previous.onPreempted?() previous.onPreempted?()
} }
Self.activeCapture = self Self.activeCapture = self
if let mouse = GCMouse.current { attach(mouse: mouse) } // Attach EVERY connected mouse, not just GCMouse.current. With two pointing devices (e.g.
// the iPad's own Magic Keyboard trackpad AND a Universal Control "V-UC Automouse"), only one
// is `current` at a time; attaching just that one left the OTHER device's motion handler
// uninstalled, so moving it did nothing. Each GCMouse delivers its own deltas through its own
// handler, so handling all of them lets either device drive. New arrivals are caught by the
// GCMouseDidConnect observer below.
for mouse in GCMouse.mice() { attach(mouse: mouse) }
if let keyboard = GCKeyboard.coalesced { attach(keyboard: keyboard) } if let keyboard = GCKeyboard.coalesced { attach(keyboard: keyboard) }
observers.append(NotificationCenter.default.addObserver( observers.append(NotificationCenter.default.addObserver(
forName: .GCMouseDidConnect, object: nil, queue: .main forName: .GCMouseDidConnect, object: nil, queue: .main
@@ -27,10 +27,35 @@ public enum Licenses {
+ apache + apache
} }
/// The bundled brand typeface (Geist Sans + Geist Mono) SIL Open Font License 1.1. The
/// license file ships alongside the OTFs in `Resources/Fonts/`, satisfying the OFL's
/// distribution requirement; this surfaces it in the Acknowledgements screen too.
public static var fontLicense: String {
guard let url = Bundle.module.url(
forResource: "Geist-OFL", withExtension: "txt", subdirectory: "Fonts"),
let text = try? String(contentsOf: url, encoding: .utf8)
else { return "" }
return text
}
/// Third-party software notices for the linked Rust crates (generated by /// Third-party software notices for the linked Rust crates (generated by
/// `scripts/gen-third-party-notices.sh`). /// `scripts/gen-third-party-notices.sh`).
public static var thirdPartyNotices: String { public static var thirdPartyNotices: String {
let text = resource("THIRD-PARTY-NOTICES") let text = resource("THIRD-PARTY-NOTICES")
return text.isEmpty ? "Third-party notices unavailable." : text return text.isEmpty ? "Third-party notices unavailable." : text
} }
/// `thirdPartyNotices` pre-split into render-sized line chunks. The full notices are ~885 KB /
/// 16k lines; a single SwiftUI `Text` that large overshoots CoreText/CoreAnimation's max
/// renderable height it lays out for ages and draws blank past the limit so the
/// Acknowledgements screen renders these chunks in a `LazyVStack` (only on-screen chunks lay
/// out, and no chunk is tall enough to clip). Split at line boundaries and joined with "\n";
/// the inter-chunk break is the `LazyVStack` row boundary, so no text is lost. Computed once.
public static let thirdPartyNoticesChunks: [String] = {
let lines = thirdPartyNotices.split(separator: "\n", omittingEmptySubsequences: false)
let chunkSize = 200
return stride(from: 0, to: lines.count, by: chunkSize).map { start in
lines[start..<min(start + chunkSize, lines.count)].joined(separator: "\n")
}
}()
} }
@@ -1,21 +1,35 @@
// Stage-2 presenter, present half: draw a decoded NV12 CVPixelBuffer into a CAMetalLayer // Stage-2 presenter, present half: draw a decoded NV12 / P010 / 4:4:4 CVPixelBuffer into a CAMetalLayer
// drawable with a BT.709 YUVRGB shader. The display link (owned by the hosting view) drives // drawable with a YCbCrRGB shader. The hosting view's CADisplayLink drives `render` once per vsync
// `render` once per vsync with the target present time, so a present can finally be stamped and // (via Stage2Pipeline.renderTick) with the target present time, so a present can be stamped and the
// the present tail hand-paced. See docs apple-stage2-presenter.md. // present tail hand-paced. See docs apple-stage2-presenter.md.
// //
// Main-thread only: created during view setup, `render` called from the view's CADisplayLink // Main-thread only: created during view setup, `render`/`configure` called from the view's CADisplayLink
// (which fires on the main runloop). The Metal objects + texture cache are touched only here. // (which fires on the main runloop). The Metal objects + texture cache are touched only here. The one
// exception is `setHdrMeta`, called from the pump thread it hops the layer write to main so every
// CALayer mutation stays on one thread.
#if canImport(Metal) && canImport(QuartzCore) #if canImport(Metal) && canImport(QuartzCore)
import CoreGraphics import CoreGraphics
import CoreVideo import CoreVideo
import Metal import Metal
import QuartzCore import QuartzCore
import os
/// Runtime-compiled (no metallib build step needed in SwiftPM): a fullscreen triangle and a private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter")
/// BT.709 limited-range NV12RGB fragment shader. uv.y is flipped (1 - p.y) so the top-left-
/// origin texture presents upright (NDC y is up), not upside down. (Colorspace is BT.709 SDR /// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the
/// for now matches the host; 10-bit/HDR + other matrices are a later tie-in.) /// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors
/// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the
/// brighter highlights into the panel's headroom. This is the missing anchor that made the old HDR path
/// render "way too bright" (no `edrMetadata` no reference-white anchoring); a LARGER value renders
/// dimmer. Matches the host's standard PQ reference white.
private let hdrReferenceWhiteNits: Float = 203.0
/// Runtime-compiled (no metallib build step needed in SwiftPM): a fullscreen triangle and BT.709 SDR
/// and BT.2020-PQ HDR YCbCrRGB fragment shaders. uv.y is flipped (1 - p.y) so the top-left-origin
/// texture presents upright (NDC y is up). The HDR shader outputs PQ-encoded RGB as-is the
/// CAMetalLayer's `itur_2100_PQ` colour space + `edrMetadata` tell the system compositor the samples
/// are PQ and how to tone-map them (no EOTF here, matching the host's BT.2020 PQ emission).
private let shaderSource = """ private let shaderSource = """
#include <metal_stdlib> #include <metal_stdlib>
using namespace metal; using namespace metal;
@@ -30,11 +44,46 @@ vertex VOut pf_vtx(uint vid [[vertex_id]]) {
return o; return o;
} }
// Bicubic (Catmull-Rom) sampling of the single-channel luma plane. When the drawable is larger
// than the decoded frame (a window/view bigger than the host's fixed mode), a bilinear upscale
// looks soft; Catmull-Rom keeps edges crisp — matching AVSampleBufferDisplayLayer's (stage-1)
// scaler — and reduces to the exact texel at 1:1, so a native-resolution present stays pixel-exact.
// Nine bilinear taps (TheRealMJP's optimisation of the 16-tap kernel); `s` MUST be a linear
// sampler. Luma carries the perceived detail, so only it gets bicubic; chroma stays bilinear.
float catmullRomLuma(texture2d<float> tex, sampler s, float2 uv) {
float2 texSize = float2(tex.get_width(), tex.get_height());
float2 samplePos = uv * texSize;
float2 tc1 = floor(samplePos - 0.5) + 0.5;
float2 f = samplePos - tc1;
float2 w0 = f * (-0.5 + f * (1.0 - 0.5 * f));
float2 w1 = 1.0 + f * f * (-2.5 + 1.5 * f);
float2 w2 = f * (0.5 + f * (2.0 - 1.5 * f));
float2 w3 = f * f * (-0.5 + 0.5 * f);
float2 w12 = w1 + w2;
float2 off12 = w2 / w12;
float2 tc0 = (tc1 - 1.0) / texSize;
float2 tc3 = (tc1 + 2.0) / texSize;
float2 tc12 = (tc1 + off12) / texSize;
float r = 0.0;
r += tex.sample(s, float2(tc0.x, tc0.y)).r * (w0.x * w0.y);
r += tex.sample(s, float2(tc12.x, tc0.y)).r * (w12.x * w0.y);
r += tex.sample(s, float2(tc3.x, tc0.y)).r * (w3.x * w0.y);
r += tex.sample(s, float2(tc0.x, tc12.y)).r * (w0.x * w12.y);
r += tex.sample(s, float2(tc12.x, tc12.y)).r * (w12.x * w12.y);
r += tex.sample(s, float2(tc3.x, tc12.y)).r * (w3.x * w12.y);
r += tex.sample(s, float2(tc0.x, tc3.y)).r * (w0.x * w3.y);
r += tex.sample(s, float2(tc12.x, tc3.y)).r * (w12.x * w3.y);
r += tex.sample(s, float2(tc3.x, tc3.y)).r * (w3.x * w3.y);
return r;
}
// SDR: 8-bit NV12 / 4:4:4 (BT.709, limited/video range) → full-range RGB. Chroma is sampled at the
// same UV as luma, so a full-size 4:4:4 chroma plane needs no shader change vs 4:2:0.
fragment float4 pf_frag(VOut in [[stage_in]], fragment float4 pf_frag(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]], texture2d<float> lumaTex [[texture(0)]],
texture2d<float> chromaTex [[texture(1)]]) { texture2d<float> chromaTex [[texture(1)]]) {
constexpr sampler s(filter::linear, address::clamp_to_edge); constexpr sampler s(filter::linear, address::clamp_to_edge);
float y = lumaTex.sample(s, in.uv).r; float y = catmullRomLuma(lumaTex, s, in.uv);
float2 c = chromaTex.sample(s, in.uv).rg; float2 c = chromaTex.sample(s, in.uv).rg;
// BT.709, 8-bit limited (video) range → full-range RGB. // BT.709, 8-bit limited (video) range → full-range RGB.
y = (y - 16.0/255.0) * (255.0/219.0); y = (y - 16.0/255.0) * (255.0/219.0);
@@ -46,18 +95,18 @@ fragment float4 pf_frag(VOut in [[stage_in]],
return float4(saturate(float3(r, g, b)), 1.0); return float4(saturate(float3(r, g, b)), 1.0);
} }
// HDR: 10-bit P010 (BT.2020, limited range), Y'CbCr that is PQ-encoded. We apply the BT.2020 // HDR: 10-bit P010 / 4:4:4 (BT.2020, limited range), YCbCr that is PQ-encoded. We apply the BT.2020
// matrix to get PQ-encoded R'G'B' and output it as-is — the CAMetalLayer's itur_2100_PQ colour // matrix to get PQ-encoded RGB and output it as-is — the CAMetalLayer's itur_2100_PQ colour space
// space + EDR tells the compositor the samples are PQ, so it does the PQ→display mapping. No EOTF // + edrMetadata tell the compositor the samples are PQ, so it does the PQ→display tone-map. No EOTF
// here (matching the host, which emitted BT.2020 PQ). P010 stores the 10-bit code in the high bits // here. P010/x444 store the 10-bit code in the high bits of each 16-bit sample, so an .r16Unorm sample
// of each 16-bit sample, so an .r16Unorm sample reads ~code/1023 (the /1024 vs /1023 error is < 0.1%). // reads ~code/1023 (the /1024 vs /1023 error is < 0.1%).
fragment float4 pf_frag_hdr(VOut in [[stage_in]], fragment float4 pf_frag_hdr(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]], texture2d<float> lumaTex [[texture(0)]],
texture2d<float> chromaTex [[texture(1)]]) { texture2d<float> chromaTex [[texture(1)]]) {
constexpr sampler s(filter::linear, address::clamp_to_edge); constexpr sampler s(filter::linear, address::clamp_to_edge);
float y = lumaTex.sample(s, in.uv).r; float y = catmullRomLuma(lumaTex, s, in.uv);
float2 c = chromaTex.sample(s, in.uv).rg; float2 c = chromaTex.sample(s, in.uv).rg;
// BT.2020 10-bit limited (video) range → full-range PQ R'G'B'. // BT.2020 10-bit limited (video) range → full-range PQ RGB.
y = (y - 64.0/1023.0) * (1023.0/876.0); y = (y - 64.0/1023.0) * (1023.0/876.0);
float u = (c.x - 512.0/1023.0) * (1023.0/896.0); float u = (c.x - 512.0/1023.0) * (1023.0/896.0);
float v = (c.y - 512.0/1023.0) * (1023.0/896.0); float v = (c.y - 512.0/1023.0) * (1023.0/896.0);
@@ -74,21 +123,34 @@ public final class MetalVideoPresenter {
private let device: MTLDevice private let device: MTLDevice
private let queue: MTLCommandQueue private let queue: MTLCommandQueue
/// SDR (BT.709 8-bit NV12 bgra8) and HDR (BT.2020 PQ 10-bit P010 rgba16Float) pipelines. /// SDR (BT.709 8-bit bgra8) and HDR (BT.2020 PQ 10-bit rgba16Float) pipelines. Selected per
/// Selected per frame by `render`; the layer is reconfigured when the mode flips (HDR toggle). /// frame in `render`; the layer is reconfigured to match when the session flips (HDR toggle).
private let pipelineSDR: MTLRenderPipelineState private let pipelineSDR: MTLRenderPipelineState
private let pipelineHDR: MTLRenderPipelineState private let pipelineHDR: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache? private var textureCache: CVMetalTextureCache?
/// Current layer configuration switched lazily in `configure(hdr:)` when a frame's mode differs.
private var hdrActive = false
/// nil if Metal is unavailable (no GPU / a headless CI) the caller falls back to stage-1. /// Current layer configuration switched in `configure(hdr:)` when a frame's HDR-ness differs.
public init?() { /// Main-thread only (read + written from `render`/`configure`, all on the display-link runloop).
private var hdrActive = false
/// Last HDR mastering grade received via `setHdrMeta` (the host's 0xCE). Cached so a mid-session
/// SDRHDR flip's `configureColor` re-applies the real grade instead of clobbering it back to the
/// bare reference-white anchor (an out-of-order race otherwise: `setHdrMeta` and the flip both write
/// `edrMetadata`). Main-thread only.
private var lastHdrMeta: PunktfunkConnection.HdrMeta?
#if DEBUG
/// Last logged "decodeddrawable" signature, so the diagnostic logs only on a size/HDR change.
private var lastSizeSig = ""
#endif
/// nil if Metal is unavailable (no GPU / a headless CI) or a shader fails to compile the caller
/// falls back to stage-1.
public static func make() -> MetalVideoPresenter? {
guard let device = MTLCreateSystemDefaultDevice(), guard let device = MTLCreateSystemDefaultDevice(),
let queue = device.makeCommandQueue() let queue = device.makeCommandQueue()
else { return nil } else { return nil }
self.device = device let pipelineSDR: MTLRenderPipelineState
self.queue = queue let pipelineHDR: MTLRenderPipelineState
do { do {
let library = try device.makeLibrary(source: shaderSource, options: nil) let library = try device.makeLibrary(source: shaderSource, options: nil)
let vtx = library.makeFunction(name: "pf_vtx") let vtx = library.makeFunction(name: "pf_vtx")
@@ -105,76 +167,148 @@ public final class MetalVideoPresenter {
} catch { } catch {
return nil return nil
} }
CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, device, nil, &textureCache) var cache: CVMetalTextureCache?
guard textureCache != nil else { return nil } CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, device, nil, &cache)
guard let textureCache = cache else { return nil }
let layer = CAMetalLayer() let layer = CAMetalLayer()
layer.device = device layer.device = device
layer.pixelFormat = .bgra8Unorm layer.pixelFormat = .bgra8Unorm
layer.framebufferOnly = true layer.framebufferOnly = true
layer.isOpaque = true layer.isOpaque = true
// Triple-buffer: more in-flight drawables before `nextDrawable()` (called on the
// display-link / MAIN thread) has to block waiting for one to free.
layer.maximumDrawableCount = 3
#if os(macOS) #if os(macOS)
// The display link already paces exactly one present per vsync. Leaving the layer's // The display link already paces exactly one present per vsync. Leaving the layer's own vsync
// own vsync wait on means `commandBuffer.present` ALSO blocks for the hardware vsync, // wait on means `commandBuffer.present` ALSO blocks for the hardware vsync, so `nextDrawable()`
// so `nextDrawable()` stalls the MAIN thread until a drawable frees windowed, the // stalls the MAIN thread until a drawable frees windowed, the WindowServer's looser
// WindowServer's looser compositing hides it; FULLSCREEN's tighter, more-direct path // compositing hides it; FULLSCREEN's tighter path serializes the main thread to the display and
// serializes the main thread to the display and the stall surfaces as bad judder. // the stall surfaces as bad judder. Disabling the layer-level sync lets present return promptly
// Disabling the layer-level sync lets present return promptly (the display link is the // (the display link is the pacing source) the fix for the fullscreen stutter. macOS-only.
// pacing source), which is what fixes the fullscreen stutter. macOS-only property.
layer.displaySyncEnabled = false layer.displaySyncEnabled = false
#endif #endif
// Render the drawable at the DECODED frame's resolution (set per-frame in `render`) and let the
// system compositor scale it to the layer's bounds the same `.resizeAspect` path stage-1's
// AVSampleBufferDisplayLayer uses. A native-resolution present is then pixel-exact (1:1, no
// shader scaling); a resized window rescales via the system's scaler.
layer.contentsGravity = .resizeAspect
// Triple-buffer: more in-flight drawables before `nextDrawable()` (called on the display-link /
// MAIN thread) has to block waiting for one to free.
layer.maximumDrawableCount = 3
return MetalVideoPresenter(
device: device, queue: queue, pipelineSDR: pipelineSDR, pipelineHDR: pipelineHDR,
textureCache: textureCache, layer: layer)
}
private init(
device: MTLDevice, queue: MTLCommandQueue, pipelineSDR: MTLRenderPipelineState,
pipelineHDR: MTLRenderPipelineState, textureCache: CVMetalTextureCache, layer: CAMetalLayer
) {
self.device = device
self.queue = queue
self.pipelineSDR = pipelineSDR
self.pipelineHDR = pipelineHDR
self.textureCache = textureCache
self.layer = layer self.layer = layer
} }
/// Track the stream mode (the host can Reconfigure mid-stream). Size is in pixels. /// Configure the layer + active pipeline for an SDR or HDR session. MAIN THREAD ONLY. Called once at
public func setDrawableSize(_ size: CGSize) { /// session start and again per-frame from `render` (idempotent the guard makes a same-state call a
guard size.width > 0, size.height > 0 else { return } /// no-op), so a mid-session HDR toggle (the host re-inits its encoder; the decoded `frame.isHDR`
if layer.drawableSize != size { layer.drawableSize = size } /// flips) reconfigures here automatically. HDR uses an rgba16Float drawable + BT.2020 PQ colour space
} /// + EDR with a 203-nit reference-white anchor; SDR uses the plain 8-bit sRGB path.
public func configure(hdr: Bool) {
/// Reconfigure the layer for SDR or HDR when the stream mode flips (HDR toggle). HDR uses an
/// rgba16Float drawable + a BT.2020 PQ colour space + EDR, so the compositor PQ-maps to the
/// display; SDR uses the plain 8-bit sRGB path. Main-thread only (called from `render`).
private func configure(hdr: Bool) {
guard hdr != hdrActive else { return } guard hdr != hdrActive else { return }
hdrActive = hdr hdrActive = hdr
configureColor(hdr: hdr)
}
/// Set the layer's pixel format + colour config for SDR or HDR. MAIN THREAD ONLY. EDR is requested
/// on macOS + iOS (the old `#if os(macOS)` guard left iOS EDR half-engaged). tvOS has NO EDR API
/// (`wantsExtendedDynamicRangeContent`/`edrMetadata`/`CAEDRMetadata` are all unavailable there), so
/// it gets the PQ pixel format + colour space only the tvOS compositor tone-maps from those.
private func configureColor(hdr: Bool) {
if hdr { if hdr {
layer.pixelFormat = .rgba16Float layer.pixelFormat = .rgba16Float
layer.colorspace = CGColorSpace(name: CGColorSpace.itur_2100_PQ) layer.colorspace = CGColorSpace(name: CGColorSpace.itur_2100_PQ)
#if os(macOS) #if !os(tvOS)
layer.wantsExtendedDynamicRangeContent = true layer.wantsExtendedDynamicRangeContent = true
// Anchor reference white. Re-apply the real grade if one already arrived (0xCE before the
// flip); otherwise the bare 203-nit anchor. Without this anchor the PQ signal is too bright.
layer.edrMetadata = makeEDR(lastHdrMeta)
#endif #endif
} else { } else {
// SDR: gamma-encoded BT.709 [0,1] in an 8-bit drawable; a nil colorspace tags it device/sRGB
// (the proven SDR path never showed the "too bright" issue, which was HDR-only).
layer.pixelFormat = .bgra8Unorm layer.pixelFormat = .bgra8Unorm
layer.colorspace = nil layer.colorspace = nil
#if os(macOS) #if !os(tvOS)
layer.wantsExtendedDynamicRangeContent = false layer.wantsExtendedDynamicRangeContent = false
layer.edrMetadata = nil
#endif #endif
} }
} }
/// Draw one decoded frame to the next drawable and present it. `isHDR` selects the 10-bit #if !os(tvOS)
/// BT.2020 PQ path (P010 input) vs the 8-bit BT.709 path (NV12 input). Returns true on success; private func makeEDR(_ meta: PunktfunkConnection.HdrMeta?) -> CAEDRMetadata {
/// false when there's no drawable yet, a texture couldn't be made, or Metal errored the CAEDRMetadata.hdr10(
/// caller then doesn't stamp a present for this frame. displayInfo: meta?.masteringDisplayColorVolume(),
contentInfo: meta?.contentLightLevelInfo(),
opticalOutputScale: hdrReferenceWhiteNits)
}
#endif
/// Update the HDR mastering metadata (drained from the host's 0xCE datagram) to refine the system
/// tone-map from the real grade. Called from the PUMP thread, so the layer write is hopped to MAIN
/// (every CALayer mutation stays on one thread). The grade is cached so a later SDRHDR
/// `configureColor` re-applies it; the `edrMetadata` write is gated on `hdrActive` (setting it on an
/// SDR layer is harmless but pointless, and the flip will apply it anyway).
public func setHdrMeta(_ meta: PunktfunkConnection.HdrMeta) {
DispatchQueue.main.async { [weak self] in
guard let self else { return }
self.lastHdrMeta = meta
// tvOS has no edrMetadata the cached grade is still kept above (harmless), it just can't
// be applied to the layer there. macOS/iOS refine the system tone-map from the real grade.
#if !os(tvOS)
if self.hdrActive { self.layer.edrMetadata = self.makeEDR(meta) }
#endif
}
}
/// Draw one decoded frame to the next drawable and present it. MAIN THREAD (the display link).
/// `isHDR` selects the 10-bit BT.2020 PQ path vs the 8-bit BT.709 path and is reconciled with the
/// layer config via `configure`. Returns true on success; false when there's no drawable yet, a
/// texture couldn't be made, or Metal errored the caller then doesn't stamp a present.
@discardableResult @discardableResult
public func render(_ pixelBuffer: CVPixelBuffer, isHDR: Bool = false) -> Bool { public func render(_ pixelBuffer: CVPixelBuffer, isHDR: Bool = false) -> Bool {
// Reconcile the layer with the decoded frame's HDR-ness (handles a mid-session SDRHDR flip).
configure(hdr: isHDR) configure(hdr: isHDR)
// P010 stores 10-bit luma/chroma in 16-bit samples R16/RG16; NV12 is 8-bit R8/RG8.
let lumaFmt: MTLPixelFormat = isHDR ? .r16Unorm : .r8Unorm // P010/x444 store 10-bit luma/chroma in 16-bit samples R16/RG16; NV12/444v is 8-bit R8/RG8.
let chromaFmt: MTLPixelFormat = isHDR ? .rg16Unorm : .rg8Unorm // Derived from the actual decoded buffer so a 4:4:4 (full chroma plane) frame just works.
let pf = CVPixelBufferGetPixelFormatType(pixelBuffer)
let tenBit =
pf == kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange
|| pf == kCVPixelFormatType_420YpCbCr10BiPlanarFullRange
|| pf == kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange
|| pf == kCVPixelFormatType_444YpCbCr10BiPlanarFullRange
guard let textureCache, guard let textureCache,
let luma = makeTexture(pixelBuffer, plane: 0, format: lumaFmt, cache: textureCache), let luma = makeTexture(
let chroma = makeTexture(pixelBuffer, plane: 1, format: chromaFmt, cache: textureCache) pixelBuffer, plane: 0, format: tenBit ? .r16Unorm : .r8Unorm, cache: textureCache),
let chroma = makeTexture(
pixelBuffer, plane: 1, format: tenBit ? .rg16Unorm : .rg8Unorm, cache: textureCache)
else { return false } else { return false }
// The hosting view owns drawableSize (aspect-fit to its bounds); skip until it's laid // Size the drawable to the decoded frame so the fullscreen triangle samples 1:1 (pixel-exact);
// out. The fullscreen triangle scales the decoded texture to fill the drawable. // the layer's contentsGravity then scales it to the on-screen bounds via the system compositor
guard layer.drawableSize.width > 0, layer.drawableSize.height > 0, // (matching stage-1). drawableSize does NOT track bounds (defaults to 0), so set it BEFORE
let drawable = layer.nextDrawable(), // nextDrawable; re-set only on a change (first frame / Reconfigure / HDR flip).
let decodedSize = CGSize(
width: CVPixelBufferGetWidth(pixelBuffer), height: CVPixelBufferGetHeight(pixelBuffer))
if layer.drawableSize != decodedSize { layer.drawableSize = decodedSize }
#if DEBUG
logSizeIfChanged(decoded: decodedSize)
#endif
guard let drawable = layer.nextDrawable(),
let commandBuffer = queue.makeCommandBuffer() let commandBuffer = queue.makeCommandBuffer()
else { return false } else { return false }
@@ -186,24 +320,23 @@ public final class MetalVideoPresenter {
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else { guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false return false
} }
encoder.setRenderPipelineState(isHDR ? pipelineHDR : pipelineSDR) encoder.setRenderPipelineState(hdrActive ? pipelineHDR : pipelineSDR)
encoder.setFragmentTexture(CVMetalTextureGetTexture(luma), index: 0) encoder.setFragmentTexture(CVMetalTextureGetTexture(luma), index: 0)
encoder.setFragmentTexture(CVMetalTextureGetTexture(chroma), index: 1) encoder.setFragmentTexture(CVMetalTextureGetTexture(chroma), index: 1)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3) encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding() encoder.endEncoding()
commandBuffer.present(drawable) // present at the next vsync lowest latency commandBuffer.present(drawable) // present at the next vsync lowest latency
// Hold the CVMetalTextures + the source pixel buffer (its IOSurface) alive until the GPU // Hold the CVMetalTextures + source pixel buffer (its IOSurface) alive until the GPU finishes
// finishes sampling releasing them at scope exit could free the backing mid-read. // sampling releasing them at scope exit could free the backing mid-read.
commandBuffer.addCompletedHandler { _ in _ = (luma, chroma, pixelBuffer) } commandBuffer.addCompletedHandler { _ in _ = (luma, chroma, pixelBuffer) }
commandBuffer.commit() commandBuffer.commit()
return true return true
} }
/// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past /// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the
/// the draw the MTLTexture is only valid while its CVMetalTexture is retained. /// draw the MTLTexture is only valid while its CVMetalTexture is retained.
private func makeTexture( private func makeTexture(
_ pixelBuffer: CVPixelBuffer, plane: Int, format: MTLPixelFormat, _ pixelBuffer: CVPixelBuffer, plane: Int, format: MTLPixelFormat, cache: CVMetalTextureCache
cache: CVMetalTextureCache
) -> CVMetalTexture? { ) -> CVMetalTexture? {
let w = CVPixelBufferGetWidthOfPlane(pixelBuffer, plane) let w = CVPixelBufferGetWidthOfPlane(pixelBuffer, plane)
let h = CVPixelBufferGetHeightOfPlane(pixelBuffer, plane) let h = CVPixelBufferGetHeightOfPlane(pixelBuffer, plane)
@@ -215,5 +348,16 @@ public final class MetalVideoPresenter {
else { return nil } else { return nil }
return cvTexture return cvTexture
} }
#if DEBUG
private func logSizeIfChanged(decoded: CGSize) {
let sig = "\(Int(decoded.width))x\(Int(decoded.height))|hdr\(hdrActive ? 1 : 0)"
if sig != lastSizeSig {
lastSizeSig = sig
let msg = "stage2: decoded \(Int(decoded.width))x\(Int(decoded.height)) hdr=\(hdrActive)"
presenterLog.info("\(msg, privacy: .public)")
}
}
#endif
} }
#endif #endif
@@ -0,0 +1,94 @@
// Steers the system's iPad pointer-lock resolution down to a chosen "anchor" view controller.
//
// `UIViewController.prefersPointerLocked` is resolved the same way as the status bar: the system
// walks DOWN from the window's root view controller through `childViewControllerForPointerLock`.
// SwiftUI's hosting / container view controllers do NOT forward that query to their children, so a
// `UIViewControllerRepresentable` controller buried in the SwiftUI tree (our StreamViewController)
// is never consulted its `prefersPointerLocked = true` is silently ignored and a Magic Keyboard
// trackpad / mouse falls through to the absolute-pointer path instead of being captured.
//
// Swizzling the DEFAULT implementation isn't enough: the controllers that break the chain
// (UIHostingController and SwiftUI's internal containers) provide their OWN implementation of the
// property, so a base-class swizzle never runs for them. Instead we walk UP the LIVE `parent`
// chain from the anchor to the window root and, on each real ancestor, force
// `childViewControllerForPointerLock` to return the next controller toward the anchor. Each forced
// value is a genuine direct child (we follow the actual containment chain), so the system's
// downward walk reaches the anchor and reads its `prefersPointerLocked`.
//
// The forcing is per-INSTANCE an associated object gated behind a one-time per-CLASS IMP
// swizzle. Only the specific controllers in the anchor's chain are affected; every other instance
// of those classes keeps its original behavior (associated object nil original IMP). The forced
// values are cleared on disengage so the long-lived SwiftUI parents don't retain a stale controller
// across sessions. Only the PUBLIC `childViewControllerForPointerLock` selector is touched
// (App-Store-safe; no private API).
#if os(iOS)
import ObjectiveC
import UIKit
enum PointerLockChain {
private static var forcedChildKey: UInt8 = 0
/// Classes whose `childViewControllerForPointerLock` we've already IMP-swizzled (keyed by the
/// class object). Main-thread only pointer-lock resolution and capture toggles are all main.
private static var swizzledClasses = Set<ObjectIdentifier>()
/// Ancestors we've stamped with a forced child this engagement, held weakly so a deallocated
/// SwiftUI controller drops out on its own (no dangling). disengage() clears every one even
/// if the live `parent` chain has since broken so a stamped parent can never retain a stale
/// controller subtree across sessions. One anchor is ever engaged at a time.
private static let stampedParents = NSHashTable<UIViewController>.weakObjects()
private static func forcedChild(of vc: UIViewController) -> UIViewController? {
objc_getAssociatedObject(vc, &forcedChildKey) as? UIViewController
}
private static func setForcedChild(_ child: UIViewController?, on vc: UIViewController) {
// RETAIN: while steering, the parent must keep the toward-anchor child alive. It's also
// already a strong child of `vc` via UIKit containment, so this adds no cycle (the reverse
// `.parent` link is weak), and disengage() always clears it so it can't outlive a session.
objc_setAssociatedObject(vc, &forcedChildKey, child, .OBJC_ASSOCIATION_RETAIN_NONATOMIC)
}
/// Ensure `cls`'s `childViewControllerForPointerLock` getter consults the per-instance forced
/// child first, falling back to the class's original implementation. Idempotent per class.
private static func ensureSwizzled(_ cls: AnyClass) {
let id = ObjectIdentifier(cls)
guard !swizzledClasses.contains(id) else { return }
swizzledClasses.insert(id)
let selector = #selector(getter: UIViewController.childViewControllerForPointerLock)
guard let method = class_getInstanceMethod(cls, selector) else { return }
let originalIMP = method_getImplementation(method)
typealias OriginalFn = @convention(c) (AnyObject, Selector) -> UIViewController?
let original = unsafeBitCast(originalIMP, to: OriginalFn.self)
let forwarding: @convention(block) (UIViewController) -> UIViewController? = { vc in
if let forced = forcedChild(of: vc) { return forced }
return original(vc, selector)
}
method_setImplementation(method, imp_implementationWithBlock(forwarding))
}
/// Force every ancestor of `anchor` to forward pointer-lock resolution toward it, then ask the
/// system to re-resolve. No-op when `anchor` isn't in a view-controller hierarchy yet (it
/// re-runs from the anchor's appearance/parent callbacks once it is).
static func engage(_ anchor: UIViewController) {
disengage(anchor) // clear any prior engagement first (reparent / re-anchor)
var child = anchor
while let parent = child.parent {
ensureSwizzled(object_getClass(parent)!)
setForcedChild(child, on: parent)
stampedParents.add(parent)
child = parent
}
anchor.setNeedsUpdateOfPrefersPointerLocked()
}
/// Clear the forced forwarding on every stamped ancestor (so the SwiftUI parents stop retaining
/// the anchor's subtree) and re-resolve to drop the lock.
static func disengage(_ anchor: UIViewController) {
for parent in stampedParents.allObjects {
setForcedChild(nil, on: parent)
}
stampedParents.removeAllObjects()
anchor.setNeedsUpdateOfPrefersPointerLocked()
}
}
#endif
@@ -0,0 +1,36 @@
// Synthetic 4:4:4 HEVC keyframes used only by `Stage444Probe` to probe decode capability.
//
// Each is the first IDR access unit (VPS + SPS + PPS + IDR slice, Annex-B) of a 256×256 HEVC
// Range-Extensions clip `chroma_format_idc = 3` generated offline with libx265:
// ffmpeg -f lavfi -i color=c=gray:s=256x256:r=30:d=0.1 -frames:v 3 \
// -pix_fmt yuv444p[10le] -c:v libx265 \
// -x265-params keyint=1:min-keyint=1:no-info=1:repeat-headers=1:aud=0 -f hevc out.hevc
// 256×256 clears the hardware decoder's minimum-dimension floor (a 16×16 clip is rejected for every
// chroma format). Validated to hardware-decode to `444v`/`x444` on Apple Silicon (M3).
enum Probe444Blobs {
/// 256×256 HEVC Range-Extensions 4:4:4 keyframe (Annex-B): 134 bytes.
static let au444_8bit: [UInt8] = [
0x00, 0x00, 0x00, 0x01, 0x40, 0x01, 0x0c, 0x01, 0xff, 0xff, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00,
0x9e, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c, 0xba, 0x02, 0x40, 0x00, 0x00, 0x00, 0x01, 0x42,
0x01, 0x01, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00, 0x9e, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c,
0x90, 0x01, 0x01, 0x00, 0x80, 0xb2, 0xdd, 0x49, 0x26, 0x57, 0x80, 0xb4, 0x04, 0x00, 0x00, 0x03,
0x00, 0x04, 0x00, 0x00, 0x03, 0x00, 0x78, 0x20, 0x00, 0x00, 0x00, 0x01, 0x44, 0x01, 0xc1, 0x72,
0x86, 0x0c, 0x06, 0x24, 0x00, 0x00, 0x00, 0x01, 0x28, 0x01, 0xaf, 0x72, 0x15, 0xe8, 0x34, 0xeb,
0xae, 0xfb, 0xfe, 0x75, 0x57, 0xca, 0xc1, 0x71, 0x43, 0x16, 0xf5, 0xc2, 0x40, 0xbd, 0x80, 0xa6,
0x65, 0x35, 0x20, 0x28, 0x81, 0xa2, 0x5e, 0xc0, 0x93, 0x04, 0x10, 0x9b, 0x00, 0x34, 0xe0, 0x87,
0x00, 0x00, 0x03, 0x00, 0x5b, 0x40,
]
/// 256×256 HEVC Range-Extensions 4:4:4 10-bit keyframe (Annex-B): 133 bytes.
static let au444_10bit: [UInt8] = [
0x00, 0x00, 0x00, 0x01, 0x40, 0x01, 0x0c, 0x01, 0xff, 0xff, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00,
0x9c, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c, 0xba, 0x02, 0x40, 0x00, 0x00, 0x00, 0x01, 0x42,
0x01, 0x01, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00, 0x9c, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c,
0x90, 0x01, 0x01, 0x00, 0x80, 0x9b, 0x2d, 0xd4, 0x92, 0x65, 0x78, 0x0b, 0x40, 0x40, 0x00, 0x00,
0x03, 0x00, 0x40, 0x00, 0x00, 0x07, 0x82, 0x00, 0x00, 0x00, 0x01, 0x44, 0x01, 0xc1, 0x72, 0x86,
0x0c, 0x06, 0x24, 0x00, 0x00, 0x00, 0x01, 0x28, 0x01, 0xaf, 0x72, 0x15, 0xe8, 0x34, 0xeb, 0xae,
0xfb, 0xfe, 0x75, 0x57, 0xca, 0xc1, 0x71, 0x43, 0x16, 0xf5, 0xc2, 0x40, 0xbd, 0x80, 0xa6, 0x65,
0x35, 0x20, 0x28, 0x81, 0xa2, 0x5e, 0xc0, 0x93, 0x04, 0x10, 0x9b, 0x00, 0x34, 0xe0, 0x87, 0x00,
0x00, 0x03, 0x00, 0x5b, 0x40,
]
}
@@ -182,6 +182,11 @@ public final class PunktfunkConnection {
case dualSense = 2 case dualSense = 2
case xboxOne = 3 case xboxOne = 3
case dualShock4 = 4 case dualShock4 = 4
// Valve Steam Controller / Steam Deck (Linux UHID hid-steam hosts). Parity only on Apple
// GameController never surfaces a 0x28DE HID device, so the client can't capture one; these
// exist so the resolved type round-trips and name parsing matches the host.
case steamController = 5
case steamDeck = 6
/// Loose name parsing for env/dev hooks, mirroring the host's /// Loose name parsing for env/dev hooks, mirroring the host's
/// `GamepadPref::from_name`. /// `GamepadPref::from_name`.
@@ -192,6 +197,8 @@ public final class PunktfunkConnection {
case "dualsense", "ds", "ds5", "ps5": self = .dualSense case "dualsense", "ds", "ds5", "ps5": self = .dualSense
case "xboxone", "xbox-one", "xboxseries", "series": self = .xboxOne case "xboxone", "xbox-one", "xboxseries", "series": self = .xboxOne
case "dualshock4", "dualshock", "ds4", "ps4": self = .dualShock4 case "dualshock4", "dualshock", "ds4", "ps4": self = .dualShock4
case "steamdeck", "steam-deck", "deck": self = .steamDeck
case "steamcontroller", "steam-controller", "steamcon": self = .steamController
default: return nil default: return nil
} }
} }
@@ -231,6 +238,13 @@ public final class PunktfunkConnection {
public private(set) var colorFullRange: Bool = false public private(set) var colorFullRange: Bool = false
/// Encoded bit depth (8 or 10). /// Encoded bit depth (8 or 10).
public private(set) var bitDepth: UInt8 = 8 public private(set) var bitDepth: UInt8 = 8
/// The chroma subsampling the host resolved for this session, as the HEVC `chroma_format_idc`:
/// `1` = 4:2:0 (every pre-4:4:4 host, and the back-compat default) or `3` = full-chroma 4:4:4
/// (only when this client advertised `videoCap444` *and* the host could open a real 4:4:4
/// encoder). Drive the decoder's requested pixel format from this. See `isChroma444`.
public private(set) var chromaFormat: UInt8 = 1
/// Convenience: the resolved stream is full-chroma 4:4:4 (`chroma_format_idc == 3`).
public var isChroma444: Bool { chromaFormat == 3 }
/// True when the negotiated stream is HDR (PQ or HLG transfer) drive an HDR present path and /// True when the negotiated stream is HDR (PQ or HLG transfer) drive an HDR present path and
/// drain `nextHdrMeta`. /// drain `nextHdrMeta`.
public var isHDR: Bool { colorTransfer == 16 || colorTransfer == 18 } public var isHDR: Bool { colorTransfer == 16 || colorTransfer == 18 }
@@ -327,6 +341,9 @@ public final class PunktfunkConnection {
colorMatrix = mtx colorMatrix = mtx
colorFullRange = fullRange != 0 colorFullRange = fullRange != 0
bitDepth = depth bitDepth = depth
var cf: UInt8 = 1
_ = punktfunk_connection_chroma_format(handle, &cf)
chromaFormat = cf
var ac: UInt8 = 2 var ac: UInt8 = 2
_ = punktfunk_connection_audio_channels(handle, &ac) _ = punktfunk_connection_audio_channels(handle, &ac)
resolvedAudioChannels = ac resolvedAudioChannels = ac
@@ -598,6 +615,10 @@ public final class PunktfunkConnection {
public static let videoCap10Bit: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_10BIT) public static let videoCap10Bit: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_10BIT)
/// Video-capability bit: the client can present BT.2020 PQ HDR10 (implies 10-bit). /// Video-capability bit: the client can present BT.2020 PQ HDR10 (implies 10-bit).
public static let videoCapHDR: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_HDR) public static let videoCapHDR: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_HDR)
/// Video-capability bit: the client can decode a full-chroma 4:4:4 HEVC stream (Range
/// Extensions). Advertise only when the device can *hardware*-decode it (`Stage444Probe`);
/// the host then emits 4:4:4 only if it too opted in. `chromaFormat` reflects the real value.
public static let videoCap444: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_444)
/// Static HDR mastering metadata (SMPTE ST.2086 + content light level) the host sent for an HDR /// Static HDR mastering metadata (SMPTE ST.2086 + content light level) the host sent for an HDR
/// session. Mirrors the wire/ABI `PunktfunkHdrMeta`; primaries are in ST.2086 **G, B, R** order, /// session. Mirrors the wire/ABI `PunktfunkHdrMeta`; primaries are in ST.2086 **G, B, R** order,
@@ -0,0 +1,93 @@
Copyright 2024 The Geist Project Authors (https://github.com/vercel/geist-font)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
https://openfontlicense.org
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
@@ -177,6 +177,16 @@ public final class SessionAudio {
private var playbackEngine: AVAudioEngine? private var playbackEngine: AVAudioEngine?
private var captureEngine: AVAudioEngine? private var captureEngine: AVAudioEngine?
private var drainStarted = false private var drainStarted = false
#if !os(macOS)
/// AVAudioSession `setCategory`/`setActive` are synchronous and block on the audio server, so
/// they must not run on the main thread (UI stall AVFoundation warns about it). PROCESS-WIDE
/// (static) so every SessionAudio shares one serial queue: the AVAudioSession is a process
/// singleton, and across a reconnect the old session's deactivate must be ordered before the
/// new session's activate (a per-instance queue would let them race and leave the new session's
/// audio deactivated). stop() enqueues its deactivate promptly so it lands before the next
/// session's activate.
private static let sessionQueue = DispatchQueue(label: "io.unom.punktfunk.audio.session")
#endif
public init(connection: PunktfunkConnection) { public init(connection: PunktfunkConnection) {
self.connection = connection self.connection = connection
@@ -189,37 +199,60 @@ public final class SessionAudio {
flag.stop() flag.stop()
} }
/// Start playback (and, if enabled+authorized, the mic uplink). Empty UIDs = system /// Start playback (and, if enabled+authorized, the mic uplink). Empty UIDs = system default
/// default device; on iOS the UIDs are ignored entirely (routes are /// device; on iOS the UIDs are ignored entirely (routes are AVAudioSession-managed). On macOS
/// AVAudioSession-managed). Main thread (engine setup); returns after the engines /// the engines start synchronously on the caller's (main) thread. On iOS/tvOS start() is
/// start the mic may start slightly later if the permission prompt is pending. /// ASYNCHRONOUS: it activates the AVAudioSession off the main thread, then starts the engines on
/// a later main-queue hop (gated by `!flag.isStopped`) so playback is live shortly after, not
/// on return. The mic may start later still if the permission prompt is pending.
public func start(speakerUID: String, micUID: String, micEnabled: Bool) { public func start(speakerUID: String, micUID: String, micEnabled: Bool) {
#if os(iOS) #if os(macOS)
// Route + policy live in the session, not per-engine: stereo playback, mic // No AVAudioSession on macOS start the engines directly (caller's thread, as before).
// capture when enabled, Bluetooth allowed. Failure is non-fatal (defaults). startEngines(speakerUID: speakerUID, micUID: micUID, micEnabled: micEnabled)
#else
// Configure + activate the session OFF the main thread (it blocks on the audio server),
// then start the engines back on the main thread once it's active engine routing/format
// depend on the active session. A stop() racing in between is caught by the flag guard.
Self.sessionQueue.async { [weak self] in
guard let self else { return }
self.activateAudioSession(micEnabled: micEnabled)
DispatchQueue.main.async { [weak self] in
guard let self, !self.flag.isStopped else { return }
self.startEngines(speakerUID: speakerUID, micUID: micUID, micEnabled: micEnabled)
}
}
#endif
}
#if !os(macOS)
/// Route + policy live in the session, not per-engine: stereo playback, mic capture when
/// enabled, Bluetooth allowed. Failure is non-fatal (defaults). Runs on `sessionQueue`.
private func activateAudioSession(micEnabled: Bool) {
let session = AVAudioSession.sharedInstance() let session = AVAudioSession.sharedInstance()
do { do {
#if os(iOS)
if micEnabled { if micEnabled {
// .defaultToSpeaker: .playAndRecord otherwise routes to the iPhone // .defaultToSpeaker: .playAndRecord otherwise routes to the iPhone EARPIECE; only
// EARPIECE; only affects the built-in route (headphones/BT still win). // affects the built-in route (headphones/BT still win).
try session.setCategory( try session.setCategory(
.playAndRecord, mode: .default, .playAndRecord, mode: .default,
options: [.allowBluetoothA2DP, .defaultToSpeaker]) options: [.allowBluetoothA2DP, .defaultToSpeaker])
} else { } else {
try session.setCategory(.playback, mode: .default) try session.setCategory(.playback, mode: .default)
} }
#else // tvOS no app-accessible mic
try session.setCategory(.playback, mode: .default)
#endif
try session.setActive(true) try session.setActive(true)
} catch { } catch {
log.warning("AVAudioSession setup failed: \(error.localizedDescription)") log.warning("AVAudioSession setup failed: \(error.localizedDescription)")
} }
#elseif os(tvOS)
do {
try AVAudioSession.sharedInstance().setCategory(.playback, mode: .default)
try AVAudioSession.sharedInstance().setActive(true)
} catch {
log.warning("AVAudioSession setup failed: \(error.localizedDescription)")
} }
#endif #endif
/// Build + start the playback engine (and the mic uplink when enabled + authorized). Main
/// thread (engine setup); on iOS/tvOS the session is already active by the time this runs.
private func startEngines(speakerUID: String, micUID: String, micEnabled: Bool) {
startPlayback(speakerUID: speakerUID) startPlayback(speakerUID: speakerUID)
#if os(tvOS) #if os(tvOS)
// No app-accessible microphone input on tvOS playback only. // No app-accessible microphone input on tvOS playback only.
@@ -258,19 +291,24 @@ public final class SessionAudio {
capture.stop() capture.stop()
} }
playback?.stop() playback?.stop()
if wasDraining {
_ = drainDone.wait(timeout: .now() + .milliseconds(400))
}
#if !os(macOS) #if !os(macOS)
// Release the session so audio we interrupted (Music, podcasts) gets its // Release the session so audio we interrupted (Music, podcasts) gets its resume cue. Like
// resume cue. // activation, setActive is synchronous/blocking run it on the shared serial session queue
// (off the main thread). Enqueued HERE engines already stopped, and BEFORE the drain wait
// below so across a reconnect it lands ahead of the next session's activate on the shared
// queue (otherwise a deferred deactivate could deactivate the new session). Fire-and-forget.
Self.sessionQueue.async {
do { do {
try AVAudioSession.sharedInstance().setActive( try AVAudioSession.sharedInstance().setActive(
false, options: .notifyOthersOnDeactivation) false, options: .notifyOthersOnDeactivation)
} catch { } catch {
log.warning("AVAudioSession deactivation failed: \(error.localizedDescription)") log.warning("AVAudioSession deactivation failed: \(error.localizedDescription)")
} }
}
#endif #endif
if wasDraining {
_ = drainDone.wait(timeout: .now() + .milliseconds(400))
}
} }
// MARK: - Playback (host speaker) // MARK: - Playback (host speaker)
@@ -1,21 +1,21 @@
// Stage-2 presenter orchestrator: a pump thread pulls AUs VideoDecoder; the decoder's async // Stage-2 presenter orchestrator: a pump thread pulls AUs VideoDecoder; the decoder's async output
// output drops the newest decoded frame into a 1-slot ring; the hosting view's display link // drops the newest decoded frame into a 1-slot ring; the hosting view's display link calls `renderTick`
// calls `renderTick` once per vsync to draw + present the newest ready frame and stamp // once per vsync to draw + present the newest ready frame and stamp capturepresent. Mirrors
// capturepresent. Mirrors StreamPump's lifecycle (one per start; cancel is permanent). // StreamPump's lifecycle (one per start; cancel is permanent).
// //
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; `renderTick` // Threading: the pump runs on its own thread; the decoder callback on a VT thread; `renderTick` +
// + `setDrawableSize` + `start`/`stop` on the MAIN thread (the view's CADisplayLink fires there). // `start`/`stop` on the MAIN thread (the view's CADisplayLink fires there). Only the ring (lock-guarded)
// Only the ring + decoder cross threads and both are internally locked. // and the decoder/presenter (internally locked / main-hopped) cross threads.
#if canImport(Metal) && canImport(QuartzCore) #if canImport(Metal) && canImport(QuartzCore)
import AVFoundation import AVFoundation
import Foundation import Foundation
import QuartzCore import QuartzCore
/// Weak-target wrapper for CADisplayLink. The link retains its target, so targeting a view /// Weak-target wrapper for CADisplayLink. The link retains its target, so targeting a view directly
/// directly makes a `view link view` cycle that only `invalidate()` breaks if a teardown /// makes a `view link view` cycle that only `invalidate()` breaks if a teardown is ever missed
/// is ever missed the view leaks and keeps ticking. This proxy holds the handler weakly, so the /// the view leaks and keeps ticking. This proxy holds the handler weakly, so the view can deallocate
/// view can deallocate and its `deinit` invalidate the link. /// and its `deinit` invalidate the link.
public final class DisplayLinkProxy: NSObject { public final class DisplayLinkProxy: NSObject {
private let onTick: (CADisplayLink) -> Void private let onTick: (CADisplayLink) -> Void
public init(_ onTick: @escaping (CADisplayLink) -> Void) { self.onTick = onTick } public init(_ onTick: @escaping (CADisplayLink) -> Void) { self.onTick = onTick }
@@ -44,10 +44,10 @@ private final class PumpToken: @unchecked Sendable {
func cancel() { lock.lock(); live = false; lock.unlock() } func cancel() { lock.lock(); live = false; lock.unlock() }
} }
/// Throttled host keyframe requests for decode recovery. The decoder's async error callback /// Throttled host keyframe requests for decode recovery. The decoder's async error callback (a VT
/// (a VT thread) and the pump thread (a submit failure) both signal a wedge; this coalesces /// thread) and the pump thread (a submit failure) both signal a wedge; this coalesces them so the
/// them so the control stream isn't flooded while the decode stays stalled for several frames /// control stream isn't flooded while the decode stays stalled for several frames until the requested
/// until the requested IDR lands. Bound to the live connection in `start`, unbound in `stop`. /// IDR lands. Bound to the live connection in `start`, unbound in `stop`.
private final class KeyframeRecovery: @unchecked Sendable { private final class KeyframeRecovery: @unchecked Sendable {
private let lock = NSLock() private let lock = NSLock()
private var connection: PunktfunkConnection? private var connection: PunktfunkConnection?
@@ -60,7 +60,7 @@ private final class KeyframeRecovery: @unchecked Sendable {
func request() { func request() {
lock.lock() lock.lock()
let now = DispatchTime.now().uptimeNanoseconds let now = DispatchTime.now().uptimeNanoseconds
let due = lastNs == 0 || now &- lastNs > 250_000_000 // 250 ms since the last request let due = lastNs == 0 || now &- lastNs > 100_000_000 // 100 ms since the last request
if due { lastNs = now } if due { lastNs = now }
let conn = due ? connection : nil let conn = due ? connection : nil
lock.unlock() lock.unlock()
@@ -76,30 +76,36 @@ public final class Stage2Pipeline {
private let recovery = KeyframeRecovery() private let recovery = KeyframeRecovery()
private var token = PumpToken() private var token = PumpToken()
private var offsetNs: Int64 = 0 private var offsetNs: Int64 = 0
/// Signalled when the pump thread exits, so `stop()` can join it (bounded) before `decoder.reset()`
/// otherwise a pump iteration already past its `token.isLive` check can rebuild a decode session
/// right after the reset (a brief orphan session). `pumpJoinable` is armed by `start`, consumed by
/// the first `stop` (so the idempotent second `stop`/deinit doesn't block on an already-drained
/// semaphore). start/stop are sequential lifecycle calls, so the plain flag is safe.
private let pumpStopped = DispatchSemaphore(value: 0)
private var pumpJoinable = false
/// The Metal layer the hosting view installs + sizes. nil-init fails when Metal is /// The Metal layer the hosting view installs + sizes.
/// unavailable so the caller can fall back to stage-1.
public var layer: CAMetalLayer { presenter.layer } public var layer: CAMetalLayer { presenter.layer }
/// `presentMeter` records capturepresent (the glass-to-glass term). Returns nil if Metal /// `presentMeter` records capturepresent (the glass-to-glass term). Returns nil if Metal can't be
/// can't be set up (headless / no GPU) caller falls back to the stage-1 presenter. /// set up (headless / no GPU) caller falls back to the stage-1 presenter.
public init?(presentMeter: LatencyMeter) { public init?(presentMeter: LatencyMeter) {
guard let presenter = MetalVideoPresenter() else { return nil } guard let presenter = MetalVideoPresenter.make() else { return nil }
self.presenter = presenter self.presenter = presenter
self.presentMeter = presentMeter self.presentMeter = presentMeter
let ring = ring let ring = ring
let recovery = recovery let recovery = recovery
self.decoder = VideoDecoder( self.decoder = VideoDecoder(
onDecoded: { ring.submit($0) }, onDecoded: { ring.submit($0) },
// Async decode failure (a bad P-frame referencing a lost/corrupt IDR): the pump // Async decode failure (a bad P-frame referencing a lost/corrupt IDR): the pump resets to
// resets to re-gate on the next IDR, and we ask the host to send one now (infinite // re-gate on the next IDR, and we ask the host to send one now (infinite GOP it wouldn't
// GOP it wouldn't otherwise come soon). Throttled in KeyframeRecovery. // otherwise come soon). Throttled in KeyframeRecovery.
onDecodeError: { _ in recovery.request() }) onDecodeError: { _ in recovery.request() })
} }
/// Start pulling AUs into the decoder. `onFrame` fires per AU at receipt (captureclient /// Start pulling AUs into the decoder. MAIN THREAD. `onFrame` fires per AU at receipt (captureclient
/// meter, exactly as stage-1); `onSessionEnd` on close. `clockOffsetNs` (host minus client) /// meter, exactly as stage-1); `onSessionEnd` on close. `clockOffsetNs` (host minus client) makes the
/// makes the present stamp cross-machine valid. /// present stamp cross-machine valid.
public func start( public func start(
connection: PunktfunkConnection, connection: PunktfunkConnection,
onFrame: (@Sendable (AccessUnit) -> Void)?, onFrame: (@Sendable (AccessUnit) -> Void)?,
@@ -108,43 +114,70 @@ public final class Stage2Pipeline {
offsetNs = connection.clockOffsetNs offsetNs = connection.clockOffsetNs
recovery.bind(connection) // arm host-keyframe recovery for this session recovery.bind(connection) // arm host-keyframe recovery for this session
token = PumpToken() // fresh token per start cancel is permanent (like StreamPump) token = PumpToken() // fresh token per start cancel is permanent (like StreamPump)
// Configure the decoder's chroma + the layer's initial colorimetry before the first frame. The
// chroma subsampling drives only the decode pixel format (orthogonal to HDR/depth); the HDR
// config is the Welcome's latched value, which a mid-session flip then overrides per-frame.
decoder.setChroma444(connection.isChroma444)
presenter.configure(hdr: connection.isHDR)
let token = token let token = token
let decoder = decoder let decoder = decoder
let recovery = recovery let recovery = recovery
let presenter = presenter
let pumpStopped = pumpStopped
let thread = Thread { let thread = Thread {
defer { pumpStopped.signal() } // let stop() join the pump (bounded) before decoder.reset()
var format: CMVideoFormatDescription? var format: CMVideoFormatDescription?
var lastFramesDropped = connection.framesDropped() var lastFramesDropped = connection.framesDropped()
// Persistent recovery WANT, not a one-shot edge (see StreamPump for the full rationale):
// keep asking until an IDR lands so a request swallowed by the throttle is re-sent.
var awaitingIDR = false
// 4:4:4 backstop: a run of decode/create failures in a 4:4:4 session means this device can't
// decode 4:4:4 at the negotiated resolution (the HW probe clears the common case but not a
// resolution-ceiling miss). End cleanly instead of looping on a black screen.
var decodeFailRun = 0
while token.isLive { while token.isLive {
do { do {
// Loss recovery (the primary recovery path). The reassembler drops unrecoverable // Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
// AUs (framesDropped) and the decoder then conceals the reference-missing delta // decoder conceals the reference-missing deltas often WITHOUT an error callback
// frames that follow often rendering them WITHOUT an error callback so the // so key off the drop count climbing, then keep asking (awaitingIDR) until a fresh
// onDecodeError trigger rarely fires after a real network blip. Ask the host for // IDR re-anchors decode.
// a fresh IDR whenever the drop count climbs (throttled in KeyframeRecovery).
// Polled every iteration so a total-loss drought recovers the moment packets
// resume and the reassembler counts the gap.
let dropped = connection.framesDropped() let dropped = connection.framesDropped()
if dropped > lastFramesDropped { if dropped > lastFramesDropped {
lastFramesDropped = dropped lastFramesDropped = dropped
recovery.request() awaitingIDR = true
} }
// Drain any HDR mastering-metadata update (0xCE) and hand it to the decoder, which if awaitingIDR { recovery.request() }
// attaches it to subsequent HDR frames. Non-blocking; only HDR sessions emit these. // Drain HDR mastering metadata (0xCE) and hand it to the PRESENTER ( CAEDRMetadata).
if connection.isHDR, let meta = try? connection.nextHdrMeta(timeoutMs: 0) { // Polled UNCONDITIONALLY (not gated on connection.isHDR, the fixed Welcome flag): the
decoder.setHdrMeta(meta) // host sends 0xCE only for HDR, INCLUDING a mid-session SDRHDR transition (a game
// entering HDR the host re-inits its encoder) the Welcome flag would never reflect.
// Non-blocking; nil for an SDR stream.
if let meta = try? connection.nextHdrMeta(timeoutMs: 0) {
presenter.setHdrMeta(meta)
} }
guard let au = try connection.nextAU(timeoutMs: 100) else { continue } guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au) onFrame?(au)
if let f = AnnexB.formatDescription(fromIDR: au.data) { if let f = AnnexB.formatDescription(fromIDR: au.data) {
format = f // refreshed on every IDR (mode changes included) format = f // refreshed on every IDR (mode changes included)
awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
} }
guard let f = format, token.isLive else { continue } guard let f = format, token.isLive else { continue }
if !decoder.decode(au: au, format: f) { if decoder.decode(au: au, format: f) {
// Submit/decoder error: drop the session and re-gate on the next IDR's decodeFailRun = 0
// in-band parameter sets (a delta frame can't recover) stage-1's policy } else {
// and ask the host for that IDR now (infinite GOP; throttled). // Submit/decoder error: drop the session and re-gate on the next IDR's in-band
// parameter sets (a delta frame can't recover) and keep asking for that IDR.
decoder.reset() decoder.reset()
recovery.request() awaitingIDR = true
decodeFailRun += 1
// ~3 s of solid failure in a 4:4:4 session (and only there a 4:2:0 loss
// recovers within a GOP) 4:4:4 isn't decodable here; end the session.
if connection.isChroma444, decodeFailRun >= 180 {
if token.isLive { onSessionEnd?() }
break
}
} }
} catch { } catch {
if token.isLive { onSessionEnd?() } if token.isLive { onSessionEnd?() }
@@ -154,27 +187,30 @@ public final class Stage2Pipeline {
} }
thread.name = "punktfunk-stage2-pump" thread.name = "punktfunk-stage2-pump"
thread.qualityOfService = .userInteractive thread.qualityOfService = .userInteractive
pumpJoinable = true
thread.start() thread.start()
} }
/// MAIN thread, once per vsync. Present the newest ready frame (if any) and stamp /// MAIN thread, once per vsync. Present the newest ready frame (if any) and stamp capturepresent at
/// capturepresent at `targetPresentNs` the display link's target present instant, already /// `targetPresentNs` the display link's target present instant, already converted to
/// converted to `CLOCK_REALTIME` (see `realtimeNs(forDisplayLinkTimestamp:)`). /// `CLOCK_REALTIME` (see `realtimeNs(forDisplayLinkTimestamp:)`).
public func renderTick(targetPresentNs: Int64) { public func renderTick(targetPresentNs: Int64) {
guard let frame = ring.take() else { return } guard let frame = ring.take() else { return }
guard presenter.render(frame.pixelBuffer, isHDR: frame.isHDR) else { return } guard presenter.render(frame.pixelBuffer, isHDR: frame.isHDR) else { return }
presentMeter.record(ptsNs: frame.ptsNs, atNs: targetPresentNs, offsetNs: offsetNs) presentMeter.record(ptsNs: frame.ptsNs, atNs: targetPresentNs, offsetNs: offsetNs)
} }
/// MAIN thread. Keep the drawable matched to the negotiated mode (host can Reconfigure). /// Stop the pump ( one poll timeout) and drop the decode session. MAIN THREAD; idempotent. Does not
public func setDrawableSize(_ size: CGSize) { /// close the connection. A restart needs a fresh Stage2Pipeline (cancel is permanent).
presenter.setDrawableSize(size)
}
/// Stop the pump ( one poll timeout) and drop the decode session. Does not close the
/// connection. A restart needs a fresh Stage2Pipeline (cancel is permanent).
public func stop() { public func stop() {
token.cancel() token.cancel()
// Join the pump (bounded: one nextAU poll + an in-flight decode) before resetting the decoder,
// so the pump can't rebuild a session right after the reset. Only the first stop joins; a
// repeat/deinit stop skips the already-drained semaphore.
if pumpJoinable {
pumpJoinable = false
_ = pumpStopped.wait(timeout: .now() + 0.5)
}
decoder.reset() decoder.reset()
recovery.bind(nil) // stop requesting keyframes once the session is torn down recovery.bind(nil) // stop requesting keyframes once the session is torn down
} }
@@ -182,8 +218,8 @@ public final class Stage2Pipeline {
deinit { token.cancel() } deinit { token.cancel() }
/// Convert a `CADisplayLink.targetTimestamp` (CACurrentMediaTime basis) to a `CLOCK_REALTIME` /// Convert a `CADisplayLink.targetTimestamp` (CACurrentMediaTime basis) to a `CLOCK_REALTIME`
/// nanosecond instant the present clock the AU pts + skew offset live in. Projects to the /// nanosecond instant the present clock the AU pts + skew offset live in. Projects to the target
/// target present time (when the frame is actually on glass), not the moment we drew. /// present time (when the frame is actually on glass), not the moment we drew.
public static func realtimeNs(forDisplayLinkTimestamp t: CFTimeInterval) -> Int64 { public static func realtimeNs(forDisplayLinkTimestamp t: CFTimeInterval) -> Int64 {
let caNow = CACurrentMediaTime() let caNow = CACurrentMediaTime()
var ts = timespec() var ts = timespec()
@@ -0,0 +1,83 @@
// Runtime 4:4:4 HEVC decode-capability probe.
//
// We advertise `VIDEO_CAP_444` (so the host upgrades to a full-chroma 4:4:4 stream) ONLY when this
// device can decode 4:4:4 HEVC *in hardware* software 4:4:4 decode works but is far too slow for a
// real-time stream at the negotiated resolution, so a software-only device must keep 4:2:0.
//
// `VTIsHardwareDecodeSupported(HEVC)` and the HEVC-decoder-capabilities dictionary report HEVC HW
// decode but expose nothing about `chroma_format_idc`, so the only reliable signal is to actually
// create a *hardware-required* `VTDecompressionSession` for a tiny synthetic 4:4:4 keyframe and
// confirm it both creates and decodes to the expected biplanar 4:4:4 pixel format. Validated on an
// Apple M3 (HW 4:4:4 8- and 10-bit decode to `444v`/`x444`); a software-only decoder fails the
// hardware-required create and we fall back to 4:2:0.
//
// The probe blobs are 256×256 (above the hardware decoder's minimum-dimension floor a 16×16 clip
// is rejected for ALL chroma formats, including 4:2:0) HEVC Range-Extensions keyframes generated
// offline with libx265; see scripts notes. Results are cached (device-static) in lazy statics.
import CoreMedia
import CoreVideo
import Foundation
import VideoToolbox
public enum Stage444Probe {
/// True iff this device hardware-decodes 8-bit 4:4:4 HEVC (the host's current 4:4:4 path
/// BT.709 limited `yuv444p`). Cached after first evaluation.
public static let hwDecode444_8bit: Bool = probeHardware444(
au: Probe444Blobs.au444_8bit,
want: kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange,
fullRangeSibling: kCVPixelFormatType_444YpCbCr8BiPlanarFullRange)
/// True iff this device hardware-decodes 10-bit 4:4:4 HEVC (the 4:4:4 HDR/10-bit intersection).
/// Cached after first evaluation.
public static let hwDecode444_10bit: Bool = probeHardware444(
au: Probe444Blobs.au444_10bit,
want: kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange,
fullRangeSibling: kCVPixelFormatType_444YpCbCr10BiPlanarFullRange)
/// Create a hardware-REQUIRED `VTDecompressionSession` for the synthetic 4:4:4 keyframe and
/// decode it, returning true only when the decoder produces the expected (video- or full-range)
/// biplanar 4:4:4 pixel format. Any failure (no hardware path, wrong output format, decode error)
/// false we keep 4:2:0.
private static func probeHardware444(
au auBytes: [UInt8], want: OSType, fullRangeSibling: OSType
) -> Bool {
let data = Data(auBytes)
guard let format = AnnexB.formatDescription(fromIDR: data) else { return false }
// Require a hardware decoder a software false-positive would make us advertise 4:4:4 and
// then decode every real frame on the CPU, blowing the latency budget.
let spec: [CFString: Any] = [
kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder: true,
]
let attrs: [CFString: Any] = [
kCVPixelBufferPixelFormatTypeKey: want,
kCVPixelBufferMetalCompatibilityKey: true,
]
var session: VTDecompressionSession?
let created = VTDecompressionSessionCreate(
allocator: kCFAllocatorDefault, formatDescription: format,
decoderSpecification: spec as CFDictionary, imageBufferAttributes: attrs as CFDictionary,
outputCallback: nil, decompressionSessionOut: &session)
guard created == noErr, let session else { return false }
defer { VTDecompressionSessionInvalidate(session) }
let au = AccessUnit(data: data, ptsNs: 0, frameIndex: 0, flags: 0)
guard let sample = AnnexB.sampleBuffer(au: au, format: format) else { return false }
var produced: OSType = 0
let done = DispatchSemaphore(value: 0)
let status = VTDecompressionSessionDecodeFrame(
session, sampleBuffer: sample,
flags: [._EnableAsynchronousDecompression], infoFlagsOut: nil
) { status, _, imageBuffer, _, _ in
if status == noErr, let imageBuffer {
produced = CVPixelBufferGetPixelFormatType(imageBuffer)
}
done.signal()
}
guard status == noErr else { return false }
VTDecompressionSessionWaitForAsynchronousFrames(session)
_ = done.wait(timeout: .now() + 1.0)
return produced == want || produced == fullRangeSibling
}
}
@@ -6,6 +6,9 @@
import AVFoundation import AVFoundation
import Foundation import Foundation
import os
private let pumpLog = Logger(subsystem: "io.unom.punktfunk", category: "video")
/// Cancellation handle owned by exactly one pump thread a restart hands the old pump /// Cancellation handle owned by exactly one pump thread a restart hands the old pump
/// its own token, so it can never be revived by a newer start(). /// its own token, so it can never be revived by a newer start().
@@ -47,44 +50,74 @@ final class StreamPump {
var format: CMVideoFormatDescription? var format: CMVideoFormatDescription?
var lastKeyframeRequest = Date.distantPast var lastKeyframeRequest = Date.distantPast
var lastFramesDropped = connection.framesDropped() var lastFramesDropped = connection.framesDropped()
// Coalesced host keyframe request: the decode stays wedged for several frames until // Recovery is a persistent WANT, not a one-shot edge: set it on detected loss (or a
// the IDR lands, so requesting on every frame would flood the control stream. // decoder reset), retry the throttled request EVERY iteration, and clear it only when a
// fresh IDR actually re-anchors decode. The old code advanced `lastFramesDropped` on the
// same edge it fired the throttled request so a request swallowed by the throttle (a
// second drop within the window, e.g. the lost recovery IDR itself being pruned) was
// never re-sent: the counter went flat, the climb never re-fired, and the picture stayed
// frozen for good while audio kept playing. The iPhone's lossy Wi-Fi hits this where the
// Mac's Ethernet never does.
var awaitingIDR = false
var awaitingSince = Date.distantPast // when the current recovery began (for the resume log)
var wasFailed = false
// Coalesced host keyframe request. 100 ms throttle (matches the working Android path):
// fast enough that a lost recovery IDR is re-requested promptly, bounded so a sustained
// freeze can't flood the control stream.
func requestKeyframeThrottled() { func requestKeyframeThrottled() {
let now = Date() let now = Date()
if now.timeIntervalSince(lastKeyframeRequest) > 0.25 { if now.timeIntervalSince(lastKeyframeRequest) > 0.1 {
connection.requestKeyframe() connection.requestKeyframe()
lastKeyframeRequest = now lastKeyframeRequest = now
} }
} }
while token.isLive { while token.isLive {
do { do {
// Loss recovery (the primary recovery path). Under the host's infinite GOP the // Loss recovery (the primary path). Under the host's infinite GOP the only
// only recovery keyframe is one we request. The reassembler drops unrecoverable // recovery keyframe is one we request. The reassembler drops unrecoverable AUs
// AUs (framesDropped); the decoder then *conceals* the reference-missing delta // (framesDropped); the decoder then *conceals* the reference-missing deltas a
// frames that follow a frozen / garbage picture, WITHOUT flipping the layer to // frozen / garbage picture that never flips the layer to .failed so key off the
// .failed so the .failed check below rarely fires after a real network blip. // drop count climbing, then keep asking (awaitingIDR) until an IDR lands. Polled
// Ask the host for a fresh IDR whenever the drop count climbs. Polled every // every iteration so a total-loss drought still recovers when packets resume.
// iteration (not just per AU) so a total-loss drought still recovers the moment
// packets resume and the reassembler counts the gap.
let dropped = connection.framesDropped() let dropped = connection.framesDropped()
if dropped > lastFramesDropped { if dropped > lastFramesDropped {
lastFramesDropped = dropped // Log only on the falsetrue transition (once per recovery cycle), not per
requestKeyframeThrottled() // dropped AU, so heavy loss doesn't spam the log.
if !awaitingIDR {
awaitingSince = Date()
pumpLog.notice(
"video: unrecoverable drop (framesDropped=\(dropped, privacy: .public)) — requesting recovery IDR")
} }
lastFramesDropped = dropped
awaitingIDR = true
}
if awaitingIDR { requestKeyframeThrottled() }
guard let au = try connection.nextAU(timeoutMs: 100) else { continue } guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au) onFrame?(au)
if let f = AnnexB.formatDescription(fromIDR: au.data) { let idrFormat = AnnexB.formatDescription(fromIDR: au.data)
if let f = idrFormat {
format = f // refreshed on every IDR (mode changes included) format = f // refreshed on every IDR (mode changes included)
if awaitingIDR {
let ms = Int(Date().timeIntervalSince(awaitingSince) * 1000)
pumpLog.notice("video: recovery IDR received — resumed after \(ms, privacy: .public) ms")
} }
if layer.status == .failed { awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
}
let failed = layer.status == .failed
if failed {
// Decode wedged hard (the cold-first-connect case a lost/corrupt opening // Decode wedged hard (the cold-first-connect case a lost/corrupt opening
// IDR): flush and re-gate on the next in-band parameter sets (resuming with // IDR): flush and, unless THIS AU is the recovering IDR (re-anchored above),
// a delta frame can't recover), AND ask the host for a fresh IDR. Throttled: // re-gate on the next in-band parameter sets and keep asking enqueuing a
// the layer stays .failed across several polls until the IDR lands. // delta into a failed layer can't recover it.
if !wasFailed { pumpLog.warning("video: display layer .failed — flushing + re-anchoring") }
layer.flush() layer.flush()
format = AnnexB.formatDescription(fromIDR: au.data) if idrFormat == nil {
requestKeyframeThrottled() format = nil
awaitingIDR = true
} }
}
wasFailed = failed
guard let f = format, guard let f = format,
let sample = AnnexB.sampleBuffer(au: au, format: f), let sample = AnnexB.sampleBuffer(au: au, format: f),
token.isLive // don't enqueue a stale frame after a restart token.isLive // don't enqueue a stale frame after a restart
@@ -137,8 +137,8 @@ public struct StreamView: NSViewRepresentable {
public final class StreamLayerView: NSView { public final class StreamLayerView: NSView {
private let displayLayer = AVSampleBufferDisplayLayer() private let displayLayer = AVSampleBufferDisplayLayer()
private var pump: StreamPump? private var pump: StreamPump?
/// Stage-2 presenter (opt-in via `punktfunk.presenter`): a CAMetalLayer sublayer driven by a /// Stage-2 presenter (default): a CAMetalLayer sublayer driven by a display link instead of the
/// display link instead of the StreamPump displayLayer path. nil = stage-1 (default). /// StreamPump displayLayer path. nil = stage-1 (Metal-unavailable fallback / DEBUG toggle).
var presentMeter: LatencyMeter? var presentMeter: LatencyMeter?
private var stage2: Stage2Pipeline? private var stage2: Stage2Pipeline?
private var stage2Link: CADisplayLink? private var stage2Link: CADisplayLink?
@@ -245,6 +245,15 @@ public final class StreamLayerView: NSView {
layoutMetalLayer() // keep the stage-2 sublayer aspect-fit to the view layoutMetalLayer() // keep the stage-2 sublayer aspect-fit to the view
} }
public override func setFrameSize(_ newSize: NSSize) {
super.setFrameSize(newSize)
// `layout()` isn't guaranteed on a manual-frame (no-Auto-Layout) live resize, so the
// stage-2 metal sublayer's drawableSize could stay at the old size while the view grows
// the compositor then upscales a too-small drawable and the video turns blocky. Resize the
// drawable here too so it always tracks the window's pixel size (no stale upscale).
layoutMetalLayer()
}
// MARK: - Capture state machine // MARK: - Capture state machine
/// Clicking into the video engages capture; that click is local (engagement), so /// Clicking into the video engages capture; that click is local (engagement), so
@@ -549,10 +558,17 @@ public final class StreamLayerView: NSView {
cursorVisible = false cursorVisible = false
_ = connection.resolvedCompositor // (was: Auto gamescope; kept to document intent) _ = connection.resolvedCompositor // (was: Auto gamescope; kept to document intent)
// Presenter choice default stage-1 (the known-good AVSampleBufferDisplayLayer). Stage-2 // Presenter choice stage-2 is the DEFAULT (explicit VTDecompressionSession decode + a
// (`punktfunk.presenter == "stage2"`) takes explicit VTDecompressionSession decode + a // CAMetalLayer/display-link present): it can detect + recover a wedged decoder where
// CAMetalLayer/display-link present; it falls back here if Metal can't be set up. // stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference. Stage-1 is
if UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage2", // reachable only via the DEBUG presenter toggle; release always takes stage-2 (the stage-1
// pump below stays the automatic fallback if Metal is missing).
#if DEBUG
let forceStage1 = UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage1"
#else
let forceStage1 = false
#endif
if !forceStage1,
let meter = presentMeter, let meter = presentMeter,
let pipeline = Stage2Pipeline(presentMeter: meter) { let pipeline = Stage2Pipeline(presentMeter: meter) {
startStage2(pipeline, connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd) startStage2(pipeline, connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
@@ -593,9 +609,11 @@ public final class StreamLayerView: NSView {
targetPresentNs: Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: link.targetTimestamp)) targetPresentNs: Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: link.targetTimestamp))
} }
/// Aspect-fit the metal sublayer in the view (the host streams at the client's native mode, /// Position the metal sublayer aspect-fit in the view (the host streams at the client's native
/// so this is usually the full bounds; it letterboxes a resized window). drawableSize is the /// mode, so this is usually the full bounds; it letterboxes a resized window). Only the layer
/// layer's pixel size the fullscreen-triangle shader scales the decoded texture to fill it. /// FRAME is set here the presenter sizes the drawable to the decoded frame and the layer's
/// contentsGravity (.resizeAspect) scales it to this frame via the system compositor, so a
/// resized window rescales through the system's filter (matching stage-1) instead of the shader.
private func layoutMetalLayer() { private func layoutMetalLayer() {
guard let metalLayer, let connection else { return } guard let metalLayer, let connection else { return }
let mode = connection.currentMode() let mode = connection.currentMode()
@@ -604,14 +622,12 @@ public final class StreamLayerView: NSView {
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)), aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds) insideRect: bounds)
: bounds : bounds
let scale = window?.backingScaleFactor ?? 1
// No implicit resize animation; refresh contentsScale on a retinanon-retina move. // No implicit resize animation; refresh contentsScale on a retinanon-retina move.
CATransaction.begin() CATransaction.begin()
CATransaction.setDisableActions(true) CATransaction.setDisableActions(true)
metalLayer.contentsScale = scale metalLayer.contentsScale = window?.backingScaleFactor ?? 1
metalLayer.frame = fit metalLayer.frame = fit
CATransaction.commit() CATransaction.commit()
stage2?.setDrawableSize(CGSize(width: fit.width * scale, height: fit.height * scale))
} }
public override func viewDidChangeBackingProperties() { public override func viewDidChangeBackingProperties() {
@@ -622,7 +638,7 @@ public final class StreamLayerView: NSView {
private func teardownStage2() { private func teardownStage2() {
stage2Link?.invalidate() stage2Link?.invalidate()
stage2Link = nil stage2Link = nil
stage2?.stop() stage2?.stop() // stops the pump (synchronous join) + drops the decode session
stage2 = nil stage2 = nil
metalLayer?.removeFromSuperlayer() metalLayer?.removeFromSuperlayer()
metalLayer = nil metalLayer = nil
@@ -11,13 +11,18 @@
// host mode, so the host's rescale is the identity). // host mode, so the host's rescale is the identity).
// //
// A hardware mouse/trackpad is a pointer, not a finger. When the scene is pointer-LOCKED // A hardware mouse/trackpad is a pointer, not a finger. When the scene is pointer-LOCKED
// (full-screen + frontmost iPad) GCMouse delivers raw relative deltas and the system hides // (full-screen + frontmost iPad, and the user hasn't disabled pointer capture in Settings
// the cursor the gaming-grade path. When it CAN'T lock (Stage Manager, not frontmost, // see PointerLockChain, which steers the lock request through SwiftUI's hosting controllers)
// iPhone) the system shows its own cursor and routes the mouse through UIKit's pointer path: // GCMouse delivers raw relative deltas and the system hides the cursor the gaming-grade path.
// hover + indirect-pointer touches, which we forward as ABSOLUTE cursor position (+ buttons) // InputCapture handles EVERY connected mouse (GCMouse.mice), not just the current one, so a
// so the host cursor tracks the visible local one. We never forward an indirect pointer as a // trackpad + a second pointer (e.g. a Universal Control mouse) both drive. When the scene CAN'T
// touch doing so hid the cursor and made the host see taps instead of a moving mouse. // lock (Stage Manager, not frontmost, iPhone, capture disabled) the system shows its own cursor
// GCMouse is gated off whenever the lock isn't held so the two paths can't double-send. // and routes the mouse through UIKit's pointer path: hover + indirect-pointer touches, which we
// forward as ABSOLUTE cursor position (+ buttons) so the host cursor tracks the visible local one.
// We never forward an indirect pointer as a touch doing so hid the cursor and made the host see
// taps instead of a moving mouse. The two paths are mutually exclusive on `gcMouseForwarding`
// (== locked): GCMouse forwards only WHILE locked, the UIKit indirect path (motion, buttons AND
// scroll) only while NOT locked so a pointer that emits both channels under lock can't double-send.
// Hardware keyboard forwarding shares InputCapture with macOS auto-engaged when streaming // Hardware keyboard forwarding shares InputCapture with macOS auto-engaged when streaming
// starts, toggles (detected from the HID stream; there is no NSEvent monitor here). // starts, toggles (detected from the HID stream; there is no NSEvent monitor here).
// //
@@ -92,8 +97,8 @@ public final class StreamViewController: UIViewController {
public private(set) var connection: PunktfunkConnection? public private(set) var connection: PunktfunkConnection?
private var pump: StreamPump? private var pump: StreamPump?
private var observers: [NSObjectProtocol] = [] private var observers: [NSObjectProtocol] = []
/// Stage-2 presenter (opt-in via `punktfunk.presenter`): a CAMetalLayer sublayer driven by a /// Stage-2 presenter (default): a CAMetalLayer sublayer driven by a CADisplayLink instead of the
/// CADisplayLink instead of the StreamPump displayLayer path. nil = stage-1 (default). /// StreamPump displayLayer path. nil = stage-1 (Metal-unavailable fallback / DEBUG toggle).
var presentMeter: LatencyMeter? var presentMeter: LatencyMeter?
private var stage2: Stage2Pipeline? private var stage2: Stage2Pipeline?
private var stage2Link: CADisplayLink? private var stage2Link: CADisplayLink?
@@ -136,6 +141,13 @@ public final class StreamViewController: UIViewController {
public override func loadView() { public override func loadView() {
view = StreamLayerUIView() view = StreamLayerUIView()
// Re-size the stage-2 drawable if the display scale changes without a bounds change (e.g.
// moving to an external display at a different scale) the iOS analogue of macOS's
// viewDidChangeBackingProperties relayout. The handler takes the VC as its argument, so it
// doesn't capture self (no retain cycle with the registration).
registerForTraitChanges([UITraitDisplayScale.self]) { (vc: StreamViewController, _) in
vc.layoutMetalLayer()
}
#if os(iOS) #if os(iOS)
// Hide the iPadOS cursor while it hovers the video: the host renders its own // Hide the iPadOS cursor while it hovers the video: the host renders its own
// cursor from our deltas, so the local one only diverges from it. This hides the // cursor from our deltas, so the local one only diverges from it. This hides the
@@ -148,19 +160,58 @@ public final class StreamViewController: UIViewController {
} }
#if os(iOS) #if os(iOS)
// Pointer lock is only meaningful on iPad (iPhone has no hardware-pointer lock) and /// Whether the user wants the mouse/trackpad pointer CAPTURED (pointer lock relative
// only when capture is engaged. The system additionally requires full-screen + frontmost /// movement, the gaming default) rather than forwarded as an absolute position (desktop
// and may drop it (Slide Over/Stage Manager/backgrounding) verified in setCaptured(). /// use). Read live from UserDefaults so it tracks the Settings toggle; defaults to on when
public override var prefersPointerLocked: Bool { /// unset. iPad-only gated again in `prefersPointerLocked`.
captured && UIDevice.current.userInterfaceIdiom == .pad private var pointerCaptureEnabled: Bool {
UserDefaults.standard.object(forKey: DefaultsKey.pointerCapture) as? Bool ?? true
} }
/// Whether the pointer should be CAPTURED right now: iPad, capture engaged, and the user
/// hasn't opted into the absolute (desktop) pointer. The system additionally requires
/// full-screen + frontmost and may drop the lock (Slide Over/Stage Manager/backgrounding)
/// syncPointerLock() handles the actual grant/drop and falls back to absolute when unlocked.
private var wantsPointerLock: Bool {
captured && pointerCaptureEnabled && UIDevice.current.userInterfaceIdiom == .pad
}
public override var prefersPointerLocked: Bool { wantsPointerLock }
public override var prefersHomeIndicatorAutoHidden: Bool { true } public override var prefersHomeIndicatorAutoHidden: Bool { true }
// If SwiftUI's UIHostingController reparents us, a plain container parent that forwards // NOTE: we deliberately do NOT override `childViewControllerForPointerLock`. The default
// its pointer-lock decision to its children will then reach this VC. (UIHostingController // returns nil, which tells the system to use THIS controller's own `prefersPointerLocked`
// itself does not consult children, which is why GCMouse deltas can never arrive there // exactly what we want, since `PointerLockChain` forces our SwiftUI ancestors to forward the
// the touch path, always forwarded, is the unconditional fallback.) // downward walk to us and we are the terminal anchor. Returning `self` here would make the
public override var childViewControllerForPointerLock: UIViewController? { self } // system ask the same controller forever (it keeps delegating to the returned child)
// unbounded recursion stack overflow once the chain actually reaches us.
/// (Re)build or tear down the forced pointer-lock forwarding chain from this controller to the
/// window root so the system actually resolves our `prefersPointerLocked`. Safe to call
/// repeatedly it no-ops until the view is in a window with a parent chain, and re-runs from
/// the appearance/parent callbacks once SwiftUI has placed us.
private func updatePointerLockChain() {
// Engaging needs a live parent chain to the window root; disengaging is always safe and
// must run even after the view has left the window (session teardown) so the stamped
// SwiftUI ancestors are cleared.
if wantsPointerLock, view.window != nil {
PointerLockChain.engage(self)
} else {
PointerLockChain.disengage(self)
}
}
public override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
// SwiftUI places us in the hierarchy AFTER start()'s setCaptured(true), and may reparent us
// later re-anchor the chain here so a lock requested before we had a parent still lands.
updatePointerLockChain()
}
public override func didMove(toParent parent: UIViewController?) {
super.didMove(toParent: parent)
updatePointerLockChain() // chain shape changed re-anchor (or no-op if not yet in a window)
}
#endif #endif
func start( func start(
@@ -190,18 +241,22 @@ public final class StreamViewController: UIViewController {
guard self?.captureEnabled == true else { return } guard self?.captureEnabled == true else { return }
connection?.send(event) connection?.send(event)
} }
// Indirect pointer (mouse/trackpad with no lock) absolute cursor + buttons, routed // Indirect pointer (mouse/trackpad) WITHOUT a lock absolute cursor + buttons + scroll.
// through InputCapture so the forwarding gate and release-on-blur apply uniformly. // While the scene is pointer-LOCKED the GCMouse path owns motion AND buttons AND scroll, so
// the whole UIKit indirect path is gated off here (`gcMouseForwarding`). The trackpad and a
// mouse BOTH report through GCMouse under lock and ALSO emit UIKit indirect-pointer events
// (pinned at the locked position) without this gate a click double-sends (GCMouse + UIKit)
// and a second pointer (e.g. a Universal Control mouse) competes with the trackpad. The gate
// is the exact mirror of the GCMouse handlers, which fire only while locked.
streamView.onPointerMoveAbs = { [weak self] p in streamView.onPointerMoveAbs = { [weak self] p in
self?.inputCapture?.sendMouseAbs( guard let self, self.inputCapture?.gcMouseForwarding == false else { return }
self.inputCapture?.sendMouseAbs(
x: p.x, y: p.y, surfaceWidth: p.w, surfaceHeight: p.h) x: p.x, y: p.y, surfaceWidth: p.w, surfaceHeight: p.h)
} }
streamView.onPointerButton = { [weak self] button, down in streamView.onPointerButton = { [weak self] button, down in
self?.inputCapture?.sendMouseButton(button, pressed: down) guard let self, self.inputCapture?.gcMouseForwarding == false else { return }
self.inputCapture?.sendMouseButton(button, pressed: down)
} }
// Trackpad two-finger / wheel scroll host scroll. The pan recognizer is the
// UNLOCKED regime; while locked, GCMouse's scroll handler owns it mirror the
// sendMouseAbs !gcMouseForwarding gate so the two can't double-send.
streamView.onScroll = { [weak self] dx, dy in streamView.onScroll = { [weak self] dx, dy in
guard let self, self.inputCapture?.gcMouseForwarding == false else { return } guard let self, self.inputCapture?.gcMouseForwarding == false else { return }
self.inputCapture?.sendScroll(dx: dx, dy: dy) self.inputCapture?.sendScroll(dx: dx, dy: dy)
@@ -219,10 +274,17 @@ public final class StreamViewController: UIViewController {
inputCapture = capture inputCapture = capture
#endif #endif
// Presenter choice default stage-1 (the known-good AVSampleBufferDisplayLayer). Stage-2 // Presenter choice stage-2 is the DEFAULT (VTDecompressionSession decode + a
// (`punktfunk.presenter == "stage2"`) takes VTDecompressionSession decode + a // CAMetalLayer/display-link present): it can detect + recover a wedged decoder, where
// CAMetalLayer/display-link present; falls back here if Metal can't be set up. // stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference frame with no
if UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage2", // way to recover. Stage-1 is reachable only via the DEBUG presenter toggle; release always
// takes stage-2 (the stage-1 pump below stays the automatic fallback if Metal is missing).
#if DEBUG
let forceStage1 = UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage1"
#else
let forceStage1 = false
#endif
if !forceStage1,
let meter = presentMeter, let meter = presentMeter,
let pipeline = Stage2Pipeline(presentMeter: meter) { let pipeline = Stage2Pipeline(presentMeter: meter) {
startStage2(pipeline, connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd) startStage2(pipeline, connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
@@ -300,8 +362,8 @@ public final class StreamViewController: UIViewController {
onFrame: (@Sendable (AccessUnit) -> Void)?, onSessionEnd: (@Sendable () -> Void)? onFrame: (@Sendable (AccessUnit) -> Void)?, onSessionEnd: (@Sendable () -> Void)?
) { ) {
let metal = pipeline.layer let metal = pipeline.layer
metal.contentsScale = streamView.contentScaleFactor
// Composites OVER the idle (un-enqueued in stage-2) AVSampleBufferDisplayLayer base. // Composites OVER the idle (un-enqueued in stage-2) AVSampleBufferDisplayLayer base.
// (contentsScale + frame are set by layoutMetalLayer() just below.)
streamView.layer.addSublayer(metal) streamView.layer.addSublayer(metal)
metalLayer = metal metalLayer = metal
stage2 = pipeline stage2 = pipeline
@@ -325,9 +387,20 @@ public final class StreamViewController: UIViewController {
layoutMetalLayer() layoutMetalLayer()
} }
/// Aspect-fit the metal sublayer in the view (the host streams at the client's native mode, /// The display scale to render the metal drawable at. `traitCollection.displayScale` is the
/// so this is usually the full bounds). drawableSize is the layer's pixel size; the shader's /// canonical render scale and is reliable once the controller is in the hierarchy;
/// fullscreen triangle scales the decoded texture to fill it. /// `view.contentScaleFactor` can read 1.0 before the view attaches to a window/screen, which
/// would size the drawable at point resolution a pixelated, upscaled mess. Falls back to the
/// main screen scale if the trait is still unspecified.
private var renderScale: CGFloat {
let s = traitCollection.displayScale
return s > 0 ? s : UIScreen.main.scale
}
/// Position the metal sublayer aspect-fit in the view (the host streams at the client's native
/// mode, so this is usually the full bounds). Only the layer FRAME is set here the presenter
/// sizes the drawable to the decoded frame and the layer's contentsGravity (.resizeAspect)
/// scales it to this frame via the system compositor (matching stage-1's videoGravity).
private func layoutMetalLayer() { private func layoutMetalLayer() {
guard let metalLayer, let connection else { return } guard let metalLayer, let connection else { return }
let mode = connection.currentMode() let mode = connection.currentMode()
@@ -337,19 +410,17 @@ public final class StreamViewController: UIViewController {
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)), aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds) insideRect: bounds)
: bounds : bounds
let scale = streamView.contentScaleFactor
CATransaction.begin() CATransaction.begin()
CATransaction.setDisableActions(true) // don't animate the resize CATransaction.setDisableActions(true) // don't animate the resize
metalLayer.contentsScale = scale metalLayer.contentsScale = renderScale
metalLayer.frame = fit metalLayer.frame = fit
CATransaction.commit() CATransaction.commit()
stage2?.setDrawableSize(CGSize(width: fit.width * scale, height: fit.height * scale))
} }
private func teardownStage2() { private func teardownStage2() {
stage2Link?.invalidate() stage2Link?.invalidate()
stage2Link = nil stage2Link = nil
stage2?.stop() stage2?.stop() // stops the pump (synchronous join) + drops the decode session
stage2 = nil stage2 = nil
metalLayer?.removeFromSuperlayer() metalLayer?.removeFromSuperlayer()
metalLayer = nil metalLayer = nil
@@ -369,6 +440,7 @@ public final class StreamViewController: UIViewController {
captured = false captured = false
} }
setNeedsUpdateOfPrefersPointerLocked() setNeedsUpdateOfPrefersPointerLocked()
updatePointerLockChain() // (re)anchor the SwiftUI ancestors so the lock actually resolves
syncPointerLock() // resolve cursor + GCMouse/absolute routing for the current state syncPointerLock() // resolve cursor + GCMouse/absolute routing for the current state
let onCaptureChange = onCaptureChange let onCaptureChange = onCaptureChange
let captured = captured let captured = captured
@@ -49,11 +49,10 @@ public final class VideoDecoder: @unchecked Sendable {
/// pump can re-gate on the next IDR. /// pump can re-gate on the next IDR.
private let onDecodeError: @Sendable (OSStatus) -> Void private let onDecodeError: @Sendable (OSStatus) -> Void
/// Latest source HDR mastering metadata (from `PunktfunkConnection.nextHdrMeta`), attached to /// Whether the negotiated stream is full-chroma 4:4:4 (`connection.isChroma444`), set once at
/// each decoded HDR pixel buffer so the compositor tone-maps from the real grade. Guarded by its /// session start before any decode. Selects the 4:4:4 decode pixel format (orthogonal to bit
/// own lock written by the pump thread, read on the VT decode callback. /// depth / HDR). Read inside `createSessionLocked` under `lock`.
private let metaLock = NSLock() private var chroma444 = false
private var hdrMeta: PunktfunkConnection.HdrMeta?
public init( public init(
onDecoded: @escaping @Sendable (ReadyFrame) -> Void, onDecoded: @escaping @Sendable (ReadyFrame) -> Void,
@@ -65,12 +64,13 @@ public final class VideoDecoder: @unchecked Sendable {
deinit { teardown() } deinit { teardown() }
/// Set the source HDR mastering metadata (drained from `PunktfunkConnection.nextHdrMeta`). It's /// Select the chroma subsampling of the decode output (4:2:0 vs full-chroma 4:4:4). Call once at
/// attached to subsequent decoded HDR pixel buffers. Thread-safe; cheap to call on each update. /// session start, before decoding, from `connection.isChroma444`. Takes effect on the next
public func setHdrMeta(_ meta: PunktfunkConnection.HdrMeta) { /// session (re)build. Thread-safe.
metaLock.lock() public func setChroma444(_ on: Bool) {
hdrMeta = meta lock.lock()
metaLock.unlock() chroma444 = on
lock.unlock()
} }
/// Submit one AU for asynchronous decode, (re)creating the session if `format` changed. The /// Submit one AU for asynchronous decode, (re)creating the session if `format` changed. The
@@ -135,8 +135,10 @@ public final class VideoDecoder: @unchecked Sendable {
/// True when `newFormat` carries a PQ (SMPTE ST 2084) or HLG transfer function i.e. the host /// True when `newFormat` carries a PQ (SMPTE ST 2084) or HLG transfer function i.e. the host
/// is sending HDR (BT.2020). VideoToolbox populates the transfer-function extension from the /// is sending HDR (BT.2020). VideoToolbox populates the transfer-function extension from the
/// HEVC VUI, so this tracks the *stream*, switching dynamically when the user toggles HDR /// HEVC VUI, so this picks the decode bit depth (10-bit P010/x444 vs 8-bit NV12/444v) from the
/// (the host re-emits parameter sets with the new VUI a new format desc session rebuild). /// stream. The present-side HDR config (colorspace/EDR/shader) is latched once per session from
/// the Welcome (`connection.isHDR`), which the host does NOT flip mid-session so this predicate
/// and that config agree for the session (a `#if DEBUG` assert in the presenter guards it).
static func isHDRFormat(_ format: CMVideoFormatDescription) -> Bool { static func isHDRFormat(_ format: CMVideoFormatDescription) -> Bool {
guard guard
let tf = CMFormatDescriptionGetExtension( let tf = CMFormatDescriptionGetExtension(
@@ -157,11 +159,18 @@ public final class VideoDecoder: @unchecked Sendable {
session = nil session = nil
format = nil format = nil
// Decode pixel format is a 2×2 of (chroma, depth/HDR), both biplanar so the presenter binds
// plane 0 = luma, plane 1 = interleaved chroma uniformly 4:4:4 just delivers a full-size
// chroma plane. 10-bit (P010 / `x444`) for HDR (PQ/HLG), 8-bit (NV12 / `444v`) otherwise.
let hdr = Self.isHDRFormat(newFormat) let hdr = Self.isHDRFormat(newFormat)
let pixelFormat = let pixelFormat: OSType = {
hdr switch (chroma444, hdr) {
? kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange // P010 (10-bit) case (false, false): return kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange // NV12
: kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange // NV12 (8-bit) case (false, true): return kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange // P010
case (true, false): return kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange // 444v
case (true, true): return kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange // x444
}
}()
let imageAttrs: [CFString: Any] = [ let imageAttrs: [CFString: Any] = [
kCVPixelBufferMetalCompatibilityKey: true, kCVPixelBufferMetalCompatibilityKey: true,
kCVPixelBufferPixelFormatTypeKey: pixelFormat, kCVPixelBufferPixelFormatTypeKey: pixelFormat,
@@ -169,11 +178,20 @@ public final class VideoDecoder: @unchecked Sendable {
var callback = VTDecompressionOutputCallbackRecord( var callback = VTDecompressionOutputCallbackRecord(
decompressionOutputCallback: decoderOutputCallback, decompressionOutputCallback: decoderOutputCallback,
decompressionOutputRefCon: Unmanaged.passUnretained(self).toOpaque()) decompressionOutputRefCon: Unmanaged.passUnretained(self).toOpaque())
// 4:4:4 sessions REQUIRE a hardware decoder: we only advertise 4:4:4 when the hardware probe
// passed, so a hardware-incapable mode (e.g. a resolution past the HW 4:4:4 ceiling) must fail
// HERE, synchronously, letting the pump's backstop end the session rather than silently
// falling back to a software 4:4:4 decoder far too slow for a real-time stream. 4:2:0 keeps the
// software fallback (nil spec) as a robustness net.
let spec: CFDictionary? =
chroma444
? [kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder: true] as CFDictionary
: nil
var newSession: VTDecompressionSession? var newSession: VTDecompressionSession?
let status = VTDecompressionSessionCreate( let status = VTDecompressionSessionCreate(
allocator: kCFAllocatorDefault, allocator: kCFAllocatorDefault,
formatDescription: newFormat, formatDescription: newFormat,
decoderSpecification: nil, // hardware by default decoderSpecification: spec,
imageBufferAttributes: imageAttrs as CFDictionary, imageBufferAttributes: imageAttrs as CFDictionary,
outputCallback: &callback, outputCallback: &callback,
decompressionSessionOut: &newSession) decompressionSessionOut: &newSession)
@@ -195,26 +213,17 @@ public final class VideoDecoder: @unchecked Sendable {
// pts was stamped at timescale 1e9 (AnnexB.sampleBuffer); normalize defensively. // pts was stamped at timescale 1e9 (AnnexB.sampleBuffer); normalize defensively.
let p = CMTimeConvertScale(pts, timescale: 1_000_000_000, method: .default) let p = CMTimeConvertScale(pts, timescale: 1_000_000_000, method: .default)
let ptsNs = p.value > 0 ? UInt64(p.value) : 0 let ptsNs = p.value > 0 ? UInt64(p.value) : 0
// HDR iff the decoder produced a 10-bit P010 buffer (we only request P010 for PQ streams). // HDR iff the decoder produced a 10-bit buffer (we only request a 10-bit format for PQ/HLG
// streams). Covers 4:2:0 (P010) and 4:4:4 (`x444`), video- and full-range, so a 10-bit 4:4:4
// HDR frame isn't misclassified as SDR. (The mastering metadata is applied to the presenter's
// CAMetalLayer via CAEDRMetadata, not to this source buffer a separate-drawable presenter
// never composites the source buffer's attachments, so attaching them here would be dead.)
let fmt = CVPixelBufferGetPixelFormatType(imageBuffer)
let isHDR = let isHDR =
CVPixelBufferGetPixelFormatType(imageBuffer) fmt == kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange
== kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange || fmt == kCVPixelFormatType_420YpCbCr10BiPlanarFullRange
// Attach the source's mastering display + content light level (ST.2086 / CEA-861.3) so the || fmt == kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange
// compositor tone-maps from the real grade rather than inferring from the PQ colourspace || fmt == kCVPixelFormatType_444YpCbCr10BiPlanarFullRange
// alone. The SEI byte payloads map 1:1 to these CVImageBuffer attachment keys.
if isHDR {
metaLock.lock()
let meta = hdrMeta
metaLock.unlock()
if let meta {
CVBufferSetAttachment(
imageBuffer, kCVImageBufferMasteringDisplayColorVolumeKey,
meta.masteringDisplayColorVolume() as CFData, .shouldPropagate)
CVBufferSetAttachment(
imageBuffer, kCVImageBufferContentLightLevelInfoKey,
meta.contentLightLevelInfo() as CFData, .shouldPropagate)
}
}
onDecoded( onDecoded(
ReadyFrame(ptsNs: ptsNs, decodedNs: decodedNs, pixelBuffer: imageBuffer, isHDR: isHDR)) ReadyFrame(ptsNs: ptsNs, decodedNs: decodedNs, pixelBuffer: imageBuffer, isHDR: isHDR))
} }
@@ -0,0 +1,69 @@
import XCTest
#if canImport(Metal)
import CoreVideo
import Metal
import QuartzCore
@testable import PunktfunkKit
final class MetalPresenterTests: XCTestCase {
/// `MetalVideoPresenter.make()` compiles the runtime Metal shaders (the BT.709/BT.2020 YUVRGB
/// fragment shaders plus the Catmull-Rom luma sampler). A `nil` result on a GPU-equipped host
/// means a shader failed to compile this catches a malformed shader before it silently
/// degrades stage-2 to a stage-1 fallback on device.
func testPresenterInitCompilesShaders() throws {
guard MTLCreateSystemDefaultDevice() != nil else {
throw XCTSkip("no Metal device available in this environment")
}
XCTAssertNotNil(
MetalVideoPresenter.make(),
"stage-2 Metal shaders failed to compile (presenter init returned nil)")
}
/// The HDR fix: `configure(hdr:)` must put the layer into the BT.2020-PQ EDR configuration with a
/// reference-white anchor (`edrMetadata`) the missing anchor was what made HDR render "too
/// bright". SDR must use the plain 8-bit path with EDR off and no metadata. A mid-session flip is a
/// per-mode reconfigure, so the round trip back to SDR must fully restore the SDR config.
func testConfigureHDRSetsEDRAnchor() throws {
guard let presenter = MetalVideoPresenter.make() else {
throw XCTSkip("no Metal device available in this environment")
}
presenter.configure(hdr: true)
XCTAssertEqual(presenter.layer.pixelFormat, .rgba16Float, "HDR uses an EDR-capable drawable")
XCTAssertNotNil(presenter.layer.colorspace, "HDR layer must be tagged (itur_2100_PQ)")
XCTAssertTrue(
presenter.layer.wantsExtendedDynamicRangeContent, "EDR must be requested on all platforms")
XCTAssertNotNil(
presenter.layer.edrMetadata,
"HDR must anchor reference white via edrMetadata (the fix for 'too bright')")
// Mid-session HDRSDR flip: the 8-bit path, EDR off, no metadata.
presenter.configure(hdr: false)
XCTAssertEqual(presenter.layer.pixelFormat, .bgra8Unorm, "SDR uses the plain 8-bit drawable")
XCTAssertFalse(presenter.layer.wantsExtendedDynamicRangeContent)
XCTAssertNil(presenter.layer.edrMetadata)
}
/// `render` with a freshly-allocated NV12 buffer must present without crashing or hanging the
/// main-thread present path is the highest-risk part of the stage-2 rewrite. (A headless CI with no
/// display can still allocate a drawable from a CAMetalLayer; if it can't, render returns false,
/// which is also a valid non-crashing outcome.)
func testRenderDoesNotCrashOnNV12Frame() throws {
guard let presenter = MetalVideoPresenter.make() else {
throw XCTSkip("no Metal device available in this environment")
}
presenter.configure(hdr: false)
var pb: CVPixelBuffer?
let attrs: [CFString: Any] = [kCVPixelBufferMetalCompatibilityKey: true]
let status = CVPixelBufferCreate(
kCFAllocatorDefault, 256, 256, kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange,
attrs as CFDictionary, &pb)
guard status == kCVReturnSuccess, let pixelBuffer = pb else {
throw XCTSkip("could not allocate a test pixel buffer")
}
// Just asserting it returns (true or false) without trapping the layer may have no drawable
// source headless, so a false return is acceptable.
_ = presenter.render(pixelBuffer, isHDR: false)
}
}
#endif
@@ -0,0 +1,68 @@
// 4:4:4 decode-path coverage: the hardware-capability probe is stable/cached, and a real 4:4:4 HEVC
// keyframe decodes through VideoDecoder to a biplanar 4:4:4 pixel buffer. Reuses the same synthetic
// 4:4:4 blobs the runtime probe ships with.
import CoreVideo
import VideoToolbox
import XCTest
@testable import PunktfunkKit
private final class FrameBox: @unchecked Sendable {
let lock = NSLock()
var frame: ReadyFrame?
var error: OSStatus?
}
final class Stage444Tests: XCTestCase {
/// The capability probe is device-static and cached reading it twice must return the same value
/// (and must never crash, including where 4:4:4 is unsupported false).
func testProbeIsStableAndCached() {
XCTAssertEqual(Stage444Probe.hwDecode444_8bit, Stage444Probe.hwDecode444_8bit)
XCTAssertEqual(Stage444Probe.hwDecode444_10bit, Stage444Probe.hwDecode444_10bit)
}
/// A real 8-bit 4:4:4 HEVC keyframe (the embedded probe blob) decodes through `VideoDecoder` with
/// `setChroma444(true)` to a 256×256 biplanar 4:4:4 (`444v`/`444f`) buffer classified SDR.
/// (4:4:4 sessions require a hardware decoder skip where there isn't one, which is exactly where
/// the client wouldn't advertise 4:4:4 anyway.)
func testVideoDecoderDecodes444() throws {
try XCTSkipUnless(
Stage444Probe.hwDecode444_8bit, "no hardware 4:4:4 decode on this device")
let data = Data(Probe444Blobs.au444_8bit)
let format = try XCTUnwrap(
AnnexB.formatDescription(fromIDR: data), "the 4:4:4 blob must yield a format description")
let au = AccessUnit(data: data, ptsNs: 7_000_000, frameIndex: 0, flags: 0)
let box = FrameBox()
let done = DispatchSemaphore(value: 0)
let decoder = VideoDecoder(
onDecoded: { f in box.lock.lock(); box.frame = f; box.lock.unlock(); done.signal() },
onDecodeError: { s in box.lock.lock(); box.error = s; box.lock.unlock(); done.signal() })
decoder.setChroma444(true)
XCTAssertTrue(decoder.decode(au: au, format: format), "4:4:4 frame submit should succeed")
XCTAssertEqual(done.wait(timeout: .now() + 10), .success, "the decode callback must fire")
decoder.reset()
box.lock.lock(); let frame = box.frame; let error = box.error; box.lock.unlock()
XCTAssertNil(error.map { "decode error \($0)" })
let ready = try XCTUnwrap(frame, "a 4:4:4 ReadyFrame must be delivered")
XCTAssertEqual(CVPixelBufferGetWidth(ready.pixelBuffer), 256)
XCTAssertEqual(CVPixelBufferGetHeight(ready.pixelBuffer), 256)
let pf = CVPixelBufferGetPixelFormatType(ready.pixelBuffer)
XCTAssertTrue(
pf == kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange
|| pf == kCVPixelFormatType_444YpCbCr8BiPlanarFullRange,
"expected a biplanar 4:4:4 8-bit buffer, got \(fourCCString(pf))")
XCTAssertFalse(ready.isHDR, "an 8-bit BT.709 4:4:4 stream is SDR")
// The chroma plane (plane 1) must be FULL resolution for 4:4:4 (vs half for 4:2:0) this is
// what lets the unchanged shader sample chroma at the luma UV.
XCTAssertEqual(CVPixelBufferGetWidthOfPlane(ready.pixelBuffer, 1), 256)
XCTAssertEqual(CVPixelBufferGetHeightOfPlane(ready.pixelBuffer, 1), 256)
}
private func fourCCString(_ t: OSType) -> String {
let b = [UInt8(t >> 24 & 0xff), UInt8(t >> 16 & 0xff), UInt8(t >> 8 & 0xff), UInt8(t & 0xff)]
return String(bytes: b, encoding: .ascii) ?? "\(t)"
}
}
+14 -5
View File
@@ -294,7 +294,13 @@ const RESOLUTIONS: [number, number, string][] = [
[2560, 1440, "2560 × 1440"], [2560, 1440, "2560 × 1440"],
]; ];
const REFRESH = [0, 30, 60, 90, 120]; const REFRESH = [0, 30, 60, 90, 120];
const GAMEPADS = ["auto", "xbox360", "dualsense"]; const GAMEPADS = ["auto", "xbox360", "dualsense", "steamdeck"];
const GAMEPAD_LABELS: Record<string, string> = {
auto: "Automatic",
xbox360: "Xbox 360",
dualsense: "DualSense",
steamdeck: "Steam Deck",
};
const SettingsSection: FC = () => { const SettingsSection: FC = () => {
const [s, setS] = useState<StreamSettings | null>(null); const [s, setS] = useState<StreamSettings | null>(null);
@@ -355,14 +361,17 @@ const SettingsSection: FC = () => {
/> />
<Field label="Gamepad type" childrenContainerWidth="max"> <Field label="Gamepad type" childrenContainerWidth="max">
<Dropdown <Dropdown
rgOptions={GAMEPADS.map((g) => ({ rgOptions={GAMEPADS.map((g) => ({ data: g, label: GAMEPAD_LABELS[g] ?? g }))}
data: g,
label: g === "auto" ? "Automatic" : g === "xbox360" ? "Xbox 360" : "DualSense",
}))}
selectedOption={s.gamepad} selectedOption={s.gamepad}
onChange={(o) => patch({ gamepad: o.data as string })} onChange={(o) => patch({ gamepad: o.data as string })}
/> />
</Field> </Field>
{s.gamepad === "steamdeck" && (
<Field
label="⚠ Disable Steam Input"
description="Steam Deck mode forwards the paddles, both trackpads, and gyro to the host. For that, Steam Input must be OFF for punktfunk: on the game page tap ⚙ → Controller Settings → set Steam Input to Off. Otherwise Steam keeps the Deck's controls and only the sticks + buttons reach the host."
/>
)}
<ToggleField <ToggleField
label="Stream microphone" label="Stream microphone"
checked={s.mic_enabled} checked={s.mic_enabled}
+23
View File
@@ -113,12 +113,35 @@ async function ensureShortcut(): Promise<number> {
return appId; return appId;
} }
/**
* Best-effort: turn Steam Input OFF for our shortcut so SDL's HIDAPI Steam Deck driver can open the
* Deck's controls (paddles · trackpads · gyro) directly. There is no confirmed-stable SteamClient
* API for this, so it is feature-detected and MUST never block or throw into the launch — the manual
* toggle (game page → ⚙ → Controller Settings → Steam Input Off, surfaced in the plugin Settings) is
* the documented source of truth. No-op when the optional API is absent.
*/
function disableSteamInputForShortcut(appId: number): void {
try {
const input = (
SteamClient as unknown as {
Input?: { SetSteamInputEnabledForApp?: (appId: number, enabled: boolean) => void };
}
).Input;
input?.SetSteamInputEnabledForApp?.(appId, false);
} catch {
/* a controller tweak must never break the launch */
}
}
/** /**
* Launch a stream to `host:port` fullscreen in Gaming Mode. Encodes the target into the * Launch a stream to `host:port` fullscreen in Gaming Mode. Encodes the target into the
* shortcut's launch options (so one generic shortcut serves every host), then RunGame. * shortcut's launch options (so one generic shortcut serves every host), then RunGame.
*/ */
export async function launchStream(host: string, port: number): Promise<void> { export async function launchStream(host: string, port: number): Promise<void> {
const appId = await ensureShortcut(); const appId = await ensureShortcut();
// Best-effort so the Deck's rich controls reach the client; no-op if the API is absent (the user
// disables Steam Input manually — see the Settings instruction).
disableSteamInputForShortcut(appId);
const target = port && port !== 9777 ? `${host}:${port}` : host; const target = port && port !== 9777 ? `${host}:${port}` : host;
// KEY=value ... %command% — the wrapper reads PF_HOST from the environment. // KEY=value ... %command% — the wrapper reads PF_HOST from the environment.
SteamClient.Apps.SetAppLaunchOptions(appId, `PF_HOST=${target} %command%`); SteamClient.Apps.SetAppLaunchOptions(appId, `PF_HOST=${target} %command%`);
+1
View File
@@ -767,6 +767,7 @@ fn start_session_with(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>,
connector, connector,
frames.take().expect("Connected delivered once"), frames.take().expect("Connected delivered once"),
app.gamepad.escape_events(), app.gamepad.escape_events(),
app.gamepad.disconnect_events(),
handle.stop.clone(), handle.stop.clone(),
inhibit, inhibit,
&title, &title,
+186 -32
View File
@@ -18,7 +18,7 @@ use punktfunk_core::quic::{HidOutput, RichInput};
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::mpsc::{Receiver, Sender}; use std::sync::mpsc::{Receiver, Sender};
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use std::time::Duration; use std::time::{Duration, Instant};
/// Motion scale constants, shared convention with the Swift client (`GamepadWire`): /// Motion scale constants, shared convention with the Swift client (`GamepadWire`):
/// derived from hid-playstation's math over the host's fixed calibration blob. SDL hands /// derived from hid-playstation's math over the host's fixed calibration blob. SDL hands
@@ -33,8 +33,15 @@ const G: f32 = 9.80665;
/// is the only way out. Four simultaneous buttons that no game uses as a deliberate /// is the only way out. Four simultaneous buttons that no game uses as a deliberate
/// combo, so it can't be triggered by normal play. Still forwarded to the host (the user /// combo, so it can't be triggered by normal play. Still forwarded to the host (the user
/// is leaving anyway); we only also raise the escape signal. /// is leaving anyway); we only also raise the escape signal.
///
/// **Escalation:** a quick press leaves fullscreen / releases capture; *holding* the same
/// chord for [`DISCONNECT_HOLD`] ends the session. Deliberately NOT the Steam / QAM buttons —
/// those are the marquee pass-through controls that now reach the host's game-mode UI.
const ESCAPE_CHORD: [u32; 4] = [wire::BTN_LB, wire::BTN_RB, wire::BTN_START, wire::BTN_BACK]; const ESCAPE_CHORD: [u32; 4] = [wire::BTN_LB, wire::BTN_RB, wire::BTN_START, wire::BTN_BACK];
/// Hold the [`ESCAPE_CHORD`] at least this long to disconnect (escalates the leave-fullscreen press).
const DISCONNECT_HOLD: Duration = Duration::from_millis(1500);
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub struct PadInfo { pub struct PadInfo {
pub id: u32, pub id: u32,
@@ -58,6 +65,7 @@ impl PadInfo {
GamepadPref::DualSense => "DualSense", GamepadPref::DualSense => "DualSense",
GamepadPref::DualShock4 => "DualShock 4", GamepadPref::DualShock4 => "DualShock 4",
GamepadPref::XboxOne => "Xbox One", GamepadPref::XboxOne => "Xbox One",
GamepadPref::SteamDeck => "Steam Deck",
_ => "", _ => "",
} }
} }
@@ -89,6 +97,9 @@ pub struct GamepadService {
/// Fires once per press of the [`ESCAPE_CHORD`]; the stream page consumes it to leave /// Fires once per press of the [`ESCAPE_CHORD`]; the stream page consumes it to leave
/// fullscreen + release capture. /// fullscreen + release capture.
escape_rx: async_channel::Receiver<()>, escape_rx: async_channel::Receiver<()>,
/// Fires once when the [`ESCAPE_CHORD`] is held past [`DISCONNECT_HOLD`]; the stream page
/// consumes it to end the session (the controller equivalent of Ctrl+Alt+Shift+D).
disconnect_rx: async_channel::Receiver<()>,
} }
impl GamepadService { impl GamepadService {
@@ -98,11 +109,12 @@ impl GamepadService {
let pinned = Arc::new(Mutex::new(None)); let pinned = Arc::new(Mutex::new(None));
let (ctl, ctl_rx) = std::sync::mpsc::channel(); let (ctl, ctl_rx) = std::sync::mpsc::channel();
let (escape_tx, escape_rx) = async_channel::unbounded(); let (escape_tx, escape_rx) = async_channel::unbounded();
let (disconnect_tx, disconnect_rx) = async_channel::unbounded();
let (p, a, pin) = (pads.clone(), active.clone(), pinned.clone()); let (p, a, pin) = (pads.clone(), active.clone(), pinned.clone());
if let Err(e) = std::thread::Builder::new() if let Err(e) = std::thread::Builder::new()
.name("punktfunk-gamepad".into()) .name("punktfunk-gamepad".into())
.spawn(move || { .spawn(move || {
if let Err(e) = run(&p, &a, &pin, &ctl_rx, &escape_tx) { if let Err(e) = run(&p, &a, &pin, &ctl_rx, &escape_tx, &disconnect_tx) {
tracing::warn!(error = %e, "gamepad service ended — pads disabled"); tracing::warn!(error = %e, "gamepad service ended — pads disabled");
} }
}) })
@@ -115,6 +127,7 @@ impl GamepadService {
pinned, pinned,
ctl, ctl,
escape_rx, escape_rx,
disconnect_rx,
} }
} }
@@ -124,6 +137,12 @@ impl GamepadService {
self.escape_rx.clone() self.escape_rx.clone()
} }
/// A receiver that yields one `()` when the escape chord is held past [`DISCONNECT_HOLD`]
/// (controller disconnect). A fresh clone per call; the stream page spawns a future on it.
pub fn disconnect_events(&self) -> async_channel::Receiver<()> {
self.disconnect_rx.clone()
}
pub fn pads(&self) -> Vec<PadInfo> { pub fn pads(&self) -> Vec<PadInfo> {
self.pads.lock().unwrap().clone() self.pads.lock().unwrap().clone()
} }
@@ -188,6 +207,13 @@ fn button_bit(b: sdl3::gamepad::Button) -> Option<u32> {
Button::DPadLeft => wire::BTN_DPAD_LEFT, Button::DPadLeft => wire::BTN_DPAD_LEFT,
Button::DPadRight => wire::BTN_DPAD_RIGHT, Button::DPadRight => wire::BTN_DPAD_RIGHT,
Button::Touchpad => wire::BTN_TOUCHPAD, Button::Touchpad => wire::BTN_TOUCHPAD,
// Back grips / paddles (Steam Deck L4/L5/R4/R5, Xbox Elite P1P4) + the misc/Share button.
// PADDLE1/2/3/4 = R4/L4/R5/L5 (see the host `input::gamepad`).
Button::RightPaddle1 => wire::BTN_PADDLE1,
Button::LeftPaddle1 => wire::BTN_PADDLE2,
Button::RightPaddle2 => wire::BTN_PADDLE3,
Button::LeftPaddle2 => wire::BTN_PADDLE4,
Button::Misc1 => wire::BTN_MISC1,
_ => return None, _ => return None,
}) })
} }
@@ -259,11 +285,22 @@ struct Worker {
/// Wire state of the active pad — zeroed on the wire at switch/detach. /// Wire state of the active pad — zeroed on the wire at switch/detach.
last_axis: [i32; 6], last_axis: [i32; 6],
held_buttons: Vec<u32>, held_buttons: Vec<u32>,
/// Touchpad contacts the host believes are down, keyed by `(surface, finger)` — lifted on pad
/// switch / detach so a contact held at that moment doesn't stick. surface 0 = the legacy single
/// touchpad, 1/2 = a Steam left/right pad.
held_touches: std::collections::HashSet<(u8, u8)>,
last_accel: [i16; 3], last_accel: [i16; 3],
/// Raises the UI escape signal; the escape chord fires it once per press. /// Raises the UI escape signal; the escape chord fires it once per press.
escape_tx: async_channel::Sender<()>, escape_tx: async_channel::Sender<()>,
/// Raises the UI disconnect signal when the escape chord is held past [`DISCONNECT_HOLD`].
disconnect_tx: async_channel::Sender<()>,
/// The escape chord is fully held — latched so it fires once, not every poll. /// The escape chord is fully held — latched so it fires once, not every poll.
chord_armed: bool, chord_armed: bool,
/// When the escape chord became fully held (drives the hold-to-disconnect escalation); `None`
/// when the chord is broken.
chord_since: Option<Instant>,
/// The disconnect signal already fired for the current hold — latched so it fires once.
disconnect_fired: bool,
} }
impl Worker { impl Worker {
@@ -275,13 +312,22 @@ impl Worker {
fn pad_info(&self, id: u32) -> Option<PadInfo> { fn pad_info(&self, id: u32) -> Option<PadInfo> {
let pad = self.opened.get(&id)?; let pad = self.opened.get(&id)?;
let mut pref = pref_for_type(
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
);
// There is no SDL gamepad type for the Steam Deck / Steam Controller, so detect Valve by
// VID/PID (Deck 0x1205, SC wired 0x1102, SC dongle 0x1142) — the host then builds the virtual
// hid-steam pad with the back grips + dual trackpads and the right glyph identity.
if pad.vendor_id() == Some(0x28DE)
&& matches!(pad.product_id(), Some(0x1205 | 0x1102 | 0x1142))
{
pref = GamepadPref::SteamDeck;
}
Some(PadInfo { Some(PadInfo {
id, id,
name: pad.name().unwrap_or_else(|| "Controller".into()), name: pad.name().unwrap_or_else(|| "Controller".into()),
pref: pref_for_type( pref,
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
),
}) })
} }
@@ -297,32 +343,90 @@ impl Worker {
} }
*v = i32::MIN; *v = i32::MIN;
} }
// Lift any touchpad contact the host still believes is down (surface 0 = legacy pad).
for (surface, finger) in self.held_touches.drain() {
let rich = if surface == 0 {
RichInput::Touchpad {
pad: 0,
finger,
active: false,
x: 0,
y: 0,
}
} else {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: false,
click: false,
x: 0,
y: 0,
pressure: 0,
}
};
let _ = c.send_rich_input(rich);
}
} else { } else {
self.held_buttons.clear(); self.held_buttons.clear();
self.last_axis = [i32::MIN; 6]; self.last_axis = [i32::MIN; 6];
self.held_touches.clear();
} }
// A held chord doesn't survive a flush (detach / pad-switch) — clear its latches too.
self.reset_chord();
} }
/// Raise the UI escape signal when the [`ESCAPE_CHORD`] just completed (latched so it /// Raise the UI escape signal when the [`ESCAPE_CHORD`] just completed (latched so it
/// fires once per press). Called after each button-down updates `held_buttons`. /// fires once per press) and start the hold-to-disconnect timer. Called after each
/// button-down updates `held_buttons`.
fn maybe_fire_escape(&mut self) { fn maybe_fire_escape(&mut self) {
if self.chord_armed { if self.chord_armed {
return; return;
} }
if ESCAPE_CHORD.iter().all(|b| self.held_buttons.contains(b)) { if ESCAPE_CHORD.iter().all(|b| self.held_buttons.contains(b)) {
self.chord_armed = true; self.chord_armed = true;
self.chord_since = Some(Instant::now());
let _ = self.escape_tx.try_send(()); let _ = self.escape_tx.try_send(());
tracing::info!("gamepad escape chord (L1+R1+Start+Select) — leaving fullscreen"); tracing::info!(
"gamepad escape chord (L1+R1+Start+Select) — leaving fullscreen (hold to disconnect)"
);
}
}
/// Fire the disconnect signal once the escape chord has been continuously held past
/// [`DISCONNECT_HOLD`]. Polled from the main loop so the hold completes without new events.
fn maybe_fire_disconnect(&mut self) {
if self.disconnect_fired {
return;
}
if let Some(since) = self.chord_since {
if since.elapsed() >= DISCONNECT_HOLD {
self.disconnect_fired = true;
let _ = self.disconnect_tx.try_send(());
tracing::info!("gamepad escape chord held — disconnecting");
}
} }
} }
/// Re-arm once the chord is broken (any of its buttons released). /// Re-arm once the chord is broken (any of its buttons released).
fn rearm_escape(&mut self) { fn rearm_escape(&mut self) {
if self.chord_armed && !ESCAPE_CHORD.iter().all(|b| self.held_buttons.contains(b)) { if self.chord_armed && !ESCAPE_CHORD.iter().all(|b| self.held_buttons.contains(b)) {
self.chord_armed = false; self.reset_chord();
} }
} }
/// Clear the escape/disconnect chord latches. Called at every session boundary
/// ([`flush_held`](Self::flush_held) on detach/pad-switch + on attach): the hold-to-disconnect
/// path *always* ends the session while the chord is still physically held, so the matching
/// button-up events arrive after detach (dropped by the `attached` guard) and `rearm_escape`
/// never runs — without this the latched state would leak into the next session and either
/// swallow its first chord press or fire a stale disconnect on connect.
fn reset_chord(&mut self) {
self.chord_armed = false;
self.chord_since = None;
self.disconnect_fired = false;
}
/// Sensors stream only while a session wants them (they cost USB/BT bandwidth). /// Sensors stream only while a session wants them (they cost USB/BT bandwidth).
fn set_sensors(&mut self, enabled: bool) { fn set_sensors(&mut self, enabled: bool) {
let Some(id) = self.active_id() else { return }; let Some(id) = self.active_id() else { return };
@@ -335,6 +439,56 @@ impl Worker {
} }
} }
} }
/// Forward one touchpad contact on the rich-input plane. A multi-touchpad pad (Steam Deck / Steam
/// Controller) sends `TouchpadEx` with the surface (SDL touchpad 0 = left → 1, 1 = right → 2) and
/// signed coordinates; a single-touchpad pad (DualSense) keeps the legacy `Touchpad` (unsigned).
fn forward_touch(
&mut self,
which: u32,
touchpad: u32,
finger: u8,
x: f32,
y: f32,
active: bool,
) {
let Some(c) = self.attached.as_ref() else {
return;
};
let multi = self
.opened
.get(&which)
.map(|p| p.touchpads_count() >= 2)
.unwrap_or(false);
let (cx, cy) = (x.clamp(0.0, 1.0), y.clamp(0.0, 1.0));
let surface = if multi { (touchpad as u8) + 1 } else { 0 };
let rich = if multi {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: active,
click: false,
x: (cx * 65535.0 - 32768.0) as i16,
y: (cy * 65535.0 - 32768.0) as i16,
pressure: 0,
}
} else {
RichInput::Touchpad {
pad: 0,
finger,
active,
x: (cx * 65535.0) as u16,
y: (cy * 65535.0) as u16,
}
};
let _ = c.send_rich_input(rich);
if active {
self.held_touches.insert((surface, finger));
} else {
self.held_touches.remove(&(surface, finger));
}
}
} }
#[allow(clippy::too_many_lines)] #[allow(clippy::too_many_lines)]
@@ -344,11 +498,18 @@ fn run(
pinned_out: &Mutex<Option<u32>>, pinned_out: &Mutex<Option<u32>>,
ctl: &Receiver<Ctl>, ctl: &Receiver<Ctl>,
escape_tx: &async_channel::Sender<()>, escape_tx: &async_channel::Sender<()>,
disconnect_tx: &async_channel::Sender<()>,
) -> Result<(), String> { ) -> Result<(), String> {
// Off-main-thread + no video subsystem: keep SDL away from signals, poll pads on its // Off-main-thread + no video subsystem: keep SDL away from signals, poll pads on its
// own thread. // own thread.
sdl3::hint::set("SDL_NO_SIGNAL_HANDLERS", "1"); sdl3::hint::set("SDL_NO_SIGNAL_HANDLERS", "1");
sdl3::hint::set("SDL_JOYSTICK_THREAD", "1"); sdl3::hint::set("SDL_JOYSTICK_THREAD", "1");
// Let SDL's HIDAPI drivers open Valve Steam Controller / Steam Deck devices directly, so the
// paddles, both trackpads, and gyro arrive as first-class SDL gamepad inputs. On a Deck in Game
// Mode, Steam Input still holds the device — the user must disable Steam Input for this app (see
// the Decky UX); on a desktop client (or a Deck with Steam Input off) the hints just work.
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAMDECK", "1");
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAM", "1");
let sdl = sdl3::init().map_err(|e| e.to_string())?; let sdl = sdl3::init().map_err(|e| e.to_string())?;
let subsystem = sdl.gamepad().map_err(|e| e.to_string())?; let subsystem = sdl.gamepad().map_err(|e| e.to_string())?;
let mut pump = sdl.event_pump().map_err(|e| e.to_string())?; let mut pump = sdl.event_pump().map_err(|e| e.to_string())?;
@@ -361,9 +522,13 @@ fn run(
attached: None, attached: None,
last_axis: [i32::MIN; 6], last_axis: [i32::MIN; 6],
held_buttons: Vec::new(), held_buttons: Vec::new(),
held_touches: std::collections::HashSet::new(),
last_accel: [0; 3], last_accel: [0; 3],
escape_tx: escape_tx.clone(), escape_tx: escape_tx.clone(),
disconnect_tx: disconnect_tx.clone(),
chord_armed: false, chord_armed: false,
chord_since: None,
disconnect_fired: false,
}; };
let publish = |w: &Worker| { let publish = |w: &Worker| {
@@ -381,6 +546,7 @@ fn run(
Ok(Ctl::Attach(c)) => { Ok(Ctl::Attach(c)) => {
w.attached = Some(c); w.attached = Some(c);
w.last_axis = [i32::MIN; 6]; w.last_axis = [i32::MIN; 6];
w.reset_chord(); // every session starts un-latched (Attach doesn't flush)
w.set_sensors(true); w.set_sensors(true);
} }
Ok(Ctl::Detach) => { Ok(Ctl::Detach) => {
@@ -474,9 +640,11 @@ fn run(
send(w.attached.as_ref().unwrap(), InputKind::GamepadAxis, id, v); send(w.attached.as_ref().unwrap(), InputKind::GamepadAxis, id, v);
} }
} }
// DualSense touchpad → the rich-input plane, normalized 0..=65535. // Touchpad contacts → the rich-input plane. One pad (DualSense) keeps the legacy
// `Touchpad`; two pads (Steam Deck / Steam Controller) send `TouchpadEx` per surface.
Event::ControllerTouchpadDown { Event::ControllerTouchpadDown {
which, which,
touchpad,
finger, finger,
x, x,
y, y,
@@ -484,41 +652,23 @@ fn run(
} }
| Event::ControllerTouchpadMotion { | Event::ControllerTouchpadMotion {
which, which,
touchpad,
finger, finger,
x, x,
y, y,
.. ..
} if active == Some(which) && w.attached.is_some() => { } if active == Some(which) && w.attached.is_some() => {
let _ = w w.forward_touch(which, touchpad as u32, finger as u8, x, y, true);
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: true,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
} }
Event::ControllerTouchpadUp { Event::ControllerTouchpadUp {
which, which,
touchpad,
finger, finger,
x, x,
y, y,
.. ..
} if active == Some(which) && w.attached.is_some() => { } if active == Some(which) && w.attached.is_some() => {
let _ = w w.forward_touch(which, touchpad as u32, finger as u8, x, y, false);
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: false,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
} }
// Motion: accel events update the cache; each gyro event ships a sample // Motion: accel events update the cache; each gyro event ships a sample
// (the DualSense reports both at ~250 Hz). Scale convention shared with // (the DualSense reports both at ~250 Hz). Scale convention shared with
@@ -559,6 +709,10 @@ fn run(
} }
} }
// Escalate a held escape chord to a disconnect (polled — the hold completes with no
// new button events; the chord itself is only detected while a session is attached).
w.maybe_fire_disconnect();
// Feedback planes (this thread is their single consumer). The host re-sends // Feedback planes (this thread is their single consumer). The host re-sends
// rumble state periodically, so a generous duration with refresh-on-update is // rumble state periodically, so a generous duration with refresh-on-update is
// safe — a dropped stop heals within ~500 ms. // safe — a dropped stop heals within ~500 ms.
+36 -3
View File
@@ -124,12 +124,13 @@ impl Capture {
} }
} }
#[allow(clippy::too_many_lines)] #[allow(clippy::too_many_lines, clippy::too_many_arguments)]
pub fn new( pub fn new(
window: &adw::ApplicationWindow, window: &adw::ApplicationWindow,
connector: Arc<NativeClient>, connector: Arc<NativeClient>,
frames: async_channel::Receiver<DecodedFrame>, frames: async_channel::Receiver<DecodedFrame>,
escape_rx: async_channel::Receiver<()>, escape_rx: async_channel::Receiver<()>,
disconnect_rx: async_channel::Receiver<()>,
stop: Arc<AtomicBool>, stop: Arc<AtomicBool>,
inhibit_shortcuts: bool, inhibit_shortcuts: bool,
title: &str, title: &str,
@@ -152,7 +153,7 @@ pub fn new(
stats_label.set_margin_top(12); stats_label.set_margin_top(12);
let hint = gtk::Label::new(Some( let hint = gtk::Label::new(Some(
"Click the stream to capture input · Ctrl+Alt+Shift+Q releases", "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · Ctrl+Alt+Shift+D disconnects",
)); ));
hint.add_css_class("osd"); hint.add_css_class("osd");
hint.set_halign(gtk::Align::Center); hint.set_halign(gtk::Align::Center);
@@ -163,7 +164,9 @@ pub fn new(
// Flashed when entering fullscreen — the only exit affordances once the header bar is // Flashed when entering fullscreen — the only exit affordances once the header bar is
// hidden (F11 on a keyboard; the L1+R1+Start+Select chord on a controller, which is the // hidden (F11 on a keyboard; the L1+R1+Start+Select chord on a controller, which is the
// only way out on a Steam Deck). // only way out on a Steam Deck).
let fs_hint = gtk::Label::new(Some("F11 · L1 + R1 + Start + Select — exit fullscreen")); let fs_hint = gtk::Label::new(Some(
"F11 · L1 + R1 + Start + Select — exit fullscreen (hold to disconnect)",
));
fs_hint.add_css_class("osd"); fs_hint.add_css_class("osd");
fs_hint.set_halign(gtk::Align::Center); fs_hint.set_halign(gtk::Align::Center);
fs_hint.set_valign(gtk::Align::Start); fs_hint.set_valign(gtk::Align::Start);
@@ -297,6 +300,7 @@ pub fn new(
key.set_propagation_phase(gtk::PropagationPhase::Capture); key.set_propagation_phase(gtk::PropagationPhase::Capture);
let cap = capture.clone(); let cap = capture.clone();
let window_k = window.clone(); let window_k = window.clone();
let stop_kb = stop.clone();
key.connect_key_pressed(move |_, keyval, keycode, state| { key.connect_key_pressed(move |_, keyval, keycode, state| {
let chord = gdk::ModifierType::CONTROL_MASK let chord = gdk::ModifierType::CONTROL_MASK
| gdk::ModifierType::ALT_MASK | gdk::ModifierType::ALT_MASK
@@ -309,6 +313,13 @@ pub fn new(
} }
return glib::Propagation::Stop; return glib::Propagation::Stop;
} }
// Ctrl+Alt+Shift+D — leave the session. Now that Steam / QAM pass through to the host,
// the capture toggle alone can't end a stream, so this is the keyboard's explicit exit.
if state.contains(chord) && keyval.to_lower() == gdk::Key::d {
cap.release();
stop_kb.store(true, Ordering::SeqCst);
return glib::Propagation::Stop;
}
if keyval == gdk::Key::F11 { if keyval == gdk::Key::F11 {
if window_k.is_fullscreen() { if window_k.is_fullscreen() {
window_k.unfullscreen(); window_k.unfullscreen();
@@ -442,6 +453,24 @@ pub fn new(
}) })
}; };
// Controller disconnect (escape chord held past the hold threshold) → end the session, the
// controller equivalent of Ctrl+Alt+Shift+D. Setting `stop` ends the session pump, which pops
// this page (and fires `hidden` below). One-shot — the session is going away.
let disconnect_future = {
let window = window.clone();
let cap = capture.clone();
let stop_d = stop.clone();
glib::spawn_future_local(async move {
if disconnect_rx.recv().await.is_ok() {
cap.release();
if window.is_fullscreen() {
window.unfullscreen();
}
stop_d.store(true, Ordering::SeqCst);
}
})
};
// The page's `hidden` fires once navigation away completes (back button, pop on // The page's `hidden` fires once navigation away completes (back button, pop on
// session end) — NOT on the transient unmap/map cycle a NavigationView push performs. // session end) — NOT on the transient unmap/map cycle a NavigationView push performs.
{ {
@@ -449,6 +478,7 @@ pub fn new(
let stop_h = stop.clone(); let stop_h = stop.clone();
let handlers = RefCell::new(Some((fs_handler, active_handler))); let handlers = RefCell::new(Some((fs_handler, active_handler)));
let escape_future = RefCell::new(Some(escape_future)); let escape_future = RefCell::new(Some(escape_future));
let disconnect_future = RefCell::new(Some(disconnect_future));
page.connect_hidden(move |_| { page.connect_hidden(move |_| {
tracing::debug!("stream page hidden — ending session"); tracing::debug!("stream page hidden — ending session");
if let Some((fs, active)) = handlers.borrow_mut().take() { if let Some((fs, active)) = handlers.borrow_mut().take() {
@@ -458,6 +488,9 @@ pub fn new(
if let Some(f) = escape_future.borrow_mut().take() { if let Some(f) = escape_future.borrow_mut().take() {
f.abort(); f.abort();
} }
if let Some(f) = disconnect_future.borrow_mut().take() {
f.abort();
}
if window.is_fullscreen() { if window.is_fullscreen() {
window.unfullscreen(); window.unfullscreen();
} }
+108 -27
View File
@@ -169,6 +169,13 @@ fn button_bit(b: sdl3::gamepad::Button) -> Option<u32> {
Button::DPadLeft => wire::BTN_DPAD_LEFT, Button::DPadLeft => wire::BTN_DPAD_LEFT,
Button::DPadRight => wire::BTN_DPAD_RIGHT, Button::DPadRight => wire::BTN_DPAD_RIGHT,
Button::Touchpad => wire::BTN_TOUCHPAD, Button::Touchpad => wire::BTN_TOUCHPAD,
// Back grips / paddles (Steam Deck L4/L5/R4/R5, Xbox Elite P1P4) + the misc/Share button.
// PADDLE1/2/3/4 = R4/L4/R5/L5 (see the host `input::gamepad`).
Button::RightPaddle1 => wire::BTN_PADDLE1,
Button::LeftPaddle1 => wire::BTN_PADDLE2,
Button::RightPaddle2 => wire::BTN_PADDLE3,
Button::LeftPaddle2 => wire::BTN_PADDLE4,
Button::Misc1 => wire::BTN_MISC1,
_ => return None, _ => return None,
}) })
} }
@@ -240,6 +247,9 @@ struct Worker {
/// Wire state of the active pad — zeroed on the wire at switch/detach. /// Wire state of the active pad — zeroed on the wire at switch/detach.
last_axis: [i32; 6], last_axis: [i32; 6],
held_buttons: Vec<u32>, held_buttons: Vec<u32>,
/// Touchpad contacts the host believes are down, keyed by `(surface, finger)` — lifted on pad
/// switch / detach. surface 0 = the legacy single touchpad, 1/2 = a Steam left/right pad.
held_touches: std::collections::HashSet<(u8, u8)>,
last_accel: [i16; 3], last_accel: [i16; 3],
} }
@@ -252,13 +262,21 @@ impl Worker {
fn pad_info(&self, id: u32) -> Option<PadInfo> { fn pad_info(&self, id: u32) -> Option<PadInfo> {
let pad = self.opened.get(&id)?; let pad = self.opened.get(&id)?;
let mut pref = pref_for_type(
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
);
// No SDL type for the Steam Deck / Steam Controller — detect Valve by VID/PID (Deck 0x1205,
// SC wired 0x1102, SC dongle 0x1142) so the host builds the virtual hid-steam pad.
if pad.vendor_id() == Some(0x28DE)
&& matches!(pad.product_id(), Some(0x1205 | 0x1102 | 0x1142))
{
pref = GamepadPref::SteamDeck;
}
Some(PadInfo { Some(PadInfo {
id, id,
name: pad.name().unwrap_or_else(|| "Controller".into()), name: pad.name().unwrap_or_else(|| "Controller".into()),
pref: pref_for_type( pref,
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
),
}) })
} }
@@ -274,9 +292,33 @@ impl Worker {
} }
*v = i32::MIN; *v = i32::MIN;
} }
for (surface, finger) in self.held_touches.drain() {
let rich = if surface == 0 {
RichInput::Touchpad {
pad: 0,
finger,
active: false,
x: 0,
y: 0,
}
} else {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: false,
click: false,
x: 0,
y: 0,
pressure: 0,
}
};
let _ = c.send_rich_input(rich);
}
} else { } else {
self.held_buttons.clear(); self.held_buttons.clear();
self.last_axis = [i32::MIN; 6]; self.last_axis = [i32::MIN; 6];
self.held_touches.clear();
} }
} }
@@ -292,6 +334,56 @@ impl Worker {
} }
} }
} }
/// Forward one touchpad contact on the rich-input plane. A multi-touchpad pad (Steam Deck / Steam
/// Controller) sends `TouchpadEx` with the surface (SDL touchpad 0 = left → 1, 1 = right → 2) and
/// signed coordinates; a single-touchpad pad (DualSense) keeps the legacy `Touchpad` (unsigned).
fn forward_touch(
&mut self,
which: u32,
touchpad: u32,
finger: u8,
x: f32,
y: f32,
active: bool,
) {
let Some(c) = self.attached.as_ref() else {
return;
};
let multi = self
.opened
.get(&which)
.map(|p| p.touchpads_count() >= 2)
.unwrap_or(false);
let (cx, cy) = (x.clamp(0.0, 1.0), y.clamp(0.0, 1.0));
let surface = if multi { (touchpad as u8) + 1 } else { 0 };
let rich = if multi {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: active,
click: false,
x: (cx * 65535.0 - 32768.0) as i16,
y: (cy * 65535.0 - 32768.0) as i16,
pressure: 0,
}
} else {
RichInput::Touchpad {
pad: 0,
finger,
active,
x: (cx * 65535.0) as u16,
y: (cy * 65535.0) as u16,
}
};
let _ = c.send_rich_input(rich);
if active {
self.held_touches.insert((surface, finger));
} else {
self.held_touches.remove(&(surface, finger));
}
}
} }
#[allow(clippy::too_many_lines)] #[allow(clippy::too_many_lines)]
@@ -305,6 +397,10 @@ fn run(
// thread. // thread.
sdl3::hint::set("SDL_NO_SIGNAL_HANDLERS", "1"); sdl3::hint::set("SDL_NO_SIGNAL_HANDLERS", "1");
sdl3::hint::set("SDL_JOYSTICK_THREAD", "1"); sdl3::hint::set("SDL_JOYSTICK_THREAD", "1");
// Let SDL's HIDAPI drivers open Valve Steam Controller / Steam Deck devices directly, so the
// paddles, both trackpads, and gyro arrive as first-class SDL gamepad inputs.
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAMDECK", "1");
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAM", "1");
let sdl = sdl3::init().map_err(|e| e.to_string())?; let sdl = sdl3::init().map_err(|e| e.to_string())?;
let subsystem = sdl.gamepad().map_err(|e| e.to_string())?; let subsystem = sdl.gamepad().map_err(|e| e.to_string())?;
let mut pump = sdl.event_pump().map_err(|e| e.to_string())?; let mut pump = sdl.event_pump().map_err(|e| e.to_string())?;
@@ -317,6 +413,7 @@ fn run(
attached: None, attached: None,
last_axis: [i32::MIN; 6], last_axis: [i32::MIN; 6],
held_buttons: Vec::new(), held_buttons: Vec::new(),
held_touches: std::collections::HashSet::new(),
last_accel: [0; 3], last_accel: [0; 3],
}; };
@@ -426,9 +523,11 @@ fn run(
send(w.attached.as_ref().unwrap(), InputKind::GamepadAxis, id, v); send(w.attached.as_ref().unwrap(), InputKind::GamepadAxis, id, v);
} }
} }
// DualSense touchpad → the rich-input plane, normalized 0..=65535. // Touchpad contacts → the rich-input plane. One pad (DualSense) keeps the legacy
// `Touchpad`; two pads (Steam Deck / Steam Controller) send `TouchpadEx` per surface.
Event::ControllerTouchpadDown { Event::ControllerTouchpadDown {
which, which,
touchpad,
finger, finger,
x, x,
y, y,
@@ -436,41 +535,23 @@ fn run(
} }
| Event::ControllerTouchpadMotion { | Event::ControllerTouchpadMotion {
which, which,
touchpad,
finger, finger,
x, x,
y, y,
.. ..
} if active == Some(which) && w.attached.is_some() => { } if active == Some(which) && w.attached.is_some() => {
let _ = w w.forward_touch(which, touchpad as u32, finger as u8, x, y, true);
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: true,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
} }
Event::ControllerTouchpadUp { Event::ControllerTouchpadUp {
which, which,
touchpad,
finger, finger,
x, x,
y, y,
.. ..
} if active == Some(which) && w.attached.is_some() => { } if active == Some(which) && w.attached.is_some() => {
let _ = w w.forward_touch(which, touchpad as u32, finger as u8, x, y, false);
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: false,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
} }
// Motion: accel events update the cache; each gyro event ships a sample (the // Motion: accel events update the cache; each gyro event ships a sample (the
// DualSense reports both at ~250 Hz). Scale convention shared with the other // DualSense reports both at ~250 Hz). Scale convention shared with the other
+30 -3
View File
@@ -80,7 +80,14 @@ pub mod control {
pub width: u32, pub width: u32,
pub height: u32, pub height: u32,
pub refresh_hz: u32, pub refresh_hz: u32,
pub _reserved: u32, /// Host-preferred per-client monitor id (`1..=15`) — the EDID serial / IddCx `ConnectorIndex` /
/// `ContainerId` the driver names this monitor by. A given client (keyed by its cert fingerprint)
/// gets a STABLE id across reconnects, so the OS device path + EDID stay identical and Windows
/// reapplies that client's saved per-monitor config (DPI scaling). `0` = AUTO: the driver
/// allocates the lowest-free id (the original slot-based behavior — used for anonymous/TOFU and
/// GameStream sessions). Byte-compatible with the old `_reserved` (offset 20): an un-upgraded
/// driver ignores it (→ auto), which the host detects via [`AddReply::resolved_monitor_id`].
pub preferred_monitor_id: u32,
} }
/// `IOCTL_ADD` reply: the OS target id + the adapter LUID the IDD landed on (split low/high to /// `IOCTL_ADD` reply: the OS target id + the adapter LUID the IDD landed on (split low/high to
@@ -91,7 +98,11 @@ pub mod control {
pub adapter_luid_low: u32, pub adapter_luid_low: u32,
pub adapter_luid_high: i32, pub adapter_luid_high: i32,
pub target_id: u32, pub target_id: u32,
pub _reserved: u32, /// The monitor id the driver ACTUALLY used — echoes [`AddRequest::preferred_monitor_id`] when the
/// preference was honored, or the auto-allocated id otherwise. Byte-compatible with the old
/// `_reserved` (offset 12): an un-upgraded driver leaves it `0`, so the host can tell its
/// preference was ignored (stale driver) and log it instead of silently losing per-client config.
pub resolved_monitor_id: u32,
} }
/// `IOCTL_REMOVE` input. /// `IOCTL_REMOVE` input.
@@ -129,11 +140,13 @@ pub mod control {
assert!(offset_of!(AddRequest, width) == 8); assert!(offset_of!(AddRequest, width) == 8);
assert!(offset_of!(AddRequest, height) == 12); assert!(offset_of!(AddRequest, height) == 12);
assert!(offset_of!(AddRequest, refresh_hz) == 16); assert!(offset_of!(AddRequest, refresh_hz) == 16);
assert!(offset_of!(AddRequest, preferred_monitor_id) == 20);
assert!(size_of::<AddReply>() == 16); assert!(size_of::<AddReply>() == 16);
assert!(offset_of!(AddReply, adapter_luid_low) == 0); assert!(offset_of!(AddReply, adapter_luid_low) == 0);
assert!(offset_of!(AddReply, adapter_luid_high) == 4); assert!(offset_of!(AddReply, adapter_luid_high) == 4);
assert!(offset_of!(AddReply, target_id) == 8); assert!(offset_of!(AddReply, target_id) == 8);
assert!(offset_of!(AddReply, resolved_monitor_id) == 12);
assert!(size_of::<RemoveRequest>() == 8); assert!(size_of::<RemoveRequest>() == 8);
assert!(offset_of!(RemoveRequest, session_id) == 0); assert!(offset_of!(RemoveRequest, session_id) == 0);
@@ -436,11 +449,25 @@ mod tests {
width: 3840, width: 3840,
height: 2160, height: 2160,
refresh_hz: 120, refresh_hz: 120,
_reserved: 0, preferred_monitor_id: 7,
}; };
let bytes = bytemuck::bytes_of(&req); let bytes = bytemuck::bytes_of(&req);
assert_eq!(bytes.len(), 24); assert_eq!(bytes.len(), 24);
assert_eq!(*bytemuck::from_bytes::<control::AddRequest>(bytes), req); assert_eq!(*bytemuck::from_bytes::<control::AddRequest>(bytes), req);
// preferred_monitor_id occupies the old `_reserved` slot at offset 20 — byte-compatible.
assert_eq!(bytes[20..24], 7u32.to_le_bytes());
let reply = control::AddReply {
adapter_luid_low: 0x1234_5678,
adapter_luid_high: -2,
target_id: 262,
resolved_monitor_id: 7,
};
let rbytes = bytemuck::bytes_of(&reply);
assert_eq!(rbytes.len(), 16);
assert_eq!(*bytemuck::from_bytes::<control::AddReply>(rbytes), reply);
// resolved_monitor_id occupies the old `_reserved` slot at offset 12 — byte-compatible.
assert_eq!(rbytes[12..16], 7u32.to_le_bytes());
} }
#[test] #[test]
+167
View File
@@ -492,6 +492,10 @@ pub const PUNKTFUNK_HIDOUT_LED: u8 = 1;
pub const PUNKTFUNK_HIDOUT_PLAYER_LEDS: u8 = 2; pub const PUNKTFUNK_HIDOUT_PLAYER_LEDS: u8 = 2;
/// `PunktfunkHidOutput::kind` — one adaptive-trigger effect (`which` + `effect`/`effect_len` valid). /// `PunktfunkHidOutput::kind` — one adaptive-trigger effect (`which` + `effect`/`effect_len` valid).
pub const PUNKTFUNK_HIDOUT_TRIGGER: u8 = 3; pub const PUNKTFUNK_HIDOUT_TRIGGER: u8 = 3;
/// `PunktfunkHidOutput::kind` — a trackpad haptic pulse (Steam Controller voice-coils). `which` =
/// side (0 = right pad, 1 = left pad); `effect[0..6]` packs `amplitude` / `period` / `count` as
/// little-endian `u16`s with `effect_len = 6`. Clients without trackpad coils drop it.
pub const PUNKTFUNK_HIDOUT_TRACKPAD_HAPTIC: u8 = 4;
/// Capacity of `PunktfunkHidOutput::effect` (the DualSense trigger parameter block). /// Capacity of `PunktfunkHidOutput::effect` (the DualSense trigger parameter block).
pub const PUNKTFUNK_HID_EFFECT_MAX: u8 = 11; pub const PUNKTFUNK_HID_EFFECT_MAX: u8 = 11;
@@ -559,6 +563,23 @@ impl PunktfunkHidOutput {
out.effect[..n].copy_from_slice(&effect[..n]); out.effect[..n].copy_from_slice(&effect[..n]);
out.effect_len = n as u8; out.effect_len = n as u8;
} }
HidOutput::TrackpadHaptic {
pad,
side,
amplitude,
period,
count,
} => {
// No new struct (PunktfunkHidOutput has no size guard): pack into the existing
// `which` (side) + `effect[0..6]` (amplitude/period/count LE), `effect_len = 6`.
out.kind = PUNKTFUNK_HIDOUT_TRACKPAD_HAPTIC;
out.pad = *pad;
out.which = *side;
out.effect[0..2].copy_from_slice(&amplitude.to_le_bytes());
out.effect[2..4].copy_from_slice(&period.to_le_bytes());
out.effect[4..6].copy_from_slice(&count.to_le_bytes());
out.effect_len = 6;
}
} }
out out
} }
@@ -618,6 +639,11 @@ impl PunktfunkHdrMeta {
pub const PUNKTFUNK_RICH_TOUCHPAD: u8 = 1; pub const PUNKTFUNK_RICH_TOUCHPAD: u8 = 1;
/// `PunktfunkRichInput::kind` — a motion sample (`gyro`/`accel` valid). /// `PunktfunkRichInput::kind` — a motion sample (`gyro`/`accel` valid).
pub const PUNKTFUNK_RICH_MOTION: u8 = 2; pub const PUNKTFUNK_RICH_MOTION: u8 = 2;
/// `RichInput::TouchpadEx` kind on the wire — an extended trackpad contact that identifies the
/// surface (0 single / 1 Steam-left / 2 Steam-right) and carries click + pressure. The host decodes
/// it today; *sending* it from a C client needs the size-prefixed `PunktfunkRichInputEx` +
/// `punktfunk_connection_send_rich_input2` (added with client capture).
pub const PUNKTFUNK_RICH_TOUCHPAD_EX: u8 = 3;
/// One rich client→host input for the host's virtual DualSense /// One rich client→host input for the host's virtual DualSense
/// ([`punktfunk_connection_send_rich_input`]): a touchpad contact or a motion sample. Set `kind` /// ([`punktfunk_connection_send_rich_input`]): a touchpad contact or a motion sample. Set `kind`
@@ -666,6 +692,77 @@ impl PunktfunkRichInput {
} }
} }
/// Forward-compatible superset of [`PunktfunkRichInput`] that can also express the rich Steam
/// surfaces: a *second* trackpad (`surface`), a distinct `click` vs touch, signed coordinates, and
/// pressure. Sent via [`punktfunk_connection_send_rich_input2`] — the only way a C client can emit a
/// `TouchpadEx`. The caller MUST set `struct_size = sizeof(PunktfunkRichInputEx)` (the ABI-skew
/// guard, like [`PunktfunkConfig`]); the legacy [`PunktfunkRichInput`] +
/// [`punktfunk_connection_send_rich_input`] stay byte-for-byte for existing callers.
#[cfg(feature = "quic")]
#[repr(C)]
#[derive(Clone, Copy)]
pub struct PunktfunkRichInputEx {
/// MUST equal `sizeof(PunktfunkRichInputEx)`.
pub struct_size: u32,
/// One of `PUNKTFUNK_RICH_*` (`TOUCHPAD` / `MOTION` / `TOUCHPAD_EX`).
pub kind: u8,
/// Gamepad index.
pub pad: u8,
/// Touchpad/TouchpadEx: contact id.
pub finger: u8,
/// Touchpad/TouchpadEx: 1 = finger down / touching, 0 = lifted.
pub active: u8,
/// TouchpadEx: which surface — 0 = single/DualSense, 1 = Steam left pad, 2 = Steam right pad.
pub surface: u8,
/// TouchpadEx: 1 = the pad is physically clicked (depressed), distinct from a touch contact.
pub click: u8,
/// Reserved for alignment; set to 0.
pub _reserved: [u8; 2],
/// TouchpadEx: x coordinate — **signed**, centred at 0 (the real Steam report convention). For a
/// legacy `TOUCHPAD` kind sent through this struct, store the unsigned `0..=65535` value's bits.
pub x: i16,
/// TouchpadEx: y coordinate — signed, centred at 0.
pub y: i16,
/// TouchpadEx: contact pressure (`0` if the surface has no force sensor).
pub pressure: u16,
/// Motion: gyro (pitch, yaw, roll), raw signed-16.
pub gyro: [i16; 3],
/// Motion: accelerometer (x, y, z), raw signed-16.
pub accel: [i16; 3],
}
#[cfg(feature = "quic")]
impl PunktfunkRichInputEx {
fn to_rich(self) -> Option<crate::quic::RichInput> {
use crate::quic::RichInput;
match self.kind {
PUNKTFUNK_RICH_TOUCHPAD_EX => Some(RichInput::TouchpadEx {
pad: self.pad,
surface: self.surface,
finger: self.finger,
touch: self.active != 0,
click: self.click != 0,
x: self.x,
y: self.y,
pressure: self.pressure,
}),
PUNKTFUNK_RICH_MOTION => Some(RichInput::Motion {
pad: self.pad,
gyro: self.gyro,
accel: self.accel,
}),
PUNKTFUNK_RICH_TOUCHPAD => Some(RichInput::Touchpad {
pad: self.pad,
finger: self.finger,
active: self.active != 0,
x: self.x as u16,
y: self.y as u16,
}),
_ => None,
}
}
}
/// Read an optional NUL-terminated UTF-8 string parameter; `Err` = invalid pointer/UTF-8. /// Read an optional NUL-terminated UTF-8 string parameter; `Err` = invalid pointer/UTF-8.
#[cfg(feature = "quic")] #[cfg(feature = "quic")]
unsafe fn opt_cstr<'a>(p: *const std::os::raw::c_char) -> std::result::Result<Option<&'a str>, ()> { unsafe fn opt_cstr<'a>(p: *const std::os::raw::c_char) -> std::result::Result<Option<&'a str>, ()> {
@@ -714,6 +811,22 @@ pub const PUNKTFUNK_GAMEPAD_XBOXONE: u32 = 3;
/// DualSense (minus adaptive triggers / player LEDs / mute). Honored only where available (Linux /// DualSense (minus adaptive triggers / player LEDs / mute). Honored only where available (Linux
/// hosts); otherwise the host falls back to X-Box 360. /// hosts); otherwise the host falls back to X-Box 360.
pub const PUNKTFUNK_GAMEPAD_DUALSHOCK4: u32 = 4; pub const PUNKTFUNK_GAMEPAD_DUALSHOCK4: u32 = 4;
/// UHID classic Steam Controller (Valve `28DE:1102`, kernel `hid-steam`): dual trackpads, gyro,
/// two grip paddles. Reserved — currently folds to `XBOX360` until its backend lands.
pub const PUNKTFUNK_GAMEPAD_STEAMCONTROLLER: u32 = 5;
/// UHID Steam Deck controller (Valve `28DE:1205`, kernel `hid-steam`): full Deck gamepad incl. the
/// four back grips, a right trackpad, and the IMU; re-grabbed by Steam Input with native glyphs when
/// Steam runs on the host. Honored only where available (Linux hosts); else folds to X-Box 360.
pub const PUNKTFUNK_GAMEPAD_STEAMDECK: u32 = 6;
/// Extended `InputEvent` gamepad button bits for embedders building raw events: the four back grips
/// (Steam L4/L5/R4/R5 ≙ Xbox-Elite P1P4) + the misc/capture button, in Moonlight's
/// `buttonFlags2 << 16` namespace. Mirror `input::gamepad::BTN_PADDLE1..4` / `BTN_MISC1`.
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE1: u32 = 0x0001_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE2: u32 = 0x0002_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE3: u32 = 0x0004_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE4: u32 = 0x0008_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_MISC1: u32 = 0x0020_0000;
/// Connect to a `punktfunk/1` host and start a session at `width`x`height`@`refresh_hz`. /// Connect to a `punktfunk/1` host and start a session at `width`x`height`@`refresh_hz`.
/// Blocks up to `timeout_ms` for the handshake. Returns NULL on failure. Equivalent to /// Blocks up to `timeout_ms` for the handshake. Returns NULL on failure. Equivalent to
@@ -742,11 +855,28 @@ const _: () = {
// Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift). // Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift).
const _: () = { const _: () = {
use crate::config::GamepadPref; use crate::config::GamepadPref;
use crate::input::gamepad as g;
assert!(PUNKTFUNK_GAMEPAD_AUTO == GamepadPref::Auto.to_u8() as u32); assert!(PUNKTFUNK_GAMEPAD_AUTO == GamepadPref::Auto.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_XBOX360 == GamepadPref::Xbox360.to_u8() as u32); assert!(PUNKTFUNK_GAMEPAD_XBOX360 == GamepadPref::Xbox360.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_DUALSENSE == GamepadPref::DualSense.to_u8() as u32); assert!(PUNKTFUNK_GAMEPAD_DUALSENSE == GamepadPref::DualSense.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_XBOXONE == GamepadPref::XboxOne.to_u8() as u32); assert!(PUNKTFUNK_GAMEPAD_XBOXONE == GamepadPref::XboxOne.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_DUALSHOCK4 == GamepadPref::DualShock4.to_u8() as u32); assert!(PUNKTFUNK_GAMEPAD_DUALSHOCK4 == GamepadPref::DualShock4.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_STEAMCONTROLLER == GamepadPref::SteamController.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_STEAMDECK == GamepadPref::SteamDeck.to_u8() as u32);
// Extended button bits mirror the wire `input::gamepad` constants.
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE1 == g::BTN_PADDLE1);
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE2 == g::BTN_PADDLE2);
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE3 == g::BTN_PADDLE3);
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE4 == g::BTN_PADDLE4);
assert!(PUNKTFUNK_GAMEPAD_BTN_MISC1 == g::BTN_MISC1);
};
// The additive M3 kinds (TouchpadEx / TrackpadHaptic) must never grow the legacy ABI structs —
// they have no `struct_size` guard, so a layout change would corrupt old-built callers' buffers.
#[cfg(feature = "quic")]
const _: () = {
assert!(core::mem::size_of::<PunktfunkRichInput>() == 20);
assert!(core::mem::size_of::<PunktfunkHidOutput>() == 19);
}; };
/// Trust: `pin_sha256` (NULL or 32 bytes) is the expected SHA-256 fingerprint of the host's /// Trust: `pin_sha256` (NULL or 32 bytes) is the expected SHA-256 fingerprint of the host's
@@ -1727,6 +1857,43 @@ pub unsafe extern "C" fn punktfunk_connection_send_rich_input(
}) })
} }
/// Send a rich client→host input via the forward-compatible [`PunktfunkRichInputEx`] — the only way
/// a C client can emit a `TouchpadEx` (a second trackpad / signed coords / pressure). Set
/// `rich->struct_size = sizeof(PunktfunkRichInputEx)`; a smaller (older-layout) value is rejected.
///
/// # Safety
/// `c` is a valid connection handle; `rich` is null or points to at least its declared
/// `struct_size` bytes.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_send_rich_input2(
c: *mut PunktfunkConnection,
rich: *const PunktfunkRichInputEx,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if rich.is_null() {
return PunktfunkStatus::NullPointer;
}
// Read only the 4-byte size prefix first to bound the subsequent full read (the
// `PunktfunkConfig` ABI-skew precedent).
let declared = unsafe { std::ptr::addr_of!((*rich).struct_size).read_unaligned() } as usize;
if declared < std::mem::size_of::<PunktfunkRichInputEx>() {
return PunktfunkStatus::InvalidArg;
}
match unsafe { *rich }.to_rich() {
Some(r) => match c.inner.send_rich_input(r) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
},
None => PunktfunkStatus::InvalidArg,
}
})
}
/// The currently active session mode — the Welcome's, until an accepted /// The currently active session mode — the Welcome's, until an accepted
/// [`punktfunk_connection_request_mode`] switches it. Safe any time after connect. /// [`punktfunk_connection_request_mode`] switches it. Safe any time after connect.
/// ///
+45 -4
View File
@@ -137,8 +137,9 @@ impl CompositorPref {
/// host decide (its `PUNKTFUNK_GAMEPAD` env var, else X-Box 360). A concrete preference is /// host decide (its `PUNKTFUNK_GAMEPAD` env var, else X-Box 360). A concrete preference is
/// honored only if that backend is available on the host (DualSense / DualShock 4 need Linux UHID); /// honored only if that backend is available on the host (DualSense / DualShock 4 need Linux UHID);
/// otherwise the host falls back and reports the real choice in `Welcome`. The wire form is a single /// otherwise the host falls back and reports the real choice in `Welcome`. The wire form is a single
/// byte (`0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`), appended to /// byte (`0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`,
/// `Hello`/`Welcome` — older peers simply omit/ignore it (an unknown byte degrades to `Auto`). /// `5 = SteamController`, `6 = SteamDeck`), appended to `Hello`/`Welcome` — older peers simply
/// omit/ignore it (an unknown byte degrades to `Auto`).
#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)] #[derive(Clone, Copy, Debug, PartialEq, Eq, Default)]
pub enum GamepadPref { pub enum GamepadPref {
/// Let the host pick (its `PUNKTFUNK_GAMEPAD` env var, else X-Box 360). /// Let the host pick (its `PUNKTFUNK_GAMEPAD` env var, else X-Box 360).
@@ -155,10 +156,19 @@ pub enum GamepadPref {
/// UHID DualShock 4 (kernel `hid-playstation`, ≥ 6.2) — lightbar, touchpad, motion, rumble. Like /// UHID DualShock 4 (kernel `hid-playstation`, ≥ 6.2) — lightbar, touchpad, motion, rumble. Like
/// `DualSense` minus adaptive triggers / player LEDs / mute. Needs Linux UHID on the host. /// `DualSense` minus adaptive triggers / player LEDs / mute. Needs Linux UHID on the host.
DualShock4, DualShock4,
/// UHID classic Steam Controller (Valve `28DE:1102`, kernel `hid-steam`) — dual trackpads, gyro,
/// two grip paddles, trackpad-only haptics. Needs Linux UHID. *(Reserved; its backend is not yet
/// built — currently folds to `Xbox360`; the Deck identity below is the implemented one.)*
SteamController,
/// UHID Steam Deck controller (Valve `28DE:1205`, kernel `hid-steam`) — full Deck gamepad incl.
/// the four back grips (L4/L5/R4/R5), a right trackpad, and the IMU; re-grabbed by Steam Input
/// with native glyphs when Steam runs on the host. Needs Linux UHID.
SteamDeck,
} }
impl GamepadPref { impl GamepadPref {
/// Wire byte. `0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`. /// Wire byte. `0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`,
/// `5 = SteamController`, `6 = SteamDeck`.
pub const fn to_u8(self) -> u8 { pub const fn to_u8(self) -> u8 {
match self { match self {
GamepadPref::Auto => 0, GamepadPref::Auto => 0,
@@ -166,6 +176,8 @@ impl GamepadPref {
GamepadPref::DualSense => 2, GamepadPref::DualSense => 2,
GamepadPref::XboxOne => 3, GamepadPref::XboxOne => 3,
GamepadPref::DualShock4 => 4, GamepadPref::DualShock4 => 4,
GamepadPref::SteamController => 5,
GamepadPref::SteamDeck => 6,
} }
} }
@@ -177,6 +189,8 @@ impl GamepadPref {
2 => GamepadPref::DualSense, 2 => GamepadPref::DualSense,
3 => GamepadPref::XboxOne, 3 => GamepadPref::XboxOne,
4 => GamepadPref::DualShock4, 4 => GamepadPref::DualShock4,
5 => GamepadPref::SteamController,
6 => GamepadPref::SteamDeck,
_ => GamepadPref::Auto, _ => GamepadPref::Auto,
} }
} }
@@ -192,12 +206,14 @@ impl GamepadPref {
GamepadPref::XboxOne GamepadPref::XboxOne
} }
"dualshock4" | "dualshock" | "ds4" | "ps4" => GamepadPref::DualShock4, "dualshock4" | "dualshock" | "ds4" | "ps4" => GamepadPref::DualShock4,
"steamdeck" | "steam-deck" | "deck" => GamepadPref::SteamDeck,
"steamcontroller" | "steam-controller" | "steamcon" => GamepadPref::SteamController,
_ => return None, _ => return None,
}) })
} }
/// Canonical lowercase identifier (`"auto"`, `"xbox360"`, `"dualsense"`, `"xboxone"`, /// Canonical lowercase identifier (`"auto"`, `"xbox360"`, `"dualsense"`, `"xboxone"`,
/// `"dualshock4"`). /// `"dualshock4"`, `"steamcontroller"`, `"steamdeck"`).
pub fn as_str(self) -> &'static str { pub fn as_str(self) -> &'static str {
match self { match self {
GamepadPref::Auto => "auto", GamepadPref::Auto => "auto",
@@ -205,6 +221,8 @@ impl GamepadPref {
GamepadPref::DualSense => "dualsense", GamepadPref::DualSense => "dualsense",
GamepadPref::XboxOne => "xboxone", GamepadPref::XboxOne => "xboxone",
GamepadPref::DualShock4 => "dualshock4", GamepadPref::DualShock4 => "dualshock4",
GamepadPref::SteamController => "steamcontroller",
GamepadPref::SteamDeck => "steamdeck",
} }
} }
} }
@@ -381,4 +399,27 @@ mod tests {
c.fec.fec_percent = 15; // 250 + ceil(250*15/100)=288 > 255 c.fec.fec_percent = 15; // 250 + ceil(250*15/100)=288 > 255
assert!(c.validate().is_err()); assert!(c.validate().is_err());
} }
#[test]
fn gamepad_pref_steam_roundtrip() {
use GamepadPref::*;
// Wire-byte round-trip for the Steam additions; an unknown byte still degrades to Auto.
for (p, b) in [(SteamController, 5u8), (SteamDeck, 6)] {
assert_eq!(p.to_u8(), b);
assert_eq!(GamepadPref::from_u8(b), p);
}
assert_eq!(GamepadPref::from_u8(99), Auto);
// Name parsing + canonical-name round-trip.
assert_eq!(GamepadPref::from_name("steamdeck"), Some(SteamDeck));
assert_eq!(GamepadPref::from_name("deck"), Some(SteamDeck));
assert_eq!(
GamepadPref::from_name("steamcontroller"),
Some(SteamController)
);
assert_eq!(SteamDeck.as_str(), "steamdeck");
assert_eq!(
GamepadPref::from_name(SteamController.as_str()),
Some(SteamController)
);
}
} }
+14
View File
@@ -66,10 +66,24 @@ pub mod gamepad {
pub const BTN_B: u32 = 0x2000; pub const BTN_B: u32 = 0x2000;
pub const BTN_X: u32 = 0x4000; pub const BTN_X: u32 = 0x4000;
pub const BTN_Y: u32 = 0x8000; pub const BTN_Y: u32 = 0x8000;
// Extended buttons in Moonlight's `buttonFlags2 << 16` namespace (see `gamestream/gamepad.rs`),
// so the GameStream paddle path and the native path share one host injector map. The four Steam
// Deck back grips (L4/L5/R4/R5) reuse the four GameStream/Xbox-Elite paddle slots — a semantic
// 1:1 for binding (the device identity carries the glyph distinction).
/// Back grip R4 — SDL `RightPaddle1` / GameStream `PADDLE1`.
pub const BTN_PADDLE1: u32 = 0x0001_0000;
/// Back grip L4 — SDL `LeftPaddle1` / GameStream `PADDLE2`.
pub const BTN_PADDLE2: u32 = 0x0002_0000;
/// Back grip R5 — SDL `RightPaddle2` / GameStream `PADDLE3`.
pub const BTN_PADDLE3: u32 = 0x0004_0000;
/// Back grip L5 — SDL `LeftPaddle2` / GameStream `PADDLE4`.
pub const BTN_PADDLE4: u32 = 0x0008_0000;
/// DualSense touchpad click. Moonlight's extended-button position (`buttonFlags2` /// DualSense touchpad click. Moonlight's extended-button position (`buttonFlags2`
/// merges in at `<< 16`, see `gamestream/gamepad.rs`), so GameStream clients land on /// merges in at `<< 16`, see `gamestream/gamepad.rs`), so GameStream clients land on
/// the same bit. Only the DualSense backend renders it; the xpad has no such button. /// the same bit. Only the DualSense backend renders it; the xpad has no such button.
pub const BTN_TOUCHPAD: u32 = 0x10_0000; pub const BTN_TOUCHPAD: u32 = 0x10_0000;
/// Misc / capture button — the Deck `…`/quick-access, Share/Capture / GameStream `MISC`.
pub const BTN_MISC1: u32 = 0x0020_0000;
/// Axis ids for `InputKind::GamepadAxis`. /// Axis ids for `InputKind::GamepadAxis`.
pub const AXIS_LS_X: u32 = 0; pub const AXIS_LS_X: u32 = 0;
+92 -1
View File
@@ -1218,6 +1218,7 @@ pub fn decode_mic_datagram(b: &[u8]) -> Option<(u32, u64, &[u8])> {
const RICH_TOUCHPAD: u8 = 0x01; const RICH_TOUCHPAD: u8 = 0x01;
const RICH_MOTION: u8 = 0x02; const RICH_MOTION: u8 = 0x02;
const RICH_TOUCHPAD_EX: u8 = 0x03;
/// A rich client→host controller input beyond the fixed [`InputEvent`](crate::input::InputEvent): /// A rich client→host controller input beyond the fixed [`InputEvent`](crate::input::InputEvent):
/// the DualSense touchpad and motion sensors. `pad` is the gamepad index. Wire form is /// the DualSense touchpad and motion sensors. `pad` is the gamepad index. Wire form is
@@ -1241,6 +1242,22 @@ pub enum RichInput {
gyro: [i16; 3], gyro: [i16; 3],
accel: [i16; 3], accel: [i16; 3],
}, },
/// A richer trackpad contact that also identifies *which* physical pad (Steam Controller / Deck
/// have two), carries a separate click vs touch state, and a pressure reading. `surface`:
/// `0` = the single / DualSense touchpad, `1` = the Steam left pad, `2` = the Steam right pad.
/// Coordinates are **signed** (centred at 0), matching the real Steam report; `pressure` is `0`
/// for a surface with no force sensor. New clients send this for every touch surface; the host
/// decodes both `Touchpad` (`0x01`) and `TouchpadEx` (`0x03`) indefinitely.
TouchpadEx {
pad: u8,
surface: u8,
finger: u8,
touch: bool,
click: bool,
x: i16,
y: i16,
pressure: u16,
},
} }
impl RichInput { impl RichInput {
@@ -1264,6 +1281,22 @@ impl RichInput {
out.extend_from_slice(&v.to_le_bytes()); out.extend_from_slice(&v.to_le_bytes());
} }
} }
RichInput::TouchpadEx {
pad,
surface,
finger,
touch,
click,
x,
y,
pressure,
} => {
let state = (touch as u8) | ((click as u8) << 1);
out.extend_from_slice(&[RICH_TOUCHPAD_EX, pad, surface, finger, state]);
out.extend_from_slice(&x.to_le_bytes());
out.extend_from_slice(&y.to_le_bytes());
out.extend_from_slice(&pressure.to_le_bytes());
}
} }
out out
} }
@@ -1288,6 +1321,16 @@ impl RichInput {
accel: [i16at(9), i16at(11), i16at(13)], accel: [i16at(9), i16at(11), i16at(13)],
}) })
} }
RICH_TOUCHPAD_EX if b.len() >= 12 => Some(RichInput::TouchpadEx {
pad: b[2],
surface: b[3],
finger: b[4],
touch: b[5] & 0x01 != 0,
click: b[5] & 0x02 != 0,
x: i16::from_le_bytes([b[6], b[7]]),
y: i16::from_le_bytes([b[8], b[9]]),
pressure: u16::from_le_bytes([b[10], b[11]]),
}),
_ => None, _ => None,
} }
} }
@@ -1296,6 +1339,7 @@ impl RichInput {
const HIDOUT_LED: u8 = 0x01; const HIDOUT_LED: u8 = 0x01;
const HIDOUT_PLAYER_LEDS: u8 = 0x02; const HIDOUT_PLAYER_LEDS: u8 = 0x02;
const HIDOUT_TRIGGER: u8 = 0x03; const HIDOUT_TRIGGER: u8 = 0x03;
const HIDOUT_TRACKPAD_HAPTIC: u8 = 0x04;
/// DualSense feedback flowing host → client (what a game wrote to the host's virtual pad). /// DualSense feedback flowing host → client (what a game wrote to the host's virtual pad).
/// Wire form `[0xCD][kind][pad][fields…]`. The rich analog of the fixed rumble datagram; /// Wire form `[0xCD][kind][pad][fields…]`. The rich analog of the fixed rumble datagram;
@@ -1309,6 +1353,16 @@ pub enum HidOutput {
/// One adaptive-trigger effect: `which` 0 = L2, 1 = R2; `effect` is the raw DualSense /// One adaptive-trigger effect: `which` 0 = L2, 1 = R2; `effect` is the raw DualSense
/// trigger parameter block (mode + params) for the client to replay on a real controller. /// trigger parameter block (mode + params) for the client to replay on a real controller.
Trigger { pad: u8, which: u8, effect: Vec<u8> }, Trigger { pad: u8, which: u8, effect: Vec<u8> },
/// A trackpad haptic pulse for a Steam Controller's voice-coil actuators (its only "rumble").
/// `side` 0 = right pad, 1 = left pad; `amplitude` + `period` (µs off-time) + `count` (pulses)
/// synthesize a buzz. A client without trackpad coils drops it (or maps it to ordinary rumble).
TrackpadHaptic {
pad: u8,
side: u8,
amplitude: u16,
period: u16,
count: u16,
},
} }
impl HidOutput { impl HidOutput {
@@ -1325,6 +1379,18 @@ impl HidOutput {
out.extend_from_slice(&[HIDOUT_TRIGGER, *pad, *which]); out.extend_from_slice(&[HIDOUT_TRIGGER, *pad, *which]);
out.extend_from_slice(effect); out.extend_from_slice(effect);
} }
HidOutput::TrackpadHaptic {
pad,
side,
amplitude,
period,
count,
} => {
out.extend_from_slice(&[HIDOUT_TRACKPAD_HAPTIC, *pad, *side]);
out.extend_from_slice(&amplitude.to_le_bytes());
out.extend_from_slice(&period.to_le_bytes());
out.extend_from_slice(&count.to_le_bytes());
}
} }
out out
} }
@@ -1349,6 +1415,13 @@ impl HidOutput {
which: b[3], which: b[3],
effect: b[4..].to_vec(), effect: b[4..].to_vec(),
}), }),
HIDOUT_TRACKPAD_HAPTIC if b.len() >= 10 => Some(HidOutput::TrackpadHaptic {
pad: b[2],
side: b[3],
amplitude: u16::from_le_bytes([b[4], b[5]]),
period: u16::from_le_bytes([b[6], b[7]]),
count: u16::from_le_bytes([b[8], b[9]]),
}),
_ => None, _ => None,
} }
} }
@@ -2486,6 +2559,16 @@ mod tests {
gyro: [-100, 200, -300], gyro: [-100, 200, -300],
accel: [16384, -8192, 1], accel: [16384, -8192, 1],
}, },
RichInput::TouchpadEx {
pad: 2,
surface: 1,
finger: 1,
touch: true,
click: false,
x: -12345,
y: 30000,
pressure: 4000,
},
] { ] {
let d = ev.encode(); let d = ev.encode();
assert_eq!(d[0], RICH_INPUT_MAGIC); assert_eq!(d[0], RICH_INPUT_MAGIC);
@@ -2494,7 +2577,8 @@ mod tests {
// Disjoint from the fixed input datagram (0xC8); unknown kind + truncation → None. // Disjoint from the fixed input datagram (0xC8); unknown kind + truncation → None.
assert!(RichInput::decode(&[crate::input::INPUT_MAGIC; 18]).is_none()); assert!(RichInput::decode(&[crate::input::INPUT_MAGIC; 18]).is_none());
assert!(RichInput::decode(&[RICH_INPUT_MAGIC, 0x7F]).is_none()); // unknown kind assert!(RichInput::decode(&[RICH_INPUT_MAGIC, 0x7F]).is_none()); // unknown kind
assert!(RichInput::decode(&[RICH_INPUT_MAGIC, RICH_TOUCHPAD, 0]).is_none()); assert!(RichInput::decode(&[RICH_INPUT_MAGIC, RICH_TOUCHPAD, 0]).is_none()); // short
assert!(RichInput::decode(&[RICH_INPUT_MAGIC, RICH_TOUCHPAD_EX, 0, 0, 0, 0]).is_none());
// short // short
} }
@@ -2516,6 +2600,13 @@ mod tests {
which: 1, which: 1,
effect: vec![0x26, 0x90, 0xA0, 0xFF, 0x00, 0x00], effect: vec![0x26, 0x90, 0xA0, 0xFF, 0x00, 0x00],
}, },
HidOutput::TrackpadHaptic {
pad: 0,
side: 1,
amplitude: 0x1234,
period: 0x5678,
count: 9,
},
]; ];
for ev in &cases { for ev in &cases {
let d = ev.encode(); let d = ev.encode();
+10
View File
@@ -89,6 +89,9 @@ tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] }
wayland-client = "0.31" wayland-client = "0.31"
wayland-protocols-wlr = { version = "0.3", features = ["client"] } wayland-protocols-wlr = { version = "0.3", features = ["client"] }
wayland-protocols-misc = { version = "0.3", features = ["client"] } wayland-protocols-misc = { version = "0.3", features = ["client"] }
# `xdg-output` (zxdg_output_v1): the per-output *logical* geometry (post-scale size + global
# position), used by the KWin fake_input backend to map absolute coordinates under display scaling.
wayland-protocols = { version = "0.32", features = ["client"] }
# Codegen for KDE's `zkde_screencast_unstable_v1` (vendored in `protocols/`): create a KWin # Codegen for KDE's `zkde_screencast_unstable_v1` (vendored in `protocols/`): create a KWin
# virtual output sized to the client's resolution and get its PipeWire node (KRdp's path). # virtual output sized to the client's resolution and get its PipeWire node (KRdp's path).
# `wayland-backend` is referenced by the generated interface tables. # `wayland-backend` is referenced by the generated interface tables.
@@ -119,6 +122,10 @@ ash = "0.38"
# `libcuda.so.1` is dlopen'd at runtime (NOT link-time) so one Linux binary runs on NVIDIA # `libcuda.so.1` is dlopen'd at runtime (NOT link-time) so one Linux binary runs on NVIDIA
# (zero-copy via CUDA) AND on AMD/Intel (VAAPI, no NVIDIA driver present) — see `zerocopy::cuda`. # (zero-copy via CUDA) AND on AMD/Intel (VAAPI, no NVIDIA driver present) — see `zerocopy::cuda`.
libloading = "0.8" libloading = "0.8"
# Vendored + trimmed `usbip` server core (no libusb) — presents a virtual Steam Deck over USB/IP
# so the local `vhci_hcd` attaches it: the shippable, Secure-Boot-clean, Steam-Input-promotable
# virtual-Deck transport on non-SteamOS hosts (`inject/linux/steam_usbip.rs`). See the crate's NOTICE.
usbip-sim = { path = "vendor/usbip-sim" }
[target.'cfg(target_os = "windows")'.dependencies] [target.'cfg(target_os = "windows")'.dependencies]
# Windows host backends. `windows` covers the Win32/CCD APIs the SudoVDA virtual-display backend # Windows host backends. `windows` covers the Win32/CCD APIs the SudoVDA virtual-display backend
@@ -175,6 +182,9 @@ windows = { version = "0.62", features = [
# Windows service supervisor (src/service.rs): a kill-on-close job object so a service crash never # Windows service supervisor (src/service.rs): a kill-on-close job object so a service crash never
# orphans the SYSTEM host it launched into the interactive session. # orphans the SYSTEM host it launched into the interactive session.
"Win32_System_JobObjects", "Win32_System_JobObjects",
# CoCreateInstance(PolicyConfigClient) — set the default audio playback/recording endpoints via the
# undocumented IPolicyConfig (audio/windows/audio_control.rs) so mic + desktop audio auto-wire.
"Win32_System_Com",
] } ] }
# The SCM plumbing for the `service` subcommand (define_windows_service! / dispatcher / control # The SCM plumbing for the `service` subcommand (define_windows_service! / dispatcher / control
# handler / ServiceManager install). Wraps the Win32 service API; the supervision loop itself uses # handler / ServiceManager install). Wraps the Win32 service API; the supervision loop itself uses
+5
View File
@@ -42,6 +42,7 @@ pub fn open_audio_capture(channels: u32) -> Result<Box<dyn AudioCapturer>> {
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub fn open_audio_capture(channels: u32) -> Result<Box<dyn AudioCapturer>> { pub fn open_audio_capture(channels: u32) -> Result<Box<dyn AudioCapturer>> {
audio_control::ensure_wired_once();
wasapi_cap::WasapiLoopbackCapturer::open(channels) wasapi_cap::WasapiLoopbackCapturer::open(channels)
.map(|c| Box::new(c) as Box<dyn AudioCapturer>) .map(|c| Box::new(c) as Box<dyn AudioCapturer>)
} }
@@ -77,6 +78,7 @@ pub fn open_virtual_mic(channels: u32) -> Result<Box<dyn VirtualMic>> {
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub fn open_virtual_mic(channels: u32) -> Result<Box<dyn VirtualMic>> { pub fn open_virtual_mic(channels: u32) -> Result<Box<dyn VirtualMic>> {
audio_control::ensure_wired_once();
wasapi_mic::WasapiVirtualMic::open(channels).map(|m| Box::new(m) as Box<dyn VirtualMic>) wasapi_mic::WasapiVirtualMic::open(channels).map(|m| Box::new(m) as Box<dyn VirtualMic>)
} }
@@ -85,6 +87,9 @@ pub fn open_virtual_mic(_channels: u32) -> Result<Box<dyn VirtualMic>> {
anyhow::bail!("virtual mic requires Linux + PipeWire or Windows + a virtual audio device") anyhow::bail!("virtual mic requires Linux + PipeWire or Windows + a virtual audio device")
} }
#[cfg(target_os = "windows")]
#[path = "audio/windows/audio_control.rs"]
mod audio_control;
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
mod linux; mod linux;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
@@ -0,0 +1,227 @@
//! Windows audio device auto-wiring — production mic + desktop-audio passthrough with zero manual
//! setup.
//!
//! A headless host has no real audio output, so BOTH the desktop-audio loopback ([`super::wasapi_cap`])
//! and the virtual mic ([`super::wasapi_mic`]) must run on VIRTUAL audio cables — and on DIFFERENT
//! ones, or the loopback re-captures the injected mic (an infinite echo). The installer bundles
//! VB-Audio Virtual Cable (the mic target: its "CABLE Input" render endpoint → "CABLE Output" capture)
//! and the host auto-installs the Steam Streaming pair (a loopback-capable render). This module wires
//! them up at startup so no manual Sound-settings fiddling is ever needed:
//!
//! * default **PLAYBACK** → a loopback-capable render that is NOT the mic cable (a real output device
//! if one exists, else the Steam Streaming Microphone; **never** the Steam Streaming Speakers, whose
//! loopback is silent — validated live). This is the endpoint [`super::wasapi_cap`] loopback-captures
//! for desktop audio.
//! * default **RECORDING** → the virtual mic's capture endpoint (VB-Cable "CABLE Output") so host apps
//! record the client's mic by default.
//!
//! [`super::wasapi_mic::find_device`] then resolves the mic INJECT target to "CABLE Input" — a render
//! candidate that is NOT the default playback — guaranteeing loopback ≠ mic, so there is no echo.
//!
//! Setting a default endpoint uses the undocumented `IPolicyConfig` COM interface (the only way to set
//! a default device programmatically — neither the `windows` nor `wasapi` crate exposes it; it is the
//! same call `mmsys.cpl` makes). Opt out with `PUNKTFUNK_KEEP_DEFAULT` to leave the user's chosen
//! defaults untouched.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::{anyhow, bail, Result};
use std::ffi::c_void;
use std::sync::Once;
use wasapi::Direction;
/// Run the audio device auto-wiring exactly once per process, before the first capturer/mic opens.
/// Blocks until done so the default playback is set before the loopback captures it. Best-effort:
/// every failure is logged, never fatal (the host then falls back to whatever the current defaults
/// are — exactly the pre-wiring behaviour).
pub(crate) fn ensure_wired_once() {
static WIRED: Once = Once::new();
WIRED.call_once(|| {
if std::env::var_os("PUNKTFUNK_KEEP_DEFAULT").is_some() {
tracing::info!("PUNKTFUNK_KEEP_DEFAULT set — leaving the audio default devices untouched");
return;
}
// Run on a dedicated COM-MTA thread so we never collide with the caller's apartment mode
// (the capture/mic threads each initialize their own COM separately).
let handle = std::thread::Builder::new()
.name("pf-audio-wiring".into())
.spawn(|| {
if wasapi::initialize_mta().ok().is_err() {
tracing::warn!("audio wiring: COM init (MTA) failed — skipping");
return;
}
if let Err(e) = ensure_audio_wiring() {
tracing::warn!(error = %format!("{e:#}"),
"audio auto-wiring failed — mic/desktop audio may need manual device defaults");
}
});
if let Ok(h) = handle {
let _ = h.join();
}
});
}
/// `(friendly_name, endpoint_id)` for every ACTIVE endpoint in direction `dir`.
fn list_endpoints(dir: Direction) -> Vec<(String, String)> {
let mut out = Vec::new();
let Ok(en) = wasapi::DeviceEnumerator::new() else {
return out;
};
let Ok(coll) = en.get_device_collection(&dir) else {
return out;
};
let Ok(n) = coll.get_nbr_devices() else {
return out;
};
for i in 0..n {
if let Ok(dev) = coll.get_device_at_index(i) {
let id = dev.get_id().unwrap_or_default();
if id.is_empty() {
continue;
}
out.push((dev.get_friendlyname().unwrap_or_default(), id));
}
}
out
}
/// Pick the loopback + mic-capture devices and set them as the default playback/recording.
fn ensure_audio_wiring() -> Result<()> {
let renders = list_endpoints(Direction::Render);
let captures = list_endpoints(Direction::Capture);
if renders.is_empty() {
bail!("no active render endpoints to wire");
}
// A render is unusable as the desktop-audio loopback if it is a VB-Cable endpoint (reserved for
// the mic inject) or the Steam Streaming Speakers (its loopback is silent — validated live).
let excluded_loopback =
|ln: &str| ln.contains("cable") || ln.contains("steam streaming speakers");
// "virtual-ish" = a known virtual cable; a render WITHOUT these markers is a real output device,
// the best loopback source (apps render there and the operator can also hear it).
let virtualish = |ln: &str| {
ln.contains("virtual")
|| ln.contains("cable")
|| ln.contains("steam streaming")
|| ln.contains("voicemeeter")
};
let loopback = renders
.iter()
.find(|(n, _)| {
let ln = n.to_lowercase();
!excluded_loopback(&ln) && !virtualish(&ln)
})
.or_else(|| {
renders
.iter()
.find(|(n, _)| n.to_lowercase().contains("steam streaming microphone"))
})
.or_else(|| {
renders
.iter()
.find(|(n, _)| !excluded_loopback(&n.to_lowercase()))
});
// The virtual mic's CAPTURE endpoint host apps record from — VB-Cable "CABLE Output" preferred.
let mic_capture = captures
.iter()
.find(|(n, _)| n.to_lowercase().contains("cable output"))
.or_else(|| {
captures
.iter()
.find(|(n, _)| n.to_lowercase().contains("steam streaming microphone"))
})
.or_else(|| {
captures.iter().find(|(n, _)| {
let ln = n.to_lowercase();
ln.contains("voicemeeter") || ln.contains("virtual")
})
});
match loopback {
Some((name, id)) => match set_default_endpoint(id) {
Ok(()) => tracing::info!(device = %name,
"audio wiring: default playback = desktop-audio loopback source"),
Err(e) => tracing::warn!(device = %name, error = %format!("{e:#}"),
"audio wiring: failed to set the default playback device"),
},
None => {
tracing::warn!("audio wiring: no usable desktop-audio loopback render endpoint found")
}
}
if let Some((name, id)) = mic_capture {
match set_default_endpoint(id) {
Ok(()) => tracing::info!(device = %name,
"audio wiring: default recording = virtual mic (apps record the client's mic)"),
Err(e) => tracing::warn!(device = %name, error = %format!("{e:#}"),
"audio wiring: failed to set the default recording device"),
}
}
Ok(())
}
// --- IPolicyConfig (undocumented): set a default audio endpoint by id, for all three roles. ---
/// The `IPolicyConfig` vtable. Only `SetDefaultEndpoint` is called; the 10 methods between `Release`
/// and it (`GetMixFormat` … `SetPropertyValue`) are placeholders so the slot offset is correct.
#[repr(C)]
struct IPolicyConfigVtbl {
query_interface: unsafe extern "system" fn(
*mut c_void,
*const windows::core::GUID,
*mut *mut c_void,
) -> windows::core::HRESULT,
add_ref: unsafe extern "system" fn(*mut c_void) -> u32,
release: unsafe extern "system" fn(*mut c_void) -> u32,
_reserved: [*const c_void; 10],
set_default_endpoint: unsafe extern "system" fn(
*mut c_void,
windows::core::PCWSTR,
u32,
) -> windows::core::HRESULT,
// SetEndpointVisibility follows — unused.
}
/// Set `device_id` as the default audio endpoint for eConsole/eMultimedia/eCommunications via the
/// undocumented `IPolicyConfig::SetDefaultEndpoint` (the call `mmsys.cpl` makes). Errs if any role
/// fails.
fn set_default_endpoint(device_id: &str) -> Result<()> {
use windows::core::{IUnknown, Interface, GUID, PCWSTR};
use windows::Win32::System::Com::{CoCreateInstance, CLSCTX_ALL};
// PolicyConfigClient coclass + IPolicyConfig (Win7+) IID.
const CLSID_POLICY_CONFIG: GUID = GUID::from_u128(0x870af99c_171d_4f9e_af0d_e63df40c2bc9);
const IID_IPOLICY_CONFIG: GUID = GUID::from_u128(0xf8679f50_850a_41cf_9c72_430f290290c8);
let wide: Vec<u16> = device_id.encode_utf16().chain(std::iter::once(0)).collect();
// SAFETY: CoCreateInstance with a valid CLSID returns an owned, refcounted IUnknown. We QI it for
// IPolicyConfig; on success (HRESULT ok + non-null pointer) we invoke its SetDefaultEndpoint slot
// through the documented vtable layout (3 IUnknown + 10 placeholder methods precede it) with a
// NUL-terminated UTF-16 id and an in-range ERole (0..=2), then Release the QI'd pointer. Every
// pointer is checked non-null before deref; `unk` is Released by its Drop on scope exit.
unsafe {
let unk: IUnknown = CoCreateInstance(&CLSID_POLICY_CONFIG, None, CLSCTX_ALL)
.map_err(|e| anyhow!("CoCreateInstance(PolicyConfig): {e}"))?;
let mut raw: *mut c_void = std::ptr::null_mut();
unk.query(&IID_IPOLICY_CONFIG, &mut raw)
.ok()
.map_err(|e| anyhow!("QueryInterface(IPolicyConfig): {e}"))?;
if raw.is_null() {
bail!("IPolicyConfig QueryInterface returned null");
}
let vtbl = *(raw as *const *const IPolicyConfigVtbl);
let mut result = Ok(());
for role in 0u32..=2 {
let hr = ((*vtbl).set_default_endpoint)(raw, PCWSTR(wide.as_ptr()), role);
if hr.is_err() {
result = hr
.ok()
.map_err(|e| anyhow!("SetDefaultEndpoint(role {role}): {e}"));
}
}
((*vtbl).release)(raw);
result
}
}
@@ -4,10 +4,12 @@
//! **capture** endpoint then surfaces as a microphone that host apps can record from. //! **capture** endpoint then surfaces as a microphone that host apps can record from.
//! //!
//! Target device, by friendly-name substring (first match wins; override with `PUNKTFUNK_MIC_DEVICE`): //! Target device, by friendly-name substring (first match wins; override with `PUNKTFUNK_MIC_DEVICE`):
//! "Steam Streaming Microphone" (ships with Steam Remote Play — exactly this purpose), VB-Audio //! VB-Audio "CABLE Input" (bundled by the installer — the preferred, dedicated mic target), the
//! "CABLE Input", VoiceMeeter, or anything with "virtual" in the name. If none is present we //! "Steam Streaming Microphone", VoiceMeeter, or anything with "virtual" in the name.
//! auto-install the Steam Streaming audio pair (see [`install_steam_audio_pair`]); failing that we //! [`super::audio_control`] sets the default playback to a DIFFERENT loopback-capable device so the
//! return an error with install guidance and the host runs without mic passthrough. //! chosen mic is never the endpoint the loopback captures. If no candidate is present we auto-install
//! the Steam Streaming audio pair (see [`install_steam_audio_pair`]); failing that we return an error
//! with install guidance and the host runs without mic passthrough.
//! //!
//! **Anti-echo guard (the whole point of this being non-trivial).** The desktop-audio plane //! **Anti-echo guard (the whole point of this being non-trivial).** The desktop-audio plane
//! ([`super::wasapi_cap`]) loopback-captures the **default render endpoint**. WASAPI loopback //! ([`super::wasapi_cap`]) loopback-captures the **default render endpoint**. WASAPI loopback
@@ -45,8 +47,8 @@ const MAX_QUEUE_BYTES: usize = (SAMPLE_RATE as usize * 80 / 1000) * BLOCK_ALIGN;
/// Render-endpoint friendly-name substrings (lowercased) we can write into so the device's capture /// Render-endpoint friendly-name substrings (lowercased) we can write into so the device's capture
/// endpoint becomes a host mic. Ordered by preference. /// endpoint becomes a host mic. Ordered by preference.
const CANDIDATES: &[&str] = &[ const CANDIDATES: &[&str] = &[
"cable input", // VB-Audio Virtual Cable — bundled by the installer; the preferred dedicated mic target
"steam streaming microphone", "steam streaming microphone",
"cable input",
"voicemeeter input", "voicemeeter input",
"voicemeeter aux input", "voicemeeter aux input",
"virtual", "virtual",
+34 -117
View File
@@ -59,7 +59,7 @@ pub struct OutputFormat {
/// Produce GPU-resident D3D11 frames (zero-copy for a GPU encoder — NVENC/AMF/QSV) rather than CPU /// Produce GPU-resident D3D11 frames (zero-copy for a GPU encoder — NVENC/AMF/QSV) rather than CPU
/// staging. `false` **only** for the GPU-less software encoder. /// staging. `false` **only** for the GPU-less software encoder.
pub gpu: bool, pub gpu: bool,
/// HDR: the capturer converts to 10-bit (IDD-push FP16 → `Rgb10a2`; the DDA secure-desktop HDR hint). /// HDR: the capturer converts to 10-bit (IDD-push FP16 → `P010`, or `Rgb10a2` for a 4:4:4 source).
/// `false` = 8-bit SDR. /// `false` = 8-bit SDR.
pub hdr: bool, pub hdr: bool,
/// Full-chroma 4:4:4 session: the capturer must keep full chroma — deliver packed **RGB** /// Full-chroma 4:4:4 session: the capturer must keep full chroma — deliver packed **RGB**
@@ -380,23 +380,12 @@ pub fn capture_virtual_output(
.map(|c| Box::new(c) as Box<dyn Capturer>) .map(|c| Box::new(c) as Box<dyn Capturer>)
} }
/// `PUNKTFUNK_NO_WGC=1` forces the pure single-process DDA (Desktop Duplication) path everywhere: it
/// skips WGC in [`capture_virtual_output`] AND bypasses the two-process secure-desktop relay (so even a
/// SYSTEM host captures in-process via DDA, the way Apollo does — one capturer for the normal AND the
/// secure desktop). For bringing DDA up to parity / validating it on its own; all the WGC code stays
/// compiled and comes back the moment the flag is unset.
#[cfg(target_os = "windows")]
pub(crate) fn wgc_disabled() -> bool {
crate::config::config().no_wgc
}
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub fn capture_virtual_output( pub fn capture_virtual_output(
vout: crate::vdisplay::VirtualOutput, vout: crate::vdisplay::VirtualOutput,
want: OutputFormat, want: OutputFormat,
capture: crate::session_plan::CaptureBackend, _capture: crate::session_plan::CaptureBackend,
) -> Result<Box<dyn Capturer>> { ) -> Result<Box<dyn Capturer>> {
use crate::session_plan::CaptureBackend;
let target = vout.win_capture.clone().ok_or_else(|| { let target = vout.win_capture.clone().ok_or_else(|| {
anyhow::anyhow!( anyhow::anyhow!(
"SudoVDA target not yet an active display (needs a WDDM GPU to activate it)" "SudoVDA target not yet an active display (needs a WDDM GPU to activate it)"
@@ -404,97 +393,36 @@ pub fn capture_virtual_output(
})?; })?;
let pref = vout.preferred_mode; let pref = vout.preferred_mode;
let keep = vout.keepalive; let keep = vout.keepalive;
// Full-chroma 4:4:4 needs a full-chroma RGB source. The IDD-push and WGC paths emit subsampled // IDD direct-push is the sole Windows capture path: consume frames straight from the pf-vdisplay
// NV12/P010 by default, which can't reconstruct 4:4:4; route a 4:4:4 session to DDA, which delivers // driver's shared ring (in-process, Session 0 — it captures the secure desktop too; no Desktop
// RGB (Bgra) when its `chroma_444` flag is set. (IDD-push/WGC 4:4:4 capture is a follow-up.) // Duplication, no WGC helper). A FRESH monitor + ring is created per session: a REUSED monitor's
if want.chroma_444 && capture != CaptureBackend::Dda { // swap-chain dies after ~2 sessions and can't be revived. The ring is always FP16 when the display
tracing::info!("4:4:4 session — using DDA capture (RGB source) instead of {capture:?}"); // is HDR (the driver composes the IDD in FP16); `want.hdr` proactively enables advanced color and
return dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444) // selects the per-frame conversion (FP16 → P010 vs BGRA → NV12). `IddPushCapturer` takes the
.map(|c| Box::new(c) as Box<dyn Capturer>); // keepalive (it owns the virtual display). There is NO fallback (DDA + the WGC relay were removed):
} // if it can't open or the driver doesn't attach, the session fails cleanly and the client reconnects.
// P2 direct frame push (kill DDA): consume frames straight from the pf-vdisplay driver's shared idd_push::IddPushCapturer::open(target, pref, want.hdr, keep)
// ring — no Desktop Duplication, no win32u reparenting hook. Resolved once in the `SessionPlan`
// (was re-derived from `config().idd_push` here); `IddPush` takes the keepalive (owns the virtual
// display) so there's no fall-through.
if capture == CaptureBackend::IddPush {
// Recreate the monitor + ring per session (fix-teardown): a FRESH monitor reliably gets a
// working IddCx swap-chain, whereas a REUSED monitor's swap-chain dies after ~2 sessions and
// the host can't revive it. The driver's recreate crash (target id resolved to 0) is fixed by
// stamping target_id onto the monitor context. The ring is always FP16 (the driver composes
// the IDD in FP16); `want_hdr` selects the per-frame conversion (FP16 → Rgb10a2 vs Bgra).
// If IDD-push can't open OR the driver doesn't attach to the ring within a few seconds (e.g. a
// hybrid-GPU render mismatch), fall back to DDA so the session is NEVER left black (audit §5.1).
// `open()` hands the keepalive back on failure so DDA can take ownership of the virtual display.
match idd_push::IddPushCapturer::open(target.clone(), pref, want.hdr, keep) {
Ok(c) => return Ok(Box::new(c) as Box<dyn Capturer>),
Err((e, keep)) => {
tracing::warn!(
error = %format!("{e:#}"),
"IDD-push open/attach failed — falling back to DDA"
);
return dxgi::DuplCapturer::open(
target,
pref,
keep,
want.gpu,
false,
want.chroma_444,
)
.map(|c| Box::new(c) as Box<dyn Capturer>);
}
}
}
// WGC (Windows.Graphics.Capture) is the default: it captures the COMPOSED desktop including the
// overlay/independent-flip planes DXGI Desktop Duplication misses (the frozen-HDR-animation bug),
// and has no ACCESS_LOST-on-overlay churn. DDA stays available via PUNKTFUNK_CAPTURE=dda and is
// the secure-desktop (lock/UAC) fallback (WGC can't capture those). `keep` is moved into the
// chosen backend (it owns the SudoVDA keepalive), so there's no open-time auto-fallback. The
// backend choice (`dda`/`dxgi`/`PUNKTFUNK_NO_WGC` → DDA, else WGC) is now resolved once in the plan.
if capture == CaptureBackend::Dda {
return dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444)
.map(|c| Box::new(c) as Box<dyn Capturer>);
}
// WGC default, with a watchdog'd DDA fallback. WGC's Direct3D11CaptureFramePool::CreateFreeThreaded
// intermittently HANGS on the headless SudoVDA (IddCx) display — a blocking call we can't error out
// of in place. So run WGC open on a dedicated thread and bound it: if it doesn't finish in time
// (hang) or errors, fall back to the reliable DDA path so the session is NEVER left black. WGC,
// when it opens, captures the composed desktop (overlay/MPO-correct HDR — fixes frozen animations);
// DDA is the safety net (+ the secure-desktop path). The encode thread is set MTA so the WGC
// objects built on the watchdog thread (also MTA) are usable here; the keepalive is handed to WGC
// only on success, else to DDA. A hung watchdog thread is abandoned (holds no keepalive).
// SAFETY: `RoInitialize` is a combase FFI call that initializes the WinRT apartment for the calling
// thread. It takes the `RO_INIT_MULTITHREADED` enum by value and borrows no memory, so there is no
// pointer/lifetime/aliasing obligation; it is safe on any thread and idempotent — a second call on a
// thread already in a compatible apartment returns S_FALSE / RPC_E_CHANGED_MODE, which we discard.
// Runs on the encode thread that goes on to use the WGC (WinRT) objects built by the watchdog thread.
unsafe {
let _ = windows::Win32::System::WinRT::RoInitialize(
windows::Win32::System::WinRT::RO_INIT_MULTITHREADED,
);
}
let (tx, rx) = std::sync::mpsc::channel();
let t = target.clone();
let _ = std::thread::Builder::new()
.name("wgc-open".into())
.spawn(move || {
let _ = tx.send(wgc::WgcCapturer::open(t, pref));
});
match rx.recv_timeout(std::time::Duration::from_secs(5)) {
Ok(Ok(mut c)) => {
c.attach_keepalive(keep);
Ok(Box::new(c) as Box<dyn Capturer>)
}
Ok(Err(e)) => {
tracing::warn!(error = %format!("{e:#}"), "WGC open failed — falling back to DDA");
dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444)
.map(|c| Box::new(c) as Box<dyn Capturer>) .map(|c| Box::new(c) as Box<dyn Capturer>)
} .map_err(|(e, _keep)| e.context("IDD-push capture open (no fallback)"))
Err(_) => { }
tracing::warn!("WGC open timed out (CreateFreeThreaded hang on the virtual display) — falling back to DDA");
dxgi::DuplCapturer::open(target, pref, keep, want.gpu, false, want.chroma_444) /// Whether the active capturer can deliver a full-chroma (RGB) source for a 4:4:4 HEVC encode. The
.map(|c| Box::new(c) as Box<dyn Capturer>) /// negotiator gates 4:4:4 on this so the host honestly downgrades to 4:2:0 when the capturer can only
} /// produce subsampled frames. Linux (the portal capturer feeding CPU RGB → `yuv444p`) can; the Windows
} /// IDD-push path delivers subsampled NV12/P010 today, so full-chroma capture there is a follow-up.
#[cfg(target_os = "linux")]
pub(crate) fn capturer_supports_444() -> bool {
true
}
#[cfg(target_os = "windows")]
pub(crate) fn capturer_supports_444() -> bool {
// IDD-push 4:4:4 (full-chroma RGB from the FP16 ring) is the next step; until then the sole Windows
// capturer delivers subsampled NV12/P010 only, so the host honestly negotiates 4:2:0.
false
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub(crate) fn capturer_supports_444() -> bool {
false
} }
#[cfg(not(any(target_os = "linux", target_os = "windows")))] #[cfg(not(any(target_os = "linux", target_os = "windows")))]
@@ -506,14 +434,9 @@ pub fn capture_virtual_output(
anyhow::bail!("virtual-output capture requires Linux or Windows") anyhow::bail!("virtual-output capture requires Linux or Windows")
} }
// Goal-1 stage 6: the Windows backends live under `capture/windows/`, the Linux one under `capture/linux/` // Goal-1 stage 6: the Windows backend lives under `capture/windows/`, the Linux one under `capture/linux/`
// (`#[path]` keeps the module names flat, so every `crate::capture::*` path is unchanged). // (`#[path]` keeps the module names flat, so every `crate::capture::*` path is unchanged). Windows capture
#[cfg(target_os = "windows")] // is IDD direct-push only — DXGI Desktop Duplication (DDA) and the WGC two-process relay were removed.
#[path = "capture/windows/composed_flip.rs"]
pub mod composed_flip;
#[cfg(target_os = "windows")]
#[path = "capture/windows/desktop_watch.rs"]
pub mod desktop_watch;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
#[path = "capture/windows/dxgi.rs"] #[path = "capture/windows/dxgi.rs"]
pub mod dxgi; pub mod dxgi;
@@ -522,9 +445,3 @@ pub mod dxgi;
pub mod idd_push; pub mod idd_push;
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
mod linux; mod linux;
#[cfg(target_os = "windows")]
#[path = "capture/windows/wgc.rs"]
pub mod wgc;
#[cfg(target_os = "windows")]
#[path = "capture/windows/wgc_relay.rs"]
pub mod wgc_relay;
@@ -1,217 +0,0 @@
//! Force-composed-flip overlay (Windows) — make the secure (Winlogon: UAC / lock / login) desktop
//! capturable by Desktop Duplication.
//!
//! The secure desktop's dialog/wallpaper presents via **fullscreen independent-flip / MPO**: it scans
//! out directly, bypassing DWM composition, so `IDXGIOutputDuplication::AcquireNextFrame` returns
//! `DXGI_ERROR_ACCESS_LOST` (born-lost) — there is no composed frame to hand out (the client sees
//! black). Independent-flip requires the presenting app to own the ENTIRE output: putting ANY other
//! visible window on that output disqualifies it, forcing DWM to **composite**, which DDA can then
//! capture. So we keep a tiny, click-through, near-invisible **topmost layered window** alive on the
//! *current input desktop* (which is Winlogon while the secure desktop is up). On a desktop switch the
//! window is orphaned, so a dedicated thread tracks the input desktop and recreates it there.
//!
//! This is the non-input alternative to "tap a key to wake the lock screen": it needs no SendInput and
//! no system-wide registry change (it does NOT disable MPO globally — it only nudges OUR output to
//! composed while a session is live). Effectiveness can be build/driver-dependent; gated by
//! `PUNKTFUNK_FORCE_COMPOSED` (default ON; set =0 to disable).
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use windows::core::w;
use windows::Win32::Foundation::{HWND, LPARAM, LRESULT, WPARAM};
use windows::Win32::System::LibraryLoader::GetModuleHandleW;
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, GetUserObjectInformationW, OpenInputDesktop, SetThreadDesktop,
DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS, UOI_NAME,
};
use windows::Win32::UI::WindowsAndMessaging::{
CreateWindowExW, DefWindowProcW, DestroyWindow, DispatchMessageW, PeekMessageW, RegisterClassW,
SetLayeredWindowAttributes, SetWindowPos, ShowWindow, TranslateMessage, HWND_TOPMOST,
LWA_ALPHA, MSG, PM_REMOVE, SWP_NOACTIVATE, SWP_NOMOVE, SWP_NOSIZE, SW_SHOWNOACTIVATE,
WNDCLASSW, WS_EX_LAYERED, WS_EX_NOACTIVATE, WS_EX_TOOLWINDOW, WS_EX_TOPMOST, WS_EX_TRANSPARENT,
WS_POPUP,
};
/// A running force-composed-flip overlay. Drop signals the thread to tear down its window + exit.
pub struct ForceComposedFlip {
stop: Arc<AtomicBool>,
}
impl ForceComposedFlip {
/// Start the overlay (no-op + `None` if disabled via `PUNKTFUNK_FORCE_COMPOSED=0`).
pub fn start() -> Option<Self> {
if std::env::var("PUNKTFUNK_FORCE_COMPOSED").as_deref() == Ok("0") {
tracing::info!("force-composed-flip overlay disabled (PUNKTFUNK_FORCE_COMPOSED=0)");
return None;
}
let stop = Arc::new(AtomicBool::new(false));
let st = stop.clone();
std::thread::Builder::new()
.name("composed-flip".into())
// SAFETY: `run` is this module's `unsafe fn` (it owns a desktop+window lifecycle via Win32
// FFI); it takes ownership of `st` (the stop `Arc<AtomicBool>`) and has no caller-side memory
// precondition. It is designed to own its thread for its whole duration — exactly the
// dedicated `composed-flip` thread spawned here.
.spawn(move || unsafe { run(st) })
.ok()?;
tracing::info!("force-composed-flip overlay started (Winlogon-aware)");
Some(ForceComposedFlip { stop })
}
}
impl Drop for ForceComposedFlip {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
}
}
extern "system" fn wndproc(hwnd: HWND, msg: u32, wp: WPARAM, lp: LPARAM) -> LRESULT {
// SAFETY: this is the window procedure the OS invokes with the window's own `hwnd` and a real
// message `(msg, wp, lp)`. `DefWindowProcW` performs default processing for exactly those
// parameters (all passed straight through by value); it borrows no Rust memory and is synchronous.
unsafe { DefWindowProcW(hwnd, msg, wp, lp) }
}
/// Read the current input-desktop name (e.g. "Default" / "Winlogon"); `None` if it can't be read.
unsafe fn input_desktop_name() -> Option<String> {
let desk = OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(0x0001),
)
.ok()?;
let mut buf = [0u16; 64];
let mut needed = 0u32;
let ok = GetUserObjectInformationW(
windows::Win32::Foundation::HANDLE(desk.0),
UOI_NAME,
Some(buf.as_mut_ptr() as *mut _),
(buf.len() * 2) as u32,
Some(&mut needed),
)
.is_ok();
let _ = CloseDesktop(desk);
if !ok {
return None;
}
Some(
String::from_utf16_lossy(&buf)
.trim_end_matches('\u{0}')
.to_string(),
)
}
/// Create the tiny topmost layered click-through window on the CURRENT thread's desktop. Caller must
/// have `SetThreadDesktop`'d to the target input desktop first.
unsafe fn make_overlay() -> Option<HWND> {
let hinst = GetModuleHandleW(None).ok()?;
let class = w!("PunktfunkComposedFlip");
// RegisterClassW is idempotent-ish: a second register for the same name fails harmlessly; we
// ignore the result and rely on the class existing. (One process, so it registers once.)
let wc = WNDCLASSW {
lpfnWndProc: Some(wndproc),
hInstance: hinst.into(),
lpszClassName: class,
..Default::default()
};
let atom = RegisterClassW(&wc);
if atom == 0 {
let e = windows::Win32::Foundation::GetLastError();
// 1410 = ERROR_CLASS_ALREADY_EXISTS is fine (re-register after a desktop switch).
if e.0 != 1410 {
tracing::warn!(err = e.0, "force-composed-flip: RegisterClassW failed");
}
}
let hwnd = match CreateWindowExW(
WS_EX_LAYERED | WS_EX_TRANSPARENT | WS_EX_TOPMOST | WS_EX_NOACTIVATE | WS_EX_TOOLWINDOW,
class,
w!(""),
WS_POPUP,
0,
0,
1,
1,
None,
None,
Some(hinst.into()),
None,
) {
Ok(h) => h,
Err(e) => {
let le = windows::Win32::Foundation::GetLastError();
tracing::warn!(err = %format!("{e:?}"), last = le.0,
"force-composed-flip: CreateWindowExW failed");
return None;
}
};
// alpha=1: technically visible (so it disqualifies independent-flip) but imperceptible.
let _ = SetLayeredWindowAttributes(hwnd, windows::Win32::Foundation::COLORREF(0), 1, LWA_ALPHA);
let _ = ShowWindow(hwnd, SW_SHOWNOACTIVATE);
let _ = SetWindowPos(
hwnd,
Some(HWND_TOPMOST),
0,
0,
0,
0,
SWP_NOMOVE | SWP_NOSIZE | SWP_NOACTIVATE,
);
Some(hwnd)
}
unsafe fn run(stop: Arc<AtomicBool>) {
let mut cur_desktop: Option<String> = None;
let mut hwnd: Option<HWND> = None;
let mut ticks: u32 = 0;
while !stop.load(Ordering::Relaxed) {
// Follow the input desktop: if it changed (Default↔Winlogon), re-attach this thread and
// recreate the window there (a window is bound to the desktop it was created on).
let name = input_desktop_name();
if name != cur_desktop {
if let Some(h) = hwnd.take() {
let _ = DestroyWindow(h);
}
if let Ok(desk) = OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(0x1000_0000), // GENERIC_ALL (incl. DESKTOP_CREATEWINDOW=0x0002)
) {
if SetThreadDesktop(desk).is_ok() {
hwnd = make_overlay();
tracing::info!(desktop = ?name, created = hwnd.is_some(),
"force-composed-flip: overlay (re)created on input desktop");
}
// Leak `desk` while it's the thread desktop (closing the current thread desktop is UB).
}
cur_desktop = name;
}
// Re-assert topmost periodically (other windows on the secure desktop can push us down) and
// pump our message queue so the window stays responsive/composited.
if let Some(h) = hwnd {
let _ = SetWindowPos(
h,
Some(HWND_TOPMOST),
0,
0,
0,
0,
SWP_NOMOVE | SWP_NOSIZE | SWP_NOACTIVATE,
);
let mut msg = MSG::default();
while PeekMessageW(&mut msg, Some(h), 0, 0, PM_REMOVE).as_bool() {
let _ = TranslateMessage(&msg);
DispatchMessageW(&msg);
}
}
ticks = ticks.wrapping_add(1);
let _ = ticks;
std::thread::sleep(std::time::Duration::from_millis(200));
}
if let Some(h) = hwnd.take() {
let _ = DestroyWindow(h);
}
tracing::info!("force-composed-flip overlay stopped");
}
@@ -1,144 +0,0 @@
//! Input-desktop watcher (Windows) — the authoritative "normal vs secure desktop" signal for the
//! two-process secure-desktop design (design/archive/windows-secure-desktop.md).
//!
//! Windows switches the *input desktop* to "Winlogon" (the secure desktop) for UAC elevation, the
//! lock screen and the login screen, and back to "Default" for the normal session. WGC captures only
//! the normal desktop; DDA-as-SYSTEM captures the secure one. A dedicated thread polls the input
//! desktop's NAME (WTS session notifications miss UAC entirely, so the name is the reliable signal)
//! and publishes it as an atomic the capture mux + input path read.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use std::sync::atomic::{AtomicBool, AtomicU8, Ordering};
use std::sync::Arc;
use std::time::Duration;
use windows::Win32::Foundation::HANDLE;
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, GetUserObjectInformationW, OpenInputDesktop, DESKTOP_ACCESS_FLAGS,
DESKTOP_CONTROL_FLAGS, UOI_NAME,
};
/// The normal interactive desktop ("Default") — WGC capture applies.
pub const DESKTOP_NORMAL: u8 = 0;
/// The secure desktop ("Winlogon": UAC / lock / login) — DDA-as-SYSTEM capture applies.
pub const DESKTOP_SECURE: u8 = 1;
/// Polls the input-desktop name on its own thread and publishes [`DESKTOP_NORMAL`]/[`DESKTOP_SECURE`].
pub struct DesktopWatcher {
state: Arc<AtomicU8>,
stop: Arc<AtomicBool>,
}
impl DesktopWatcher {
pub fn start() -> Self {
// Compute the CURRENT desktop synchronously before returning, so the first reader (the capture
// mux) sees the real state immediately. Otherwise a session that begins already on the secure
// desktop (e.g. a reconnect to a locked box) would read DESKTOP_NORMAL for the first poll
// interval and relay one stale normal-desktop frame — the "flash of the login screen" bug.
// SAFETY: `is_secure_desktop` is this module's `unsafe fn` — unsafe only because it calls Win32
// desktop FFI (`OpenInputDesktop`/`GetUserObjectInformationW`/`CloseDesktop`), with no caller
// precondition; it opens, names, and closes the input-desktop handle internally and is safe to
// call from any thread (here, on the thread running `DesktopWatcher::start`).
let initial = if unsafe { is_secure_desktop() } {
DESKTOP_SECURE
} else {
DESKTOP_NORMAL
};
let state = Arc::new(AtomicU8::new(initial));
let stop = Arc::new(AtomicBool::new(false));
let s = state.clone();
let st = stop.clone();
let _ = std::thread::Builder::new()
.name("desktop-watch".into())
.spawn(move || {
// Debounce: only publish a change after the raw reading has been stable for several
// polls. The input desktop flaps Default↔Winlogon transiently during a lock/UAC
// transition; publishing every flap makes the capture mux thrash (rebuild storms).
const STABLE_POLLS: u32 = 4; // ~80ms
let mut published = initial;
let mut candidate = initial;
let mut stable = 0u32;
while !st.load(Ordering::Relaxed) {
// SAFETY: same as in `start` — `is_secure_desktop` is self-contained Win32 desktop
// FFI with no caller precondition, called here on the dedicated `desktop-watch`
// polling thread.
let v = if unsafe { is_secure_desktop() } {
DESKTOP_SECURE
} else {
DESKTOP_NORMAL
};
if v == candidate {
stable = stable.saturating_add(1);
} else {
candidate = v;
stable = 1;
}
if stable >= STABLE_POLLS && candidate != published {
s.store(candidate, Ordering::Release);
published = candidate;
tracing::info!(
desktop = if candidate == DESKTOP_SECURE {
"Winlogon(secure)"
} else {
"Default"
},
"input desktop changed (debounced)"
);
}
std::thread::sleep(Duration::from_millis(20));
}
});
DesktopWatcher { state, stop }
}
/// The shared atomic ([`DESKTOP_NORMAL`]/[`DESKTOP_SECURE`]) for the capture mux to read.
pub fn state(&self) -> Arc<AtomicU8> {
self.state.clone()
}
/// True when the secure (Winlogon) desktop is the input desktop right now.
pub fn is_secure(&self) -> bool {
self.state.load(Ordering::Acquire) == DESKTOP_SECURE
}
}
impl Drop for DesktopWatcher {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
}
}
/// True if the current input desktop is "Winlogon" (the secure desktop). Best-effort: if the desktop
/// can't be opened or named, report not-secure (the safe default — keep WGC/normal capture).
pub(crate) unsafe fn is_secure_desktop() -> bool {
let desk = match OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(DESKTOP_READOBJECTS),
) {
Ok(d) => d,
Err(_) => return false,
};
let mut buf = [0u16; 64];
let mut needed = 0u32;
let ok = GetUserObjectInformationW(
HANDLE(desk.0),
UOI_NAME,
Some(buf.as_mut_ptr() as *mut _),
(buf.len() * 2) as u32,
Some(&mut needed),
)
.is_ok();
let _ = CloseDesktop(desk);
if !ok {
return false;
}
let name = String::from_utf16_lossy(&buf);
name.trim_end_matches('\u{0}')
.eq_ignore_ascii_case("Winlogon")
}
/// `DESKTOP_READOBJECTS` access right (the windows crate exposes it as a typed flag; we need the raw
/// bit for `OpenInputDesktop`'s access mask).
const DESKTOP_READOBJECTS: u32 = 0x0001;
File diff suppressed because it is too large Load Diff
@@ -1,8 +1,8 @@
//! P2 direct frame push (kill DDA) — HOST side. The pf-vdisplay driver runs in a restricted WUDFHost //! P2 direct frame push (kill DDA) — HOST side. The pf-vdisplay driver's WUDFHost canNOT create named
//! token that canNOT create named kernel objects, so — exactly like the gamepad UMDF drivers //! kernel objects, so — exactly like the gamepad UMDF drivers (`inject/dualsense_windows.rs`) — the
//! (`inject/dualsense_windows.rs`) — the HOST (privileged) CREATES the shared header + frame-ready //! HOST (privileged) CREATES the shared header + frame-ready event + ring of keyed-mutex textures
//! event + ring of keyed-mutex textures (`Global\` names, permissive `D:(A;;GA;;;WD)` SDDL) on the //! (`Global\` names, scoped `D:(A;;GA;;;SY)(A;;GA;;;LS)` to SYSTEM + the driver's LocalService host —
//! discrete render GPU, and the driver only OPENS them and copies frames in. We then consume the ring //! see `shared_object_sa`) on the discrete render GPU, and the driver only OPENS them and copies frames in. We then consume the ring
//! straight into the zero-copy NVENC path — no DXGI Desktop Duplication, no `win32u` hook. Gated by //! straight into the zero-copy NVENC path — no DXGI Desktop Duplication, no `win32u` hook. Gated by
//! `PUNKTFUNK_IDD_PUSH`. Driver counterpart: `packaging/windows/drivers/pf-vdisplay/src/ //! `PUNKTFUNK_IDD_PUSH`. Driver counterpart: `packaging/windows/drivers/pf-vdisplay/src/
//! frame_transport.rs`. The shared `SharedHeader` layout, `MAGIC`/`VERSION`/`RING_LEN`, the //! frame_transport.rs`. The shared `SharedHeader` layout, `MAGIC`/`VERSION`/`RING_LEN`, the
@@ -236,13 +236,17 @@ pub struct IddPushCapturer {
// ownership to one thread at a time with NO concurrent access; we do not (and must not) claim `Sync`. // ownership to one thread at a time with NO concurrent access; we do not (and must not) claim `Sync`.
unsafe impl Send for IddPushCapturer {} unsafe impl Send for IddPushCapturer {}
/// Build a permissive (Everyone:GenericAll) `SECURITY_ATTRIBUTES` so the restricted WUDFHost driver /// Build a `SECURITY_ATTRIBUTES` granting GENERIC_ALL to **SYSTEM** (the host creates+publishes the
/// can OPEN the host-created objects — the same `D:(A;;GA;;;WD)` SDDL the gamepad shared section uses. /// shared event + texture ring) and **LocalService** (the account the pf_vdisplay WUDFHost runs under,
/// The returned `psd` backing must outlive `sa`; both are dropped when the process exits. /// which consumes them) — `D:(A;;GA;;;SY)(A;;GA;;;LS)`, the same scoping as the gamepad section. The
unsafe fn permissive_sa() -> Result<(SECURITY_ATTRIBUTES, PSECURITY_DESCRIPTOR)> { /// old SDDL granted **Everyone** (`WD`), which let any local user open the `Global\pfvd-*` objects and
/// read captured screen frames (security-review 2026-06-28 #5). Verified on the RTX box (2026-06-29):
/// the WUDFHost token is `S-1-5-19` (LocalService), SYSTEM integrity, zero restricted SIDs — so SY+LS
/// suffices for the driver and excludes normal user processes. `psd` must outlive `sa`.
unsafe fn shared_object_sa() -> Result<(SECURITY_ATTRIBUTES, PSECURITY_DESCRIPTOR)> {
let mut psd = PSECURITY_DESCRIPTOR::default(); let mut psd = PSECURITY_DESCRIPTOR::default();
ConvertStringSecurityDescriptorToSecurityDescriptorW( ConvertStringSecurityDescriptorToSecurityDescriptorW(
w!("D:(A;;GA;;;WD)"), w!("D:(A;;GA;;;SY)(A;;GA;;;LS)"),
SDDL_REVISION_1, SDDL_REVISION_1,
&mut psd, &mut psd,
None, None,
@@ -269,7 +273,7 @@ impl IddPushCapturer {
h: u32, h: u32,
format: DXGI_FORMAT, format: DXGI_FORMAT,
) -> Result<Vec<HostSlot>> { ) -> Result<Vec<HostSlot>> {
let (sa, _psd) = permissive_sa()?; let (sa, _psd) = shared_object_sa()?;
let mut slots = Vec::new(); let mut slots = Vec::new();
for k in 0..RING_LEN { for k in 0..RING_LEN {
let desc = D3D11_TEXTURE2D_DESC { let desc = D3D11_TEXTURE2D_DESC {
@@ -375,7 +379,7 @@ impl IddPushCapturer {
// SAFETY: one block over the whole ring setup; every operation in it is sound: // SAFETY: one block over the whole ring setup; every operation in it is sound:
// - `set_advanced_color`/`advanced_color_enabled` are `unsafe fn`s taking only a copy of the plain // - `set_advanced_color`/`advanced_color_enabled` are `unsafe fn`s taking only a copy of the plain
// `u32` target id; they read/flip CCD display config and return owned values, borrowing nothing. // `u32` target id; they read/flip CCD display config and return owned values, borrowing nothing.
// - `CreateDXGIFactory1`, `EnumAdapterByLuid`, `make_device`, `permissive_sa`, `CreateFileMappingW`, // - `CreateDXGIFactory1`, `EnumAdapterByLuid`, `make_device`, `shared_object_sa`, `CreateFileMappingW`,
// `MapViewOfFile`, `CreateEventW`, and `create_ring_slots` are all `?`-checked, so every returned // `MapViewOfFile`, `CreateEventW`, and `create_ring_slots` are all `?`-checked, so every returned
// interface/handle/view is non-error before use; `&sa`/`&adapter`/`&device`/the `&HSTRING` names // interface/handle/view is non-error before use; `&sa`/`&adapter`/`&device`/the `&HSTRING` names
// are live borrows that outlive each synchronous call, and `sa.lpSecurityDescriptor` stays valid // are live borrows that outlive each synchronous call, and `sa.lpSecurityDescriptor` stays valid
@@ -421,7 +425,7 @@ impl IddPushCapturer {
.context("EnumAdapterByLuid(render adapter) for IDD push")?; .context("EnumAdapterByLuid(render adapter) for IDD push")?;
let (device, context) = make_device(&adapter).context("make_device for IDD push")?; let (device, context) = make_device(&adapter).context("make_device for IDD push")?;
let (sa, _psd) = permissive_sa()?; let (sa, _psd) = shared_object_sa()?;
let bytes = std::mem::size_of::<SharedHeader>().max(64); let bytes = std::mem::size_of::<SharedHeader>().max(64);
// Header. // Header.
@@ -1,816 +0,0 @@
//! Windows.Graphics.Capture (WGC) capture backend — the HDR/animation-correct path.
//!
//! Why WGC over DXGI Desktop Duplication: DDA duplicates only the DWM-composed primary surface, so
//! HDR desktop animations the OS routes onto hardware overlay / independent-flip / MPO planes (Start
//! menu, Win11 Mica/acrylic, window resize) never enter the surface DDA reads — the stream shows a
//! frozen desktop ("broken HDR animations"). Engaging WGC capture pulls that content back through DWM
//! composition, so the surface WGC hands back contains the animations. WGC also has no
//! ACCESS_LOST-on-overlay-flip churn.
//!
//! It reuses the rest of the pipeline UNCHANGED: the frame's GPU texture (the OS already composited
//! the cursor into it — `IsCursorCaptureEnabled(true)`) goes through the same scRGB→BT.2020-PQ shader
//! ([`super::dxgi::HdrConverter`]) into a host-owned `R10G10B10A2` texture (HDR) or is copied into a
//! BGRA texture (SDR), which is handed to NVENC zero-copy (registered by pointer, encoded in place).
//! Shares the D3D11 device with NVENC via `FramePayload::D3d11`.
//!
//! Limitation: WGC cannot capture the secure desktop (lock / UAC / login) — the caller falls back to
//! the DDA backend ([`super::dxgi::DuplCapturer`]) for those (see capture.rs).
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::dxgi::{
find_output, hdr_shader_p010_enabled, make_device, nudge_cursor_onto, D3d11Frame, HdrConverter,
HdrP010Converter, VideoConverter, WinCaptureTarget,
};
use super::{CapturedFrame, Capturer, FramePayload, PixelFormat};
use anyhow::{bail, Context, Result};
use std::collections::VecDeque;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Condvar, Mutex};
use std::time::{Duration, Instant};
use windows::core::{IInspectable, Interface};
use windows::Foundation::{TimeSpan, TypedEventHandler};
use windows::Graphics::Capture::{
Direct3D11CaptureFrame, Direct3D11CaptureFramePool, GraphicsCaptureItem, GraphicsCaptureSession,
};
use windows::Graphics::DirectX::DirectXPixelFormat;
use windows::Win32::Foundation::{CloseHandle, HANDLE};
use windows::Win32::Graphics::Direct3D11::{
ID3D11Device, ID3D11DeviceContext, ID3D11RenderTargetView, ID3D11ShaderResourceView,
ID3D11Texture2D, D3D11_BIND_RENDER_TARGET, D3D11_BIND_SHADER_RESOURCE, D3D11_TEXTURE2D_DESC,
D3D11_USAGE_DEFAULT,
};
use windows::Win32::Graphics::Dxgi::Common::{
DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020, DXGI_FORMAT_R10G10B10A2_UNORM,
DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_SAMPLE_DESC,
};
use windows::Win32::Graphics::Dxgi::{IDXGIDevice, IDXGIOutput6};
use windows::Win32::Security::{ImpersonateLoggedOnUser, RevertToSelf};
use windows::Win32::System::RemoteDesktop::{WTSGetActiveConsoleSessionId, WTSQueryUserToken};
use windows::Win32::System::WinRT::Direct3D11::{
CreateDirect3D11DeviceFromDXGIDevice, IDirect3DDxgiInterfaceAccess,
};
use windows::Win32::System::WinRT::Graphics::Capture::IGraphicsCaptureItemInterop;
use windows::Win32::System::WinRT::{RoInitialize, RO_INIT_MULTITHREADED};
/// Output texture ring depth. The encode loop pipelines one frame deep (NVENC encodes frame N while
/// the capturer produces N+1), so two live textures suffice; three gives headroom against a slow
/// `lock_bitstream` and matches the WGC frame-pool depth.
// Sized for the deep encode pipeline (`PUNKTFUNK_ENCODE_DEPTH`, default 4, clamped ≤ 6): up to DEPTH
// frames are in flight in NVENC at once, so the HDR convert ring and the SDR held-frame set must each
// keep DEPTH(+headroom) live textures, and the WGC pool needs spare buffers beyond what we hold.
const OUT_RING: usize = 8;
/// SDR zero-copy: how many recent WGC frames to keep alive so NVENC can encode the pool texture in
/// place (no `CopyResource`). Each in-flight encode reads a distinct frame, so this must exceed the
/// pipeline depth; the oldest is released once `HELD_FRAMES` newer ones exist.
const HELD_FRAMES: usize = 8;
/// WGC frame-pool buffer count. Must exceed `HELD_FRAMES` so the compositor always has free buffers
/// to render into while we hold frames for in-place (zero-copy) SDR encode.
const WGC_POOL_BUFFERS: i32 = 10;
/// The host runs as SYSTEM (so the DDA secure-desktop path works), but WGC will NOT activate under
/// the SYSTEM account (`CreateForMonitor` → 0x80070424). Impersonate the interactive console user
/// for the WGC activation. Returns the user token (the caller reverts + closes it after activation)
/// or `None` (no active user, or the host already runs AS the user — WTSQueryUserToken then fails and
/// WGC works without impersonation). SYSTEM-only; harmless under a user-token host.
unsafe fn impersonate_active_user() -> Option<HANDLE> {
let session = WTSGetActiveConsoleSessionId();
if session == 0xFFFF_FFFF {
return None;
}
let mut token = HANDLE::default();
if WTSQueryUserToken(session, &mut token).is_ok() {
if ImpersonateLoggedOnUser(token).is_ok() {
return Some(token);
}
let _ = CloseHandle(token);
}
None
}
/// RAII: reverts the WGC-activation impersonation when it drops (covers every `?` early-return).
struct Deimpersonate(Option<HANDLE>);
impl Drop for Deimpersonate {
fn drop(&mut self) {
if let Some(tok) = self.0.take() {
// SAFETY: `RevertToSelf` takes no arguments and undoes the thread impersonation set during
// WGC activation; `tok` is the impersonation token `HANDLE` from `impersonate_active_user`,
// owned by this `Deimpersonate` and closed exactly once here (taken out of the `Option`, so
// no double-close). Both are FFI calls borrowing no Rust memory.
unsafe {
let _ = RevertToSelf();
let _ = CloseHandle(tok);
}
}
}
}
/// Signal from the free-threaded FrameArrived callback to the encode thread: a monotonically
/// increasing count of arrived frames + a condvar to wake `next_frame`. The encode thread tracks how
/// many it has consumed; `TryGetNextFrame` is called exactly `available - consumed` times so we never
/// hit the empty-pool ambiguity, and draining to the newest keeps latency at one frame.
struct WgcSignal {
available: AtomicU64,
mtx: Mutex<()>,
cv: Condvar,
}
pub struct WgcCapturer {
device: ID3D11Device,
context: ID3D11DeviceContext,
// WGC objects — kept alive for the session's lifetime.
pool: Direct3D11CaptureFramePool,
session: GraphicsCaptureSession,
_item: GraphicsCaptureItem,
_frame_arrived_token: i64,
signal: Arc<WgcSignal>,
consumed: u64,
width: u32,
height: u32,
timeout_ms: u64,
first_frame: bool,
hdr: bool,
/// The source display's static HDR mastering metadata (ST.2086 + content light level), read from
/// `IDXGIOutput6::GetDesc1` at open when the output is HDR. Forwarded to the encoder (in-band SEI)
/// and the client (0xCE) by the stream loop. `None` when SDR. (The helper relay path also encodes,
/// so this is what gives the secure/normal-desktop HDR stream its mastering SEI.)
hdr_meta: Option<punktfunk_core::quic::HdrMeta>,
hdr_conv: Option<HdrConverter>,
fp16_src: Option<ID3D11Texture2D>,
fp16_srv: Option<ID3D11ShaderResourceView>,
/// `PUNKTFUNK_HDR_SHADER_P010` path: emit P010 (BT.2020 PQ 10-bit limited range) DIRECTLY from our
/// own shader (`HdrP010Converter`) so NVENC takes native P010 and skips its SM-side RGB→YUV CSC.
/// Gated by [`hdr_shader_p010_enabled`] AND `self.hdr`; `None`/empty when off → the existing R10 +
/// VideoProcessor paths run unchanged. `p010_disabled` latches a runtime failure (e.g. a driver
/// that rejects the planar plane RTV) so we fall back to the R10 path and stop retrying.
hdr_p010_conv: Option<HdrP010Converter>,
p010_out: Vec<ID3D11Texture2D>,
p010_idx: usize,
p010_disabled: bool,
/// Ring of host-owned output textures (BGRA for SDR, R10G10B10A2 for HDR), rotated per processed
/// frame. A ring — not one texture — is required because the encode loop is PIPELINED: NVENC
/// encodes frame N (in place, registered by pointer) while this capturer produces frame N+1, so
/// N+1 must land in a DIFFERENT texture or it clobbers the in-flight encode. (`fp16_src` stays
/// single: it's only touched within the D3D11 immediate context, whose op ordering already
/// serializes the convert's read against the next copy's write — NVENC's async engine read is the
/// only consumer that escapes that ordering, and it reads the ring output, never `fp16_src`.)
out_ring: Vec<ID3D11Texture2D>,
ring_idx: usize,
/// Video-processor RGB→YUV converter (off the 3D engine where possible) + its NV12/P010 output
/// ring. Preferred path: the OS-composited capture (cursor already in it) is converted DIRECTLY to
/// NVENC's native YUV — no `CopyResource`, no cursor draw, and NVENC skips its internal RGB→YUV.
/// `None`/error → falls back to the legacy SDR-zero-copy / HDR-shader paths.
video_conv: Option<VideoConverter>,
yuv_out: Vec<ID3D11Texture2D>,
yuv_idx: usize,
yuv_is_hdr: bool,
vp_disabled: bool,
/// SDR zero-copy: the recent WGC frames we hand to NVENC in place. Held so the pool doesn't
/// recycle the texture mid-encode; the oldest is released once `HELD_FRAMES` newer ones exist.
held: VecDeque<Direct3D11CaptureFrame>,
/// Last presentable GPU texture + format, repeated when no new frame arrived (static desktop).
last_present: Option<(ID3D11Texture2D, PixelFormat)>,
/// Owns the SudoVDA keepalive once attached (after WGC is confirmed open) — dropping the capturer
/// then REMOVEs the virtual output. `None` between open and attach so a WGC-open failure leaves
/// the keepalive with the caller for the DDA fallback.
_keepalive: Option<Box<dyn Send>>,
}
// SAFETY: like `DuplCapturer`. `WgcCapturer` holds D3D11 (free-threaded device/context) plus WGC WinRT
// objects (`Direct3D11CaptureFramePool` etc., created free-threaded via `CreateFreeThreaded`). COM/WinRT
// reference counting is interlocked, and the capturer is owned + used by exactly one encode thread,
// moved to it once and never shared (no `Sync`), so transferring ownership across threads is sound. The
// free-threaded `FrameArrived` callback touches only the `Arc<WgcSignal>` (itself `Send + Sync`), not
// the capturer's COM fields.
unsafe impl Send for WgcCapturer {}
impl WgcCapturer {
/// Open WGC capture. Does NOT take the keepalive — the caller attaches it via
/// [`attach_keepalive`](Self::attach_keepalive) only after open succeeds, so a failure leaves the
/// keepalive with the caller to hand to the DDA fallback.
pub fn open(target: WinCaptureTarget, preferred: Option<(u32, u32, u32)>) -> Result<Self> {
// SAFETY: runs on the thread opening the WGC session. `RoInitialize` inits this thread's WinRT
// apartment (idempotent; result ignored). `impersonate_active_user()` and `find_output()` are
// this module's `unsafe fn`s whose contracts (call on the activating thread; pass a GDI name)
// are met, and the impersonation is reverted by `_deimp`'s Drop on every return path. Every
// COM/WinRT call thereafter operates on an object obtained + `?`-checked earlier in this same
// block on this single thread — the `IDXGIOutput1` from `find_output`, the device/context from
// `make_device`, the factory/interop/item/pool/session — and the `TypedEventHandler` closure
// captures an `Arc<WgcSignal>` (Send+Sync) by move. No raw pointers are dereferenced; borrowed
// locals outlive their synchronous calls.
unsafe {
// WGC is WinRT — the calling thread needs a COM/WinRT apartment for the GraphicsCaptureItem
// activation factory (RoGetActivationFactory). Initialize MTA; ignore "already initialized"
// / "changed mode" (another component on this thread may have init'd a compatible apartment).
let ro = RoInitialize(RO_INIT_MULTITHREADED);
// Impersonate the interactive user for the duration of WGC activation (host runs as
// SYSTEM; WGC won't activate under SYSTEM). Reverted by the guard's Drop on return. The
// WGC objects, once created, are accessed from the (SYSTEM) encode thread thereafter.
let imp = impersonate_active_user();
let _deimp = Deimpersonate(imp);
tracing::info!(ro_result = ?ro, impersonated = imp.is_some(), "WGC: RoInitialize(MTA)");
// The SudoVDA output appears a beat after the display is created — settle-retry like DDA.
let deadline = Instant::now() + Duration::from_millis(2000);
let (adapter, output) = loop {
if let Some(n) = crate::win_display::resolve_gdi_name(target.target_id) {
if let Ok(found) = find_output(&n) {
break found;
}
}
if let Ok(found) = find_output(&target.gdi_name) {
break found;
}
if Instant::now() >= deadline {
bail!(
"WGC: no DXGI output for SudoVDA target {} yet",
target.target_id
);
}
std::thread::sleep(Duration::from_millis(100));
};
let (device, context) = make_device(&adapter)?;
let od = output.GetDesc().context("output GetDesc")?;
let hmonitor = od.Monitor;
// HDR iff the output's colour space is BT.2020 PQ (G2084) — matches the DDA FP16 detection.
// From the same desc, read the source display's mastering metadata (ST.2086) when HDR.
let desc1 = output
.cast::<IDXGIOutput6>()
.ok()
.and_then(|o6| o6.GetDesc1().ok());
let hdr = desc1
.as_ref()
.map(|d1| d1.ColorSpace == DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020)
.unwrap_or(false);
let hdr_meta = if hdr {
desc1.as_ref().map(|d| {
crate::hdr::hdr_meta_from_display(
(d.RedPrimary[0], d.RedPrimary[1]),
(d.GreenPrimary[0], d.GreenPrimary[1]),
(d.BluePrimary[0], d.BluePrimary[1]),
(d.WhitePoint[0], d.WhitePoint[1]),
d.MaxLuminance,
d.MinLuminance,
0, // MaxCLL: GetDesc1 has no content light level (Apollo zeroes it)
0, // MaxFALL
)
})
} else {
None
};
// Wrap our D3D11 device as a WinRT IDirect3DDevice so the frame pool allocates on it (the
// pool textures land on our device → CopyResource + NVENC are same-device, no readback).
let dxgi_device: IDXGIDevice = device.cast().context("ID3D11Device as IDXGIDevice")?;
let inspectable: IInspectable = CreateDirect3D11DeviceFromDXGIDevice(&dxgi_device)
.context("CreateDirect3D11DeviceFromDXGIDevice")?;
let d3d_device: windows::Graphics::DirectX::Direct3D11::IDirect3DDevice = inspectable
.cast()
.context("IInspectable as IDirect3DDevice")?;
tracing::info!(hdr, "WGC: device ready, creating capture item");
// GraphicsCaptureItem for the monitor (the SudoVDA output enumerates as a normal monitor).
let interop: IGraphicsCaptureItemInterop =
windows::core::factory::<GraphicsCaptureItem, IGraphicsCaptureItemInterop>()
.context("GraphicsCaptureItem interop factory")?;
let item: GraphicsCaptureItem = interop
.CreateForMonitor(hmonitor)
.context("CreateForMonitor")?;
let size = item.Size().context("item Size")?;
let (width, height) = (size.Width.max(0) as u32, size.Height.max(0) as u32);
tracing::info!(
width,
height,
"WGC: capture item created, creating frame pool"
);
let pixel_format = if hdr {
DirectXPixelFormat::R16G16B16A16Float // scRGB FP16 — same surface DDA gives on HDR
} else {
DirectXPixelFormat::B8G8R8A8UIntNormalized
};
// Extra buffers: SDR zero-copy holds the last `HELD_FRAMES` frames (encoded in place), so
// the pool needs headroom beyond that for the producer to keep rendering at 240 Hz.
let pool = Direct3D11CaptureFramePool::CreateFreeThreaded(
&d3d_device,
pixel_format,
WGC_POOL_BUFFERS,
size,
)
.context("CreateFreeThreaded frame pool")?;
let signal = Arc::new(WgcSignal {
available: AtomicU64::new(0),
mtx: Mutex::new(()),
cv: Condvar::new(),
});
let sig = signal.clone();
let handler = TypedEventHandler::<Direct3D11CaptureFramePool, IInspectable>::new(
move |_pool, _arg| {
sig.available.fetch_add(1, Ordering::Release);
sig.cv.notify_one();
Ok(())
},
);
let token = pool.FrameArrived(&handler).context("FrameArrived")?;
tracing::info!("WGC: creating capture session");
let session = pool
.CreateCaptureSession(&item)
.context("CreateCaptureSession")?;
// OS composites the cursor into the frame (HDR-correct, no manual composite pass).
let _ = session.SetIsCursorCaptureEnabled(true);
// Drop the yellow capture border (best-effort — older builds reject it).
let _ = session.SetIsBorderRequired(false);
// Lift the 60 Hz cap: allow up to the client's refresh (Win11 24H2+; below that this is a
// no-op and WGC caps ~60). 100 ns ticks per frame.
let refresh = preferred
.map(|(_, _, hz)| hz)
.filter(|&hz| hz > 0)
.unwrap_or(60);
let ticks = (10_000_000i64 / refresh.max(1) as i64).max(1);
let _ = session.SetMinUpdateInterval(TimeSpan { Duration: ticks });
tracing::info!("WGC: StartCapture");
session.StartCapture().context("StartCapture")?;
// WGC fires FrameArrived on CHANGE; a static desktop may never deliver the first frame
// (→ black, then the next_frame deadline ends the session). Nudge the cursor onto the
// output to force the first composition change, exactly like the DDA path does.
nudge_cursor_onto(&output);
let timeout_ms = (2000 / refresh.max(1) as u64).max(8);
tracing::info!(
width,
height,
hdr,
refresh,
"WGC capture started ({})",
if hdr {
"HDR FP16→BT.2020 PQ"
} else {
"SDR BGRA"
}
);
Ok(Self {
device,
context,
pool,
session,
_item: item,
_frame_arrived_token: token,
signal,
consumed: 0,
width,
height,
timeout_ms,
first_frame: true,
hdr,
hdr_meta,
hdr_conv: None,
fp16_src: None,
fp16_srv: None,
hdr_p010_conv: None,
p010_out: Vec::new(),
p010_idx: 0,
p010_disabled: false,
out_ring: Vec::new(),
ring_idx: 0,
video_conv: None,
yuv_out: Vec::new(),
yuv_idx: 0,
yuv_is_hdr: false,
vp_disabled: std::env::var_os("PUNKTFUNK_NO_VIDEO_PROCESSOR").is_some(),
held: VecDeque::new(),
last_present: None,
_keepalive: None,
})
}
}
/// Take ownership of the SudoVDA keepalive once the WGC session is confirmed open.
pub fn attach_keepalive(&mut self, keepalive: Box<dyn Send>) {
self._keepalive = Some(keepalive);
}
/// Block until a new frame arrives (cv), then drain `TryGetNextFrame` to the NEWEST queued frame
/// (skip stale → one-frame latency). Returns `None` on timeout (no new frame → caller repeats).
fn wait_and_drain(&mut self) -> Option<Direct3D11CaptureFrame> {
let wait_ms = if self.first_frame {
2000
} else {
self.timeout_ms
};
{
let mut g = self.signal.mtx.lock().unwrap();
while self.signal.available.load(Ordering::Acquire) <= self.consumed {
let (ng, res) = self
.signal
.cv
.wait_timeout(g, Duration::from_millis(wait_ms))
.unwrap();
g = ng;
if res.timed_out() {
return None;
}
}
}
let target = self.signal.available.load(Ordering::Acquire);
let mut last = None;
while self.consumed < target {
if let Ok(f) = self.pool.TryGetNextFrame() {
last = Some(f);
}
self.consumed += 1;
}
last
}
unsafe fn ensure_fp16_src(&mut self) -> Result<()> {
if self.fp16_src.is_some() {
return Ok(());
}
let desc = tex_desc(
self.width,
self.height,
DXGI_FORMAT_R16G16B16A16_FLOAT,
(D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
);
let mut t = None;
self.device
.CreateTexture2D(&desc, None, Some(&mut t))
.context("CreateTexture2D(wgc fp16 src)")?;
let t = t.context("fp16 src")?;
let mut srv = None;
self.device
.CreateShaderResourceView(&t, None, Some(&mut srv))?;
self.fp16_srv = Some(srv.context("fp16 srv")?);
self.fp16_src = Some(t);
Ok(())
}
/// Lazily allocate the HDR output texture ring (R10G10B10A2, the convert pass's render target →
/// NVENC input), `RENDER_TARGET`-bindable. SDR is zero-copy (encodes the WGC pool texture in
/// place) and uses no ring.
unsafe fn ensure_out_ring(
&mut self,
format: windows::Win32::Graphics::Dxgi::Common::DXGI_FORMAT,
) -> Result<()> {
if !self.out_ring.is_empty() {
return Ok(());
}
let desc = tex_desc(
self.width,
self.height,
format,
D3D11_BIND_RENDER_TARGET.0 as u32,
);
for _ in 0..OUT_RING {
let mut t = None;
self.device
.CreateTexture2D(&desc, None, Some(&mut t))
.context("CreateTexture2D(wgc out ring)")?;
self.out_ring.push(t.context("wgc out ring tex")?);
}
Ok(())
}
/// Convert `input` (the OS-composited WGC pool texture: BGRA or scRGB FP16) → NVENC's native YUV
/// (NV12 / P010) on the video processor. Returns the YUV texture (from a ring so consecutive
/// encodes don't collide), or `None` to fall back to the legacy RGB paths.
unsafe fn convert_to_yuv(
&mut self,
input: &ID3D11Texture2D,
hdr: bool,
) -> Option<ID3D11Texture2D> {
if self.vp_disabled {
return None;
}
if self.video_conv.is_none() || self.yuv_out.is_empty() || self.yuv_is_hdr != hdr {
self.video_conv = None;
self.yuv_out.clear();
self.yuv_idx = 0;
let vc = match VideoConverter::new(
&self.device,
&self.context,
self.width,
self.height,
hdr,
) {
Ok(vc) => vc,
Err(e) => {
tracing::warn!(error = %format!("{e:#}"),
"WGC: video processor unavailable — falling back to RGB path");
self.vp_disabled = true;
return None;
}
};
let fmt = if hdr {
windows::Win32::Graphics::Dxgi::Common::DXGI_FORMAT_P010
} else {
windows::Win32::Graphics::Dxgi::Common::DXGI_FORMAT_NV12
};
let desc = tex_desc(
self.width,
self.height,
fmt,
D3D11_BIND_RENDER_TARGET.0 as u32,
);
for _ in 0..OUT_RING {
let mut t = None;
if self
.device
.CreateTexture2D(&desc, None, Some(&mut t))
.is_err()
{
tracing::warn!("WGC: CreateTexture2D(YUV) failed — falling back to RGB path");
self.vp_disabled = true;
self.yuv_out.clear();
return None;
}
let Some(tex) = t else {
self.vp_disabled = true;
self.yuv_out.clear();
return None;
};
self.yuv_out.push(tex);
}
self.video_conv = Some(vc);
self.yuv_is_hdr = hdr;
tracing::info!(
hdr,
"WGC: video-processor YUV path active ({})",
if hdr { "P010" } else { "NV12" }
);
}
let slot = self.yuv_idx;
self.yuv_idx = (self.yuv_idx + 1) % self.yuv_out.len();
let out = self.yuv_out[slot].clone();
if let Err(e) = self.video_conv.as_ref()?.convert(input, &out) {
tracing::warn!(error = %format!("{e:#}"),
"WGC: VideoProcessorBlt failed — falling back to RGB path");
self.vp_disabled = true;
self.video_conv = None;
self.yuv_out.clear();
return None;
}
Some(out)
}
/// `PUNKTFUNK_HDR_SHADER_P010` path: convert the OS-composited FP16 scRGB capture DIRECTLY to a
/// host-owned P010 texture (BT.2020 PQ, 10-bit limited range) via [`HdrP010Converter`] — two
/// shader passes writing the P010 planes. NVENC then takes native P010 and skips its internal
/// RGB→YUV CSC. Returns the next ring slot's P010 texture, or `Err` if the converter / a planar
/// plane RTV fails (the caller latches `p010_disabled` and falls back to the R10 path).
unsafe fn hdr_to_p010(&mut self, src: &ID3D11Texture2D) -> Result<ID3D11Texture2D> {
let slot = self.p010_idx;
// Lazily allocate the FP16 source (shared with the R10 path) + the P010 output ring.
self.ensure_fp16_src()?;
let fp16 = self.fp16_src.clone().context("fp16 src")?;
self.context.CopyResource(&fp16, src);
if self.p010_out.is_empty() {
let desc = tex_desc(
self.width,
self.height,
windows::Win32::Graphics::Dxgi::Common::DXGI_FORMAT_P010,
D3D11_BIND_RENDER_TARGET.0 as u32,
);
for _ in 0..OUT_RING {
let mut t = None;
self.device
.CreateTexture2D(&desc, None, Some(&mut t))
.context("CreateTexture2D(wgc p010 ring)")?;
self.p010_out.push(t.context("wgc p010 ring tex")?);
}
}
self.p010_idx = (self.p010_idx + 1) % self.p010_out.len();
let out = self.p010_out[slot].clone();
if self.hdr_p010_conv.is_none() {
self.hdr_p010_conv = Some(HdrP010Converter::new(&self.device)?);
}
let srv = self.fp16_srv.clone().context("fp16 srv")?;
self.hdr_p010_conv.as_ref().unwrap().convert(
&self.device,
&self.context,
&srv,
&out,
self.width,
self.height,
)?;
Ok(out)
}
fn process_frame(&mut self, frame: Direct3D11CaptureFrame) -> Result<CapturedFrame> {
// SAFETY: runs on the capturer's single owning thread. `frame` is a live
// `Direct3D11CaptureFrame` from `self.pool`; `frame.Surface().cast::<IDirect3DDxgiInterfaceAccess
// >().GetInterface()` yields the frame's backing `ID3D11Texture2D`, which belongs to
// `self.device` (the pool was created on it via `CreateDirect3D11DeviceFromDXGIDevice`). Every
// helper called here — `hdr_to_p010`, `convert_to_yuv`, `ensure_fp16_src`, `ensure_out_ring`,
// `HdrConverter::convert`, `CopyResource`, `CreateRenderTargetView` — operates on
// `self.device`/`self.context` and that same-device texture, so all resources share one device.
// The frame is held in `self.held` until its async GPU read completes for the zero-copy paths.
// Single-threaded immediate-context use; borrowed textures/SRVs/RTVs outlive each synchronous call.
unsafe {
let surface = frame.Surface().context("frame Surface")?;
let access: IDirect3DDxgiInterfaceAccess = surface
.cast()
.context("surface as IDirect3DDxgiInterfaceAccess")?;
let src: ID3D11Texture2D = access
.GetInterface()
.context("GetInterface ID3D11Texture2D")?;
// GATED P010-shader path (`PUNKTFUNK_HDR_SHADER_P010`): for HDR, emit P010 (BT.2020 PQ
// 10-bit limited range) DIRECTLY from our shader so NVENC takes native P010 and skips its
// SM-side RGB→YUV CSC. Runs BEFORE the R10 + VideoProcessor path. A converter/plane-RTV
// failure latches `p010_disabled` → we fall through to the unchanged R10 path for the rest
// of the session. Default OFF → none of this executes and behaviour is byte-for-byte as
// today.
if self.hdr && !self.p010_disabled && hdr_shader_p010_enabled() {
match self.hdr_to_p010(&src) {
Ok(p010) => {
// The P010 output is host-owned (the ring), and the FP16 CopyResource read
// `src` synchronously on the immediate context before the shader passes — so we
// do NOT need to hold `frame` past here (unlike the SDR/R10 in-place paths).
// Dropping it returns the pool buffer to WGC immediately.
drop(frame);
self.last_present = Some((p010.clone(), PixelFormat::P010));
return Ok(self.d3d11_frame(p010, PixelFormat::P010));
}
Err(e) => {
tracing::warn!(error = %format!("{e:#}"),
"WGC: HDR P010 shader path failed — disabling it, falling back to R10");
self.p010_disabled = true;
self.hdr_p010_conv = None;
self.p010_out.clear();
}
}
}
// Preferred path: convert the OS-composited capture (cursor already in it) DIRECTLY to
// NVENC's native YUV on the video processor — no CopyResource, no cursor draw, and NVENC
// skips its internal RGB→YUV (the contended 3D step). WGC's multi-buffer pool + held set
// means reading the pool texture directly does NOT serialize (unlike DDA's single-frame
// model). The frame is held until the async Blt finishes. (HDR: the video processor can't
// ingest FP16 scRGB, so the Blt fails and we fall back to the R10 path below; the
// `PUNKTFUNK_HDR_SHADER_P010` branch above is the off-the-SM HDR path.)
if let Some(yuv) = self.convert_to_yuv(&src, self.hdr) {
let fmt = if self.hdr {
PixelFormat::P010
} else {
PixelFormat::Nv12
};
self.last_present = Some((yuv.clone(), fmt));
let out = self.d3d11_frame(yuv, fmt);
self.held.push_back(frame);
while self.held.len() > HELD_FRAMES {
self.held.pop_front();
}
return Ok(out);
}
// --- fallback (video processor unavailable) ---
if self.hdr {
// Next ring slot — the in-flight encode reads the slot we handed out last time, so
// this capture must land in a different one (see `out_ring`).
let slot = self.ring_idx;
self.ring_idx = (self.ring_idx + 1) % OUT_RING;
// FP16 (cursor already composited by the OS) → BT.2020 PQ 10-bit for NVENC.
self.ensure_fp16_src()?;
let fp16 = self.fp16_src.clone().context("fp16 src")?;
self.context.CopyResource(&fp16, &src);
self.ensure_out_ring(DXGI_FORMAT_R10G10B10A2_UNORM)?;
let out = self.out_ring[slot].clone();
if self.hdr_conv.is_none() {
self.hdr_conv = Some(HdrConverter::new(&self.device)?);
}
let srv = self.fp16_srv.clone().context("fp16 srv")?;
let mut rtv: Option<ID3D11RenderTargetView> = None;
self.device
.CreateRenderTargetView(&out, None, Some(&mut rtv))?;
let rtv = rtv.context("hdr10 rtv")?;
self.hdr_conv.as_ref().unwrap().convert(
&self.context,
&srv,
&rtv,
self.width,
self.height,
);
self.last_present = Some((out.clone(), PixelFormat::Rgb10a2));
Ok(self.d3d11_frame(out, PixelFormat::Rgb10a2))
} else {
// SDR ZERO-COPY: hand NVENC the WGC pool texture DIRECTLY — no `CopyResource`. The
// per-frame copy otherwise queues on the graphics engine behind a GPU-saturating game
// and stalls `lock_bitstream` ~20 ms (NVENC sits idle waiting for its input). Encoding
// the pool texture in place removes that graphics-queue dependency (Apollo's model).
// We must keep the frame alive until its async encode finishes, so retain the last
// `HELD_FRAMES`; the pool has spare buffers so the producer never starves.
self.last_present = Some((src.clone(), PixelFormat::Bgra));
let out = self.d3d11_frame(src, PixelFormat::Bgra);
self.held.push_back(frame);
while self.held.len() > HELD_FRAMES {
self.held.pop_front();
}
Ok(out)
}
}
}
fn d3d11_frame(&self, texture: ID3D11Texture2D, format: PixelFormat) -> CapturedFrame {
CapturedFrame {
width: self.width,
height: self.height,
pts_ns: now_ns(),
format,
payload: FramePayload::D3d11(D3d11Frame {
texture,
device: self.device.clone(),
}),
}
}
}
impl Capturer for WgcCapturer {
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
self.hdr_meta
}
fn next_frame(&mut self) -> Result<CapturedFrame> {
let overall = Instant::now() + Duration::from_secs(20);
loop {
if let Some(frame) = self.wait_and_drain() {
self.first_frame = false;
return self.process_frame(frame);
}
// No new frame within the wait — repeat the last presented frame (static desktop).
if let Some((tex, fmt)) = &self.last_present {
return Ok(self.d3d11_frame(tex.clone(), *fmt));
}
if Instant::now() > overall {
bail!("no WGC frame within 20s (SudoVDA monitor not lit / no capture access?)");
}
}
}
fn try_latest(&mut self) -> Result<Option<CapturedFrame>> {
let target = self.signal.available.load(Ordering::Acquire);
if target <= self.consumed {
return Ok(None);
}
let mut last = None;
while self.consumed < target {
if let Ok(f) = self.pool.TryGetNextFrame() {
last = Some(f);
}
self.consumed += 1;
}
match last {
Some(frame) => self.process_frame(frame).map(Some),
None => Ok(None),
}
}
// set_active: the trait default (no-op) is correct — WGC keeps its session running across the
// active/idle gate (cheap; the frame pool just recycles), like the DDA duplication.
}
impl Drop for WgcCapturer {
fn drop(&mut self) {
let _ = self.session.Close();
let _ = self.pool.Close();
// _keepalive drops after, REMOVEing the SudoVDA monitor.
}
}
fn tex_desc(
width: u32,
height: u32,
format: windows::Win32::Graphics::Dxgi::Common::DXGI_FORMAT,
bind: u32,
) -> D3D11_TEXTURE2D_DESC {
D3D11_TEXTURE2D_DESC {
Width: width,
Height: height,
MipLevels: 1,
ArraySize: 1,
Format: format,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: bind,
CPUAccessFlags: 0,
MiscFlags: 0,
}
}
fn now_ns() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as u64)
.unwrap_or(0)
}
@@ -1,484 +0,0 @@
//! Host-side WGC helper relay (Windows two-process secure-desktop design,
//! design/archive/windows-secure-desktop.md — step 4).
//!
//! WGC won't activate under the SYSTEM account, so the SYSTEM host can't capture the normal desktop
//! itself. Instead it spawns `punktfunk-host wgc-helper` in the **interactive user session** (so WGC works)
//! via `CreateProcessAsUserW`, with the helper's **stdout** redirected to an anonymous pipe the host
//! reads. The helper ships framed Annex-B access units; this module parses them back into AUs the
//! host relays onto the live QUIC session (same `EncodedFrame` flow, just sourced over a pipe instead
//! of a local encoder). A second pipe carries a tiny **control** channel to the helper (stdin: force
//! keyframe), and the helper's **stderr** is forwarded line-by-line into host tracing so its logs are
//! visible from the SYSTEM host's console.
//!
//! Wire framing (must match `wgc_helper::write_au`): per AU
//! `[u32 magic "PFAU" LE][u32 len LE][u64 pts_ns LE][u8 keyframe][len bytes data]`.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use crate::capture::dxgi::WinCaptureTarget;
use anyhow::{bail, Context, Result};
use std::io::{BufRead, BufReader, Read};
use std::sync::mpsc::{Receiver, SyncSender};
use std::sync::Mutex;
use windows::core::PWSTR;
use windows::Win32::Foundation::SetHandleInformation;
use windows::Win32::Foundation::{CloseHandle, HANDLE};
use windows::Win32::Foundation::{HANDLE_FLAGS, HANDLE_FLAG_INHERIT};
use windows::Win32::Security::{
DuplicateTokenEx, SecurityImpersonation, TokenPrimary, SECURITY_ATTRIBUTES, TOKEN_ALL_ACCESS,
};
use windows::Win32::System::Environment::{CreateEnvironmentBlock, DestroyEnvironmentBlock};
use windows::Win32::System::Pipes::CreatePipe;
use windows::Win32::System::RemoteDesktop::{WTSGetActiveConsoleSessionId, WTSQueryUserToken};
use windows::Win32::System::Threading::{
CreateProcessAsUserW, TerminateProcess, CREATE_NO_WINDOW, CREATE_UNICODE_ENVIRONMENT,
PROCESS_INFORMATION, STARTF_USESTDHANDLES, STARTUPINFOW,
};
/// Must match [`crate::wgc_helper`]'s `AU_MAGIC` ("PFAU").
const AU_MAGIC: u32 = 0x5046_4155;
/// One access unit relayed from the helper, in the helper's (= the host's, same machine) monotonic
/// clock — `pts_ns` is directly comparable to the host's `now_ns()`.
pub struct RelayAu {
pub data: Vec<u8>,
pub pts_ns: u64,
pub keyframe: bool,
}
/// A running USER-session WGC helper whose AUs the SYSTEM host relays. Drop kills the child + closes
/// the pipes; the reader threads then end on the broken pipe.
pub struct HelperRelay {
proc: HANDLE,
thread: HANDLE,
/// Host write end of the helper's stdin — control commands (force keyframe). Mutex so the relay
/// can be shared while the encode thread requests keyframes.
stdin_w: Mutex<HANDLE>,
/// Parsed AUs from the helper's stdout reader thread.
rx: Receiver<RelayAu>,
}
// SAFETY: every field is itself `Send`: the `proc`/`thread` `HANDLE`s are process-global kernel
// handle values (plain integers valid from any thread, owned for the relay's lifetime and closed once
// on Drop), `stdin_w` is a `Mutex<HANDLE>`, and `rx` is an mpsc `Receiver<RelayAu>` (which is `Send`).
// The relay is moved to one thread and owned there, so transferring it across threads is sound.
unsafe impl Send for HelperRelay {}
// NOTE: `HelperRelay` is deliberately NOT `Sync`. Its `rx: Receiver<RelayAu>` is `!Sync` (std mpsc
// is single-consumer), and the relay is only ever a single-owner local in the punktfunk1 two-process
// mux loop — never shared by `&` across threads — so `Sync` is neither sound nor needed. (A prior
// `unsafe impl Sync` here asserted more than the fields support; removed.)
/// Control byte on the helper's stdin: force the next encoded frame to be an IDR (client decode
/// recovery). Mirrors `enc.request_keyframe()` in the single-process path.
const CTL_KEYFRAME: u8 = 0x01;
impl HelperRelay {
/// Spawn the helper in the interactive user session and start relaying its AUs. `target` is the
/// SudoVDA output the host already created (captured by GDI name only — the helper never touches
/// display topology). `(w, h, hz)` is the negotiated mode; `bitrate_kbps` the negotiated bitrate.
pub fn spawn(
target: &WinCaptureTarget,
mode: (u32, u32, u32),
bitrate_kbps: u32,
bit_depth: u8,
) -> Result<HelperRelay> {
let exe = std::env::current_exe().context("current_exe for helper spawn")?;
let exe = exe.to_string_lossy().into_owned();
let (w, h, hz) = mode;
// CreateProcessAsUserW takes a single mutable command line (argv[0] = exe).
let cmdline = format!(
"\"{exe}\" wgc-helper --gdi \"{}\" --target-id {} --mode {w}x{h}x{hz} --bitrate {bitrate_kbps} --bit-depth {bit_depth}",
target.gdi_name, target.target_id
);
tracing::info!(cmd = %cmdline, "spawning WGC helper in user session");
// SAFETY: `spawn_inner` is an `unsafe fn` only because it drives raw Win32 token/pipe/process
// FFI; it imposes no caller-side memory precondition beyond valid arguments. `cmdline` is a live
// `&str` borrowed for the synchronous call and `(w, h, hz)` are plain `u32`s. It validates its
// own runtime requirements (active console session, SYSTEM token) and returns `Err` otherwise.
unsafe { spawn_inner(&cmdline, w, h, hz) }
}
/// Receive the next relayed AU. Distinguishes a `Timeout` (helper slow/stalled — keep waiting)
/// from `Disconnected` (helper exited → its stdout closed → reader thread ended → channel
/// dropped), which returns *immediately* and means the relay must stop, not spin.
pub fn recv_timeout(
&self,
dur: std::time::Duration,
) -> Result<RelayAu, std::sync::mpsc::RecvTimeoutError> {
self.rx.recv_timeout(dur)
}
/// Non-blocking receive — used to drain stale buffered AUs (encoded while the secure desktop was
/// the live source) before resuming the relay. `Ok` while AUs remain, `Err` once empty.
pub fn try_recv(&self) -> Result<RelayAu, std::sync::mpsc::TryRecvError> {
self.rx.try_recv()
}
/// Ask the helper's encoder for an IDR on the next frame (client decode recovery). Best-effort:
/// a write failure means the helper is gone — the caller's recv loop will see the disconnect.
pub fn request_keyframe(&self) {
let h = self.stdin_w.lock().unwrap();
let mut written = 0u32;
// SAFETY: `*h` is the host's write end of the helper's stdin pipe — a live `HANDLE` owned by
// this `HelperRelay` (held under the `stdin_w` Mutex, locked here), closed only in Drop.
// `WriteFile` reads the 1-byte `&[CTL_KEYFRAME]` buffer and writes the byte count into
// `written`; both are live locals that outlive the synchronous call. A failure (helper gone) is
// discarded as documented.
unsafe {
let _ = windows::Win32::Storage::FileSystem::WriteFile(
*h,
Some(&[CTL_KEYFRAME]),
Some(&mut written),
None,
);
}
}
}
impl Drop for HelperRelay {
fn drop(&mut self) {
// SAFETY: `self.proc`/`self.thread` are the child process/thread `HANDLE`s from
// `CreateProcessAsUserW`, and `stdin_w` is the host's pipe write end — all owned by this
// `HelperRelay` and closed exactly once here in Drop (no double-close). `TerminateProcess` and
// the three `CloseHandle`s are FFI calls taking those handles by value, borrowing no Rust memory.
unsafe {
// Terminate the child first so its WGC capture + NVENC session tear down, then close our
// handles (the reader threads end on the resulting broken pipe).
let _ = TerminateProcess(self.proc, 1);
let _ = CloseHandle(*self.stdin_w.lock().unwrap());
let _ = CloseHandle(self.proc);
let _ = CloseHandle(self.thread);
}
tracing::info!("WGC helper relay torn down");
}
}
/// Inheritable anonymous pipe (read, write). The caller marks whichever end the host keeps as
/// non-inheritable so the child only inherits its own end.
unsafe fn make_pipe() -> Result<(HANDLE, HANDLE)> {
let mut read = HANDLE::default();
let mut write = HANDLE::default();
let sa = SECURITY_ATTRIBUTES {
nLength: std::mem::size_of::<SECURITY_ATTRIBUTES>() as u32,
lpSecurityDescriptor: std::ptr::null_mut(),
bInheritHandle: true.into(),
};
CreatePipe(&mut read, &mut write, Some(&sa), 0).context("CreatePipe")?;
Ok((read, write))
}
/// Mark a handle non-inheritable (the host keeps it; the child must not get a copy).
unsafe fn no_inherit(h: HANDLE) {
let _ = SetHandleInformation(h, HANDLE_FLAG_INHERIT.0, HANDLE_FLAGS(0));
}
/// Build a child environment block: the target session's block (so DLL/PATH/SystemRoot resolve) with
/// this process's `PUNKTFUNK_*` vars overlaid, so the child runs with the SAME settings this process
/// has (`PUNKTFUNK_ENCODER=nvenc`, `PUNKTFUNK_ZEROCOPY`, …) instead of the target shell's. Returns a
/// UTF-16, double-null-terminated block suitable for `CREATE_UNICODE_ENVIRONMENT`. Shared by the WGC
/// helper spawn (here) and the Windows service launching the host into the active session.
pub(crate) unsafe fn merged_env_block(user_block: *const u16) -> Vec<u16> {
// Parse the user block ("VAR=VALUE\0" … "\0") into entries.
let mut entries: Vec<String> = Vec::new();
if !user_block.is_null() {
let mut p = user_block;
loop {
let mut len = 0isize;
while *p.offset(len) != 0 {
len += 1;
}
if len == 0 {
break; // the trailing empty string = end of block
}
let slice = std::slice::from_raw_parts(p, len as usize);
entries.push(String::from_utf16_lossy(slice));
p = p.offset(len + 1);
}
}
// Overlay "our" settings — PUNKTFUNK_* and RUST_LOG — dropping whatever the target block had.
let is_ours = |k: &str| k.starts_with("PUNKTFUNK_") || k == "RUST_LOG";
entries.retain(|e| !is_ours(e.split('=').next().unwrap_or("")));
for (k, v) in std::env::vars().filter(|(k, _)| is_ours(k)) {
entries.push(format!("{k}={v}"));
}
// Serialize back to a UTF-16 double-null-terminated block.
let mut block: Vec<u16> = Vec::new();
for e in entries {
block.extend(e.encode_utf16());
block.push(0);
}
block.push(0);
block
}
unsafe fn spawn_inner(cmdline: &str, w: u32, h: u32, hz: u32) -> Result<HelperRelay> {
// The user token of the active console session (requires the host to be SYSTEM).
let session = WTSGetActiveConsoleSessionId();
if session == 0xFFFF_FFFF {
bail!("no active console session (WTSGetActiveConsoleSessionId)");
}
let mut user_token = HANDLE::default();
WTSQueryUserToken(session, &mut user_token)
.context("WTSQueryUserToken (host must run as SYSTEM)")?;
// A primary token for CreateProcessAsUserW.
let mut primary = HANDLE::default();
let dup = DuplicateTokenEx(
user_token,
TOKEN_ALL_ACCESS,
None,
SecurityImpersonation,
TokenPrimary,
&mut primary,
);
let _ = CloseHandle(user_token);
dup.context("DuplicateTokenEx(TokenPrimary)")?;
// The user's environment block (PATH, USERPROFILE, SystemRoot → DLL resolution), MERGED with the
// host's PUNKTFUNK_* vars. CreateProcessAsUserW would otherwise give the helper the *user's* env
// only, dropping PUNKTFUNK_ENCODER=nvenc / PUNKTFUNK_ZEROCOPY/… that the host runs with — so the
// helper would fall back to the software (H.264-only) encoder. We parse the user block, strip any
// PUNKTFUNK_* it has, append the host's, and pass the merged block.
let mut env_block: *mut core::ffi::c_void = std::ptr::null_mut();
let _ = CreateEnvironmentBlock(&mut env_block, Some(primary), false);
let merged_env = merged_env_block(env_block as *const u16);
if !env_block.is_null() {
let _ = DestroyEnvironmentBlock(env_block);
}
// Three pipes: stdout (helper→host AUs), stdin (host→helper control), stderr (helper→host logs).
let (out_r, out_w) = make_pipe().context("stdout pipe")?;
let (in_r, in_w) = make_pipe().context("stdin pipe")?;
let (err_r, err_w) = make_pipe().context("stderr pipe")?;
// The host keeps out_r / in_w / err_r — none inheritable; the child inherits out_w/in_r/err_w.
no_inherit(out_r);
no_inherit(in_w);
no_inherit(err_r);
let mut si = STARTUPINFOW {
cb: std::mem::size_of::<STARTUPINFOW>() as u32,
dwFlags: STARTF_USESTDHANDLES,
hStdInput: in_r,
hStdOutput: out_w,
hStdError: err_w,
..Default::default()
};
// WGC needs the interactive desktop.
let mut desktop: Vec<u16> = "winsta0\\default\0".encode_utf16().collect();
si.lpDesktop = PWSTR(desktop.as_mut_ptr());
let mut cmd: Vec<u16> = cmdline.encode_utf16().chain(std::iter::once(0)).collect();
let mut pi = PROCESS_INFORMATION::default();
let created = CreateProcessAsUserW(
Some(primary),
None,
Some(PWSTR(cmd.as_mut_ptr())),
None,
None,
true, // inherit handles (the child's std ends)
CREATE_UNICODE_ENVIRONMENT | CREATE_NO_WINDOW,
Some(merged_env.as_ptr() as *const core::ffi::c_void),
None,
&si,
&mut pi,
);
// Clean up regardless of outcome: the child now owns its inherited ends; close our copies.
let _ = CloseHandle(out_w);
let _ = CloseHandle(in_r);
let _ = CloseHandle(err_w);
let _ = CloseHandle(primary);
if let Err(e) = created {
let _ = CloseHandle(out_r);
let _ = CloseHandle(in_w);
let _ = CloseHandle(err_r);
return Err(e).context("CreateProcessAsUserW(wgc-helper)");
}
tracing::info!(pid = pi.dwProcessId, mode = %format!("{w}x{h}@{hz}"), "WGC helper spawned");
// The helper does the WGC capture + NVENC encode, but it runs under the user's UAC-FILTERED token
// (no SE_INC_BASE_PRIORITY), so it can't raise its OWN GPU scheduling-priority class — under a
// GPU-saturating game NVENC then gets starved (the "240→40 fps in-game collapse"). The SYSTEM host
// holds the privilege, so stamp the HIGH GPU priority class onto the child here, right after spawn
// (the process-level class applies to the GPU contexts the helper creates afterwards).
crate::capture::dxgi::set_child_gpu_priority_class(pi.hProcess);
// stderr → host tracing, line by line.
let err_handle = HandleReader(err_r);
std::thread::Builder::new()
.name("wgc-helper-log".into())
.spawn(move || {
let r = BufReader::new(err_handle);
for line in r.lines() {
match line {
Ok(l) if !l.trim().is_empty() => tracing::info!(target: "wgc_helper", "{l}"),
Ok(_) => {}
Err(_) => break,
}
}
})
.ok();
// stdout → parsed AUs. Bounded so a stalled relay applies backpressure (the pipe then fills and
// the helper blocks on write — the same backpressure the single-process channel gives).
let (tx, rx) = std::sync::mpsc::sync_channel::<RelayAu>(3);
let out_handle = HandleReader(out_r);
std::thread::Builder::new()
.name("wgc-helper-au".into())
.spawn(move || au_reader(out_handle, tx))
.ok();
Ok(HelperRelay {
proc: pi.hProcess,
thread: pi.hThread,
stdin_w: Mutex::new(in_w),
rx,
})
}
/// Parse the AU framing off the helper's stdout and forward each AU. Ends (returns) when the pipe
/// breaks (helper exit) or the channel's receiver is dropped (relay torn down).
fn au_reader(mut r: HandleReader, tx: SyncSender<RelayAu>) {
loop {
let mut hdr = [0u8; 4 + 4 + 8 + 1];
if r.read_exact(&mut hdr).is_err() {
break;
}
let magic = u32::from_le_bytes([hdr[0], hdr[1], hdr[2], hdr[3]]);
if magic != AU_MAGIC {
tracing::error!(
magic = format!("{magic:#x}"),
"WGC helper AU stream desync — aborting relay"
);
break;
}
let len = u32::from_le_bytes([hdr[4], hdr[5], hdr[6], hdr[7]]) as usize;
let pts_ns = u64::from_le_bytes([
hdr[8], hdr[9], hdr[10], hdr[11], hdr[12], hdr[13], hdr[14], hdr[15],
]);
let keyframe = hdr[16] != 0;
// Bound the allocation — a corrupt length must not OOM the host. 64 MiB is far above any real
// AU (a 5K keyframe is a few MB).
if len > 64 * 1024 * 1024 {
tracing::error!(len, "WGC helper AU length implausible — aborting relay");
break;
}
let mut data = vec![0u8; len];
if r.read_exact(&mut data).is_err() {
break;
}
if tx
.send(RelayAu {
data,
pts_ns,
keyframe,
})
.is_err()
{
break; // relay dropped
}
}
}
/// Minimal `Read` over a Win32 pipe HANDLE (the windows crate doesn't impl `Read` on HANDLE).
struct HandleReader(HANDLE);
// SAFETY: `HandleReader` owns a single pipe `HANDLE` (a process-global kernel handle value, valid from
// any thread). It is moved into the dedicated reader thread and used only there (and closed once on
// Drop), never shared — so transferring ownership across threads is sound.
unsafe impl Send for HandleReader {}
impl Read for HandleReader {
fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
let mut read = 0u32;
// SAFETY: `self.0` is the live read end of an anonymous pipe owned by this `HandleReader`
// (closed only in Drop). `ReadFile` fills the caller-provided `buf` (writing at most `buf.len()`
// bytes) and stores the count in `read`; both outlive the synchronous call. A broken pipe
// surfaces as `Err` and is mapped to EOF below.
let ok = unsafe {
windows::Win32::Storage::FileSystem::ReadFile(self.0, Some(buf), Some(&mut read), None)
};
match ok {
Ok(()) => Ok(read as usize),
// A broken pipe (helper exited) reads as ERROR_BROKEN_PIPE → report EOF (0).
Err(_) => Ok(0),
}
}
}
impl Drop for HandleReader {
fn drop(&mut self) {
// SAFETY: `self.0` is the pipe `HANDLE` this `HandleReader` owns; `CloseHandle` (an FFI call
// taking the handle by value) is invoked exactly once here in Drop, so there is no double-close.
unsafe {
let _ = CloseHandle(self.0);
}
}
}
/// Is this process running as the LOCAL SYSTEM account? Used to decide whether the two-process
/// secure-desktop path applies (only SYSTEM can `WTSQueryUserToken` + capture the Winlogon desktop).
pub fn running_as_system() -> bool {
use windows::Win32::Security::{GetTokenInformation, TokenUser, TOKEN_QUERY, TOKEN_USER};
use windows::Win32::System::Threading::{GetCurrentProcess, OpenProcessToken};
// SAFETY: `OpenProcessToken(GetCurrentProcess(), TOKEN_QUERY, &mut token)` opens the current-process
// token (the pseudo-handle is always valid) into `token`, which is closed once before each return.
// The first `GetTokenInformation` (null buffer) queries the required `len`; `buf` is then a
// `Vec<u8>` of exactly `len` bytes and the second call fills it, so `&*(buf.as_ptr() as *const
// TOKEN_USER)` reads a `TOKEN_USER` the kernel just wrote into a sufficiently-sized buffer (the
// variable-length SID it points at also lies within `buf`, which outlives the borrow).
// `is_local_system_sid` is this module's `unsafe fn`, given that in-buffer `PSID`. Safe on any thread.
unsafe {
let mut token = HANDLE::default();
if OpenProcessToken(GetCurrentProcess(), TOKEN_QUERY, &mut token).is_err() {
return false;
}
let mut len = 0u32;
let _ = GetTokenInformation(token, TokenUser, None, 0, &mut len);
if len == 0 {
let _ = CloseHandle(token);
return false;
}
let mut buf = vec![0u8; len as usize];
let ok = GetTokenInformation(
token,
TokenUser,
Some(buf.as_mut_ptr() as *mut _),
len,
&mut len,
)
.is_ok();
let _ = CloseHandle(token);
if !ok {
return false;
}
let tu = &*(buf.as_ptr() as *const TOKEN_USER);
// The well-known LocalSystem SID is S-1-5-18.
is_local_system_sid(tu.User.Sid)
}
}
/// True iff `sid` is S-1-5-18 (LocalSystem).
unsafe fn is_local_system_sid(sid: windows::Win32::Security::PSID) -> bool {
use windows::Win32::Security::{
GetSidIdentifierAuthority, GetSidSubAuthority, GetSidSubAuthorityCount, IsValidSid,
};
if !IsValidSid(sid).as_bool() {
return false;
}
let auth = GetSidIdentifierAuthority(sid);
if auth.is_null() {
return false;
}
// NT Authority = {0,0,0,0,0,5}.
let a = (*auth).Value;
if a != [0, 0, 0, 0, 0, 5] {
return false;
}
let count = *GetSidSubAuthorityCount(sid);
if count != 1 {
return false;
}
*GetSidSubAuthority(sid, 0) == 18 // SECURITY_LOCAL_SYSTEM_RID
}
+7 -24
View File
@@ -6,8 +6,8 @@
//! //!
//! **Goal-1 stages 12** (`design/windows-host-rewrite.md` §2.2): stage 1 stood this up; stage 2 migrated the //! **Goal-1 stages 12** (`design/windows-host-rewrite.md` §2.2): stage 1 stood this up; stage 2 migrated the
//! genuinely-constant operator/dispatch knobs onto it (the dispatch-disagreement bug class: `idd_push`, //! genuinely-constant operator/dispatch knobs onto it (the dispatch-disagreement bug class: `idd_push`,
//! `capture_backend`, `encoder_pref`, `render_adapter`, `no_wgc`, the vdisplay backend select — plus the //! `encoder_pref`, `render_adapter`, the vdisplay backend select — plus the plan-named
//! plan-named `secure_dda`/`idd_depth`/`zerocopy`/`ten_bit`/`four_four_four` and the multi-site `perf`/`compositor`/ //! `idd_depth`/`zerocopy`/`ten_bit`/`four_four_four` and the multi-site `perf`/`compositor`/
//! `video_source`/`gamepad`). `SessionPlan` (stage 3) consumes it as the single owner of the //! `video_source`/`gamepad`). `SessionPlan` (stage 3) consumes it as the single owner of the
//! capture/topology/encoder decision. //! capture/topology/encoder decision.
//! //!
@@ -36,27 +36,17 @@ use std::sync::OnceLock;
/// derived `Debug` impl, so the parser can stay a single platform-neutral function. /// derived `Debug` impl, so the parser can stay a single platform-neutral function.
#[derive(Debug, Clone, Default)] #[derive(Debug, Clone, Default)]
pub struct HostConfig { pub struct HostConfig {
/// `PUNKTFUNK_IDD_PUSH` — capture from the pf-vdisplay driver's shared ring (in-process Session-0 /// `PUNKTFUNK_IDD_PUSH` — IDD direct-push monitor mode (the per-session monitor + ring recreate and
/// capture; no WGC helper). **Value-aware** (`0`/`false`/`no`/`off`/empty ⇒ off, else on); unset ⇒ off. /// the discrete-render-GPU pin in [`crate::vdisplay::manager`]). IDD-push is the sole Windows capture
/// The installer's default `host.env` sets it on, so a fresh install runs the validated IDD-push path /// path (DXGI Desktop Duplication and the WGC relay were removed), so this should stay on — the
/// (it falls back to DDA if the driver can't attach — see [`crate::capture`]). NOT a bare presence flag /// installer's `host.env` sets it. **Value-aware** (`0`/`false`/`no`/`off`/empty ⇒ off, else on);
/// (so an operator can turn it OFF in `host.env` with `=0`, which a `var_os` presence check can't). /// unset ⇒ off. NOT a bare presence flag (so an operator can turn it OFF with `=0`).
pub idd_push: bool, pub idd_push: bool,
/// `PUNKTFUNK_ENCODER` — explicit encoder-backend override (lowercased; empty = auto-detect by GPU vendor). /// `PUNKTFUNK_ENCODER` — explicit encoder-backend override (lowercased; empty = auto-detect by GPU vendor).
pub encoder_pref: String, pub encoder_pref: String,
/// `PUNKTFUNK_NO_HELPER` — never spawn the user-session WGC helper.
pub no_helper: bool,
/// `PUNKTFUNK_FORCE_HELPER` — force the WGC helper even when not running as SYSTEM.
pub force_helper: bool,
/// `PUNKTFUNK_NO_WGC` — force the pure single-process DDA path (skip WGC and the two-process relay).
pub no_wgc: bool,
/// `PUNKTFUNK_CAPTURE` — explicit Windows capture-backend override (lowercased; `dda`/`dxgi` vs the WGC default).
pub capture_backend: String,
/// `PUNKTFUNK_RENDER_ADAPTER` — discrete render-GPU pin by description substring (`Some` even when empty: /// `PUNKTFUNK_RENDER_ADAPTER` — discrete render-GPU pin by description substring (`Some` even when empty:
/// the empty string still counts as "set" for the presence checks, and the value reader filters it). /// the empty string still counts as "set" for the presence checks, and the value reader filters it).
pub render_adapter: Option<String>, pub render_adapter: Option<String>,
/// `PUNKTFUNK_SECURE_DDA` — enable the experimental DDA-on-secure-desktop (Winlogon/UAC) mux leg.
pub secure_dda: bool,
/// `PUNKTFUNK_IDD_DEPTH` — IDD-push pipeline depth override (default 2; the call site clamps to its `OUT_RING`). /// `PUNKTFUNK_IDD_DEPTH` — IDD-push pipeline depth override (default 2; the call site clamps to its `OUT_RING`).
pub idd_depth: usize, pub idd_depth: usize,
/// `PUNKTFUNK_ZEROCOPY` — opt into the Windows D3D11 zero-copy encode path (presence semantics; see module docs). /// `PUNKTFUNK_ZEROCOPY` — opt into the Windows D3D11 zero-copy encode path (presence semantics; see module docs).
@@ -103,14 +93,7 @@ impl HostConfig {
encoder_pref: std::env::var("PUNKTFUNK_ENCODER") encoder_pref: std::env::var("PUNKTFUNK_ENCODER")
.unwrap_or_default() .unwrap_or_default()
.to_ascii_lowercase(), .to_ascii_lowercase(),
no_helper: flag("PUNKTFUNK_NO_HELPER"),
force_helper: flag("PUNKTFUNK_FORCE_HELPER"),
no_wgc: flag("PUNKTFUNK_NO_WGC"),
capture_backend: std::env::var("PUNKTFUNK_CAPTURE")
.unwrap_or_default()
.to_ascii_lowercase(),
render_adapter: val("PUNKTFUNK_RENDER_ADAPTER"), render_adapter: val("PUNKTFUNK_RENDER_ADAPTER"),
secure_dda: flag("PUNKTFUNK_SECURE_DDA"),
idd_depth: val("PUNKTFUNK_IDD_DEPTH") idd_depth: val("PUNKTFUNK_IDD_DEPTH")
.and_then(|s| s.parse::<usize>().ok()) .and_then(|s| s.parse::<usize>().ok())
.unwrap_or(2), .unwrap_or(2),
@@ -66,6 +66,13 @@ pub const BTN_A: u32 = 0x1000;
pub const BTN_B: u32 = 0x2000; pub const BTN_B: u32 = 0x2000;
pub const BTN_X: u32 = 0x4000; pub const BTN_X: u32 = 0x4000;
pub const BTN_Y: u32 = 0x8000; pub const BTN_Y: u32 = 0x8000;
// Extended buttons in the `buttonFlags2 << 16` namespace (mirror `punktfunk_core::input::gamepad`):
// the four back-grip paddles. `decode` already merges `buttonFlags2 << 16` into `buttons`, but the
// injector map dropped these bits — Sunshine/Moonlight paddle clients were silently no-op'd.
pub const BTN_PADDLE1: u32 = 0x0001_0000;
pub const BTN_PADDLE2: u32 = 0x0002_0000;
pub const BTN_PADDLE3: u32 = 0x0004_0000;
pub const BTN_PADDLE4: u32 = 0x0008_0000;
/// Decode one decrypted control plaintext into a controller event, if it is one. Mouse, /// Decode one decrypted control plaintext into a controller event, if it is one. Mouse,
/// keyboard, keepalives etc. yield `None` (they're handled by [`super::input::decode`]). /// keyboard, keepalives etc. yield `None` (they're handled by [`super::input::decode`]).
@@ -101,6 +101,10 @@ struct Session {
server_challenge: [u8; 16], server_challenge: [u8; 16],
/// The client's phase-3 hash, recomputed + checked in phase 4. /// The client's phase-3 hash, recomputed + checked in phase 4.
client_hash: Vec<u8>, client_hash: Vec<u8>,
/// Set once phase 3 has produced the RSA-signed serversecret. A repeated phase 3 is refused so a
/// peer past phase 1 can't loop phase2/phase3 to harvest many signing-time samples (a passive
/// timing-oracle amplifier vs. the rsa-crate Marvin side-channel; see `.cargo/audit.toml`).
responded: bool,
} }
pub struct Pairing { pub struct Pairing {
@@ -155,6 +159,7 @@ impl Pairing {
serversecret: [0; 16], serversecret: [0; 16],
server_challenge: [0; 16], server_challenge: [0; 16],
client_hash: Vec::new(), client_hash: Vec::new(),
responded: false,
}, },
); );
tracing::info!( tracing::info!(
@@ -216,6 +221,14 @@ impl Pairing {
bail!("short challenge response"); bail!("short challenge response");
} }
s.client_hash = client_hash[..32].to_vec(); s.client_hash = client_hash[..32].to_vec();
// Sign the serversecret exactly ONCE per ceremony: refuse a repeated phase 3 so a peer that
// cleared phase 1 (operator PIN) can't replay it to collect many RSA signing-time samples
// (timing-oracle amplifier vs. RUSTSEC-2023-0071; see `.cargo/audit.toml`). A legit client
// signs once. The session stays for phase 4 (the cert-pin step) but won't re-sign.
if s.responded {
bail!("serverchallengeresp already answered for this pairing session");
}
s.responded = true;
let sig: Signature = id.signing_key.sign(&s.serversecret); let sig: Signature = id.signing_key.sign(&s.serversecret);
let mut secret = Vec::with_capacity(16 + 256); let mut secret = Vec::with_capacity(16 + 256);
secret.extend_from_slice(&s.serversecret); secret.extend_from_slice(&s.serversecret);
@@ -213,6 +213,17 @@ fn open_gs_virtual_source(
let compositor = if let Some(c) = app.and_then(|a| a.compositor) { let compositor = if let Some(c) = app.and_then(|a| a.compositor) {
c c
} else { } else {
// Windows has a single virtual-display backend (pf-vdisplay); `vdisplay::open` ignores the
// compositor arg there, so short-circuit the Linux session-detection state machine with a
// placeholder — mirrors `punktfunk1::resolve_compositor`. Without this, the Linux `detect()`
// below bails on Windows ("could not detect compositor … XDG_CURRENT_DESKTOP=''"), which
// killed the GameStream video thread → black screen (the native plane was already guarded).
#[cfg(target_os = "windows")]
{
crate::vdisplay::Compositor::Kwin
}
#[cfg(not(target_os = "windows"))]
{
let active = crate::vdisplay::detect_active_session(); let active = crate::vdisplay::detect_active_session();
crate::vdisplay::apply_session_env(&active); crate::vdisplay::apply_session_env(&active);
let c = crate::vdisplay::compositor_for_kind(active.kind) let c = crate::vdisplay::compositor_for_kind(active.kind)
@@ -221,6 +232,7 @@ fn open_gs_virtual_source(
.context("detect compositor")?; .context("detect compositor")?;
crate::vdisplay::apply_input_env(c); crate::vdisplay::apply_input_env(c);
c c
}
}; };
let mut vd = crate::vdisplay::open(compositor).context("open virtual display")?; let mut vd = crate::vdisplay::open(compositor).context("open virtual display")?;
// Carry the resolved launch command on the backend instance (per-session) rather than a // Carry the resolved launch command on the backend instance (per-session) rather than a
+25
View File
@@ -491,6 +491,31 @@ pub mod gamepad;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
#[path = "inject/windows/gamepad_raii.rs"] #[path = "inject/windows/gamepad_raii.rs"]
mod gamepad_raii; mod gamepad_raii;
/// Linux: virtual Steam Deck via UHID — the kernel `hid-steam` driver binds it as a real Deck.
#[cfg(target_os = "linux")]
#[path = "inject/linux/steam_controller.rs"]
pub mod steam_controller;
/// Linux: virtual Steam Deck via the USB gadget subsystem (`raw_gadget` + `dummy_hcd`) — the only
/// virtual-Deck transport Steam Input promotes (presents the controller on USB interface 2).
/// SteamOS-host only (needs `dummy_hcd` + `raw_gadget`).
#[cfg(target_os = "linux")]
#[path = "inject/linux/steam_gadget.rs"]
pub mod steam_gadget;
/// Transport-independent Steam Controller / Steam Deck HID contract (descriptor, byte-exact Deck
/// serializer, XInput/rich mappers, rumble parser), used by the Linux UHID backend ([`steam_controller`]).
#[cfg(target_os = "linux")]
#[path = "inject/proto/steam_proto.rs"]
pub mod steam_proto;
/// Pure fallback-remap policy (Steam-only inputs onto a non-Steam backend) + the Deck motion rescale.
#[cfg(target_os = "linux")]
#[path = "inject/proto/steam_remap.rs"]
pub mod steam_remap;
/// Linux: virtual Steam Deck over **USB/IP** (`vhci_hcd`) — the shippable, Secure-Boot-clean,
/// Steam-Input-promotable virtual-Deck transport on non-SteamOS hosts (Bazzite/generic), where
/// `dummy_hcd`/`raw_gadget` aren't built. In-tree + signed; no module build, no MOK.
#[cfg(target_os = "linux")]
#[path = "inject/linux/steam_usbip.rs"]
pub mod steam_usbip;
/// Stub — virtual gamepads need Linux uinput or the Windows UMDF drivers; events are dropped elsewhere. /// Stub — virtual gamepads need Linux uinput or the Windows UMDF drivers; events are dropped elsewhere.
#[cfg(not(any(target_os = "linux", target_os = "windows")))] #[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub mod gamepad { pub mod gamepad {
@@ -182,6 +182,9 @@ pub struct DualSenseManager {
last_write: Vec<Instant>, last_write: Vec<Instant>,
/// Pad creation failed (e.g. /dev/uhid permissions) — warn once, drop events. /// Pad creation failed (e.g. /dev/uhid permissions) — warn once, drop events.
broken: bool, broken: bool,
/// Fallback policy for the Steam back grips a client may send (the DualSense has no back-button
/// HID slot). `PUNKTFUNK_STEAM_REMAP=paddles=…`; default drop.
remap: crate::inject::steam_remap::RemapConfig,
} }
impl Default for DualSenseManager { impl Default for DualSenseManager {
@@ -198,6 +201,7 @@ impl DualSenseManager {
last_rumble: vec![(0, 0); MAX_PADS], last_rumble: vec![(0, 0); MAX_PADS],
last_write: vec![Instant::now(); MAX_PADS], last_write: vec![Instant::now(); MAX_PADS],
broken: false, broken: false,
remap: crate::inject::steam_remap::RemapConfig::from_env(),
} }
} }
@@ -229,8 +233,12 @@ impl DualSenseManager {
// Merge buttons/sticks/triggers from the frame, preserving touch + motion (those // Merge buttons/sticks/triggers from the frame, preserving touch + motion (those
// come on the rich-input plane and must survive a button-only frame). // come on the rich-input plane and must survive a button-only frame).
let prev = self.state[idx]; let prev = self.state[idx];
// Steam back grips have no DualSense slot — fold them onto standard buttons per the
// configured policy (default drop) so they aren't silently lost.
let buttons =
crate::inject::steam_remap::fold_paddles(f.buttons, self.remap.paddles);
let mut s = DsState::from_gamepad( let mut s = DsState::from_gamepad(
f.buttons, buttons,
f.ls_x, f.ls_x,
f.ls_y, f.ls_y,
f.rs_x, f.rs_x,
@@ -252,7 +260,9 @@ impl DualSenseManager {
/// arrived first); they're dropped if the pad isn't present. /// arrived first); they're dropped if the pad isn't present.
pub fn apply_rich(&mut self, rich: RichInput) { pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich { let idx = match rich {
RichInput::Touchpad { pad, .. } | RichInput::Motion { pad, .. } => pad as usize, RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
}; };
if idx >= MAX_PADS || self.pads[idx].is_none() { if idx >= MAX_PADS || self.pads[idx].is_none() {
return; return;
@@ -280,6 +290,26 @@ impl DualSenseManager {
self.state[idx].gyro = gyro; self.state[idx].gyro = gyro;
self.state[idx].accel = accel; self.state[idx].accel = accel;
} }
RichInput::TouchpadEx {
surface,
finger,
touch,
x,
y,
..
} => {
// A Steam right/single pad maps onto the one DualSense touchpad (signed centre-0 →
// 0..=65535); surface 1 (the Steam left pad) has no DualSense equivalent.
if surface != 1 {
let slot = (finger as usize).min(1);
let n = |v: i16| ((v as i32) + 32768) as u32;
let t = &mut self.state[idx].touch[slot];
t.active = touch;
t.id = slot as u8;
t.x = (n(x) * (DS_TOUCH_W - 1) as u32 / u16::MAX as u32) as u16;
t.y = (n(y) * (DS_TOUCH_H - 1) as u32 / u16::MAX as u32) as u16;
}
}
} }
self.write(idx); self.write(idx);
} }
@@ -367,6 +367,9 @@ pub struct DualShock4Manager {
last_write: Vec<Instant>, last_write: Vec<Instant>,
/// Pad creation failed (e.g. /dev/uhid permissions) — warn once, drop events. /// Pad creation failed (e.g. /dev/uhid permissions) — warn once, drop events.
broken: bool, broken: bool,
/// Fallback policy for the Steam back grips a client may send (the DS4 has no back-button HID
/// slot). `PUNKTFUNK_STEAM_REMAP=paddles=…`; default drop.
remap: crate::inject::steam_remap::RemapConfig,
} }
impl Default for DualShock4Manager { impl Default for DualShock4Manager {
@@ -384,6 +387,7 @@ impl DualShock4Manager {
last_led: vec![None; MAX_PADS], last_led: vec![None; MAX_PADS],
last_write: vec![Instant::now(); MAX_PADS], last_write: vec![Instant::now(); MAX_PADS],
broken: false, broken: false,
remap: crate::inject::steam_remap::RemapConfig::from_env(),
} }
} }
@@ -416,8 +420,12 @@ impl DualShock4Manager {
// Merge buttons/sticks/triggers, preserving touch + motion (those arrive on the // Merge buttons/sticks/triggers, preserving touch + motion (those arrive on the
// rich-input plane and must survive a button-only frame). // rich-input plane and must survive a button-only frame).
let prev = self.state[idx]; let prev = self.state[idx];
// Steam back grips have no DS4 slot — fold them onto standard buttons per the
// configured policy (default drop) so they aren't silently lost.
let buttons =
crate::inject::steam_remap::fold_paddles(f.buttons, self.remap.paddles);
let mut s = DsState::from_gamepad( let mut s = DsState::from_gamepad(
f.buttons, buttons,
f.ls_x, f.ls_x,
f.ls_y, f.ls_y,
f.rs_x, f.rs_x,
@@ -439,7 +447,9 @@ impl DualShock4Manager {
/// pad isn't present. /// pad isn't present.
pub fn apply_rich(&mut self, rich: RichInput) { pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich { let idx = match rich {
RichInput::Touchpad { pad, .. } | RichInput::Motion { pad, .. } => pad as usize, RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
}; };
if idx >= MAX_PADS || self.pads[idx].is_none() { if idx >= MAX_PADS || self.pads[idx].is_none() {
return; return;
@@ -466,6 +476,26 @@ impl DualShock4Manager {
self.state[idx].gyro = gyro; self.state[idx].gyro = gyro;
self.state[idx].accel = accel; self.state[idx].accel = accel;
} }
RichInput::TouchpadEx {
surface,
finger,
touch,
x,
y,
..
} => {
// A Steam right/single pad maps onto the one DS4 touchpad (signed centre-0 →
// 0..=65535); surface 1 (the Steam left pad) has no DS4 equivalent.
if surface != 1 {
let slot = (finger as usize).min(1);
let n = |v: i16| ((v as i32) + 32768) as u32;
let t = &mut self.state[idx].touch[slot];
t.active = touch;
t.id = slot as u8;
t.x = (n(x) * (DS4_TOUCH_W - 1) as u32 / u16::MAX as u32) as u16;
t.y = (n(y) * (DS4_TOUCH_H - 1) as u32 / u16::MAX as u32) as u16;
}
}
} }
self.write(idx); self.write(idx);
} }
@@ -69,9 +69,16 @@ const BTN_START: u16 = 0x13b;
const BTN_MODE: u16 = 0x13c; const BTN_MODE: u16 = 0x13c;
const BTN_THUMBL: u16 = 0x13d; const BTN_THUMBL: u16 = 0x13d;
const BTN_THUMBR: u16 = 0x13e; const BTN_THUMBR: u16 = 0x13e;
// Xbox-Elite paddle codes (the xpad convention SDL / Steam Input recognize). A client's back grips —
// and the GameStream `buttonFlags2` paddle bits, which were silently dropped before — land here, so
// the virtual X-Box pad exposes paddles like an Elite controller. PADDLE1/2/3/4 = R4/L4/R5/L5.
const BTN_TRIGGER_HAPPY5: u16 = 0x2c4;
const BTN_TRIGGER_HAPPY6: u16 = 0x2c5;
const BTN_TRIGGER_HAPPY7: u16 = 0x2c6;
const BTN_TRIGGER_HAPPY8: u16 = 0x2c7;
/// `(GameStream button bit, evdev key code)` — D-pad is emitted as HAT axes instead. /// `(GameStream button bit, evdev key code)` — D-pad is emitted as HAT axes instead.
const BUTTON_MAP: [(u32, u16); 11] = [ const BUTTON_MAP: [(u32, u16); 15] = [
(gamepad::BTN_A, BTN_SOUTH), (gamepad::BTN_A, BTN_SOUTH),
(gamepad::BTN_B, BTN_EAST), (gamepad::BTN_B, BTN_EAST),
(gamepad::BTN_X, BTN_NORTH), (gamepad::BTN_X, BTN_NORTH),
@@ -83,6 +90,10 @@ const BUTTON_MAP: [(u32, u16); 11] = [
(gamepad::BTN_GUIDE, BTN_MODE), (gamepad::BTN_GUIDE, BTN_MODE),
(gamepad::BTN_LS_CLK, BTN_THUMBL), (gamepad::BTN_LS_CLK, BTN_THUMBL),
(gamepad::BTN_RS_CLK, BTN_THUMBR), (gamepad::BTN_RS_CLK, BTN_THUMBR),
(gamepad::BTN_PADDLE1, BTN_TRIGGER_HAPPY5),
(gamepad::BTN_PADDLE2, BTN_TRIGGER_HAPPY6),
(gamepad::BTN_PADDLE3, BTN_TRIGGER_HAPPY7),
(gamepad::BTN_PADDLE4, BTN_TRIGGER_HAPPY8),
]; ];
/// The USB identity a virtual uinput pad presents. SDL/Steam/Proton key their built-in mapping off /// The USB identity a virtual uinput pad presents. SDL/Steam/Proton key their built-in mapping off
@@ -7,9 +7,14 @@
//! which the libei/portal path cannot. We connect as an ordinary Wayland client on the KWin session's //! which the libei/portal path cannot. We connect as an ordinary Wayland client on the KWin session's
//! `$WAYLAND_DISPLAY` and translate events into fake-input requests; keyboard keys are raw Linux //! `$WAYLAND_DISPLAY` and translate events into fake-input requests; keyboard keys are raw Linux
//! evdev codes that KWin resolves through the session's own keymap (no keymap upload, unlike the wlr //! evdev codes that KWin resolves through the session's own keymap (no keymap upload, unlike the wlr
//! virtual-keyboard path), and absolute pointer/touch coordinates are global compositor space — which //! virtual-keyboard path), and absolute pointer/touch coordinates are global compositor space.
//! on a headless box (single per-session virtual output at the origin, scale 1) equals the streamed //!
//! output's pixels. //! Global compositor space is *logical* pixels (post display-scaling), which only equals the streamed
//! output's physical pixels at scale 1. Under a fractional/integer scale the logical edge sits at
//! `physical / scale`, so feeding the raw streamed pixel coordinate lands the cursor `scale×` too far
//! toward the bottom-right (top-left stays put). We therefore track each output's logical geometry
//! (position + size) via `xdg-output` and map the normalized client position into the matching
//! output's logical rectangle — the same shape the libei backend uses with its EI region.
#![allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)] #![allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program). // Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
@@ -18,8 +23,14 @@
use super::{gs_button_to_evdev, vk_to_evdev, InputEvent, InputInjector}; use super::{gs_button_to_evdev, vk_to_evdev, InputEvent, InputInjector};
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use punktfunk_core::input::InputKind; use punktfunk_core::input::InputKind;
use std::time::{Duration, Instant};
use wayland_client::protocol::wl_output::{self, WlOutput};
use wayland_client::protocol::wl_registry::{self, WlRegistry}; use wayland_client::protocol::wl_registry::{self, WlRegistry};
use wayland_client::{Connection, Dispatch, EventQueue, Proxy, QueueHandle}; use wayland_client::{Connection, Dispatch, EventQueue, Proxy, QueueHandle, WEnum};
use wayland_protocols::xdg::xdg_output::zv1::client::{
zxdg_output_manager_v1::ZxdgOutputManagerV1,
zxdg_output_v1::{self, ZxdgOutputV1},
};
// Generate the client bindings for the vendored protocol XML inline (no build.rs), exactly like the // Generate the client bindings for the vendored protocol XML inline (no build.rs), exactly like the
// KWin virtual-output backend. Path is relative to CARGO_MANIFEST_DIR. // KWin virtual-output backend. Path is relative to CARGO_MANIFEST_DIR.
@@ -48,10 +59,39 @@ const AXIS_HORIZONTAL: u32 = 1;
/// `code` value marking a horizontal scroll event (mirrors `gamestream::input` / the wlr backend). /// `code` value marking a horizontal scroll event (mirrors `gamestream::input` / the wlr backend).
const SCROLL_HORIZONTAL: u32 = 1; const SCROLL_HORIZONTAL: u32 = 1;
/// One tracked output: its physical mode (to match the streamed resolution) and its logical geometry
/// (the global-compositor-space rectangle absolute coordinates are addressed in). `logical_w == 0`
/// means xdg-output hasn't reported its size yet.
struct OutputTrack {
/// Registry global id — also the dispatch user-data, so events route back to this entry.
name: u32,
wl_output: WlOutput,
xdg_output: Option<ZxdgOutputV1>,
/// Physical pixel mode from `wl_output.mode` (the `current` mode); matched against the streamed WxH.
mode_w: i32,
mode_h: i32,
/// Logical (post-scale) geometry from `xdg-output`.
logical_x: i32,
logical_y: i32,
logical_w: i32,
logical_h: i32,
}
/// Registry-bound globals (the Wayland dispatch state). /// Registry-bound globals (the Wayland dispatch state).
#[derive(Default)] #[derive(Default)]
struct State { struct State {
fake: Option<FakeInput>, fake: Option<FakeInput>,
xdg_mgr: Option<ZxdgOutputManagerV1>,
outputs: Vec<OutputTrack>,
}
impl State {
/// Create the `xdg_output` for a tracked output once both it and the manager exist.
fn ensure_xdg_output(o: &mut OutputTrack, mgr: &ZxdgOutputManagerV1, qh: &QueueHandle<State>) {
if o.xdg_output.is_none() {
o.xdg_output = Some(mgr.get_xdg_output(&o.wl_output, qh, o.name));
}
}
} }
impl Dispatch<WlRegistry, ()> for State { impl Dispatch<WlRegistry, ()> for State {
@@ -63,15 +103,57 @@ impl Dispatch<WlRegistry, ()> for State {
_: &Connection, _: &Connection,
qh: &QueueHandle<Self>, qh: &QueueHandle<Self>,
) { ) {
if let wl_registry::Event::Global { match event {
wl_registry::Event::Global {
name, name,
interface, interface,
version, version,
} = event } => match interface.as_str() {
{ "org_kde_kwin_fake_input" => {
if interface == "org_kde_kwin_fake_input" {
state.fake = Some(registry.bind(name, version.min(MAX_VERSION), qh, ())); state.fake = Some(registry.bind(name, version.min(MAX_VERSION), qh, ()));
} }
"wl_output" => {
// v1 carries `mode` (all we need); bind no higher than the proxy's max (4).
let wl_output: WlOutput = registry.bind(name, version.min(4), qh, name);
let mut o = OutputTrack {
name,
wl_output,
xdg_output: None,
mode_w: 0,
mode_h: 0,
logical_x: 0,
logical_y: 0,
logical_w: 0,
logical_h: 0,
};
if let Some(mgr) = state.xdg_mgr.clone() {
State::ensure_xdg_output(&mut o, &mgr, qh);
}
state.outputs.push(o);
}
"zxdg_output_manager_v1" => {
let mgr: ZxdgOutputManagerV1 = registry.bind(name, version.min(3), qh, ());
// Outputs bound before the manager have no xdg_output yet — create them now.
for o in state.outputs.iter_mut() {
State::ensure_xdg_output(o, &mgr, qh);
}
state.xdg_mgr = Some(mgr);
}
_ => {}
},
wl_registry::Event::GlobalRemove { name } => {
state.outputs.retain(|o| {
if o.name == name {
if let Some(x) = &o.xdg_output {
x.destroy();
}
false
} else {
true
}
});
}
_ => {}
} }
} }
} }
@@ -89,13 +171,86 @@ impl Dispatch<FakeInput, ()> for State {
} }
} }
impl Dispatch<WlOutput, u32> for State {
fn event(
state: &mut Self,
_: &WlOutput,
event: wl_output::Event,
name: &u32,
_: &Connection,
_: &QueueHandle<Self>,
) {
// Only the *current* mode matters — a real monitor also advertises its other supported modes.
if let wl_output::Event::Mode {
flags: WEnum::Value(flags),
width,
height,
..
} = event
{
if flags.contains(wl_output::Mode::Current) {
if let Some(o) = state.outputs.iter_mut().find(|o| o.name == *name) {
o.mode_w = width;
o.mode_h = height;
}
}
}
}
}
impl Dispatch<ZxdgOutputV1, u32> for State {
fn event(
state: &mut Self,
_: &ZxdgOutputV1,
event: zxdg_output_v1::Event,
name: &u32,
_: &Connection,
_: &QueueHandle<Self>,
) {
if let Some(o) = state.outputs.iter_mut().find(|o| o.name == *name) {
match event {
zxdg_output_v1::Event::LogicalPosition { x, y } => {
o.logical_x = x;
o.logical_y = y;
}
zxdg_output_v1::Event::LogicalSize { width, height } => {
o.logical_w = width;
o.logical_h = height;
}
_ => {}
}
}
}
}
// The manager has no events.
impl Dispatch<ZxdgOutputManagerV1, ()> for State {
fn event(
_: &mut Self,
_: &ZxdgOutputManagerV1,
_: <ZxdgOutputManagerV1 as Proxy>::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
pub struct KwinFakeInjector { pub struct KwinFakeInjector {
conn: Connection, conn: Connection,
queue: EventQueue<State>, queue: EventQueue<State>,
state: State, state: State,
fake: FakeInput, fake: FakeInput,
/// When output geometry was last re-read; throttles the per-event roundtrip (see `refresh_geometry`).
last_refresh: Option<Instant>,
} }
/// How often the fake_input backend re-reads output geometry from the compositor. Output add/remove
/// (a new session's virtual output) and live scale/resolution changes are infrequent, so a lazy
/// poll on the injector's own thread is plenty and adds at most one local-socket roundtrip twice a
/// second — versus a blocking roundtrip on every single mouse-move event.
const GEO_REFRESH: Duration = Duration::from_millis(500);
impl KwinFakeInjector { impl KwinFakeInjector {
pub fn open() -> Result<Self> { pub fn open() -> Result<Self> {
let conn = Connection::connect_to_env() let conn = Connection::connect_to_env()
@@ -122,13 +277,77 @@ impl KwinFakeInjector {
.context("fake_input authenticate roundtrip")?; .context("fake_input authenticate roundtrip")?;
conn.flush().ok(); conn.flush().ok();
tracing::info!("KWin fake_input ready (headless keyboard/mouse/touch — no portal)"); // Settle output geometry (wl_output + xdg-output were bound during the registry roundtrip
Ok(Self { // above; their logical_size arrives on a follow-up roundtrip). Best-effort — falls back to
// scale-1 mapping if xdg-output is absent.
let mut injector = Self {
conn, conn,
queue, queue,
state, state,
fake, fake,
}) last_refresh: None,
};
injector.refresh_geometry();
tracing::info!(
outputs = injector.state.outputs.len(),
"KWin fake_input ready (headless keyboard/mouse/touch — no portal)"
);
Ok(injector)
}
/// Re-read output geometry, throttled to [`GEO_REFRESH`]. A `roundtrip` both flushes any pending
/// `get_xdg_output` requests and reads the geometry events back. A wl_output that *appeared* this
/// round only gets its xdg_output created mid-dispatch, so its `logical_size` lands on a later
/// roundtrip — keep going (bounded) until every output is settled.
fn refresh_geometry(&mut self) {
let now = Instant::now();
if let Some(t) = self.last_refresh {
if now.duration_since(t) < GEO_REFRESH {
return;
}
}
self.last_refresh = Some(now);
for _ in 0..3 {
if self.queue.roundtrip(&mut self.state).is_err() {
return;
}
let pending =
self.state.xdg_mgr.is_some() && self.state.outputs.iter().any(|o| o.logical_w == 0);
if !pending {
break;
}
}
}
/// Resolve the logical (global-compositor-space) rectangle to map a normalized client position
/// into. Prefer the output whose physical mode matches the streamed `phys_w`×`phys_h` (the
/// per-session virtual output); fall back to the sole output, then — if xdg-output is unavailable
/// — to the streamed pixels at the origin (the pre-scaling behavior, correct at scale 1).
fn logical_target(&self, phys_w: i32, phys_h: i32) -> (f64, f64, f64, f64) {
let usable = || {
self.state
.outputs
.iter()
.filter(|o| o.logical_w > 0 && o.logical_h > 0)
};
let chosen = usable()
.find(|o| o.mode_w == phys_w && o.mode_h == phys_h)
.or_else(|| {
let mut it = usable();
match (it.next(), it.next()) {
(Some(only), None) => Some(only),
_ => None,
}
});
match chosen {
Some(o) => (
o.logical_x as f64,
o.logical_y as f64,
o.logical_w as f64,
o.logical_h as f64,
),
None => (0.0, 0.0, phys_w as f64, phys_h as f64),
}
} }
} }
@@ -139,12 +358,17 @@ impl InputInjector for KwinFakeInjector {
self.fake.pointer_motion(event.x as f64, event.y as f64); self.fake.pointer_motion(event.x as f64, event.y as f64);
} }
InputKind::MouseMoveAbs => { InputKind::MouseMoveAbs => {
let w = (event.flags >> 16) & 0xffff; let w = ((event.flags >> 16) & 0xffff) as i32;
let h = event.flags & 0xffff; let h = (event.flags & 0xffff) as i32;
if w > 0 && h > 0 { if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64; self.refresh_geometry();
let y = event.y.clamp(0, h as i32) as f64; let (lx, ly, lw, lh) = self.logical_target(w, h);
self.fake.pointer_motion_absolute(x, y); // Normalize in the streamed (physical) pixel space, then place inside the output's
// logical rectangle — so display scaling no longer offsets the cursor.
let nx = (event.x as f64 / w as f64).clamp(0.0, 1.0);
let ny = (event.y as f64 / h as f64).clamp(0.0, 1.0);
self.fake
.pointer_motion_absolute(lx + nx * lw, ly + ny * lh);
} }
} }
InputKind::MouseButtonDown | InputKind::MouseButtonUp => { InputKind::MouseButtonDown | InputKind::MouseButtonUp => {
@@ -179,11 +403,15 @@ impl InputInjector for KwinFakeInjector {
// Touch: id = event.code, coords in the client surface w×h packed into flags (same // Touch: id = event.code, coords in the client surface w×h packed into flags (same
// absolute mapping as MouseMoveAbs). Each event is its own frame. // absolute mapping as MouseMoveAbs). Each event is its own frame.
InputKind::TouchDown | InputKind::TouchMove => { InputKind::TouchDown | InputKind::TouchMove => {
let w = (event.flags >> 16) & 0xffff; let w = ((event.flags >> 16) & 0xffff) as i32;
let h = event.flags & 0xffff; let h = (event.flags & 0xffff) as i32;
if w > 0 && h > 0 { if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64; self.refresh_geometry();
let y = event.y.clamp(0, h as i32) as f64; let (lx, ly, lw, lh) = self.logical_target(w, h);
let nx = (event.x as f64 / w as f64).clamp(0.0, 1.0);
let ny = (event.y as f64 / h as f64).clamp(0.0, 1.0);
let x = lx + nx * lw;
let y = ly + ny * lh;
if event.kind == InputKind::TouchDown { if event.kind == InputKind::TouchDown {
self.fake.touch_down(event.code, x, y); self.fake.touch_down(event.code, x, y);
} else { } else {
@@ -0,0 +1,575 @@
//! Virtual Steam Deck controller via UHID — the Steam analogue of the virtual DualSense
//! ([`super::dualsense`]). A UHID device with Valve VID `28DE` / Deck PID `1205` is bound by the
//! kernel `hid-steam` driver, which exposes a full Steam Deck gamepad evdev (incl. the four back
//! grips) **plus** a separate IMU evdev, and — when Steam runs on the host — is re-grabbed by Steam
//! Input with native glyphs + trackpad/gyro/back-button bindings.
//!
//! The transport-independent contract (descriptor, byte-exact serializer, the `XInput`/rich
//! mappers, the rumble parser) lives in [`super::steam_proto`]; this module is the `/dev/uhid`
//! plumbing + the two Steam-specific lifecycle quirks the DualSense path lacks:
//!
//! 1. **`gamepad_mode` entry.** `steam_do_deck_input_event` early-returns under the default
//! `lizard_mode` until `gamepad_mode` is toggled on — which the kernel only does when the `b9.6`
//! Steam/menu-right button is held ~450 ms with no hidraw client open. So on the first pad we
//! best-effort clear `lizard_mode` via sysfs (needs root; bypasses the gate entirely) AND every
//! pad pulses `b9.6` for [`MODE_ENTER`] at creation. After that an **anti-toggle guard** caps any
//! continuous `b9.6` (a long in-game Start-hold) below the kernel's 450 ms threshold so play can
//! never accidentally flip `gamepad_mode` back off.
//! 2. **`UHID_SET_REPORT`.** Steam feedback (`0xEB` rumble) + the kernel's settings/serial writes
//! arrive as FEATURE set-reports that MUST be answered `err = 0`, or the kernel stalls ~5 s per
//! command (the DualSense backend only services GET_REPORT + OUTPUT).
use super::steam_proto::{
btn, parse_steam_output, serial_reply, serialize_deck_state, SteamState, STEAMDECK_PRODUCT,
STEAMDECK_RDESC, STEAM_REPORT_LEN, STEAM_VENDOR,
};
use crate::gamestream::gamepad::{GamepadEvent, MAX_PADS};
use anyhow::{Context, Result};
use punktfunk_core::quic::{HidOutput, RichInput};
use std::fs::{File, OpenOptions};
use std::io::{Read, Write};
use std::os::unix::fs::OpenOptionsExt;
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::{Duration, Instant};
// /dev/uhid event ABI — same layout as the DualSense backend.
const UHID_PATH: &str = "/dev/uhid";
const UHID_DESTROY: u32 = 1;
const UHID_OUTPUT: u32 = 6;
const UHID_GET_REPORT: u32 = 9;
const UHID_GET_REPORT_REPLY: u32 = 10;
const UHID_CREATE2: u32 = 11;
const UHID_INPUT2: u32 = 12;
const UHID_SET_REPORT: u32 = 13;
const UHID_SET_REPORT_REPLY: u32 = 14;
const HID_MAX_DESCRIPTOR_SIZE: usize = 4096;
const UHID_EVENT_SIZE: usize = 4 + 4372;
const BUS_USB: u16 = 0x03;
/// Hold the `b9.6` mode-switch this long at creation to toggle `gamepad_mode` on (the kernel needs
/// ~450 ms continuous; give margin).
const MODE_ENTER: Duration = Duration::from_millis(650);
/// Cap continuous `b9.6` (Start) below the kernel's 450 ms mode-switch threshold: after this long
/// we insert a one-frame release so an in-game long-Start-hold can't toggle `gamepad_mode` off.
const MENU_HOLD_CAP: Duration = Duration::from_millis(350);
fn put_cstr(ev: &mut [u8], off: usize, cap: usize, s: &str) {
let n = s.len().min(cap - 1);
ev[off..off + n].copy_from_slice(&s.as_bytes()[..n]);
}
/// Best-effort, once per process: clear `hid_steam`'s `lizard_mode` so `steam_do_deck_input_event`
/// stops gating on `gamepad_mode` (gamepad events then always flow). Needs root; on failure the
/// per-pad `b9.6` pulse + guard handle it instead.
fn try_clear_lizard_mode() {
static TRIED: AtomicBool = AtomicBool::new(false);
if TRIED.swap(true, Ordering::Relaxed) {
return;
}
match std::fs::write("/sys/module/hid_steam/parameters/lizard_mode", "N") {
Ok(()) => {
tracing::info!("cleared hid_steam lizard_mode (Steam Deck gamepad events always flow)")
}
Err(e) => tracing::debug!(
error = %e,
"could not clear hid_steam lizard_mode (no root?) — using the gamepad_mode pulse + guard"
),
}
}
/// A virtual Steam Deck backed by `/dev/uhid`. Dropping it destroys the device (the kernel tears
/// down the bound `hid-steam` interface + both evdevs).
pub struct SteamDeckPad {
fd: File,
seq: u32,
created: Instant,
/// When `b9.6` started being continuously held in our OUTPUT (anti-toggle guard); `None` = not.
menu_hold_since: Option<Instant>,
}
impl SteamDeckPad {
pub fn open(index: u8) -> Result<SteamDeckPad> {
try_clear_lizard_mode();
let fd = OpenOptions::new()
.read(true)
.write(true)
.custom_flags(libc::O_NONBLOCK)
.open(UHID_PATH)
.with_context(|| {
format!("open {UHID_PATH} (is the uhid udev rule installed + are you in 'input'?)")
})?;
let mut pad = SteamDeckPad {
fd,
seq: 0,
created: Instant::now(),
menu_hold_since: None,
};
pad.send_create2(index).context("UHID_CREATE2 Steam Deck")?;
Ok(pad)
}
fn send_create2(&mut self, index: u8) -> Result<()> {
let mut ev = [0u8; UHID_EVENT_SIZE];
ev[0..4].copy_from_slice(&UHID_CREATE2.to_ne_bytes());
put_cstr(&mut ev, 4, 128, &format!("Punktfunk Steam Deck {index}")); // name[128]
put_cstr(&mut ev, 132, 64, &format!("punktfunk/steam/{index}")); // phys[64]
put_cstr(&mut ev, 196, 64, &format!("punktfunk-steam-{index}")); // uniq[64]
ev[260..262].copy_from_slice(&(STEAMDECK_RDESC.len() as u16).to_ne_bytes()); // rd_size
ev[262..264].copy_from_slice(&BUS_USB.to_ne_bytes()); // bus
ev[264..268].copy_from_slice(&STEAM_VENDOR.to_ne_bytes());
ev[268..272].copy_from_slice(&STEAMDECK_PRODUCT.to_ne_bytes());
ev[272..276].copy_from_slice(&0x0100u32.to_ne_bytes()); // version
ev[276..280].copy_from_slice(&0u32.to_ne_bytes()); // country
ev[280..280 + STEAMDECK_RDESC.len()].copy_from_slice(STEAMDECK_RDESC);
self.fd.write_all(&ev).context("write UHID_CREATE2")?;
Ok(())
}
/// Serialize `st` (with the gamepad-mode entry overlay + anti-toggle guard applied) and write it.
pub fn write_state(&mut self, st: &SteamState) -> Result<()> {
self.seq = self.seq.wrapping_add(1);
let mut s = *st;
s.buttons = self.effective_buttons(st.buttons);
let mut r = [0u8; STEAM_REPORT_LEN];
serialize_deck_state(&mut r, &s, self.seq);
let mut ev = [0u8; UHID_EVENT_SIZE];
ev[0..4].copy_from_slice(&UHID_INPUT2.to_ne_bytes());
ev[4..6].copy_from_slice(&(r.len() as u16).to_ne_bytes()); // input2.size
ev[6..6 + r.len()].copy_from_slice(&r); // input2.data
self.fd.write_all(&ev).context("write UHID_INPUT2")?;
Ok(())
}
/// True while still pulsing the mode-switch at creation (the caller force-writes during this).
fn in_mode_entry(&self) -> bool {
self.created.elapsed() < MODE_ENTER
}
/// During mode entry, force `b9.6` held (override). Afterwards, pass the real buttons through but
/// drop `b9.6` for one frame whenever it's been continuously held past [`MENU_HOLD_CAP`].
fn effective_buttons(&mut self, mut buttons: u64) -> u64 {
if self.in_mode_entry() {
return btn::STEAM_MENU_RIGHT;
}
if buttons & btn::MENU != 0 {
let now = Instant::now();
match self.menu_hold_since {
None => self.menu_hold_since = Some(now),
Some(since) if now.duration_since(since) >= MENU_HOLD_CAP => {
buttons &= !btn::MENU; // one-frame release resets the kernel's mode-switch timer
self.menu_hold_since = None;
}
Some(_) => {}
}
} else {
self.menu_hold_since = None;
}
buttons
}
/// Service the device, non-blocking: answer the kernel's GET_REPORT (serial) + SET_REPORT
/// (settings / rumble — ack `err=0`) and parse any rumble feedback (`0xEB`, on either the
/// SET_REPORT or OUTPUT path) into `(low, high)` for the universal rumble plane.
pub fn service(&mut self) -> Option<(u16, u16)> {
let mut rumble = None;
let mut ev = [0u8; UHID_EVENT_SIZE];
while let Ok(n) = self.fd.read(&mut ev) {
if n < UHID_EVENT_SIZE {
break;
}
match u32::from_ne_bytes([ev[0], ev[1], ev[2], ev[3]]) {
UHID_OUTPUT => {
let size = u16::from_ne_bytes([ev[4100], ev[4101]]) as usize;
let end = 4 + size.min(HID_MAX_DESCRIPTOR_SIZE);
if let Some(r) = parse_steam_output(&ev[4..end]).rumble {
rumble = Some(r);
}
}
UHID_GET_REPORT => {
let id = u32::from_ne_bytes([ev[4], ev[5], ev[6], ev[7]]);
let _ = self.reply_get_report(id, &serial_reply("PUNKTFUNK01"));
}
UHID_SET_REPORT => {
let id = u32::from_ne_bytes([ev[4], ev[5], ev[6], ev[7]]);
// SET_REPORT data: [report-id 0, cmd, …] at ev[12..]. Surface rumble, then ack.
let end = (12 + 16).min(UHID_EVENT_SIZE);
if let Some(r) = parse_steam_output(&ev[12..end]).rumble {
rumble = Some(r);
}
let _ = self.reply_set_report(id);
}
_ => {} // Start/Stop/Open/Close — ignore
}
}
rumble
}
fn reply_get_report(&mut self, id: u32, data: &[u8]) -> Result<()> {
let mut ev = [0u8; UHID_EVENT_SIZE];
ev[0..4].copy_from_slice(&UHID_GET_REPORT_REPLY.to_ne_bytes());
ev[4..8].copy_from_slice(&id.to_ne_bytes());
ev[8..10].copy_from_slice(&0u16.to_ne_bytes()); // err 0
ev[10..12].copy_from_slice(&(data.len() as u16).to_ne_bytes());
ev[12..12 + data.len()].copy_from_slice(data);
self.fd.write_all(&ev).context("UHID_GET_REPORT_REPLY")?;
Ok(())
}
fn reply_set_report(&mut self, id: u32) -> Result<()> {
let mut ev = [0u8; UHID_EVENT_SIZE];
ev[0..4].copy_from_slice(&UHID_SET_REPORT_REPLY.to_ne_bytes());
ev[4..8].copy_from_slice(&id.to_ne_bytes());
ev[8..10].copy_from_slice(&0u16.to_ne_bytes()); // err 0 (ack)
self.fd.write_all(&ev).context("UHID_SET_REPORT_REPLY")?;
Ok(())
}
}
impl Drop for SteamDeckPad {
fn drop(&mut self) {
let mut ev = [0u8; UHID_EVENT_SIZE];
ev[0..4].copy_from_slice(&UHID_DESTROY.to_ne_bytes());
let _ = self.fd.write_all(&ev);
}
}
/// All virtual Steam Deck pads of a session — the Steam analogue of
/// [`DualSenseManager`](super::dualsense::DualSenseManager), selected with `PUNKTFUNK_GAMEPAD=steamdeck`.
/// Button/stick frames arrive via [`handle`](Self::handle); the right trackpad + motion via
/// [`apply_rich`](Self::apply_rich); [`pump`](Self::pump) services the kernel handshake + routes
/// rumble back; [`heartbeat`](Self::heartbeat) keeps the pad alive (and drives the mode-entry pulse).
/// The transport a manager pad drives. UHID is universal but Steam Input won't promote it (a UHID
/// device has no USB interface number, `Interface: -1`); the USB **gadget** (`raw_gadget`, SteamOS)
/// and **usbip** (`vhci_hcd`, universal) both present the controller on USB interface 2, which Steam
/// Input *does* promote. Selected per-pad by [`open_transport`].
enum DeckTransport {
Uhid(SteamDeckPad),
Gadget(crate::inject::steam_gadget::SteamDeckGadget),
Usbip(crate::inject::steam_usbip::SteamDeckUsbip),
}
impl DeckTransport {
fn write_state(&mut self, st: &SteamState) {
match self {
DeckTransport::Uhid(p) => {
let _ = p.write_state(st);
}
DeckTransport::Gadget(g) => g.write_state(st),
DeckTransport::Usbip(u) => u.write_state(st),
}
}
fn service(&mut self) -> Option<(u16, u16)> {
match self {
DeckTransport::Uhid(p) => p.service(),
DeckTransport::Gadget(g) => g.service().rumble,
DeckTransport::Usbip(u) => u.service().rumble,
}
}
fn in_mode_entry(&self) -> bool {
match self {
// Only the UHID pad needs the gamepad-mode entry pulse: the promoted transports are
// read raw via hidraw by Steam Input, which bypasses the kernel's evdev mode gate.
DeckTransport::Uhid(p) => p.in_mode_entry(),
DeckTransport::Gadget(_) | DeckTransport::Usbip(_) => false,
}
}
}
/// Open the best Steam-Input-promotable Deck transport available, in preference order:
/// **`raw_gadget` (SteamOS validated fast-path) → `usbip`/`vhci_hcd` (universal, Secure-Boot-clean)
/// → UHID (universal, but `Interface: -1` so Steam Input won't promote it).** Each rung degrades to
/// the next on failure, so a host lacking the gadget modules still gets a *promotable* Deck via
/// usbip, and one lacking both still gets a working (if non-promoted) UHID pad.
fn open_transport(idx: u8) -> Result<DeckTransport> {
use crate::inject::{steam_gadget, steam_usbip};
// 1. raw_gadget — the validated SteamOS fast-path (default on there).
if steam_gadget::gadget_preferred() {
steam_gadget::ensure_modules();
match steam_gadget::SteamDeckGadget::open(idx) {
Ok(g) => {
tracing::info!(
index = idx,
"virtual Steam Deck created (USB gadget — Steam Input recognizes it)"
);
return Ok(DeckTransport::Gadget(g));
}
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "USB-gadget Deck unavailable — trying usbip")
}
}
}
// 2. usbip/vhci_hcd — the universal, in-tree, Secure-Boot-clean transport (default on elsewhere).
if steam_usbip::usbip_preferred() {
match steam_usbip::SteamDeckUsbip::open(idx) {
Ok(u) => return Ok(DeckTransport::Usbip(u)),
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "usbip Deck unavailable — falling back to UHID")
}
}
}
// 3. UHID — universal fallback (works everywhere; Steam Input won't promote it).
let p = SteamDeckPad::open(idx)?;
tracing::info!(
index = idx,
"virtual Steam Deck created (UHID hid-steam — not Steam-Input-promoted)"
);
Ok(DeckTransport::Uhid(p))
}
pub struct SteamControllerManager {
pads: Vec<Option<DeckTransport>>,
state: Vec<SteamState>,
last_rumble: Vec<(u16, u16)>,
last_write: Vec<Instant>,
broken: bool,
}
impl Default for SteamControllerManager {
fn default() -> SteamControllerManager {
SteamControllerManager::new()
}
}
impl SteamControllerManager {
pub fn new() -> SteamControllerManager {
SteamControllerManager {
pads: (0..MAX_PADS).map(|_| None).collect(),
state: vec![SteamState::neutral(); MAX_PADS],
last_rumble: vec![(0, 0); MAX_PADS],
last_write: vec![Instant::now(); MAX_PADS],
broken: false,
}
}
pub fn handle(&mut self, ev: &GamepadEvent) {
match ev {
GamepadEvent::Arrival { index, kind, .. } => {
tracing::info!(index, kind, "controller arrival (Steam Deck)");
self.ensure(*index as usize);
}
GamepadEvent::State(f) => {
let idx = f.index as usize;
if idx >= MAX_PADS {
return;
}
for (i, slot) in self.pads.iter_mut().enumerate() {
if slot.is_some() && f.active_mask & (1 << i) == 0 {
tracing::info!(index = i, "controller unplugged (Steam Deck)");
*slot = None;
self.state[i] = SteamState::neutral();
self.last_rumble[i] = (0, 0);
}
}
if f.active_mask & (1 << idx) == 0 {
return;
}
self.ensure(idx);
// Merge buttons/sticks/triggers, preserving the rich-plane fields (trackpad + motion
// arrive separately and must survive a button-only frame).
let prev = self.state[idx];
let mut s = SteamState::from_gamepad(
f.buttons,
f.ls_x,
f.ls_y,
f.rs_x,
f.rs_y,
f.left_trigger,
f.right_trigger,
);
s.rpad_x = prev.rpad_x;
s.rpad_y = prev.rpad_y;
s.lpad_x = prev.lpad_x;
s.lpad_y = prev.lpad_y;
s.gyro = prev.gyro;
s.accel = prev.accel;
s.buttons |= prev.buttons & (btn::RPAD_TOUCH | btn::LPAD_TOUCH);
self.state[idx] = s;
self.write(idx);
}
}
}
/// Apply a rich client→host event (right trackpad / motion) to an existing pad.
pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich {
RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
};
if idx >= MAX_PADS || self.pads[idx].is_none() {
return;
}
self.state[idx].apply_rich(rich);
self.write(idx);
}
fn write(&mut self, idx: usize) {
let st = self.state[idx];
if let Some(pad) = self.pads[idx].as_mut() {
pad.write_state(&st);
}
self.last_write[idx] = Instant::now();
}
/// Re-emit each live pad's current report when silent past `max_gap`, and force a steady stream
/// while a pad is still pulsing its gamepad-mode entry (so the `b9.6` toggle completes even with
/// no game input).
pub fn heartbeat(&mut self, max_gap: Duration) {
let now = Instant::now();
for i in 0..self.pads.len() {
let Some(pad) = self.pads[i].as_ref() else {
continue;
};
if pad.in_mode_entry() || now.duration_since(self.last_write[i]) >= max_gap {
self.write(i);
}
}
}
fn ensure(&mut self, idx: usize) {
if idx >= MAX_PADS || self.pads[idx].is_some() || self.broken {
return;
}
match open_transport(idx as u8) {
Ok(t) => {
self.pads[idx] = Some(t);
self.state[idx] = SteamState::neutral();
self.last_rumble[idx] = (0, 0);
self.last_write[idx] = Instant::now();
}
Err(e) => {
tracing::error!(error = %format!("{e:#}"), "virtual Steam Deck creation failed — controller input disabled");
self.broken = true;
}
}
}
/// Service every pad: answer the kernel handshake and forward rumble on the universal plane.
/// `rumble` fires `(index, low, high)` only on a level change. The Steam Deck has no rich
/// host→client feedback plane (no lightbar / adaptive triggers), so `hidout` goes unused.
pub fn pump(&mut self, mut rumble: impl FnMut(u16, u16, u16), _hidout: impl FnMut(HidOutput)) {
for i in 0..self.pads.len() {
let Some(pad) = self.pads[i].as_mut() else {
continue;
};
if let Some(r) = pad.service() {
if self.last_rumble[i] != r {
self.last_rumble[i] = r;
rumble(i as u16, r.0, r.1);
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Find the evdev node for a kernel input device by exact name (e.g. `"Steam Deck"`).
fn find_node(name: &str) -> Option<String> {
let devs = std::fs::read_to_string("/proc/bus/input/devices").ok()?;
for block in devs.split("\n\n") {
if !block
.lines()
.any(|l| l.trim() == format!("N: Name=\"{name}\""))
{
continue;
}
for l in block.lines() {
if let Some(h) = l.strip_prefix("H: Handlers=") {
if let Some(ev) = h.split_whitespace().find(|t| t.starts_with("event")) {
return Some(format!("/dev/input/{ev}"));
}
}
}
}
None
}
/// Read the evdev's current key bitmap (`EVIOCGKEY`) and test whether `code` is down.
fn key_is_down(node: &str, code: u16) -> bool {
use std::os::unix::io::AsRawFd;
let Ok(f) = std::fs::File::open(node) else {
return false;
};
let mut bits = [0u8; 96];
const EVIOCGKEY: libc::c_ulong = (2 << 30) | (96 << 16) | (0x45 << 8) | 0x18;
// SAFETY: EVIOCGKEY copies the current key-state bitmap of the evdev behind the valid fd
// `f` into `bits`; 96 bytes covers KEY_MAX/8, so the kernel never writes past the buffer.
let rc = unsafe { libc::ioctl(f.as_raw_fd(), EVIOCGKEY, bits.as_mut_ptr()) };
rc >= 0 && (bits[(code / 8) as usize] >> (code % 8)) & 1 == 1
}
/// Read the current value of an absolute axis (`EVIOCGABS`) — the first `i32` of `input_absinfo`.
fn abs_value(node: &str, abs: u16) -> Option<i32> {
use std::os::unix::io::AsRawFd;
let f = std::fs::File::open(node).ok()?;
let mut info = [0u8; 24]; // struct input_absinfo { value, min, max, fuzz, flat, resolution }
let req: libc::c_ulong =
(2 << 30) | (24 << 16) | (0x45 << 8) | (0x40 + abs as libc::c_ulong);
// SAFETY: EVIOCGABS fills the 24-byte input_absinfo for the valid evdev fd `f`; we read only
// the leading i32 `value`. The buffer is exactly sizeof(input_absinfo), so no overflow.
let rc = unsafe { libc::ioctl(f.as_raw_fd(), req, info.as_mut_ptr()) };
(rc >= 0).then(|| i32::from_ne_bytes([info[0], info[1], info[2], info[3]]))
}
/// On-box smoke test for the real backend: a `SteamDeckPad` must bind `hid-steam` (creating both
/// the gamepad + IMU evdevs), enter `gamepad_mode` via the creation pulse, and land a held button
/// on the evdev (`BTN_A`, code 0x130) — proving the entry overlay + byte-exact serialize path —
/// then tear the device down on drop. Touches `/dev/uhid`, so it is `#[ignore]`d in CI; run on a
/// box with `hid-steam` + `input`-group access: `cargo test -p punktfunk-host -- --ignored`.
#[test]
#[ignore = "creates a real /dev/uhid device; needs hid-steam + the input group"]
fn backend_binds_and_input_flows() {
use punktfunk_core::input::gamepad as gs;
const BTN_A: u16 = 0x130;
const ABS_HAT0X: u16 = 0x10; // left trackpad X
let mut pad = SteamDeckPad::open(0).expect("open SteamDeckPad (/dev/uhid + input group?)");
// Drive the full M3 wire path: build state through `from_gamepad` (BTN_A + the L4 back grip)
// and `apply_rich` (a left-pad TouchpadEx contact), then hold it past MODE_ENTER (the b9.6
// pulse), servicing the handshake.
let mut st = SteamState::from_gamepad(gs::BTN_A | gs::BTN_PADDLE2, 0, 0, 0, 0, 0, 0);
st.apply_rich(RichInput::TouchpadEx {
pad: 0,
surface: 1,
finger: 0,
touch: true,
click: false,
x: -8000,
y: 9000,
pressure: 0,
});
let start = Instant::now();
while start.elapsed() < Duration::from_millis(1200) {
let _ = pad.service();
pad.write_state(&st).expect("write_state");
std::thread::sleep(Duration::from_millis(4));
}
let devs = std::fs::read_to_string("/proc/bus/input/devices").unwrap_or_default();
assert!(devs.contains("Steam Deck"), "gamepad evdev not created");
assert!(
devs.contains("Steam Deck Motion Sensors"),
"IMU evdev not created"
);
let node = find_node("Steam Deck").expect("gamepad evdev node");
assert!(
key_is_down(&node, BTN_A),
"BTN_A not down — gamepad_mode entry or serialize failed"
);
// The left trackpad contact (TouchpadEx surface 1, gated on LPAD_TOUCH) reaches ABS_HAT0X.
assert_eq!(
abs_value(&node, ABS_HAT0X),
Some(-8000),
"left trackpad (TouchpadEx surface 1) did not reach ABS_HAT0X"
);
drop(pad);
std::thread::sleep(Duration::from_millis(200));
let devs = std::fs::read_to_string("/proc/bus/input/devices").unwrap_or_default();
assert!(
!devs.contains("Steam Deck Motion Sensors"),
"device not torn down on drop"
);
}
}
@@ -0,0 +1,558 @@
//! Virtual Steam Deck via the USB **gadget** subsystem (`raw_gadget` + `dummy_hcd`) — the only
//! virtual-Deck transport Steam Input recognizes.
//!
//! The UHID [`super::steam_controller::SteamDeckPad`] binds the kernel `hid-steam` driver, but Steam's
//! own controller driver filters the Deck's controller to USB **interface 2**, and a UHID device has no
//! USB interface number (`Interface: -1`), so Steam enumerates it but never promotes it. This backend
//! instead presents a *real* 3-interface USB Deck (mouse = interface 0, keyboard = 1, **controller =
//! 2**) on a `dummy_hcd` loopback UDC, driven from userspace via `/dev/raw-gadget` so we can answer
//! every control transfer (including the HID feature reports `f_hid` can't). Proven on a real Deck:
//! hid-steam binds it, Steam reserves an XInput slot and emits an X-Box pad. Descriptors are captured
//! verbatim from a physical Deck; see `packaging/linux/steam-deck-gadget/` for the original PoC + the
//! USB-stack gotchas. **SteamOS-host only** (needs `dummy_hcd` + `raw_gadget`, which SteamOS ships).
//!
//! The transport here is self-contained (libc + std); the report bytes it streams are produced by
//! [`super::steam_proto`] in the wrapping backend.
use anyhow::{bail, Context, Result};
use std::mem::size_of;
use std::os::fd::RawFd;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::thread::JoinHandle;
// ---- raw_gadget UAPI (mirrors linux/usb/raw_gadget.h; inlined like the C PoC) ----
const UDC_NAME_MAX: usize = 128;
#[repr(C)]
struct UsbRawInit {
driver_name: [u8; UDC_NAME_MAX],
device_name: [u8; UDC_NAME_MAX],
speed: u8,
}
// usb_raw_event { u32 type; u32 length; u8 data[]; } — we read it into a fixed buffer.
const EVENT_HDR: usize = 8; // type + length
const EVENT_BUF: usize = EVENT_HDR + 64; // setup packet (8) fits easily
// usb_raw_ep_io { u16 ep; u16 flags; u32 length; u8 data[]; }
const EPIO_HDR: usize = 8;
// usb_endpoint_descriptor is 9 bytes in the kernel (audio bRefresh/bSynchAddress); EP_ENABLE wants it.
#[repr(C, packed)]
#[derive(Clone, Copy, Default)]
struct UsbEndpointDescriptor {
b_length: u8,
b_descriptor_type: u8,
b_endpoint_address: u8,
bm_attributes: u8,
w_max_packet_size: u16,
b_interval: u8,
b_refresh: u8,
b_synch_address: u8,
}
const fn ioc(dir: u64, nr: u64, size: usize) -> libc::c_ulong {
((dir << 30) | ((size as u64) << 16) | ((b'U' as u64) << 8) | nr) as libc::c_ulong
}
const IOCTL_INIT: libc::c_ulong = ioc(1, 0, size_of::<UsbRawInit>());
const IOCTL_RUN: libc::c_ulong = ioc(0, 1, 0);
const IOCTL_EVENT_FETCH: libc::c_ulong = ioc(2, 2, EVENT_HDR); // size is the header; kernel copies more
const IOCTL_EP0_WRITE: libc::c_ulong = ioc(1, 3, EPIO_HDR);
const IOCTL_EP0_READ: libc::c_ulong = ioc(2 | 1, 4, EPIO_HDR); // _IOWR
const IOCTL_EP_ENABLE: libc::c_ulong = ioc(1, 5, size_of::<UsbEndpointDescriptor>());
const IOCTL_EP_WRITE: libc::c_ulong = ioc(1, 7, EPIO_HDR);
const IOCTL_CONFIGURE: libc::c_ulong = ioc(0, 9, 0);
const IOCTL_VBUS_DRAW: libc::c_ulong = ioc(1, 10, 4);
const IOCTL_EP0_STALL: libc::c_ulong = ioc(0, 12, 0);
const USB_RAW_EVENT_CONNECT: u32 = 1;
const USB_RAW_EVENT_CONTROL: u32 = 2;
const USB_SPEED_HIGH: u8 = 3;
// Captured-from-hardware Deck descriptors + the `0x83`/`0xAE` feature contract live in the shared
// [`super::steam_proto`] module (single source of truth, also used by the usbip transport).
use super::steam_proto::{
deck_serial, deck_unit_id, feature_reply, neutral_deck_report, RDESC_DECK_CTRL as RDESC_CTRL,
RDESC_DECK_KBD as RDESC_KBD, RDESC_DECK_MOUSE as RDESC_MOUSE,
};
const DEV_DESC: [u8; 18] = [
18, 1, 0x00, 0x02, // bLength, DEVICE, bcdUSB 2.00
0, 0, 0, 64, // class/sub/proto, bMaxPacketSize0
0xDE, 0x28, 0x05, 0x12, // idVendor 28DE, idProduct 1205
0x00, 0x03, // bcdDevice 3.00
1, 2, 3, 1, // iManufacturer, iProduct, iSerial, bNumConfigurations
];
const HID_DT: u8 = 0x21;
const HID_RPT_DT: u8 = 0x22;
/// Assemble the 84-byte config descriptor: config + 3×(interface + HID + 7-byte endpoint).
fn build_config() -> Vec<u8> {
let mut c = Vec::with_capacity(84);
// config descriptor (wTotalLength patched after)
c.extend_from_slice(&[9, 2, 84, 0, 3, 1, 0, 0x80, 250]);
// helper closures
let iface = |n: u8, sub: u8, proto: u8| [9u8, 4, n, 0, 1, 3, sub, proto, 0];
let hid = |rlen: u16, country: u8| {
[
9u8,
HID_DT,
0x10,
0x01,
country,
1,
HID_RPT_DT,
(rlen & 0xff) as u8,
(rlen >> 8) as u8,
]
};
let ep = |addr: u8, mps: u16| [7u8, 5, addr, 0x03, (mps & 0xff) as u8, (mps >> 8) as u8, 4];
// interface 0: mouse, EP 0x81
c.extend_from_slice(&iface(0, 0, 2));
c.extend_from_slice(&hid(RDESC_MOUSE.len() as u16, 0));
c.extend_from_slice(&ep(0x81, 8));
// interface 1: keyboard (boot), EP 0x82
c.extend_from_slice(&iface(1, 1, 1));
c.extend_from_slice(&hid(RDESC_KBD.len() as u16, 0));
c.extend_from_slice(&ep(0x82, 8));
// interface 2: controller, EP 0x83, bCountryCode 33
c.extend_from_slice(&iface(2, 0, 0));
c.extend_from_slice(&hid(RDESC_CTRL.len() as u16, 33));
c.extend_from_slice(&ep(0x83, 64));
debug_assert_eq!(c.len(), 84);
c
}
fn string_desc(idx: u8, serial: &str) -> Vec<u8> {
if idx == 0 {
return vec![4, 3, 0x09, 0x04]; // LANGID en-US
}
let s: &str = match idx {
1 => "Valve Software",
2 => "Steam Deck Controller",
3 => serial,
_ => "",
};
let mut v = vec![(2 + s.len() * 2) as u8, 3];
for ch in s.encode_utf16() {
v.push((ch & 0xff) as u8);
v.push((ch >> 8) as u8);
}
v
}
// ---- ioctl wrappers (the only unsafe surface for the raw_gadget UAPI; documented once) ----
fn ioctl_ptr<T>(fd: RawFd, req: libc::c_ulong, arg: *const T) -> i32 {
// SAFETY: `fd` is our open /dev/raw-gadget descriptor; `arg` points to a correctly-sized,
// initialized argument for `req` (a raw_gadget UAPI struct or an owned usb_raw_ep_io buffer)
// that lives for the duration of the call. `ioctl` is variadic, so passing a thin pointer is ABI-correct.
unsafe { libc::ioctl(fd, req as _, arg) as i32 }
}
fn ioctl_mut<T>(fd: RawFd, req: libc::c_ulong, arg: *mut T) -> i32 {
// SAFETY: as `ioctl_ptr`, but `arg` is a writable buffer the kernel fills for `req` (EVENT_FETCH / EP0_READ).
unsafe { libc::ioctl(fd, req as _, arg) as i32 }
}
fn ioctl_val(fd: RawFd, req: libc::c_ulong, val: libc::c_ulong) -> i32 {
// SAFETY: `req` (VBUS_DRAW) takes an integer argument by value; `fd` is our descriptor.
unsafe { libc::ioctl(fd, req as _, val) as i32 }
}
fn ioctl_none(fd: RawFd, req: libc::c_ulong) -> i32 {
// SAFETY: `req` (RUN / CONFIGURE / EP0_STALL) takes no argument, but raw_gadget rejects a non-zero
// `value` with EINVAL — pass an explicit 0 (an omitted vararg would be an indeterminate register).
unsafe { libc::ioctl(fd, req as _, 0) as i32 }
}
// ---- low-level ep0 helpers (operate on the shared fd) ----
fn ep0_write(fd: RawFd, data: &[u8]) -> i32 {
let mut buf = vec![0u8; EPIO_HDR + data.len()];
buf[0..2].copy_from_slice(&0u16.to_ne_bytes()); // ep 0
buf[4..8].copy_from_slice(&(data.len() as u32).to_ne_bytes());
buf[EPIO_HDR..].copy_from_slice(data);
ioctl_ptr(fd, IOCTL_EP0_WRITE, buf.as_ptr())
}
fn ep0_read(fd: RawFd, len: usize) -> (i32, Vec<u8>) {
let mut buf = vec![0u8; EPIO_HDR + len.max(1)];
buf[4..8].copy_from_slice(&(len as u32).to_ne_bytes());
let r = ioctl_mut(fd, IOCTL_EP0_READ, buf.as_mut_ptr());
let n = if r > 0 { r as usize } else { 0 };
(r, buf[EPIO_HDR..EPIO_HDR + n.min(len.max(1))].to_vec())
}
/// Complete a no-data OUT control (status stage is an IN, handled by a zero-length read).
fn ep0_ack(fd: RawFd) {
ep0_read(fd, 0);
}
fn ep0_stall(fd: RawFd) {
ioctl_none(fd, IOCTL_EP0_STALL);
}
/// Owns the `/dev/raw-gadget` fd; closing it tears the device down.
struct GadgetFd(RawFd);
impl Drop for GadgetFd {
fn drop(&mut self) {
// SAFETY: `self.0` is the fd we opened in `SteamDeckGadget::open` and own uniquely here.
unsafe { libc::close(self.0) };
}
}
/// A virtual Steam Deck presented over the USB gadget subsystem. Dropping it stops the threads and
/// closes the gadget (the kernel tears down the device).
pub struct SteamDeckGadget {
report: Arc<Mutex<[u8; 64]>>,
feedback: Arc<Mutex<super::steam_proto::SteamFeedback>>,
running: Arc<AtomicBool>,
threads: Vec<JoinHandle<()>>,
_fd: Arc<GadgetFd>,
seq: u32,
}
impl SteamDeckGadget {
/// Bind a virtual Deck on a fresh `dummy_hcd` UDC. `index` only varies the serial. Requires
/// `dummy_hcd` + `raw_gadget` loaded and write access to `/dev/raw-gadget` (root on SteamOS).
pub fn open(index: u8) -> Result<SteamDeckGadget> {
// SAFETY: opening a constant NUL-terminated device path with O_RDWR; returns a fd or -1.
let fd = unsafe { libc::open(c"/dev/raw-gadget".as_ptr(), libc::O_RDWR) };
if fd < 0 {
bail!(
"open /dev/raw-gadget ({}) — is raw_gadget+dummy_hcd loaded and are we root?",
std::io::Error::last_os_error()
);
}
let fd = Arc::new(GadgetFd(fd));
let raw = fd.0;
// INIT against the dummy UDC, then RUN.
// SAFETY: `UsbRawInit` is a plain-old-data struct (byte arrays + u8); all-zero is a valid value.
let mut init: UsbRawInit = unsafe { std::mem::zeroed() };
copy_cstr(&mut init.driver_name, "dummy_udc");
copy_cstr(&mut init.device_name, "dummy_udc.0");
init.speed = USB_SPEED_HIGH;
if ioctl_ptr(raw, IOCTL_INIT, &init as *const _) < 0 {
bail!("raw_gadget INIT: {}", std::io::Error::last_os_error());
}
if ioctl_none(raw, IOCTL_RUN) < 0 {
bail!("raw_gadget RUN: {}", std::io::Error::last_os_error());
}
let serial = deck_serial(index);
let unit_id = deck_unit_id(index); // "PF" + index — a synthetic per-instance device id
let report = Arc::new(Mutex::new(neutral_deck_report()));
let feedback = Arc::new(Mutex::new(Default::default()));
let running = Arc::new(AtomicBool::new(true));
let ctrl_ep = Arc::new(std::sync::atomic::AtomicI32::new(-1));
let configured = Arc::new(AtomicBool::new(false));
// Control thread: enumerate + answer every control transfer.
let control = {
let fd = fd.clone();
let running = running.clone();
let ctrl_ep = ctrl_ep.clone();
let configured = configured.clone();
let feedback = feedback.clone();
std::thread::Builder::new()
.name("pf-deck-gadget-ctrl".into())
.spawn(move || {
control_loop(fd, running, ctrl_ep, configured, feedback, serial, unit_id)
})
.context("spawn gadget control thread")?
};
// Stream thread: push the current report on the controller interrupt-IN endpoint.
let stream = {
let fd = fd.clone();
let running = running.clone();
let ctrl_ep = ctrl_ep.clone();
let configured = configured.clone();
let report = report.clone();
std::thread::Builder::new()
.name("pf-deck-gadget-stream".into())
.spawn(move || stream_loop(fd, running, ctrl_ep, configured, report))
.context("spawn gadget stream thread")?
};
Ok(SteamDeckGadget {
report,
feedback,
running,
threads: vec![control, stream],
_fd: fd,
seq: 0,
})
}
/// Serialize `st` into the 64-byte Deck state report streamed to the kernel.
pub fn write_state(&mut self, st: &super::steam_proto::SteamState) {
self.seq = self.seq.wrapping_add(1);
let mut r = [0u8; 64];
super::steam_proto::serialize_deck_state(&mut r, st, self.seq);
if let Ok(mut g) = self.report.lock() {
*g = r;
}
}
/// Drain any feedback (rumble) the kernel/Steam wrote to the device.
pub fn service(&mut self) -> super::steam_proto::SteamFeedback {
self.feedback
.lock()
.map(|mut f| std::mem::take(&mut *f))
.unwrap_or_default()
}
}
impl Drop for SteamDeckGadget {
fn drop(&mut self) {
self.running.store(false, Ordering::SeqCst);
for t in self.threads.drain(..) {
let _ = t.join();
}
}
}
fn copy_cstr(dst: &mut [u8], s: &str) {
let n = s.len().min(dst.len() - 1);
dst[..n].copy_from_slice(&s.as_bytes()[..n]);
}
fn control_loop(
fd: Arc<GadgetFd>,
running: Arc<AtomicBool>,
ctrl_ep: Arc<std::sync::atomic::AtomicI32>,
configured: Arc<AtomicBool>,
feedback: Arc<Mutex<super::steam_proto::SteamFeedback>>,
serial: String,
unit_id: u32,
) {
let raw = fd.0;
let cfg = build_config();
let mut last_set: Vec<u8> = Vec::new();
let mut evbuf = [0u8; EVENT_BUF];
while running.load(Ordering::SeqCst) {
// EVENT_FETCH: type(4) length(4) data[].
evbuf[4..8].copy_from_slice(&(8u32).to_ne_bytes()); // request setup-sized payload
let r = ioctl_mut(raw, IOCTL_EVENT_FETCH, evbuf.as_mut_ptr());
if r < 0 {
if running.load(Ordering::SeqCst) {
// transient; brief backoff
std::thread::sleep(std::time::Duration::from_millis(2));
}
continue;
}
let etype = u32::from_ne_bytes([evbuf[0], evbuf[1], evbuf[2], evbuf[3]]);
match etype {
USB_RAW_EVENT_CONNECT => {}
USB_RAW_EVENT_CONTROL => {
let s = &evbuf[EVENT_HDR..EVENT_HDR + 8];
let ctrl = Setup {
bm_request_type: s[0],
b_request: s[1],
w_value: u16::from_le_bytes([s[2], s[3]]),
w_index: u16::from_le_bytes([s[4], s[5]]),
w_length: u16::from_le_bytes([s[6], s[7]]),
};
handle_control(
raw,
&ctrl,
&cfg,
&serial,
unit_id,
&ctrl_ep,
&configured,
&mut last_set,
&feedback,
);
}
_ => {}
}
}
}
struct Setup {
bm_request_type: u8,
b_request: u8,
w_value: u16,
w_index: u16,
w_length: u16,
}
#[allow(clippy::too_many_arguments)]
fn handle_control(
raw: RawFd,
ctrl: &Setup,
cfg: &[u8],
serial: &str,
unit_id: u32,
ctrl_ep: &std::sync::atomic::AtomicI32,
configured: &AtomicBool,
last_set: &mut Vec<u8>,
feedback: &Mutex<super::steam_proto::SteamFeedback>,
) {
let idx = (ctrl.w_index & 0xff) as u8;
let type_class = ctrl.bm_request_type & 0x60;
let wl = ctrl.w_length as usize;
if type_class == 0x00 {
// standard
match ctrl.b_request {
0x06 => {
// GET_DESCRIPTOR
let dt = (ctrl.w_value >> 8) as u8;
let di = (ctrl.w_value & 0xff) as u8;
let resp: Vec<u8> = match dt {
1 => DEV_DESC.to_vec(),
2 => cfg.to_vec(),
3 => string_desc(di, serial),
HID_RPT_DT => match idx {
0 => RDESC_MOUSE.to_vec(),
1 => RDESC_KBD.to_vec(),
_ => RDESC_CTRL.to_vec(),
},
HID_DT => {
// re-emit the interface's HID descriptor from the config blob (best effort)
hid_desc_for(cfg, idx)
}
_ => {
ep0_stall(raw);
return;
}
};
let n = resp.len().min(wl);
ep0_write(raw, &resp[..n]);
}
0x09 => {
// SET_CONFIGURATION
ioctl_val(raw, IOCTL_VBUS_DRAW, 0x32);
ioctl_none(raw, IOCTL_CONFIGURE);
enable_endpoints(raw, ctrl_ep);
ep0_ack(raw);
configured.store(true, Ordering::SeqCst);
}
0x0b => ep0_ack(raw), // SET_INTERFACE
0x00 => {
let st = 0u16;
ep0_write(raw, &st.to_le_bytes());
}
_ => ep0_stall(raw),
}
} else if type_class == 0x20 {
// HID class
match ctrl.b_request {
0x01 => {
// GET_REPORT — serve the Deck feature reply for the last requested command.
let resp = feature_reply(last_set, serial, unit_id);
let n = resp.len().min(wl);
ep0_write(raw, &resp[..n]);
}
0x09 => {
// SET_REPORT — read the host's data; remember it + extract feedback.
let (r, data) = ep0_read(raw, wl);
if r > 0 {
*last_set = data.clone();
// parse_steam_output expects [report-id(0), cmd, …]; EP0 OUT data is [cmd, …].
let mut framed = Vec::with_capacity(data.len() + 1);
framed.push(0);
framed.extend_from_slice(&data);
let fb = super::steam_proto::parse_steam_output(&framed);
if fb.rumble.is_some() {
if let Ok(mut g) = feedback.lock() {
*g = fb;
}
}
}
}
0x0a | 0x0b => ep0_ack(raw), // SET_IDLE / SET_PROTOCOL
0x03 => {
ep0_write(raw, &[0u8]);
} // GET_PROTOCOL
_ => ep0_stall(raw),
}
} else {
ep0_stall(raw);
}
}
fn hid_desc_for(cfg: &[u8], idx: u8) -> Vec<u8> {
// The HID descriptors live right after each interface descriptor in the config blob.
// Offsets: cfg(9) | i0(9) h0(9) e0(7) | i1(9) h1(9) e1(7) | i2(9) h2(9) e2(7)
let off = match idx {
0 => 9 + 9,
1 => 9 + 25 + 9,
_ => 9 + 50 + 9,
};
cfg.get(off..off + 9)
.map(|s| s.to_vec())
.unwrap_or_default()
}
fn enable_endpoints(raw: RawFd, ctrl_ep: &std::sync::atomic::AtomicI32) {
let mk = |addr: u8, mps: u16| UsbEndpointDescriptor {
b_length: 7,
b_descriptor_type: 5,
b_endpoint_address: addr,
bm_attributes: 0x03,
w_max_packet_size: mps,
b_interval: 4,
..Default::default()
};
let e0 = mk(0x81, 8);
let e1 = mk(0x82, 8);
let e2 = mk(0x83, 64);
ioctl_ptr(raw, IOCTL_EP_ENABLE, &e0 as *const _);
ioctl_ptr(raw, IOCTL_EP_ENABLE, &e1 as *const _);
let h2 = ioctl_ptr(raw, IOCTL_EP_ENABLE, &e2 as *const _);
ctrl_ep.store(h2, Ordering::SeqCst);
}
fn stream_loop(
fd: Arc<GadgetFd>,
running: Arc<AtomicBool>,
ctrl_ep: Arc<std::sync::atomic::AtomicI32>,
configured: Arc<AtomicBool>,
report: Arc<Mutex<[u8; 64]>>,
) {
let raw = fd.0;
while running.load(Ordering::SeqCst) {
let ep = ctrl_ep.load(Ordering::SeqCst);
if configured.load(Ordering::SeqCst) && ep >= 0 {
let r = report
.lock()
.map(|g| *g)
.unwrap_or_else(|_| neutral_deck_report());
let mut buf = [0u8; EPIO_HDR + 64];
buf[0..2].copy_from_slice(&(ep as u16).to_ne_bytes());
buf[4..8].copy_from_slice(&(64u32).to_ne_bytes());
buf[EPIO_HDR..].copy_from_slice(&r);
// Blocks until the host polls the interrupt-IN endpoint; that's fine on its own thread.
ioctl_ptr(raw, IOCTL_EP_WRITE, buf.as_ptr());
}
std::thread::sleep(std::time::Duration::from_millis(8));
}
}
/// Best-effort load of the gadget modules (SteamOS ships `dummy_hcd` + `raw_gadget`). Failures are
/// ignored — the caller falls back to UHID if `/dev/raw-gadget` is then still unusable.
pub fn ensure_modules() {
for m in ["dummy_hcd", "raw_gadget"] {
let _ = std::process::Command::new("modprobe").arg(m).status();
}
}
/// Whether to prefer the USB-gadget Deck over the UHID `SteamDeckPad` — the only transport Steam Input
/// promotes (validated glass-to-glass on a Deck). Defaults **on for SteamOS** hosts (which ship the
/// gadget modules + run Steam Input); off elsewhere, where the universal UHID path stays the default.
/// `PUNKTFUNK_STEAM_GADGET=1`/`0` forces it on/off. A Deck-as-host with a *physical* Deck never reaches
/// here: `resolve_gamepad`'s conflict gate degrades `SteamDeck` → DualSense before the manager is built.
pub fn gadget_preferred() -> bool {
if let Ok(v) = std::env::var("PUNKTFUNK_STEAM_GADGET") {
return v == "1" || v.eq_ignore_ascii_case("true");
}
is_steamos()
}
/// True on SteamOS-class hosts (`/etc/os-release` `ID=steamos`, or `ID_LIKE` naming it).
fn is_steamos() -> bool {
std::fs::read_to_string("/etc/os-release")
.map(|s| {
s.lines()
.any(|l| l == "ID=steamos" || (l.starts_with("ID_LIKE=") && l.contains("steamos")))
})
.unwrap_or(false)
}
@@ -0,0 +1,733 @@
//! Virtual Steam Deck over **USB/IP** (`vhci_hcd`) — the shippable, Secure-Boot-clean, universal
//! alternative to [`super::steam_gadget`] (`raw_gadget` + `dummy_hcd`, SteamOS-only).
//!
//! Like the gadget, this presents a *real* 3-interface USB Steam Deck (mouse = interface 0, keyboard
//! = 1, **controller = 2**) — the interface-2 layout Steam's own driver filters on, so Steam Input
//! promotes it (a UHID Deck, `Interface: -1`, never is). Unlike the gadget it needs no out-of-tree
//! module: `vhci_hcd` is in-tree + signed on SteamOS, Bazzite, and ~every distro, loads under Secure
//! Boot, and needs no MOK. A userspace [`usbip_sim`] server emulates the Deck; the local `vhci_hcd`
//! attaches it. **Validated on Bazzite**: `vhci_hcd` enumerates the 3-interface Deck, `hid-steam`
//! binds it, and Steam reserves an XInput slot — identical recognition to the gadget.
//!
//! The device model + the USB/IP protocol come from the vendored [`usbip_sim`] crate (the upstream
//! `usbip` crate trimmed of its libusb host mode); the captured descriptors + the `0x83`/`0xAE`
//! feature contract come from the shared [`super::steam_proto`] (one source of truth with the gadget).
//!
//! **Attach** is in-process by default (no external `usbip` CLI dependency — the production goal): we
//! run the emulation server on a loopback TCP port, connect to it ourselves, perform the
//! `OP_REQ_IMPORT` handshake, then hand the connected socket fd to `vhci_hcd` via its sysfs `attach`
//! file. If anything in that path fails we fall back to the widely-packaged `usbip` CLI; if *that*
//! also fails, [`open`](SteamDeckUsbip::open) returns `Err` and the caller degrades to UHID.
use super::steam_proto::{
deck_serial, deck_unit_id, feature_reply, neutral_deck_report, parse_steam_output,
SteamFeedback, SteamState, RDESC_DECK_CTRL, RDESC_DECK_KBD, RDESC_DECK_MOUSE,
};
use anyhow::{bail, Context, Result};
use std::any::Any;
use std::collections::HashSet;
use std::io::{Read, Write};
use std::net::TcpStream;
use std::os::fd::AsRawFd;
use std::path::{Path, PathBuf};
use std::process::Command;
use std::sync::{Arc, Mutex};
use std::thread::JoinHandle;
use std::time::{Duration, Instant};
use usbip_sim::{
Direction, SetupPacket, UsbDevice, UsbEndpoint, UsbInterface, UsbInterfaceHandler, UsbIpServer,
Version,
};
const STEAM_VENDOR: u16 = 0x28DE;
const STEAMDECK_PRODUCT: u16 = 0x1205;
/// The single device's USB/IP bus id (one device per server, so the fixed default is fine).
const BUS_ID: &str = "0-0-0";
/// The usbip default TCP port — the server must listen here for the `usbip` CLI fallback to attach.
const USBIP_TCP_PORT: u16 = 3240;
/// Build the 9-byte HID class descriptor inserted between the interface and endpoint descriptors.
fn hid_desc(report_len: usize, country: u8) -> Vec<u8> {
let l = report_len as u16;
#[rustfmt::skip]
let d = vec![0x09, 0x21, 0x10, 0x01, country, 1, 0x22, (l & 0xff) as u8, (l >> 8) as u8];
d
}
/// The Deck **controller** interface (vendor HID, interface 2): answers the HID feature reports
/// (descriptor / `0x83` attributes / `0xAE` serial), streams the current 64-byte state on the
/// interrupt-IN endpoint, and surfaces rumble written via SET_REPORT.
#[derive(Debug)]
struct ControllerHandler {
/// The current 64-byte Deck input report, shared with [`SteamDeckUsbip::write_state`].
report: Arc<Mutex<[u8; 64]>>,
/// Rumble extracted from the kernel's SET_REPORTs, drained by [`SteamDeckUsbip::service`].
feedback: Arc<Mutex<SteamFeedback>>,
/// The host's last SET_REPORT command (drives [`feature_reply`]).
last_set: Vec<u8>,
serial: String,
unit_id: u32,
}
impl UsbInterfaceHandler for ControllerHandler {
fn get_class_specific_descriptor(&self) -> Vec<u8> {
hid_desc(RDESC_DECK_CTRL.len(), 33)
}
fn handle_urb(
&mut self,
_interface: &UsbInterface,
ep: UsbEndpoint,
_len: u32,
setup: SetupPacket,
req: &[u8],
) -> std::io::Result<Vec<u8>> {
if ep.is_ep0() {
Ok(match (setup.request_type, setup.request) {
// GET report descriptor (standard, interface recipient).
(0x81, 0x06) if (setup.value >> 8) == 0x22 => RDESC_DECK_CTRL.to_vec(),
// HID GET_REPORT (feature) — the Deck `0x83`/`0xAE` contract.
(0xA1, 0x01) => feature_reply(&self.last_set, &self.serial, self.unit_id).to_vec(),
// HID SET_REPORT — remember the command (for the next feature reply) + surface rumble.
(0x21, 0x09) => {
self.last_set = req.to_vec();
// `parse_steam_output` expects `[report-id(0), cmd, …]`; EP0 OUT data is `[cmd, …]`.
let mut framed = Vec::with_capacity(req.len() + 1);
framed.push(0);
framed.extend_from_slice(req);
let fb = parse_steam_output(&framed);
if fb.rumble.is_some() {
if let Ok(mut g) = self.feedback.lock() {
*g = fb;
}
}
vec![]
}
(0x21, 0x0A) | (0x21, 0x0B) => vec![], // SET_IDLE / SET_PROTOCOL
_ => vec![],
})
} else if let Direction::In = ep.direction() {
// Interrupt-IN poll: return the current report. The vendored sim paces interrupt-IN by
// bInterval (vhci_hcd does NOT throttle the server side), so this isn't a busy spin.
let r = self
.report
.lock()
.map(|g| *g)
.unwrap_or_else(|_| neutral_deck_report());
Ok(r.to_vec())
} else {
Ok(vec![])
}
}
fn as_any(&mut self) -> &mut dyn Any {
self
}
}
/// A minimal idle HID interface (mouse / keyboard) — serves only its report descriptor.
#[derive(Debug)]
struct IdleHidHandler {
report_desc: Vec<u8>,
}
impl UsbInterfaceHandler for IdleHidHandler {
fn get_class_specific_descriptor(&self) -> Vec<u8> {
hid_desc(self.report_desc.len(), 0)
}
fn handle_urb(
&mut self,
_i: &UsbInterface,
ep: UsbEndpoint,
_l: u32,
setup: SetupPacket,
_req: &[u8],
) -> std::io::Result<Vec<u8>> {
if ep.is_ep0() && setup.request == 0x06 && (setup.value >> 8) == 0x22 {
Ok(self.report_desc.clone())
} else {
Ok(vec![])
}
}
fn as_any(&mut self) -> &mut dyn Any {
self
}
}
fn boxed(
h: impl UsbInterfaceHandler + Send + 'static,
) -> Arc<Mutex<Box<dyn UsbInterfaceHandler + Send>>> {
Arc::new(Mutex::new(Box::new(h)))
}
fn ep(addr: u8, mps: u16) -> UsbEndpoint {
UsbEndpoint {
address: addr,
attributes: 0x03, // interrupt
max_packet_size: mps,
interval: 4,
}
}
/// Assemble the simulated 3-interface USB Deck. The controller handler shares `report` + `feedback`
/// with the owning [`SteamDeckUsbip`].
fn build_device(
index: u8,
report: &Arc<Mutex<[u8; 64]>>,
feedback: &Arc<Mutex<SteamFeedback>>,
) -> UsbDevice {
let mut dev = UsbDevice::new(0); // one device per server; bus_id stays the default "0-0-0".
dev.vendor_id = STEAM_VENDOR;
dev.product_id = STEAMDECK_PRODUCT;
dev.usb_version = Version::from(0x0200u16); // bcdUSB 2.00
dev.device_bcd = Version::from(0x0300u16); // bcdDevice 3.00 (matches the gadget)
dev.set_manufacturer_name("Valve Software");
dev.set_product_name("Steam Deck Controller");
dev.set_serial_number(&deck_serial(index));
dev.with_interface(
0x03,
0x00,
0x02,
Some("mouse"),
vec![ep(0x81, 8)],
boxed(IdleHidHandler {
report_desc: RDESC_DECK_MOUSE.to_vec(),
}),
)
.with_interface(
0x03,
0x01,
0x01,
Some("keyboard"),
vec![ep(0x82, 8)],
boxed(IdleHidHandler {
report_desc: RDESC_DECK_KBD.to_vec(),
}),
)
.with_interface(
0x03,
0x00,
0x00,
Some("controller"),
vec![ep(0x83, 64)],
boxed(ControllerHandler {
report: report.clone(),
feedback: feedback.clone(),
last_set: vec![],
serial: deck_serial(index),
unit_id: deck_unit_id(index),
}),
)
}
/// Owns the emulation-server thread (a dedicated current-thread tokio runtime) and stops it on drop.
/// Run on its own thread so `SteamDeckUsbip::open` works whether or not the caller is inside a tokio
/// runtime (creating a runtime inside one would panic).
struct ServerThread {
stop: Arc<tokio::sync::Notify>,
join: Option<JoinHandle<()>>,
}
impl ServerThread {
/// Spawn the server on `listener`, serving exactly the one simulated `dev`.
fn spawn(listener: std::net::TcpListener, dev: UsbDevice) -> Result<ServerThread> {
let stop = Arc::new(tokio::sync::Notify::new());
let stop_t = stop.clone();
let join = std::thread::Builder::new()
.name("pf-deck-usbip".into())
.spawn(move || {
let rt = match tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
{
Ok(rt) => rt,
Err(e) => {
tracing::error!(error = %e, "usbip server runtime build failed");
return;
}
};
rt.block_on(run_server(
listener,
Arc::new(UsbIpServer::new_simulated(vec![dev])),
stop_t,
));
})
.context("spawn usbip server thread")?;
Ok(ServerThread {
stop,
join: Some(join),
})
}
}
impl Drop for ServerThread {
fn drop(&mut self) {
self.stop.notify_one();
if let Some(j) = self.join.take() {
let _ = j.join();
}
}
}
/// Accept loop: serve each USB/IP connection with the vendored `usbip_sim::handler` until stopped.
async fn run_server(
listener: std::net::TcpListener,
server: Arc<UsbIpServer>,
stop: Arc<tokio::sync::Notify>,
) {
let listener = match tokio::net::TcpListener::from_std(listener) {
Ok(l) => l,
Err(e) => {
tracing::error!(error = %e, "usbip TcpListener::from_std failed");
return;
}
};
loop {
tokio::select! {
_ = stop.notified() => break,
r = listener.accept() => match r {
Ok((mut sock, _)) => {
let server = server.clone();
tokio::spawn(async move {
let _ = usbip_sim::handler(&mut sock, server).await;
});
}
Err(e) => {
tracing::warn!(error = %e, "usbip accept error");
break;
}
}
}
}
}
/// A virtual Steam Deck presented over USB/IP. Dropping it detaches the `vhci_hcd` port (the device
/// disappears, Steam releases its slot) and stops the emulation server.
pub struct SteamDeckUsbip {
report: Arc<Mutex<[u8; 64]>>,
feedback: Arc<Mutex<SteamFeedback>>,
/// The `vhci_hcd` port we attached to — written to the sysfs `detach` file on drop.
vhci_port: u16,
/// Kept alive so the connected socket fd we handed to `vhci_hcd` stays valid (in-process attach
/// only; the CLI hands its own fd to the kernel and exits, so this is `None` there).
_client_sock: Option<TcpStream>,
/// Emulation-server thread; dropped (stopped) after the detach.
_server: ServerThread,
seq: u32,
}
impl SteamDeckUsbip {
/// Bind a virtual Deck and attach it locally via `vhci_hcd`. `index` varies only the serial.
/// Requires `vhci_hcd` loaded and root (the sysfs attach / the CLI both need it). Tries the
/// in-process sysfs attach first, then the `usbip` CLI; `PUNKTFUNK_USBIP_ATTACH=inproc|cli`
/// pins one path (for debugging).
pub fn open(index: u8) -> Result<SteamDeckUsbip> {
ensure_modules();
if vhci_base().is_none() {
bail!(
"vhci_hcd unavailable (no /sys/devices/platform/vhci_hcd*/status) — is it loaded?"
);
}
let mode = std::env::var("PUNKTFUNK_USBIP_ATTACH").ok();
if mode.as_deref() != Some("cli") {
match Self::open_in_process(index) {
Ok(d) => return Ok(d),
Err(e) if mode.as_deref() == Some("inproc") => return Err(e),
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "in-process vhci attach failed — trying the usbip CLI")
}
}
}
Self::open_via_cli(index)
}
/// In-process attach: emulate on a loopback port, do the import handshake ourselves, hand the
/// connected socket to `vhci_hcd` via sysfs. No external dependency.
fn open_in_process(index: u8) -> Result<SteamDeckUsbip> {
let report = Arc::new(Mutex::new(neutral_deck_report()));
let feedback = Arc::new(Mutex::new(SteamFeedback::default()));
let dev = build_device(index, &report, &feedback);
// An ephemeral loopback port (avoids contending the usbip default with another pad).
let listener =
std::net::TcpListener::bind(("127.0.0.1", 0)).context("bind loopback usbip server")?;
let port = listener
.local_addr()
.context("usbip server local_addr")?
.port();
listener
.set_nonblocking(true)
.context("usbip listener set_nonblocking")?;
let server = ServerThread::spawn(listener, dev)?;
// Connect to our own server and run the OP_REQ_IMPORT handshake.
let mut sock = connect_loopback(port).context("connect to usbip server")?;
let (devid, speed) = import_handshake(&mut sock).context("usbip import handshake")?;
// Hand the connected socket to vhci_hcd. Clear BOTH timeouts first: the kernel's vhci rx/tx
// threads honour SO_RCVTIMEO/SO_SNDTIMEO on this socket, so the 3s handshake timeouts would
// otherwise tear the device down after 3s idle (rx) or a 3s-blocked send (tx).
let vhci_port = vhci_find_free_port(speed).context("find a free vhci port")?;
sock.set_read_timeout(None).ok();
sock.set_write_timeout(None).ok();
vhci_attach(vhci_port, sock.as_raw_fd(), devid, speed).context("write vhci_hcd attach")?;
tracing::info!(
index,
vhci_port,
"virtual Steam Deck attached via usbip (in-process — Steam Input recognizes it)"
);
Ok(SteamDeckUsbip {
report,
feedback,
vhci_port,
_client_sock: Some(sock),
_server: server,
seq: 0,
})
}
/// Fallback: emulate on the usbip default port and let the `usbip` CLI attach (it picks the vhci
/// port itself; we recover it by diffing the sysfs status).
fn open_via_cli(index: u8) -> Result<SteamDeckUsbip> {
let report = Arc::new(Mutex::new(neutral_deck_report()));
let feedback = Arc::new(Mutex::new(SteamFeedback::default()));
let dev = build_device(index, &report, &feedback);
let listener = std::net::TcpListener::bind(("127.0.0.1", USBIP_TCP_PORT))
.with_context(|| format!("bind usbip default port {USBIP_TCP_PORT} for CLI attach"))?;
listener
.set_nonblocking(true)
.context("usbip listener set_nonblocking")?;
let server = ServerThread::spawn(listener, dev)?;
let before = vhci_used_ports();
usbip_attach_cli().context("usbip CLI attach")?;
let vhci_port = wait_for_new_port(&before)
.context("could not determine the vhci port the usbip CLI attached to")?;
tracing::info!(
index,
vhci_port,
"virtual Steam Deck attached via usbip (CLI — Steam Input recognizes it)"
);
Ok(SteamDeckUsbip {
report,
feedback,
vhci_port,
_client_sock: None,
_server: server,
seq: 0,
})
}
/// Serialize `st` into the 64-byte Deck report streamed on the controller interrupt-IN endpoint.
pub fn write_state(&mut self, st: &SteamState) {
self.seq = self.seq.wrapping_add(1);
let mut r = [0u8; 64];
super::steam_proto::serialize_deck_state(&mut r, st, self.seq);
if let Ok(mut g) = self.report.lock() {
*g = r;
}
}
/// Drain any rumble feedback the kernel/Steam wrote to the device.
pub fn service(&mut self) -> SteamFeedback {
self.feedback
.lock()
.map(|mut f| std::mem::take(&mut *f))
.unwrap_or_default()
}
}
impl Drop for SteamDeckUsbip {
fn drop(&mut self) {
// Detach the vhci port first (the kernel closes its end of the socket + tears down the
// device); `_client_sock` + `_server` then drop, closing our side + stopping the server.
if let Err(e) = vhci_detach(self.vhci_port) {
tracing::debug!(port = self.vhci_port, error = %e, "vhci detach failed (device may already be gone)");
}
}
}
// ---- USB/IP import handshake (we act as the usbip *client* before handing the fd to the kernel) ----
const USBIP_VERSION: u16 = 0x0111;
const OP_REQ_IMPORT: u16 = 0x8003;
/// Connect to our own loopback server, retrying briefly while the server thread comes up.
fn connect_loopback(port: u16) -> Result<TcpStream> {
let addr = ("127.0.0.1", port);
let mut last = None;
for _ in 0..50 {
match TcpStream::connect(addr) {
Ok(s) => {
s.set_nodelay(true).ok();
return Ok(s);
}
Err(e) => {
last = Some(e);
std::thread::sleep(Duration::from_millis(10));
}
}
}
Err(anyhow::anyhow!(
"connect 127.0.0.1:{port}: {}",
last.map(|e| e.to_string()).unwrap_or_default()
))
}
/// Send `OP_REQ_IMPORT` for [`BUS_ID`] and read `OP_REP_IMPORT`, returning `(devid, speed)` parsed
/// from the device record (the same `devid = bus_num<<16 | dev_num` + speed `vhci_hcd` wants). The
/// whole 320-byte reply MUST be consumed here so the socket starts clean at the kernel's first
/// `USBIP_CMD_SUBMIT`.
fn import_handshake(sock: &mut TcpStream) -> Result<(u32, u32)> {
// Bounded so a non-responsive server can't head-block the per-session input thread (this talks
// to our own in-process loopback server, so a working handshake completes in well under a ms).
sock.set_read_timeout(Some(Duration::from_secs(1))).ok();
sock.set_write_timeout(Some(Duration::from_secs(1))).ok();
let mut req = Vec::with_capacity(40);
req.extend_from_slice(&USBIP_VERSION.to_be_bytes());
req.extend_from_slice(&OP_REQ_IMPORT.to_be_bytes());
req.extend_from_slice(&0u32.to_be_bytes()); // status
let mut busid = [0u8; 32];
let b = BUS_ID.as_bytes();
busid[..b.len()].copy_from_slice(b);
req.extend_from_slice(&busid);
sock.write_all(&req).context("send OP_REQ_IMPORT")?;
// Reply: version(2) code(2) status(4), then the 312-byte device record on success.
let mut header = [0u8; 8];
sock.read_exact(&mut header)
.context("read OP_REP_IMPORT header")?;
let status = u32::from_be_bytes([header[4], header[5], header[6], header[7]]);
if status != 0 {
bail!("OP_REP_IMPORT refused (status={status}) — device {BUS_ID} not exported?");
}
let mut dev = [0u8; 312];
sock.read_exact(&mut dev)
.context("read OP_REP_IMPORT device record")?;
// Device record layout: path[256], bus_id[32], bus_num(4 BE)@288, dev_num(4 BE)@292, speed(4)@296.
let be = |o: usize| u32::from_be_bytes([dev[o], dev[o + 1], dev[o + 2], dev[o + 3]]);
let bus_num = be(288);
let dev_num = be(292);
let speed = be(296);
Ok(((bus_num << 16) | dev_num, speed))
}
// ---- vhci_hcd sysfs plumbing ----
/// Best-effort load of `vhci_hcd` (in-tree + signed on SteamOS/Bazzite/most distros).
pub fn ensure_modules() {
let _ = Command::new("modprobe").arg("vhci_hcd").status();
}
/// Run `usbip attach -r 127.0.0.1 -b 0-0-0`, bounded by a deadline so a hung CLI can't head-block
/// the per-session input thread indefinitely (the caller runs this inline on that thread).
fn usbip_attach_cli() -> Result<()> {
let mut child = Command::new("usbip")
.args(["attach", "-r", "127.0.0.1", "-b", BUS_ID])
.spawn()
.context("spawn `usbip attach` (is usbip-utils installed?)")?;
let deadline = Instant::now() + Duration::from_secs(6);
loop {
match child.try_wait().context("wait on `usbip attach`")? {
Some(st) if st.success() => return Ok(()),
Some(st) => bail!("`usbip attach` exited with {st}"),
None if Instant::now() >= deadline => {
let _ = child.kill();
let _ = child.wait();
bail!("`usbip attach` timed out (>6s) — killed");
}
None => std::thread::sleep(Duration::from_millis(20)),
}
}
}
/// Whether a usbip attach should be attempted at all. Default on (the universal Steam-promotable
/// transport on non-SteamOS hosts); `PUNKTFUNK_STEAM_USBIP=0` forces it off, `=1` forces it on.
/// [`open`](SteamDeckUsbip::open) still degrades gracefully if `vhci_hcd` turns out to be absent.
pub fn usbip_preferred() -> bool {
!matches!(
std::env::var("PUNKTFUNK_STEAM_USBIP").ok().as_deref(),
Some("0") | Some("false")
)
}
/// The `vhci_hcd.0` (or legacy `vhci_hcd`) platform sysfs directory, if present.
fn vhci_base() -> Option<PathBuf> {
for p in [
"/sys/devices/platform/vhci_hcd.0",
"/sys/devices/platform/vhci_hcd",
] {
let base = Path::new(p);
if base.join("status").exists() {
return Some(base.to_path_buf());
}
}
None
}
fn read_status() -> Result<String> {
let base = vhci_base().context("vhci_hcd sysfs not present")?;
std::fs::read_to_string(base.join("status")).context("read vhci_hcd status")
}
/// One parsed `status` row: `(port, hub_is_superspeed, sta)`. Handles both the modern
/// `hub port sta …` and the legacy `port sta …` column layouts; returns `None` for header/blank rows.
fn parse_status_row(line: &str) -> Option<(u16, bool, u32)> {
let t: Vec<&str> = line.split_whitespace().collect();
if t.is_empty() {
return None;
}
let (hub_ss, port_str, sta_str) = if t[0] == "hs" || t[0] == "ss" {
(Some(t[0] == "ss"), *t.get(1)?, *t.get(2)?)
} else if t[0].chars().all(|c| c.is_ascii_digit()) {
(None, t[0], *t.get(1)?) // legacy: port sta …
} else {
return None; // header ("hub"/"prt"/"port" …)
};
let port = port_str.parse::<u16>().ok()?;
let sta = sta_str.parse::<u32>().ok()?;
Some((port, hub_ss.unwrap_or(false), sta))
}
/// `sta == 4` is `VDEV_ST_NULL` (a free port).
const VDEV_ST_NULL: u32 = 4;
/// Pick a free `vhci_hcd` port matching the device speed (`usbip_speed >= 5` ⇒ SuperSpeed hub).
fn vhci_find_free_port(usbip_speed: u32) -> Result<u16> {
let want_ss = usbip_speed >= 5;
let status = read_status()?;
for line in status.lines() {
if let Some((port, is_ss, sta)) = parse_status_row(line) {
if sta == VDEV_ST_NULL && is_ss == want_ss {
return Ok(port);
}
}
}
// Speed-class match failed (legacy single-hub status): take any free port.
for line in status.lines() {
if let Some((port, _, sta)) = parse_status_row(line) {
if sta == VDEV_ST_NULL {
return Ok(port);
}
}
}
bail!("no free vhci_hcd port (all ports in use?)")
}
/// Ports currently in use (`sta != VDEV_ST_NULL`) — snapshotted around a CLI attach to recover its port.
fn vhci_used_ports() -> HashSet<u16> {
read_status()
.unwrap_or_default()
.lines()
.filter_map(parse_status_row)
.filter(|&(_, _, sta)| sta != VDEV_ST_NULL)
.map(|(port, _, _)| port)
.collect()
}
/// Poll the status file (briefly) for a port that became used since `before` — the one the CLI attached.
fn wait_for_new_port(before: &HashSet<u16>) -> Result<u16> {
let deadline = Instant::now() + Duration::from_secs(2);
loop {
if let Some(p) = vhci_used_ports().difference(before).copied().min() {
return Ok(p);
}
if Instant::now() >= deadline {
bail!("no newly-attached vhci port appeared after `usbip attach`");
}
std::thread::sleep(Duration::from_millis(50));
}
}
fn vhci_attach(port: u16, sockfd: i32, devid: u32, speed: u32) -> Result<()> {
let base = vhci_base().context("vhci_hcd sysfs not present")?;
let line = format!("{port} {sockfd} {devid} {speed}");
std::fs::write(base.join("attach"), line)
.with_context(|| format!("write vhci_hcd attach (port {port}) — root?"))
}
fn vhci_detach(port: u16) -> Result<()> {
let base = vhci_base().context("vhci_hcd sysfs not present")?;
std::fs::write(base.join("detach"), format!("{port}")).context("write vhci_hcd detach")
}
#[cfg(test)]
mod tests {
use super::*;
/// The `status` parser handles the modern `hub port sta …` layout, the legacy `port sta …`
/// layout, and skips header/blank lines — a slip here would mean attaching to a busy port.
#[test]
fn status_parser_handles_both_layouts() {
// modern
assert_eq!(
parse_status_row("hs 0000 004 000 00000000 000000 0-0"),
Some((0, false, 4))
);
assert_eq!(
parse_status_row("ss 0008 006 000 00000000 000000 0-0"),
Some((8, true, 6))
);
// legacy (no hub column)
assert_eq!(
parse_status_row("0001 004 000 00000000 000000 0-0"),
Some((1, false, 4))
);
// header / blank
assert_eq!(
parse_status_row("hub port sta spd dev sockfd local_busid"),
None
);
assert_eq!(parse_status_row(""), None);
}
/// A free HS port is preferred for an HS device; a free SS port for an SS device.
#[test]
fn free_port_selection_matches_speed() {
let status = "hub port sta spd dev sockfd local_busid\n\
hs 0000 006 000 00000000 000000 0-0\n\
hs 0001 004 000 00000000 000000 0-0\n\
ss 0008 004 000 00000000 000000 0-0\n";
// Reuse the parser directly (vhci_find_free_port reads sysfs; test the selection logic).
let hs = status
.lines()
.filter_map(parse_status_row)
.find(|&(_, is_ss, sta)| sta == VDEV_ST_NULL && !is_ss)
.map(|(p, _, _)| p);
let ss = status
.lines()
.filter_map(parse_status_row)
.find(|&(_, is_ss, sta)| sta == VDEV_ST_NULL && is_ss)
.map(|(p, _, _)| p);
assert_eq!(hs, Some(1));
assert_eq!(ss, Some(8));
}
/// On-box smoke test (needs root + `vhci_hcd`): attach a virtual Deck, confirm `hid-steam` binds
/// it (the `Steam Deck` evdev appears) and that it tears down on drop. `#[ignore]`d in CI.
#[test]
#[ignore = "attaches a real vhci_hcd device; needs root + vhci_hcd"]
fn usbip_deck_binds_and_tears_down() {
ensure_modules();
let mut pad = SteamDeckUsbip::open(0).expect("open SteamDeckUsbip (root + vhci_hcd?)");
let st = SteamState::from_gamepad(punktfunk_core::input::gamepad::BTN_A, 0, 0, 0, 0, 0, 0);
let start = Instant::now();
while start.elapsed() < Duration::from_millis(800) {
pad.write_state(&st);
let _ = pad.service();
std::thread::sleep(Duration::from_millis(8));
}
let devs = std::fs::read_to_string("/proc/bus/input/devices").unwrap_or_default();
assert!(
devs.contains("Steam Deck"),
"hid-steam did not bind the usbip Deck"
);
drop(pad);
std::thread::sleep(Duration::from_millis(300));
let devs = std::fs::read_to_string("/proc/bus/input/devices").unwrap_or_default();
assert!(
!devs.contains("Steam Deck Motion Sensors"),
"device not torn down on drop"
);
}
}
@@ -0,0 +1,684 @@
//! Transport-independent Steam Controller / Steam Deck HID contract — the Steam analogue of
//! [`super::dualsense_proto`]. The report descriptor, the command/feature IDs, the byte-exact
//! Deck input-report serializer, the `XInput`/rich-input → state mappers, and the rumble-feedback
//! parser. Pure logic, shared by the Linux UHID backend and (later) a Windows UMDF backend.
//!
//! **Layout source of truth:** the kernel `drivers/hid/hid-steam.c` `steam_do_deck_input_event`
//! (+ `steam_do_deck_sensors_event`) — every offset/bit/sign below is transcribed verbatim from
//! it and on-box-validated against kernel 7.0 (see `design/steam-controller-deck-support.md`).
//! M0 proved the device binds + parses; M1 (here) makes the serializer byte-exact.
//!
//! Three load-bearing details the DualSense path does NOT have:
//! * **report id 0 / unnumbered**: input reports are the raw 64 bytes starting `[0x01,0x00,0x09]`
//! (no report-id prefix); FEATURE get/set reports DO carry a leading `0x00` report-id byte
//! (`steam_send_report` does `memcpy(buf+1, cmd, …)`, `steam_recv_report` strips `buf[0]`).
//! * **`gamepad_mode` gate**: `steam_do_deck_input_event` early-returns when
//! `!gamepad_mode && lizard_mode` (the module param, default on). `gamepad_mode` starts false
//! and TOGGLES when [`btn::STEAM_MENU_RIGHT`] (`b9.6`, the mode-switch) is held ~450 ms while
//! no hidraw client is open. The backend enters gamepad mode at session start (pulse that bit,
//! or load `hid_steam lizard_mode=0`) — see the backend, not this module.
//! * **the `UHID_SET_REPORT` handshake** must be answered (DualSense omits it).
#![allow(dead_code)] // Some of the full model is consumed only once the M2 backend + M3 wire land.
use punktfunk_core::input::gamepad as gs;
use punktfunk_core::quic::RichInput;
/// Valve. `hid-steam` matches purely by VID/PID over `BUS_USB`.
pub const STEAM_VENDOR: u32 = 0x28DE;
/// Steam Deck built-in controller (same PID on LCD + OLED).
pub const STEAMDECK_PRODUCT: u32 = 0x1205;
/// Classic Steam Controller, wired (report id 1 / `ID_CONTROLLER_STATE`; a later model).
pub const STEAMCTRL_WIRED_PRODUCT: u32 = 0x1102;
/// The Steam HID state/command report is a fixed 64-byte, **unnumbered** (report-id-0) frame.
pub const STEAM_REPORT_LEN: usize = 64;
// Command IDs (drivers/hid/hid-steam.c), confirmed against the kernel source.
pub const ID_CLEAR_DIGITAL_MAPPINGS: u8 = 0x81;
pub const ID_GET_ATTRIBUTES_VALUES: u8 = 0x83;
pub const ID_SET_SETTINGS_VALUES: u8 = 0x87;
pub const ID_LOAD_DEFAULT_SETTINGS: u8 = 0x8E;
pub const ID_GET_DEVICE_INFO: u8 = 0xA1;
pub const ID_GET_STRING_ATTRIBUTE: u8 = 0xAE;
pub const ATTRIB_STR_UNIT_SERIAL: u8 = 0x01;
/// Host→client feedback: `steam_haptic_rumble` emits report `[0xEB, 9, …]` (FF_RUMBLE → trackpad
/// actuators / Deck motors). The Deck's rumble path; the classic SC also has `0x8F` pad pulses.
pub const ID_TRIGGER_RUMBLE_CMD: u8 = 0xEB;
pub const ID_TRIGGER_HAPTIC_PULSE: u8 = 0x8F;
/// Input report message types: SC = `ID_CONTROLLER_STATE`, Deck = `ID_CONTROLLER_DECK_STATE`.
pub const ID_CONTROLLER_STATE: u8 = 0x01;
pub const ID_CONTROLLER_DECK_STATE: u8 = 0x09;
/// Which Steam device identity to present. M1 implements the Deck fully; the classic Controller
/// (dual trackpads, report id 1, trackpad-only haptics) is a later identity behind the same path.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum SteamModel {
Deck,
Controller,
}
impl SteamModel {
pub fn product(self) -> u32 {
match self {
SteamModel::Deck => STEAMDECK_PRODUCT,
SteamModel::Controller => STEAMCTRL_WIRED_PRODUCT,
}
}
}
/// Minimal vendor-defined HID report descriptor: one application collection with a 64-byte input
/// report and a 64-byte feature report, both UNNUMBERED (report id 0). `hid-steam` is a raw-event
/// driver, so the field layout is cosmetic — but `steam_probe` requires `hid_parse` to succeed AND
/// a non-empty FEATURE report list (`steam_is_valve_interface`), so the feature item is mandatory.
#[rustfmt::skip]
pub const STEAMDECK_RDESC: &[u8] = &[
0x06, 0x00, 0xFF, // Usage Page (Vendor-Defined 0xFF00)
0x09, 0x01, // Usage (0x01)
0xA1, 0x01, // Collection (Application)
0x15, 0x00, // Logical Minimum (0)
0x26, 0xFF, 0x00, // Logical Maximum (255)
0x75, 0x08, // Report Size (8 bits)
0x95, 0x40, // Report Count (64)
0x09, 0x01, // Usage (0x01)
0x81, 0x02, // Input (Data,Var,Abs) — the 64-byte state report
0x09, 0x01, // Usage (0x01)
0x95, 0x40, // Report Count (64)
0xB1, 0x02, // Feature (Data,Var,Abs) — makes steam_is_valve_interface() true
0xC0, // End Collection
];
/// Deck button bits, indexed in the `u64` packed across report bytes 8..16 — bit `(byte-8)*8 + bit`,
/// transcribed verbatim from `steam_do_deck_input_event` (bytes 12 + 15 carry no buttons). Naming
/// follows the physical Deck control; the trailing comment is the kernel `BTN_*` it maps to.
pub mod btn {
// byte 8
pub const RT_FULL: u64 = 1 << 0; // BTN_TR2 — right trigger fully pressed
pub const LT_FULL: u64 = 1 << 1; // BTN_TL2 — left trigger fully pressed
pub const RB: u64 = 1 << 2; // BTN_TR — right shoulder
pub const LB: u64 = 1 << 3; // BTN_TL — left shoulder
pub const Y: u64 = 1 << 4;
pub const B: u64 = 1 << 5;
pub const X: u64 = 1 << 6;
pub const A: u64 = 1 << 7;
// byte 9
pub const DPAD_UP: u64 = 1 << 8;
pub const DPAD_RIGHT: u64 = 1 << 9;
pub const DPAD_LEFT: u64 = 1 << 10;
pub const DPAD_DOWN: u64 = 1 << 11;
pub const VIEW: u64 = 1 << 12; // BTN_SELECT — "menu left" (View / Back)
pub const STEAM: u64 = 1 << 13; // BTN_MODE — Steam logo button
pub const MENU: u64 = 1 << 14; // BTN_START — "menu right" (Start / Options)
pub const L5: u64 = 1 << 15; // BTN_GRIPL2 — left BOTTOM back grip
// byte 10
pub const R5: u64 = 1 << 16; // BTN_GRIPR2 — right BOTTOM back grip
pub const LPAD_CLICK: u64 = 1 << 17; // BTN_THUMB — left pad pressed (click)
pub const RPAD_CLICK: u64 = 1 << 18; // BTN_THUMB2 — right pad pressed (click)
pub const LPAD_TOUCH: u64 = 1 << 19; // gates ABS_HAT0 (left pad coords)
pub const RPAD_TOUCH: u64 = 1 << 20; // gates ABS_HAT1 (right pad coords)
pub const L3: u64 = 1 << 22; // BTN_THUMBL — left joystick click
// byte 11
pub const R3: u64 = 1 << 26; // BTN_THUMBR — right joystick click
// byte 13
pub const L4: u64 = 1 << 41; // BTN_GRIPL — left TOP back grip
pub const R4: u64 = 1 << 42; // BTN_GRIPR — right TOP back grip
pub const LJOY_TOUCH: u64 = 1 << 46;
pub const RJOY_TOUCH: u64 = 1 << 47;
// byte 14
pub const QAM: u64 = 1 << 50; // BTN_BASE — quick-access (…) button
/// `b9.6` doubles as the mode-switch: held ~450 ms (no hidraw client) it toggles `gamepad_mode`.
pub const STEAM_MENU_RIGHT: u64 = MENU;
}
/// Full virtual Steam Deck controller state. All analog fields are stored as the RAW little-endian
/// report values the kernel reads (so [`serialize_deck_state`] is a pure memcpy); the kernel applies
/// its own sign conventions on top (`ABS_Y = -raw`, etc.) — see [`SteamState::from_gamepad`].
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct SteamState {
/// Packed button bits (see [`btn`]); occupies report bytes 8..16.
pub buttons: u64,
/// Left / right joystick, raw s16 (report 48/50/52/54). The kernel negates the Y axes.
pub lx: i16,
pub ly: i16,
pub rx: i16,
pub ry: i16,
/// Left / right analog trigger, raw u16 (report 44/46 → ABS_HAT2Y/X).
pub lt: u16,
pub rt: u16,
/// Left / right trackpad position, raw s16, centred 0 (report 16/18/20/22). Only surfaced by
/// the kernel while the matching `*PAD_TOUCH` button bit is set.
pub lpad_x: i16,
pub lpad_y: i16,
pub rpad_x: i16,
pub rpad_y: i16,
pub lpad_pressure: u16,
pub rpad_pressure: u16,
/// IMU, raw s16. `accel`/`gyro` are `[X, Y, Z]`; the kernel maps them to ABS_X/Z/Y + ABS_RX/RZ/RY
/// (with Z/RZ negated) on the separate sensors evdev.
pub accel: [i16; 3],
pub gyro: [i16; 3],
}
impl SteamState {
pub fn neutral() -> SteamState {
SteamState::default()
}
/// Set/clear a button (or group) by its [`btn`] mask.
pub fn press(&mut self, mask: u64, down: bool) {
if down {
self.buttons |= mask;
} else {
self.buttons &= !mask;
}
}
/// Map an `XInput`/GameStream pad frame (button bitmask + i16 sticks + u8 triggers) into the Deck
/// state. Sticks pass through (the kernel negates Y, which yields the conventional direction —
/// validated on-box); triggers scale u8 0..255 → u16 0..32640 and set the full-pull bit when
/// pressed. Trackpad + motion + the back grips arrive separately ([`apply_rich`], the M3 wire).
pub fn from_gamepad(
buttons: u32,
lx: i16,
ly: i16,
rx: i16,
ry: i16,
lt: u8,
rt: u8,
) -> SteamState {
let on = |bit: u32| buttons & bit != 0;
let mut s = SteamState {
lx,
ly,
rx,
ry,
lt: (lt as u16) * 128,
rt: (rt as u16) * 128,
..SteamState::neutral()
};
let mut b = 0u64;
let set = |b: &mut u64, on: bool, m: u64| {
if on {
*b |= m;
}
};
set(&mut b, on(gs::BTN_A), btn::A);
set(&mut b, on(gs::BTN_B), btn::B);
set(&mut b, on(gs::BTN_X), btn::X);
set(&mut b, on(gs::BTN_Y), btn::Y);
set(&mut b, on(gs::BTN_LB), btn::LB);
set(&mut b, on(gs::BTN_RB), btn::RB);
set(&mut b, lt > 0, btn::LT_FULL);
set(&mut b, rt > 0, btn::RT_FULL);
set(&mut b, on(gs::BTN_BACK), btn::VIEW);
set(&mut b, on(gs::BTN_START), btn::MENU);
set(&mut b, on(gs::BTN_GUIDE), btn::STEAM);
set(&mut b, on(gs::BTN_LS_CLICK), btn::L3);
set(&mut b, on(gs::BTN_RS_CLICK), btn::R3);
set(&mut b, on(gs::BTN_DPAD_UP), btn::DPAD_UP);
set(&mut b, on(gs::BTN_DPAD_DOWN), btn::DPAD_DOWN);
set(&mut b, on(gs::BTN_DPAD_LEFT), btn::DPAD_LEFT);
set(&mut b, on(gs::BTN_DPAD_RIGHT), btn::DPAD_RIGHT);
// The DualSense touchpad-click wire bit maps to the Deck's RIGHT pad click (the pad that
// stands in for the DualSense touchpad — see apply_rich).
set(&mut b, on(gs::BTN_TOUCHPAD), btn::RPAD_CLICK);
// Back grips (the whole reason for the Deck identity): the wire paddle bits map to the four
// Deck grips — PADDLE1/2/3/4 = R4/L4/R5/L5 (see `input::gamepad`); MISC1 = the QAM '…' button.
set(&mut b, on(gs::BTN_PADDLE1), btn::R4);
set(&mut b, on(gs::BTN_PADDLE2), btn::L4);
set(&mut b, on(gs::BTN_PADDLE3), btn::R5);
set(&mut b, on(gs::BTN_PADDLE4), btn::L5);
set(&mut b, on(gs::BTN_MISC1), btn::QAM);
s.buttons = b;
s
}
/// Apply one rich client→host event into this state, preserving everything else. The single-pad
/// wire [`RichInput::Touchpad`] maps to the **right** trackpad (the Deck pad analogous to the
/// DualSense touchpad); the left pad arrives via the M3 `TouchpadEx` surface. [`RichInput::Motion`]
/// passes gyro/accel straight through (raw i16; cross-device unit scaling is M3).
pub fn apply_rich(&mut self, rich: RichInput) {
match rich {
RichInput::Touchpad { active, x, y, .. } => {
self.press(btn::RPAD_TOUCH, active);
// Normalized 0..=65535 (centre 32768) → the pad's centred s16 range.
self.rpad_x = ((x as i32) - 32768) as i16;
self.rpad_y = ((y as i32) - 32768) as i16;
}
RichInput::Motion { gyro, accel, .. } => {
// The wire carries DualSense-convention units (what every client capture emits); the
// Deck's hid-steam report wants 16 LSB/°·s + 16384 LSB/g, so rescale here.
let (g, a) = super::steam_remap::motion_wire_to_deck(gyro, accel);
self.gyro = g;
self.accel = a;
}
RichInput::TouchpadEx {
surface,
touch,
click,
x,
y,
..
} => {
// Steam pads are natively signed (centre 0), so x/y map straight in. surface 1 =
// left pad, anything else (0 single / 2 right) = right pad.
if surface == 1 {
self.press(btn::LPAD_TOUCH, touch);
self.press(btn::LPAD_CLICK, click);
self.lpad_x = x;
self.lpad_y = y;
} else {
self.press(btn::RPAD_TOUCH, touch);
self.press(btn::RPAD_CLICK, click);
self.rpad_x = x;
self.rpad_y = y;
}
}
}
}
}
/// Serialize the full Deck input report (`ID_CONTROLLER_DECK_STATE`) into the 64-byte unnumbered
/// frame `hid-steam` parses. Pure + byte-exact against `steam_do_deck_input_event`; the report-id
/// constant is `data[0]=0x01` (NOT a HID report id — this report is unnumbered).
pub fn serialize_deck_state(r: &mut [u8; STEAM_REPORT_LEN], st: &SteamState, seq: u32) {
r.fill(0);
r[0] = 0x01;
r[1] = 0x00;
r[2] = ID_CONTROLLER_DECK_STATE;
r[3] = 0x3C; // payload length; the kernel ignores it
r[4..8].copy_from_slice(&seq.to_le_bytes());
r[8..16].copy_from_slice(&st.buttons.to_le_bytes()); // bytes 8..16 (12+15 stay 0)
r[16..18].copy_from_slice(&st.lpad_x.to_le_bytes());
r[18..20].copy_from_slice(&st.lpad_y.to_le_bytes());
r[20..22].copy_from_slice(&st.rpad_x.to_le_bytes());
r[22..24].copy_from_slice(&st.rpad_y.to_le_bytes());
r[24..26].copy_from_slice(&st.accel[0].to_le_bytes()); // accel X → IMU ABS_X
r[26..28].copy_from_slice(&st.accel[1].to_le_bytes()); // accel Y → IMU ABS_Z (kernel negates)
r[28..30].copy_from_slice(&st.accel[2].to_le_bytes()); // accel Z → IMU ABS_Y
r[30..32].copy_from_slice(&st.gyro[0].to_le_bytes()); // gyro X → IMU ABS_RX
r[32..34].copy_from_slice(&st.gyro[1].to_le_bytes()); // gyro Y → IMU ABS_RZ (kernel negates)
r[34..36].copy_from_slice(&st.gyro[2].to_le_bytes()); // gyro Z → IMU ABS_RY
// 36..44 quaternion — left 0 (optional; the kernel does not surface it)
r[44..46].copy_from_slice(&st.lt.to_le_bytes()); // left trigger → ABS_HAT2Y
r[46..48].copy_from_slice(&st.rt.to_le_bytes()); // right trigger → ABS_HAT2X
r[48..50].copy_from_slice(&st.lx.to_le_bytes()); // left joystick X → ABS_X
r[50..52].copy_from_slice(&st.ly.to_le_bytes()); // left joystick Y → ABS_Y (kernel negates)
r[52..54].copy_from_slice(&st.rx.to_le_bytes()); // right joystick X → ABS_RX
r[54..56].copy_from_slice(&st.ry.to_le_bytes()); // right joystick Y → ABS_RY (kernel negates)
r[56..58].copy_from_slice(&st.lpad_pressure.to_le_bytes());
r[58..60].copy_from_slice(&st.rpad_pressure.to_le_bytes());
}
/// Build the `steam_get_serial` GET_REPORT reply. The Steam feature path is report-id-0 with a
/// leading report-id byte the kernel strips (`steam_recv_report` does `memcpy(data, buf+1, …)`), so
/// the wire is `[0x00, 0xAE, len, 0x01, ascii…]`; the kernel then validates `reply[0]==0xAE`,
/// `1<=reply[1]<=21`, `reply[2]==0x01`. Non-fatal (a bad reply → the `"XXXXXXXXXX"` fallback).
pub fn serial_reply(serial: &str) -> [u8; STEAM_REPORT_LEN] {
let mut buf = [0u8; STEAM_REPORT_LEN];
let bytes = serial.as_bytes();
let len = bytes.len().clamp(1, 21);
buf[0] = 0x00; // report id 0 — stripped by steam_recv_report
buf[1] = ID_GET_STRING_ATTRIBUTE;
buf[2] = len as u8;
buf[3] = ATTRIB_STR_UNIT_SERIAL;
buf[4..4 + len].copy_from_slice(&bytes[..len]);
buf
}
/// One service pass's extracted feedback. Rumble rides the universal 0xCA plane (so any client
/// feels it); the classic SC's trackpad-pulse haptics (`0x8F`) are a later, model-specific add.
#[derive(Default, Debug, PartialEq, Eq)]
pub struct SteamFeedback {
/// `(low, high)` motor levels (left/strong, right/weak), if a rumble report carried them.
pub rumble: Option<(u16, u16)>,
}
/// Parse a feature/output report the kernel wrote to our device. The Steam feedback path is a
/// FEATURE `SET_REPORT` whose wire data is `[0x00 report-id, cmd, len, …]`; `cmd == 0xEB`
/// (`steam_haptic_rumble`) carries `[…, 0, intensity(2), left_speed(2), right_speed(2), gains(2)]`.
/// We surface `(left_speed, right_speed)` as `(low, high)` for the 0xCA rumble plane.
pub fn parse_steam_output(data: &[u8]) -> SteamFeedback {
let mut fb = SteamFeedback::default();
// data[0] is the stripped report-id byte (0); the command id follows.
if data.len() >= 10 && data[1] == ID_TRIGGER_RUMBLE_CMD {
let le = |o: usize| u16::from_le_bytes([data[o], data[o + 1]]);
let left = le(6); // left_speed (report[5..7]) → low / strong motor
let right = le(8); // right_speed (report[7..9]) → high / weak motor
fb.rumble = Some((left, right));
}
fb
}
// ===========================================================================================
// Real-USB Deck device contract (the gadget + usbip transports present a *real* 3-interface USB
// Deck so Steam Input promotes it; the UHID path above uses the minimal [`STEAMDECK_RDESC`]).
//
// These descriptors are captured verbatim from a physical Steam Deck (28DE:1205): mouse =
// interface 0, keyboard = interface 1, **controller = interface 2** (the interface number Steam's
// own driver filters on — the reason a UHID Deck, `Interface: -1`, is never promoted). The
// `0x83`/`0xAE` feature contract is what stops Steam re-probing (the gamepad-evdev churn). Shared
// by [`super::super::steam_gadget`] (raw_gadget) and [`super::super::steam_usbip`] (usbip/vhci).
// ===========================================================================================
/// Captured Deck **mouse** report descriptor (interface 0, EP 0x81).
#[rustfmt::skip]
pub const RDESC_DECK_MOUSE: &[u8] = &[
0x05,0x01,0x09,0x02,0xa1,0x01,0x09,0x01,0xa1,0x00,0x05,0x09,0x19,0x01,0x29,0x02,
0x15,0x00,0x25,0x01,0x75,0x01,0x95,0x02,0x81,0x02,0x75,0x06,0x95,0x01,0x81,0x01,
0x05,0x01,0x09,0x30,0x09,0x31,0x15,0x81,0x25,0x7f,0x75,0x08,0x95,0x02,0x81,0x06,
0x95,0x01,0x09,0x38,0x81,0x06,0x05,0x0c,0x0a,0x38,0x02,0x95,0x01,0x81,0x06,0xc0,0xc0];
/// Captured Deck **keyboard** (boot) report descriptor (interface 1, EP 0x82).
#[rustfmt::skip]
pub const RDESC_DECK_KBD: &[u8] = &[
0x05,0x01,0x09,0x06,0xa1,0x01,0x05,0x07,0x19,0xe0,0x29,0xe7,0x15,0x00,0x25,0x01,
0x75,0x01,0x95,0x08,0x81,0x02,0x81,0x01,0x19,0x00,0x29,0x65,0x15,0x00,0x25,0x65,
0x75,0x08,0x95,0x06,0x81,0x00,0xc0];
/// Captured Deck **controller** report descriptor (interface 2, EP 0x83; Usage Page `0xFFFF`,
/// `bCountryCode 33`). The vendor-defined report the `hid-steam` driver binds.
#[rustfmt::skip]
pub const RDESC_DECK_CTRL: &[u8] = &[
0x06,0xff,0xff,0x09,0x01,0xa1,0x01,0x09,0x02,0x09,0x03,0x15,0x00,0x26,0xff,0x00,
0x75,0x08,0x95,0x40,0x81,0x02,0x09,0x06,0x09,0x07,0x15,0x00,0x26,0xff,0x00,0x75,
0x08,0x95,0x40,0xb1,0x02,0xc0];
/// Per-instance Deck unit id stamped into the `0x83` GET_ATTRIBUTES device-id attrs (`0x0a`/`0x04`)
/// so a virtual Deck never collides with a real one or another instance. `"PF"` high word + index.
pub fn deck_unit_id(index: u8) -> u32 {
0x5046_0000 | index as u32
}
/// A Steam-accepted alphanumeric unit serial (a real Deck's is e.g. `"FVZZ4200469B"`; Steam rejects
/// a too-short/oddly-formatted one as "Invalid or missing unit serial number" and substitutes its
/// own — benign, but we present a clean 12-char one). Derived from [`deck_unit_id`] so the `0xAE`
/// serial reply and the `0x83` unit-id attrs stay consistent.
pub fn deck_serial(index: u8) -> String {
format!("PFDK{:08X}", deck_unit_id(index))
}
/// The neutral 64-byte Deck input report (header only, all controls released) — the report the
/// real-USB transports stream until the first [`serialize_deck_state`] call updates it.
pub fn neutral_deck_report() -> [u8; STEAM_REPORT_LEN] {
let mut r = [0u8; STEAM_REPORT_LEN];
r[0] = 0x01;
r[2] = ID_CONTROLLER_DECK_STATE;
r[3] = 0x3C;
r
}
/// Build the HID feature GET_REPORT reply for the host's last SET_REPORT command, for the *real-USB*
/// Deck (gadget + usbip). Steam's `GetControllerInfo` reads the `0x83` attributes + the `0xAE`
/// serial; **serving the real `0x83` blob is what stops Steam re-probing** (the gamepad-evdev churn).
/// The 9-attribute `0x83` layout + the `0xAE` string format were captured from a physical Deck via
/// hidraw. `unit_id` (see [`deck_unit_id`]) stamps a per-instance value into the device-id attrs.
///
/// Note this is the raw 64-byte EP0 feature payload (command id first, no report-id prefix) — the USB
/// control path, distinct from [`serial_reply`] which carries the UHID report-id byte the kernel
/// strips.
pub fn feature_reply(last_set: &[u8], serial: &str, unit_id: u32) -> [u8; STEAM_REPORT_LEN] {
let cmd = last_set.first().copied().unwrap_or(ID_GET_STRING_ATTRIBUTE);
let mut r = [0u8; STEAM_REPORT_LEN];
match cmd {
ID_GET_ATTRIBUTES_VALUES => {
// GET_ATTRIBUTES_VALUES: [0x83, 0x2d, then 9× (attr-id, value u32-LE)].
r[0] = ID_GET_ATTRIBUTES_VALUES;
r[1] = 0x2d;
let attrs: [(u8, u32); 9] = [
(0x01, 0x1205), // product id
(0x02, 0),
(0x0a, unit_id), // unit serial number (per-instance)
(0x04, unit_id ^ 0x5555_5555),
(0x09, 0x2e),
(0x0b, 0x0fa0),
(0x0d, 0),
(0x0c, 0),
(0x0e, 0),
];
let mut o = 2;
for (id, val) in attrs {
r[o] = id;
r[o + 1..o + 5].copy_from_slice(&val.to_le_bytes());
o += 5;
}
}
ID_GET_STRING_ATTRIBUTE => {
// GET_STRING_ATTRIBUTE: [0xAE, len, attr, ascii…]. The kernel validates the serial (attr
// 0x01) wants reply[2]==0x01 and 1<=len<=21; for other attrs we echo the requested id.
let attr = last_set.get(2).copied().unwrap_or(ATTRIB_STR_UNIT_SERIAL);
let b = serial.as_bytes();
let len = b.len().clamp(1, 20);
r[0] = ID_GET_STRING_ATTRIBUTE;
r[1] = len as u8;
r[2] = attr;
r[3..3 + len].copy_from_slice(&b[..len]);
}
_ => {
// Settings read-back (e.g. 0x87): echo the host's last command + data.
let n = last_set.len().min(STEAM_REPORT_LEN);
r[..n].copy_from_slice(&last_set[..n]);
}
}
r
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn descriptor_declares_input_and_feature_reports() {
assert!(
STEAMDECK_RDESC.contains(&0xB1),
"missing Feature main item — steam_is_valve_interface() would fail"
);
assert!(STEAMDECK_RDESC.contains(&0x81), "missing Input main item");
assert_eq!(
*STEAMDECK_RDESC.last().unwrap(),
0xC0,
"unterminated collection"
);
}
/// Every analog field lands at the exact offset `steam_do_deck_input_event` reads, the header is
/// what `steam_raw_event` requires, and the buttons pack into bytes 8..16 (12+15 zero). A
/// one-byte slip here turns the whole controller into noise.
#[test]
fn serialize_is_byte_exact() {
let mut st = SteamState::neutral();
st.buttons = btn::A | btn::L4 | btn::R5 | btn::QAM;
st.lx = 0x1122;
st.ly = 0x3344;
st.rx = 0x5566;
st.ry = 0x778;
st.lt = 0xABCD;
st.rt = 0xEF01;
st.lpad_x = 0x0A0B;
st.lpad_y = 0x0C0D;
st.rpad_x = 0x0E0F;
st.rpad_y = 0x1011;
st.accel = [0x0102, 0x0304, 0x0506];
st.gyro = [0x0708, 0x090A, 0x0B0C];
st.lpad_pressure = 0x1314;
st.rpad_pressure = 0x1516;
let mut r = [0u8; STEAM_REPORT_LEN];
serialize_deck_state(&mut r, &st, 0xAABB_CCDD);
assert_eq!(&r[0..4], &[0x01, 0x00, 0x09, 0x3C]);
assert_eq!(&r[4..8], &[0xDD, 0xCC, 0xBB, 0xAA]); // seq LE
// buttons: A=bit7 (byte8), L4=bit41 (byte13.1), R5=bit16 (byte10.0), QAM=bit50 (byte14.2).
assert_eq!(r[8], 0x80); // A
assert_eq!(r[10], 0x01); // R5
assert_eq!(r[12], 0x00); // unused button byte
assert_eq!(r[13], 0x02); // L4 (bit 1)
assert_eq!(r[14], 0x04); // QAM (bit 2)
assert_eq!(r[15], 0x00); // unused button byte
assert_eq!(&r[16..18], &0x0A0Bi16.to_le_bytes()); // lpad X
assert_eq!(&r[20..22], &0x0E0Fi16.to_le_bytes()); // rpad X
assert_eq!(&r[24..26], &0x0102i16.to_le_bytes()); // accel X
assert_eq!(&r[26..28], &0x0304i16.to_le_bytes()); // accel Y
assert_eq!(&r[28..30], &0x0506i16.to_le_bytes()); // accel Z
assert_eq!(&r[30..32], &0x0708i16.to_le_bytes()); // gyro X
assert_eq!(&r[44..46], &0xABCDu16.to_le_bytes()); // left trigger
assert_eq!(&r[46..48], &0xEF01u16.to_le_bytes()); // right trigger
assert_eq!(&r[48..50], &0x1122i16.to_le_bytes()); // left joy X
assert_eq!(&r[50..52], &0x3344i16.to_le_bytes()); // left joy Y
assert_eq!(&r[52..54], &0x5566i16.to_le_bytes()); // right joy X
assert_eq!(&r[56..58], &0x1314u16.to_le_bytes()); // left pad pressure
assert_eq!(&r[58..60], &0x1516u16.to_le_bytes()); // right pad pressure
}
/// `from_gamepad` sets the right Deck bits + scales triggers, and a touched flag is merged when
/// a trackpad contact arrives via `apply_rich`.
#[test]
fn from_gamepad_and_rich_mapping() {
let s = SteamState::from_gamepad(
gs::BTN_A | gs::BTN_START | gs::BTN_GUIDE | gs::BTN_LB,
1000,
-2000,
0,
0,
255,
0,
);
assert_ne!(s.buttons & btn::A, 0);
assert_ne!(s.buttons & btn::MENU, 0);
assert_ne!(s.buttons & btn::STEAM, 0);
assert_ne!(s.buttons & btn::LB, 0);
assert_ne!(s.buttons & btn::LT_FULL, 0); // lt=255 → full-pull bit
assert_eq!(s.lt, 255 * 128);
assert_eq!(s.lx, 1000);
assert_eq!(s.ly, -2000);
let mut s = SteamState::neutral();
s.apply_rich(RichInput::Touchpad {
pad: 0,
finger: 0,
active: true,
x: 65535,
y: 0,
});
assert_ne!(s.buttons & btn::RPAD_TOUCH, 0);
assert_eq!(s.rpad_x, 32767); // 65535-32768
assert_eq!(s.rpad_y, -32768); // 0-32768
// Motion is rescaled from the wire (DualSense) convention into Deck units (gyro ×16/20,
// accel ×16384/10000) — see steam_remap::motion_wire_to_deck.
s.apply_rich(RichInput::Motion {
pad: 0,
gyro: [1000, -2000, 0],
accel: [10000, -5000, 0],
});
assert_eq!(s.gyro, [800, -1600, 0]);
assert_eq!(s.accel, [16384, -8192, 0]);
}
/// M3: the wire back-button bits map to the four Deck grips + QAM, and `TouchpadEx` routes the
/// left / right surfaces to the matching pad (signed coords pass straight through).
#[test]
fn back_buttons_and_dual_trackpad_mapping() {
let s = SteamState::from_gamepad(
gs::BTN_PADDLE1 | gs::BTN_PADDLE2 | gs::BTN_PADDLE3 | gs::BTN_PADDLE4 | gs::BTN_MISC1,
0,
0,
0,
0,
0,
0,
);
assert_ne!(s.buttons & btn::R4, 0); // PADDLE1 = R4
assert_ne!(s.buttons & btn::L4, 0); // PADDLE2 = L4
assert_ne!(s.buttons & btn::R5, 0); // PADDLE3 = R5
assert_ne!(s.buttons & btn::L5, 0); // PADDLE4 = L5
assert_ne!(s.buttons & btn::QAM, 0); // MISC1 = QAM
let mut s = SteamState::neutral();
s.apply_rich(RichInput::TouchpadEx {
pad: 0,
surface: 1,
finger: 0,
touch: true,
click: true,
x: -5000,
y: 6000,
pressure: 100,
});
assert_ne!(s.buttons & btn::LPAD_TOUCH, 0);
assert_ne!(s.buttons & btn::LPAD_CLICK, 0);
assert_eq!((s.lpad_x, s.lpad_y), (-5000, 6000));
s.apply_rich(RichInput::TouchpadEx {
pad: 0,
surface: 2,
finger: 0,
touch: true,
click: false,
x: 7000,
y: -8000,
pressure: 0,
});
assert_ne!(s.buttons & btn::RPAD_TOUCH, 0);
assert_eq!((s.rpad_x, s.rpad_y), (7000, -8000));
}
/// The serial reply carries the leading report-id byte the kernel strips, so the *stripped*
/// view (`reply[1..]`) is what `steam_get_serial` validates: `[0xAE, len, 0x01, ascii…]`.
#[test]
fn serial_reply_has_stripped_prefix() {
let r = serial_reply("PUNKTFUNK01");
assert_eq!(r[0], 0x00); // report id, stripped by steam_recv_report
assert_eq!(r[1], ID_GET_STRING_ATTRIBUTE); // becomes reply[0] after strip
assert!((1..=21).contains(&r[2]));
assert_eq!(r[3], ATTRIB_STR_UNIT_SERIAL);
assert_eq!(&r[4..4 + r[2] as usize], b"PUNKTFUNK01");
}
/// A `0xEB` rumble feature report parses to `(left_speed, right_speed)`; other commands don't.
#[test]
fn parse_rumble_feedback() {
// [report-id 0, 0xEB, len 9, 0, intensity(2), left(2), right(2), gains(2)]
let mut d = vec![0u8; 12];
d[1] = ID_TRIGGER_RUMBLE_CMD;
d[2] = 9;
d[6..8].copy_from_slice(&0x8000u16.to_le_bytes()); // left_speed
d[8..10].copy_from_slice(&0x4000u16.to_le_bytes()); // right_speed
assert_eq!(parse_steam_output(&d).rumble, Some((0x8000, 0x4000)));
let mut d = vec![0u8; 12];
d[1] = ID_SET_SETTINGS_VALUES; // a settings write — no rumble
assert_eq!(parse_steam_output(&d).rumble, None);
}
/// The shared real-USB Deck feature contract (gadget + usbip): the `0x83` GET_ATTRIBUTES reply
/// carries the 9-attribute blob with the per-instance unit id, and the `0xAE` reply carries the
/// Steam-accepted serial — both keyed off the host's last SET_REPORT command. A slip here is the
/// gamepad-evdev churn (Steam re-probing).
#[test]
fn deck_feature_reply_contract() {
let serial = deck_serial(0);
let unit_id = deck_unit_id(0);
assert_eq!(serial, "PFDK50460000"); // 12-char alphanumeric, derived from the unit id
assert_eq!(serial.len(), 12);
// 0x83 GET_ATTRIBUTES_VALUES: header + (0x0a, unit_id) at the 3rd attribute slot.
let r = feature_reply(&[ID_GET_ATTRIBUTES_VALUES], &serial, unit_id);
assert_eq!(r[0], ID_GET_ATTRIBUTES_VALUES);
assert_eq!(r[1], 0x2d);
assert_eq!(r[12], 0x0a); // 3rd attr id (slots at 2,7,12,…)
assert_eq!(
u32::from_le_bytes([r[13], r[14], r[15], r[16]]),
unit_id,
"unit serial attribute must carry the per-instance unit id"
);
// 0xAE GET_STRING_ATTRIBUTE: [0xAE, len, attr(0x01), ascii serial…].
let r = feature_reply(
&[ID_GET_STRING_ATTRIBUTE, 0, ATTRIB_STR_UNIT_SERIAL],
&serial,
unit_id,
);
assert_eq!(r[0], ID_GET_STRING_ATTRIBUTE);
assert_eq!(r[1] as usize, serial.len());
assert_eq!(r[2], ATTRIB_STR_UNIT_SERIAL);
assert_eq!(&r[3..3 + serial.len()], serial.as_bytes());
// Distinct pad indices get distinct unit ids + serials (no collision between virtual Decks).
assert_ne!(deck_unit_id(0), deck_unit_id(1));
assert_ne!(deck_serial(0), deck_serial(1));
}
}
@@ -0,0 +1,149 @@
//! Pure fallback-remap policy for the Steam Controller / Steam Deck rich inputs when the resolved
//! host backend is **not** the virtual `hid-steam` device (DualSense / DualShock 4 / Xbox), so a
//! client's Steam-only inputs aren't silently dropped — plus the cross-device motion rescale the
//! Deck backend itself needs.
//!
//! Driven by the host's `PUNKTFUNK_STEAM_REMAP` env (`key=value`, `,`/`;`-separated, e.g.
//! `paddles=stickclicks`). No I/O beyond [`RemapConfig::from_env`]; everything else is pure +
//! unit-testable. The uinput Xbox pad already exposes the back grips as Elite paddles
//! (`BTN_TRIGGER_HAPPY5-8`), so only the slot-less DualSense / DS4 backends fold them.
use punktfunk_core::input::gamepad as gs;
/// Where the four Steam back grips go on a backend with no native back-button HID slot.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)]
pub enum PaddleFallback {
/// Drop them — the back buttons are simply absent on this pad. The honest default: don't fire
/// buttons the user didn't ask for. Set the env to map them instead.
#[default]
Drop,
/// L4/L5 → left-stick click, R4/R5 → right-stick click.
StickClicks,
/// L4/L5 → left bumper, R4/R5 → right bumper.
Shoulders,
}
/// Fallback-remap knobs parsed from `PUNKTFUNK_STEAM_REMAP`.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct RemapConfig {
pub paddles: PaddleFallback,
}
impl RemapConfig {
/// Parse the host's `PUNKTFUNK_STEAM_REMAP` env (absent / unrecognized → defaults).
pub fn from_env() -> RemapConfig {
std::env::var("PUNKTFUNK_STEAM_REMAP")
.map(|s| RemapConfig::parse(&s))
.unwrap_or_default()
}
/// Pure parse of the `key=value[,key=value…]` string (the testable core of [`from_env`]).
pub fn parse(s: &str) -> RemapConfig {
let mut cfg = RemapConfig::default();
for kv in s.split([',', ';']) {
let mut it = kv.splitn(2, '=');
if let (Some(k), Some(v)) = (it.next(), it.next()) {
if k.trim().eq_ignore_ascii_case("paddles") {
cfg.paddles = match v.trim().to_ascii_lowercase().as_str() {
"stickclicks" | "l3r3" | "sticks" => PaddleFallback::StickClicks,
"shoulders" | "lbrb" | "bumpers" => PaddleFallback::Shoulders,
_ => PaddleFallback::Drop,
};
}
}
}
cfg
}
}
/// Fold the wire back-grip bits (`BTN_PADDLE1..4`) into standard buttons per `policy` for a pad with
/// no native back-button slot, clearing the paddle bits. Pure. PADDLE1/2/3/4 = R4/L4/R5/L5.
pub fn fold_paddles(mut buttons: u32, policy: PaddleFallback) -> u32 {
let left = buttons & (gs::BTN_PADDLE2 | gs::BTN_PADDLE4) != 0; // L4 | L5
let right = buttons & (gs::BTN_PADDLE1 | gs::BTN_PADDLE3) != 0; // R4 | R5
buttons &= !(gs::BTN_PADDLE1 | gs::BTN_PADDLE2 | gs::BTN_PADDLE3 | gs::BTN_PADDLE4);
let (lbit, rbit) = match policy {
PaddleFallback::Drop => return buttons,
PaddleFallback::StickClicks => (gs::BTN_LS_CLICK, gs::BTN_RS_CLICK),
PaddleFallback::Shoulders => (gs::BTN_LB, gs::BTN_RB),
};
if left {
buttons |= lbit;
}
if right {
buttons |= rbit;
}
buttons
}
// Motion rescale. The wire uses the DualSense convention (20 LSB/°·s gyro, 10000 LSB/g accel — the
// scale every client capture applies). The Steam Deck's `hid-steam` report wants 16 LSB/°·s and
// 16384 LSB/g, so the Deck backend rescales; the DualSense / DS4 backends consume the wire 1:1.
const GYRO_NUM: i32 = 16;
const GYRO_DEN: i32 = 20;
const ACCEL_NUM: i32 = 16384;
const ACCEL_DEN: i32 = 10000;
fn scale(v: i16, num: i32, den: i32) -> i16 {
((v as i32 * num) / den).clamp(i16::MIN as i32, i16::MAX as i32) as i16
}
/// Rescale a wire (DualSense-convention) motion sample into the Steam Deck's `hid-steam` units.
pub fn motion_wire_to_deck(gyro: [i16; 3], accel: [i16; 3]) -> ([i16; 3], [i16; 3]) {
(
gyro.map(|g| scale(g, GYRO_NUM, GYRO_DEN)),
accel.map(|a| scale(a, ACCEL_NUM, ACCEL_DEN)),
)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_paddle_policy() {
assert_eq!(RemapConfig::parse("").paddles, PaddleFallback::Drop);
assert_eq!(
RemapConfig::parse("paddles=stickclicks").paddles,
PaddleFallback::StickClicks
);
assert_eq!(
RemapConfig::parse("foo=bar; paddles = Shoulders").paddles,
PaddleFallback::Shoulders
);
assert_eq!(
RemapConfig::parse("paddles=nonsense").paddles,
PaddleFallback::Drop
);
}
#[test]
fn fold_paddles_maps_and_clears() {
// All four grips set + a real A button.
let b = gs::BTN_A | gs::BTN_PADDLE1 | gs::BTN_PADDLE2 | gs::BTN_PADDLE3 | gs::BTN_PADDLE4;
// Drop: paddle bits cleared, A preserved, nothing added.
assert_eq!(fold_paddles(b, PaddleFallback::Drop), gs::BTN_A);
// StickClicks: left grips → L3, right grips → R3.
assert_eq!(
fold_paddles(b, PaddleFallback::StickClicks),
gs::BTN_A | gs::BTN_LS_CLICK | gs::BTN_RS_CLICK
);
// Only a left grip (L4 = PADDLE2) → only the left bumper under Shoulders.
assert_eq!(
fold_paddles(gs::BTN_PADDLE2, PaddleFallback::Shoulders),
gs::BTN_LB
);
}
#[test]
fn motion_rescale_to_deck_units() {
// gyro × 16/20 = 0.8; accel × 16384/10000 = 1.6384.
let (g, a) = motion_wire_to_deck([1000, -2000, 0], [10000, -5000, 0]);
assert_eq!(g, [800, -1600, 0]);
assert_eq!(a, [16384, -8192, 0]);
// Saturates rather than wraps.
let (_, a) = motion_wire_to_deck([0; 3], [32767, i16::MIN, 0]);
assert_eq!(a[0], i16::MAX);
assert_eq!(a[1], i16::MIN);
}
}
@@ -385,7 +385,9 @@ impl DualSenseWindowsManager {
/// Apply one rich client→host event (touchpad contact / motion sample) to an existing pad. /// Apply one rich client→host event (touchpad contact / motion sample) to an existing pad.
pub fn apply_rich(&mut self, rich: RichInput) { pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich { let idx = match rich {
RichInput::Touchpad { pad, .. } | RichInput::Motion { pad, .. } => pad as usize, RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
}; };
if idx >= MAX_PADS || self.pads[idx].is_none() { if idx >= MAX_PADS || self.pads[idx].is_none() {
return; return;
@@ -409,6 +411,26 @@ impl DualSenseWindowsManager {
self.state[idx].gyro = gyro; self.state[idx].gyro = gyro;
self.state[idx].accel = accel; self.state[idx].accel = accel;
} }
RichInput::TouchpadEx {
surface,
finger,
touch,
x,
y,
..
} => {
// A Steam right/single pad maps onto the one DualSense touchpad (signed centre-0 →
// 0..=65535); surface 1 (the Steam left pad) has no DualSense equivalent.
if surface != 1 {
let slot = (finger as usize).min(1);
let n = |v: i16| ((v as i32) + 32768) as u32;
let t = &mut self.state[idx].touch[slot];
t.active = touch;
t.id = slot as u8;
t.x = (n(x) * (DS_TOUCH_W - 1) as u32 / u16::MAX as u32) as u16;
t.y = (n(y) * (DS_TOUCH_H - 1) as u32 / u16::MAX as u32) as u16;
}
}
} }
self.write(idx); self.write(idx);
} }
@@ -186,7 +186,9 @@ impl DualShock4WindowsManager {
/// Apply one rich client→host event (touchpad contact / motion sample) to an existing pad. /// Apply one rich client→host event (touchpad contact / motion sample) to an existing pad.
pub fn apply_rich(&mut self, rich: RichInput) { pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich { let idx = match rich {
RichInput::Touchpad { pad, .. } | RichInput::Motion { pad, .. } => pad as usize, RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
}; };
if idx >= MAX_PADS || self.pads[idx].is_none() { if idx >= MAX_PADS || self.pads[idx].is_none() {
return; return;
@@ -210,6 +212,26 @@ impl DualShock4WindowsManager {
self.state[idx].gyro = gyro; self.state[idx].gyro = gyro;
self.state[idx].accel = accel; self.state[idx].accel = accel;
} }
RichInput::TouchpadEx {
surface,
finger,
touch,
x,
y,
..
} => {
// A Steam right/single pad maps onto the one DS4 touchpad (signed centre-0 →
// 0..=65535); surface 1 (the Steam left pad) has no DS4 equivalent.
if surface != 1 {
let slot = (finger as usize).min(1);
let n = |v: i16| ((v as i32) + 32768) as u32;
let t = &mut self.state[idx].touch[slot];
t.active = touch;
t.id = slot as u8;
t.x = (n(x) * (DS4_TOUCH_W - 1) as u32 / u16::MAX as u32) as u16;
t.y = (n(y) * (DS4_TOUCH_H - 1) as u32 / u16::MAX as u32) as u16;
}
}
} }
self.write(idx); self.write(idx);
} }
@@ -21,10 +21,18 @@ use windows::Win32::System::Memory::{
MEMORY_MAPPED_VIEW_ADDRESS, PAGE_READWRITE, MEMORY_MAPPED_VIEW_ADDRESS, PAGE_READWRITE,
}; };
/// A named, anonymous (pagefile-backed) shared section + its mapped read/write view, created with the /// A named, anonymous (pagefile-backed) shared section + its mapped read/write view. RAII: drop unmaps
/// permissive `D:(A;;GA;;;WD)` SDDL the restricted-token driver needs to open it. RAII: drop unmaps the /// the view, then the [`OwnedHandle`] closes the section handle (in that order). Replaces the three
/// view, then the [`OwnedHandle`] closes the section handle (in that order). Replaces the three backends' /// backends' hand-duplicated `CreateFileMappingW` + `MapViewOfFile` + manual `Drop`.
/// hand-duplicated `CreateFileMappingW` + `MapViewOfFile` + manual `Drop`. ///
/// SDDL `D:(A;;GA;;;SY)(A;;GA;;;LS)`: GENERIC_ALL to **SYSTEM** (the host creates the section and
/// writes the live HID input report into it) and **LocalService** (the account the UMDF driver's
/// WUDFHost runs under, which reads it). The old SDDL granted **Everyone** (`WD`) — on the (mistaken)
/// assumption the driver needed a restricted token's broad access — letting any local user
/// `OpenFileMapping` the section to inject controller input or tamper the trusted channel
/// (security-review 2026-06-28 #5). Verified on the RTX box (2026-06-29): the WUDFHost token is
/// `S-1-5-19` (LocalService), SYSTEM integrity, with **zero restricted SIDs** — so scoping to SY+LS is
/// sufficient for the driver and excludes normal (medium-IL, non-service) user processes.
pub(super) struct Shm { pub(super) struct Shm {
/// Owns the section handle (closed on drop). Held only for ownership — never read after construction. /// Owns the section handle (closed on drop). Held only for ownership — never read after construction.
_handle: OwnedHandle, _handle: OwnedHandle,
@@ -40,7 +48,7 @@ impl Shm {
// exit — acceptable for a host-lifetime object). // exit — acceptable for a host-lifetime object).
unsafe { unsafe {
ConvertStringSecurityDescriptorToSecurityDescriptorW( ConvertStringSecurityDescriptorToSecurityDescriptorW(
w!("D:(A;;GA;;;WD)"), w!("D:(A;;GA;;;SY)(A;;GA;;;LS)"),
SDDL_REVISION_1, SDDL_REVISION_1,
&mut psd, &mut psd,
None, None,
-32
View File
@@ -56,9 +56,6 @@ mod spike;
mod stats_recorder; mod stats_recorder;
mod vdisplay; mod vdisplay;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
#[path = "windows/wgc_helper.rs"]
mod wgc_helper;
#[cfg(target_os = "windows")]
#[path = "windows/win_adapter.rs"] #[path = "windows/win_adapter.rs"]
mod win_adapter; mod win_adapter;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
@@ -392,35 +389,6 @@ fn real_main() -> Result<()> {
paired_store: None, paired_store: None,
}) })
} }
// USER-session WGC helper (Windows two-process secure-desktop design): capture the EXISTING
// SudoVDA via WGC + NVENC, stream AUs on stdout to the SYSTEM host. Spawned by the host
// (CreateProcessAsUser), not run by hand. See design/archive/windows-secure-desktop.md.
#[cfg(target_os = "windows")]
Some("wgc-helper") => {
let get = |flag: &str| {
args.iter()
.skip_while(|a| *a != flag)
.nth(1)
.map(String::as_str)
};
let (width, height, fps) = get("--mode")
.and_then(|m| {
let p: Vec<u32> = m.split('x').filter_map(|s| s.parse().ok()).collect();
(p.len() == 3).then(|| (p[0], p[1], p[2]))
})
.unwrap_or((1920, 1080, 60));
wgc_helper::run(wgc_helper::HelperOptions {
target_id: get("--target-id").and_then(|s| s.parse().ok()).unwrap_or(0),
gdi_name: get("--gdi").unwrap_or("").to_string(),
width,
height,
fps,
bitrate_kbps: get("--bitrate")
.and_then(|s| s.parse().ok())
.unwrap_or(20000),
bit_depth: get("--bit-depth").and_then(|s| s.parse().ok()).unwrap_or(8),
})
}
// Windows service control: install/uninstall/start/stop/status + the SCM `run` entry point. // Windows service control: install/uninstall/start/stop/status + the SCM `run` entry point.
// Replaces the ad-hoc launch chain — `service install` registers an auto-start SYSTEM service // Replaces the ad-hoc launch chain — `service install` registers an auto-start SYSTEM service
// that launches the host into the active interactive session. // that launches the host into the active interactive session.
+23 -4
View File
@@ -394,6 +394,12 @@ struct ArmNativePairing {
/// Window length in seconds (default 120; clamped to 15600). /// Window length in seconds (default 120; clamped to 15600).
#[schema(example = 120)] #[schema(example = 120)]
ttl_secs: Option<u32>, ttl_secs: Option<u32>,
/// Optional: bind the window to ONE device fingerprint (hex SHA-256, e.g. from a pending knock).
/// When set, only a pairing attempt from that fingerprint consumes the window — so an unpaired
/// LAN peer can neither pair nor burn a window armed for a specific device (security-review #9).
/// Omit for an unbound window (any device may use the PIN — trusted-LAN only).
#[schema(example = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08")]
fingerprint: Option<String>,
} }
/// A paired native (punktfunk/1) client. /// A paired native (punktfunk/1) client.
@@ -879,8 +885,21 @@ async fn arm_native_pairing(
); );
}; };
let ttl = req.ttl_secs.unwrap_or(120).clamp(15, 600); let ttl = req.ttl_secs.unwrap_or(120).clamp(15, 600);
let _pin = np.arm(std::time::Duration::from_secs(ttl as u64)); // A bound window (operator selected a specific device) is DoS-proof: only that fingerprint can
tracing::info!(ttl_secs = ttl, "management API: native pairing armed"); // consume it (#9). An unbound window (no fingerprint) keeps the legacy any-device behavior.
let bound = req
.fingerprint
.as_deref()
.map(str::trim)
.filter(|s| !s.is_empty())
.map(|s| s.to_ascii_lowercase());
let bound_to_device = bound.is_some();
let _pin = np.arm_for(std::time::Duration::from_secs(ttl as u64), bound);
tracing::info!(
ttl_secs = ttl,
bound_to_device,
"management API: native pairing armed"
);
Json(native_status(&st)).into_response() Json(native_status(&st)).into_response()
} }
@@ -1975,8 +1994,8 @@ mod tests {
assert_eq!(b.as_array().unwrap().len(), 0); assert_eq!(b.as_array().unwrap().len(), 0);
// Two devices knock (what the QUIC gate records); they appear in the list. // Two devices knock (what the QUIC gate records); they appear in the list.
np.note_pending("Enrico's MacBook", "aa11"); np.note_pending("Enrico's MacBook", "aa11", None);
np.note_pending("device bb22cc33", "bb22"); np.note_pending("device bb22cc33", "bb22", None);
let (_, b) = send(&app, get_req("/api/v1/native/pending")).await; let (_, b) = send(&app, get_req("/api/v1/native/pending")).await;
assert_eq!(b.as_array().unwrap().len(), 2); assert_eq!(b.as_array().unwrap().len(), 2);
assert_eq!(b[0]["name"], "Enrico's MacBook"); assert_eq!(b[0]["name"], "Enrico's MacBook");
+204 -21
View File
@@ -8,6 +8,7 @@
//! armed on demand for a short window — rather than accepting one. //! armed on demand for a short window — rather than accepting one.
use anyhow::Result; use anyhow::Result;
use std::net::IpAddr;
use std::path::PathBuf; use std::path::PathBuf;
use std::sync::Mutex; use std::sync::Mutex;
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
@@ -42,10 +43,29 @@ struct PairedState {
/// The current arming window. `pin == None` ⇒ disarmed. `expires_at == None` ⇒ armed with no /// The current arming window. `pin == None` ⇒ disarmed. `expires_at == None` ⇒ armed with no
/// expiry (the CLI `--allow-pairing` flag); `Some(t)` ⇒ a web-armed window that auto-disarms. /// expiry (the CLI `--allow-pairing` flag); `Some(t)` ⇒ a web-armed window that auto-disarms.
///
/// `bound_fp == Some(fp)` ⇒ the window is **bound to one operator-selected device fingerprint**:
/// only a pairing attempt from that fingerprint may consume it (security-review 2026-06-28 #9). This
/// closes the window-burn DoS — an unpaired LAN peer cannot consume a window armed for a specific
/// device, because the QUIC client-auth proves cert possession (it can't forge the bound fingerprint).
/// `None` ⇒ unbound (the CLI flag / a console "arm open"): any well-formed attempt consumes it (the
/// legacy behavior, retaining the window-burn DoS — acceptable only on a trusted LAN).
#[derive(Default)] #[derive(Default)]
struct Armed { struct Armed {
pin: Option<String>, pin: Option<String>,
expires_at: Option<Instant>, expires_at: Option<Instant>,
bound_fp: Option<String>,
}
/// The result of resolving the armed PIN for a specific client fingerprint ([`NativePairing::pin_for_attempt`]).
pub enum PinAttempt {
/// No window is armed (disarmed/expired) — reject; do not run the ceremony.
Disarmed,
/// A window IS armed but **bound to a different fingerprint** — reject WITHOUT consuming it, so
/// an unrelated (attacker) fingerprint can't burn the operator's armed window (#9).
BoundToOther,
/// Proceed: the PIN to run the ceremony with (the window is unbound, or bound to this fingerprint).
Pin(String),
} }
/// An unpaired (but identified) device that knocked on a pairing-required host — held for /// An unpaired (but identified) device that knocked on a pairing-required host — held for
@@ -57,6 +77,13 @@ struct Pending {
name: String, name: String,
fp_hex: String, fp_hex: String,
requested_at: Instant, requested_at: Instant,
/// QUIC-validated source address of the knock — used for the per-source cap (#13), so one host
/// can't fill the queue. `None` if unknown (e.g. tests / a caller that doesn't supply it).
src_ip: Option<IpAddr>,
/// True while a connection is held open in [`NativePairing::wait_for_decision`] for this knock.
/// A live parked knock is a genuine device waiting for the operator — eviction skips it unless
/// every entry is parked, so a cert-rotating flood can't evict the device being onboarded (#13).
parked: bool,
} }
#[derive(Default)] #[derive(Default)]
@@ -94,6 +121,10 @@ pub enum PairingDecision {
const PENDING_TTL: Duration = Duration::from_secs(10 * 60); const PENDING_TTL: Duration = Duration::from_secs(10 * 60);
/// Cap on the pending list — a LAN scanner must not grow it unboundedly. Oldest entries drop. /// Cap on the pending list — a LAN scanner must not grow it unboundedly. Oldest entries drop.
const PENDING_CAP: usize = 32; const PENDING_CAP: usize = 32;
/// Max pending knocks one source IP may occupy, so a single host can't fill the whole queue and hide
/// / evict a genuine device's knock (security-review 2026-06-28 #13). The QUIC path is address-
/// validated, so the source IP isn't off-path spoofable; an attacker would need that many real hosts.
const MAX_PENDING_PER_IP: usize = 4;
/// Shared native-pairing state: the arming PIN window + the persistent trust store + the /// Shared native-pairing state: the arming PIN window + the persistent trust store + the
/// pending-approval queue. /// pending-approval queue.
@@ -209,6 +240,7 @@ impl NativePairing {
Armed { Armed {
pin: Some(fixed_pin.unwrap_or_else(random_pin)), pin: Some(fixed_pin.unwrap_or_else(random_pin)),
expires_at: None, expires_at: None,
bound_fp: None,
} }
} else { } else {
Armed::default() Armed::default()
@@ -221,16 +253,43 @@ impl NativePairing {
}) })
} }
/// Arm pairing with a fresh random PIN, valid for `ttl`. Returns the PIN to display. /// Arm pairing with a fresh random PIN, valid for `ttl`, **unbound** (any well-formed attempt
/// consumes it). Returns the PIN to display. Prefer [`Self::arm_for`] with a specific device
/// fingerprint on untrusted LANs — an unbound window is burnable by any peer (#9).
pub fn arm(&self, ttl: Duration) -> String { pub fn arm(&self, ttl: Duration) -> String {
self.arm_for(ttl, None)
}
/// Arm pairing with a fresh random PIN, valid for `ttl`. If `bound_fp` is `Some`, the window is
/// bound to that device fingerprint: only a pairing attempt from it consumes the window, so an
/// unrelated (attacker) fingerprint can neither pair nor burn the window (#9). Returns the PIN.
pub fn arm_for(&self, ttl: Duration, bound_fp: Option<String>) -> String {
let pin = random_pin(); let pin = random_pin();
*self.arm.lock().unwrap() = Armed { *self.arm.lock().unwrap() = Armed {
pin: Some(pin.clone()), pin: Some(pin.clone()),
expires_at: Some(Instant::now() + ttl), expires_at: Some(Instant::now() + ttl),
bound_fp,
}; };
pin pin
} }
/// Resolve the PIN for an attempt from `client_fp_hex`, honoring fingerprint binding (#9):
/// `Disarmed` if no window is armed; `BoundToOther` if a window is armed but bound to a different
/// fingerprint (the caller MUST reject without consuming it); else `Pin` to run the ceremony.
pub fn pin_for_attempt(&self, client_fp_hex: &str) -> PinAttempt {
let mut arm = self.arm.lock().unwrap();
Self::expire(&mut arm);
match &arm.pin {
None => PinAttempt::Disarmed,
Some(pin) => match &arm.bound_fp {
Some(bound) if !bound.eq_ignore_ascii_case(client_fp_hex) => {
PinAttempt::BoundToOther
}
_ => PinAttempt::Pin(pin.clone()),
},
}
}
/// Disarm pairing (no new ceremonies accepted). /// Disarm pairing (no new ceremonies accepted).
pub fn disarm(&self) { pub fn disarm(&self) {
*self.arm.lock().unwrap() = Armed::default(); *self.arm.lock().unwrap() = Armed::default();
@@ -342,11 +401,30 @@ impl NativePairing {
.retain(|p| p.requested_at.elapsed() < PENDING_TTL); .retain(|p| p.requested_at.elapsed() < PENDING_TTL);
} }
/// Record an unpaired device's knock for delegated approval. Re-knocks from the same /// Pick the entry to evict, optionally restricted to a single source IP: the least-recently-active
/// fingerprint refresh the existing entry in place (same id; a connect-retry loop must not spam /// **non-parked** entry (a live parked knock is a genuine device awaiting the operator — never
/// the list); a fresh fingerprint gets a new id, evicting the **least-recently-active** entry /// evict it under load); only if every candidate is parked does it fall back to the oldest of
/// past [`PENDING_CAP`]. The name is sanitized (untrusted; see [`sanitize_device_name`]). /// those (#13). Returns the index, or `None` if there's nothing to evict.
pub fn note_pending(&self, name: &str, fp_hex: &str) { fn evict_index(items: &[Pending], only_ip: Option<IpAddr>) -> Option<usize> {
let pick = |allow_parked: bool| {
items
.iter()
.enumerate()
.filter(|(_, p)| only_ip.is_none_or(|ip| p.src_ip == Some(ip)))
.filter(|(_, p)| allow_parked || !p.parked)
.min_by_key(|(_, p)| p.requested_at)
.map(|(i, _)| i)
};
pick(false).or_else(|| pick(true))
}
/// Record an unpaired device's knock for delegated approval. Re-knocks from the same fingerprint
/// refresh the existing entry in place (same id; a connect-retry loop must not spam the list). A
/// fresh fingerprint gets a new id; the queue is bounded two ways so a flood can't crowd out a
/// genuine knock (#13): a **per-source-IP cap** ([`MAX_PENDING_PER_IP`]) means one host can hold at
/// most a few slots, and the global [`PENDING_CAP`] evicts the least-recently-active **non-parked**
/// entry (never a live, held-open parked knock). The name is sanitized (untrusted).
pub fn note_pending(&self, name: &str, fp_hex: &str, src_ip: Option<IpAddr>) {
let name = sanitize_device_name(name, fp_hex); let name = sanitize_device_name(name, fp_hex);
let mut pending = self.pending.lock().unwrap(); let mut pending = self.pending.lock().unwrap();
Self::expire_pending(&mut pending); Self::expire_pending(&mut pending);
@@ -357,19 +435,31 @@ impl NativePairing {
{ {
p.requested_at = Instant::now(); p.requested_at = Instant::now();
p.name = name; p.name = name;
if p.src_ip.is_none() {
p.src_ip = src_ip;
}
return; return;
} }
if pending.items.len() >= PENDING_CAP { // Per-source-IP cap: a single host can't occupy more than MAX_PENDING_PER_IP slots — evict its
// Evict the least-recently-active entry. NOT index 0: the in-place refresh above means // own oldest entry first so it can't crowd out other devices' knocks (#13).
// Vec order no longer tracks recency, so pick the minimum `requested_at` explicitly. if let Some(ip) = src_ip {
if let Some(at) = pending if pending
.items .items
.iter() .iter()
.enumerate() .filter(|p| p.src_ip == Some(ip))
.min_by_key(|(_, p)| p.requested_at) .count()
.map(|(i, _)| i) >= MAX_PENDING_PER_IP
{ {
pending.items.remove(at); if let Some(i) = Self::evict_index(&pending.items, Some(ip)) {
pending.items.remove(i);
}
}
}
// Global cap: evict the least-recently-active non-parked entry (Vec order no longer tracks
// recency after in-place refreshes, so pick explicitly).
if pending.items.len() >= PENDING_CAP {
if let Some(i) = Self::evict_index(&pending.items, None) {
pending.items.remove(i);
} }
} }
let id = pending.next_id; let id = pending.next_id;
@@ -379,9 +469,24 @@ impl NativePairing {
name, name,
fp_hex: fp_hex.to_string(), fp_hex: fp_hex.to_string(),
requested_at: Instant::now(), requested_at: Instant::now(),
src_ip,
parked: false,
}); });
} }
/// Mark/unmark the pending entry for `fp_hex` as having a live parked waiter (no-op if it's gone).
/// A parked entry is protected from eviction under load (#13).
fn set_parked(&self, fp_hex: &str, parked: bool) {
let mut pending = self.pending.lock().unwrap();
if let Some(p) = pending
.items
.iter_mut()
.find(|p| p.fp_hex.eq_ignore_ascii_case(fp_hex))
{
p.parked = parked;
}
}
/// The devices currently awaiting approval (for the management API). /// The devices currently awaiting approval (for the management API).
pub fn pending(&self) -> Vec<PendingRequest> { pub fn pending(&self) -> Vec<PendingRequest> {
let mut pending = self.pending.lock().unwrap(); let mut pending = self.pending.lock().unwrap();
@@ -462,6 +567,23 @@ impl NativePairing {
/// to keep the knocking connection open until a human clicks Approve — so the device pairs and /// to keep the knocking connection open until a human clicks Approve — so the device pairs and
/// streams with no reconnect (delegated approval, roadmap §8b-1). /// streams with no reconnect (delegated approval, roadmap §8b-1).
pub async fn wait_for_decision(&self, fp_hex: &str, timeout: Duration) -> PairingDecision { pub async fn wait_for_decision(&self, fp_hex: &str, timeout: Duration) -> PairingDecision {
// Mark this knock parked so a cert-rotating flood can't evict the genuine, held-open
// connection out of the pending queue while the operator decides (#13). Cleared on every
// exit path by the guard's Drop.
self.set_parked(fp_hex, true);
struct ParkGuard<'a> {
np: &'a NativePairing,
fp: &'a str,
}
impl Drop for ParkGuard<'_> {
fn drop(&mut self) {
self.np.set_parked(self.fp, false);
}
}
let _park = ParkGuard {
np: self,
fp: fp_hex,
};
let deadline = tokio::time::Instant::now() + timeout; let deadline = tokio::time::Instant::now() + timeout;
loop { loop {
// Arm the wakeup BEFORE re-reading state, and `enable()` it, so an approve/deny that // Arm the wakeup BEFORE re-reading state, and `enable()` it, so an approve/deny that
@@ -548,8 +670,8 @@ mod tests {
// A knock appears; a re-knock from the same fingerprint refreshes (same id, new name) // A knock appears; a re-knock from the same fingerprint refreshes (same id, new name)
// instead of duplicating. // instead of duplicating.
np.note_pending("device aa11", "AA11"); np.note_pending("device aa11", "AA11", None);
np.note_pending("Bedroom TV", "aa11"); np.note_pending("Bedroom TV", "aa11", None);
let pend = np.pending(); let pend = np.pending();
assert_eq!(pend.len(), 1, "re-knock dedups by fingerprint"); assert_eq!(pend.len(), 1, "re-knock dedups by fingerprint");
assert_eq!(pend[0].name, "Bedroom TV"); assert_eq!(pend[0].name, "Bedroom TV");
@@ -562,7 +684,7 @@ mod tests {
assert!(!np.is_paired("aa11")); assert!(!np.is_paired("aa11"));
// Approve pairs the fingerprint (operator label wins) and clears the entry. // Approve pairs the fingerprint (operator label wins) and clears the entry.
np.note_pending("device bb22", "BB22"); np.note_pending("device bb22", "BB22", None);
let id = np.pending()[0].id; let id = np.pending()[0].id;
assert!( assert!(
np.approve_pending(9999, None).unwrap().is_none(), np.approve_pending(9999, None).unwrap().is_none(),
@@ -578,8 +700,11 @@ mod tests {
assert_eq!(np.list()[0].name, "Living Room"); assert_eq!(np.list()[0].name, "Living Room");
// The cap evicts the oldest knock. // The cap evicts the oldest knock.
// Flood from many DISTINCT source IPs (so the per-IP cap doesn't kick in) → the global cap
// holds at PENDING_CAP, evicting the oldest non-parked entries first.
for i in 0..(PENDING_CAP + 3) { for i in 0..(PENDING_CAP + 3) {
np.note_pending("flood", &format!("f{i:03}")); let ip = IpAddr::from([10, 0, (i / 256) as u8, (i % 256) as u8]);
np.note_pending("flood", &format!("f{i:03}"), Some(ip));
} }
let pend = np.pending(); let pend = np.pending();
assert_eq!(pend.len(), PENDING_CAP); assert_eq!(pend.len(), PENDING_CAP);
@@ -610,7 +735,7 @@ mod tests {
let p = temp(); let p = temp();
let _ = std::fs::remove_file(&p); let _ = std::fs::remove_file(&p);
let np = NativePairing::load_with(Some(p.clone()), None, false).unwrap(); let np = NativePairing::load_with(Some(p.clone()), None, false).unwrap();
np.note_pending("Knocker", "cc44"); np.note_pending("Knocker", "cc44", None);
assert_eq!(np.pending().len(), 1); assert_eq!(np.pending().len(), 1);
// Pairing the same fingerprint (e.g. via the PIN ceremony) drops the stale pending entry. // Pairing the same fingerprint (e.g. via the PIN ceremony) drops the stale pending entry.
np.add("Knocker", "CC44").unwrap(); np.add("Knocker", "CC44").unwrap();
@@ -656,7 +781,7 @@ mod tests {
let np = Arc::new(NativePairing::load_with(Some(p.clone()), None, false).unwrap()); let np = Arc::new(NativePairing::load_with(Some(p.clone()), None, false).unwrap());
// TimedOut: a parked knock with no decision returns TimedOut; the entry survives. // TimedOut: a parked knock with no decision returns TimedOut; the entry survives.
np.note_pending("Knocker", "ab01"); np.note_pending("Knocker", "ab01", None);
let d = np let d = np
.wait_for_decision("ab01", Duration::from_millis(80)) .wait_for_decision("ab01", Duration::from_millis(80))
.await; .await;
@@ -681,7 +806,7 @@ mod tests {
assert!(np.is_paired("ab01")); assert!(np.is_paired("ab01"));
// Denied: denying WHILE parked wakes the waiter with Denied (not held until timeout). // Denied: denying WHILE parked wakes the waiter with Denied (not held until timeout).
np.note_pending("Knock2", "cd02"); np.note_pending("Knock2", "cd02", None);
let np3 = np.clone(); let np3 = np.clone();
let waiter = let waiter =
tokio::spawn( tokio::spawn(
@@ -703,4 +828,62 @@ mod tests {
assert_eq!(d, PairingDecision::Approved); assert_eq!(d, PairingDecision::Approved);
let _ = std::fs::remove_file(&p); let _ = std::fs::remove_file(&p);
} }
/// #9: a window can be bound to one operator-selected fingerprint, so an unrelated (attacker)
/// fingerprint can neither pair nor BURN the window (it's rejected without a PIN).
#[test]
fn armed_pin_is_fingerprint_bindable() {
let p = temp();
let _ = std::fs::remove_file(&p);
let np = NativePairing::load_with(Some(p.clone()), None, false).unwrap();
// Unbound: any fingerprint resolves to the PIN (legacy behavior).
let pin = np.arm(Duration::from_secs(60));
assert!(matches!(np.pin_for_attempt("aa11"), PinAttempt::Pin(x) if x == pin));
assert!(matches!(np.pin_for_attempt("bb22"), PinAttempt::Pin(_)));
// Bound to AA11: only that fp (case-insensitive) gets the PIN; another fp is BoundToOther —
// the caller rejects it WITHOUT consuming the window.
let pin = np.arm_for(Duration::from_secs(60), Some("AA11".into()));
assert!(matches!(np.pin_for_attempt("aa11"), PinAttempt::Pin(x) if x == pin));
assert!(matches!(
np.pin_for_attempt("bb22"),
PinAttempt::BoundToOther
));
np.disarm();
assert!(matches!(np.pin_for_attempt("aa11"), PinAttempt::Disarmed));
let _ = std::fs::remove_file(&p);
}
/// #13: one source IP can't exceed the per-IP cap, and a parked (held-open) genuine knock is
/// never evicted by a flood — even one that fills the global cap from many distinct IPs.
#[test]
fn pending_per_ip_cap_and_parked_protection() {
let p = temp();
let _ = std::fs::remove_file(&p);
let np = NativePairing::load_with(Some(p.clone()), None, false).unwrap();
// Per-IP cap: one source flooding distinct fingerprints holds at most MAX_PENDING_PER_IP.
let attacker = IpAddr::from([192, 168, 1, 66]);
for i in 0..20 {
np.note_pending("flood", &format!("atk{i:03}"), Some(attacker));
}
assert_eq!(
np.pending().len(),
MAX_PENDING_PER_IP,
"one IP can't exceed the per-IP cap"
);
// A genuine knock from a different IP, parked (a live held-open connection), survives a flood
// from many distinct IPs that fills the global cap.
let legit = IpAddr::from([192, 168, 1, 50]);
np.note_pending("Living Room", "legit01", Some(legit));
np.set_parked("legit01", true);
for i in 0..(PENDING_CAP * 2) {
let ip = IpAddr::from([10, 0, (i / 256) as u8, (i % 256) as u8]);
np.note_pending("flood2", &format!("g{i:04}"), Some(ip));
}
assert!(
np.pending_contains("legit01"),
"a parked, held-open knock is never evicted by a flood"
);
assert!(np.pending().len() <= PENDING_CAP, "global cap still holds");
let _ = std::fs::remove_file(&p);
}
} }
+165 -508
View File
@@ -532,10 +532,25 @@ async fn serve_session(
.await .await
.map_err(|_| anyhow!("first message timeout"))??; .map_err(|_| anyhow!("first message timeout"))??;
if let Ok(req) = PairRequest::decode(&first) { if let Ok(req) = PairRequest::decode(&first) {
// Read the live arming PIN per attempt, so a window that lapsed no longer pairs. // The client fingerprint (cert possession is proven by the QUIC handshake) is needed to honor
let pin = np // a fingerprint-bound PIN window (#9): a window the operator armed for a SPECIFIC device must
.current_pin() // not be consumable — or burnable — by any other fingerprint.
.context("pairing not armed (arm it in the console, or start with --allow-pairing)")?; let client_fp = endpoint::peer_fingerprint(&conn)
.ok_or_else(|| anyhow!("pairing requires the client to present a certificate"))?;
let client_fp_hex = fingerprint_hex(&client_fp);
// Resolve the live arming PIN per attempt (so a lapsed window no longer pairs), honoring any
// fingerprint binding.
let pin = match np.pin_for_attempt(&client_fp_hex) {
crate::native_pairing::PinAttempt::Pin(pin) => pin,
crate::native_pairing::PinAttempt::Disarmed => anyhow::bail!(
"pairing not armed (arm it in the console, or start with --allow-pairing)"
),
// Armed for a DIFFERENT device — reject without running the ceremony, so this attempt does
// NOT consume (burn) the operator's window for the device they actually selected (#9).
crate::native_pairing::PinAttempt::BoundToOther => anyhow::bail!(
"pairing is armed for a different device — this attempt does not consume the window"
),
};
{ {
let mut last = last_pairing.lock().unwrap(); let mut last = last_pairing.lock().unwrap();
if let Some(t) = *last { if let Some(t) = *last {
@@ -589,7 +604,9 @@ async fn serve_session(
); );
tracing::info!(name = %label, fingerprint = %fp_hex, tracing::info!(name = %label, fingerprint = %fp_hex,
"unpaired device knocked — parking connection for delegated approval in the console"); "unpaired device knocked — parking connection for delegated approval in the console");
np.note_pending(&label, &fp_hex); // Record the QUIC-validated source IP so the pending queue's per-source cap can stop one
// host from flooding/evicting genuine knocks (#13).
np.note_pending(&label, &fp_hex, Some(peer.ip()));
// Free the session slot while a human decides — a parked knock must not hold an NVENC // Free the session slot while a human decides — a parked knock must not hold an NVENC
// permit (a handful of parked knocks would otherwise block every real session). // permit (a handful of parked knocks would otherwise block every real session).
drop(permit); drop(permit);
@@ -738,14 +755,18 @@ async fn serve_session(
// opens a tiny encoder; it runs only when both opt-ins are set and is cached after the first. // opens a tiny encoder; it runs only when both opt-ins are set and is cached after the first.
let host_wants_444 = crate::config::config().four_four_four; let host_wants_444 = crate::config::config().four_four_four;
let client_supports_444 = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_444 != 0; let client_supports_444 = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_444 != 0;
let single_process = crate::session_plan::resolve_topology() // The active capturer must be able to deliver a full-chroma (RGB) source — the honest-downgrade
== crate::session_plan::SessionTopology::SingleProcess; // gate. Linux's portal capturer can; the Windows IDD-push path delivers subsampled NV12/P010
// today (full-chroma IDD-push capture is a follow-up), so it returns false there and the host
// negotiates 4:2:0. (Replaces the old `single_process` gate — single-process is now the only
// topology, and 4:4:4 routed to DDA, which was removed.)
let capture_supports_444 = crate::capture::capturer_supports_444();
// The GPU probe opens a real (tiny) encoder on first use, so run it off the reactor like the // The GPU probe opens a real (tiny) encoder on first use, so run it off the reactor like the
// compositor probe above (blocking probes → spawn_blocking). Short-circuit so it only runs when // compositor probe above (blocking probes → spawn_blocking). Short-circuit so it only runs when
// the cheap gates already pass. The result is cached process-wide (a negative latches until // the cheap gates already pass. The result is cached process-wide (a negative latches until
// restart — acceptable: a GPU either supports HEVC 4:4:4 or it doesn't, and a transient open // restart — acceptable: a GPU either supports HEVC 4:4:4 or it doesn't, and a transient open
// failure here is rare since the session's own encoder isn't open yet). // failure here is rare since the session's own encoder isn't open yet).
let gpu_supports_444 = if host_wants_444 && client_supports_444 && single_process { let gpu_supports_444 = if host_wants_444 && client_supports_444 && capture_supports_444 {
tokio::task::spawn_blocking(|| { tokio::task::spawn_blocking(|| {
crate::encode::can_encode_444(crate::encode::Codec::H265) crate::encode::can_encode_444(crate::encode::Codec::H265)
}) })
@@ -763,7 +784,7 @@ async fn serve_session(
chroma = ?chroma, chroma = ?chroma,
host_wants_444, host_wants_444,
client_supports_444, client_supports_444,
single_process, capture_supports_444,
"encode chroma" "encode chroma"
); );
@@ -1382,6 +1403,8 @@ enum PadBackend {
DualSense(crate::inject::dualsense::DualSenseManager), DualSense(crate::inject::dualsense::DualSenseManager),
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
DualShock4(crate::inject::dualshock4::DualShock4Manager), DualShock4(crate::inject::dualshock4::DualShock4Manager),
#[cfg(target_os = "linux")]
SteamDeck(crate::inject::steam_controller::SteamControllerManager),
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
DualSenseWindows(crate::inject::dualsense_windows::DualSenseWindowsManager), DualSenseWindows(crate::inject::dualsense_windows::DualSenseWindowsManager),
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
@@ -1403,6 +1426,12 @@ impl PadBackend {
tracing::info!("gamepad backend: virtual DualShock 4 (UHID hid-playstation)"); tracing::info!("gamepad backend: virtual DualShock 4 (UHID hid-playstation)");
return PadBackend::DualShock4(crate::inject::dualshock4::DualShock4Manager::new()); return PadBackend::DualShock4(crate::inject::dualshock4::DualShock4Manager::new());
} }
GamepadPref::SteamDeck => {
tracing::info!("gamepad backend: virtual Steam Deck (UHID hid-steam)");
return PadBackend::SteamDeck(
crate::inject::steam_controller::SteamControllerManager::new(),
);
}
GamepadPref::XboxOne => { GamepadPref::XboxOne => {
tracing::info!("gamepad backend: uinput X-Box One/Series pad"); tracing::info!("gamepad backend: uinput X-Box One/Series pad");
return PadBackend::Xbox360(crate::inject::gamepad::GamepadManager::with_identity( return PadBackend::Xbox360(crate::inject::gamepad::GamepadManager::with_identity(
@@ -1438,6 +1467,8 @@ impl PadBackend {
PadBackend::DualSense(m) => m.handle(ev), PadBackend::DualSense(m) => m.handle(ev),
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
PadBackend::DualShock4(m) => m.handle(ev), PadBackend::DualShock4(m) => m.handle(ev),
#[cfg(target_os = "linux")]
PadBackend::SteamDeck(m) => m.handle(ev),
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
PadBackend::DualSenseWindows(m) => m.handle(ev), PadBackend::DualSenseWindows(m) => m.handle(ev),
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
@@ -1454,6 +1485,8 @@ impl PadBackend {
PadBackend::DualSense(m) => m.apply_rich(_rich), PadBackend::DualSense(m) => m.apply_rich(_rich),
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
PadBackend::DualShock4(m) => m.apply_rich(_rich), PadBackend::DualShock4(m) => m.apply_rich(_rich),
#[cfg(target_os = "linux")]
PadBackend::SteamDeck(m) => m.apply_rich(_rich),
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
PadBackend::DualSenseWindows(m) => m.apply_rich(_rich), PadBackend::DualSenseWindows(m) => m.apply_rich(_rich),
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
@@ -1479,6 +1512,8 @@ impl PadBackend {
PadBackend::DualSense(m) => m.pump(rumble, hidout), PadBackend::DualSense(m) => m.pump(rumble, hidout),
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
PadBackend::DualShock4(m) => m.pump(rumble, hidout), PadBackend::DualShock4(m) => m.pump(rumble, hidout),
#[cfg(target_os = "linux")]
PadBackend::SteamDeck(m) => m.pump(rumble, hidout),
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
PadBackend::DualSenseWindows(m) => m.pump(rumble, hidout), PadBackend::DualSenseWindows(m) => m.pump(rumble, hidout),
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
@@ -1498,6 +1533,8 @@ impl PadBackend {
PadBackend::DualSense(m) => m.heartbeat(std::time::Duration::from_millis(8)), PadBackend::DualSense(m) => m.heartbeat(std::time::Duration::from_millis(8)),
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
PadBackend::DualShock4(m) => m.heartbeat(std::time::Duration::from_millis(8)), PadBackend::DualShock4(m) => m.heartbeat(std::time::Duration::from_millis(8)),
#[cfg(target_os = "linux")]
PadBackend::SteamDeck(m) => m.heartbeat(std::time::Duration::from_millis(8)),
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
PadBackend::DualSenseWindows(m) => m.heartbeat(std::time::Duration::from_millis(8)), PadBackend::DualSenseWindows(m) => m.heartbeat(std::time::Duration::from_millis(8)),
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
@@ -1877,10 +1914,94 @@ fn pick_gamepad(pref: GamepadPref, env: Option<&str>, linux: bool, windows: bool
// One/Series: a real, distinct uinput identity on Linux; folded into the 360 backend on // One/Series: a real, distinct uinput identity on Linux; folded into the 360 backend on
// Windows (XInput can't tell them apart anyway). // Windows (XInput can't tell them apart anyway).
GamepadPref::XboxOne if linux => GamepadPref::XboxOne, GamepadPref::XboxOne if linux => GamepadPref::XboxOne,
// Steam Deck: Linux UHID hid-steam. The classic Steam Controller's backend isn't built yet,
// so it folds to Xbox360 for now (Windows Steam devices are M7).
GamepadPref::SteamDeck if linux => GamepadPref::SteamDeck,
_ => GamepadPref::Xbox360, _ => GamepadPref::Xbox360,
} }
} }
/// Runtime degrade for the Linux UHID backends (DualSense / DualShock 4 / Steam Deck): if
/// `/dev/uhid` can't be opened for write *now*, fall back to the uinput X-Box 360 pad rather than a
/// dead controller (the UHID device-create would just fail). Cheap — opens + drops the char device,
/// no `UHID_CREATE2`, so no device is created. A no-op on non-Linux (those backends are UMDF/uinput).
#[cfg(target_os = "linux")]
fn degrade_if_no_uhid(chosen: GamepadPref) -> GamepadPref {
let needs_uhid = matches!(
chosen,
GamepadPref::DualSense | GamepadPref::DualShock4 | GamepadPref::SteamDeck
);
if needs_uhid
&& std::fs::OpenOptions::new()
.write(true)
.open("/dev/uhid")
.is_err()
{
tracing::warn!(
wanted = chosen.as_str(),
"/dev/uhid not writable — falling back to the X-Box 360 pad"
);
return GamepadPref::Xbox360;
}
chosen
}
#[cfg(not(target_os = "linux"))]
fn degrade_if_no_uhid(chosen: GamepadPref) -> GamepadPref {
chosen
}
/// True if a **physical** Valve Steam controller (`28DE`) is already attached. The host's own Steam
/// Input is then managing a `28DE` device, and presenting a second (virtual) one makes Steam juggle
/// two Decks — confirmed conflict-prone on a Deck-as-host (the physical `28DE:1205` + Steam's
/// `28DE:11FF` XInput output pad are both live). HID device dirs are named `BUS:VID:PID.INST`
/// (uppercase); a UHID virtual device resolves through `/devices/virtual/…`, a real one does not.
#[cfg(target_os = "linux")]
fn physical_steam_controller_present() -> bool {
let Ok(entries) = std::fs::read_dir("/sys/bus/hid/devices") else {
return false;
};
entries.flatten().any(|e| {
if !e.file_name().to_string_lossy().contains(":28DE:") {
return false;
}
match std::fs::read_link(e.path()) {
Ok(target) => !target.to_string_lossy().contains("/virtual/"),
Err(_) => true,
}
})
}
/// Gate a virtual Steam pad off when a physical Steam controller is attached (§ conflict). Degrade to
/// DualSense (then the uhid ladder), which Steam treats as an ordinary, distinct pad. Override with
/// `PUNKTFUNK_STEAM_FORCE=1` when the host has no competing Steam Input (e.g. a remote-only box).
#[cfg(target_os = "linux")]
fn degrade_steam_on_conflict(chosen: GamepadPref) -> GamepadPref {
if !matches!(
chosen,
GamepadPref::SteamDeck | GamepadPref::SteamController
) {
return chosen;
}
let forced = std::env::var("PUNKTFUNK_STEAM_FORCE")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(false);
if !forced && physical_steam_controller_present() {
tracing::warn!(
wanted = chosen.as_str(),
"a physical Steam controller is attached — the host's Steam Input would manage two 28DE \
devices; falling back to DualSense (set PUNKTFUNK_STEAM_FORCE=1 to override)"
);
return degrade_if_no_uhid(GamepadPref::DualSense);
}
chosen
}
#[cfg(not(target_os = "linux"))]
fn degrade_steam_on_conflict(chosen: GamepadPref) -> GamepadPref {
chosen
}
/// Resolve the client's gamepad-backend preference (the env/logging shell around /// Resolve the client's gamepad-backend preference (the env/logging shell around
/// [`pick_gamepad`]). Always concrete — the `Welcome` reports what the session will drive. /// [`pick_gamepad`]). Always concrete — the `Welcome` reports what the session will drive.
fn resolve_gamepad(pref: GamepadPref) -> GamepadPref { fn resolve_gamepad(pref: GamepadPref) -> GamepadPref {
@@ -1891,6 +2012,14 @@ fn resolve_gamepad(pref: GamepadPref) -> GamepadPref {
cfg!(target_os = "linux"), cfg!(target_os = "linux"),
cfg!(target_os = "windows"), cfg!(target_os = "windows"),
); );
// Runtime degrade (separate from the compile-time platform check above): the Linux UHID
// backends need `/dev/uhid` usable *now*, else creating the device just fails and the controller
// goes dead — fall back to the always-available uinput X-Box 360 pad instead.
let chosen = degrade_if_no_uhid(chosen);
// Conflict gate: don't present a virtual Steam (28DE) pad when the host already has a physical
// Steam controller — its own Steam Input would then manage two Decks (confirmed conflict-prone on
// a Deck-as-host). `PUNKTFUNK_STEAM_FORCE=1` overrides.
let chosen = degrade_steam_on_conflict(chosen);
match pref { match pref {
GamepadPref::Auto => { GamepadPref::Auto => {
// The operator's env knob deserves a diagnostic when it didn't drive the // The operator's env knob deserves a diagnostic when it didn't drive the
@@ -2571,7 +2700,7 @@ fn session_watcher_loop(tx: std::sync::mpsc::Sender<SessionSwitch>, stop: Arc<At
} }
} }
/// All per-session inputs for [`virtual_stream`] / [`virtual_stream_relay`], bundled so the session entry /// All per-session inputs for [`virtual_stream`], bundled so the session entry
/// is one moved value instead of a 13-positional-argument `#[allow(too_many_arguments)]` signature /// is one moved value instead of a 13-positional-argument `#[allow(too_many_arguments)]` signature
/// (Goal-1 stage 4, plan §2.4). Everything is **owned** — the receivers move in (`virtual_stream` is their /// (Goal-1 stage 4, plan §2.4). Everything is **owned** — the receivers move in (`virtual_stream` is their
/// only consumer) — so the whole context moves into the stream thread and the borrow plumbing disappears. /// only consumer) — so the whole context moves into the stream thread and the borrow plumbing disappears.
@@ -2619,8 +2748,9 @@ struct SessionContext {
} }
fn virtual_stream(ctx: SessionContext) -> Result<()> { fn virtual_stream(ctx: SessionContext) -> Result<()> {
// This thread runs the capture+encode loop (single-process: Linux / synthetic / NO_WGC DDA) — or // This thread runs the capture+encode loop (single-process — the only topology: Linux portal /
// tail-calls the relay below. Elevate it so a CPU-heavy game can't deschedule our GPU submission. // synthetic, Windows in-process IDD-push). Elevate it so a CPU-heavy game can't deschedule our GPU
// submission.
boost_thread_priority(true); boost_thread_priority(true);
// Resolve the per-session capture / topology / encoder decision ONCE (Goal-1 stage 3): the deployed // Resolve the per-session capture / topology / encoder decision ONCE (Goal-1 stage 3): the deployed
// path now reads this typed `SessionPlan` instead of re-deriving from config at each dispatch site // path now reads this typed `SessionPlan` instead of re-deriving from config at each dispatch site
@@ -2628,14 +2758,6 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
// only per-session input — capture/topology/encoder are otherwise pure functions of `HostConfig`. // only per-session input — capture/topology/encoder are otherwise pure functions of `HostConfig`.
let plan = crate::session_plan::SessionPlan::resolve(ctx.bit_depth, ctx.chroma); let plan = crate::session_plan::SessionPlan::resolve(ctx.bit_depth, ctx.chroma);
tracing::info!(?plan, "resolved session plan"); tracing::info!(?plan, "resolved session plan");
// Windows two-process secure-desktop path: when the host runs as SYSTEM (required for the secure
// desktop + SendInput), WGC can't activate in-process, so we capture the normal desktop via a
// helper spawned in the user session and relay its AUs. (Single-process WGC/DDA is used as the
// user, and stays the path on Linux.) See design/archive/windows-secure-desktop.md.
#[cfg(target_os = "windows")]
if plan.topology == crate::session_plan::SessionTopology::TwoProcessRelay {
return virtual_stream_relay(ctx);
}
// Single-process path: unpack the context into the locals the loop below uses (names unchanged, so the // Single-process path: unpack the context into the locals the loop below uses (names unchanged, so the
// body is byte-for-byte the same; the receivers are now owned but `try_recv()` is identical). // body is byte-for-byte the same; the receivers are now owned but `try_recv()` is identical).
let SessionContext { let SessionContext {
@@ -2670,6 +2792,11 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
// host-lifetime VirtualDisplayManager (§2.5). It does NO monitor work, so it must precede the IDD-push // host-lifetime VirtualDisplayManager (§2.5). It does NO monitor work, so it must precede the IDD-push
// preempt below (which reaches the manager) — otherwise `vdm()` is called before init and panics. // preempt below (which reaches the manager) — otherwise `vdm()` is called before init and panics.
let mut vd = crate::vdisplay::open(compositor)?; let mut vd = crate::vdisplay::open(compositor)?;
// Per-client STABLE monitor identity (Phase 2): hand the backend the connecting client's cert
// fingerprint so a freshly CREATED virtual monitor gets this client's persistent id — Windows then
// reapplies the client's saved per-monitor config (DPI scaling) on reconnect. No-op on Linux backends
// and for anonymous/GameStream clients (no fingerprint → the driver auto-allocates).
vd.set_client_identity(endpoint::peer_fingerprint(&conn));
// IDD-push reconnect preempt (the dance now lives in the manager, Goal-1 §2.5): serialize setup so a // IDD-push reconnect preempt (the dance now lives in the manager, Goal-1 §2.5): serialize setup so a
// reconnect FLOOD can't run concurrent monitor create/teardown, STOP the prior session + WAIT for it // reconnect FLOOD can't run concurrent monitor create/teardown, STOP the prior session + WAIT for it
// to release its monitor (instead of tearing a monitor out from under a still-live session), and // to release its monitor (instead of tearing a monitor out from under a still-live session), and
@@ -2685,20 +2812,7 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
drop(_idd_setup_guard); drop(_idd_setup_guard);
// Windows single-process DDA path (PUNKTFUNK_NO_WGC=1): the SudoVDA virtual display, isolated as the // Windows: capture is live — launch the requested library title into the
// SOLE active output, goes into fullscreen independent-flip (one plane on one display) which Desktop
// Duplication cannot capture → the born-lost ACCESS_LOST storm we measured on the RTX4090+iGPU box
// (hook verified-firing, DPI=2, yet 100% DuplicateOutput1 E_ACCESSDENIED + born-lost). A tiny topmost
// layered overlay disqualifies independent-flip and forces DWM composition, which DDA CAN capture.
// (Apollo never hits this because it runs WITH a physical monitor attached — multi-display is already
// DWM-composited; we isolate to sole-display, so we must force composition ourselves.) Unlike the WGC
// relay path — where WGC owns the normal desktop and the overlay is secure-only — here DDA owns the
// normal desktop too, so it must run unconditionally. Held for the session; Drop tears it down.
// Best-effort; disable with PUNKTFUNK_FORCE_COMPOSED=0.
#[cfg(target_os = "windows")]
let _composed_flip = crate::capture::composed_flip::ForceComposedFlip::start();
// Windows: capture is live (and composition forced) — launch the requested library title into the
// interactive user session so it renders onto the captured desktop and grabs foreground. Linux // interactive user session so it renders onto the captured desktop and grabs foreground. Linux
// nests its launch in gamescope instead (the handshake `PUNKTFUNK_GAMESCOPE_APP` path). Best-effort: // nests its launch in gamescope instead (the handshake `PUNKTFUNK_GAMESCOPE_APP` path). Best-effort:
// a launch failure (no recipe for the kind, no interactive user) leaves the user on the desktop. // a launch failure (no recipe for the kind, no interactive user) leaves the user on the desktop.
@@ -3170,480 +3284,6 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
Ok(()) Ok(())
} }
/// Windows two-process video stream: the SYSTEM host creates the SudoVDA virtual output (and holds
/// its keepalive = the sole topology/isolation owner), spawns the WGC helper in the user session to
/// capture+encode the NORMAL desktop, and relays the helper's AUs onto the QUIC data plane via the
/// same send thread as the single-process path. A [`DesktopWatcher`](crate::capture::desktop_watch)
/// muxes the source: while the input desktop is Winlogon (UAC / lock / login — which WGC can't
/// capture), the host captures it with its OWN DDA encoder; back on Default it resumes the relay.
/// Every source switch latches a "wait for IDR" so the client's decoder resumes on a keyframe (the
/// two encoders keep independent infinite-GOP state). Reconfigure rebuilds the output + re-spawns the
/// helper at the new mode (and drops the stale-target DDA); keyframe requests forward to the active
/// source.
#[cfg(target_os = "windows")]
fn virtual_stream_relay(ctx: SessionContext) -> Result<()> {
use crate::capture::dxgi::WinCaptureTarget;
use crate::capture::wgc_relay::HelperRelay;
use crate::capture::Capturer; // trait methods (set_active/next_frame) on the concrete DuplCapturer
// Unpack the context (names unchanged so the body is identical). The relay doesn't yet send the
// source's 0xCE HDR metadata — the helper's in-band SEI carries it (a Windows follow-up) — so `conn`
// is held unused.
let SessionContext {
session,
mode,
seconds,
stop,
reconfig,
keyframe,
compositor,
bitrate_kbps,
bit_depth,
// The two-process WGC relay encodes 4:2:0 in v1 — the handshake's `single_process` gate already
// forced `chroma` to Yuv420 for this topology, so the helper + secure-desktop DDA stay 4:2:0.
chroma: _,
probe_rx,
probe_result_tx,
fec_target,
conn: _conn,
stats,
client_label,
launch,
} = ctx;
tracing::info!(
?mode,
bitrate_kbps,
bit_depth,
"punktfunk/1 two-process stream (SYSTEM host + user-session WGC helper)"
);
let mut vd = crate::vdisplay::open(compositor)?;
// Create the SudoVDA output + spawn a helper capturing it by GDI name. Returns the keepalive
// (held for the output's life — the sole isolation owner), the running relay, the capture target
// (so the host can also open DDA on it for the secure desktop), and the achieved refresh.
type Built = (Box<dyn Send>, HelperRelay, WinCaptureTarget, u32);
let build = |vd: &mut Box<dyn crate::vdisplay::VirtualDisplay>,
mode: punktfunk_core::Mode|
-> Result<Built> {
let vout = vd.create(mode).context("create virtual output")?;
let effective_hz = vout
.preferred_mode
.map(|(_, _, hz)| hz)
.filter(|&hz| hz > 0)
.unwrap_or(mode.refresh_hz);
let target = vout.win_capture.clone().ok_or_else(|| {
anyhow!("SudoVDA target not yet an active display (needs a WDDM GPU to activate it)")
})?;
// HDR is driven by the SudoVDA monitor's ACTUAL advanced-color state, not the handshake bit
// depth: the whole pipeline follows the monitor (WGC captures FP16 when HDR is on; NVENC forces
// Main10 + BT.2020 PQ from the 10-bit capture format regardless of the negotiated depth; the
// client auto-detects PQ from the HEVC VUI). So:
// - a negotiated 10-bit session PROACTIVELY enables HDR on the monitor (below), but
// - we must NEVER force HDR *off* here — that would wipe out a user's deliberate Windows HDR
// toggle on the virtual display on every build (the "HDR doesn't persist" bug). Leaving the
// monitor's state alone lets a user-enabled HDR session flow through end-to-end.
// The secure-desktop HDR drop (for the DDA leg) keys off the monitor's real state in the mux loop.
#[cfg(target_os = "windows")]
if bit_depth >= 10 {
// SAFETY: `set_advanced_color` is marked `unsafe` only because it drives the Win32 CCD API
// internally; it takes `target_id` by value (Copy `u32` — this session's live SudoVDA
// monitor's CCD target id) and sizes + owns every buffer it hands the OS on its own stack.
// We pass no pointers, so nothing must outlive the call and there is no aliasing; an
// unknown/absent target id simply returns false.
unsafe {
if crate::win_display::set_advanced_color(target.target_id, true) {
// Let the colorspace change settle before WGC creates its capture item / detects HDR.
std::thread::sleep(std::time::Duration::from_millis(250));
}
}
}
let relay = HelperRelay::spawn(
&target,
(mode.width, mode.height, effective_hz),
bitrate_kbps,
bit_depth,
)
.context("spawn WGC helper")?;
Ok((vout.keepalive, relay, target, effective_hz))
};
let (mut _keepalive, mut relay, mut target, mut effective_hz) = build(&mut vd, mode)?;
let mut cur_mode = mode;
// Capture is live (the WGC helper is relaying) — launch the requested library title into the
// interactive user session so it renders onto the captured desktop and grabs foreground.
// Best-effort: a failure (no recipe for the kind, no interactive user) leaves the user on the desktop.
if let Some(id) = launch.as_deref() {
if let Err(e) = crate::library::launch_title(id) {
tracing::warn!(launch_id = id, error = %e, "could not launch requested library title");
}
}
// O3.1: optionally observe the IDD-push ring alongside WGC (WGC = the presentation trigger) to
// confirm the 0257 driver pushes frames into a HOST-created ring. Diagnostic only; gated.
if std::env::var_os("PUNKTFUNK_IDD_PUSH_OBSERVE").is_some() {
crate::capture::idd_push::spawn_observer(
target.clone(),
Some((cur_mode.width, cur_mode.height, effective_hz)),
);
}
// The host's own DDA capturer+encoder for the SECURE (Winlogon) desktop, which WGC — and thus the
// helper — cannot capture. Opened lazily on the first secure transition (so a session that never
// hits a UAC/lock screen never pays for a second NVENC session), then kept for fast re-switch.
struct DdaPipe {
cap: Box<dyn crate::capture::Capturer>,
enc: Box<dyn crate::encode::Encoder>,
frame: crate::capture::CapturedFrame,
}
// Note: takes the dimensions as args rather than capturing `cur_mode` — `cur_mode` is reassigned
// on reconfig, and a closure holding a shared borrow of it for the whole fn would forbid that.
let open_dda =
|target: &WinCaptureTarget, w: u32, h: u32, hz: u32, hdr: bool| -> Result<DdaPipe> {
// The host already holds the real keepalive (sole isolation owner), so DDA gets a no-op one.
// `hdr` requests an FP16 DuplicateOutput1 so the secure desktop is captured in HDR (→ BT.2020
// PQ Main10) instead of black — legacy DuplicateOutput can't capture an HDR/FP16 desktop.
let mut cap = crate::capture::dxgi::DuplCapturer::open(
target.clone(),
Some((w, h, hz)),
Box::new(()),
// The relay's host encoder is GPU (NVENC/AMF/QSV unless software) — pass `gpu` in (Goal-1
// stage 5) so the DDA capturer doesn't re-derive it.
crate::capture::gpu_encode(),
hdr,
false, // the two-process relay path is 4:2:0 in v1
)
.context("open DDA for secure desktop")?;
cap.set_active(true);
let frame = cap.next_frame().context("DDA first frame")?;
let enc = crate::encode::open_video(
crate::encode::Codec::H265,
frame.format,
frame.width,
frame.height,
hz,
bitrate_kbps as u64 * 1000,
frame.is_cuda(),
bit_depth,
// Secure-desktop DDA on the two-process relay path: 4:2:0 in v1 (matches the helper).
crate::encode::ChromaFormat::Yuv420,
)
.context("open video encoder for DDA")?;
Ok(DdaPipe {
cap: Box::new(cap),
enc,
frame,
})
};
let perf = crate::config::config().perf;
let burst_cap = std::env::var("PUNKTFUNK_PACE_BURST_KB")
.ok()
.and_then(|s| s.parse::<usize>().ok())
.unwrap_or(128)
* 1024;
// Same encode|send split as the single-process path: this thread relays AUs, a dedicated send
// thread owns the Session and does FEC+seal+paced-send. The relay encodes in the helper process,
// so this path's FrameMsgs carry no cap/submit/encode split (those stages stay 0 in the sample);
// the send thread still emits fps/goodput/pacing/loss from `session.stats()`.
let send_stats = SendStats {
rec: stats,
width: mode.width,
height: mode.height,
fps: effective_hz,
codec: "hevc",
client: client_label,
bitrate_kbps,
};
let (frame_tx, frame_rx) = std::sync::mpsc::sync_channel::<FrameMsg>(3);
let send_thread = std::thread::Builder::new()
.name("punktfunk-send".into())
.spawn({
let stop = stop.clone();
move || {
send_loop(
session,
frame_rx,
probe_rx,
probe_result_tx,
stop,
perf,
burst_cap,
fec_target,
send_stats,
)
}
})
.context("spawn send thread")?;
// Test hook: PUNKTFUNK_SECURE_TEST_PERIOD_MS=N drives a square-wave secure/normal toggle every N ms
// instead of the real watcher — exercises the mid-session helper↔DDA mux without a live UAC/lock.
let secure_test_ms: Option<u128> = std::env::var("PUNKTFUNK_SECURE_TEST_PERIOD_MS")
.ok()
.and_then(|s| s.parse().ok())
.filter(|&n| n > 0);
// Switching to the host DDA on the secure (Winlogon) desktop is OPT-IN: DDA can't reliably capture
// the secure desktop's HDR independent-flip (it storms ACCESS_LOST → black), whereas the WGC helper
// STAYS LIVE through a lock/UAC. So by default the mux keeps WGC the whole time (no DesktopWatcher
// switch, no overlay). Enable the experimental DDA-on-secure path with PUNKTFUNK_SECURE_DDA=1.
let dda_secure = crate::config::config().secure_dda || secure_test_ms.is_some();
// The authoritative Default↔Winlogon signal (requires SYSTEM to read the Winlogon desktop name);
// only needed when the DDA-on-secure path is enabled.
let watcher = dda_secure.then(crate::capture::desktop_watch::DesktopWatcher::start);
// Force-composed-flip overlay (only with DDA-on-secure): keeps the secure desktop out of fullscreen
// independent-flip so DDA can duplicate it. Off by default to avoid touching the normal desktop.
let _composed_flip = dda_secure
.then(crate::capture::composed_flip::ForceComposedFlip::start)
.flatten();
let start = std::time::Instant::now();
let mut interval = std::time::Duration::from_secs_f64(1.0 / effective_hz.max(1) as f64);
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(seconds as u64);
let mut sent: u64 = 0;
// Mux state: which source is live, the lazily-opened DDA pipe, a DDA pacing clock, and a
// "wait for the next IDR before forwarding" latch set on every source switch (the client's
// decoder must resume on a keyframe — the two encoders keep independent infinite-GOP state).
let mut dda: Option<DdaPipe> = None;
let mut on_secure = false;
let mut next = std::time::Instant::now();
let mut await_idr = false;
// Step 6 relaunch watchdog: how many times in a row the helper has died without producing a frame.
// A console disconnect/reconnect or a helper crash kills it; we respawn (the new helper picks up
// the now-active session via WTSGetActiveConsoleSessionId). Reset on the first relayed frame; only
// give up (end the stream) after a run of failures spanning a few seconds.
let mut helper_fails = 0u32;
const MAX_HELPER_FAILS: u32 = 20;
// Build a FrameMsg + hand it to the send thread; returns false if the send thread is gone (caller
// breaks the loop). Kept as a macro (not a closure) so each use borrows `frame_tx`/`sent`/`interval`
// at its own site without a long-lived capture, and `break 'outer` stays a literal at the call site
// (a `break 'outer` inside the macro body risks label-hygiene resolution failures).
macro_rules! forward {
($data:expr, $capture_ns:expr, $keyframe:expr) => {{
let flags = if $keyframe {
(FLAG_PIC | FLAG_SOF) as u32
} else {
FLAG_PIC as u32
};
let capture_ns = $capture_ns;
let encode_us = (now_ns().saturating_sub(capture_ns) / 1000) as u32;
let msg = FrameMsg {
data: $data,
capture_ns,
flags,
deadline: std::time::Instant::now() + interval,
encode_us,
cap_us: 0,
submit_us: 0,
wait_us: 0,
repeat: false,
was_measured: false,
};
let ok = frame_tx.send(msg).is_ok();
if ok {
sent += 1;
}
ok
}};
}
'outer: while !stop.load(Ordering::SeqCst) && std::time::Instant::now() < deadline {
// Mode switch: rebuild the output + re-spawn the helper at the new mode (drop the old relay +
// keepalive only after the new pair is up, so a failed rebuild keeps the current stream). The
// DDA pipe (on the old target) is dropped — it reopens on the next secure transition.
let mut want = None;
while let Ok(m) = reconfig.try_recv() {
want = Some(m);
}
if let Some(new_mode) = want {
tracing::info!(?new_mode, "two-process: rebuilding for mode switch");
match build(&mut vd, new_mode) {
Ok((ka, rl, tg, hz)) => {
relay = rl; // drops the old relay (kills old helper) ...
_keepalive = ka; // ... then releases the old output
target = tg;
effective_hz = hz;
cur_mode = new_mode;
dda = None; // old-target DDA is stale; reopen on next secure
interval = std::time::Duration::from_secs_f64(1.0 / hz.max(1) as f64);
}
Err(e) => {
tracing::error!(error = %format!("{e:#}"), ?new_mode,
"two-process mode-switch rebuild failed — staying on the current mode");
}
}
}
// Coalesce client decode-recovery keyframe requests and forward to the active source.
let mut want_kf = false;
while keyframe.try_recv().is_ok() {
want_kf = true;
}
// Source mux: capture the secure (Winlogon) desktop via the host's DDA, the normal desktop via
// the helper relay. On a switch, latch await_idr + force the now-active source to emit an IDR
// so the client resumes cleanly.
let secure = dda_secure
&& match secure_test_ms {
Some(p) => (start.elapsed().as_millis() / p) % 2 == 1,
None => watcher.as_ref().is_some_and(|w| w.is_secure()),
};
if secure != on_secure {
on_secure = secure;
await_idr = true;
tracing::info!(
to = if secure {
"secure(DDA)"
} else {
"normal(WGC relay)"
},
"two-process: source switch"
);
if secure {
// Capture the secure (Winlogon) desktop in its NATIVE colorspace. Don't try to drop the
// SudoVDA out of HDR for the DDA leg — display-config changes are denied on the secure
// desktop (the drop just churned + still went black). Instead, if the monitor is in HDR,
// open DDA in HDR (FP16 DuplicateOutput1 → BT.2020 PQ Main10); the normal-desktop DDA
// overlay/flip issues that drove us to WGC don't apply to the composed Winlogon UI.
// SAFETY: `advanced_color_enabled` is `unsafe` only because it queries the Win32 CCD
// API; it takes `target_id` by value (the live SudoVDA monitor's CCD target id) and
// allocates + owns every buffer it passes the OS internally. No caller pointer is
// involved, so nothing must outlive the call and there is no aliasing; a missing
// target id just yields false.
let hdr = unsafe { crate::win_display::advanced_color_enabled(target.target_id) };
dda = None; // reopen to capture the secure desktop
match open_dda(&target, cur_mode.width, cur_mode.height, effective_hz, hdr) {
Ok(mut p) => {
tracing::info!(hdr, "two-process: opened DDA for the secure desktop");
p.enc.request_keyframe();
dda = Some(p);
}
Err(e) => {
tracing::error!(error = %format!("{e:#}"),
"two-process: DDA open failed — secure desktop will freeze on last frame");
}
}
next = std::time::Instant::now();
} else {
// Returning to the normal desktop: RESUME from the still-alive WGC helper. Do NOT
// recreate the SudoVDA monitor or respawn the helper — build()'s vd.create() is an
// IOCTL_REMOVE+ADD of the monitor (the audible disconnect/connect chime + the
// teardown/recreate kernel stress that broke DDA, now applied to the mux). The monitor +
// helper persist for the WHOLE session; only the host-DDA leg opens (secure) and closes
// (normal). Apply the DDA learning here: reuse, don't tear down.
dda = None; // free the secure DDA encoder; the relay (helper) is the source again
while relay.try_recv().is_ok() {} // drop secure-dwell backlog
relay.request_keyframe(); // client decoder resumes on the helper's next IDR
// Nothing to restore: we no longer toggle the SudoVDA's HDR state for the DDA leg, so the
// monitor's colorspace is unchanged and the still-alive WGC helper just resumes.
next = std::time::Instant::now();
}
}
if want_kf {
if secure {
if let Some(d) = dda.as_mut() {
d.enc.request_keyframe();
}
} else {
relay.request_keyframe();
}
await_idr = true;
}
if secure {
// DDA capture+encode for the secure desktop, paced to the frame interval.
let Some(d) = dda.as_mut() else {
std::thread::sleep(interval);
continue;
};
if let Some(f) = d.cap.try_latest().context("DDA capture")? {
d.frame = f;
}
let capture_ns = now_ns();
d.enc.submit(&d.frame).context("DDA encoder submit")?;
next += interval;
while let Some(au) = d.enc.poll().context("DDA encoder poll")? {
if await_idr && !au.keyframe {
continue;
}
await_idr = false;
if !forward!(au.data, capture_ns, au.keyframe) {
break 'outer; // send thread gone
}
}
match next.checked_duration_since(std::time::Instant::now()) {
Some(dur) => std::thread::sleep(dur),
None => next = std::time::Instant::now(),
}
} else {
// Relay the helper's AUs for the normal desktop. Timeout → keep servicing the loop;
// Disconnected → the helper exited (step 6 adds the relaunch watchdog).
let au = match relay.recv_timeout(std::time::Duration::from_millis(500)) {
Ok(au) => au,
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {
if stop.load(Ordering::SeqCst) {
break;
}
tracing::warn!("two-process: no AU from helper within 500ms");
continue;
}
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => {
// The helper exited (crash, or a console disconnect killed its session). REBUILD
// the whole output + helper (not just respawn on the old target): an abruptly-killed
// helper leaves the SudoVDA's DXGI output briefly unresolvable ("no DXGI output for
// target N yet"), and a console reconnect needs a fresh output in the new session —
// `build` recreates both. Back off so a hard-failing rebuild (e.g. no active session
// yet) doesn't spin; give up only after a sustained run of failures.
helper_fails += 1;
if helper_fails > MAX_HELPER_FAILS {
tracing::error!(
fails = helper_fails,
"two-process: WGC helper keeps dying — ending stream"
);
break;
}
std::thread::sleep(std::time::Duration::from_millis(500));
match build(&mut vd, cur_mode) {
Ok((ka, rl, tg, hz)) => {
tracing::warn!(
fails = helper_fails,
"two-process: WGC helper exited — rebuilt output + helper"
);
relay = rl;
_keepalive = ka;
target = tg;
effective_hz = hz;
dda = None; // old-target DDA is stale
interval = std::time::Duration::from_secs_f64(1.0 / hz.max(1) as f64);
await_idr = true; // resume on the new helper's opening IDR
}
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), fails = helper_fails,
"two-process: helper rebuild failed — will retry");
}
}
continue;
}
};
if await_idr && !au.keyframe {
continue; // skip stale deltas until the post-switch IDR
}
await_idr = false;
helper_fails = 0; // a frame flowed → the helper is healthy again
// The helper's pts_ns is on this machine's monotonic clock (same `now_ns()` source).
if !forward!(au.data, au.pts_ns, au.keyframe) {
break 'outer; // send thread gone
}
}
}
drop(frame_tx);
let _ = send_thread.join();
drop(watcher);
tracing::info!(sent, "punktfunk/1 two-process stream complete");
Ok(())
}
/// One mode's capture/encode pipeline: (capturer, encoder, first frame, frame interval). /// One mode's capture/encode pipeline: (capturer, encoder, first frame, frame interval).
/// Dropping the capturer tears down the PipeWire stream and the virtual output with it. /// Dropping the capturer tears down the PipeWire stream and the virtual output with it.
type Pipeline = ( type Pipeline = (
@@ -3675,6 +3315,23 @@ fn build_pipeline_with_retry(
// 30-60s to produce its first frame, and a first-connect timeout would tear down the warm // 30-60s to produce its first frame, and a first-connect timeout would tear down the warm
// session (forcing another cold start on reconnect). A genuinely permanent failure still fails // session (forcing another cold start on reconnect). A genuinely permanent failure still fails
// fast via `is_permanent_build_error`; only transient "no frame yet" retries consume the budget. // fast via `is_permanent_build_error`; only transient "no frame yet" retries consume the budget.
// IDD-push only: HOLD one monitor lease across all build attempts. A failed attempt's capturer
// drop releases ITS lease, but this held lease keeps the shared monitor Active (refs >= 1), so the
// next attempt's `vd.create` JOINS it (refcount++) instead of finding it Lingering and tripping the
// IDD-push reconnect PREEMPT (teardown + recreate). That preempt-per-retry was the REMOVE→ADD churn
// that exhausts the IddCx monitor-slot pool and wedges ADD at 0x80070490 — one ADD per cold start
// now, not one per attempt. Non-IDD-push backends (Linux portal, WGC) don't use the refcount manager
// and aren't churn-wedge-prone, so they keep create-per-attempt (a held lease there would allocate a
// second virtual output). Dropped when this fn returns — on success the Pipeline's own lease keeps
// the monitor Active; on failure refs falls to 0 → Lingering → linger-timeout teardown.
let _retry_hold = if matches!(plan.capture, crate::session_plan::CaptureBackend::IddPush) {
Some(
vd.create(mode)
.context("acquire virtual output for the session (retry-hold lease)")?,
)
} else {
None
};
const MAX_ATTEMPTS: u32 = 8; const MAX_ATTEMPTS: u32 = 8;
let mut backoff = std::time::Duration::from_millis(500); let mut backoff = std::time::Duration::from_millis(500);
for attempt in 1..=MAX_ATTEMPTS { for attempt in 1..=MAX_ATTEMPTS {
+9 -42
View File
@@ -26,12 +26,9 @@ pub enum CaptureBackend {
/// Linux: the xdg ScreenCast portal → PipeWire (the only Linux capture path). /// Linux: the xdg ScreenCast portal → PipeWire (the only Linux capture path).
Portal, Portal,
/// Windows: IDD direct-push — frames pulled straight from the pf-vdisplay driver's shared ring /// Windows: IDD direct-push — frames pulled straight from the pf-vdisplay driver's shared ring
/// (in-process, Session 0; no Desktop Duplication, no WGC helper). /// (in-process, Session 0; captures the secure desktop too). The sole Windows capture path —
/// DXGI Desktop Duplication (DDA) and the WGC two-process relay were removed.
IddPush, IddPush,
/// Windows: DXGI Desktop Duplication (`PUNKTFUNK_CAPTURE=dda|dxgi` or `PUNKTFUNK_NO_WGC`).
Dda,
/// Windows: Windows.Graphics.Capture (the composed-desktop default), with a DDA watchdog fallback.
Wgc,
} }
impl CaptureBackend { impl CaptureBackend {
@@ -42,20 +39,10 @@ impl CaptureBackend {
CaptureBackend::Portal CaptureBackend::Portal
} }
/// Windows precedence (identical to the pre-stage-3 `capture_virtual_output` branch order): /// Windows: IDD direct-push is the sole capture path (DDA + the WGC two-process relay were removed).
/// IDD-push wins; else an explicit `dda`/`dxgi` request or `PUNKTFUNK_NO_WGC` selects DDA; else WGC.
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub fn resolve() -> Self { pub fn resolve() -> Self {
let cfg = crate::config::config();
if cfg.idd_push {
CaptureBackend::IddPush CaptureBackend::IddPush
} else if matches!(cfg.capture_backend.as_str(), "dda" | "dxgi")
|| crate::capture::wgc_disabled()
{
CaptureBackend::Dda
} else {
CaptureBackend::Wgc
}
} }
#[cfg(not(any(target_os = "linux", target_os = "windows")))] #[cfg(not(any(target_os = "linux", target_os = "windows")))]
@@ -67,11 +54,9 @@ impl CaptureBackend {
/// How a session is structured across processes. /// How a session is structured across processes.
#[derive(Clone, Copy, Debug, PartialEq, Eq)] #[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum SessionTopology { pub enum SessionTopology {
/// One process captures + encodes (Linux; Windows non-SYSTEM / IDD-push / `NO_WGC`). /// One process captures + encodes. The only topology: Linux (portal) and Windows (in-process
/// IDD-push in Session 0). The SYSTEM-host + user-session WGC relay was removed with DDA/WGC.
SingleProcess, SingleProcess,
/// SYSTEM host + a user-session WGC helper relay (the Windows normal-desktop path under SYSTEM,
/// where in-process WGC can't activate). See `virtual_stream_relay`.
TwoProcessRelay,
} }
/// The resolved encode backend (recorded for logging / stages 45; the per-session encoder open still /// The resolved encode backend (recorded for logging / stages 45; the per-session encoder open still
@@ -103,8 +88,8 @@ pub struct SessionPlan {
pub encoder: EncoderBackend, pub encoder: EncoderBackend,
/// Handshake-negotiated encode bit depth (8, or 10 = HEVC Main10). /// Handshake-negotiated encode bit depth (8, or 10 = HEVC Main10).
pub bit_depth: u8, pub bit_depth: u8,
/// The IDD-push HDR hint (`bit_depth >= 10`) — the want-HDR flag the capturer was passed before. /// The IDD-push HDR hint (`bit_depth >= 10`) — the want-HDR flag handed to the capturer so it
/// Non-IDD-push Windows backends ignore it and auto-detect HDR from the monitor; Linux is 8-bit. /// proactively enables advanced color on the virtual display. Linux is 8-bit (HDR blocked upstream).
pub hdr: bool, pub hdr: bool,
/// Handshake-negotiated chroma subsampling (4:2:0, or full-chroma 4:4:4 when the client + host + /// Handshake-negotiated chroma subsampling (4:2:0, or full-chroma 4:4:4 when the client + host +
/// GPU all support it). Resolved before the Welcome; `Yuv420` on every backend that declined it. /// GPU all support it). Resolved before the Welcome; `Yuv420` on every backend that declined it.
@@ -151,26 +136,8 @@ impl SessionPlan {
} }
} }
/// Process topology. On Windows this is the former `punktfunk1::should_use_helper` logic verbatim; on /// Process topology. Single-process is the only topology now: Linux (portal) and Windows (in-process
/// every other platform the session is always single-process. /// IDD-push in Session 0). The Windows SYSTEM-host + user-session WGC relay was removed with DDA/WGC.
#[cfg(target_os = "windows")]
pub(crate) fn resolve_topology() -> SessionTopology {
let cfg = crate::config::config();
// `NO_HELPER`/`NO_WGC` force single-process; IDD-push captures in-process in Session 0 (no helper);
// otherwise the helper runs when forced or when we're SYSTEM (in-process WGC can't activate there).
let helper = if cfg.no_helper || crate::capture::wgc_disabled() || cfg.idd_push {
false
} else {
cfg.force_helper || crate::capture::wgc_relay::running_as_system()
};
if helper {
SessionTopology::TwoProcessRelay
} else {
SessionTopology::SingleProcess
}
}
#[cfg(not(target_os = "windows"))]
pub(crate) fn resolve_topology() -> SessionTopology { pub(crate) fn resolve_topology() -> SessionTopology {
SessionTopology::SingleProcess SessionTopology::SingleProcess
} }

Some files were not shown because too many files have changed in this diff Show More