34 Commits

Author SHA1 Message Date
enricobuehler 1c04e77293 feat(apple): Improve presenter
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / web (push) Has been cancelled
ci / rust (push) Has been cancelled
android-screenshots / screenshots (push) Successful in 2m16s
deb / build-publish (push) Successful in 3m26s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
windows-host / package (push) Successful in 6m48s
release / apple (push) Successful in 7m45s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m22s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m37s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
android / android (push) Successful in 9m35s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m32s
linux-client-screenshots / screenshots (push) Successful in 2m31s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m53s
web-screenshots / screenshots (push) Successful in 2m32s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m37s
docker / deploy-docs (push) Failing after 1m4s
flatpak / build-publish (push) Failing after 3m44s
feat(apple): add cursor capture on iPad
2026-06-30 01:31:48 +02:00
enricobuehler e2d4c40167 feat(android): HDR toggle, video-feed stats, landscape lock
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 5m14s
ci / web (push) Successful in 49s
apple / screenshots (push) Successful in 5m19s
ci / docs-site (push) Successful in 1m0s
ci / bench (push) Successful in 4m36s
ci / rust (push) Successful in 13m31s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 40s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
deb / build-publish (push) Successful in 10m9s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m50s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m19s
docker / deploy-docs (push) Failing after 1m4s
- HDR toggle in Settings → Display. Persisted (hdr_enabled, default on); the
  host is advertised HDR only when the toggle is on AND the panel can present
  HDR10 (displaySupportsHdr), so SDR panels never get PQ they'd mis-tone-map.
  The toggle is disabled/greyed on non-HDR displays (ToggleRow gained `enabled`).
- Extend the stats HUD with a video-feed line, e.g.
  "HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0". nativeVideoStats now returns 14
  doubles (appends bitDepth, CICP primaries/transfer, chroma_format_idc from the
  negotiated Welcome); older/shorter layouts just omit the line.
- Lock the stream to landscape while streaming (SENSOR_LANDSCAPE), restoring the
  prior orientation on exit. The activity declares configChanges=orientation, so
  it re-lays-out in place with no stream restart; harmless no-op on TV.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 22:16:04 +02:00
enricobuehler 580b1ea7a7 feat(host/steam): shippable usbip/vhci_hcd virtual Deck + client leave-shortcuts
apple / screenshots (push) Has been cancelled
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
audit / cargo-audit (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
Steam Deck pass-through (design/steam-deck-passthrough-plan.md), code-complete +
all CI checks green on Linux + adversarially reviewed; on-glass validation pending:

- usbip/`vhci_hcd` virtual Deck transport (inject/linux/steam_usbip.rs) for
  non-SteamOS hosts (Bazzite/generic) — presents a real interface-2 USB Deck so
  Steam Input promotes it. In-process vhci attach (loopback OP_REQ_IMPORT handshake
  → sysfs attach) with a bounded `usbip`-CLI fallback; detach on drop.
- Backed by a vendored, libusb-free trim of the `usbip` crate
  (crates/punktfunk-host/vendor/usbip-sim, MIT + NOTICE; host/cdc/hid + rusb/nusb
  removed; interrupt-IN paced by bInterval).
- Selection ladder raw_gadget (SteamOS fast-path) → usbip (universal) → UHID,
  with PUNKTFUNK_STEAM_USBIP / PUNKTFUNK_USBIP_ATTACH knobs.
- Shared Deck descriptors + the 0x83/0xAE feature contract + a Steam-accepted
  serial consolidated into steam_proto.rs; the raw_gadget backend reuses them.
- Linux client leave-shortcuts: Ctrl+Alt+Shift+D + holding the escape chord
  (L1+R1+Start+Select) >=1.5s end the session (short press still exits
  fullscreen); the chord state resets across sessions.

Also bundles in-progress work already staged in the tree:
- host(kwin): xdg-output logical-geometry mapping so the KWin fake_input backend
  places absolute coordinates correctly under display scaling.
- docs: design/README index entries + design/controller-only-mode.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 831b37b4b7 build: exclude the usbip-poc from the workspace (standalone PoC, pulls libusb)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 4f0b4aa68f docs(steam): production plan for Deck client pass-through + shippable usbip host
Write design/steam-deck-passthrough-plan.md — the build plan to ship exact Steam
Deck pass-through from the Linux client (incl. the Steam + QAM buttons) plus a
virtual Deck on any Linux host. Key validated facts captured so the next session
doesn't re-investigate:

- Client capture is ALREADY correct: SDL3 maps Steam->Guide, QAM->Misc1; the
  client forwards BTN_GUIDE/BTN_MISC1; the host maps them to btn::STEAM/btn::QAM.
  Only precondition: Steam Input disabled on the client (the Decky UX).
- Shippable host transport = usbip + vhci_hcd (in-tree + signed everywhere, no
  module build, no MOK) — PROVEN on Bazzite: Steam promotes the usbip interface-2
  Deck (XInput slot + X-Box pad), identical to raw_gadget on SteamOS.
- Build steps: refactor steam_gadget.rs into shared Deck-logic + a transport
  trait; add the usbip transport (vendor-trim the usbip crate to drop rusb/libusb,
  in-process vhci attach); transport-select raw_gadget->usbip->UHID/DualSense;
  client leave-shortcut (controller chord + Ctrl+Alt+Shift+D); serial polish.

Also checks in the working usbip Deck PoC (packaging/linux/steam-deck-gadget/
usbip-poc/) for the next session to build on. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 963c406f33 feat(host/steam): default the gadget Deck on for SteamOS (glass-confirmed)
The virtual Steam Deck is validated glass-to-glass on a Deck: it appears as a
distinct second Steam controller, a held A drives Steam's overlay ("Resume
Game"), and a button press registers in a real game (confirmed in-game).

gadget_preferred() now defaults ON for SteamOS hosts (/etc/os-release ID=steamos
or ID_LIKE), OFF elsewhere where the universal UHID path stays the default;
PUNKTFUNK_STEAM_GADGET=1/0 forces it. A Deck-as-host with a physical Deck never
reaches this path — resolve_gamepad's conflict gate degrades SteamDeck → DualSense
first, so the two-Deck case never happens in production (it was only a test-rig
confound on the dev Deck).

The feature is complete: a virtual Steam Deck that Steam Input recognizes +
promotes, churn-free, with input flowing to games. Workspace clippy/fmt/test
green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 7ab8acaf55 feat(host/steam): harden the gadget feature contract — fixes the evdev churn
The virtual Deck's gamepad evdev was churning (destroyed + recreated) because
Steam kept re-probing: GetControllerInfo reads HID feature reports, and the gadget
served zeros for them. Captured the real contract off a physical Deck
(packaging/linux/steam-deck-gadget/get_deck_attrs.c, hidraw HIDIOCGFEATURE — usbmon
truncates to 32B) and implemented it in steam_gadget.rs::feature_reply:

- 0x83 GET_ATTRIBUTES_VALUES: [83, 2d, 9×(attr-id, u32-LE)] — product id 0x1205, a
  per-instance unit serial (0x0a/0x04, so a gadget never collides with a real Deck
  or another gadget), and the capability attrs (0x09=0x2e, 0x0b=0x0fa0, rest 0).
- 0xAE GET_STRING_ATTRIBUTE: [ae, len, attr, ascii] — serial (attr 1) / board
  serial (attr 0).
- other commands (0x87 settings): echo the last write.

Validated on the Deck: 1 connect / 0 disconnect / 1 gamepad evdev (was constant
churn), Steam activates the gadget cleanly (no GetControllerInfo failed, no zombie)
and emits its X-Box 360 pad. usbmon on the gadget's bus confirms our state reports
(pressed button at byte 8) are delivered on the interrupt-IN and consumed by
hid-steam — so with M1/M2's byte-8→BTN_SOUTH decode the input chain is proven
end-to-end. Remaining: a foreground-game confirmation of Steam Input's XInput
mapping, then default the gadget on for SteamOS.

Workspace clippy/fmt/test green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler c8e19396e4 feat(host/steam): raw_gadget Deck host backend (Steam-Input path, opt-in)
Port the proven raw_gadget virtual Deck to a Rust host gamepad backend, the
SteamOS-only transport that gets Steam Input to actually promote the Deck.

- inject/linux/steam_gadget.rs (new): SteamDeckGadget — a userspace raw_gadget
  emulator of the real 3-interface USB Deck (mouse=0/keyboard=1/controller=2,
  28DE:1205) on a dummy_hcd loopback UDC, descriptors captured from a physical
  Deck, answering every control transfer incl. the HID feature reports. Driven by
  the same steam_proto::serialize_deck_state as the UHID pad; rumble feedback via
  parse_steam_output. The raw_gadget UAPI is funneled through 4 documented ioctl
  wrappers (the crate denies undocumented unsafe).
- inject/linux/steam_controller.rs: the manager pad is now a DeckTransport enum
  (Uhid | Gadget); ensure() prefers the gadget when PUNKTFUNK_STEAM_GADGET=1
  (best-effort modprobe dummy_hcd+raw_gadget), gracefully falling back to the
  universal UHID SteamDeckPad. write/pump/heartbeat dispatch through the enum.

Validated on a real Deck via a static musl harness that #[path]-includes the
module: enumerates, hid-steam binds + reads our serial + creates the Steam Deck +
Motion Sensors evdevs — identical to the C PoC. Caught a real portability bug:
raw_gadget's no-arg ioctls (RUN/CONFIGURE/EP0_STALL) reject a non-zero `value`
with EINVAL, and on musl an omitted ioctl vararg is a garbage register — so they
must pass an explicit 0.

Opt-in (default off) while the Steam GetControllerInfo feature contract is
hardened (to stop the gamepad-evdev churn). Workspace clippy/fmt/test green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 78020cd66c docs(steam): gadget input-flow status — reports delivered + format-validated
On the Deck, a pressa build shows hid-steam polls our interface-2 interrupt-IN
endpoint and our 64-byte state reports are delivered ("STREAM: first input report
delivered"). The report format is already validated (M1 serializer on-box + M2's
EVIOCGKEY/EVIOCGABS test on the same hid-steam decode). The "Steam Deck" gamepad
evdev forms but is transient (hid-steam recreates it as gamepad_mode toggles —
Steam keeps re-probing because the PoC serves the serial but not Steam's full
GetControllerInfo attribute set, on a heavily-churned test Deck), so a stable live
EVIOCGKEY catch of the held A wasn't obtained. Delivery + format proven; the
evdev transience is a feature-report-completeness gap the host backend resolves.
Doc §11. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 8870e85233 feat(steam): raw_gadget virtual Deck — full Steam Input recognition (proven on Deck)
The interface-2 wall is climbed. packaging/linux/steam-deck-gadget/deck_raw_gadget.c
is a raw_gadget userspace emulator of a real 3-interface USB Steam Deck (28DE:1205,
mouse=0/keyboard=1/controller=2) on a dummy_hcd loopback UDC, with descriptors
captured verbatim from a physical Deck and full HID feature-report handling.

Live on a real Deck (SteamOS 3.8.11): hid-steam reads our serial (PFDECK000),
creates the Steam Deck + Motion Sensors evdevs, and Steam Input PROMOTES it —
controller.txt "Interface: 2 ... device opened ... reserving XInput slot 1" +
"input: Microsoft X-Box 360 pad 1". Stable (1 connect, 0 disconnects, no zombie);
the kernel Steam Deck evdev is then grabbed by Steam Input which exposes its own
X-Box pad, exactly like a real Deck. First time a virtual Deck is fully Steam-Input
promoted (UHID can't — it has no USB interface number, so Steam filters it).

Also includes the configfs f_hid variant (configfs_gadget_up/down.sh) — the minimal
reproducer that proved interface 2 makes Steam open+XInput-reserve the device, but
f_hid can't serve feature reports so Steam dropped it as a zombie.

Gotchas documented in the README: 7-byte vs 9-byte endpoint descriptor, no-data OUT
controls acked via zero-length EP0_READ (not WRITE, else error -110), streamer must
not start before SET_CONFIGURATION is acked. SteamOS-host only (needs dummy_hcd +
raw_gadget). Recognition proven; feeding real client reports + a host backend is next.
Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler a81f1304cd docs(steam): gadget PoC — interface 2 PROVEN (Steam opens + XInput-reserves it)
On the Deck (which ships dummy_hcd + raw_gadget + configfs f_hid), a pure-shell
configfs gadget stood up a real 3-interface USB Deck (kbd=0/mouse=1/controller=2,
28de:1205) on a dummy_hcd loopback UDC. hid-steam bound all 3 interfaces, and
crucially Steam PROMOTED the interface-2 controller: "Local Device Found ...
Interface: 2 ... Steam controller device opened for index 14 ... Steam Controller
reserving XInput slot 1" — exactly where the interface -1 UHID Deck was filtered.

It then failed only at feature-report exchange (f_hid can't serve HID GET_REPORT:
"steam_send_report: error -32", "couldn't get controller details ... zombie
controller"), and no gamepad evdev formed for the same reason. So interface 2 is
necessary AND sufficient for Steam to open+XInput-reserve the Deck; the remaining
piece is serving feature/output reports, which raw_gadget can (full control,
like UHID). Next: a raw_gadget 3-interface Deck emulator. Doc §11. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler c75f39fd8e docs(steam): VALIDATED — virtual DualSense IS Steam-Input recognized; isolates the Deck wall to interface 2
Definitive hardware test (Bazzite running Steam): a virtual DualSense (UHID,
054c:0ce6, Interface: -1) is FULLY promoted by Steam — controller.txt logs
"Local Device Found 054c 0ce6 DualSense Wireless Controller", then "Controller
using HIDAPI driver vid=0x054c pid=0x0ce6" and loads configset_controller_ps5.vdf
(our calibration/pairing/firmware feature blobs read back). The SAME Interface:
-1 that the Deck is rejected at is accepted for the DualSense.

So the wall is specifically the Deck's MULTI-INTERFACE requirement (Steam must
pick interface 2 among kbd/mouse/controller), NOT a UHID limitation. The
DualSense path delivers real Steam Input (gyro + touchpad + glyphs + bindings)
for a streamed Deck/SC client; it loses only Deck glyphs, the 2nd trackpad, and
the 4 back grips as distinct Steam-Input paddles (M5 folds them to buttons).

Full Deck-identity Steam Input would need interface 2 -> a USB gadget (dummy_hcd
+ configfs HID, controller on interface 2). Feasible but heavy/non-portable:
dummy_hcd isn't built on Bazzite/Deck/dev-box, so it'd be a per-kernel build +
(on immutable SteamOS/Bazzite) a package-layer + reboot per host.

Doc-only (design §11). Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 37c3e2bed2 docs(steam): criterion-4 RESOLVED — the interface-2 ceiling; recommend dropping M7
Hardware finding (a SteamOS Deck @ .253 + a Bazzite host @ .41, both running
Steam, via a minimal C UHID probe on Bazzite): a UHID virtual Steam Deck binds
the kernel hid-steam and creates the evdevs (so kernel-evdev + SDL-hidapi
consumers see the full grips/trackpads/IMU surface), but Steam Input will NOT
manage it. Steam's controller.txt enumerates it ("Local Device Found, 28de 1205,
Product Punktfunk Steam Deck") but logs Interface: -1 and never promotes it (no
28de:11ff XInput pad). The physical Deck on the same logs is Interface: 2 — a
real Deck is a 3-interface USB device (kbd 0 / mouse 1 / controller 2) and Steam
binds the controller on interface 2; a single UHID device has no USB interface
number, so Steam reads -1 and filters it out. (The feared 0x83/0xA1 attribute
probes never fired — it's an interface filter, not a probe-reject.)

Consequences (design §11):
- The virtual Deck's value is non-Steam / SDL games on Linux (grips + trackpads
  + gyro via evdev / SDL HIDAPI), NOT Steam Input.
- The virtual DualSense stays the Steam-Input path everywhere (Steam recognizes
  a single-interface DualSense); M5's paddle-fold carries the back grips.
- M7 (a Windows UMDF virtual Deck) is NOT recommended: same interface filter,
  and Windows has no kernel-hid-steam evdev fallback, so nothing would consume
  it; the existing Windows virtual DualSense already covers that case.
- M0-M6 is not wasted: the protocol/wire + client capture feed the DualSense
  path too, and the virtual Deck is the best option for non-Steam Linux games.

Doc-only (design/steam-controller-deck-support.md): added §11, updated the status
+ pending-validation. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 4f40fa3cb7 feat(host/steam): M6 — Steam-pad conflict gate (hardware-validated)
Don't present a virtual Steam (28DE) pad on a host that already has a physical
Steam controller — the host's own Steam Input would then manage two Decks and
confuse player assignment.

- physical_steam_controller_present(): scans /sys/bus/hid/devices for a 28DE HID
  device on a real (non-/virtual/) path.
- degrade_steam_on_conflict() in resolve_gamepad: a resolved SteamDeck /
  SteamController with a physical Steam controller attached degrades to DualSense
  (then the M5 uhid ladder); PUNKTFUNK_STEAM_FORCE=1 overrides (e.g. a remote-only
  box with no competing Steam Input).

Validated on real hardware (a SteamOS Steam Deck @ .253 + a Bazzite host @ .41,
both running Steam):
- Conflict confirmed: the Deck-as-host already has its physical 28DE:1205 AND
  Steam's 28DE:11FF XInput output pad live; a 2nd virtual 28DE = two Decks.
- Bind robustness: the virtual Deck binds hid-steam on a SECOND kernel (Bazzite
  6.17.7, vs the dev box 7.0) and the kernel accepted our serial (the M1 fix).
- Criterion-4 (running-Steam recognition) PARTIAL: a userspace consumer (Steam/
  SDL) engaged the virtual Deck (opened the hidraw, ran the lizard-disable +
  settings sequence the kernel's Deck path skips) but emitted NO 28DE:11FF XInput
  pad on the desktop — so Steam recognizes it enough to manage lizard mode but did
  not promote it to a managed XInput controller (likely needs a Big-Picture/game
  context, or a richer device; the 0x83/0xA1 attribute probes never fired, so it
  wasn't a probe-reject either).
- The heuristic itself checks TRUE on the Deck, FALSE on Bazzite.

Workspace clippy/fmt/test green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 486a292845 feat(host/steam): M5 — fallback remap, motion rescale, degrade ladder
Keep the rich Steam inputs from silently dropping when the resolved backend
isn't the virtual hid-steam device, and fix a cross-device motion-scale bug.

- inject/proto/steam_remap.rs (new, pure + unit-tested):
  * motion_wire_to_deck — the wire carries DualSense-convention units (20 LSB/
    deg.s gyro, 10000 LSB/g accel — what every client capture emits), but the
    Deck's hid-steam report wants 16 LSB/deg.s + 16384 LSB/g. The Deck backend
    now rescales (gyro x16/20, accel x16384/10000): a real Deck<->Deck gyro/
    accel correctness fix (the DualSense/DS4 backends consume the wire 1:1).
  * fold_paddles + RemapConfig (PUNKTFUNK_STEAM_REMAP=paddles=drop|stickclicks|
    shoulders, default drop) — the DualSense + DS4 managers fold a client's back
    grips onto standard buttons rather than dropping them (those pads have no
    back-button HID slot; the uinput Xbox pad already exposes them as Elite
    paddles BTN_TRIGGER_HAPPY5-8).

- resolve_gamepad: a runtime degrade ladder — a UHID backend (DualSense / DS4 /
  Steam Deck) on a host where /dev/uhid isn't writable now falls back to the
  uinput Xbox 360 pad instead of a dead controller (the device-create would
  just fail). Separate from pick_gamepad's compile-time platform check, so the
  existing pick_gamepad tests are untouched.

- Delete the throwaway M0/M1 spike (src/bin/steam_uhid_spike.rs) — M2's
  #[ignore]d backend test subsumes its validation, and removing it frees
  steam_proto to reference steam_remap cleanly.

On-box backend test still green; workspace clippy/fmt/test green (incl. the new
steam_remap tests). Deferred as optional RemapConfig growth: gyro->mouse /
trackpad->stick synthesis on an Xbox target (no slot — documented drop today).
Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler d8c254281e feat(steam): M4 complete — C-ABI send path, Decky UX, Apple/Android parity
Finish the client side of the Steam Controller / Steam Deck pipeline.

- C-ABI (core abi.rs): PunktfunkRichInputEx — a size-prefixed superset of
  PunktfunkRichInput that can express the second trackpad (surface), a distinct
  click vs touch, signed coords + pressure — plus
  punktfunk_connection_send_rich_input2 (the struct_size ABI-skew-guard
  precedent). The only way a C client (Apple/embedders) can emit a TouchpadEx;
  the legacy struct + send_rich_input stay byte-for-byte. punktfunk_core.h
  regenerated.

- Decky (clients/decky): a "Steam Deck" gamepad type in Settings + an unmissable
  Disable-Steam-Input instruction shown when it's selected (in Game Mode Steam
  Input holds 0x1205, so the SDL HIDAPI Steam driver can't open the Deck's
  controls until the user disables Steam Input for the shortcut). Plus a
  best-effort, feature-detected disableSteamInputForShortcut() in launchStream —
  never blocks/throws; the manual toggle is the documented source of truth.

- Apple parity (PunktfunkConnection.swift): GamepadType.steamController/steamDeck
  (wire 5/6) + name parsing, so the resolved type round-trips. Capture is blocked
  (GameController never surfaces a 0x28DE HID device).

- Android parity (Gamepad.kt): PREF_STEAMCONTROLLER/STEAMDECK + the Valve 0x28DE
  PIDs in prefFor(). Rich-input capture stays out of scope (no rich-input plane
  yet) — standard buttons/sticks resolve to the host's Steam Deck pad.

Rust workspace clippy/fmt/test green; Decky src/ typechecks clean (only a
pre-existing @decky/api dep resolution error remains); Swift/Kotlin compile on
their CI. The full pipeline is now BUILT; what remains is validation that needs
hardware we don't have (a running Steam on the host, a live Deck client, the
Moonlight paddle regression). Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler ae71e4628d feat(clients/steam): M4 — desktop SDL clients capture the rich Steam inputs
The Linux + Windows native clients (clients/{linux,windows}/src/gamepad.rs) now
capture and send the Steam Controller / Steam Deck rich inputs, so a real Deck
(off Steam Input) or a Steam Controller on a desktop client drives the host's
virtual hid-steam pad end-to-end:

- Set SDL's HIDAPI Steam hints (SDL_JOYSTICK_HIDAPI_STEAMDECK / _STEAM) before
  init so SDL opens Valve devices directly (paddles + both trackpads + gyro as
  first-class SDL gamepad inputs).
- Detect the Deck/SC by VID/PID (0x28DE + 0x1205 / 0x1102 / 0x1142) ->
  GamepadPref::SteamDeck (there is no SDL gamepad type for it), so the host
  builds the virtual Deck with the right identity.
- Map the SDL paddle + Misc1 buttons -> BTN_PADDLE1..4 / BTN_MISC1 (a free win
  for Xbox Elite paddles too).
- Route a SECOND touchpad -> RichInput::TouchpadEx (SDL touchpad 0 = left ->
  surface 1, 1 = right -> surface 2, signed coords); a single touchpad keeps the
  legacy Touchpad. New forward_touch() helper centralizes the choice.
- Track held touchpad contacts per (surface, finger) and lift them on pad
  switch/detach so a contact held at that moment can't stick.
- Sensor (gyro/accel) capture was already generic across pad types.

Linux client builds + clippy clean; the Windows client is a near-verbatim
mirror (windows CI compiles it). On a Deck in Game Mode, Steam Input still holds
the device — the user disables Steam Input for the client (the Decky UX, next);
on a desktop client (or a Deck with Steam Input off) the hints just work.

Remaining M4: Decky Disable-Steam-Input UX, Apple/Android parity, and the C-ABI
PunktfunkRichInputEx + send_rich_input2 (Apple/embedder send path). Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 01c55aed38 feat(proto/steam): M3 — rich Steam wire (back buttons + 2nd trackpad)
Carry the rich Steam Controller / Steam Deck inputs end-to-end on the wire —
strictly additive + forward-compatible (unknown kinds/bits drop on old peers).

Core (punktfunk-core):
- input.rs: BTN_PADDLE1..4 + BTN_MISC1 in Moonlight's buttonFlags2<<16 namespace
  (so the GameStream paddle path and native grips share one host injector map;
  Steam L4/L5/R4/R5 reuse the four Xbox-Elite paddle slots).
- quic.rs: RichInput::TouchpadEx (kind 0x03 — surface 0/1/2, touch+click, signed
  coords, pressure; the second trackpad the single Touchpad can't express) and
  HidOutput::TrackpadHaptic (kind 0x04 — the SC voice-coil pulse). Round-tripped.
- abi.rs: PUNKTFUNK_GAMEPAD_STEAMDECK=6 / _STEAMCONTROLLER=5, the paddle bits,
  RICH_TOUCHPAD_EX / HIDOUT_TRACKPAD_HAPTIC constants. from_hid packs
  TrackpadHaptic into the existing which + effect[0..6] — the legacy structs do
  NOT grow (guarded by new size_of==20/19 asserts); GamepadPref lockstep +
  paddle-bit lockstep asserts extended. include/punktfunk_core.h regenerated.

Host (punktfunk-host):
- steam_proto::from_gamepad maps the wire paddles -> the four Deck grips + QAM;
  apply_rich routes TouchpadEx left/right -> the matching pad.
- every DualSense/DS4 manager (Linux + Windows) gained a TouchpadEx arm
  (surface 0/2 -> its one touchpad; surface 1 ignored) so the variant compiles
  everywhere and a Steam client streaming to a DS host keeps its right pad.
- the xpad BUTTON_MAP finally consumes the GameStream paddle bits
  (BTN_TRIGGER_HAPPY5-8) — Sunshine/Moonlight paddle clients were silently
  no-op'd before (design §5.6).
- Android feedback: drop TrackpadHaptic (no coils; rumble rides 0xCA).

Validated on-box: the ignored backend test now drives the full wire path —
from_gamepad (BTN_A + the L4 grip) + apply_rich (a left-pad TouchpadEx) reach the
evdev as BTN_A + ABS_HAT0X=-8000. Wire round-trips + paddle/TouchpadEx mapping
unit-tested. Workspace clippy/fmt/test green. Not pushed.

Deferred to M4: the C-ABI PunktfunkRichInputEx + send_rich_input2 (only the
Apple/embedder *send* path needs it; the host decodes TouchpadEx today).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 95308d352b feat(host/steam): M2 — virtual Steam Deck as a wired PadBackend (Linux)
Make the virtual hid-steam device a selectable per-session host gamepad,
end-to-end on Linux: PUNKTFUNK_GAMEPAD=steamdeck now builds a
SteamControllerManager that creates a /dev/uhid 28DE:1205 Deck, enters
gamepad_mode, and feeds the byte-exact Deck report (M1).

- inject/linux/steam_controller.rs: SteamControllerManager / SteamDeckPad,
  mirroring dualsense.rs (open/create2, GET/SET_REPORT pump, heartbeat, RAII
  destroy). Two Steam-specific quirks beyond the DualSense path:
    * gamepad_mode entry — best-effort `lizard_mode=0` via sysfs, plus a b9.6
      creation pulse (MODE_ENTER) so steam_do_deck_input_event stops
      early-returning, plus an anti-toggle guard (MENU_HOLD_CAP) so a long
      in-game Start-hold can't flip gamepad_mode back off.
    * UHID_SET_REPORT answered err=0 (DualSense omits it; the kernel stalls
      ~5s/cmd otherwise); the 0xEB rumble report parsed onto the 0xCA plane.
- core config.rs: GamepadPref::SteamDeck (wire byte 6) + SteamController
  (byte 5, reserved — folds to Xbox360 until its backend lands); from_u8 /
  from_name / as_str. Forward-compatible (unknown byte -> Auto); the C-ABI
  PUNKTFUNK_GAMEPAD_* constants stay M3, so no generated-header drift.
- punktfunk1.rs: PadBackend::SteamDeck variant + select / handle / apply_rich
  / pump / heartbeat arms; pick_gamepad Linux arm.

On-box: an #[ignore]d backend test (backend_binds_and_input_flows) drives the
real SteamDeckPad — it binds hid-steam (gamepad + IMU evdevs), enters gamepad
mode, BTN_A reaches the evdev, and the device tears down on drop. Workspace
clippy/fmt/test green. Not pushed. Next: M3 (protocol/ABI wire) + M4 (client
capture).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 9ff7d41bfe feat(host/steam): M1 — byte-exact Deck input serializer, on-box validated
Flesh out inject/proto/steam_proto.rs into the full Steam Deck HID contract,
transcribed verbatim from the kernel steam_do_deck_input_event /
steam_do_deck_sensors_event and validated field-for-field against kernel 7.0:

- SteamState: the u64 button map (bytes 8..16), sticks/triggers/trackpads/IMU
  stored as raw little-endian report values; serialize_deck_state is a pure,
  byte-exact memcpy into the 64-byte unnumbered frame.
- from_gamepad (XInput frame -> Deck buttons/sticks/triggers) + apply_rich
  (RichInput touchpad -> right pad, motion -> IMU).
- parse_steam_output: the 0xEB ID_TRIGGER_RUMBLE_CMD feedback -> (low, high)
  for the universal rumble plane.
- serial_reply fixed: prepend the report-id-0 byte the kernel strips
  (steam_recv_report does memcpy(data, buf+1, ...)); M0's reply lacked it, so
  the kernel fell back to the "XXXXXXXXXX" serial.
- SteamModel (Deck now; classic Controller later), command/feature IDs.

The spike is repurposed as the M1 validator: it pulses the b9.6 mode-switch to
enter gamepad_mode (steam_do_deck_input_event early-returns under the default
lizard_mode otherwise), then holds a known test pattern. Reading both evdevs via
EVIOCGABS/EVIOCGKEY, every field matched: ABS_X/Y/RX/RY (incl. the kernel
Y-negation), both triggers, the touched right-pad HAT1X/Y, the IMU accel/gyro
(with ABS_Z/RZ negations), and the 6 expected buttons incl. the L4/R5 grips.

5 unit tests + workspace clippy/fmt/test green. Next: M2 (SteamControllerManager
UHID backend + PadBackend wiring). Not pushed — pipeline not yet shippable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 2b47d8cc28 feat(host/steam): M0 — virtual hid-steam UHID device binds + parses (Linux)
Greenfield virtual Steam Deck controller, the Steam analogue of the shipped
virtual DualSense. Proves the kernel hid-steam driver binds a /dev/uhid
28DE:1205 device, registers it as a real Steam Deck, and parses our input
reports — the go/no-go gate for the full Steam Controller/Deck pipeline.

- inject/proto/steam_proto.rs: keeper module — the vendor HID descriptor (one
  feature report, the sole thing steam_is_valve_interface() checks), the
  command/feature IDs, serialize_deck_state, and the serial GET_REPORT reply.
  Unit-tested.
- src/bin/steam_uhid_spike.rs: throwaway M0 spike (Linux-only) — opens
  /dev/uhid, creates the device, services the handshake including
  UHID_SET_REPORT (which the DualSense backend omits and which hid-steam
  stalls ~5s/cmd without), and heartbeats a neutral report.
- design/steam-controller-deck-support.md: full design + M0–M7 plan; the two
  walls (Steam Input capture ownership; virtual-Steam recognition) and the
  fidelity ceiling. Status: M0 GREEN.

On-box (headless Ubuntu 26.04, kernel 7.0, no Steam): journalctl -k shows
hid-steam binding the device (rebind off hid-generic), "Steam Controller
connected", and the kernel creating BOTH a "Steam Deck" gamepad evdev and a
"Steam Deck Motion Sensors" IMU evdev (INPUT_PROP_ACCELEROMETER). A
layout-agnostic mash-probe drove 23 distinct BTN_* codes through
hid-steam -> evdev, proving the input-report parse path. M1 line-checks the
exact per-bit report layout against the lab kernel.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 7cd9364c9e style(host): rustfmt the #9/#13 pairing edits
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 3e498cd40d docs(security): #9/#13 fixed, S7 rationale corrected (red-team follow-up)
17/18 now fixed. A red-team of the three accepted findings showed #9 and #13
rested on a circular premise (each was the other's "safe fallback") and S7's
written rationale was wrong (signing exercises the same modexp Marvin targets).
#9/#13 closed; S7 accept retained for the corrected reasons + amplifier hardened.
See f0574a5, f6c9576.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 60de506f66 fix(host/gamestream): correct the rsa-Marvin (S7) rationale + cap pairing signatures
Red-team found the .cargo/audit.toml justification for RUSTSEC-2023-0071 was
materially wrong: it claimed "Marvin targets decryption, so the vulnerable path
isn't exercised" — but the advisory is a variable-time modexp of the secret
exponent, which RSA *signing* (signing_key.sign) also runs. The accept is still
correct, for the RIGHT reasons (no decryption/padding oracle; the signed
serversecret is host-random not attacker-chosen; signing is operator-PIN-gated;
GameStream is off by default and the native QUIC plane uses rustls, not rsa;
Moonlight mandates RSA-2048 so the GameStream key can't move off it). Rewrite
the rationale accordingly.

Also shut the timing-sample amplifier the review surfaced: the pairing session
was never marked after phase 3, so a peer past phase 1 could loop phase2/phase3
to harvest many RSA signing-time samples. Sign exactly once per ceremony
(reject a repeated serverchallengeresp).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 2865368771 fix(host/pairing): close native-pairing DoS findings #9 + #13 (red-team follow-up)
The accepts for #9 (PIN-window burn) and #13 (knock-queue flood) rested on a
circular premise — each cited the other as the safe fallback — and a re-review
showed one LAN attacker could defeat BOTH, denying all onboarding. Close them:

- #13 per-source-IP cap on the pending-knock queue (MAX_PENDING_PER_IP) so one
  host can't fill/evict the 32-slot queue (QUIC validates the source address);
  and eviction now NEVER drops a live *parked* knock (a held-open connection
  awaiting operator approval), so a cert-rotating flood can't evict the genuine
  device being onboarded. This makes the delegated-approval path genuinely
  flood-resistant — restoring the validity of #9's "use delegated approval on
  hostile LANs" fallback.

- #9 fingerprint-bindable PIN window: `NativePairing::arm_for(ttl, Some(fp))`
  binds the window to one operator-selected device; `pin_for_attempt` returns
  `BoundToOther` for any other fingerprint, which the QUIC pair path rejects
  WITHOUT consuming the window — so an unpaired peer can neither pair nor BURN a
  window armed for a specific device (it can't forge the bound fingerprint). The
  mgmt `POST /native/pair/arm` gains an optional `fingerprint` (from a pending
  knock); unbound arming keeps the legacy any-device behavior (trusted-LAN).
  (Web-console "pair this pending device with a PIN" UX is a follow-up; the
  flood-resistant knock path above is the immediate hostile-LAN onboarding path.)

+ regression tests (armed_pin_is_fingerprint_bindable,
  pending_per_ip_cap_and_parked_protection); api/openapi.json regenerated.
110 host tests + clippy + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 6e2e946bc9 docs(security): #5 fixed + on-box validated (RTX box, 2026-06-29)
15/18 now fixed; no finding remains open and actionable. SDDL scoped to
SYSTEM+LocalService, validated live (6943-frame DualSense+IDD session works;
non-SYSTEM OpenFileMapping now ACCESS_DENIED). See e59fa60.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler b5f02000d6 fix(host/security): scope Windows shared-section SDDL to SYSTEM+LocalService (close #5)
The gamepad host<->UMDF-driver shared sections (Global\pfds-shm-*, pfxusb-shm-*)
and the IDD-push frame ring/event (Global\pfvd-*) were created with
`D:(A;;GA;;;WD)` — GENERIC_ALL to **Everyone** — on the assumption the driver's
WUDFHost ran under a restricted token needing broad access. So any local
unprivileged user could OpenFileMapping the section to inject controller input,
tamper the trusted HID channel, or read captured screen frames
(security-review 2026-06-28 #5).

On-box validation (RTX box, 2026-06-29) disproved the restricted-token premise:
the WUDFHost token is NT AUTHORITY\LocalService (S-1-5-19), SYSTEM integrity,
with ZERO restricted SIDs. So the section only needs SYSTEM (the host creates +
writes it) and LocalService (the driver opens it). Scope both SDDL sites to
`D:(A;;GA;;;SY)(A;;GA;;;LS)`; rename the now-misnamed `permissive_sa` ->
`shared_object_sa`; correct the stale "restricted-token / Everyone" docs.

Validated live: a full DualSense + 1280x720x60 session — 6943 frames received,
HID output round-tripped, device status OK (pf_dualsense + pf_vdisplay WUDFHosts
both LocalService open the scoped sections fine), while OpenFileMapping from a
non-SYSTEM admin session now returns ACCESS_DENIED (was a granted handle under
WD). Host-only change (the SDDL is set when the host CREATES the section);
drivers unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler fe562f0562 chore(apple): rename app display name to "Punktfunk"
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
release / apple (push) Has been cancelled
android / android (push) Has been cancelled
ci / rust (push) Has been cancelled
decky / build-publish (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
CFBundleDisplayName was "Punktfunkempfänger" across all targets/configs; the
in-app title is already "Punktfunk", so make the home-screen name match. Built
iOS app resolves CFBundleDisplayName = "Punktfunk".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:34:16 +02:00
enricobuehler 4e00037a89 feat(apple): stage-2 default + pixel-perfect, decode robustness, UI/rumble polish
apple / swift (push) Successful in 1m4s
android / android (push) Successful in 4m33s
ci / rust (push) Successful in 5m4s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 59s
deb / build-publish (push) Successful in 3m12s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
release / apple (push) Successful in 8m30s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m45s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m48s
apple / screenshots (push) Successful in 5m43s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m24s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m46s
Stream reliability
- Default to the stage-2 presenter (VTDecompressionSession + CAMetalLayer): it detects
  and recovers a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard
  on a lost HEVC reference frame with no app-side recovery (confirmed Apple limitation).
  Stage 1 is now a DEBUG-only presenter toggle, plus the automatic no-Metal fallback.
- Stage-2 pixel-perfect: render the drawable at the decoded size (shader stays 1:1 =
  identity) and let the layer's contentsGravity scale via the system compositor — the
  same path stage-1's videoGravity used — instead of scaling in-shader.
- Loss recovery in both pumps is now a persistent awaitingIDR want, retried until an IDR
  actually lands, so a keyframe request swallowed by the throttle can't strand a frozen
  frame; 100 ms keyframe throttle to match the Android path.
- Fix "Publishing changes from within view updates": defer the HostStore writes out of
  the .onChange(of: model.phase) callback.
- Move AVAudioSession setActive/setCategory off the main thread (async on a shared serial
  queue) to stop the UI-stall warning.

Controllers
- Rumble: capped-exponential backoff when the gamecontrollerd.haptics XPC breaks (-4811)
  so a transient server interruption self-heals instead of cascading; playsHapticsOnly so
  a controller engine doesn't join the always-active streaming audio session.
- Host cards: iPad pointer "magnet" hover effect; iPhone press scale + light haptic.

UI / design
- Ship Geist (SIL OFL 1.1) as the app font (bundled OTFs + registration), with the
  license surfaced in Acknowledgements.
- Restructure iOS/iPadOS Settings into a category NavigationSplitView; resolution wheel
  with custom-resolution entry; 10-bit HDR toggle in Display.
- Industrial host-card redesign (left-aligned, bold, brand monogram tiles).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:26:10 +02:00
enricobuehler 46b9aa8cf0 fix(windows-host): IDD activation — resolve-first, force-EXTEND only as fallback
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 4m23s
ci / rust (push) Successful in 5m9s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 57s
apple / screenshots (push) Successful in 5m33s
windows-host / package (push) Successful in 8m46s
deb / build-publish (push) Successful in 3m13s
decky / build-publish (push) Successful in 27s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m51s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m15s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m19s
docker / deploy-docs (push) Successful in 21s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m44s
force_extend_topology() was added before the resolve loop to de-clone a fresh IDD on
integrated-screen boxes (laptops), but its bare SDC_TOPOLOGY_EXTEND preset is
ACCESS_DENIED from the Session-0 service context on a HEADLESS box and broke the IDD
auto-activation there: resolve_gdi_name stayed None -> "not an active display path" ->
black screen. That regressed the headless/primary platform (live RTX box).

Revert to the proven e2c9bfd flow: resolve FIRST (Windows auto-activates the IDD as its
own extended path), and force-EXTEND only as the FALLBACK when resolve returns None (the
integrated-screen clone case, observed live to leave resolve None). The success path is
byte-identical to e2c9bfd (resolve -> set_active_mode -> isolate_displays_ccd).

Validated live: the headless RTX box streams again (probe: frames flow, driver attaches
to the ring, host/driver render LUIDs match).

Reviewed multi-agent + adversarial: no regression on the validated headless path or the
observed Optimus-laptop clone path (a cloned IddCx target resolves to None there, so the
is_none() fallback fires + de-clones). Known theoretical caveat, documented inline and
unobserved for IddCx but untested across GPU/driver/OS: a CCD clone that manifests as a
shared-source ACTIVE path would resolve to Some and bypass the is_none() gate. Follow-up:
widen the gate (a target_is_cloned helper) once an integrated-screen box is available to
validate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 16:15:48 +02:00
enricobuehler 372b27540b fix(apple): render Acknowledgements notices in lazy chunks
apple / swift (push) Successful in 1m11s
android / android (push) Successful in 4m19s
ci / web (push) Successful in 51s
ci / rust (push) Successful in 5m14s
ci / docs-site (push) Successful in 58s
release / apple (push) Successful in 7m54s
deb / build-publish (push) Successful in 4m0s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
ci / bench (push) Successful in 5m19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
apple / screenshots (push) Successful in 5m37s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m26s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m35s
docker / deploy-docs (push) Successful in 18s
THIRD-PARTY-NOTICES.txt is ~885 KB / 16k lines; rendering it in a single
SwiftUI Text overshot the text-rendering height limit — it laid out for ages
and drew blank below the cutoff (only the small punktfunk licenses above it
showed). Split the notices into ~80 line-chunks (<=200 lines / <=18 KB each,
computed once as Licenses.thirdPartyNoticesChunks) and render them in a
top-level LazyVStack so only on-screen chunks lay out and no chunk is tall
enough to clip. Chunking is lossless — rejoining the chunks reproduces the
original byte-for-byte, so no notice text is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:34:19 +02:00
enricobuehler db4d15bf8b fix(apple): stop the iOS/iPadOS Add Host sheet from scrolling
apple / swift (push) Successful in 1m5s
android / android (push) Successful in 4m15s
ci / rust (push) Successful in 4m56s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 58s
deb / build-publish (push) Successful in 3m11s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
release / apple (push) Successful in 8m32s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m40s
apple / screenshots (push) Successful in 5m44s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m29s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m31s
docker / deploy-docs (push) Successful in 17s
The add-host content is a SwiftUI Form (backed by a scrollable list), so it
bounced/scrolled inside the fixed .height(320) detent even though the three
rows + action button fit exactly. Lock it with .scrollDisabled(true) on iOS
(covers iPadOS); macOS (fixed-size panel) and tvOS (custom rows, no Form) are
untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:12:08 +02:00
enricobuehler 8e24ea9ed7 fix(ci): archive Apple release builds with Automatic signing
The in-app OSS license screens (7591425) added a `resources:` array to the
PunktfunkKit SwiftPM target, which makes SwiftPM emit a resource-bundle target
(PunktfunkKit_PunktfunkKit). A resource bundle is a product type that cannot
carry a provisioning profile, so the explicit PROVISIONING_PROFILE_SPECIFIER
each release.yml archive step set — global on macOS, sdk-scoped on iOS/tvOS —
now lands on it and fails the archive ("does not support provisioning profiles")
on all three platforms. (Before that commit there was no resource bundle, so the
profile was harmless.)

Switch all three archive steps to CODE_SIGN_STYLE=Automatic (development):
Automatic signing assigns a profile only to the app target and leaves the
resource bundle (and the macOS-host SwiftPM macro plugins) alone, and bakes the
sandbox entitlements in. No -allowProvisioningUpdates, so it stays offline and
never cloud-signs (the App-Manager ASC key can't). DISTRIBUTION signing is
unchanged — still manual, in the -exportArchive step (which maps the profile to
io.unom.punktfunk only). Drops the now-unneeded manual signing xcconfigs.

Requires the runner to have a development provisioning profile for
io.unom.punktfunk on each platform (now installed for macOS/iOS/tvOS).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:12:08 +02:00
enricobuehler 73c0125843 fix(mgmt): regenerate api/openapi.json for 0.3.0
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 4m16s
ci / rust (push) Successful in 5m5s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 58s
apple / screenshots (push) Successful in 5m17s
deb / build-publish (push) Successful in 3m11s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / bench (push) Successful in 4m43s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 26s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m28s
docker / deploy-docs (push) Successful in 17s
The OpenAPI 'info.version' tracks CARGO_PKG_VERSION; the 0.3.0 bump made the
checked-in spec stale (the openapi_document_is_complete_and_checked_in test).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 07:54:30 +00:00
114 changed files with 10398 additions and 749 deletions
+20 -6
View File
@@ -13,11 +13,25 @@
[advisories]
ignore = [
# rsa "Marvin Attack" a timing sidechannel in RSA *decryption* (PKCS#1 v1.5 padding oracle).
# There is NO fixed rsa release (the constant-time rewrite is still unreleased upstream), and rsa
# is required for GameStream/Moonlight pairing. Crucially, the host uses rsa ONLY for PKCS#1 v1.5
# SIGNING / VERIFYING (gamestream/cert.rs + gamestream/pairing.rs: SigningKey / VerifyingKey /
# Signer / Verifier) — it never performs RSA decryption, which is the operation Marvin targets.
# So the vulnerable code path is not exercised. Revisit if a fixed rsa ships or we add RSA decrypt.
# rsa "Marvin Attack" (RUSTSEC-2023-0071): a timing side-channel in the rsa crate's variable-time
# modular exponentiation of the SECRET exponent. IMPORTANT — this affects the RSA private-key op in
# general, INCLUDING signing (m^d mod n), which the host DOES perform (gamestream/pairing.rs
# `signing_key.sign(&serversecret)`). It is NOT, as an earlier version of this note wrongly claimed,
# limited to decryption — so "the vulnerable path isn't exercised" is false; signing exercises it.
# We accept it because the attack is not practically reachable here, NOT because the path is unused:
# * No RSA decryption / PKCS#1v1.5 padding oracle exists anywhere (every `decrypt` in the tree is
# AES/AES-GCM), so the classic Bleichenbacher/Marvin chosen-ciphertext oracle is absent.
# * The only signed message (`serversecret`) is HOST-generated random, never attacker-chosen — so
# there's no adaptive chosen-input probing (the lever remote RSA-timing key recovery needs); and
# signing is gated behind the operator-entered pairing PIN, ONE signature per ceremony (a
# repeated phase-3 is rejected — gamestream/pairing.rs — to deny a passive timing-sample harvester).
# * GameStream is OFF by default (bare `serve` is native-only); the secure native QUIC plane uses
# rustls' constant-time backend, NOT the rsa crate. RSA is touched only on the opt-in,
# trusted-LAN GameStream/Moonlight pairing handshake. Moonlight mandates RSA-2048, so the
# GameStream identity cannot move to Ed25519/ECDSA (only the native identity could, and it
# already avoids the rsa crate).
# There is NO fixed rsa release (the constant-time rewrite is still unreleased upstream). Revisit if:
# a constant-time rsa ships (then drop this), the host ever signs an attacker-chosen message with
# this key, or any RSA decryption / key-transport using the private key is added.
"RUSTSEC-2023-0071",
]
+40 -48
View File
@@ -207,10 +207,20 @@ jobs:
# (Config/Punktfunk-macOS.entitlements) — mandatory for the Mac App Store.
continue-on-error: true
run: |
# Separate archive from the Developer ID one above: App Store needs a profile-signed
# archive (manual signing), not the unsigned-then-codesign DMG path. Same App-Manager
# ASC-key constraint as iOS/tvOS — MANUAL signing, NOT -allowProvisioningUpdates
# (cloud signing the key can't do). Quit Xcode so it can't prune the dropped profile.
# Separate archive from the Developer ID one above: App Store needs a signed, entitled
# archive that -exportArchive can re-sign for distribution, not the unsigned-then-codesign
# DMG path. Archive with AUTOMATIC signing (development). Why not a manually-specified
# profile (as this step used to do): the in-app license screens added a SwiftPM resource
# bundle (PunktfunkKit_PunktfunkKit), and a resource bundle is a product type that cannot
# carry a provisioning profile — a global PROVISIONING_PROFILE_SPECIFIER (here) or an
# sdk-scoped one (iOS/tvOS) lands on it and fails the archive ("does not support
# provisioning profiles"). Automatic signing assigns a profile only to the app and leaves
# the resource bundle (and the macOS-host macro plugins) alone, and bakes the sandbox
# entitlements in. No -allowProvisioningUpdates → it stays OFFLINE and never cloud-signs
# (the App-Manager ASC key can't), so the runner must have a macOS *development* profile
# for io.unom.punktfunk installed. DISTRIBUTION signing happens in the export step below
# (manual, via the plist). Quit Xcode so it can't prune the manually-installed App Store
# distribution profile that export needs.
osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true
pkill -x Xcode 2>/dev/null || true
PROFILE="Punktfunk macOS App Store Distribution"
@@ -218,11 +228,10 @@ jobs:
-project "$PROJECT" -scheme Punktfunk \
-destination 'generic/platform=macOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-macos-appstore.xcarchive" \
-skipMacroValidation -skipPackagePluginValidation \
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \
CODE_SIGN_STYLE=Manual \
CODE_SIGN_IDENTITY="Apple Distribution" \
DEVELOPMENT_TEAM="$TEAM_ID" \
PROVISIONING_PROFILE_SPECIFIER="$PROFILE"
CODE_SIGN_STYLE=Automatic \
DEVELOPMENT_TEAM="$TEAM_ID"
cat > "$RUNNER_TEMP/export-macos-appstore.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
@@ -252,35 +261,27 @@ jobs:
# Best-effort until the App Store Connect app record for io.unom.punktfunk exists.
continue-on-error: true
run: |
# MANUAL App Store signing: the local (valid) Apple Distribution identity + the App
# Store provisioning profile. NOT -allowProvisioningUpdates — with an App-Manager-role
# ASC key that forces Xcode's CLOUD-managed signing, which the role can't do ("Cloud
# signing permission error"). The profile must be installed on the runner under
# ~/Library/Developer/Xcode/UserData/Provisioning Profiles/ (install it once with
# Xcode.app quit, or it prunes the manually-dropped distribution profile).
# A running Xcode.app prunes unrecognized profiles from that dir — quit it so the App
# Store profile survives this build; headless xcodebuild doesn't need the GUI app.
# Archive with AUTOMATIC signing (development) — see the macOS App Store step for the full
# rationale. The SwiftPM resource bundle (PunktfunkKit_PunktfunkKit, added with the in-app
# license screens) builds for iphoneos, so even the sdk-scoped PROVISIONING_PROFILE_SPECIFIER
# this step used to set matched it and failed the archive ("does not support provisioning
# profiles"). Automatic signing profiles only the app and leaves the resource bundle (and
# the macOS-host macro plugins) alone. No -allowProvisioningUpdates → OFFLINE, never
# cloud-signs (the App-Manager ASC key can't), so the runner needs an iOS *development*
# profile for io.unom.punktfunk installed. DISTRIBUTION signing is the export step below
# (manual, via the plist). A running Xcode.app prunes unrecognized profiles — quit it so the
# manually-installed App Store distribution profile survives for export.
osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true
pkill -x Xcode 2>/dev/null || true
PROFILE="Punktfunk iOS App Store Distribution"
# Scope signing to the iOS device SDK via an xcconfig — see the tvOS step below for the
# full rationale. A global (CLI) profile specifier would also be forced onto the shared
# macOS-host SwiftPM macro plugins, which reject it and fail the archive; [sdk=iphoneos*]
# in an xcconfig lands it on the app/framework slices only.
SIGN_XCCONFIG="$RUNNER_TEMP/sign-ios.xcconfig"
cat > "$SIGN_XCCONFIG" <<XCCONF
CODE_SIGN_STYLE = Manual
DEVELOPMENT_TEAM = $TEAM_ID
CODE_SIGN_IDENTITY[sdk=iphoneos*] = Apple Distribution
PROVISIONING_PROFILE_SPECIFIER[sdk=iphoneos*] = $PROFILE
XCCONF
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild archive \
-project "$PROJECT" -scheme Punktfunk-iOS \
-destination 'generic/platform=iOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-ios.xcarchive" \
-skipMacroValidation -skipPackagePluginValidation \
-xcconfig "$SIGN_XCCONFIG" \
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM"
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \
CODE_SIGN_STYLE=Automatic \
DEVELOPMENT_TEAM="$TEAM_ID"
cat > "$RUNNER_TEMP/export-appstore.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
@@ -312,33 +313,24 @@ jobs:
# on the runner (xcodebuild -downloadPlatform tvOS).
continue-on-error: true
run: |
# Same manual App Store signing as iOS (the App-Manager ASC key can't cloud-sign).
# Archive with AUTOMATIC signing (development) — see the macOS App Store step. The SwiftPM
# resource bundle (PunktfunkKit_PunktfunkKit) builds for appletvos and rejected the
# sdk-scoped profile this step used to set; Automatic signing profiles only the app and
# leaves the resource bundle + the macOS-host macro plugins (OnceMacro/SwizzlingMacro/
# AssociationMacro) alone. No -allowProvisioningUpdates → OFFLINE, never cloud-signs (the
# App-Manager ASC key can't), so the runner needs a tvOS *development* profile for
# io.unom.punktfunk installed. DISTRIBUTION signing is the export step below (manual, plist).
osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true
pkill -x Xcode 2>/dev/null || true
PROFILE="Punktfunk tvOS App Store Distribution"
# Scope signing to the tvOS device SDK via an xcconfig. A global (CLI) profile specifier
# hits EVERY target, including the shared SwiftPM macro plugins (OnceMacro/SwizzlingMacro/
# AssociationMacro) which build for the macOS host and reject a provisioning profile
# ("<macro> does not support provisioning profiles"), failing the archive. Conditionals
# work only in an xcconfig (xcodebuild mis-parses a CLI "SETTING[sdk=..]=val"), and a
# command-line -xcconfig outranks target settings, so [sdk=appletvos*] puts the profile on
# the app/framework slices only — the macosx-host macros get nothing. (The macOS archive
# above is immune: its host-SDK macros are CODE_SIGNING_ALLOWED=NO, so a global specifier
# is ignored there.)
SIGN_XCCONFIG="$RUNNER_TEMP/sign-tvos.xcconfig"
cat > "$SIGN_XCCONFIG" <<XCCONF
CODE_SIGN_STYLE = Manual
DEVELOPMENT_TEAM = $TEAM_ID
CODE_SIGN_IDENTITY[sdk=appletvos*] = Apple Distribution
PROVISIONING_PROFILE_SPECIFIER[sdk=appletvos*] = $PROFILE
XCCONF
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild archive \
-project "$PROJECT" -scheme Punktfunk-tvOS \
-destination 'generic/platform=tvOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-tvos.xcarchive" \
-skipMacroValidation -skipPackagePluginValidation \
-xcconfig "$SIGN_XCCONFIG" \
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM"
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \
CODE_SIGN_STYLE=Automatic \
DEVELOPMENT_TEAM="$TEAM_ID"
cat > "$RUNNER_TEMP/export-tvos.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+19 -5
View File
@@ -144,11 +144,25 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
`test-loopback.sh` (Swift client vs synthetic punktfunk1-hosts on loopback — runs on macOS;
includes the pairing ceremony + `--require-pairing` gate),
`RemoteFirstLightTests` (full pipeline over the LAN). See
[`clients/apple/README.md`](clients/apple/README.md). **Stage 2 presenter**
(`VTDecompressionSession` + `CAMetalLayer`) is built and live-validated on glass behind the opt-in
`punktfunk.presenter` flag (~11 ms p50 capture→present), to become the default after a few
resolution/HDR checks. Next: make stage 2 the default, glass-to-glass numbers via
`tools/latency-probe`, iOS/iPadOS/tvOS variants.
[`clients/apple/README.md`](clients/apple/README.md). **Stage 2 presenter is now the DEFAULT**
(stage-1 is the Metal-unavailable / DEBUG fallback): explicit `VTDecompressionSession` decode →
`CAMetalLayer`, presented from the hosting view's **main-runloop `CADisplayLink`** (`renderTick` pops
the newest ready frame per vsync; macOS `displaySyncEnabled = false` is the real fullscreen-judder fix,
~11 ms p50). *(An off-main `CAMetalDisplayLink` and an off-main blocking-render present thread were
both tried and reverted — both measured slower on macOS and iPad.)* **HDR fixed**
(`design/apple-stage2-presenter.md`): the "too bright" bug was a missing reference-white anchor — the
fix keeps the PQ-passthrough shader and adds `CAEDRMetadata.hdr10(…, opticalOutputScale: 203)` +
`wantsExtendedDynamicRangeContent` on the layer (on all platforms; the old `#if os(macOS)` guard left
iOS/tvOS EDR half-engaged), routing the 0xCE mastering metadata to the layer (via `setHdrMeta`) instead
of a never-composited source buffer. **Mid-session SDR↔HDR** is handled: `render` reconciles the layer
per-frame from the decoded `frame.isHDR` (per-mode pixel format `bgra8`/`rgba16Float`), so a game
entering HDR mid-stream just reconfigures (last 0xCE grade cached + re-applied; pump drains 0xCE
unconditionally). **4:4:4 added**: decode format is a 2×2 `(chroma, HDR)` matrix
(`420v/x420/444v/x444`, all biplanar so the shaders are unchanged), advertised (`VIDEO_CAP_444`) only
behind a **hardware-required `VTDecompressionSession` probe** (`Stage444Probe`, validated on M3) with a
Settings opt-out + a bounded pump backstop for an undecodable 4:4:4 session. *HDR brightness + 4:4:4
still need on-glass validation (Windows-HDR / `PUNKTFUNK_444` host).* Next: glass-to-glass numbers via
`tools/latency-probe`.
**Linux stage 1 done, first light 2026-06-12** (`clients/linux`, binary
`punktfunk-client`): GTK4/libadwaita shell linking `punktfunk-core` directly (no C ABI;
`NativeClient` is now `Sync` — mutexed plane receivers), mDNS host list, TOFU + SPAKE2
Generated
+24
View File
@@ -2331,6 +2331,17 @@ version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441"
[[package]]
name = "num-derive"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed3955f1a9c7c0c15e092f9c887db08b1fc683305fdf6eb6684f22555355e202"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "num-integer"
version = "0.1.46"
@@ -2839,12 +2850,14 @@ dependencies = [
"tracing",
"tracing-subscriber",
"ureq",
"usbip-sim",
"utoipa",
"utoipa-axum",
"utoipa-scalar",
"wasapi",
"wayland-backend",
"wayland-client",
"wayland-protocols",
"wayland-protocols-misc",
"wayland-protocols-wlr",
"wayland-scanner",
@@ -4236,6 +4249,17 @@ dependencies = [
"serde",
]
[[package]]
name = "usbip-sim"
version = "0.8.0"
dependencies = [
"log",
"num-derive",
"num-traits",
"serde",
"tokio",
]
[[package]]
name = "utf8_iter"
version = "1.0.4"
+3
View File
@@ -3,6 +3,7 @@ resolver = "2"
members = [
"crates/punktfunk-core",
"crates/punktfunk-host",
"crates/punktfunk-host/vendor/usbip-sim",
"crates/pf-driver-proto",
"clients/probe",
"clients/linux",
@@ -11,6 +12,8 @@ members = [
"tools/latency-probe",
"tools/loss-harness",
]
# Standalone PoC (built on its own; pulls usbip/tokio/libusb we don't want in the workspace).
exclude = ["packaging/linux/steam-deck-gadget/usbip-poc"]
[workspace.package]
version = "0.3.0"
+9 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0"
},
"version": "0.0.1"
"version": "0.3.0"
},
"paths": {
"/api/v1/clients": {
@@ -1354,6 +1354,14 @@
"type": "object",
"description": "Arm-native-pairing request body.",
"properties": {
"fingerprint": {
"type": [
"string",
"null"
],
"description": "Optional: bind the window to ONE device fingerprint (hex SHA-256, e.g. from a pending knock).\nWhen set, only a pairing attempt from that fingerprint consumes the window — so an unpaired\nLAN peer can neither pair nor burn a window armed for a specific device (security-review #9).\nOmit for an unbound window (any device may use the PIN — trusted-LAN only).",
"example": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
},
"ttl_secs": {
"type": [
"integer",
@@ -175,9 +175,9 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
status = "Connecting to $targetHost:$targetPort"
discovery.stop() // free the Wi-Fi radio before the stream session
scope.launch {
// Advertise HDR only when this device's display can present it (else the host sends a
// proper SDR stream rather than PQ the panel would mis-tone-map).
val hdrEnabled = displaySupportsHdr(context)
// Advertise HDR only when the user enabled it AND this device's display can present it
// (else the host sends a proper SDR stream rather than PQ the panel would mis-tone-map).
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
// "Automatic" resolves to a concrete pad type from the connected controller's VID/PID
// (Android exposes no controller-type enum) — parity with the Linux/Apple clients. An
// explicit choice is passed through unchanged.
@@ -224,7 +224,7 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
status = null
discovery.stop() // free the Wi-Fi radio before the (parked) stream session
scope.launch {
val hdrEnabled = displaySupportsHdr(context)
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
val gamepadPref = Gamepad.resolvePref(settings.gamepad)
// Pin the advertised fingerprint for a discovered host (defence against an impostor while
// we wait); a manually-typed host has none, so trust-on-first-use.
@@ -14,6 +14,13 @@ data class Settings(
val height: Int = 0,
val hz: Int = 0,
val bitrateKbps: Int = 0,
/**
* Advertise HDR (10-bit BT.2020 PQ) to the host. Default on, but only *effective* on a panel that
* can actually present HDR10 (see [displaySupportsHdr]) — on an SDR display HDR is never
* advertised regardless, so the host sends a proper 8-bit BT.709 stream rather than PQ the panel
* would mis-tone-map. Turning this off forces SDR even on a capable panel.
*/
val hdrEnabled: Boolean = true,
val compositor: Int = 0,
val gamepad: Int = 0,
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
@@ -40,6 +47,7 @@ class SettingsStore(context: Context) {
height = prefs.getInt(K_H, 0),
hz = prefs.getInt(K_HZ, 0),
bitrateKbps = prefs.getInt(K_BITRATE, 0),
hdrEnabled = prefs.getBoolean(K_HDR, true),
compositor = prefs.getInt(K_COMPOSITOR, 0),
gamepad = prefs.getInt(K_GAMEPAD, 0),
audioChannels = prefs.getInt(K_AUDIO_CH, 2),
@@ -54,6 +62,7 @@ class SettingsStore(context: Context) {
.putInt(K_H, s.height)
.putInt(K_HZ, s.hz)
.putInt(K_BITRATE, s.bitrateKbps)
.putBoolean(K_HDR, s.hdrEnabled)
.putInt(K_COMPOSITOR, s.compositor)
.putInt(K_GAMEPAD, s.gamepad)
.putInt(K_AUDIO_CH, s.audioChannels)
@@ -68,6 +77,7 @@ class SettingsStore(context: Context) {
const val K_H = "height"
const val K_HZ = "hz"
const val K_BITRATE = "bitrate_kbps"
const val K_HDR = "hdr_enabled"
const val K_COMPOSITOR = "compositor"
const val K_GAMEPAD = "gamepad"
const val K_AUDIO_CH = "audio_channels"
@@ -94,6 +94,22 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
options = BITRATE_OPTIONS,
selected = s.bitrateKbps,
) { kbps -> update(s.copy(bitrateKbps = kbps)) }
// HDR is only meaningful on a panel that can present HDR10; on an SDR display the toggle
// is disabled (and HDR is never advertised regardless) so the host doesn't send PQ the
// panel would mis-tone-map. The capability is fixed for the device, so read it once.
val hdrCapable = remember { displaySupportsHdr(context) }
ToggleRow(
title = "HDR",
subtitle = if (hdrCapable) {
"Stream 10-bit HDR (BT.2020 PQ) when the host supports it"
} else {
"This display can't present HDR10 — streams stay SDR"
},
checked = s.hdrEnabled && hdrCapable,
enabled = hdrCapable,
onCheckedChange = { on -> update(s.copy(hdrEnabled = on)) },
)
}
SettingsGroup("Host") {
@@ -181,24 +197,31 @@ private fun SettingsGroup(title: String, content: @Composable ColumnScope.() ->
}
}
/** A title + subtitle on the left, a Switch on the right. */
/** A title + subtitle on the left, a Switch on the right. [enabled] greys out the whole row. */
@Composable
private fun ToggleRow(
title: String,
subtitle: String,
checked: Boolean,
onCheckedChange: (Boolean) -> Unit,
enabled: Boolean = true,
) {
// Dim the labels when disabled so the row reads as inactive (the Switch dims itself).
val labelAlpha = if (enabled) 1f else 0.38f
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Column(Modifier.weight(1f)) {
Text(title, style = MaterialTheme.typography.bodyLarge)
Text(
title,
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurface.copy(alpha = labelAlpha),
)
Text(
subtitle,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = labelAlpha),
)
}
Switch(checked = checked, onCheckedChange = onCheckedChange)
Switch(checked = checked, onCheckedChange = onCheckedChange, enabled = enabled)
}
}
@@ -1,6 +1,7 @@
package io.unom.punktfunk
import android.Manifest
import android.content.pm.ActivityInfo
import android.content.pm.PackageManager
import android.view.SurfaceHolder
import android.view.SurfaceView
@@ -102,6 +103,13 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
it.systemBarsBehavior = WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
it.hide(WindowInsetsCompat.Type.systemBars())
}
// Lock to landscape while streaming — the host streams a landscape desktop, so pin the device
// there (either landscape direction is fine) and stop it rotating to portrait mid-session. The
// activity declares configChanges=orientation, so this re-lays out the surface in place without
// recreating the activity (no stream restart). On TV (fixed landscape) it's a harmless no-op.
// The prior request is captured and restored on the way out.
val priorOrientation = activity?.requestedOrientation
activity?.requestedOrientation = ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE
activity?.streamHandle = handle // route hardware keys to this session
activity?.axisMapper = Gamepad.AxisMapper(handle) // route joystick axes
// Host→client feedback (rumble + DualSense lightbar/LEDs); poll threads stopped before close.
@@ -114,6 +122,9 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
activity?.streamHandle = 0L
controller?.show(WindowInsetsCompat.Type.systemBars())
window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
// Release the landscape lock so the rest of the app follows the device/system again.
activity?.requestedOrientation =
priorOrientation ?: ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED
// Leaving the stream: stop the mic + audio + decode threads and tear down the session.
NativeBridge.nativeStopMic(handle)
NativeBridge.nativeStopAudio(handle)
@@ -314,9 +325,11 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
}
/**
* The live stats overlay — mirrors the Apple client's HUD. Reads the 10-double layout from
* The live stats overlay — mirrors the Apple client's HUD. Reads the 14-double layout from
* [NativeBridge.nativeVideoStats]:
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skew, w, h, hz, dropped]`.
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skew, w, h, hz, dropped, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc]`. The trailing four (present on a current native lib) describe the
* negotiated video feed and render as a codec/depth/colour/chroma line; older layouts just omit it.
*/
@Composable
internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
@@ -338,6 +351,14 @@ internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
videoFeedLine(s)?.let { feed ->
Text(
feed,
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
if (latValid) {
val tag = if (skew) "" else " (same-host)"
Text(
@@ -357,3 +378,31 @@ internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
}
}
}
/**
* Format the negotiated video-feed descriptor from the trailing four stats doubles
* `[bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`, e.g.
* `HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0`. Returns `null` on a pre-video-feed layout (< 14 doubles)
* so the overlay simply omits the line. The codes are CICP / H.273: transfer 16 = PQ, 18 = HLG (else
* SDR); primaries 9 = BT.2020, 1 = BT.709; chroma_format_idc 1 = 4:2:0, 2 = 4:2:2, 3 = 4:4:4. The
* Android decoder is always HEVC (`video/hevc`).
*/
private fun videoFeedLine(s: DoubleArray): String? {
if (s.size < 14) return null
val bitDepth = s[10].toInt()
val primaries = s[11].toInt()
val transfer = s[12].toInt()
val chromaIdc = s[13].toInt()
val depthLabel = if (bitDepth > 0) "$bitDepth-bit" else "8-bit"
val (dynamicRange, colorSpace) = when (transfer) {
16 -> "HDR" to "BT.2020 PQ"
18 -> "HDR" to "BT.2020 HLG"
else -> "SDR" to if (primaries == 9) "BT.2020" else "BT.709"
}
val chromaLabel = when (chromaIdc) {
3 -> "4:4:4"
2 -> "4:2:2"
else -> "4:2:0"
}
return "HEVC · $depthLabel · $dynamicRange ($colorSpace) · $chromaLabel"
}
@@ -186,9 +186,11 @@ internal fun StreamScene() {
Brush.linearGradient(listOf(Color(0xFF2A1E5C), Color(0xFF0E1B3D), Color(0xFF06122B))),
),
) {
// [fps, mbps, latP50, latP95, latValid, skew, w, h, hz, dropped]
// [fps, mbps, latP50, latP95, latValid, skew, w, h, hz, dropped,
// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc] — the last four = a 10-bit
// BT.2020 PQ (HDR) 4:2:0 feed, so the HUD renders its video-feed line.
StatsOverlay(
doubleArrayOf(238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 0.0),
doubleArrayOf(238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 0.0, 10.0, 9.0, 16.0, 1.0),
Modifier.align(Alignment.TopStart).padding(12.dp),
)
}
@@ -50,15 +50,25 @@ object Gamepad {
const val PREF_DUALSENSE = 2
const val PREF_XBOXONE = 3
const val PREF_DUALSHOCK4 = 4
const val PREF_STEAMCONTROLLER = 5
const val PREF_STEAMDECK = 6
// USB vendor ids of the controllers we can identify by VID/PID.
private const val VID_SONY = 0x054C
private const val VID_MICROSOFT = 0x045E
private const val VID_VALVE = 0x28DE
// Sony product ids. DualSense (PS5) and DualShock 4 (PS4) map to distinct host pad types.
private val PID_DUALSENSE = setOf(0x0CE6, 0x0DF2)
private val PID_DUALSHOCK4 = setOf(0x05C4, 0x09CC)
// Valve: Steam Deck built-in controller (0x1205); classic Steam Controller wired (0x1102) /
// dongle (0x1142). The host builds the virtual hid-steam pad; rich-input capture (paddles /
// trackpads / gyro) is out of scope on Android (no rich-input plane yet), so only the standard
// buttons + sticks reach the host for now — parity with the desktop type resolution.
private val PID_STEAMDECK = setOf(0x1205)
private val PID_STEAMCONTROLLER = setOf(0x1102, 0x1142)
// Microsoft Xbox One / Series product ids (wired + the common Bluetooth/dongle revisions). All
// behave like Xbox 360 on the host minus the glyph identity, so they share one pref byte.
private val PID_XBOXONE = setOf(
@@ -82,6 +92,8 @@ object Gamepad {
vid == VID_SONY && pid in PID_DUALSENSE -> PREF_DUALSENSE
vid == VID_SONY && pid in PID_DUALSHOCK4 -> PREF_DUALSHOCK4
vid == VID_MICROSOFT && pid in PID_XBOXONE -> PREF_XBOXONE
vid == VID_VALVE && pid in PID_STEAMDECK -> PREF_STEAMDECK
vid == VID_VALVE && pid in PID_STEAMCONTROLLER -> PREF_STEAMCONTROLLER
else -> PREF_XBOX360
}
}
@@ -103,9 +103,12 @@ object NativeBridge {
/**
* Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs.
* Returns 10 doubles:
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped]`
* (the two flags are 1.0/0.0). Poll ~1 Hz; each call resets the measurement window.
* Returns 14 doubles:
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped,
* bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`
* (the two flags are 1.0/0.0; the trailing four describe the negotiated video feed — bit depth
* 8/10, CICP primaries/transfer, and the HEVC chroma_format_idc 1=4:2:0 / 3=4:4:4). Poll ~1 Hz;
* each call resets the measurement window.
*/
external fun nativeVideoStats(handle: Long): DoubleArray?
+5
View File
@@ -114,6 +114,11 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextHidout(
out[2..n].copy_from_slice(&effect);
n
}
HidOutput::TrackpadHaptic { .. } => {
// Steam Controller trackpad-coil haptics — no Android equivalent; drop it (motor
// rumble already rides the universal 0xCA plane).
return -1;
}
};
n as jint
})
+17 -6
View File
@@ -409,11 +409,13 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
}
/// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD.
/// Returns 10 doubles
/// `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped]`
/// (the two flags are 1.0/0.0), or `null` when no decode thread is running. Poll ~1 Hz from the UI;
/// each call resets the measurement window. Not android-gated — pure `jni` + connector reads, so it
/// links on the host build too (Kotlin only ever calls it on device).
/// Returns 14 doubles
/// `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped,
/// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`
/// (the two flags are 1.0/0.0; the trailing four describe the negotiated video feed — see below), or
/// `null` when no decode thread is running. Poll ~1 Hz from the UI; each call resets the measurement
/// window. Not android-gated — pure `jni` + connector reads, so it links on the host build too
/// (Kotlin only ever calls it on device).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
env: JNIEnv,
@@ -431,7 +433,8 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
None => return std::ptr::null_mut(), // not streaming → no stats
};
let mode = h.client.mode();
let buf: [f64; 10] = [
let color = h.client.color;
let buf: [f64; 14] = [
snap.fps,
snap.mbps,
snap.lat_p50_ms,
@@ -442,6 +445,14 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
mode.height as f64,
mode.refresh_hz as f64,
h.client.frames_dropped() as f64,
// Video-feed properties the host resolved at the handshake (Welcome): encode bit depth
// (8 / 10), the CICP colour primaries + transfer code points (Kotlin maps these to a
// colour-space / HDR label — transfer 16 = PQ, 18 = HLG ⇒ HDR), and the HEVC
// chroma_format_idc (1 = 4:2:0, 3 = 4:4:4). Static for the session unless renegotiated.
h.client.bit_depth as f64,
color.primaries as f64,
color.transfer as f64,
h.client.chroma_format as f64,
];
let arr = match env.new_double_array(buf.len() as jsize) {
Ok(a) => a,
+3
View File
@@ -24,6 +24,9 @@ let package = Package(
.copy("Resources/THIRD-PARTY-NOTICES.txt"),
.copy("Resources/LICENSE-MIT.txt"),
.copy("Resources/LICENSE-APACHE.txt"),
// Geist (SIL OFL 1.1) the brand typeface, shared with punktfunk-website.
// Registered with Core Text at first use; see BrandFont.swift.
.copy("Resources/Fonts"),
],
linkerSettings: [
// Rust staticlib system deps.
@@ -364,7 +364,7 @@
ENABLE_HARDENED_RUNTIME = YES;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
@@ -398,7 +398,7 @@
ENABLE_HARDENED_RUNTIME = YES;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
@@ -429,7 +429,7 @@
CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -468,7 +468,7 @@
CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -506,7 +506,7 @@
CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -536,7 +536,7 @@
CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -10,12 +10,17 @@ struct AcknowledgementsView: View {
var body: some View {
ScrollView {
// Top-level LazyVStack so the third-party-notices chunks (Licenses.thirdPartyNoticesChunks,
// ~885 KB total) load lazily as they scroll into view a single Text that large overshoots
// the text-rendering height limit (blank below the limit + very slow). spacing 0 keeps the
// notice chunks visually continuous; the header block carries its own spacing + bottom pad.
LazyVStack(alignment: .leading, spacing: 0) {
VStack(alignment: .leading, spacing: 18) {
Text("punktfunk")
.font(.title2).bold()
.font(.geist(22, .bold, relativeTo: .title2))
if let version {
Text("Version \(version)")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
Text(Licenses.appLicense)
@@ -24,19 +29,41 @@ struct AcknowledgementsView: View {
Divider()
Text("Bundled font")
.font(.geist(17, .semibold, relativeTo: .headline))
Text("punktfunk ships the Geist typeface (Geist Sans), "
+ "© The Geist Project Authors / Vercel, used under the SIL Open Font "
+ "License 1.1.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
if !Licenses.fontLicense.isEmpty {
Text(Licenses.fontLicense)
.font(.caption2.monospaced())
.modifier(SelectableText())
}
Divider()
Text("Third-party software")
.font(.headline)
.font(.geist(17, .semibold, relativeTo: .headline))
Text(
"punktfunk uses the open-source components below, each under its own license. "
+ "On some platforms FFmpeg is additionally bundled under the LGPL v2.1+ "
+ "(dynamically linked, replaceable)."
)
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
Text(Licenses.thirdPartyNotices)
}
.frame(maxWidth: .infinity, alignment: .leading)
.padding(.bottom, 18)
ForEach(Licenses.thirdPartyNoticesChunks.indices, id: \.self) { i in
Text(Licenses.thirdPartyNoticesChunks[i])
.font(.caption2.monospaced())
.frame(maxWidth: .infinity, alignment: .leading)
.modifier(SelectableText())
}
}
.frame(maxWidth: 900, alignment: .leading)
.frame(maxWidth: .infinity, alignment: .leading)
.padding()
@@ -80,6 +80,11 @@ struct AddHostSheet: View {
}
#if !os(tvOS)
.formStyle(.grouped)
#endif
#if os(iOS)
// The detent below is sized to fit all 3 rows + the action button exactly, so the
// Form must NOT scroll/bounce inside it lock it. (iOS 16+; safe at iOS 17.)
.scrollDisabled(true)
#endif
#if os(macOS)
// macOS: UNCHANGED Cancel + Spacer + Add in an HStack, both wired to the
@@ -120,8 +125,8 @@ struct AddHostSheet: View {
// Form + the full-width action row, instead of the half-screen .medium it used to rest
// at. A single fixed detent is enough: the system keeps the content above the keyboard
// when Address/Port is focused, and on iPadOS this renders as a short bottom sheet (not a
// centered formSheet card). If Dynamic Type grows the rows past this height the Form just
// scrolls inside the detent nothing is clipped. (.height(_:) is iOS 16+, safe at iOS 17.)
// centered formSheet card). The Form itself is .scrollDisabled (above) so it can't
// bounce/scroll inside this fixed detent. (.height(_:) is iOS 16+, safe at iOS 17.)
.presentationDetents([.height(320)])
.presentationDragIndicator(.visible)
#endif
@@ -0,0 +1,39 @@
// App-wide brand chrome. SwiftUI has no single switch to put a custom font on every navigation
// title, so the iOS large/inline nav titles are themed through UINavigationBar's appearance proxy
// (set once at launch). Backgrounds are left at the system defaults transparent at the scroll
// edge (the large title floats on the content), blurred once scrolled so only the typeface
// changes: Geist, matching the cards and the website.
#if os(iOS)
import PunktfunkKit
import UIKit
enum BrandTheme {
static func apply() {
BrandFont.registerIfNeeded()
let scrollEdge = UINavigationBarAppearance()
scrollEdge.configureWithTransparentBackground()
applyFonts(to: scrollEdge)
let standard = UINavigationBarAppearance()
standard.configureWithDefaultBackground()
applyFonts(to: standard)
let proxy = UINavigationBar.appearance()
proxy.scrollEdgeAppearance = scrollEdge
proxy.standardAppearance = standard
proxy.compactAppearance = standard
}
/// Override only the title fonts; leave colors/backgrounds at the configured defaults.
private static func applyFonts(to appearance: UINavigationBarAppearance) {
if let large = UIFont(name: "Geist-Bold", size: 34) {
appearance.largeTitleTextAttributes[.font] = large
}
if let inline = UIFont(name: "Geist-SemiBold", size: 17) {
appearance.titleTextAttributes[.font] = inline
}
}
}
#endif
@@ -28,6 +28,7 @@ struct ContentView: View {
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2
@AppStorage(DefaultsKey.hdrEnabled) private var hdrEnabled = true
@AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.hudEnabled) private var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) private var hudPlacement = HUDPlacement.topTrailing.rawValue
@@ -68,15 +69,19 @@ struct ContentView: View {
// A session actually started remember it on the card ("Connected ago"
// plus the accent ring on the most recent host).
guard let host = model.activeHost else { break }
store.markConnected(host.id)
// Delegated approval just succeeded: the operator let this device in, so pin the
// host's observed fingerprint and remember it as paired future connects are then
// silent (rule 1), exactly like after a PIN/TOFU success. Dismisses the wait prompt.
if awaitingApproval?.host.id == host.id {
if let fp = model.connection?.hostFingerprint {
store.pin(host.id, fingerprint: fp)
}
awaitingApproval = nil
let approvedFingerprint = awaitingApproval?.host.id == host.id
? model.connection?.hostFingerprint : nil
if awaitingApproval?.host.id == host.id { awaitingApproval = nil }
// Persist on the next runloop tick: HostStore is an ObservableObject, and mutating
// its @Published from inside .onChange (a view-update callback) trips SwiftUI's
// "Publishing changes from within view updates". A one-tick delay is imperceptible.
let store = store
DispatchQueue.main.async {
store.markConnected(host.id)
if let approvedFingerprint { store.pin(host.id, fingerprint: approvedFingerprint) }
}
case .idle:
// The delegated-approval connect failed, timed out, or was cancelled drop the
@@ -333,6 +338,7 @@ struct ContentView: View {
rawValue: UInt32(clamping: gamepadType)) ?? .auto),
bitrateKbps: UInt32(clamping: bitrateKbps),
audioChannels: UInt8(clamping: audioChannels),
hdrEnabled: hdrEnabled,
launchID: launchID,
allowTofu: allowTofu,
requestAccess: requestAccess)
@@ -475,6 +481,7 @@ struct ContentView: View {
gamepad: pad,
bitrateKbps: bitrate,
audioChannels: UInt8(clamping: audioChannels),
hdrEnabled: hdrEnabled,
autoTrust: true)
}
}
@@ -54,7 +54,7 @@ struct ControllerTestView: View {
var body: some View {
VStack(spacing: 0) {
HStack {
Text("Test Controller").font(.headline)
Text("Test Controller").font(.geist(17, .semibold, relativeTo: .headline))
Spacer()
Button("Done") { dismiss() }.keyboardShortcut(.cancelAction)
}
@@ -99,8 +99,8 @@ struct ControllerTestView: View {
.font(.title2)
.foregroundStyle(.secondary)
VStack(alignment: .leading, spacing: 2) {
Text(c.name).font(.headline)
Text(c.productCategory).font(.caption).foregroundStyle(.secondary)
Text(c.name).font(.geist(17, .semibold, relativeTo: .headline))
Text(c.productCategory).font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
}
Spacer()
}
@@ -209,7 +209,7 @@ struct ControllerTestView: View {
) -> some View {
VStack(alignment: .leading, spacing: 4) {
Text("Touchpad\(tp.button.isPressed ? " — click" : "")")
.font(.caption2).foregroundStyle(.secondary)
.font(.geist(11, relativeTo: .caption2)).foregroundStyle(.secondary)
ZStack {
RoundedRectangle(cornerRadius: 8).stroke(Color.secondary.opacity(0.3))
fingerDot(tp.primary, color: .accentColor)
@@ -230,7 +230,7 @@ struct ControllerTestView: View {
private func motionReadout(_ m: GCMotion) -> some View {
let a = Self.totalAccel(m)
return VStack(alignment: .leading, spacing: 2) {
Text("Motion").font(.caption2).foregroundStyle(.secondary)
Text("Motion").font(.geist(11, relativeTo: .caption2)).foregroundStyle(.secondary)
Text(String(format: "gyro %+.2f %+.2f %+.2f",
m.rotationRate.x, m.rotationRate.y, m.rotationRate.z))
.font(.caption2.monospaced())
@@ -254,11 +254,11 @@ struct ControllerTestView: View {
Toggle("Heavy motor (left)", isOn: $heavyOn)
Toggle("Light motor (right)", isOn: $lightOn)
Label("Backend: \(tester.rumbleBackend)", systemImage: "waveform")
.font(.caption).foregroundStyle(.secondary)
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
Text("Toggle a motor to feel it. The host maps a game's low/high-frequency "
+ "rumble onto these two. A DualSense is driven over raw HID (CoreHaptics "
+ "can't reach its motors on macOS).")
.font(.caption).foregroundStyle(.secondary)
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
}
.onChange(of: heavyOn) { _, _ in applyRumble() }
.onChange(of: lightOn) { _, _ in applyRumble() }
@@ -289,11 +289,11 @@ struct ControllerTestView: View {
}
}
Text("Pick an effect, then pull L2/R2 to feel the resistance.")
.font(.caption).foregroundStyle(.secondary)
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
}
} else {
Text("Adaptive triggers need a DualSense.")
.font(.caption).foregroundStyle(.secondary)
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
}
}
}
@@ -348,7 +348,7 @@ struct ControllerTestView: View {
_ title: String, @ViewBuilder _ content: () -> Content
) -> some View {
VStack(alignment: .leading, spacing: 10) {
Text(title).font(.subheadline.weight(.semibold))
Text(title).font(.geist(15, .semibold, relativeTo: .subheadline))
content()
}
.frame(maxWidth: .infinity, alignment: .leading)
@@ -127,14 +127,13 @@ struct HomeView: View {
AddHostSheet { store.add($0) }
}
#if os(iOS)
// SettingsView owns its own NavigationSplitView (sidebar + detail) and Done button, so it
// is presented directly wrapping it in a NavigationStack here would nest a split view in
// a stack (double title bars). `settingsSheetSizing()` widens the sheet on iPad for the
// two-column layout.
.sheet(isPresented: $showSettings) {
NavigationStack {
SettingsView()
.navigationTitle("Settings")
.toolbar {
Button("Done") { showSettings = false }
}
}
.settingsSheetSizing()
}
#endif
#endif
@@ -172,7 +171,7 @@ struct HomeView: View {
private var discoveredSection: some View {
VStack(alignment: .leading, spacing: 10) {
Label("On this network", systemImage: "antenna.radiowaves.left.and.right")
.font(.headline)
.font(.geist(15, .semibold, relativeTo: .headline))
.foregroundStyle(.secondary)
.padding(.horizontal)
LazyVGrid(columns: gridColumns, spacing: gridSpacing) {
@@ -249,8 +248,10 @@ struct HomeView: View {
/// the width so the cards stay edge-aligned with the title and bars sized touch-first: one
/// column on iPhone portrait, 34 generous cards on iPad.
private var gridColumns: [GridItem] {
// Wider than before: the monogram card is a horizontal module (tile + address line), so
// it needs room for a monospaced "IP:port" without truncating.
#if os(macOS)
[GridItem(.adaptive(minimum: 180, maximum: 240), spacing: 16)]
[GridItem(.adaptive(minimum: 250, maximum: 320), spacing: 16)]
#elseif os(tvOS)
[GridItem(.adaptive(minimum: 320), spacing: 48)]
#else
@@ -1,26 +1,75 @@
// The host grid's cards: a saved host (tap to connect, context menu) and an mDNS-discovered
// host (tap to save + connect). Both share the same platform-tuned sizing.
// host (tap to save + connect). Both share the "monogram module" look a squared brand-purple
// monogram tile + a left-aligned bold Geist name over monospaced technical metadata
// (address, status), framed by a hairline panel border. Industrial, not soft.
import PunktfunkKit
import SwiftUI
/// Shared host-card sizing touch-first on iOS, compact on macOS/tvOS.
/// Shared host-card sizing touch-first on iOS, compact on macOS, roomy on tvOS.
private struct CardMetrics {
let iconSize: CGFloat
let iconBox: CGFloat
let cardPadding: CGFloat
let nameFont: Font
let tile: CGFloat // monogram tile side
let monogram: CGFloat // monogram letter point size
let name: CGFloat // host-name point size
let meta: CGFloat // address (mono) point size
let status: CGFloat // status-label (mono) point size
let padding: CGFloat
let spacing: CGFloat // tile text gap
let radius: CGFloat
static var current: CardMetrics {
#if os(iOS)
CardMetrics(iconSize: 56, iconBox: 76, cardPadding: 28, nameFont: .title3.weight(.semibold))
CardMetrics(tile: 54, monogram: 26, name: 19, meta: 13, status: 11,
padding: 16, spacing: 14, radius: 12)
#elseif os(tvOS)
CardMetrics(tile: 64, monogram: 32, name: 24, meta: 16, status: 14,
padding: 18, spacing: 18, radius: 14)
#else
CardMetrics(iconSize: 42, iconBox: 56, cardPadding: 18, nameFont: .headline)
CardMetrics(tile: 44, monogram: 21, name: 15, meta: 12, status: 10.5,
padding: 13, spacing: 12, radius: 10)
#endif
}
}
/// A saved host. The accent ring marks the most-recently-connected one; the context menu
/// First letter of a host name, uppercased the monogram glyph. Falls back to a bullet.
private func monogram(_ name: String) -> String {
guard let first = name.trimmingCharacters(in: .whitespacesAndNewlines).first else { return "" }
return String(first).uppercased()
}
/// The squared monogram tile. `filled` = a solid brand-purple chip (saved hosts); otherwise a
/// tinted outline (discovered hosts). Shows a spinner in place of the glyph while connecting.
private func monogramTile(_ letter: String, m: CardMetrics, connecting: Bool, filled: Bool) -> some View {
let shape = RoundedRectangle(cornerRadius: m.radius - 3, style: .continuous)
return ZStack {
shape.fill(filled
? AnyShapeStyle(LinearGradient(
colors: [Color.brand, Color.brand.opacity(0.72)],
startPoint: .top, endPoint: .bottom))
: AnyShapeStyle(Color.brand.opacity(0.14)))
if connecting {
ProgressView().tint(filled ? .white : Color.brand)
} else {
// Fixed size (not Dynamic Type): the glyph is pinned inside a fixed tile, so it must
// not scale up and spill out at large accessibility text sizes. minimumScaleFactor +
// the clip below are belt-and-suspenders for an unusually wide glyph.
Text(letter)
.font(.geistFixed(m.monogram, .bold))
.minimumScaleFactor(0.5)
.lineLimit(1)
.foregroundStyle(filled ? Color.white : Color.brand)
}
}
.frame(width: m.tile, height: m.tile)
.clipShape(shape)
.overlay {
if !filled {
shape.strokeBorder(Color.brand.opacity(0.45), lineWidth: 1)
}
}
}
/// A saved host. A left accent bar marks the most-recently-connected one; the context menu
/// pairs / speed-tests / forgets / removes. Disabled while a session is busy.
struct HostCardView: View {
let host: StoredHost
@@ -41,66 +90,44 @@ struct HostCardView: View {
var body: some View {
let m = CardMetrics.current
return Button(action: onConnect) {
VStack(spacing: 10) {
ZStack {
Image(systemName: "play.display")
.font(.system(size: m.iconSize, weight: .light))
.foregroundStyle(.tint)
.opacity(isConnecting ? 0.3 : 1)
if isConnecting {
ProgressView()
}
}
.frame(height: m.iconBox)
VStack(spacing: 2) {
HStack(spacing: 6) {
// Presence dot: green = advertising on the LAN now; grey = not seen.
Circle()
.fill(isOnline ? Color.green : Color.secondary.opacity(0.35))
.frame(width: 7, height: 7)
.accessibilityLabel(isOnline ? "Online" : "Offline")
HStack(spacing: m.spacing) {
monogramTile(monogram(host.displayName), m: m, connecting: isConnecting, filled: true)
VStack(alignment: .leading, spacing: 4) {
Text(host.displayName)
.font(m.nameFont)
.font(.geist(m.name, .bold, relativeTo: .title3))
.foregroundStyle(.primary)
.lineLimit(1)
}
HStack(spacing: 4) {
if host.pinnedSHA256 != nil {
Image(systemName: "lock.fill")
.font(.system(size: 9))
.foregroundStyle(.secondary)
}
Text("\(host.address):\(String(host.port))")
.font(.caption)
.font(.geist(m.meta, relativeTo: .caption))
.foregroundStyle(.secondary)
.lineLimit(1)
statusRow(m)
}
if let last = host.lastConnected {
Text("Connected \(last, format: .relative(presentation: .named))")
.font(.caption2)
.foregroundStyle(.tertiary)
.lineLimit(1)
Spacer(minLength: 0)
}
}
}
.frame(maxWidth: .infinity)
.padding(.vertical, m.cardPadding)
.padding(.horizontal, 12)
.padding(m.padding)
.frame(maxWidth: .infinity, alignment: .leading)
#if !os(tvOS)
// tvOS: the .card button style owns platter + focus motion extra chrome
// inside it mutes the grow/tilt. Material + accent ring are for pointer UIs.
// Deliberately .regularMaterial, not Liquid Glass: HIG keeps glass off content
// tiles (it flattens hierarchy over an opaque grid) see GlassStyle.swift.
.background(.regularMaterial, in: RoundedRectangle(cornerRadius: 14))
.overlay {
// tvOS: the .card button style owns platter + focus motion; extra chrome mutes it.
// Elsewhere: a flat material panel with a hairline border (industrial, not a soft blob),
// and a brand accent bar down the leading edge for the most-recent host.
.background(.regularMaterial)
.overlay(alignment: .leading) {
if isMostRecent {
RoundedRectangle(cornerRadius: 14)
.strokeBorder(Color.accentColor.opacity(0.35), lineWidth: 1.5)
Rectangle().fill(Color.brand).frame(width: 3)
}
}
.clipShape(RoundedRectangle(cornerRadius: m.radius, style: .continuous))
.overlay {
RoundedRectangle(cornerRadius: m.radius, style: .continuous)
.strokeBorder(.quaternary, lineWidth: 1)
}
#endif
}
#if os(tvOS)
.buttonStyle(.card)
#elseif os(iOS)
.buttonStyle(HostCardButtonStyle(cornerRadius: m.radius))
#else
.buttonStyle(.plain)
#endif
@@ -119,10 +146,31 @@ struct HostCardView: View {
Button("Remove", role: .destructive, action: onRemove)
}
}
/// Technical status line: a square presence pip + monospaced ONLINE/OFFLINE, and PAIRED when a
/// certificate is pinned (the lock state, spelled out).
@ViewBuilder private func statusRow(_ m: CardMetrics) -> some View {
HStack(spacing: 6) {
RoundedRectangle(cornerRadius: 1.5)
.fill(isOnline ? Color.green : Color.secondary.opacity(0.4))
.frame(width: 6, height: 6)
// The state is spelled out in the adjacent text, so the pip is decorative
// otherwise VoiceOver reads the status twice ("Online, ONLINE ").
.accessibilityHidden(true)
Text(isOnline ? "ONLINE" : "OFFLINE")
if host.pinnedSHA256 != nil {
Text("· PAIRED")
}
}
.font(.geist(m.status, .medium, relativeTo: .caption2))
.tracking(0.8)
.foregroundStyle(.secondary)
.lineLimit(1)
}
}
/// A host found on the LAN but not yet saved. A dashed ring distinguishes it from saved cards;
/// tapping saves it and connects (or pairs, if the host requires it).
/// A host found on the LAN but not yet saved. A tinted-outline monogram + dashed panel border
/// distinguish it from saved cards; tapping saves it and connects (or pairs, if required).
struct DiscoveredCardView: View {
let discovered: DiscoveredHost
let isBusy: Bool
@@ -131,47 +179,77 @@ struct DiscoveredCardView: View {
var body: some View {
let m = CardMetrics.current
return Button(action: onConnect) {
VStack(spacing: 10) {
Image(systemName: "play.display")
.font(.system(size: m.iconSize, weight: .light))
.foregroundStyle(.tint)
.frame(height: m.iconBox)
VStack(spacing: 2) {
HStack(spacing: m.spacing) {
monogramTile(monogram(discovered.name), m: m, connecting: false, filled: false)
VStack(alignment: .leading, spacing: 4) {
Text(discovered.name)
.font(m.nameFont)
.font(.geist(m.name, .bold, relativeTo: .title3))
.foregroundStyle(.primary)
.lineLimit(1)
HStack(spacing: 4) {
Image(systemName: discovered.requiresPairing ? "lock.fill" : "wifi")
.font(.system(size: 9))
.foregroundStyle(.secondary)
Text("\(discovered.host):\(String(discovered.port))")
.font(.caption)
.font(.geist(m.meta, relativeTo: .caption))
.foregroundStyle(.secondary)
.lineLimit(1)
HStack(spacing: 6) {
Image(systemName: discovered.requiresPairing
? "lock.fill" : "antenna.radiowaves.left.and.right")
.font(.system(size: m.status))
.accessibilityHidden(true) // decorative; the adjacent text says the state
Text(discovered.requiresPairing ? "PAIRING REQUIRED" : "DISCOVERED")
}
.font(.geist(m.status, .medium, relativeTo: .caption2))
.tracking(0.8)
.foregroundStyle(.secondary)
.lineLimit(1)
}
Text(discovered.requiresPairing ? "Pairing required" : "Discovered")
.font(.caption2)
.foregroundStyle(.tertiary)
Spacer(minLength: 0)
}
}
.frame(maxWidth: .infinity)
.padding(.vertical, m.cardPadding)
.padding(.horizontal, 12)
.padding(m.padding)
.frame(maxWidth: .infinity, alignment: .leading)
#if !os(tvOS)
.background(.regularMaterial, in: RoundedRectangle(cornerRadius: 14))
.background(.regularMaterial)
.clipShape(RoundedRectangle(cornerRadius: m.radius, style: .continuous))
.overlay {
RoundedRectangle(cornerRadius: 14)
RoundedRectangle(cornerRadius: m.radius, style: .continuous)
.strokeBorder(
Color.secondary.opacity(0.25),
Color.secondary.opacity(0.3),
style: StrokeStyle(lineWidth: 1, dash: [4, 3]))
}
#endif
}
#if os(tvOS)
.buttonStyle(.card)
#elseif os(iOS)
.buttonStyle(HostCardButtonStyle(cornerRadius: m.radius))
#else
.buttonStyle(.plain)
#endif
.disabled(isBusy)
}
}
#if os(iOS)
/// The iOS host-card press/hover treatment, one style for both idioms:
/// - iPhone: a subtle scale-down on press + a light impact haptic on press-down. (`hoverEffect` is
/// inert without a pointer.)
/// - iPad: the system pointer "magnet" the cursor morphs into a highlight that conforms to the
/// card's rounded rect on hover. (`sensoryFeedback` is inert without a Taptic Engine, and the
/// press scale doubles as click feedback.)
struct HostCardButtonStyle: ButtonStyle {
var cornerRadius: CGFloat
func makeBody(configuration: Configuration) -> some View {
configuration.label
.scaleEffect(configuration.isPressed ? 0.96 : 1)
.animation(.spring(response: 0.3, dampingFraction: 0.65), value: configuration.isPressed)
// Conform the pointer highlight to the card's rounded rect, not its square bounds.
.contentShape(.hoverEffect, RoundedRectangle(cornerRadius: cornerRadius, style: .continuous))
.hoverEffect(.highlight)
// Light tap on press-down (nil on release so it fires once, on touch). No haptic
// hardware on iPad silently ignored there.
.sensoryFeedback(trigger: configuration.isPressed) { _, pressed in
pressed ? .impact(weight: .light) : nil
}
}
}
#endif
@@ -146,7 +146,7 @@ private struct GameCard: View {
.clipShape(RoundedRectangle(cornerRadius: 10, style: .continuous))
.overlay(alignment: .topLeading) { storeBadge }
Text(game.title)
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.lineLimit(2)
.foregroundStyle(.secondary)
}
@@ -154,7 +154,7 @@ private struct GameCard: View {
private var storeBadge: some View {
Text(game.isCustom ? "Custom" : "Steam")
.font(.caption2.weight(.semibold))
.font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 6)
.padding(.vertical, 3)
.background(.ultraThinMaterial, in: Capsule())
@@ -193,7 +193,7 @@ private struct PosterImage: View {
ZStack {
Rectangle().fill(.quaternary)
Text(title)
.font(.headline)
.font(.geist(17, .semibold, relativeTo: .headline))
.multilineTextAlignment(.center)
.foregroundStyle(.secondary)
.padding(8)
@@ -48,7 +48,7 @@ struct PairSheet: View {
+ "(http://<host>:3000 → Pairing). "
+ "Pairing verifies both sides at once — no fingerprint comparison "
+ "needed.")
.font(.callout)
.font(.geist(16, relativeTo: .callout))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
TVFieldRow(
@@ -59,7 +59,7 @@ struct PairSheet: View {
) { editing = .clientName }
if let errorText {
Text(errorText)
.font(.callout)
.font(.geist(16, relativeTo: .callout))
.foregroundStyle(.red)
}
HStack(spacing: 32) {
@@ -121,13 +121,13 @@ struct PairSheet: View {
+ "(http://<host>:3000 → Pairing). "
+ "Pairing verifies both sides at once — no fingerprint "
+ "comparison needed.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
if let errorText {
Section {
Text(errorText)
.font(.callout)
.font(.geist(16, relativeTo: .callout))
.foregroundStyle(.red)
}
}
@@ -12,8 +12,19 @@ struct PunktfunkClientApp: App {
@NSApplicationDelegateAdaptor(AppDelegate.self) private var appDelegate
#endif
init() {
#if os(iOS)
// Put Geist on the navigation titles before any bar is built.
BrandTheme.apply()
#endif
}
var body: some Scene {
WindowGroup("Punktfunk") {
// Pin the whole app's tint to the brand purple explicitly the asset-catalog accent
// resolution is environment/timing-sensitive and can fall back to system blue. Wraps the
// screenshot harness too, so captured screens are on-brand.
Group {
#if DEBUG
// PUNKTFUNK_SHOT_SCENE=<name> show that single mock-populated screen full-bleed for
// the App Store screenshot capture (tools/screenshots.sh). Normal launch otherwise;
@@ -27,6 +38,11 @@ struct PunktfunkClientApp: App {
ContentView()
#endif
}
.tint(.brand)
// Geist Sans is the app's typeface. This sets the default for unstyled text and the
// form row labels; views that pick an explicit size/weight use `.geist()` directly.
.font(.geist(17, relativeTo: .body))
}
// The Stream menu (Disconnect D, Show/Hide Statistics S) a real menu bar on
// macOS, hardware-keyboard shortcuts on iPad. tvOS has neither.
#if !os(tvOS)
@@ -34,7 +50,10 @@ struct PunktfunkClientApp: App {
#endif
#if os(macOS)
Settings {
// A separate scene `.tint` does not cross scene boundaries, so re-apply the brand
// tint here or the Preferences window falls back to the (unreliable) asset accent.
SettingsView()
.tint(.brand)
}
#endif
}
@@ -103,11 +103,11 @@ private struct ShotSettings: View {
.shadow(radius: 40, y: 16)
}
#elseif os(iOS)
NavigationStack {
SettingsView()
.navigationTitle("Settings")
.navigationBarTitleDisplayMode(.inline)
}
// SettingsView owns its NavigationSplitView (sidebar + detail) and Done button, so it is
// rendered directly a wrapping NavigationStack would nest a split view in a stack. Open
// on General so the shot lands on real controls (iPad: sidebar + General detail; iPhone:
// the General page) instead of the bare category list.
SettingsView(initialCategory: .general)
#else
NavigationStack { SettingsView() }
#endif
@@ -175,10 +175,10 @@ private struct ShotHUD: View {
.foregroundStyle(.secondary)
#if os(macOS)
Text("⌘⎋ releases the mouse")
.font(.caption2).foregroundStyle(.secondary)
.font(.geist(11, relativeTo: .caption2)).foregroundStyle(.secondary)
#elseif os(tvOS)
Text("Press Menu to disconnect")
.font(.caption).foregroundStyle(.secondary)
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
#endif
}
.padding(10)
@@ -259,7 +259,7 @@ private struct ShotDesktopFrame: View {
HStack(spacing: 8) {
Image(systemName: "gamecontroller.fill")
Text("Streaming from Battlestation")
.font(.system(.callout, weight: .semibold))
.font(.geist(16, .semibold, relativeTo: .callout))
}
.padding(.horizontal, 14).padding(.vertical, 9)
.glassBackground(Capsule())
@@ -129,6 +129,8 @@ final class SessionModel: ObservableObject {
#endif
}()
let hdrCapable = hdrEnabled && displayHDR
// 4:4:4 opt-out (default on); the hardware-decode probe below is the real gate.
let want444 = (UserDefaults.standard.object(forKey: DefaultsKey.enable444) as? Bool) ?? true
Task.detached(priority: .userInitiated) {
// PunktfunkConnection.init blocks on the QUIC handshake keep it off the main
// actor. The persistent identity is presented on every connect so a paired
@@ -138,9 +140,21 @@ final class SessionModel: ObservableObject {
// Advertise 10-bit + HDR10 when enabled: the host upgrades to a BT.2020 PQ Main10 stream
// only for actual HDR content (its own gate); the VideoToolbox/Metal present path is
// HDR-capable (P010 + itur_2100_PQ + EDR). 0 keeps the 8-bit BT.709 SDR stream.
let videoCaps: UInt8 = hdrCapable
var videoCaps: UInt8 = hdrCapable
? (PunktfunkConnection.videoCap10Bit | PunktfunkConnection.videoCapHDR)
: 0
// Advertise full-chroma 4:4:4 only when allowed AND this device can HARDWARE-decode it
// (software 4:4:4 is too slow for real-time). The host content-gates depth, so an
// HDR-advertised session can still receive an 8-bit 4:4:4 stream (SDR content) require
// BOTH depths there. Otherwise a no-op (the host emits 4:4:4 only if it too opted in);
// `chromaFormat` on the connection reflects what was actually resolved.
let canDecode444 =
hdrCapable
? (Stage444Probe.hwDecode444_8bit && Stage444Probe.hwDecode444_10bit)
: Stage444Probe.hwDecode444_8bit
if want444, canDecode444 {
videoCaps |= PunktfunkConnection.videoCap444
}
let result = Result { try PunktfunkConnection(
host: host.address, port: host.port,
width: width, height: height, refreshHz: hz,
@@ -1,10 +1,12 @@
// App settings. The host creates a native virtual output at exactly the chosen size/refresh
// there is no scaling anywhere in the pipeline.
//
// Navigation differs per platform: macOS uses a tabbed preferences window (the sections had
// outgrown one scrolling pane); iOS uses a single grouped Form; tvOS uses a focus-native
// pushed-picker layout. The individual sections (`streamModeSection`, `audioSection`, ) are
// shared across all three so a setting is defined exactly once.
// Navigation differs per platform, but all three group the same categories (General, Display,
// Audio, Controllers, Advanced, About): macOS uses a tabbed preferences window; iOS/iPadOS uses
// an adaptive NavigationSplitView a category sidebar + detail pane on iPad, auto-collapsing to
// a hierarchical push list on iPhone (the system Settings idiom on each); tvOS uses a
// focus-native pushed-picker layout. The individual sections (`streamModeSection`,
// `audioSection`, ) are shared across all three so a setting is defined exactly once.
#if os(macOS)
import AppKit
@@ -21,7 +23,9 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.compositor) private var compositor = 0
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.presenter) private var presenter = "stage1"
@AppStorage(DefaultsKey.presenter) private var presenter = "stage2"
@AppStorage(DefaultsKey.hdrEnabled) private var hdrEnabled = true
@AppStorage(DefaultsKey.enable444) private var enable444 = true
@AppStorage(DefaultsKey.libraryEnabled) private var libraryEnabled = false
@AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.micEnabled) private var micEnabled = true
@@ -32,6 +36,22 @@ struct SettingsView: View {
#if DEBUG && !os(tvOS)
@State private var showControllerTest = false
#endif
#if os(iOS)
@AppStorage(DefaultsKey.pointerCapture) private var pointerCapture = true
// The sidebar selection drives the detail pane on iPad and the pushed sub-page on iPhone.
// Width class decides the initial value: nil on iPhone (show the category list first),
// General on iPad (a two-column layout should never open with an empty detail).
@Environment(\.horizontalSizeClass) private var horizontalSizeClass
@State private var settingsSelection: SettingsCategory?
// Tracked so the detail can show its own Done whenever the sidebar (and its Done) is off screen
// not just on iPhone, but on any iPad layout that collapses the sidebar to an overlay. Starts
// .doubleColumn so iPad reliably opens with the sidebar (and its Done) visible.
@State private var columnVisibility: NavigationSplitViewVisibility = .doubleColumn
// Sticky once the wheel lands on "Custom", so editing a width/height that briefly equals a
// preset doesn't snap the wheel back off Custom. A stored non-preset value reads as custom even
// when this is false (see `isCustomResolution`), so it survives relaunches without persisting.
@State private var customMode = false
#endif
#if os(macOS)
@AppStorage(DefaultsKey.speakerUID) private var speakerUID = ""
@AppStorage(DefaultsKey.micUID) private var micUID = ""
@@ -39,6 +59,15 @@ struct SettingsView: View {
@State private var inputDevices: [AudioDevice] = []
#endif
#if os(iOS)
/// `initialCategory` is nil in the app (the list opens un-selected on iPhone; iPad lands on
/// General via `onAppear`). The screenshot harness passes an explicit category so the captured
/// shot opens on a real settings page (a populated detail) rather than the bare category list.
init(initialCategory: SettingsCategory? = nil) {
_settingsSelection = State(initialValue: initialCategory)
}
#endif
var body: some View {
#if os(tvOS)
// Native tv pattern: no inline text entry (typing numbers with a remote is
@@ -66,6 +95,7 @@ struct SettingsView: View {
Form {
presenterSection
hdrSection
windowSection
statisticsSection
}
@@ -106,29 +136,116 @@ struct SettingsView: View {
}
#endif
// MARK: - iOS: one grouped Form
// MARK: - iOS / iPadOS: adaptive split view
#if os(iOS)
private var iosBody: some View {
Form {
streamModeSection
audioSection
compositorSection
presenterSection
statisticsSection
experimentalSection
controllersSection
Section {
NavigationLink("Acknowledgements") { AcknowledgementsView() }
NavigationSplitView(columnVisibility: $columnVisibility) {
List(selection: $settingsSelection) {
ForEach(SettingsCategory.allCases) { category in
// On iPhone the split view collapses to a push list, but a selection List
// draws no disclosure indicator of its own add one in compact width for the
// expected drill-in affordance. On iPad the selected row highlights instead, so
// the chevron is omitted there.
HStack {
Label(category.title, systemImage: category.symbol)
if horizontalSizeClass == .compact {
Spacer()
Image(systemName: "chevron.forward")
.font(.footnote.weight(.semibold))
.foregroundStyle(.tertiary)
// Purely a drill-in affordance the row's button trait already
// conveys "opens"; keep it out of the VoiceOver announcement.
.accessibilityHidden(true)
}
}
.tag(category)
}
}
.navigationTitle("Settings")
.toolbar {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
}
}
} detail: {
// NavigationSplitView hosts the detail in its own navigation context (its title bar),
// so no inner NavigationStack that would double the bar on iPad. On iPhone the split
// view collapses to one stack and pushes this when a row is tapped. `?? .general` only
// backs the brief pre-selection window; the list never auto-pushes on a nil selection.
settingsDetail(settingsSelection ?? .general)
// Keep a Done on the detail whenever the sidebar (and its Done) isn't on screen: the
// iPhone push, or any iPad layout that collapsed the sidebar to an overlay. When the
// sidebar is showing, its Done is the only one so this stays hidden to avoid two.
.toolbar {
if horizontalSizeClass == .compact || columnVisibility == .detailOnly {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
}
}
}
}
.formStyle(.grouped)
.onAppear {
if horizontalSizeClass == .regular, settingsSelection == nil {
settingsSelection = .general
}
gamepads.refresh()
gamepads.startDiscovery()
}
// A regularregular launch sets the default above; this catches a compactregular change
// (e.g. an iPad leaving narrow split-screen multitasking) so the detail pane fills in.
.onChange(of: horizontalSizeClass) { _, newValue in
if newValue == .regular, settingsSelection == nil {
settingsSelection = .general
}
}
.onDisappear { gamepads.stopDiscovery() }
}
@ViewBuilder
private func settingsDetail(_ category: SettingsCategory) -> some View {
switch category {
case .general:
Form {
streamModeSection
pointerSection
compositorSection
}
.formStyle(.grouped)
.navigationTitle("General")
.navigationBarTitleDisplayMode(.inline)
case .display:
Form {
presenterSection
hdrSection
statisticsSection
}
.formStyle(.grouped)
.navigationTitle("Display")
.navigationBarTitleDisplayMode(.inline)
case .audio:
Form { audioSection }
.formStyle(.grouped)
.navigationTitle("Audio")
.navigationBarTitleDisplayMode(.inline)
case .controllers:
Form { controllersSection }
.formStyle(.grouped)
.navigationTitle("Controllers")
.navigationBarTitleDisplayMode(.inline)
case .advanced:
Form { experimentalSection }
.formStyle(.grouped)
.navigationTitle("Advanced")
.navigationBarTitleDisplayMode(.inline)
case .about:
// Already a full scrollable view that sets its own "Acknowledgements" title; pin the
// display mode inline to match the five sibling detail pages (it would otherwise inherit
// the large title from the "Settings" sidebar root).
AcknowledgementsView()
.navigationBarTitleDisplayMode(.inline)
}
}
#endif
// MARK: - tvOS
@@ -156,6 +273,10 @@ struct SettingsView: View {
Binding(get: { hudEnabled ? "on" : "off" }, set: { hudEnabled = $0 == "on" })
}
private var hdrEnabledTag: Binding<String> {
Binding(get: { hdrEnabled ? "on" : "off" }, set: { hdrEnabled = $0 == "on" })
}
private var tvBody: some View {
let currentTag = "\(width)x\(height)x\(hz)"
let bounds = UIScreen.main.nativeBounds
@@ -186,20 +307,25 @@ struct SettingsView: View {
selection: $audioChannels)
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange)
.multilineTextAlignment(.center)
}
TVSelectionRow(
title: "Compositor", options: compositors, selection: $compositor)
#if DEBUG
TVSelectionRow(
title: "Presenter",
options: [("Stage 1 (default)", "stage1"), ("Stage 2 (experimental)", "stage2")],
title: "Presenter (debug)",
options: [("Stage 2 (default)", "stage2"), ("Stage 1 (debug)", "stage1")],
selection: $presenter)
#endif
TVSelectionRow(
title: "10-bit HDR",
options: [("On", "on"), ("Off", "off")], selection: hdrEnabledTag)
Text("The host creates a virtual output at exactly this mode — native "
+ "resolution, no scaling. \(Self.bitrateFooter) A specific compositor "
+ "is honored only if available on the host.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
.padding(.top, 8)
@@ -219,7 +345,7 @@ struct SettingsView: View {
TVSelectionRow(
title: "Controller type", options: Self.padTypes, selection: $gamepadType)
Text(Self.controllersFooter)
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
.padding(.top, 8)
@@ -243,6 +369,63 @@ struct SettingsView: View {
@ViewBuilder private var streamModeSection: some View {
Section {
#if os(iOS)
// Touch-first: a rotating wheel of common resolutions (this device's own mode first) and
// a segmented refresh-rate control the same family as the Clock/Timer pickers. The host
// renders a virtual output at exactly the chosen mode, so these are real pixel sizes. The
// last wheel row, "Custom", reveals width/height/refresh fields for an arbitrary mode.
VStack(alignment: .leading, spacing: 4) {
Text("Resolution")
.font(.geist(15, relativeTo: .subheadline))
.foregroundStyle(.secondary)
Picker("Resolution", selection: resolutionSelection) {
ForEach(resolutionChoices, id: \.tag) { choice in
Text(choice.label).tag(choice.tag)
}
}
.labelsHidden()
.pickerStyle(.wheel)
.frame(maxHeight: 140)
}
if isCustomResolution {
// Arbitrary entry: type the exact width × height (and refresh) the host should drive.
HStack {
TextField("Width", value: $width, format: .number.grouping(.never))
.keyboardType(.numberPad)
Text("×")
TextField("Height", value: $height, format: .number.grouping(.never))
.labelsHidden()
.keyboardType(.numberPad)
}
// A row built from an HStack of TextFields otherwise insets its bottom separator to
// the inner content, clipping the hairline under "Width"; pin it to the cell edge.
.alignmentGuide(.listRowSeparatorLeading) { _ in 0 }
LabeledContent("Refresh rate") {
TextField("Hz", value: $hz, format: .number.grouping(.never))
.keyboardType(.numberPad)
.multilineTextAlignment(.trailing)
}
} else if refreshChoices.count > 1 {
VStack(alignment: .leading, spacing: 6) {
Text("Refresh rate")
.font(.geist(15, relativeTo: .subheadline))
.foregroundStyle(.secondary)
Picker("Refresh rate", selection: $hz) {
ForEach(refreshChoices, id: \.self) { rate in
Text("\(rate) Hz").tag(rate)
}
}
.labelsHidden()
.pickerStyle(.segmented)
}
} else {
// A device with a single supported rate (e.g. 60 Hz) has nothing to pick.
LabeledContent("Refresh rate") {
Text("\(hz) Hz").foregroundStyle(.secondary)
}
}
Button("Use this display's mode") { fillFromMainScreen() }
#elseif os(macOS)
HStack {
TextField("Resolution", value: $width, format: .number.grouping(.never))
Text("×")
@@ -253,6 +436,7 @@ struct SettingsView: View {
LabeledContent("") {
Button("Use this display's mode") { fillFromMainScreen() }
}
#endif
#if !os(tvOS)
Toggle("Automatic bitrate", isOn: automaticBitrate)
if bitrateKbps != 0 {
@@ -267,7 +451,7 @@ struct SettingsView: View {
}
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange)
}
}
@@ -277,11 +461,85 @@ struct SettingsView: View {
} footer: {
Text("The host creates a virtual output at exactly this mode — "
+ "native resolution, no scaling. \(Self.bitrateFooter)")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
#if os(iOS)
// MARK: - Stream mode (iOS wheel)
/// Sentinel wheel tag for the "Custom" row. Real tags are "WxH" (digits + "x"), so this can't
/// collide with a resolution.
private static let customResolutionTag = "custom"
/// 16:9 then ultrawide presets; the device's native mode is prepended at runtime.
private static let resolutionPresets: [(name: String, w: Int, h: Int)] = [
("720p", 1280, 720),
("1080p", 1920, 1080),
("1440p", 2560, 1440),
("4K", 3840, 2160),
("Ultrawide 1080p", 2560, 1080),
("Ultrawide 1440p", 3440, 1440),
("Super ultrawide", 5120, 1440),
]
/// The non-custom wheel rows: this device's native mode first, then the presets, deduped by
/// dimensions (native wins a tie).
private var resolutionModes: [(name: String, w: Int, h: Int)] {
let bounds = UIScreen.main.nativeBounds // portrait-oriented pixels
let native = (w: Int(max(bounds.width, bounds.height)), h: Int(min(bounds.width, bounds.height)))
let all = [(name: "This device", w: native.w, h: native.h)] + Self.resolutionPresets
var seen = Set<String>()
return all.filter { seen.insert("\($0.w)x\($0.h)").inserted }
}
/// Wheel rows: the resolution modes, then a "Custom" row that reveals the numeric fields.
private var resolutionChoices: [(label: String, tag: String)] {
resolutionModes.map { (label: "\($0.name) · \($0.w) × \($0.h)", tag: "\($0.w)x\($0.h)") }
+ [(label: "Custom…", tag: Self.customResolutionTag)]
}
private var presetResolutionTags: Set<String> {
Set(resolutionModes.map { "\($0.w)x\($0.h)" })
}
/// True when the editable custom fields should show: the wheel is parked on "Custom" (sticky),
/// or the stored size simply isn't one of the presets (e.g. a value synced from a Mac) so a
/// non-preset mode stays editable across relaunches without a persisted flag.
private var isCustomResolution: Bool {
customMode || !presetResolutionTags.contains("\(width)x\(height)")
}
/// The wheel works in "WxH" tags so one selection drives both width and height; the custom
/// sentinel toggles `customMode` instead of writing a size.
private var resolutionSelection: Binding<String> {
Binding(
get: { isCustomResolution ? Self.customResolutionTag : "\(width)x\(height)" },
set: { tag in
if tag == Self.customResolutionTag {
customMode = true
return
}
customMode = false
let parts = tag.split(separator: "x").compactMap { Int($0) }
guard parts.count == 2 else { return }
width = parts[0]
height = parts[1]
})
}
/// Refresh rates the device can actually display (no point asking the host to render frames the
/// screen can't show), plus any stored custom value so it stays selectable.
private var refreshChoices: [Int] {
let maxHz = UIScreen.main.maximumFramesPerSecond
var rates = [60, 120, 240].filter { $0 <= maxHz }
if rates.isEmpty { rates = [maxHz] }
if !rates.contains(hz) { rates.append(hz) }
return rates.sorted()
}
#endif
@ViewBuilder private var audioSection: some View {
Section {
Picker("Audio channels", selection: $audioChannels) {
@@ -321,11 +579,35 @@ struct SettingsView: View {
Text("Host audio plays through the speaker; the microphone feeds the "
+ "host's virtual mic. System default follows macOS device changes. "
+ "Applies from the next session.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
#if os(iOS)
/// iPad-only pointer-capture toggle: lock the mouse/trackpad for relative movement (games) vs
/// forward an absolute cursor position (desktop). Empty on iPhone (no hardware-pointer lock
/// the mouse path there is always the absolute fallback).
@ViewBuilder private var pointerSection: some View {
if UIDevice.current.userInterfaceIdiom == .pad {
Section {
Toggle("Capture pointer for games", isOn: $pointerCapture)
} header: {
Text("Pointer")
} footer: {
Text("With a mouse or trackpad connected, lock the pointer and send relative "
+ "movement — the expected behavior for games (mouse-look). Turn this off for "
+ "desktop use to keep the pointer free and send its absolute position instead. "
+ "The lock needs the stream full-screen and frontmost; it falls back to the "
+ "absolute pointer automatically (Stage Manager, Slide Over). Finger touch is "
+ "unaffected. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
}
#endif
@ViewBuilder private var compositorSection: some View {
Section {
Picker("Compositor", selection: $compositor) {
@@ -341,7 +623,7 @@ struct SettingsView: View {
Text("Which compositor drives the virtual output on the host. A specific "
+ "choice is honored only if that backend is available there — "
+ "otherwise the host falls back to auto-detection.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@@ -355,26 +637,50 @@ struct SettingsView: View {
} footer: {
Text("Take the window fullscreen when a session starts and restore it on the host "
+ "list, so only the stream is fullscreen — not the picker.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
// Stage-2 (Metal/VTDecompressionSession) is the default and only user-visible presenter it
// recovers from a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard on a
// lost HEVC reference. Stage-1 is kept reachable as a DEBUG-only override for diagnostics, like
// the controller test. Empty in release builds (no presenter UI; stage-2 always).
@ViewBuilder private var presenterSection: some View {
#if DEBUG
Section {
Picker("Presenter", selection: $presenter) {
Text("Stage 1 (default)").tag("stage1")
Text("Stage 2 (experimental)").tag("stage2")
Text("Stage 2 (default)").tag("stage2")
Text("Stage 1 (debug)").tag("stage1")
}
} header: {
Text("Video presenter")
Text("Video presenter · debug")
} footer: {
Text("Stage 1 feeds compressed video to the system display layer (known-good). "
+ "Stage 2 decodes explicitly and presents through Metal with a display "
+ "link — it adds a capture→present (glass-to-glass) latency line in the HUD "
+ "and shortens the present tail. Applies from the next session.")
.font(.caption)
Text("Stage 2 (default) decodes explicitly and presents through Metal with a display "
+ "link — it adds a capture→present (glass-to-glass) latency line in the HUD and "
+ "self-recovers from decode stalls. Stage 1 feeds compressed video straight to the "
+ "system display layer; it freezes on a lost HEVC reference frame, so it's a debug "
+ "fallback only. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
@ViewBuilder private var hdrSection: some View {
Section {
Toggle("10-bit HDR", isOn: $hdrEnabled)
Toggle("Full chroma (4:4:4)", isOn: $enable444)
} header: {
Text("Video quality")
} footer: {
Text("HDR requests a 10-bit BT.2020 PQ (HDR10) stream — it only engages when the host is "
+ "sending HDR content AND this display supports HDR. 4:4:4 requests full chroma "
+ "(sharper text/UI, more bandwidth) — it only engages when this device can "
+ "hardware-decode it AND the host opted in. Otherwise the stream stays 8-bit "
+ "4:2:0 SDR. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@@ -392,7 +698,7 @@ struct SettingsView: View {
Text("Statistics")
} footer: {
Text(Self.statisticsFooter)
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@@ -407,7 +713,7 @@ struct SettingsView: View {
+ "(Steam + custom) via the host's management API; tap a title to launch it. "
+ "The host must expose that API on the LAN with a token "
+ "(serve --mgmt-bind 0.0.0.0 --mgmt-token …).")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@@ -441,7 +747,7 @@ struct SettingsView: View {
Text("Controllers")
} footer: {
Text(Self.controllersFooter)
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@@ -593,13 +899,13 @@ struct SettingsView: View {
}
}
}
.font(.caption2)
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
}
Spacer()
if gamepads.active?.id == controller.id {
Text("In use")
.font(.caption2.weight(.semibold))
.font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 8)
.padding(.vertical, 3)
.background(Capsule().fill(.green.opacity(0.2)))
@@ -621,6 +927,10 @@ struct SettingsView: View {
width = Int(max(bounds.width, bounds.height))
height = Int(min(bounds.width, bounds.height))
hz = UIScreen.main.maximumFramesPerSecond
#if os(iOS)
// The native mode is the "This device" wheel row, so leave Custom mode if it was on.
customMode = false
#endif
#endif
}
}
@@ -631,3 +941,52 @@ extension Double {
Swift.min(Swift.max(self, lo), hi)
}
}
#if os(iOS)
/// The settings groups, mirroring the macOS preference tabs. On iPad each is a sidebar row that
/// drives the detail pane; on iPhone the same list collapses to pushed sub-pages. Internal (not
/// private) so the screenshot harness can open SettingsView on a specific category.
enum SettingsCategory: String, CaseIterable, Identifiable {
case general, display, audio, controllers, advanced, about
var id: Self { self }
var title: String {
switch self {
case .general: return "General"
case .display: return "Display"
case .audio: return "Audio"
case .controllers: return "Controllers"
case .advanced: return "Advanced"
case .about: return "About"
}
}
var symbol: String {
switch self {
case .general: return "gearshape"
case .display: return "display"
case .audio: return "speaker.wave.2"
case .controllers: return "gamecontroller"
case .advanced: return "slider.horizontal.3"
case .about: return "info.circle"
}
}
}
extension View {
/// Present the settings sheet large on iPad so the NavigationSplitView has room for its
/// sidebar + detail a default form sheet is too narrow and the split view would collapse to
/// the iPhone push list. No-op on iPhone (the standard sheet is already right) and on iOS 17
/// (no `presentationSizing` it falls back to the default sheet, which still degrades cleanly
/// to the push list).
@ViewBuilder
func settingsSheetSizing() -> some View {
if UIDevice.current.userInterfaceIdiom == .pad, #available(iOS 18, *) {
presentationSizing(.page)
} else {
self
}
}
}
#endif
@@ -52,7 +52,7 @@ struct SpeedTestSheet: View {
var body: some View {
VStack(spacing: 20) {
Label("Speed test — \(host.displayName)", systemImage: "gauge.with.needle")
.font(.headline)
.font(.geist(17, .semibold, relativeTo: .headline))
.foregroundStyle(.tint)
switch phase {
@@ -73,7 +73,7 @@ struct SpeedTestSheet: View {
resultView(result)
case .failed(let message):
Text(message)
.font(.callout)
.font(.geist(16, relativeTo: .callout))
.foregroundStyle(.red)
.multilineTextAlignment(.center)
}
@@ -149,13 +149,13 @@ struct SpeedTestSheet: View {
if let rec = Self.recommendedKbps(result) {
Text("Recommended bitrate: \(Self.mbpsLabel(kbps: rec)) "
+ "(~70% of measured, headroom for encoder bursts).")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
} else {
Text("Too little data made it through to recommend a bitrate — "
+ "check the network and retry.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
}
@@ -69,19 +69,19 @@ struct StreamHUDView: View {
Text(model.mouseCaptured
? "⌘⎋ releases the mouse"
: "Click the stream to capture input")
.font(.caption2)
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
// The client-side cursor (C) draws the local cursor over the stream instead of
// capturing it the only accurate cursor for gamescope, whose capture has none.
Text("⌘⇧C toggles the on-screen cursor")
.font(.caption2)
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
#elseif os(iOS)
// Touch always plays directly; (hardware keyboard) toggles kb/mouse.
Text(model.mouseCaptured
? "⌘⎋ releases keyboard & mouse"
: "⌘⎋ captures keyboard & mouse")
.font(.caption2)
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
#endif
#if os(tvOS)
@@ -89,13 +89,13 @@ struct StreamHUDView: View {
// A press (the focus engine consumes it before the host sees it). Disconnect is
// the Siri Remote's Menu button (.onExitCommand on the stream) just hint it.
Text("Press Menu to disconnect")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
#else
// D lives on the app's Stream menu (so it still works when the HUD is hidden);
// this button is the in-overlay, click-to-disconnect affordance.
Button("Disconnect (⌘D)") { model.disconnect() }
.font(.caption)
.font(.geist(12, relativeTo: .caption))
#endif
}
.padding(10)
@@ -3,6 +3,7 @@
// or drops this and runs the PIN pairing ceremony instead.
import Foundation
import PunktfunkKit
import SwiftUI
struct TrustCardView: View {
@@ -18,11 +19,11 @@ struct TrustCardView: View {
.font(.system(size: 36, weight: .light))
.foregroundStyle(.tint)
Text("Verify \(hostName)")
.font(.title3.weight(.semibold))
.font(.geist(20, .semibold, relativeTo: .title3))
Text("First connection. Compare this fingerprint with the one "
+ "punktfunk-host logged at startup (\u{201C}clients pin this "
+ "fingerprint\u{201D}):")
.font(.callout)
.font(.geist(16, relativeTo: .callout))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
Text(Self.format(fingerprint: fingerprint))
@@ -58,7 +59,7 @@ struct TrustCardView: View {
#else
.buttonStyle(.borderless)
#endif
.font(.callout)
.font(.geist(16, relativeTo: .callout))
}
.padding(28)
.frame(maxWidth: 440)
@@ -0,0 +1,101 @@
// Geist the punktfunk brand typeface (the same family the website ships). Bundled as static
// OTF weights in this kit's resources and registered with Core Text at first use, so it works
// identically in the Xcode app and the `swift run` dev shell (Bundle.module resolves to the
// package resource bundle in both). Geist Sans carries titles/UI; Geist Mono carries the technical
// readouts host addresses, status labels, the stream-stats HUD for the industrial look.
//
// Licensed under the SIL Open Font License 1.1 (Resources/Fonts/Geist-OFL.txt).
import CoreText
import SwiftUI
#if canImport(UIKit)
import UIKit
#elseif canImport(AppKit)
import AppKit
#endif
public enum BrandFont {
public enum Weight {
case regular, medium, semibold, bold
}
/// PostScript names of the bundled faces (verified from each OTF's name table). Geist Sans only
/// Geist Mono is intentionally not shipped; the app's typeface is Geist Sans throughout.
private static let sansFaces = ["Geist-Regular", "Geist-Medium", "Geist-SemiBold", "Geist-Bold"]
/// Registered exactly once per process a static `let` initializer is run lazily and is
/// guaranteed thread-safe + run-at-most-once by the runtime.
private static let registered: Void = {
for face in sansFaces {
guard let url = Bundle.module.url(
forResource: face, withExtension: "otf", subdirectory: "Fonts") else {
#if DEBUG
print("BrandFont: bundled face \(face).otf not found — text will fall back to system")
#endif
continue
}
var error: Unmanaged<CFError>?
if !CTFontManagerRegisterFontsForURL(url as CFURL, .process, &error) {
#if DEBUG
let message = error?.takeRetainedValue().localizedDescription ?? "unknown error"
print("BrandFont: failed to register \(face): \(message)")
#endif
}
}
}()
/// Force registration before the first `Font.custom` lookup. Cheap to call repeatedly.
public static func registerIfNeeded() { _ = registered }
fileprivate static func sansFace(_ weight: Weight) -> String {
switch weight {
case .regular: return "Geist-Regular"
case .medium: return "Geist-Medium"
case .semibold: return "Geist-SemiBold"
case .bold: return "Geist-Bold"
}
}
}
public extension Color {
/// The punktfunk brand purple (the app-icon lens / website `--brand`). Defined explicitly,
/// independent of the asset-catalog accent `Color.accentColor` resolution is environment- and
/// timing-sensitive (it can fall back to system blue), and the brand mark must never drift.
/// Light: #6656F2, Dark: #8678F5 (the lighter violet reads better on dark surfaces).
static let brand: Color = {
#if canImport(UIKit)
return Color(UIColor { traits in
traits.userInterfaceStyle == .dark
? UIColor(red: 0x86 / 255, green: 0x78 / 255, blue: 0xF5 / 255, alpha: 1)
: UIColor(red: 0x66 / 255, green: 0x56 / 255, blue: 0xF2 / 255, alpha: 1)
})
#elseif canImport(AppKit)
return Color(NSColor(name: nil) { appearance in
appearance.bestMatch(from: [.aqua, .darkAqua]) == .darkAqua
? NSColor(red: 0x86 / 255, green: 0x78 / 255, blue: 0xF5 / 255, alpha: 1)
: NSColor(red: 0x66 / 255, green: 0x56 / 255, blue: 0xF2 / 255, alpha: 1)
})
#else
// Non-Apple fallback: the light brand value, so all branches agree on a canonical color.
return Color(red: 0x66 / 255, green: 0x56 / 255, blue: 0xF2 / 255)
#endif
}()
}
public extension Font {
/// Geist Sans at an explicit point size, scaling with Dynamic Type relative to `textStyle`.
static func geist(
_ size: CGFloat, _ weight: BrandFont.Weight = .regular,
relativeTo textStyle: TextStyle = .body
) -> Font {
BrandFont.registerIfNeeded()
return .custom(BrandFont.sansFace(weight), size: size, relativeTo: textStyle)
}
/// Geist Sans at a FIXED point size that does not scale with Dynamic Type for glyphs pinned
/// inside a fixed-size container (e.g. the monogram tile), where a scaled letter would overflow.
static func geistFixed(_ size: CGFloat, _ weight: BrandFont.Weight = .regular) -> Font {
BrandFont.registerIfNeeded()
return .custom(BrandFont.sansFace(weight), fixedSize: size)
}
}
@@ -22,9 +22,22 @@ public enum DefaultsKey {
public static let speakerUID = "punktfunk.speakerUID"
public static let micUID = "punktfunk.micUID"
public static let presenter = "punktfunk.presenter"
/// Request a 10-bit BT.2020 PQ (HDR10) stream. On by default; only takes effect when the host
/// has HDR content AND this display supports HDR otherwise the stream stays 8-bit SDR.
public static let hdrEnabled = "punktfunk.hdrEnabled"
/// Request a full-chroma 4:4:4 stream when this device can HARDWARE-decode it (`Stage444Probe`).
/// On by default; only takes effect when the host also opted in to 4:4:4 (otherwise the stream
/// stays 4:2:0). Sharper text/UI at the cost of more bandwidth.
public static let enable444 = "punktfunk.enable444"
public static let hosts = "punktfunk.hosts"
/// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never".
public static let cursorMode = "punktfunk.cursorMode"
/// iPad: capture the mouse/trackpad pointer (pointer lock relative movement) for games,
/// rather than forwarding an absolute cursor position. On by default. Only meaningful on iPad
/// with a hardware mouse/trackpad; the system grants the lock only to a full-screen, frontmost
/// scene and silently falls back to the absolute pointer when it can't (Stage Manager / Slide
/// Over). Read by `StreamViewController.prefersPointerLocked`.
public static let pointerCapture = "punktfunk.pointerCapture"
/// Experimental: show the host's game library (browsed over the management API). Off by default.
public static let libraryEnabled = "punktfunk.libraryEnabled"
/// macOS: take the window fullscreen while streaming and restore it on the host list. On by default.
@@ -68,6 +68,14 @@ private final class RumbleRenderer: @unchecked Sendable {
private var broken = false
/// Last logged active/silent state for a one-line transition log, not per-event spam.
private var wasActive = false
// Backoff after an engine failure. A broken `gamecontrollerd.haptics` XPC connection (CoreHaptics
// -4811 "server connection broke") fails EVERY rebuild until the service relaunches and that
// break fires neither stoppedHandler nor resetHandler, so without a cooldown the next rumble
// update immediately rebuilds into the same dead connection, flooding the log and never
// recovering. Delay the next setup() growing 0.5124 s on repeated failure and clear it
// the moment a player runs cleanly (or the controller changes).
private var retryAfter = Date.distantPast
private var consecutiveFailures = 0
/// CHHapticEvent sharpness = actuator frequency. A DualSense's voice-coil motors need a
/// defined frequency to move at all an intensity-only event (no sharpness) left them
@@ -91,6 +99,8 @@ private final class RumbleRenderer: @unchecked Sendable {
self.closeHID()
self.controller = c
self.broken = false
self.consecutiveFailures = 0
self.retryAfter = .distantPast
_ = self.openHIDIfDualSense(c)
onBackend?(self.backendNote(for: c))
}
@@ -108,7 +118,7 @@ private final class RumbleRenderer: @unchecked Sendable {
// other pad (and for a DualSense whose HID device could not be opened).
if self.hidRumble(low: lowAmp, high: highAmp) { return }
guard !self.broken else { return }
if active, self.low == nil, self.high == nil {
if active, self.low == nil, self.high == nil, Date() >= self.retryAfter {
self.setup()
}
let ok: Bool
@@ -124,8 +134,15 @@ private final class RumbleRenderer: @unchecked Sendable {
}
// Rebuild on the next nonzero amplitude if an engine errored and tear down OUTSIDE
// the `inout` accesses above, so teardown() never mutates a motor that a `drive` call
// still holds an exclusive reference to.
if !ok { self.teardown() }
// still holds an exclusive reference to. Back off so a broken XPC isn't re-hit every
// update; once a player is actually running the path has recovered, so clear the backoff.
if !ok {
self.teardown()
self.scheduleRetryBackoff()
} else if self.low?.player != nil || self.high?.player != nil {
self.consecutiveFailures = 0
self.retryAfter = .distantPast
}
}
}
@@ -157,14 +174,29 @@ private final class RumbleRenderer: @unchecked Sendable {
low = makeMotor(haptics, .default)
}
if low == nil, high == nil {
// Haptics present but no engine could be built right now (server busy / a transient
// error). Do NOT latch broken the next nonzero amplitude retries setup().
log.warning("rumble: haptics present but engine setup failed — will retry on next rumble")
// Haptics present but no engine could be built right now (server busy / XPC broken). Do
// NOT latch broken back off and the next nonzero amplitude past the cooldown retries.
log.warning("rumble: haptics present but engine setup failed — backing off, will retry")
scheduleRetryBackoff()
}
}
/// Push the next engine-build attempt out after a failure (capped exponential backoff), so a
/// broken `gamecontrollerd.haptics` connection gets time to relaunch instead of being re-hit on
/// every rumble update.
private func scheduleRetryBackoff() {
consecutiveFailures += 1
let shift = min(consecutiveFailures - 1, 4)
retryAfter = Date().addingTimeInterval(min(0.5 * Double(1 << shift), 4))
}
private func makeMotor(_ haptics: GCDeviceHaptics, _ locality: GCHapticsLocality) -> Motor? {
guard let engine = haptics.createEngine(withLocality: locality) else { return nil }
// A controller's motors carry no audio, so keep this engine OUT of the app's audio session
// (the default is to join it). Streaming keeps an AVAudioSession active the whole time;
// letting a haptics-only engine join it is a needless coupling that can get its
// gamecontrollerd XPC connection interrupted (the repeated -4811 server-connection breaks).
engine.playsHapticsOnly = true
// The haptic server can stop or reset the engine out from under us app backgrounding, an
// audio-session interruption (a call, Siri, another audio app), or a server crash. Left
// unhandled the players go dead and every later rumble throws, latching rumble off for the
@@ -338,29 +370,32 @@ public final class GamepadFeedback {
// Hidout traffic (lightbar / player LEDs / triggers) only exists on a PlayStation-pad
// session a DualSense or a DualShock 4 (lightbar only). Block briefly on it there and
// let rumble own the wait elsewhere; on an Xbox session it stays nonblocking.
let hasHidout = connection.resolvedGamepad == .dualSense
|| connection.resolvedGamepad == .dualShock4
let hidTimeout: UInt32 = hasHidout ? 10 : 0
let thread = Thread { [connection, flag, drainDone, weak self] in
while !flag.isStopped {
do {
if let r = try connection.nextRumble(timeoutMs: 10), r.pad == 0 {
// Poll the feedback planes NON-BLOCKING. A blocking poll (timeoutMs > 0) holds
// the connection's shared feedback lock for its whole wait; the video pump drains
// HDR mastering metadata (nextHdrMeta) on the SAME lock every frame, so a blocking
// poll here starved it and throttled HDR to ~1 fps (SDR, which never drains HDR
// meta, was unaffected). Pacing with a short sleep OUTSIDE the lock (below) keeps
// rumble/HID latency low while leaving the lock free between polls.
if let r = try connection.nextRumble(timeoutMs: 0), r.pad == 0 {
self?.rumble.apply(low: r.low, high: r.high)
}
// Drain a BOUNDED burst of hidout events: only the first poll waits,
// and the cap + stop check keep sustained 0xCD traffic (a game writing
// per-frame LED/trigger reports) from starving the rumble poll above
// or blocking stop() past one cycle.
// Drain a BOUNDED burst of hidout events so sustained 0xCD traffic (a game writing
// per-frame LED/trigger reports) can't spin here or block stop() past one cycle.
var burst = 0
while burst < 64, !flag.isStopped,
let ev = try connection.nextHidOutput(
timeoutMs: burst == 0 ? hidTimeout : 0) {
let ev = try connection.nextHidOutput(timeoutMs: 0) {
self?.render(ev)
burst += 1
}
} catch {
break // .closed (or fatal) the session is over
}
// ~8 ms poll cadence (125 Hz), slept OUTSIDE the feedback lock low rumble/HID
// latency without holding the lock the HDR-meta drain needs.
if !flag.isStopped { Thread.sleep(forTimeInterval: 0.008) }
}
drainDone.signal()
}
@@ -107,6 +107,23 @@ public final class InputCapture {
/// macOS (no GCMouse handlers installed; `sendMouseAbs` is never called there). Main-queue.
public var gcMouseForwarding = false
#if os(iOS)
/// Whether any device is attached as a `GCMouse` right now. The Magic Keyboard TRACKPAD does
/// not always register as a GCMouse on iPadOS (only a standalone mouse does) when no GCMouse
/// is present the relative GCMouse path can't carry pointer motion. Main-queue.
public var hasGCMouse: Bool { !mice.isEmpty }
/// Diagnostic: a one-line description of every attached GCMouse (count + GCDevice identity), so
/// PUNKTFUNK_INPUT_DEBUG can reveal whether the trackpad showed up as a mouse at all.
public var attachedMiceSummary: String {
guard !mice.isEmpty else { return "0 mice" }
let parts = mice.map { mouse -> String in
"\(mouse.productCategory)/\(mouse.vendorName ?? "?")"
}
return "\(mice.count) mice: \(parts.joined(separator: ", "))"
}
#endif
/// Fired on (the capture toggle detected here so it works in both states; the
/// event itself is swallowed). Main queue.
public var onToggleCapture: (() -> Void)?
@@ -394,6 +411,12 @@ public final class InputCapture {
!mice.contains(where: { $0 === mouse }) // re-delivered on wake attach once
else { return }
mice.append(mouse)
#if os(iOS)
if inputDebug {
inputLog.debug(
"GCMouse attached: \(mouse.productCategory, privacy: .public)/\(mouse.vendorName ?? "?", privacy: .public) — now \(self.attachedMiceSummary, privacy: .public)")
}
#endif
// macOS drives motion + buttons from NSEvent (StreamLayerView's local monitor
// sendMotion/sendMouseButton) because GCMouse's handlers proved unreliable there;
// installing them too would double-send. iOS keeps GCMouse (raw deltas under
@@ -27,10 +27,35 @@ public enum Licenses {
+ apache
}
/// The bundled brand typeface (Geist Sans + Geist Mono) SIL Open Font License 1.1. The
/// license file ships alongside the OTFs in `Resources/Fonts/`, satisfying the OFL's
/// distribution requirement; this surfaces it in the Acknowledgements screen too.
public static var fontLicense: String {
guard let url = Bundle.module.url(
forResource: "Geist-OFL", withExtension: "txt", subdirectory: "Fonts"),
let text = try? String(contentsOf: url, encoding: .utf8)
else { return "" }
return text
}
/// Third-party software notices for the linked Rust crates (generated by
/// `scripts/gen-third-party-notices.sh`).
public static var thirdPartyNotices: String {
let text = resource("THIRD-PARTY-NOTICES")
return text.isEmpty ? "Third-party notices unavailable." : text
}
/// `thirdPartyNotices` pre-split into render-sized line chunks. The full notices are ~885 KB /
/// 16k lines; a single SwiftUI `Text` that large overshoots CoreText/CoreAnimation's max
/// renderable height it lays out for ages and draws blank past the limit so the
/// Acknowledgements screen renders these chunks in a `LazyVStack` (only on-screen chunks lay
/// out, and no chunk is tall enough to clip). Split at line boundaries and joined with "\n";
/// the inter-chunk break is the `LazyVStack` row boundary, so no text is lost. Computed once.
public static let thirdPartyNoticesChunks: [String] = {
let lines = thirdPartyNotices.split(separator: "\n", omittingEmptySubsequences: false)
let chunkSize = 200
return stride(from: 0, to: lines.count, by: chunkSize).map { start in
lines[start..<min(start + chunkSize, lines.count)].joined(separator: "\n")
}
}()
}
@@ -1,21 +1,35 @@
// Stage-2 presenter, present half: draw a decoded NV12 CVPixelBuffer into a CAMetalLayer
// drawable with a BT.709 YUVRGB shader. The display link (owned by the hosting view) drives
// `render` once per vsync with the target present time, so a present can finally be stamped and
// the present tail hand-paced. See docs apple-stage2-presenter.md.
// Stage-2 presenter, present half: draw a decoded NV12 / P010 / 4:4:4 CVPixelBuffer into a CAMetalLayer
// drawable with a YCbCrRGB shader. The hosting view's CADisplayLink drives `render` once per vsync
// (via Stage2Pipeline.renderTick) with the target present time, so a present can be stamped and the
// present tail hand-paced. See docs apple-stage2-presenter.md.
//
// Main-thread only: created during view setup, `render` called from the view's CADisplayLink
// (which fires on the main runloop). The Metal objects + texture cache are touched only here.
// Main-thread only: created during view setup, `render`/`configure` called from the view's CADisplayLink
// (which fires on the main runloop). The Metal objects + texture cache are touched only here. The one
// exception is `setHdrMeta`, called from the pump thread it hops the layer write to main so every
// CALayer mutation stays on one thread.
#if canImport(Metal) && canImport(QuartzCore)
import CoreGraphics
import CoreVideo
import Metal
import QuartzCore
import os
/// Runtime-compiled (no metallib build step needed in SwiftPM): a fullscreen triangle and a
/// BT.709 limited-range NV12RGB fragment shader. uv.y is flipped (1 - p.y) so the top-left-
/// origin texture presents upright (NDC y is up), not upside down. (Colorspace is BT.709 SDR
/// for now matches the host; 10-bit/HDR + other matrices are a later tie-in.)
private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter")
/// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the
/// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors
/// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the
/// brighter highlights into the panel's headroom. This is the missing anchor that made the old HDR path
/// render "way too bright" (no `edrMetadata` no reference-white anchoring); a LARGER value renders
/// dimmer. Matches the host's standard PQ reference white.
private let hdrReferenceWhiteNits: Float = 203.0
/// Runtime-compiled (no metallib build step needed in SwiftPM): a fullscreen triangle and BT.709 SDR
/// and BT.2020-PQ HDR YCbCrRGB fragment shaders. uv.y is flipped (1 - p.y) so the top-left-origin
/// texture presents upright (NDC y is up). The HDR shader outputs PQ-encoded RGB as-is the
/// CAMetalLayer's `itur_2100_PQ` colour space + `edrMetadata` tell the system compositor the samples
/// are PQ and how to tone-map them (no EOTF here, matching the host's BT.2020 PQ emission).
private let shaderSource = """
#include <metal_stdlib>
using namespace metal;
@@ -30,11 +44,46 @@ vertex VOut pf_vtx(uint vid [[vertex_id]]) {
return o;
}
// Bicubic (Catmull-Rom) sampling of the single-channel luma plane. When the drawable is larger
// than the decoded frame (a window/view bigger than the host's fixed mode), a bilinear upscale
// looks soft; Catmull-Rom keeps edges crisp — matching AVSampleBufferDisplayLayer's (stage-1)
// scaler — and reduces to the exact texel at 1:1, so a native-resolution present stays pixel-exact.
// Nine bilinear taps (TheRealMJP's optimisation of the 16-tap kernel); `s` MUST be a linear
// sampler. Luma carries the perceived detail, so only it gets bicubic; chroma stays bilinear.
float catmullRomLuma(texture2d<float> tex, sampler s, float2 uv) {
float2 texSize = float2(tex.get_width(), tex.get_height());
float2 samplePos = uv * texSize;
float2 tc1 = floor(samplePos - 0.5) + 0.5;
float2 f = samplePos - tc1;
float2 w0 = f * (-0.5 + f * (1.0 - 0.5 * f));
float2 w1 = 1.0 + f * f * (-2.5 + 1.5 * f);
float2 w2 = f * (0.5 + f * (2.0 - 1.5 * f));
float2 w3 = f * f * (-0.5 + 0.5 * f);
float2 w12 = w1 + w2;
float2 off12 = w2 / w12;
float2 tc0 = (tc1 - 1.0) / texSize;
float2 tc3 = (tc1 + 2.0) / texSize;
float2 tc12 = (tc1 + off12) / texSize;
float r = 0.0;
r += tex.sample(s, float2(tc0.x, tc0.y)).r * (w0.x * w0.y);
r += tex.sample(s, float2(tc12.x, tc0.y)).r * (w12.x * w0.y);
r += tex.sample(s, float2(tc3.x, tc0.y)).r * (w3.x * w0.y);
r += tex.sample(s, float2(tc0.x, tc12.y)).r * (w0.x * w12.y);
r += tex.sample(s, float2(tc12.x, tc12.y)).r * (w12.x * w12.y);
r += tex.sample(s, float2(tc3.x, tc12.y)).r * (w3.x * w12.y);
r += tex.sample(s, float2(tc0.x, tc3.y)).r * (w0.x * w3.y);
r += tex.sample(s, float2(tc12.x, tc3.y)).r * (w12.x * w3.y);
r += tex.sample(s, float2(tc3.x, tc3.y)).r * (w3.x * w3.y);
return r;
}
// SDR: 8-bit NV12 / 4:4:4 (BT.709, limited/video range) → full-range RGB. Chroma is sampled at the
// same UV as luma, so a full-size 4:4:4 chroma plane needs no shader change vs 4:2:0.
fragment float4 pf_frag(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> chromaTex [[texture(1)]]) {
constexpr sampler s(filter::linear, address::clamp_to_edge);
float y = lumaTex.sample(s, in.uv).r;
float y = catmullRomLuma(lumaTex, s, in.uv);
float2 c = chromaTex.sample(s, in.uv).rg;
// BT.709, 8-bit limited (video) range → full-range RGB.
y = (y - 16.0/255.0) * (255.0/219.0);
@@ -46,18 +95,18 @@ fragment float4 pf_frag(VOut in [[stage_in]],
return float4(saturate(float3(r, g, b)), 1.0);
}
// HDR: 10-bit P010 (BT.2020, limited range), Y'CbCr that is PQ-encoded. We apply the BT.2020
// matrix to get PQ-encoded R'G'B' and output it as-is — the CAMetalLayer's itur_2100_PQ colour
// space + EDR tells the compositor the samples are PQ, so it does the PQ→display mapping. No EOTF
// here (matching the host, which emitted BT.2020 PQ). P010 stores the 10-bit code in the high bits
// of each 16-bit sample, so an .r16Unorm sample reads ~code/1023 (the /1024 vs /1023 error is < 0.1%).
// HDR: 10-bit P010 / 4:4:4 (BT.2020, limited range), YCbCr that is PQ-encoded. We apply the BT.2020
// matrix to get PQ-encoded RGB and output it as-is — the CAMetalLayer's itur_2100_PQ colour space
// + edrMetadata tell the compositor the samples are PQ, so it does the PQ→display tone-map. No EOTF
// here. P010/x444 store the 10-bit code in the high bits of each 16-bit sample, so an .r16Unorm sample
// reads ~code/1023 (the /1024 vs /1023 error is < 0.1%).
fragment float4 pf_frag_hdr(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> chromaTex [[texture(1)]]) {
constexpr sampler s(filter::linear, address::clamp_to_edge);
float y = lumaTex.sample(s, in.uv).r;
float y = catmullRomLuma(lumaTex, s, in.uv);
float2 c = chromaTex.sample(s, in.uv).rg;
// BT.2020 10-bit limited (video) range → full-range PQ R'G'B'.
// BT.2020 10-bit limited (video) range → full-range PQ RGB.
y = (y - 64.0/1023.0) * (1023.0/876.0);
float u = (c.x - 512.0/1023.0) * (1023.0/896.0);
float v = (c.y - 512.0/1023.0) * (1023.0/896.0);
@@ -74,21 +123,34 @@ public final class MetalVideoPresenter {
private let device: MTLDevice
private let queue: MTLCommandQueue
/// SDR (BT.709 8-bit NV12 bgra8) and HDR (BT.2020 PQ 10-bit P010 rgba16Float) pipelines.
/// Selected per frame by `render`; the layer is reconfigured when the mode flips (HDR toggle).
/// SDR (BT.709 8-bit bgra8) and HDR (BT.2020 PQ 10-bit rgba16Float) pipelines. Selected per
/// frame in `render`; the layer is reconfigured to match when the session flips (HDR toggle).
private let pipelineSDR: MTLRenderPipelineState
private let pipelineHDR: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache?
/// Current layer configuration switched lazily in `configure(hdr:)` when a frame's mode differs.
private var hdrActive = false
/// nil if Metal is unavailable (no GPU / a headless CI) the caller falls back to stage-1.
public init?() {
/// Current layer configuration switched in `configure(hdr:)` when a frame's HDR-ness differs.
/// Main-thread only (read + written from `render`/`configure`, all on the display-link runloop).
private var hdrActive = false
/// Last HDR mastering grade received via `setHdrMeta` (the host's 0xCE). Cached so a mid-session
/// SDRHDR flip's `configureColor` re-applies the real grade instead of clobbering it back to the
/// bare reference-white anchor (an out-of-order race otherwise: `setHdrMeta` and the flip both write
/// `edrMetadata`). Main-thread only.
private var lastHdrMeta: PunktfunkConnection.HdrMeta?
#if DEBUG
/// Last logged "decodeddrawable" signature, so the diagnostic logs only on a size/HDR change.
private var lastSizeSig = ""
#endif
/// nil if Metal is unavailable (no GPU / a headless CI) or a shader fails to compile the caller
/// falls back to stage-1.
public static func make() -> MetalVideoPresenter? {
guard let device = MTLCreateSystemDefaultDevice(),
let queue = device.makeCommandQueue()
else { return nil }
self.device = device
self.queue = queue
let pipelineSDR: MTLRenderPipelineState
let pipelineHDR: MTLRenderPipelineState
do {
let library = try device.makeLibrary(source: shaderSource, options: nil)
let vtx = library.makeFunction(name: "pf_vtx")
@@ -105,76 +167,137 @@ public final class MetalVideoPresenter {
} catch {
return nil
}
CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, device, nil, &textureCache)
guard textureCache != nil else { return nil }
var cache: CVMetalTextureCache?
CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, device, nil, &cache)
guard let textureCache = cache else { return nil }
let layer = CAMetalLayer()
layer.device = device
layer.pixelFormat = .bgra8Unorm
layer.framebufferOnly = true
layer.isOpaque = true
// Triple-buffer: more in-flight drawables before `nextDrawable()` (called on the
// display-link / MAIN thread) has to block waiting for one to free.
layer.maximumDrawableCount = 3
#if os(macOS)
// The display link already paces exactly one present per vsync. Leaving the layer's
// own vsync wait on means `commandBuffer.present` ALSO blocks for the hardware vsync,
// so `nextDrawable()` stalls the MAIN thread until a drawable frees windowed, the
// WindowServer's looser compositing hides it; FULLSCREEN's tighter, more-direct path
// serializes the main thread to the display and the stall surfaces as bad judder.
// Disabling the layer-level sync lets present return promptly (the display link is the
// pacing source), which is what fixes the fullscreen stutter. macOS-only property.
// The display link already paces exactly one present per vsync. Leaving the layer's own vsync
// wait on means `commandBuffer.present` ALSO blocks for the hardware vsync, so `nextDrawable()`
// stalls the MAIN thread until a drawable frees windowed, the WindowServer's looser
// compositing hides it; FULLSCREEN's tighter path serializes the main thread to the display and
// the stall surfaces as bad judder. Disabling the layer-level sync lets present return promptly
// (the display link is the pacing source) the fix for the fullscreen stutter. macOS-only.
layer.displaySyncEnabled = false
#endif
// Render the drawable at the DECODED frame's resolution (set per-frame in `render`) and let the
// system compositor scale it to the layer's bounds the same `.resizeAspect` path stage-1's
// AVSampleBufferDisplayLayer uses. A native-resolution present is then pixel-exact (1:1, no
// shader scaling); a resized window rescales via the system's scaler.
layer.contentsGravity = .resizeAspect
// Triple-buffer: more in-flight drawables before `nextDrawable()` (called on the display-link /
// MAIN thread) has to block waiting for one to free.
layer.maximumDrawableCount = 3
return MetalVideoPresenter(
device: device, queue: queue, pipelineSDR: pipelineSDR, pipelineHDR: pipelineHDR,
textureCache: textureCache, layer: layer)
}
private init(
device: MTLDevice, queue: MTLCommandQueue, pipelineSDR: MTLRenderPipelineState,
pipelineHDR: MTLRenderPipelineState, textureCache: CVMetalTextureCache, layer: CAMetalLayer
) {
self.device = device
self.queue = queue
self.pipelineSDR = pipelineSDR
self.pipelineHDR = pipelineHDR
self.textureCache = textureCache
self.layer = layer
}
/// Track the stream mode (the host can Reconfigure mid-stream). Size is in pixels.
public func setDrawableSize(_ size: CGSize) {
guard size.width > 0, size.height > 0 else { return }
if layer.drawableSize != size { layer.drawableSize = size }
}
/// Reconfigure the layer for SDR or HDR when the stream mode flips (HDR toggle). HDR uses an
/// rgba16Float drawable + a BT.2020 PQ colour space + EDR, so the compositor PQ-maps to the
/// display; SDR uses the plain 8-bit sRGB path. Main-thread only (called from `render`).
private func configure(hdr: Bool) {
/// Configure the layer + active pipeline for an SDR or HDR session. MAIN THREAD ONLY. Called once at
/// session start and again per-frame from `render` (idempotent the guard makes a same-state call a
/// no-op), so a mid-session HDR toggle (the host re-inits its encoder; the decoded `frame.isHDR`
/// flips) reconfigures here automatically. HDR uses an rgba16Float drawable + BT.2020 PQ colour space
/// + EDR with a 203-nit reference-white anchor; SDR uses the plain 8-bit sRGB path.
public func configure(hdr: Bool) {
guard hdr != hdrActive else { return }
hdrActive = hdr
configureColor(hdr: hdr)
}
/// Set the layer's pixel format + colour config for SDR or HDR. MAIN THREAD ONLY. EDR is requested
/// on ALL platforms the property is available on macOS/iOS/tvOS at our deployment floor, and the
/// old `#if os(macOS)` guard left iOS/tvOS EDR half-engaged.
private func configureColor(hdr: Bool) {
if hdr {
layer.pixelFormat = .rgba16Float
layer.colorspace = CGColorSpace(name: CGColorSpace.itur_2100_PQ)
#if os(macOS)
layer.wantsExtendedDynamicRangeContent = true
#endif
// Anchor reference white. Re-apply the real grade if one already arrived (0xCE before the
// flip); otherwise the bare 203-nit anchor. Without this anchor the PQ signal is too bright.
layer.edrMetadata = makeEDR(lastHdrMeta)
} else {
// SDR: gamma-encoded BT.709 [0,1] in an 8-bit drawable; a nil colorspace tags it device/sRGB
// (the proven SDR path never showed the "too bright" issue, which was HDR-only).
layer.pixelFormat = .bgra8Unorm
layer.colorspace = nil
#if os(macOS)
layer.wantsExtendedDynamicRangeContent = false
#endif
layer.edrMetadata = nil
}
}
/// Draw one decoded frame to the next drawable and present it. `isHDR` selects the 10-bit
/// BT.2020 PQ path (P010 input) vs the 8-bit BT.709 path (NV12 input). Returns true on success;
/// false when there's no drawable yet, a texture couldn't be made, or Metal errored the
/// caller then doesn't stamp a present for this frame.
private func makeEDR(_ meta: PunktfunkConnection.HdrMeta?) -> CAEDRMetadata {
CAEDRMetadata.hdr10(
displayInfo: meta?.masteringDisplayColorVolume(),
contentInfo: meta?.contentLightLevelInfo(),
opticalOutputScale: hdrReferenceWhiteNits)
}
/// Update the HDR mastering metadata (drained from the host's 0xCE datagram) to refine the system
/// tone-map from the real grade. Called from the PUMP thread, so the layer write is hopped to MAIN
/// (every CALayer mutation stays on one thread). The grade is cached so a later SDRHDR
/// `configureColor` re-applies it; the `edrMetadata` write is gated on `hdrActive` (setting it on an
/// SDR layer is harmless but pointless, and the flip will apply it anyway).
public func setHdrMeta(_ meta: PunktfunkConnection.HdrMeta) {
DispatchQueue.main.async { [weak self] in
guard let self else { return }
self.lastHdrMeta = meta
if self.hdrActive { self.layer.edrMetadata = self.makeEDR(meta) }
}
}
/// Draw one decoded frame to the next drawable and present it. MAIN THREAD (the display link).
/// `isHDR` selects the 10-bit BT.2020 PQ path vs the 8-bit BT.709 path and is reconciled with the
/// layer config via `configure`. Returns true on success; false when there's no drawable yet, a
/// texture couldn't be made, or Metal errored the caller then doesn't stamp a present.
@discardableResult
public func render(_ pixelBuffer: CVPixelBuffer, isHDR: Bool = false) -> Bool {
// Reconcile the layer with the decoded frame's HDR-ness (handles a mid-session SDRHDR flip).
configure(hdr: isHDR)
// P010 stores 10-bit luma/chroma in 16-bit samples R16/RG16; NV12 is 8-bit R8/RG8.
let lumaFmt: MTLPixelFormat = isHDR ? .r16Unorm : .r8Unorm
let chromaFmt: MTLPixelFormat = isHDR ? .rg16Unorm : .rg8Unorm
// P010/x444 store 10-bit luma/chroma in 16-bit samples R16/RG16; NV12/444v is 8-bit R8/RG8.
// Derived from the actual decoded buffer so a 4:4:4 (full chroma plane) frame just works.
let pf = CVPixelBufferGetPixelFormatType(pixelBuffer)
let tenBit =
pf == kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange
|| pf == kCVPixelFormatType_420YpCbCr10BiPlanarFullRange
|| pf == kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange
|| pf == kCVPixelFormatType_444YpCbCr10BiPlanarFullRange
guard let textureCache,
let luma = makeTexture(pixelBuffer, plane: 0, format: lumaFmt, cache: textureCache),
let chroma = makeTexture(pixelBuffer, plane: 1, format: chromaFmt, cache: textureCache)
let luma = makeTexture(
pixelBuffer, plane: 0, format: tenBit ? .r16Unorm : .r8Unorm, cache: textureCache),
let chroma = makeTexture(
pixelBuffer, plane: 1, format: tenBit ? .rg16Unorm : .rg8Unorm, cache: textureCache)
else { return false }
// The hosting view owns drawableSize (aspect-fit to its bounds); skip until it's laid
// out. The fullscreen triangle scales the decoded texture to fill the drawable.
guard layer.drawableSize.width > 0, layer.drawableSize.height > 0,
let drawable = layer.nextDrawable(),
// Size the drawable to the decoded frame so the fullscreen triangle samples 1:1 (pixel-exact);
// the layer's contentsGravity then scales it to the on-screen bounds via the system compositor
// (matching stage-1). drawableSize does NOT track bounds (defaults to 0), so set it BEFORE
// nextDrawable; re-set only on a change (first frame / Reconfigure / HDR flip).
let decodedSize = CGSize(
width: CVPixelBufferGetWidth(pixelBuffer), height: CVPixelBufferGetHeight(pixelBuffer))
if layer.drawableSize != decodedSize { layer.drawableSize = decodedSize }
#if DEBUG
logSizeIfChanged(decoded: decodedSize)
#endif
guard let drawable = layer.nextDrawable(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
@@ -186,24 +309,23 @@ public final class MetalVideoPresenter {
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(isHDR ? pipelineHDR : pipelineSDR)
encoder.setRenderPipelineState(hdrActive ? pipelineHDR : pipelineSDR)
encoder.setFragmentTexture(CVMetalTextureGetTexture(luma), index: 0)
encoder.setFragmentTexture(CVMetalTextureGetTexture(chroma), index: 1)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
commandBuffer.present(drawable) // present at the next vsync lowest latency
// Hold the CVMetalTextures + the source pixel buffer (its IOSurface) alive until the GPU
// finishes sampling releasing them at scope exit could free the backing mid-read.
// Hold the CVMetalTextures + source pixel buffer (its IOSurface) alive until the GPU finishes
// sampling releasing them at scope exit could free the backing mid-read.
commandBuffer.addCompletedHandler { _ in _ = (luma, chroma, pixelBuffer) }
commandBuffer.commit()
return true
}
/// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past
/// the draw the MTLTexture is only valid while its CVMetalTexture is retained.
/// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the
/// draw the MTLTexture is only valid while its CVMetalTexture is retained.
private func makeTexture(
_ pixelBuffer: CVPixelBuffer, plane: Int, format: MTLPixelFormat,
cache: CVMetalTextureCache
_ pixelBuffer: CVPixelBuffer, plane: Int, format: MTLPixelFormat, cache: CVMetalTextureCache
) -> CVMetalTexture? {
let w = CVPixelBufferGetWidthOfPlane(pixelBuffer, plane)
let h = CVPixelBufferGetHeightOfPlane(pixelBuffer, plane)
@@ -215,5 +337,16 @@ public final class MetalVideoPresenter {
else { return nil }
return cvTexture
}
#if DEBUG
private func logSizeIfChanged(decoded: CGSize) {
let sig = "\(Int(decoded.width))x\(Int(decoded.height))|hdr\(hdrActive ? 1 : 0)"
if sig != lastSizeSig {
lastSizeSig = sig
let msg = "stage2: decoded \(Int(decoded.width))x\(Int(decoded.height)) hdr=\(hdrActive)"
presenterLog.info("\(msg, privacy: .public)")
}
}
#endif
}
#endif
@@ -0,0 +1,94 @@
// Steers the system's iPad pointer-lock resolution down to a chosen "anchor" view controller.
//
// `UIViewController.prefersPointerLocked` is resolved the same way as the status bar: the system
// walks DOWN from the window's root view controller through `childViewControllerForPointerLock`.
// SwiftUI's hosting / container view controllers do NOT forward that query to their children, so a
// `UIViewControllerRepresentable` controller buried in the SwiftUI tree (our StreamViewController)
// is never consulted its `prefersPointerLocked = true` is silently ignored and a Magic Keyboard
// trackpad / mouse falls through to the absolute-pointer path instead of being captured.
//
// Swizzling the DEFAULT implementation isn't enough: the controllers that break the chain
// (UIHostingController and SwiftUI's internal containers) provide their OWN implementation of the
// property, so a base-class swizzle never runs for them. Instead we walk UP the LIVE `parent`
// chain from the anchor to the window root and, on each real ancestor, force
// `childViewControllerForPointerLock` to return the next controller toward the anchor. Each forced
// value is a genuine direct child (we follow the actual containment chain), so the system's
// downward walk reaches the anchor and reads its `prefersPointerLocked`.
//
// The forcing is per-INSTANCE an associated object gated behind a one-time per-CLASS IMP
// swizzle. Only the specific controllers in the anchor's chain are affected; every other instance
// of those classes keeps its original behavior (associated object nil original IMP). The forced
// values are cleared on disengage so the long-lived SwiftUI parents don't retain a stale controller
// across sessions. Only the PUBLIC `childViewControllerForPointerLock` selector is touched
// (App-Store-safe; no private API).
#if os(iOS)
import ObjectiveC
import UIKit
enum PointerLockChain {
private static var forcedChildKey: UInt8 = 0
/// Classes whose `childViewControllerForPointerLock` we've already IMP-swizzled (keyed by the
/// class object). Main-thread only pointer-lock resolution and capture toggles are all main.
private static var swizzledClasses = Set<ObjectIdentifier>()
/// Ancestors we've stamped with a forced child this engagement, held weakly so a deallocated
/// SwiftUI controller drops out on its own (no dangling). disengage() clears every one even
/// if the live `parent` chain has since broken so a stamped parent can never retain a stale
/// controller subtree across sessions. One anchor is ever engaged at a time.
private static let stampedParents = NSHashTable<UIViewController>.weakObjects()
private static func forcedChild(of vc: UIViewController) -> UIViewController? {
objc_getAssociatedObject(vc, &forcedChildKey) as? UIViewController
}
private static func setForcedChild(_ child: UIViewController?, on vc: UIViewController) {
// RETAIN: while steering, the parent must keep the toward-anchor child alive. It's also
// already a strong child of `vc` via UIKit containment, so this adds no cycle (the reverse
// `.parent` link is weak), and disengage() always clears it so it can't outlive a session.
objc_setAssociatedObject(vc, &forcedChildKey, child, .OBJC_ASSOCIATION_RETAIN_NONATOMIC)
}
/// Ensure `cls`'s `childViewControllerForPointerLock` getter consults the per-instance forced
/// child first, falling back to the class's original implementation. Idempotent per class.
private static func ensureSwizzled(_ cls: AnyClass) {
let id = ObjectIdentifier(cls)
guard !swizzledClasses.contains(id) else { return }
swizzledClasses.insert(id)
let selector = #selector(getter: UIViewController.childViewControllerForPointerLock)
guard let method = class_getInstanceMethod(cls, selector) else { return }
let originalIMP = method_getImplementation(method)
typealias OriginalFn = @convention(c) (AnyObject, Selector) -> UIViewController?
let original = unsafeBitCast(originalIMP, to: OriginalFn.self)
let forwarding: @convention(block) (UIViewController) -> UIViewController? = { vc in
if let forced = forcedChild(of: vc) { return forced }
return original(vc, selector)
}
method_setImplementation(method, imp_implementationWithBlock(forwarding))
}
/// Force every ancestor of `anchor` to forward pointer-lock resolution toward it, then ask the
/// system to re-resolve. No-op when `anchor` isn't in a view-controller hierarchy yet (it
/// re-runs from the anchor's appearance/parent callbacks once it is).
static func engage(_ anchor: UIViewController) {
disengage(anchor) // clear any prior engagement first (reparent / re-anchor)
var child = anchor
while let parent = child.parent {
ensureSwizzled(object_getClass(parent)!)
setForcedChild(child, on: parent)
stampedParents.add(parent)
child = parent
}
anchor.setNeedsUpdateOfPrefersPointerLocked()
}
/// Clear the forced forwarding on every stamped ancestor (so the SwiftUI parents stop retaining
/// the anchor's subtree) and re-resolve to drop the lock.
static func disengage(_ anchor: UIViewController) {
for parent in stampedParents.allObjects {
setForcedChild(nil, on: parent)
}
stampedParents.removeAllObjects()
anchor.setNeedsUpdateOfPrefersPointerLocked()
}
}
#endif
@@ -0,0 +1,36 @@
// Synthetic 4:4:4 HEVC keyframes used only by `Stage444Probe` to probe decode capability.
//
// Each is the first IDR access unit (VPS + SPS + PPS + IDR slice, Annex-B) of a 256×256 HEVC
// Range-Extensions clip `chroma_format_idc = 3` generated offline with libx265:
// ffmpeg -f lavfi -i color=c=gray:s=256x256:r=30:d=0.1 -frames:v 3 \
// -pix_fmt yuv444p[10le] -c:v libx265 \
// -x265-params keyint=1:min-keyint=1:no-info=1:repeat-headers=1:aud=0 -f hevc out.hevc
// 256×256 clears the hardware decoder's minimum-dimension floor (a 16×16 clip is rejected for every
// chroma format). Validated to hardware-decode to `444v`/`x444` on Apple Silicon (M3).
enum Probe444Blobs {
/// 256×256 HEVC Range-Extensions 4:4:4 keyframe (Annex-B): 134 bytes.
static let au444_8bit: [UInt8] = [
0x00, 0x00, 0x00, 0x01, 0x40, 0x01, 0x0c, 0x01, 0xff, 0xff, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00,
0x9e, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c, 0xba, 0x02, 0x40, 0x00, 0x00, 0x00, 0x01, 0x42,
0x01, 0x01, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00, 0x9e, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c,
0x90, 0x01, 0x01, 0x00, 0x80, 0xb2, 0xdd, 0x49, 0x26, 0x57, 0x80, 0xb4, 0x04, 0x00, 0x00, 0x03,
0x00, 0x04, 0x00, 0x00, 0x03, 0x00, 0x78, 0x20, 0x00, 0x00, 0x00, 0x01, 0x44, 0x01, 0xc1, 0x72,
0x86, 0x0c, 0x06, 0x24, 0x00, 0x00, 0x00, 0x01, 0x28, 0x01, 0xaf, 0x72, 0x15, 0xe8, 0x34, 0xeb,
0xae, 0xfb, 0xfe, 0x75, 0x57, 0xca, 0xc1, 0x71, 0x43, 0x16, 0xf5, 0xc2, 0x40, 0xbd, 0x80, 0xa6,
0x65, 0x35, 0x20, 0x28, 0x81, 0xa2, 0x5e, 0xc0, 0x93, 0x04, 0x10, 0x9b, 0x00, 0x34, 0xe0, 0x87,
0x00, 0x00, 0x03, 0x00, 0x5b, 0x40,
]
/// 256×256 HEVC Range-Extensions 4:4:4 10-bit keyframe (Annex-B): 133 bytes.
static let au444_10bit: [UInt8] = [
0x00, 0x00, 0x00, 0x01, 0x40, 0x01, 0x0c, 0x01, 0xff, 0xff, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00,
0x9c, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c, 0xba, 0x02, 0x40, 0x00, 0x00, 0x00, 0x01, 0x42,
0x01, 0x01, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00, 0x9c, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c,
0x90, 0x01, 0x01, 0x00, 0x80, 0x9b, 0x2d, 0xd4, 0x92, 0x65, 0x78, 0x0b, 0x40, 0x40, 0x00, 0x00,
0x03, 0x00, 0x40, 0x00, 0x00, 0x07, 0x82, 0x00, 0x00, 0x00, 0x01, 0x44, 0x01, 0xc1, 0x72, 0x86,
0x0c, 0x06, 0x24, 0x00, 0x00, 0x00, 0x01, 0x28, 0x01, 0xaf, 0x72, 0x15, 0xe8, 0x34, 0xeb, 0xae,
0xfb, 0xfe, 0x75, 0x57, 0xca, 0xc1, 0x71, 0x43, 0x16, 0xf5, 0xc2, 0x40, 0xbd, 0x80, 0xa6, 0x65,
0x35, 0x20, 0x28, 0x81, 0xa2, 0x5e, 0xc0, 0x93, 0x04, 0x10, 0x9b, 0x00, 0x34, 0xe0, 0x87, 0x00,
0x00, 0x03, 0x00, 0x5b, 0x40,
]
}
@@ -182,6 +182,11 @@ public final class PunktfunkConnection {
case dualSense = 2
case xboxOne = 3
case dualShock4 = 4
// Valve Steam Controller / Steam Deck (Linux UHID hid-steam hosts). Parity only on Apple
// GameController never surfaces a 0x28DE HID device, so the client can't capture one; these
// exist so the resolved type round-trips and name parsing matches the host.
case steamController = 5
case steamDeck = 6
/// Loose name parsing for env/dev hooks, mirroring the host's
/// `GamepadPref::from_name`.
@@ -192,6 +197,8 @@ public final class PunktfunkConnection {
case "dualsense", "ds", "ds5", "ps5": self = .dualSense
case "xboxone", "xbox-one", "xboxseries", "series": self = .xboxOne
case "dualshock4", "dualshock", "ds4", "ps4": self = .dualShock4
case "steamdeck", "steam-deck", "deck": self = .steamDeck
case "steamcontroller", "steam-controller", "steamcon": self = .steamController
default: return nil
}
}
@@ -231,6 +238,13 @@ public final class PunktfunkConnection {
public private(set) var colorFullRange: Bool = false
/// Encoded bit depth (8 or 10).
public private(set) var bitDepth: UInt8 = 8
/// The chroma subsampling the host resolved for this session, as the HEVC `chroma_format_idc`:
/// `1` = 4:2:0 (every pre-4:4:4 host, and the back-compat default) or `3` = full-chroma 4:4:4
/// (only when this client advertised `videoCap444` *and* the host could open a real 4:4:4
/// encoder). Drive the decoder's requested pixel format from this. See `isChroma444`.
public private(set) var chromaFormat: UInt8 = 1
/// Convenience: the resolved stream is full-chroma 4:4:4 (`chroma_format_idc == 3`).
public var isChroma444: Bool { chromaFormat == 3 }
/// True when the negotiated stream is HDR (PQ or HLG transfer) drive an HDR present path and
/// drain `nextHdrMeta`.
public var isHDR: Bool { colorTransfer == 16 || colorTransfer == 18 }
@@ -327,6 +341,9 @@ public final class PunktfunkConnection {
colorMatrix = mtx
colorFullRange = fullRange != 0
bitDepth = depth
var cf: UInt8 = 1
_ = punktfunk_connection_chroma_format(handle, &cf)
chromaFormat = cf
var ac: UInt8 = 2
_ = punktfunk_connection_audio_channels(handle, &ac)
resolvedAudioChannels = ac
@@ -598,6 +615,10 @@ public final class PunktfunkConnection {
public static let videoCap10Bit: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_10BIT)
/// Video-capability bit: the client can present BT.2020 PQ HDR10 (implies 10-bit).
public static let videoCapHDR: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_HDR)
/// Video-capability bit: the client can decode a full-chroma 4:4:4 HEVC stream (Range
/// Extensions). Advertise only when the device can *hardware*-decode it (`Stage444Probe`);
/// the host then emits 4:4:4 only if it too opted in. `chromaFormat` reflects the real value.
public static let videoCap444: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_444)
/// Static HDR mastering metadata (SMPTE ST.2086 + content light level) the host sent for an HDR
/// session. Mirrors the wire/ABI `PunktfunkHdrMeta`; primaries are in ST.2086 **G, B, R** order,
@@ -0,0 +1,93 @@
Copyright 2024 The Geist Project Authors (https://github.com/vercel/geist-font)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
https://openfontlicense.org
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
@@ -177,6 +177,16 @@ public final class SessionAudio {
private var playbackEngine: AVAudioEngine?
private var captureEngine: AVAudioEngine?
private var drainStarted = false
#if !os(macOS)
/// AVAudioSession `setCategory`/`setActive` are synchronous and block on the audio server, so
/// they must not run on the main thread (UI stall AVFoundation warns about it). PROCESS-WIDE
/// (static) so every SessionAudio shares one serial queue: the AVAudioSession is a process
/// singleton, and across a reconnect the old session's deactivate must be ordered before the
/// new session's activate (a per-instance queue would let them race and leave the new session's
/// audio deactivated). stop() enqueues its deactivate promptly so it lands before the next
/// session's activate.
private static let sessionQueue = DispatchQueue(label: "io.unom.punktfunk.audio.session")
#endif
public init(connection: PunktfunkConnection) {
self.connection = connection
@@ -189,37 +199,60 @@ public final class SessionAudio {
flag.stop()
}
/// Start playback (and, if enabled+authorized, the mic uplink). Empty UIDs = system
/// default device; on iOS the UIDs are ignored entirely (routes are
/// AVAudioSession-managed). Main thread (engine setup); returns after the engines
/// start the mic may start slightly later if the permission prompt is pending.
/// Start playback (and, if enabled+authorized, the mic uplink). Empty UIDs = system default
/// device; on iOS the UIDs are ignored entirely (routes are AVAudioSession-managed). On macOS
/// the engines start synchronously on the caller's (main) thread. On iOS/tvOS start() is
/// ASYNCHRONOUS: it activates the AVAudioSession off the main thread, then starts the engines on
/// a later main-queue hop (gated by `!flag.isStopped`) so playback is live shortly after, not
/// on return. The mic may start later still if the permission prompt is pending.
public func start(speakerUID: String, micUID: String, micEnabled: Bool) {
#if os(iOS)
// Route + policy live in the session, not per-engine: stereo playback, mic
// capture when enabled, Bluetooth allowed. Failure is non-fatal (defaults).
#if os(macOS)
// No AVAudioSession on macOS start the engines directly (caller's thread, as before).
startEngines(speakerUID: speakerUID, micUID: micUID, micEnabled: micEnabled)
#else
// Configure + activate the session OFF the main thread (it blocks on the audio server),
// then start the engines back on the main thread once it's active engine routing/format
// depend on the active session. A stop() racing in between is caught by the flag guard.
Self.sessionQueue.async { [weak self] in
guard let self else { return }
self.activateAudioSession(micEnabled: micEnabled)
DispatchQueue.main.async { [weak self] in
guard let self, !self.flag.isStopped else { return }
self.startEngines(speakerUID: speakerUID, micUID: micUID, micEnabled: micEnabled)
}
}
#endif
}
#if !os(macOS)
/// Route + policy live in the session, not per-engine: stereo playback, mic capture when
/// enabled, Bluetooth allowed. Failure is non-fatal (defaults). Runs on `sessionQueue`.
private func activateAudioSession(micEnabled: Bool) {
let session = AVAudioSession.sharedInstance()
do {
#if os(iOS)
if micEnabled {
// .defaultToSpeaker: .playAndRecord otherwise routes to the iPhone
// EARPIECE; only affects the built-in route (headphones/BT still win).
// .defaultToSpeaker: .playAndRecord otherwise routes to the iPhone EARPIECE; only
// affects the built-in route (headphones/BT still win).
try session.setCategory(
.playAndRecord, mode: .default,
options: [.allowBluetoothA2DP, .defaultToSpeaker])
} else {
try session.setCategory(.playback, mode: .default)
}
#else // tvOS no app-accessible mic
try session.setCategory(.playback, mode: .default)
#endif
try session.setActive(true)
} catch {
log.warning("AVAudioSession setup failed: \(error.localizedDescription)")
}
#elseif os(tvOS)
do {
try AVAudioSession.sharedInstance().setCategory(.playback, mode: .default)
try AVAudioSession.sharedInstance().setActive(true)
} catch {
log.warning("AVAudioSession setup failed: \(error.localizedDescription)")
}
#endif
/// Build + start the playback engine (and the mic uplink when enabled + authorized). Main
/// thread (engine setup); on iOS/tvOS the session is already active by the time this runs.
private func startEngines(speakerUID: String, micUID: String, micEnabled: Bool) {
startPlayback(speakerUID: speakerUID)
#if os(tvOS)
// No app-accessible microphone input on tvOS playback only.
@@ -258,19 +291,24 @@ public final class SessionAudio {
capture.stop()
}
playback?.stop()
if wasDraining {
_ = drainDone.wait(timeout: .now() + .milliseconds(400))
}
#if !os(macOS)
// Release the session so audio we interrupted (Music, podcasts) gets its
// resume cue.
// Release the session so audio we interrupted (Music, podcasts) gets its resume cue. Like
// activation, setActive is synchronous/blocking run it on the shared serial session queue
// (off the main thread). Enqueued HERE engines already stopped, and BEFORE the drain wait
// below so across a reconnect it lands ahead of the next session's activate on the shared
// queue (otherwise a deferred deactivate could deactivate the new session). Fire-and-forget.
Self.sessionQueue.async {
do {
try AVAudioSession.sharedInstance().setActive(
false, options: .notifyOthersOnDeactivation)
} catch {
log.warning("AVAudioSession deactivation failed: \(error.localizedDescription)")
}
}
#endif
if wasDraining {
_ = drainDone.wait(timeout: .now() + .milliseconds(400))
}
}
// MARK: - Playback (host speaker)
@@ -1,21 +1,21 @@
// Stage-2 presenter orchestrator: a pump thread pulls AUs VideoDecoder; the decoder's async
// output drops the newest decoded frame into a 1-slot ring; the hosting view's display link
// calls `renderTick` once per vsync to draw + present the newest ready frame and stamp
// capturepresent. Mirrors StreamPump's lifecycle (one per start; cancel is permanent).
// Stage-2 presenter orchestrator: a pump thread pulls AUs VideoDecoder; the decoder's async output
// drops the newest decoded frame into a 1-slot ring; the hosting view's display link calls `renderTick`
// once per vsync to draw + present the newest ready frame and stamp capturepresent. Mirrors
// StreamPump's lifecycle (one per start; cancel is permanent).
//
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; `renderTick`
// + `setDrawableSize` + `start`/`stop` on the MAIN thread (the view's CADisplayLink fires there).
// Only the ring + decoder cross threads and both are internally locked.
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; `renderTick` +
// `start`/`stop` on the MAIN thread (the view's CADisplayLink fires there). Only the ring (lock-guarded)
// and the decoder/presenter (internally locked / main-hopped) cross threads.
#if canImport(Metal) && canImport(QuartzCore)
import AVFoundation
import Foundation
import QuartzCore
/// Weak-target wrapper for CADisplayLink. The link retains its target, so targeting a view
/// directly makes a `view link view` cycle that only `invalidate()` breaks if a teardown
/// is ever missed the view leaks and keeps ticking. This proxy holds the handler weakly, so the
/// view can deallocate and its `deinit` invalidate the link.
/// Weak-target wrapper for CADisplayLink. The link retains its target, so targeting a view directly
/// makes a `view link view` cycle that only `invalidate()` breaks if a teardown is ever missed
/// the view leaks and keeps ticking. This proxy holds the handler weakly, so the view can deallocate
/// and its `deinit` invalidate the link.
public final class DisplayLinkProxy: NSObject {
private let onTick: (CADisplayLink) -> Void
public init(_ onTick: @escaping (CADisplayLink) -> Void) { self.onTick = onTick }
@@ -44,10 +44,10 @@ private final class PumpToken: @unchecked Sendable {
func cancel() { lock.lock(); live = false; lock.unlock() }
}
/// Throttled host keyframe requests for decode recovery. The decoder's async error callback
/// (a VT thread) and the pump thread (a submit failure) both signal a wedge; this coalesces
/// them so the control stream isn't flooded while the decode stays stalled for several frames
/// until the requested IDR lands. Bound to the live connection in `start`, unbound in `stop`.
/// Throttled host keyframe requests for decode recovery. The decoder's async error callback (a VT
/// thread) and the pump thread (a submit failure) both signal a wedge; this coalesces them so the
/// control stream isn't flooded while the decode stays stalled for several frames until the requested
/// IDR lands. Bound to the live connection in `start`, unbound in `stop`.
private final class KeyframeRecovery: @unchecked Sendable {
private let lock = NSLock()
private var connection: PunktfunkConnection?
@@ -60,7 +60,7 @@ private final class KeyframeRecovery: @unchecked Sendable {
func request() {
lock.lock()
let now = DispatchTime.now().uptimeNanoseconds
let due = lastNs == 0 || now &- lastNs > 250_000_000 // 250 ms since the last request
let due = lastNs == 0 || now &- lastNs > 100_000_000 // 100 ms since the last request
if due { lastNs = now }
let conn = due ? connection : nil
lock.unlock()
@@ -76,30 +76,36 @@ public final class Stage2Pipeline {
private let recovery = KeyframeRecovery()
private var token = PumpToken()
private var offsetNs: Int64 = 0
/// Signalled when the pump thread exits, so `stop()` can join it (bounded) before `decoder.reset()`
/// otherwise a pump iteration already past its `token.isLive` check can rebuild a decode session
/// right after the reset (a brief orphan session). `pumpJoinable` is armed by `start`, consumed by
/// the first `stop` (so the idempotent second `stop`/deinit doesn't block on an already-drained
/// semaphore). start/stop are sequential lifecycle calls, so the plain flag is safe.
private let pumpStopped = DispatchSemaphore(value: 0)
private var pumpJoinable = false
/// The Metal layer the hosting view installs + sizes. nil-init fails when Metal is
/// unavailable so the caller can fall back to stage-1.
/// The Metal layer the hosting view installs + sizes.
public var layer: CAMetalLayer { presenter.layer }
/// `presentMeter` records capturepresent (the glass-to-glass term). Returns nil if Metal
/// can't be set up (headless / no GPU) caller falls back to the stage-1 presenter.
/// `presentMeter` records capturepresent (the glass-to-glass term). Returns nil if Metal can't be
/// set up (headless / no GPU) caller falls back to the stage-1 presenter.
public init?(presentMeter: LatencyMeter) {
guard let presenter = MetalVideoPresenter() else { return nil }
guard let presenter = MetalVideoPresenter.make() else { return nil }
self.presenter = presenter
self.presentMeter = presentMeter
let ring = ring
let recovery = recovery
self.decoder = VideoDecoder(
onDecoded: { ring.submit($0) },
// Async decode failure (a bad P-frame referencing a lost/corrupt IDR): the pump
// resets to re-gate on the next IDR, and we ask the host to send one now (infinite
// GOP it wouldn't otherwise come soon). Throttled in KeyframeRecovery.
// Async decode failure (a bad P-frame referencing a lost/corrupt IDR): the pump resets to
// re-gate on the next IDR, and we ask the host to send one now (infinite GOP it wouldn't
// otherwise come soon). Throttled in KeyframeRecovery.
onDecodeError: { _ in recovery.request() })
}
/// Start pulling AUs into the decoder. `onFrame` fires per AU at receipt (captureclient
/// meter, exactly as stage-1); `onSessionEnd` on close. `clockOffsetNs` (host minus client)
/// makes the present stamp cross-machine valid.
/// Start pulling AUs into the decoder. MAIN THREAD. `onFrame` fires per AU at receipt (captureclient
/// meter, exactly as stage-1); `onSessionEnd` on close. `clockOffsetNs` (host minus client) makes the
/// present stamp cross-machine valid.
public func start(
connection: PunktfunkConnection,
onFrame: (@Sendable (AccessUnit) -> Void)?,
@@ -108,43 +114,70 @@ public final class Stage2Pipeline {
offsetNs = connection.clockOffsetNs
recovery.bind(connection) // arm host-keyframe recovery for this session
token = PumpToken() // fresh token per start cancel is permanent (like StreamPump)
// Configure the decoder's chroma + the layer's initial colorimetry before the first frame. The
// chroma subsampling drives only the decode pixel format (orthogonal to HDR/depth); the HDR
// config is the Welcome's latched value, which a mid-session flip then overrides per-frame.
decoder.setChroma444(connection.isChroma444)
presenter.configure(hdr: connection.isHDR)
let token = token
let decoder = decoder
let recovery = recovery
let presenter = presenter
let pumpStopped = pumpStopped
let thread = Thread {
defer { pumpStopped.signal() } // let stop() join the pump (bounded) before decoder.reset()
var format: CMVideoFormatDescription?
var lastFramesDropped = connection.framesDropped()
// Persistent recovery WANT, not a one-shot edge (see StreamPump for the full rationale):
// keep asking until an IDR lands so a request swallowed by the throttle is re-sent.
var awaitingIDR = false
// 4:4:4 backstop: a run of decode/create failures in a 4:4:4 session means this device can't
// decode 4:4:4 at the negotiated resolution (the HW probe clears the common case but not a
// resolution-ceiling miss). End cleanly instead of looping on a black screen.
var decodeFailRun = 0
while token.isLive {
do {
// Loss recovery (the primary recovery path). The reassembler drops unrecoverable
// AUs (framesDropped) and the decoder then conceals the reference-missing delta
// frames that follow often rendering them WITHOUT an error callback so the
// onDecodeError trigger rarely fires after a real network blip. Ask the host for
// a fresh IDR whenever the drop count climbs (throttled in KeyframeRecovery).
// Polled every iteration so a total-loss drought recovers the moment packets
// resume and the reassembler counts the gap.
// Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
// decoder conceals the reference-missing deltas often WITHOUT an error callback
// so key off the drop count climbing, then keep asking (awaitingIDR) until a fresh
// IDR re-anchors decode.
let dropped = connection.framesDropped()
if dropped > lastFramesDropped {
lastFramesDropped = dropped
recovery.request()
awaitingIDR = true
}
// Drain any HDR mastering-metadata update (0xCE) and hand it to the decoder, which
// attaches it to subsequent HDR frames. Non-blocking; only HDR sessions emit these.
if connection.isHDR, let meta = try? connection.nextHdrMeta(timeoutMs: 0) {
decoder.setHdrMeta(meta)
if awaitingIDR { recovery.request() }
// Drain HDR mastering metadata (0xCE) and hand it to the PRESENTER ( CAEDRMetadata).
// Polled UNCONDITIONALLY (not gated on connection.isHDR, the fixed Welcome flag): the
// host sends 0xCE only for HDR, INCLUDING a mid-session SDRHDR transition (a game
// entering HDR the host re-inits its encoder) the Welcome flag would never reflect.
// Non-blocking; nil for an SDR stream.
if let meta = try? connection.nextHdrMeta(timeoutMs: 0) {
presenter.setHdrMeta(meta)
}
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
if let f = AnnexB.formatDescription(fromIDR: au.data) {
format = f // refreshed on every IDR (mode changes included)
awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
}
guard let f = format, token.isLive else { continue }
if !decoder.decode(au: au, format: f) {
// Submit/decoder error: drop the session and re-gate on the next IDR's
// in-band parameter sets (a delta frame can't recover) stage-1's policy
// and ask the host for that IDR now (infinite GOP; throttled).
if decoder.decode(au: au, format: f) {
decodeFailRun = 0
} else {
// Submit/decoder error: drop the session and re-gate on the next IDR's in-band
// parameter sets (a delta frame can't recover) and keep asking for that IDR.
decoder.reset()
recovery.request()
awaitingIDR = true
decodeFailRun += 1
// ~3 s of solid failure in a 4:4:4 session (and only there a 4:2:0 loss
// recovers within a GOP) 4:4:4 isn't decodable here; end the session.
if connection.isChroma444, decodeFailRun >= 180 {
if token.isLive { onSessionEnd?() }
break
}
}
} catch {
if token.isLive { onSessionEnd?() }
@@ -154,27 +187,30 @@ public final class Stage2Pipeline {
}
thread.name = "punktfunk-stage2-pump"
thread.qualityOfService = .userInteractive
pumpJoinable = true
thread.start()
}
/// MAIN thread, once per vsync. Present the newest ready frame (if any) and stamp
/// capturepresent at `targetPresentNs` the display link's target present instant, already
/// converted to `CLOCK_REALTIME` (see `realtimeNs(forDisplayLinkTimestamp:)`).
/// MAIN thread, once per vsync. Present the newest ready frame (if any) and stamp capturepresent at
/// `targetPresentNs` the display link's target present instant, already converted to
/// `CLOCK_REALTIME` (see `realtimeNs(forDisplayLinkTimestamp:)`).
public func renderTick(targetPresentNs: Int64) {
guard let frame = ring.take() else { return }
guard presenter.render(frame.pixelBuffer, isHDR: frame.isHDR) else { return }
presentMeter.record(ptsNs: frame.ptsNs, atNs: targetPresentNs, offsetNs: offsetNs)
}
/// MAIN thread. Keep the drawable matched to the negotiated mode (host can Reconfigure).
public func setDrawableSize(_ size: CGSize) {
presenter.setDrawableSize(size)
}
/// Stop the pump ( one poll timeout) and drop the decode session. Does not close the
/// connection. A restart needs a fresh Stage2Pipeline (cancel is permanent).
/// Stop the pump ( one poll timeout) and drop the decode session. MAIN THREAD; idempotent. Does not
/// close the connection. A restart needs a fresh Stage2Pipeline (cancel is permanent).
public func stop() {
token.cancel()
// Join the pump (bounded: one nextAU poll + an in-flight decode) before resetting the decoder,
// so the pump can't rebuild a session right after the reset. Only the first stop joins; a
// repeat/deinit stop skips the already-drained semaphore.
if pumpJoinable {
pumpJoinable = false
_ = pumpStopped.wait(timeout: .now() + 0.5)
}
decoder.reset()
recovery.bind(nil) // stop requesting keyframes once the session is torn down
}
@@ -182,8 +218,8 @@ public final class Stage2Pipeline {
deinit { token.cancel() }
/// Convert a `CADisplayLink.targetTimestamp` (CACurrentMediaTime basis) to a `CLOCK_REALTIME`
/// nanosecond instant the present clock the AU pts + skew offset live in. Projects to the
/// target present time (when the frame is actually on glass), not the moment we drew.
/// nanosecond instant the present clock the AU pts + skew offset live in. Projects to the target
/// present time (when the frame is actually on glass), not the moment we drew.
public static func realtimeNs(forDisplayLinkTimestamp t: CFTimeInterval) -> Int64 {
let caNow = CACurrentMediaTime()
var ts = timespec()
@@ -0,0 +1,83 @@
// Runtime 4:4:4 HEVC decode-capability probe.
//
// We advertise `VIDEO_CAP_444` (so the host upgrades to a full-chroma 4:4:4 stream) ONLY when this
// device can decode 4:4:4 HEVC *in hardware* software 4:4:4 decode works but is far too slow for a
// real-time stream at the negotiated resolution, so a software-only device must keep 4:2:0.
//
// `VTIsHardwareDecodeSupported(HEVC)` and the HEVC-decoder-capabilities dictionary report HEVC HW
// decode but expose nothing about `chroma_format_idc`, so the only reliable signal is to actually
// create a *hardware-required* `VTDecompressionSession` for a tiny synthetic 4:4:4 keyframe and
// confirm it both creates and decodes to the expected biplanar 4:4:4 pixel format. Validated on an
// Apple M3 (HW 4:4:4 8- and 10-bit decode to `444v`/`x444`); a software-only decoder fails the
// hardware-required create and we fall back to 4:2:0.
//
// The probe blobs are 256×256 (above the hardware decoder's minimum-dimension floor a 16×16 clip
// is rejected for ALL chroma formats, including 4:2:0) HEVC Range-Extensions keyframes generated
// offline with libx265; see scripts notes. Results are cached (device-static) in lazy statics.
import CoreMedia
import CoreVideo
import Foundation
import VideoToolbox
public enum Stage444Probe {
/// True iff this device hardware-decodes 8-bit 4:4:4 HEVC (the host's current 4:4:4 path
/// BT.709 limited `yuv444p`). Cached after first evaluation.
public static let hwDecode444_8bit: Bool = probeHardware444(
au: Probe444Blobs.au444_8bit,
want: kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange,
fullRangeSibling: kCVPixelFormatType_444YpCbCr8BiPlanarFullRange)
/// True iff this device hardware-decodes 10-bit 4:4:4 HEVC (the 4:4:4 HDR/10-bit intersection).
/// Cached after first evaluation.
public static let hwDecode444_10bit: Bool = probeHardware444(
au: Probe444Blobs.au444_10bit,
want: kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange,
fullRangeSibling: kCVPixelFormatType_444YpCbCr10BiPlanarFullRange)
/// Create a hardware-REQUIRED `VTDecompressionSession` for the synthetic 4:4:4 keyframe and
/// decode it, returning true only when the decoder produces the expected (video- or full-range)
/// biplanar 4:4:4 pixel format. Any failure (no hardware path, wrong output format, decode error)
/// false we keep 4:2:0.
private static func probeHardware444(
au auBytes: [UInt8], want: OSType, fullRangeSibling: OSType
) -> Bool {
let data = Data(auBytes)
guard let format = AnnexB.formatDescription(fromIDR: data) else { return false }
// Require a hardware decoder a software false-positive would make us advertise 4:4:4 and
// then decode every real frame on the CPU, blowing the latency budget.
let spec: [CFString: Any] = [
kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder: true,
]
let attrs: [CFString: Any] = [
kCVPixelBufferPixelFormatTypeKey: want,
kCVPixelBufferMetalCompatibilityKey: true,
]
var session: VTDecompressionSession?
let created = VTDecompressionSessionCreate(
allocator: kCFAllocatorDefault, formatDescription: format,
decoderSpecification: spec as CFDictionary, imageBufferAttributes: attrs as CFDictionary,
outputCallback: nil, decompressionSessionOut: &session)
guard created == noErr, let session else { return false }
defer { VTDecompressionSessionInvalidate(session) }
let au = AccessUnit(data: data, ptsNs: 0, frameIndex: 0, flags: 0)
guard let sample = AnnexB.sampleBuffer(au: au, format: format) else { return false }
var produced: OSType = 0
let done = DispatchSemaphore(value: 0)
let status = VTDecompressionSessionDecodeFrame(
session, sampleBuffer: sample,
flags: [._EnableAsynchronousDecompression], infoFlagsOut: nil
) { status, _, imageBuffer, _, _ in
if status == noErr, let imageBuffer {
produced = CVPixelBufferGetPixelFormatType(imageBuffer)
}
done.signal()
}
guard status == noErr else { return false }
VTDecompressionSessionWaitForAsynchronousFrames(session)
_ = done.wait(timeout: .now() + 1.0)
return produced == want || produced == fullRangeSibling
}
}
@@ -6,6 +6,9 @@
import AVFoundation
import Foundation
import os
private let pumpLog = Logger(subsystem: "io.unom.punktfunk", category: "video")
/// Cancellation handle owned by exactly one pump thread a restart hands the old pump
/// its own token, so it can never be revived by a newer start().
@@ -47,44 +50,74 @@ final class StreamPump {
var format: CMVideoFormatDescription?
var lastKeyframeRequest = Date.distantPast
var lastFramesDropped = connection.framesDropped()
// Coalesced host keyframe request: the decode stays wedged for several frames until
// the IDR lands, so requesting on every frame would flood the control stream.
// Recovery is a persistent WANT, not a one-shot edge: set it on detected loss (or a
// decoder reset), retry the throttled request EVERY iteration, and clear it only when a
// fresh IDR actually re-anchors decode. The old code advanced `lastFramesDropped` on the
// same edge it fired the throttled request so a request swallowed by the throttle (a
// second drop within the window, e.g. the lost recovery IDR itself being pruned) was
// never re-sent: the counter went flat, the climb never re-fired, and the picture stayed
// frozen for good while audio kept playing. The iPhone's lossy Wi-Fi hits this where the
// Mac's Ethernet never does.
var awaitingIDR = false
var awaitingSince = Date.distantPast // when the current recovery began (for the resume log)
var wasFailed = false
// Coalesced host keyframe request. 100 ms throttle (matches the working Android path):
// fast enough that a lost recovery IDR is re-requested promptly, bounded so a sustained
// freeze can't flood the control stream.
func requestKeyframeThrottled() {
let now = Date()
if now.timeIntervalSince(lastKeyframeRequest) > 0.25 {
if now.timeIntervalSince(lastKeyframeRequest) > 0.1 {
connection.requestKeyframe()
lastKeyframeRequest = now
}
}
while token.isLive {
do {
// Loss recovery (the primary recovery path). Under the host's infinite GOP the
// only recovery keyframe is one we request. The reassembler drops unrecoverable
// AUs (framesDropped); the decoder then *conceals* the reference-missing delta
// frames that follow a frozen / garbage picture, WITHOUT flipping the layer to
// .failed so the .failed check below rarely fires after a real network blip.
// Ask the host for a fresh IDR whenever the drop count climbs. Polled every
// iteration (not just per AU) so a total-loss drought still recovers the moment
// packets resume and the reassembler counts the gap.
// Loss recovery (the primary path). Under the host's infinite GOP the only
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs
// (framesDropped); the decoder then *conceals* the reference-missing deltas a
// frozen / garbage picture that never flips the layer to .failed so key off the
// drop count climbing, then keep asking (awaitingIDR) until an IDR lands. Polled
// every iteration so a total-loss drought still recovers when packets resume.
let dropped = connection.framesDropped()
if dropped > lastFramesDropped {
lastFramesDropped = dropped
requestKeyframeThrottled()
// Log only on the falsetrue transition (once per recovery cycle), not per
// dropped AU, so heavy loss doesn't spam the log.
if !awaitingIDR {
awaitingSince = Date()
pumpLog.notice(
"video: unrecoverable drop (framesDropped=\(dropped, privacy: .public)) — requesting recovery IDR")
}
lastFramesDropped = dropped
awaitingIDR = true
}
if awaitingIDR { requestKeyframeThrottled() }
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
if let f = AnnexB.formatDescription(fromIDR: au.data) {
let idrFormat = AnnexB.formatDescription(fromIDR: au.data)
if let f = idrFormat {
format = f // refreshed on every IDR (mode changes included)
if awaitingIDR {
let ms = Int(Date().timeIntervalSince(awaitingSince) * 1000)
pumpLog.notice("video: recovery IDR received — resumed after \(ms, privacy: .public) ms")
}
if layer.status == .failed {
awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
}
let failed = layer.status == .failed
if failed {
// Decode wedged hard (the cold-first-connect case a lost/corrupt opening
// IDR): flush and re-gate on the next in-band parameter sets (resuming with
// a delta frame can't recover), AND ask the host for a fresh IDR. Throttled:
// the layer stays .failed across several polls until the IDR lands.
// IDR): flush and, unless THIS AU is the recovering IDR (re-anchored above),
// re-gate on the next in-band parameter sets and keep asking enqueuing a
// delta into a failed layer can't recover it.
if !wasFailed { pumpLog.warning("video: display layer .failed — flushing + re-anchoring") }
layer.flush()
format = AnnexB.formatDescription(fromIDR: au.data)
requestKeyframeThrottled()
if idrFormat == nil {
format = nil
awaitingIDR = true
}
}
wasFailed = failed
guard let f = format,
let sample = AnnexB.sampleBuffer(au: au, format: f),
token.isLive // don't enqueue a stale frame after a restart
@@ -137,8 +137,8 @@ public struct StreamView: NSViewRepresentable {
public final class StreamLayerView: NSView {
private let displayLayer = AVSampleBufferDisplayLayer()
private var pump: StreamPump?
/// Stage-2 presenter (opt-in via `punktfunk.presenter`): a CAMetalLayer sublayer driven by a
/// display link instead of the StreamPump displayLayer path. nil = stage-1 (default).
/// Stage-2 presenter (default): a CAMetalLayer sublayer driven by a display link instead of the
/// StreamPump displayLayer path. nil = stage-1 (Metal-unavailable fallback / DEBUG toggle).
var presentMeter: LatencyMeter?
private var stage2: Stage2Pipeline?
private var stage2Link: CADisplayLink?
@@ -245,6 +245,15 @@ public final class StreamLayerView: NSView {
layoutMetalLayer() // keep the stage-2 sublayer aspect-fit to the view
}
public override func setFrameSize(_ newSize: NSSize) {
super.setFrameSize(newSize)
// `layout()` isn't guaranteed on a manual-frame (no-Auto-Layout) live resize, so the
// stage-2 metal sublayer's drawableSize could stay at the old size while the view grows
// the compositor then upscales a too-small drawable and the video turns blocky. Resize the
// drawable here too so it always tracks the window's pixel size (no stale upscale).
layoutMetalLayer()
}
// MARK: - Capture state machine
/// Clicking into the video engages capture; that click is local (engagement), so
@@ -549,10 +558,17 @@ public final class StreamLayerView: NSView {
cursorVisible = false
_ = connection.resolvedCompositor // (was: Auto gamescope; kept to document intent)
// Presenter choice default stage-1 (the known-good AVSampleBufferDisplayLayer). Stage-2
// (`punktfunk.presenter == "stage2"`) takes explicit VTDecompressionSession decode + a
// CAMetalLayer/display-link present; it falls back here if Metal can't be set up.
if UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage2",
// Presenter choice stage-2 is the DEFAULT (explicit VTDecompressionSession decode + a
// CAMetalLayer/display-link present): it can detect + recover a wedged decoder where
// stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference. Stage-1 is
// reachable only via the DEBUG presenter toggle; release always takes stage-2 (the stage-1
// pump below stays the automatic fallback if Metal is missing).
#if DEBUG
let forceStage1 = UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage1"
#else
let forceStage1 = false
#endif
if !forceStage1,
let meter = presentMeter,
let pipeline = Stage2Pipeline(presentMeter: meter) {
startStage2(pipeline, connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
@@ -593,9 +609,11 @@ public final class StreamLayerView: NSView {
targetPresentNs: Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: link.targetTimestamp))
}
/// Aspect-fit the metal sublayer in the view (the host streams at the client's native mode,
/// so this is usually the full bounds; it letterboxes a resized window). drawableSize is the
/// layer's pixel size the fullscreen-triangle shader scales the decoded texture to fill it.
/// Position the metal sublayer aspect-fit in the view (the host streams at the client's native
/// mode, so this is usually the full bounds; it letterboxes a resized window). Only the layer
/// FRAME is set here the presenter sizes the drawable to the decoded frame and the layer's
/// contentsGravity (.resizeAspect) scales it to this frame via the system compositor, so a
/// resized window rescales through the system's filter (matching stage-1) instead of the shader.
private func layoutMetalLayer() {
guard let metalLayer, let connection else { return }
let mode = connection.currentMode()
@@ -604,14 +622,12 @@ public final class StreamLayerView: NSView {
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds)
: bounds
let scale = window?.backingScaleFactor ?? 1
// No implicit resize animation; refresh contentsScale on a retinanon-retina move.
CATransaction.begin()
CATransaction.setDisableActions(true)
metalLayer.contentsScale = scale
metalLayer.contentsScale = window?.backingScaleFactor ?? 1
metalLayer.frame = fit
CATransaction.commit()
stage2?.setDrawableSize(CGSize(width: fit.width * scale, height: fit.height * scale))
}
public override func viewDidChangeBackingProperties() {
@@ -622,7 +638,7 @@ public final class StreamLayerView: NSView {
private func teardownStage2() {
stage2Link?.invalidate()
stage2Link = nil
stage2?.stop()
stage2?.stop() // stops the pump (synchronous join) + drops the decode session
stage2 = nil
metalLayer?.removeFromSuperlayer()
metalLayer = nil
@@ -92,8 +92,8 @@ public final class StreamViewController: UIViewController {
public private(set) var connection: PunktfunkConnection?
private var pump: StreamPump?
private var observers: [NSObjectProtocol] = []
/// Stage-2 presenter (opt-in via `punktfunk.presenter`): a CAMetalLayer sublayer driven by a
/// CADisplayLink instead of the StreamPump displayLayer path. nil = stage-1 (default).
/// Stage-2 presenter (default): a CAMetalLayer sublayer driven by a CADisplayLink instead of the
/// StreamPump displayLayer path. nil = stage-1 (Metal-unavailable fallback / DEBUG toggle).
var presentMeter: LatencyMeter?
private var stage2: Stage2Pipeline?
private var stage2Link: CADisplayLink?
@@ -136,6 +136,13 @@ public final class StreamViewController: UIViewController {
public override func loadView() {
view = StreamLayerUIView()
// Re-size the stage-2 drawable if the display scale changes without a bounds change (e.g.
// moving to an external display at a different scale) the iOS analogue of macOS's
// viewDidChangeBackingProperties relayout. The handler takes the VC as its argument, so it
// doesn't capture self (no retain cycle with the registration).
registerForTraitChanges([UITraitDisplayScale.self]) { (vc: StreamViewController, _) in
vc.layoutMetalLayer()
}
#if os(iOS)
// Hide the iPadOS cursor while it hovers the video: the host renders its own
// cursor from our deltas, so the local one only diverges from it. This hides the
@@ -148,19 +155,58 @@ public final class StreamViewController: UIViewController {
}
#if os(iOS)
// Pointer lock is only meaningful on iPad (iPhone has no hardware-pointer lock) and
// only when capture is engaged. The system additionally requires full-screen + frontmost
// and may drop it (Slide Over/Stage Manager/backgrounding) verified in setCaptured().
public override var prefersPointerLocked: Bool {
captured && UIDevice.current.userInterfaceIdiom == .pad
/// Whether the user wants the mouse/trackpad pointer CAPTURED (pointer lock relative
/// movement, the gaming default) rather than forwarded as an absolute position (desktop
/// use). Read live from UserDefaults so it tracks the Settings toggle; defaults to on when
/// unset. iPad-only gated again in `prefersPointerLocked`.
private var pointerCaptureEnabled: Bool {
UserDefaults.standard.object(forKey: DefaultsKey.pointerCapture) as? Bool ?? true
}
/// Whether the pointer should be CAPTURED right now: iPad, capture engaged, and the user
/// hasn't opted into the absolute (desktop) pointer. The system additionally requires
/// full-screen + frontmost and may drop the lock (Slide Over/Stage Manager/backgrounding)
/// syncPointerLock() handles the actual grant/drop and falls back to absolute when unlocked.
private var wantsPointerLock: Bool {
captured && pointerCaptureEnabled && UIDevice.current.userInterfaceIdiom == .pad
}
public override var prefersPointerLocked: Bool { wantsPointerLock }
public override var prefersHomeIndicatorAutoHidden: Bool { true }
// If SwiftUI's UIHostingController reparents us, a plain container parent that forwards
// its pointer-lock decision to its children will then reach this VC. (UIHostingController
// itself does not consult children, which is why GCMouse deltas can never arrive there
// the touch path, always forwarded, is the unconditional fallback.)
public override var childViewControllerForPointerLock: UIViewController? { self }
// NOTE: we deliberately do NOT override `childViewControllerForPointerLock`. The default
// returns nil, which tells the system to use THIS controller's own `prefersPointerLocked`
// exactly what we want, since `PointerLockChain` forces our SwiftUI ancestors to forward the
// downward walk to us and we are the terminal anchor. Returning `self` here would make the
// system ask the same controller forever (it keeps delegating to the returned child)
// unbounded recursion stack overflow once the chain actually reaches us.
/// (Re)build or tear down the forced pointer-lock forwarding chain from this controller to the
/// window root so the system actually resolves our `prefersPointerLocked`. Safe to call
/// repeatedly it no-ops until the view is in a window with a parent chain, and re-runs from
/// the appearance/parent callbacks once SwiftUI has placed us.
private func updatePointerLockChain() {
// Engaging needs a live parent chain to the window root; disengaging is always safe and
// must run even after the view has left the window (session teardown) so the stamped
// SwiftUI ancestors are cleared.
if wantsPointerLock, view.window != nil {
PointerLockChain.engage(self)
} else {
PointerLockChain.disengage(self)
}
}
public override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
// SwiftUI places us in the hierarchy AFTER start()'s setCaptured(true), and may reparent us
// later re-anchor the chain here so a lock requested before we had a parent still lands.
updatePointerLockChain()
}
public override func didMove(toParent parent: UIViewController?) {
super.didMove(toParent: parent)
updatePointerLockChain() // chain shape changed re-anchor (or no-op if not yet in a window)
}
#endif
func start(
@@ -193,7 +239,14 @@ public final class StreamViewController: UIViewController {
// Indirect pointer (mouse/trackpad with no lock) absolute cursor + buttons, routed
// through InputCapture so the forwarding gate and release-on-blur apply uniformly.
streamView.onPointerMoveAbs = { [weak self] p in
self?.inputCapture?.sendMouseAbs(
guard let self else { return }
if iosInputDebug {
// Whether ANY UIKit pointer movement reaches us while the scene is LOCKED tells us
// if the trackpad (which may not be a GCMouse) can still be captured via UIKit.
iosInputLog.debug(
"UIKit pointer move x=\(p.x, privacy: .public) y=\(p.y, privacy: .public) locked=\(self.pointerLockEngaged() == true, privacy: .public) gcFwd=\(self.inputCapture?.gcMouseForwarding == true, privacy: .public)")
}
self.inputCapture?.sendMouseAbs(
x: p.x, y: p.y, surfaceWidth: p.w, surfaceHeight: p.h)
}
streamView.onPointerButton = { [weak self] button, down in
@@ -203,7 +256,12 @@ public final class StreamViewController: UIViewController {
// UNLOCKED regime; while locked, GCMouse's scroll handler owns it mirror the
// sendMouseAbs !gcMouseForwarding gate so the two can't double-send.
streamView.onScroll = { [weak self] dx, dy in
guard let self, self.inputCapture?.gcMouseForwarding == false else { return }
guard let self else { return }
if iosInputDebug {
iosInputLog.debug(
"UIKit scroll dx=\(dx, privacy: .public) dy=\(dy, privacy: .public) locked=\(self.pointerLockEngaged() == true, privacy: .public)")
}
guard self.inputCapture?.gcMouseForwarding == false else { return }
self.inputCapture?.sendScroll(dx: dx, dy: dy)
}
@@ -219,10 +277,17 @@ public final class StreamViewController: UIViewController {
inputCapture = capture
#endif
// Presenter choice default stage-1 (the known-good AVSampleBufferDisplayLayer). Stage-2
// (`punktfunk.presenter == "stage2"`) takes VTDecompressionSession decode + a
// CAMetalLayer/display-link present; falls back here if Metal can't be set up.
if UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage2",
// Presenter choice stage-2 is the DEFAULT (VTDecompressionSession decode + a
// CAMetalLayer/display-link present): it can detect + recover a wedged decoder, where
// stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference frame with no
// way to recover. Stage-1 is reachable only via the DEBUG presenter toggle; release always
// takes stage-2 (the stage-1 pump below stays the automatic fallback if Metal is missing).
#if DEBUG
let forceStage1 = UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage1"
#else
let forceStage1 = false
#endif
if !forceStage1,
let meter = presentMeter,
let pipeline = Stage2Pipeline(presentMeter: meter) {
startStage2(pipeline, connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
@@ -300,8 +365,8 @@ public final class StreamViewController: UIViewController {
onFrame: (@Sendable (AccessUnit) -> Void)?, onSessionEnd: (@Sendable () -> Void)?
) {
let metal = pipeline.layer
metal.contentsScale = streamView.contentScaleFactor
// Composites OVER the idle (un-enqueued in stage-2) AVSampleBufferDisplayLayer base.
// (contentsScale + frame are set by layoutMetalLayer() just below.)
streamView.layer.addSublayer(metal)
metalLayer = metal
stage2 = pipeline
@@ -325,9 +390,20 @@ public final class StreamViewController: UIViewController {
layoutMetalLayer()
}
/// Aspect-fit the metal sublayer in the view (the host streams at the client's native mode,
/// so this is usually the full bounds). drawableSize is the layer's pixel size; the shader's
/// fullscreen triangle scales the decoded texture to fill it.
/// The display scale to render the metal drawable at. `traitCollection.displayScale` is the
/// canonical render scale and is reliable once the controller is in the hierarchy;
/// `view.contentScaleFactor` can read 1.0 before the view attaches to a window/screen, which
/// would size the drawable at point resolution a pixelated, upscaled mess. Falls back to the
/// main screen scale if the trait is still unspecified.
private var renderScale: CGFloat {
let s = traitCollection.displayScale
return s > 0 ? s : UIScreen.main.scale
}
/// Position the metal sublayer aspect-fit in the view (the host streams at the client's native
/// mode, so this is usually the full bounds). Only the layer FRAME is set here the presenter
/// sizes the drawable to the decoded frame and the layer's contentsGravity (.resizeAspect)
/// scales it to this frame via the system compositor (matching stage-1's videoGravity).
private func layoutMetalLayer() {
guard let metalLayer, let connection else { return }
let mode = connection.currentMode()
@@ -337,19 +413,17 @@ public final class StreamViewController: UIViewController {
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds)
: bounds
let scale = streamView.contentScaleFactor
CATransaction.begin()
CATransaction.setDisableActions(true) // don't animate the resize
metalLayer.contentsScale = scale
metalLayer.contentsScale = renderScale
metalLayer.frame = fit
CATransaction.commit()
stage2?.setDrawableSize(CGSize(width: fit.width * scale, height: fit.height * scale))
}
private func teardownStage2() {
stage2Link?.invalidate()
stage2Link = nil
stage2?.stop()
stage2?.stop() // stops the pump (synchronous join) + drops the decode session
stage2 = nil
metalLayer?.removeFromSuperlayer()
metalLayer = nil
@@ -369,6 +443,7 @@ public final class StreamViewController: UIViewController {
captured = false
}
setNeedsUpdateOfPrefersPointerLocked()
updatePointerLockChain() // (re)anchor the SwiftUI ancestors so the lock actually resolves
syncPointerLock() // resolve cursor + GCMouse/absolute routing for the current state
let onCaptureChange = onCaptureChange
let captured = captured
@@ -397,7 +472,7 @@ public final class StreamViewController: UIViewController {
pointerInteraction?.invalidate() // re-resolve the hidden/visible cursor for the state
if iosInputDebug {
iosInputLog.debug(
"pointer lock isLocked=\(locked, privacy: .public) captured=\(self.captured, privacy: .public)")
"pointer lock isLocked=\(locked, privacy: .public) captured=\(self.captured, privacy: .public) useGCMouse=\(useGCMouse, privacy: .public) [\(self.inputCapture?.attachedMiceSummary ?? "n/a", privacy: .public)]")
}
}
#endif
@@ -49,11 +49,10 @@ public final class VideoDecoder: @unchecked Sendable {
/// pump can re-gate on the next IDR.
private let onDecodeError: @Sendable (OSStatus) -> Void
/// Latest source HDR mastering metadata (from `PunktfunkConnection.nextHdrMeta`), attached to
/// each decoded HDR pixel buffer so the compositor tone-maps from the real grade. Guarded by its
/// own lock written by the pump thread, read on the VT decode callback.
private let metaLock = NSLock()
private var hdrMeta: PunktfunkConnection.HdrMeta?
/// Whether the negotiated stream is full-chroma 4:4:4 (`connection.isChroma444`), set once at
/// session start before any decode. Selects the 4:4:4 decode pixel format (orthogonal to bit
/// depth / HDR). Read inside `createSessionLocked` under `lock`.
private var chroma444 = false
public init(
onDecoded: @escaping @Sendable (ReadyFrame) -> Void,
@@ -65,12 +64,13 @@ public final class VideoDecoder: @unchecked Sendable {
deinit { teardown() }
/// Set the source HDR mastering metadata (drained from `PunktfunkConnection.nextHdrMeta`). It's
/// attached to subsequent decoded HDR pixel buffers. Thread-safe; cheap to call on each update.
public func setHdrMeta(_ meta: PunktfunkConnection.HdrMeta) {
metaLock.lock()
hdrMeta = meta
metaLock.unlock()
/// Select the chroma subsampling of the decode output (4:2:0 vs full-chroma 4:4:4). Call once at
/// session start, before decoding, from `connection.isChroma444`. Takes effect on the next
/// session (re)build. Thread-safe.
public func setChroma444(_ on: Bool) {
lock.lock()
chroma444 = on
lock.unlock()
}
/// Submit one AU for asynchronous decode, (re)creating the session if `format` changed. The
@@ -135,8 +135,10 @@ public final class VideoDecoder: @unchecked Sendable {
/// True when `newFormat` carries a PQ (SMPTE ST 2084) or HLG transfer function i.e. the host
/// is sending HDR (BT.2020). VideoToolbox populates the transfer-function extension from the
/// HEVC VUI, so this tracks the *stream*, switching dynamically when the user toggles HDR
/// (the host re-emits parameter sets with the new VUI a new format desc session rebuild).
/// HEVC VUI, so this picks the decode bit depth (10-bit P010/x444 vs 8-bit NV12/444v) from the
/// stream. The present-side HDR config (colorspace/EDR/shader) is latched once per session from
/// the Welcome (`connection.isHDR`), which the host does NOT flip mid-session so this predicate
/// and that config agree for the session (a `#if DEBUG` assert in the presenter guards it).
static func isHDRFormat(_ format: CMVideoFormatDescription) -> Bool {
guard
let tf = CMFormatDescriptionGetExtension(
@@ -157,11 +159,18 @@ public final class VideoDecoder: @unchecked Sendable {
session = nil
format = nil
// Decode pixel format is a 2×2 of (chroma, depth/HDR), both biplanar so the presenter binds
// plane 0 = luma, plane 1 = interleaved chroma uniformly 4:4:4 just delivers a full-size
// chroma plane. 10-bit (P010 / `x444`) for HDR (PQ/HLG), 8-bit (NV12 / `444v`) otherwise.
let hdr = Self.isHDRFormat(newFormat)
let pixelFormat =
hdr
? kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange // P010 (10-bit)
: kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange // NV12 (8-bit)
let pixelFormat: OSType = {
switch (chroma444, hdr) {
case (false, false): return kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange // NV12
case (false, true): return kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange // P010
case (true, false): return kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange // 444v
case (true, true): return kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange // x444
}
}()
let imageAttrs: [CFString: Any] = [
kCVPixelBufferMetalCompatibilityKey: true,
kCVPixelBufferPixelFormatTypeKey: pixelFormat,
@@ -169,11 +178,20 @@ public final class VideoDecoder: @unchecked Sendable {
var callback = VTDecompressionOutputCallbackRecord(
decompressionOutputCallback: decoderOutputCallback,
decompressionOutputRefCon: Unmanaged.passUnretained(self).toOpaque())
// 4:4:4 sessions REQUIRE a hardware decoder: we only advertise 4:4:4 when the hardware probe
// passed, so a hardware-incapable mode (e.g. a resolution past the HW 4:4:4 ceiling) must fail
// HERE, synchronously, letting the pump's backstop end the session rather than silently
// falling back to a software 4:4:4 decoder far too slow for a real-time stream. 4:2:0 keeps the
// software fallback (nil spec) as a robustness net.
let spec: CFDictionary? =
chroma444
? [kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder: true] as CFDictionary
: nil
var newSession: VTDecompressionSession?
let status = VTDecompressionSessionCreate(
allocator: kCFAllocatorDefault,
formatDescription: newFormat,
decoderSpecification: nil, // hardware by default
decoderSpecification: spec,
imageBufferAttributes: imageAttrs as CFDictionary,
outputCallback: &callback,
decompressionSessionOut: &newSession)
@@ -195,26 +213,17 @@ public final class VideoDecoder: @unchecked Sendable {
// pts was stamped at timescale 1e9 (AnnexB.sampleBuffer); normalize defensively.
let p = CMTimeConvertScale(pts, timescale: 1_000_000_000, method: .default)
let ptsNs = p.value > 0 ? UInt64(p.value) : 0
// HDR iff the decoder produced a 10-bit P010 buffer (we only request P010 for PQ streams).
// HDR iff the decoder produced a 10-bit buffer (we only request a 10-bit format for PQ/HLG
// streams). Covers 4:2:0 (P010) and 4:4:4 (`x444`), video- and full-range, so a 10-bit 4:4:4
// HDR frame isn't misclassified as SDR. (The mastering metadata is applied to the presenter's
// CAMetalLayer via CAEDRMetadata, not to this source buffer a separate-drawable presenter
// never composites the source buffer's attachments, so attaching them here would be dead.)
let fmt = CVPixelBufferGetPixelFormatType(imageBuffer)
let isHDR =
CVPixelBufferGetPixelFormatType(imageBuffer)
== kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange
// Attach the source's mastering display + content light level (ST.2086 / CEA-861.3) so the
// compositor tone-maps from the real grade rather than inferring from the PQ colourspace
// alone. The SEI byte payloads map 1:1 to these CVImageBuffer attachment keys.
if isHDR {
metaLock.lock()
let meta = hdrMeta
metaLock.unlock()
if let meta {
CVBufferSetAttachment(
imageBuffer, kCVImageBufferMasteringDisplayColorVolumeKey,
meta.masteringDisplayColorVolume() as CFData, .shouldPropagate)
CVBufferSetAttachment(
imageBuffer, kCVImageBufferContentLightLevelInfoKey,
meta.contentLightLevelInfo() as CFData, .shouldPropagate)
}
}
fmt == kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange
|| fmt == kCVPixelFormatType_420YpCbCr10BiPlanarFullRange
|| fmt == kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange
|| fmt == kCVPixelFormatType_444YpCbCr10BiPlanarFullRange
onDecoded(
ReadyFrame(ptsNs: ptsNs, decodedNs: decodedNs, pixelBuffer: imageBuffer, isHDR: isHDR))
}
@@ -0,0 +1,69 @@
import XCTest
#if canImport(Metal)
import CoreVideo
import Metal
import QuartzCore
@testable import PunktfunkKit
final class MetalPresenterTests: XCTestCase {
/// `MetalVideoPresenter.make()` compiles the runtime Metal shaders (the BT.709/BT.2020 YUVRGB
/// fragment shaders plus the Catmull-Rom luma sampler). A `nil` result on a GPU-equipped host
/// means a shader failed to compile this catches a malformed shader before it silently
/// degrades stage-2 to a stage-1 fallback on device.
func testPresenterInitCompilesShaders() throws {
guard MTLCreateSystemDefaultDevice() != nil else {
throw XCTSkip("no Metal device available in this environment")
}
XCTAssertNotNil(
MetalVideoPresenter.make(),
"stage-2 Metal shaders failed to compile (presenter init returned nil)")
}
/// The HDR fix: `configure(hdr:)` must put the layer into the BT.2020-PQ EDR configuration with a
/// reference-white anchor (`edrMetadata`) the missing anchor was what made HDR render "too
/// bright". SDR must use the plain 8-bit path with EDR off and no metadata. A mid-session flip is a
/// per-mode reconfigure, so the round trip back to SDR must fully restore the SDR config.
func testConfigureHDRSetsEDRAnchor() throws {
guard let presenter = MetalVideoPresenter.make() else {
throw XCTSkip("no Metal device available in this environment")
}
presenter.configure(hdr: true)
XCTAssertEqual(presenter.layer.pixelFormat, .rgba16Float, "HDR uses an EDR-capable drawable")
XCTAssertNotNil(presenter.layer.colorspace, "HDR layer must be tagged (itur_2100_PQ)")
XCTAssertTrue(
presenter.layer.wantsExtendedDynamicRangeContent, "EDR must be requested on all platforms")
XCTAssertNotNil(
presenter.layer.edrMetadata,
"HDR must anchor reference white via edrMetadata (the fix for 'too bright')")
// Mid-session HDRSDR flip: the 8-bit path, EDR off, no metadata.
presenter.configure(hdr: false)
XCTAssertEqual(presenter.layer.pixelFormat, .bgra8Unorm, "SDR uses the plain 8-bit drawable")
XCTAssertFalse(presenter.layer.wantsExtendedDynamicRangeContent)
XCTAssertNil(presenter.layer.edrMetadata)
}
/// `render` with a freshly-allocated NV12 buffer must present without crashing or hanging the
/// main-thread present path is the highest-risk part of the stage-2 rewrite. (A headless CI with no
/// display can still allocate a drawable from a CAMetalLayer; if it can't, render returns false,
/// which is also a valid non-crashing outcome.)
func testRenderDoesNotCrashOnNV12Frame() throws {
guard let presenter = MetalVideoPresenter.make() else {
throw XCTSkip("no Metal device available in this environment")
}
presenter.configure(hdr: false)
var pb: CVPixelBuffer?
let attrs: [CFString: Any] = [kCVPixelBufferMetalCompatibilityKey: true]
let status = CVPixelBufferCreate(
kCFAllocatorDefault, 256, 256, kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange,
attrs as CFDictionary, &pb)
guard status == kCVReturnSuccess, let pixelBuffer = pb else {
throw XCTSkip("could not allocate a test pixel buffer")
}
// Just asserting it returns (true or false) without trapping the layer may have no drawable
// source headless, so a false return is acceptable.
_ = presenter.render(pixelBuffer, isHDR: false)
}
}
#endif
@@ -0,0 +1,68 @@
// 4:4:4 decode-path coverage: the hardware-capability probe is stable/cached, and a real 4:4:4 HEVC
// keyframe decodes through VideoDecoder to a biplanar 4:4:4 pixel buffer. Reuses the same synthetic
// 4:4:4 blobs the runtime probe ships with.
import CoreVideo
import VideoToolbox
import XCTest
@testable import PunktfunkKit
private final class FrameBox: @unchecked Sendable {
let lock = NSLock()
var frame: ReadyFrame?
var error: OSStatus?
}
final class Stage444Tests: XCTestCase {
/// The capability probe is device-static and cached reading it twice must return the same value
/// (and must never crash, including where 4:4:4 is unsupported false).
func testProbeIsStableAndCached() {
XCTAssertEqual(Stage444Probe.hwDecode444_8bit, Stage444Probe.hwDecode444_8bit)
XCTAssertEqual(Stage444Probe.hwDecode444_10bit, Stage444Probe.hwDecode444_10bit)
}
/// A real 8-bit 4:4:4 HEVC keyframe (the embedded probe blob) decodes through `VideoDecoder` with
/// `setChroma444(true)` to a 256×256 biplanar 4:4:4 (`444v`/`444f`) buffer classified SDR.
/// (4:4:4 sessions require a hardware decoder skip where there isn't one, which is exactly where
/// the client wouldn't advertise 4:4:4 anyway.)
func testVideoDecoderDecodes444() throws {
try XCTSkipUnless(
Stage444Probe.hwDecode444_8bit, "no hardware 4:4:4 decode on this device")
let data = Data(Probe444Blobs.au444_8bit)
let format = try XCTUnwrap(
AnnexB.formatDescription(fromIDR: data), "the 4:4:4 blob must yield a format description")
let au = AccessUnit(data: data, ptsNs: 7_000_000, frameIndex: 0, flags: 0)
let box = FrameBox()
let done = DispatchSemaphore(value: 0)
let decoder = VideoDecoder(
onDecoded: { f in box.lock.lock(); box.frame = f; box.lock.unlock(); done.signal() },
onDecodeError: { s in box.lock.lock(); box.error = s; box.lock.unlock(); done.signal() })
decoder.setChroma444(true)
XCTAssertTrue(decoder.decode(au: au, format: format), "4:4:4 frame submit should succeed")
XCTAssertEqual(done.wait(timeout: .now() + 10), .success, "the decode callback must fire")
decoder.reset()
box.lock.lock(); let frame = box.frame; let error = box.error; box.lock.unlock()
XCTAssertNil(error.map { "decode error \($0)" })
let ready = try XCTUnwrap(frame, "a 4:4:4 ReadyFrame must be delivered")
XCTAssertEqual(CVPixelBufferGetWidth(ready.pixelBuffer), 256)
XCTAssertEqual(CVPixelBufferGetHeight(ready.pixelBuffer), 256)
let pf = CVPixelBufferGetPixelFormatType(ready.pixelBuffer)
XCTAssertTrue(
pf == kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange
|| pf == kCVPixelFormatType_444YpCbCr8BiPlanarFullRange,
"expected a biplanar 4:4:4 8-bit buffer, got \(fourCCString(pf))")
XCTAssertFalse(ready.isHDR, "an 8-bit BT.709 4:4:4 stream is SDR")
// The chroma plane (plane 1) must be FULL resolution for 4:4:4 (vs half for 4:2:0) this is
// what lets the unchanged shader sample chroma at the luma UV.
XCTAssertEqual(CVPixelBufferGetWidthOfPlane(ready.pixelBuffer, 1), 256)
XCTAssertEqual(CVPixelBufferGetHeightOfPlane(ready.pixelBuffer, 1), 256)
}
private func fourCCString(_ t: OSType) -> String {
let b = [UInt8(t >> 24 & 0xff), UInt8(t >> 16 & 0xff), UInt8(t >> 8 & 0xff), UInt8(t & 0xff)]
return String(bytes: b, encoding: .ascii) ?? "\(t)"
}
}
+14 -5
View File
@@ -294,7 +294,13 @@ const RESOLUTIONS: [number, number, string][] = [
[2560, 1440, "2560 × 1440"],
];
const REFRESH = [0, 30, 60, 90, 120];
const GAMEPADS = ["auto", "xbox360", "dualsense"];
const GAMEPADS = ["auto", "xbox360", "dualsense", "steamdeck"];
const GAMEPAD_LABELS: Record<string, string> = {
auto: "Automatic",
xbox360: "Xbox 360",
dualsense: "DualSense",
steamdeck: "Steam Deck",
};
const SettingsSection: FC = () => {
const [s, setS] = useState<StreamSettings | null>(null);
@@ -355,14 +361,17 @@ const SettingsSection: FC = () => {
/>
<Field label="Gamepad type" childrenContainerWidth="max">
<Dropdown
rgOptions={GAMEPADS.map((g) => ({
data: g,
label: g === "auto" ? "Automatic" : g === "xbox360" ? "Xbox 360" : "DualSense",
}))}
rgOptions={GAMEPADS.map((g) => ({ data: g, label: GAMEPAD_LABELS[g] ?? g }))}
selectedOption={s.gamepad}
onChange={(o) => patch({ gamepad: o.data as string })}
/>
</Field>
{s.gamepad === "steamdeck" && (
<Field
label="⚠ Disable Steam Input"
description="Steam Deck mode forwards the paddles, both trackpads, and gyro to the host. For that, Steam Input must be OFF for punktfunk: on the game page tap ⚙ → Controller Settings → set Steam Input to Off. Otherwise Steam keeps the Deck's controls and only the sticks + buttons reach the host."
/>
)}
<ToggleField
label="Stream microphone"
checked={s.mic_enabled}
+23
View File
@@ -113,12 +113,35 @@ async function ensureShortcut(): Promise<number> {
return appId;
}
/**
* Best-effort: turn Steam Input OFF for our shortcut so SDL's HIDAPI Steam Deck driver can open the
* Deck's controls (paddles · trackpads · gyro) directly. There is no confirmed-stable SteamClient
* API for this, so it is feature-detected and MUST never block or throw into the launch — the manual
* toggle (game page → ⚙ → Controller Settings → Steam Input Off, surfaced in the plugin Settings) is
* the documented source of truth. No-op when the optional API is absent.
*/
function disableSteamInputForShortcut(appId: number): void {
try {
const input = (
SteamClient as unknown as {
Input?: { SetSteamInputEnabledForApp?: (appId: number, enabled: boolean) => void };
}
).Input;
input?.SetSteamInputEnabledForApp?.(appId, false);
} catch {
/* a controller tweak must never break the launch */
}
}
/**
* Launch a stream to `host:port` fullscreen in Gaming Mode. Encodes the target into the
* shortcut's launch options (so one generic shortcut serves every host), then RunGame.
*/
export async function launchStream(host: string, port: number): Promise<void> {
const appId = await ensureShortcut();
// Best-effort so the Deck's rich controls reach the client; no-op if the API is absent (the user
// disables Steam Input manually — see the Settings instruction).
disableSteamInputForShortcut(appId);
const target = port && port !== 9777 ? `${host}:${port}` : host;
// KEY=value ... %command% — the wrapper reads PF_HOST from the environment.
SteamClient.Apps.SetAppLaunchOptions(appId, `PF_HOST=${target} %command%`);
+1
View File
@@ -767,6 +767,7 @@ fn start_session_with(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>,
connector,
frames.take().expect("Connected delivered once"),
app.gamepad.escape_events(),
app.gamepad.disconnect_events(),
handle.stop.clone(),
inhibit,
&title,
+186 -32
View File
@@ -18,7 +18,7 @@ use punktfunk_core::quic::{HidOutput, RichInput};
use std::collections::HashMap;
use std::sync::mpsc::{Receiver, Sender};
use std::sync::{Arc, Mutex};
use std::time::Duration;
use std::time::{Duration, Instant};
/// Motion scale constants, shared convention with the Swift client (`GamepadWire`):
/// derived from hid-playstation's math over the host's fixed calibration blob. SDL hands
@@ -33,8 +33,15 @@ const G: f32 = 9.80665;
/// is the only way out. Four simultaneous buttons that no game uses as a deliberate
/// combo, so it can't be triggered by normal play. Still forwarded to the host (the user
/// is leaving anyway); we only also raise the escape signal.
///
/// **Escalation:** a quick press leaves fullscreen / releases capture; *holding* the same
/// chord for [`DISCONNECT_HOLD`] ends the session. Deliberately NOT the Steam / QAM buttons —
/// those are the marquee pass-through controls that now reach the host's game-mode UI.
const ESCAPE_CHORD: [u32; 4] = [wire::BTN_LB, wire::BTN_RB, wire::BTN_START, wire::BTN_BACK];
/// Hold the [`ESCAPE_CHORD`] at least this long to disconnect (escalates the leave-fullscreen press).
const DISCONNECT_HOLD: Duration = Duration::from_millis(1500);
#[derive(Clone, Debug)]
pub struct PadInfo {
pub id: u32,
@@ -58,6 +65,7 @@ impl PadInfo {
GamepadPref::DualSense => "DualSense",
GamepadPref::DualShock4 => "DualShock 4",
GamepadPref::XboxOne => "Xbox One",
GamepadPref::SteamDeck => "Steam Deck",
_ => "",
}
}
@@ -89,6 +97,9 @@ pub struct GamepadService {
/// Fires once per press of the [`ESCAPE_CHORD`]; the stream page consumes it to leave
/// fullscreen + release capture.
escape_rx: async_channel::Receiver<()>,
/// Fires once when the [`ESCAPE_CHORD`] is held past [`DISCONNECT_HOLD`]; the stream page
/// consumes it to end the session (the controller equivalent of Ctrl+Alt+Shift+D).
disconnect_rx: async_channel::Receiver<()>,
}
impl GamepadService {
@@ -98,11 +109,12 @@ impl GamepadService {
let pinned = Arc::new(Mutex::new(None));
let (ctl, ctl_rx) = std::sync::mpsc::channel();
let (escape_tx, escape_rx) = async_channel::unbounded();
let (disconnect_tx, disconnect_rx) = async_channel::unbounded();
let (p, a, pin) = (pads.clone(), active.clone(), pinned.clone());
if let Err(e) = std::thread::Builder::new()
.name("punktfunk-gamepad".into())
.spawn(move || {
if let Err(e) = run(&p, &a, &pin, &ctl_rx, &escape_tx) {
if let Err(e) = run(&p, &a, &pin, &ctl_rx, &escape_tx, &disconnect_tx) {
tracing::warn!(error = %e, "gamepad service ended — pads disabled");
}
})
@@ -115,6 +127,7 @@ impl GamepadService {
pinned,
ctl,
escape_rx,
disconnect_rx,
}
}
@@ -124,6 +137,12 @@ impl GamepadService {
self.escape_rx.clone()
}
/// A receiver that yields one `()` when the escape chord is held past [`DISCONNECT_HOLD`]
/// (controller disconnect). A fresh clone per call; the stream page spawns a future on it.
pub fn disconnect_events(&self) -> async_channel::Receiver<()> {
self.disconnect_rx.clone()
}
pub fn pads(&self) -> Vec<PadInfo> {
self.pads.lock().unwrap().clone()
}
@@ -188,6 +207,13 @@ fn button_bit(b: sdl3::gamepad::Button) -> Option<u32> {
Button::DPadLeft => wire::BTN_DPAD_LEFT,
Button::DPadRight => wire::BTN_DPAD_RIGHT,
Button::Touchpad => wire::BTN_TOUCHPAD,
// Back grips / paddles (Steam Deck L4/L5/R4/R5, Xbox Elite P1P4) + the misc/Share button.
// PADDLE1/2/3/4 = R4/L4/R5/L5 (see the host `input::gamepad`).
Button::RightPaddle1 => wire::BTN_PADDLE1,
Button::LeftPaddle1 => wire::BTN_PADDLE2,
Button::RightPaddle2 => wire::BTN_PADDLE3,
Button::LeftPaddle2 => wire::BTN_PADDLE4,
Button::Misc1 => wire::BTN_MISC1,
_ => return None,
})
}
@@ -259,11 +285,22 @@ struct Worker {
/// Wire state of the active pad — zeroed on the wire at switch/detach.
last_axis: [i32; 6],
held_buttons: Vec<u32>,
/// Touchpad contacts the host believes are down, keyed by `(surface, finger)` — lifted on pad
/// switch / detach so a contact held at that moment doesn't stick. surface 0 = the legacy single
/// touchpad, 1/2 = a Steam left/right pad.
held_touches: std::collections::HashSet<(u8, u8)>,
last_accel: [i16; 3],
/// Raises the UI escape signal; the escape chord fires it once per press.
escape_tx: async_channel::Sender<()>,
/// Raises the UI disconnect signal when the escape chord is held past [`DISCONNECT_HOLD`].
disconnect_tx: async_channel::Sender<()>,
/// The escape chord is fully held — latched so it fires once, not every poll.
chord_armed: bool,
/// When the escape chord became fully held (drives the hold-to-disconnect escalation); `None`
/// when the chord is broken.
chord_since: Option<Instant>,
/// The disconnect signal already fired for the current hold — latched so it fires once.
disconnect_fired: bool,
}
impl Worker {
@@ -275,13 +312,22 @@ impl Worker {
fn pad_info(&self, id: u32) -> Option<PadInfo> {
let pad = self.opened.get(&id)?;
let mut pref = pref_for_type(
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
);
// There is no SDL gamepad type for the Steam Deck / Steam Controller, so detect Valve by
// VID/PID (Deck 0x1205, SC wired 0x1102, SC dongle 0x1142) — the host then builds the virtual
// hid-steam pad with the back grips + dual trackpads and the right glyph identity.
if pad.vendor_id() == Some(0x28DE)
&& matches!(pad.product_id(), Some(0x1205 | 0x1102 | 0x1142))
{
pref = GamepadPref::SteamDeck;
}
Some(PadInfo {
id,
name: pad.name().unwrap_or_else(|| "Controller".into()),
pref: pref_for_type(
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
),
pref,
})
}
@@ -297,32 +343,90 @@ impl Worker {
}
*v = i32::MIN;
}
// Lift any touchpad contact the host still believes is down (surface 0 = legacy pad).
for (surface, finger) in self.held_touches.drain() {
let rich = if surface == 0 {
RichInput::Touchpad {
pad: 0,
finger,
active: false,
x: 0,
y: 0,
}
} else {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: false,
click: false,
x: 0,
y: 0,
pressure: 0,
}
};
let _ = c.send_rich_input(rich);
}
} else {
self.held_buttons.clear();
self.last_axis = [i32::MIN; 6];
self.held_touches.clear();
}
// A held chord doesn't survive a flush (detach / pad-switch) — clear its latches too.
self.reset_chord();
}
/// Raise the UI escape signal when the [`ESCAPE_CHORD`] just completed (latched so it
/// fires once per press). Called after each button-down updates `held_buttons`.
/// fires once per press) and start the hold-to-disconnect timer. Called after each
/// button-down updates `held_buttons`.
fn maybe_fire_escape(&mut self) {
if self.chord_armed {
return;
}
if ESCAPE_CHORD.iter().all(|b| self.held_buttons.contains(b)) {
self.chord_armed = true;
self.chord_since = Some(Instant::now());
let _ = self.escape_tx.try_send(());
tracing::info!("gamepad escape chord (L1+R1+Start+Select) — leaving fullscreen");
tracing::info!(
"gamepad escape chord (L1+R1+Start+Select) — leaving fullscreen (hold to disconnect)"
);
}
}
/// Fire the disconnect signal once the escape chord has been continuously held past
/// [`DISCONNECT_HOLD`]. Polled from the main loop so the hold completes without new events.
fn maybe_fire_disconnect(&mut self) {
if self.disconnect_fired {
return;
}
if let Some(since) = self.chord_since {
if since.elapsed() >= DISCONNECT_HOLD {
self.disconnect_fired = true;
let _ = self.disconnect_tx.try_send(());
tracing::info!("gamepad escape chord held — disconnecting");
}
}
}
/// Re-arm once the chord is broken (any of its buttons released).
fn rearm_escape(&mut self) {
if self.chord_armed && !ESCAPE_CHORD.iter().all(|b| self.held_buttons.contains(b)) {
self.chord_armed = false;
self.reset_chord();
}
}
/// Clear the escape/disconnect chord latches. Called at every session boundary
/// ([`flush_held`](Self::flush_held) on detach/pad-switch + on attach): the hold-to-disconnect
/// path *always* ends the session while the chord is still physically held, so the matching
/// button-up events arrive after detach (dropped by the `attached` guard) and `rearm_escape`
/// never runs — without this the latched state would leak into the next session and either
/// swallow its first chord press or fire a stale disconnect on connect.
fn reset_chord(&mut self) {
self.chord_armed = false;
self.chord_since = None;
self.disconnect_fired = false;
}
/// Sensors stream only while a session wants them (they cost USB/BT bandwidth).
fn set_sensors(&mut self, enabled: bool) {
let Some(id) = self.active_id() else { return };
@@ -335,6 +439,56 @@ impl Worker {
}
}
}
/// Forward one touchpad contact on the rich-input plane. A multi-touchpad pad (Steam Deck / Steam
/// Controller) sends `TouchpadEx` with the surface (SDL touchpad 0 = left → 1, 1 = right → 2) and
/// signed coordinates; a single-touchpad pad (DualSense) keeps the legacy `Touchpad` (unsigned).
fn forward_touch(
&mut self,
which: u32,
touchpad: u32,
finger: u8,
x: f32,
y: f32,
active: bool,
) {
let Some(c) = self.attached.as_ref() else {
return;
};
let multi = self
.opened
.get(&which)
.map(|p| p.touchpads_count() >= 2)
.unwrap_or(false);
let (cx, cy) = (x.clamp(0.0, 1.0), y.clamp(0.0, 1.0));
let surface = if multi { (touchpad as u8) + 1 } else { 0 };
let rich = if multi {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: active,
click: false,
x: (cx * 65535.0 - 32768.0) as i16,
y: (cy * 65535.0 - 32768.0) as i16,
pressure: 0,
}
} else {
RichInput::Touchpad {
pad: 0,
finger,
active,
x: (cx * 65535.0) as u16,
y: (cy * 65535.0) as u16,
}
};
let _ = c.send_rich_input(rich);
if active {
self.held_touches.insert((surface, finger));
} else {
self.held_touches.remove(&(surface, finger));
}
}
}
#[allow(clippy::too_many_lines)]
@@ -344,11 +498,18 @@ fn run(
pinned_out: &Mutex<Option<u32>>,
ctl: &Receiver<Ctl>,
escape_tx: &async_channel::Sender<()>,
disconnect_tx: &async_channel::Sender<()>,
) -> Result<(), String> {
// Off-main-thread + no video subsystem: keep SDL away from signals, poll pads on its
// own thread.
sdl3::hint::set("SDL_NO_SIGNAL_HANDLERS", "1");
sdl3::hint::set("SDL_JOYSTICK_THREAD", "1");
// Let SDL's HIDAPI drivers open Valve Steam Controller / Steam Deck devices directly, so the
// paddles, both trackpads, and gyro arrive as first-class SDL gamepad inputs. On a Deck in Game
// Mode, Steam Input still holds the device — the user must disable Steam Input for this app (see
// the Decky UX); on a desktop client (or a Deck with Steam Input off) the hints just work.
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAMDECK", "1");
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAM", "1");
let sdl = sdl3::init().map_err(|e| e.to_string())?;
let subsystem = sdl.gamepad().map_err(|e| e.to_string())?;
let mut pump = sdl.event_pump().map_err(|e| e.to_string())?;
@@ -361,9 +522,13 @@ fn run(
attached: None,
last_axis: [i32::MIN; 6],
held_buttons: Vec::new(),
held_touches: std::collections::HashSet::new(),
last_accel: [0; 3],
escape_tx: escape_tx.clone(),
disconnect_tx: disconnect_tx.clone(),
chord_armed: false,
chord_since: None,
disconnect_fired: false,
};
let publish = |w: &Worker| {
@@ -381,6 +546,7 @@ fn run(
Ok(Ctl::Attach(c)) => {
w.attached = Some(c);
w.last_axis = [i32::MIN; 6];
w.reset_chord(); // every session starts un-latched (Attach doesn't flush)
w.set_sensors(true);
}
Ok(Ctl::Detach) => {
@@ -474,9 +640,11 @@ fn run(
send(w.attached.as_ref().unwrap(), InputKind::GamepadAxis, id, v);
}
}
// DualSense touchpad → the rich-input plane, normalized 0..=65535.
// Touchpad contacts → the rich-input plane. One pad (DualSense) keeps the legacy
// `Touchpad`; two pads (Steam Deck / Steam Controller) send `TouchpadEx` per surface.
Event::ControllerTouchpadDown {
which,
touchpad,
finger,
x,
y,
@@ -484,41 +652,23 @@ fn run(
}
| Event::ControllerTouchpadMotion {
which,
touchpad,
finger,
x,
y,
..
} if active == Some(which) && w.attached.is_some() => {
let _ = w
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: true,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
w.forward_touch(which, touchpad as u32, finger as u8, x, y, true);
}
Event::ControllerTouchpadUp {
which,
touchpad,
finger,
x,
y,
..
} if active == Some(which) && w.attached.is_some() => {
let _ = w
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: false,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
w.forward_touch(which, touchpad as u32, finger as u8, x, y, false);
}
// Motion: accel events update the cache; each gyro event ships a sample
// (the DualSense reports both at ~250 Hz). Scale convention shared with
@@ -559,6 +709,10 @@ fn run(
}
}
// Escalate a held escape chord to a disconnect (polled — the hold completes with no
// new button events; the chord itself is only detected while a session is attached).
w.maybe_fire_disconnect();
// Feedback planes (this thread is their single consumer). The host re-sends
// rumble state periodically, so a generous duration with refresh-on-update is
// safe — a dropped stop heals within ~500 ms.
+36 -3
View File
@@ -124,12 +124,13 @@ impl Capture {
}
}
#[allow(clippy::too_many_lines)]
#[allow(clippy::too_many_lines, clippy::too_many_arguments)]
pub fn new(
window: &adw::ApplicationWindow,
connector: Arc<NativeClient>,
frames: async_channel::Receiver<DecodedFrame>,
escape_rx: async_channel::Receiver<()>,
disconnect_rx: async_channel::Receiver<()>,
stop: Arc<AtomicBool>,
inhibit_shortcuts: bool,
title: &str,
@@ -152,7 +153,7 @@ pub fn new(
stats_label.set_margin_top(12);
let hint = gtk::Label::new(Some(
"Click the stream to capture input · Ctrl+Alt+Shift+Q releases",
"Click the stream to capture input · Ctrl+Alt+Shift+Q releases · Ctrl+Alt+Shift+D disconnects",
));
hint.add_css_class("osd");
hint.set_halign(gtk::Align::Center);
@@ -163,7 +164,9 @@ pub fn new(
// Flashed when entering fullscreen — the only exit affordances once the header bar is
// hidden (F11 on a keyboard; the L1+R1+Start+Select chord on a controller, which is the
// only way out on a Steam Deck).
let fs_hint = gtk::Label::new(Some("F11 · L1 + R1 + Start + Select — exit fullscreen"));
let fs_hint = gtk::Label::new(Some(
"F11 · L1 + R1 + Start + Select — exit fullscreen (hold to disconnect)",
));
fs_hint.add_css_class("osd");
fs_hint.set_halign(gtk::Align::Center);
fs_hint.set_valign(gtk::Align::Start);
@@ -297,6 +300,7 @@ pub fn new(
key.set_propagation_phase(gtk::PropagationPhase::Capture);
let cap = capture.clone();
let window_k = window.clone();
let stop_kb = stop.clone();
key.connect_key_pressed(move |_, keyval, keycode, state| {
let chord = gdk::ModifierType::CONTROL_MASK
| gdk::ModifierType::ALT_MASK
@@ -309,6 +313,13 @@ pub fn new(
}
return glib::Propagation::Stop;
}
// Ctrl+Alt+Shift+D — leave the session. Now that Steam / QAM pass through to the host,
// the capture toggle alone can't end a stream, so this is the keyboard's explicit exit.
if state.contains(chord) && keyval.to_lower() == gdk::Key::d {
cap.release();
stop_kb.store(true, Ordering::SeqCst);
return glib::Propagation::Stop;
}
if keyval == gdk::Key::F11 {
if window_k.is_fullscreen() {
window_k.unfullscreen();
@@ -442,6 +453,24 @@ pub fn new(
})
};
// Controller disconnect (escape chord held past the hold threshold) → end the session, the
// controller equivalent of Ctrl+Alt+Shift+D. Setting `stop` ends the session pump, which pops
// this page (and fires `hidden` below). One-shot — the session is going away.
let disconnect_future = {
let window = window.clone();
let cap = capture.clone();
let stop_d = stop.clone();
glib::spawn_future_local(async move {
if disconnect_rx.recv().await.is_ok() {
cap.release();
if window.is_fullscreen() {
window.unfullscreen();
}
stop_d.store(true, Ordering::SeqCst);
}
})
};
// The page's `hidden` fires once navigation away completes (back button, pop on
// session end) — NOT on the transient unmap/map cycle a NavigationView push performs.
{
@@ -449,6 +478,7 @@ pub fn new(
let stop_h = stop.clone();
let handlers = RefCell::new(Some((fs_handler, active_handler)));
let escape_future = RefCell::new(Some(escape_future));
let disconnect_future = RefCell::new(Some(disconnect_future));
page.connect_hidden(move |_| {
tracing::debug!("stream page hidden — ending session");
if let Some((fs, active)) = handlers.borrow_mut().take() {
@@ -458,6 +488,9 @@ pub fn new(
if let Some(f) = escape_future.borrow_mut().take() {
f.abort();
}
if let Some(f) = disconnect_future.borrow_mut().take() {
f.abort();
}
if window.is_fullscreen() {
window.unfullscreen();
}
+108 -27
View File
@@ -169,6 +169,13 @@ fn button_bit(b: sdl3::gamepad::Button) -> Option<u32> {
Button::DPadLeft => wire::BTN_DPAD_LEFT,
Button::DPadRight => wire::BTN_DPAD_RIGHT,
Button::Touchpad => wire::BTN_TOUCHPAD,
// Back grips / paddles (Steam Deck L4/L5/R4/R5, Xbox Elite P1P4) + the misc/Share button.
// PADDLE1/2/3/4 = R4/L4/R5/L5 (see the host `input::gamepad`).
Button::RightPaddle1 => wire::BTN_PADDLE1,
Button::LeftPaddle1 => wire::BTN_PADDLE2,
Button::RightPaddle2 => wire::BTN_PADDLE3,
Button::LeftPaddle2 => wire::BTN_PADDLE4,
Button::Misc1 => wire::BTN_MISC1,
_ => return None,
})
}
@@ -240,6 +247,9 @@ struct Worker {
/// Wire state of the active pad — zeroed on the wire at switch/detach.
last_axis: [i32; 6],
held_buttons: Vec<u32>,
/// Touchpad contacts the host believes are down, keyed by `(surface, finger)` — lifted on pad
/// switch / detach. surface 0 = the legacy single touchpad, 1/2 = a Steam left/right pad.
held_touches: std::collections::HashSet<(u8, u8)>,
last_accel: [i16; 3],
}
@@ -252,13 +262,21 @@ impl Worker {
fn pad_info(&self, id: u32) -> Option<PadInfo> {
let pad = self.opened.get(&id)?;
let mut pref = pref_for_type(
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
);
// No SDL type for the Steam Deck / Steam Controller — detect Valve by VID/PID (Deck 0x1205,
// SC wired 0x1102, SC dongle 0x1142) so the host builds the virtual hid-steam pad.
if pad.vendor_id() == Some(0x28DE)
&& matches!(pad.product_id(), Some(0x1205 | 0x1102 | 0x1142))
{
pref = GamepadPref::SteamDeck;
}
Some(PadInfo {
id,
name: pad.name().unwrap_or_else(|| "Controller".into()),
pref: pref_for_type(
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
),
pref,
})
}
@@ -274,9 +292,33 @@ impl Worker {
}
*v = i32::MIN;
}
for (surface, finger) in self.held_touches.drain() {
let rich = if surface == 0 {
RichInput::Touchpad {
pad: 0,
finger,
active: false,
x: 0,
y: 0,
}
} else {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: false,
click: false,
x: 0,
y: 0,
pressure: 0,
}
};
let _ = c.send_rich_input(rich);
}
} else {
self.held_buttons.clear();
self.last_axis = [i32::MIN; 6];
self.held_touches.clear();
}
}
@@ -292,6 +334,56 @@ impl Worker {
}
}
}
/// Forward one touchpad contact on the rich-input plane. A multi-touchpad pad (Steam Deck / Steam
/// Controller) sends `TouchpadEx` with the surface (SDL touchpad 0 = left → 1, 1 = right → 2) and
/// signed coordinates; a single-touchpad pad (DualSense) keeps the legacy `Touchpad` (unsigned).
fn forward_touch(
&mut self,
which: u32,
touchpad: u32,
finger: u8,
x: f32,
y: f32,
active: bool,
) {
let Some(c) = self.attached.as_ref() else {
return;
};
let multi = self
.opened
.get(&which)
.map(|p| p.touchpads_count() >= 2)
.unwrap_or(false);
let (cx, cy) = (x.clamp(0.0, 1.0), y.clamp(0.0, 1.0));
let surface = if multi { (touchpad as u8) + 1 } else { 0 };
let rich = if multi {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: active,
click: false,
x: (cx * 65535.0 - 32768.0) as i16,
y: (cy * 65535.0 - 32768.0) as i16,
pressure: 0,
}
} else {
RichInput::Touchpad {
pad: 0,
finger,
active,
x: (cx * 65535.0) as u16,
y: (cy * 65535.0) as u16,
}
};
let _ = c.send_rich_input(rich);
if active {
self.held_touches.insert((surface, finger));
} else {
self.held_touches.remove(&(surface, finger));
}
}
}
#[allow(clippy::too_many_lines)]
@@ -305,6 +397,10 @@ fn run(
// thread.
sdl3::hint::set("SDL_NO_SIGNAL_HANDLERS", "1");
sdl3::hint::set("SDL_JOYSTICK_THREAD", "1");
// Let SDL's HIDAPI drivers open Valve Steam Controller / Steam Deck devices directly, so the
// paddles, both trackpads, and gyro arrive as first-class SDL gamepad inputs.
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAMDECK", "1");
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAM", "1");
let sdl = sdl3::init().map_err(|e| e.to_string())?;
let subsystem = sdl.gamepad().map_err(|e| e.to_string())?;
let mut pump = sdl.event_pump().map_err(|e| e.to_string())?;
@@ -317,6 +413,7 @@ fn run(
attached: None,
last_axis: [i32::MIN; 6],
held_buttons: Vec::new(),
held_touches: std::collections::HashSet::new(),
last_accel: [0; 3],
};
@@ -426,9 +523,11 @@ fn run(
send(w.attached.as_ref().unwrap(), InputKind::GamepadAxis, id, v);
}
}
// DualSense touchpad → the rich-input plane, normalized 0..=65535.
// Touchpad contacts → the rich-input plane. One pad (DualSense) keeps the legacy
// `Touchpad`; two pads (Steam Deck / Steam Controller) send `TouchpadEx` per surface.
Event::ControllerTouchpadDown {
which,
touchpad,
finger,
x,
y,
@@ -436,41 +535,23 @@ fn run(
}
| Event::ControllerTouchpadMotion {
which,
touchpad,
finger,
x,
y,
..
} if active == Some(which) && w.attached.is_some() => {
let _ = w
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: true,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
w.forward_touch(which, touchpad as u32, finger as u8, x, y, true);
}
Event::ControllerTouchpadUp {
which,
touchpad,
finger,
x,
y,
..
} if active == Some(which) && w.attached.is_some() => {
let _ = w
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: false,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
w.forward_touch(which, touchpad as u32, finger as u8, x, y, false);
}
// Motion: accel events update the cache; each gyro event ships a sample (the
// DualSense reports both at ~250 Hz). Scale convention shared with the other
+167
View File
@@ -492,6 +492,10 @@ pub const PUNKTFUNK_HIDOUT_LED: u8 = 1;
pub const PUNKTFUNK_HIDOUT_PLAYER_LEDS: u8 = 2;
/// `PunktfunkHidOutput::kind` — one adaptive-trigger effect (`which` + `effect`/`effect_len` valid).
pub const PUNKTFUNK_HIDOUT_TRIGGER: u8 = 3;
/// `PunktfunkHidOutput::kind` — a trackpad haptic pulse (Steam Controller voice-coils). `which` =
/// side (0 = right pad, 1 = left pad); `effect[0..6]` packs `amplitude` / `period` / `count` as
/// little-endian `u16`s with `effect_len = 6`. Clients without trackpad coils drop it.
pub const PUNKTFUNK_HIDOUT_TRACKPAD_HAPTIC: u8 = 4;
/// Capacity of `PunktfunkHidOutput::effect` (the DualSense trigger parameter block).
pub const PUNKTFUNK_HID_EFFECT_MAX: u8 = 11;
@@ -559,6 +563,23 @@ impl PunktfunkHidOutput {
out.effect[..n].copy_from_slice(&effect[..n]);
out.effect_len = n as u8;
}
HidOutput::TrackpadHaptic {
pad,
side,
amplitude,
period,
count,
} => {
// No new struct (PunktfunkHidOutput has no size guard): pack into the existing
// `which` (side) + `effect[0..6]` (amplitude/period/count LE), `effect_len = 6`.
out.kind = PUNKTFUNK_HIDOUT_TRACKPAD_HAPTIC;
out.pad = *pad;
out.which = *side;
out.effect[0..2].copy_from_slice(&amplitude.to_le_bytes());
out.effect[2..4].copy_from_slice(&period.to_le_bytes());
out.effect[4..6].copy_from_slice(&count.to_le_bytes());
out.effect_len = 6;
}
}
out
}
@@ -618,6 +639,11 @@ impl PunktfunkHdrMeta {
pub const PUNKTFUNK_RICH_TOUCHPAD: u8 = 1;
/// `PunktfunkRichInput::kind` — a motion sample (`gyro`/`accel` valid).
pub const PUNKTFUNK_RICH_MOTION: u8 = 2;
/// `RichInput::TouchpadEx` kind on the wire — an extended trackpad contact that identifies the
/// surface (0 single / 1 Steam-left / 2 Steam-right) and carries click + pressure. The host decodes
/// it today; *sending* it from a C client needs the size-prefixed `PunktfunkRichInputEx` +
/// `punktfunk_connection_send_rich_input2` (added with client capture).
pub const PUNKTFUNK_RICH_TOUCHPAD_EX: u8 = 3;
/// One rich client→host input for the host's virtual DualSense
/// ([`punktfunk_connection_send_rich_input`]): a touchpad contact or a motion sample. Set `kind`
@@ -666,6 +692,77 @@ impl PunktfunkRichInput {
}
}
/// Forward-compatible superset of [`PunktfunkRichInput`] that can also express the rich Steam
/// surfaces: a *second* trackpad (`surface`), a distinct `click` vs touch, signed coordinates, and
/// pressure. Sent via [`punktfunk_connection_send_rich_input2`] — the only way a C client can emit a
/// `TouchpadEx`. The caller MUST set `struct_size = sizeof(PunktfunkRichInputEx)` (the ABI-skew
/// guard, like [`PunktfunkConfig`]); the legacy [`PunktfunkRichInput`] +
/// [`punktfunk_connection_send_rich_input`] stay byte-for-byte for existing callers.
#[cfg(feature = "quic")]
#[repr(C)]
#[derive(Clone, Copy)]
pub struct PunktfunkRichInputEx {
/// MUST equal `sizeof(PunktfunkRichInputEx)`.
pub struct_size: u32,
/// One of `PUNKTFUNK_RICH_*` (`TOUCHPAD` / `MOTION` / `TOUCHPAD_EX`).
pub kind: u8,
/// Gamepad index.
pub pad: u8,
/// Touchpad/TouchpadEx: contact id.
pub finger: u8,
/// Touchpad/TouchpadEx: 1 = finger down / touching, 0 = lifted.
pub active: u8,
/// TouchpadEx: which surface — 0 = single/DualSense, 1 = Steam left pad, 2 = Steam right pad.
pub surface: u8,
/// TouchpadEx: 1 = the pad is physically clicked (depressed), distinct from a touch contact.
pub click: u8,
/// Reserved for alignment; set to 0.
pub _reserved: [u8; 2],
/// TouchpadEx: x coordinate — **signed**, centred at 0 (the real Steam report convention). For a
/// legacy `TOUCHPAD` kind sent through this struct, store the unsigned `0..=65535` value's bits.
pub x: i16,
/// TouchpadEx: y coordinate — signed, centred at 0.
pub y: i16,
/// TouchpadEx: contact pressure (`0` if the surface has no force sensor).
pub pressure: u16,
/// Motion: gyro (pitch, yaw, roll), raw signed-16.
pub gyro: [i16; 3],
/// Motion: accelerometer (x, y, z), raw signed-16.
pub accel: [i16; 3],
}
#[cfg(feature = "quic")]
impl PunktfunkRichInputEx {
fn to_rich(self) -> Option<crate::quic::RichInput> {
use crate::quic::RichInput;
match self.kind {
PUNKTFUNK_RICH_TOUCHPAD_EX => Some(RichInput::TouchpadEx {
pad: self.pad,
surface: self.surface,
finger: self.finger,
touch: self.active != 0,
click: self.click != 0,
x: self.x,
y: self.y,
pressure: self.pressure,
}),
PUNKTFUNK_RICH_MOTION => Some(RichInput::Motion {
pad: self.pad,
gyro: self.gyro,
accel: self.accel,
}),
PUNKTFUNK_RICH_TOUCHPAD => Some(RichInput::Touchpad {
pad: self.pad,
finger: self.finger,
active: self.active != 0,
x: self.x as u16,
y: self.y as u16,
}),
_ => None,
}
}
}
/// Read an optional NUL-terminated UTF-8 string parameter; `Err` = invalid pointer/UTF-8.
#[cfg(feature = "quic")]
unsafe fn opt_cstr<'a>(p: *const std::os::raw::c_char) -> std::result::Result<Option<&'a str>, ()> {
@@ -714,6 +811,22 @@ pub const PUNKTFUNK_GAMEPAD_XBOXONE: u32 = 3;
/// DualSense (minus adaptive triggers / player LEDs / mute). Honored only where available (Linux
/// hosts); otherwise the host falls back to X-Box 360.
pub const PUNKTFUNK_GAMEPAD_DUALSHOCK4: u32 = 4;
/// UHID classic Steam Controller (Valve `28DE:1102`, kernel `hid-steam`): dual trackpads, gyro,
/// two grip paddles. Reserved — currently folds to `XBOX360` until its backend lands.
pub const PUNKTFUNK_GAMEPAD_STEAMCONTROLLER: u32 = 5;
/// UHID Steam Deck controller (Valve `28DE:1205`, kernel `hid-steam`): full Deck gamepad incl. the
/// four back grips, a right trackpad, and the IMU; re-grabbed by Steam Input with native glyphs when
/// Steam runs on the host. Honored only where available (Linux hosts); else folds to X-Box 360.
pub const PUNKTFUNK_GAMEPAD_STEAMDECK: u32 = 6;
/// Extended `InputEvent` gamepad button bits for embedders building raw events: the four back grips
/// (Steam L4/L5/R4/R5 ≙ Xbox-Elite P1P4) + the misc/capture button, in Moonlight's
/// `buttonFlags2 << 16` namespace. Mirror `input::gamepad::BTN_PADDLE1..4` / `BTN_MISC1`.
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE1: u32 = 0x0001_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE2: u32 = 0x0002_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE3: u32 = 0x0004_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE4: u32 = 0x0008_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_MISC1: u32 = 0x0020_0000;
/// Connect to a `punktfunk/1` host and start a session at `width`x`height`@`refresh_hz`.
/// Blocks up to `timeout_ms` for the handshake. Returns NULL on failure. Equivalent to
@@ -742,11 +855,28 @@ const _: () = {
// Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift).
const _: () = {
use crate::config::GamepadPref;
use crate::input::gamepad as g;
assert!(PUNKTFUNK_GAMEPAD_AUTO == GamepadPref::Auto.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_XBOX360 == GamepadPref::Xbox360.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_DUALSENSE == GamepadPref::DualSense.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_XBOXONE == GamepadPref::XboxOne.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_DUALSHOCK4 == GamepadPref::DualShock4.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_STEAMCONTROLLER == GamepadPref::SteamController.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_STEAMDECK == GamepadPref::SteamDeck.to_u8() as u32);
// Extended button bits mirror the wire `input::gamepad` constants.
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE1 == g::BTN_PADDLE1);
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE2 == g::BTN_PADDLE2);
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE3 == g::BTN_PADDLE3);
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE4 == g::BTN_PADDLE4);
assert!(PUNKTFUNK_GAMEPAD_BTN_MISC1 == g::BTN_MISC1);
};
// The additive M3 kinds (TouchpadEx / TrackpadHaptic) must never grow the legacy ABI structs —
// they have no `struct_size` guard, so a layout change would corrupt old-built callers' buffers.
#[cfg(feature = "quic")]
const _: () = {
assert!(core::mem::size_of::<PunktfunkRichInput>() == 20);
assert!(core::mem::size_of::<PunktfunkHidOutput>() == 19);
};
/// Trust: `pin_sha256` (NULL or 32 bytes) is the expected SHA-256 fingerprint of the host's
@@ -1727,6 +1857,43 @@ pub unsafe extern "C" fn punktfunk_connection_send_rich_input(
})
}
/// Send a rich client→host input via the forward-compatible [`PunktfunkRichInputEx`] — the only way
/// a C client can emit a `TouchpadEx` (a second trackpad / signed coords / pressure). Set
/// `rich->struct_size = sizeof(PunktfunkRichInputEx)`; a smaller (older-layout) value is rejected.
///
/// # Safety
/// `c` is a valid connection handle; `rich` is null or points to at least its declared
/// `struct_size` bytes.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_send_rich_input2(
c: *mut PunktfunkConnection,
rich: *const PunktfunkRichInputEx,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if rich.is_null() {
return PunktfunkStatus::NullPointer;
}
// Read only the 4-byte size prefix first to bound the subsequent full read (the
// `PunktfunkConfig` ABI-skew precedent).
let declared = unsafe { std::ptr::addr_of!((*rich).struct_size).read_unaligned() } as usize;
if declared < std::mem::size_of::<PunktfunkRichInputEx>() {
return PunktfunkStatus::InvalidArg;
}
match unsafe { *rich }.to_rich() {
Some(r) => match c.inner.send_rich_input(r) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
},
None => PunktfunkStatus::InvalidArg,
}
})
}
/// The currently active session mode — the Welcome's, until an accepted
/// [`punktfunk_connection_request_mode`] switches it. Safe any time after connect.
///
+45 -4
View File
@@ -137,8 +137,9 @@ impl CompositorPref {
/// host decide (its `PUNKTFUNK_GAMEPAD` env var, else X-Box 360). A concrete preference is
/// honored only if that backend is available on the host (DualSense / DualShock 4 need Linux UHID);
/// otherwise the host falls back and reports the real choice in `Welcome`. The wire form is a single
/// byte (`0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`), appended to
/// `Hello`/`Welcome` — older peers simply omit/ignore it (an unknown byte degrades to `Auto`).
/// byte (`0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`,
/// `5 = SteamController`, `6 = SteamDeck`), appended to `Hello`/`Welcome` — older peers simply
/// omit/ignore it (an unknown byte degrades to `Auto`).
#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)]
pub enum GamepadPref {
/// Let the host pick (its `PUNKTFUNK_GAMEPAD` env var, else X-Box 360).
@@ -155,10 +156,19 @@ pub enum GamepadPref {
/// UHID DualShock 4 (kernel `hid-playstation`, ≥ 6.2) — lightbar, touchpad, motion, rumble. Like
/// `DualSense` minus adaptive triggers / player LEDs / mute. Needs Linux UHID on the host.
DualShock4,
/// UHID classic Steam Controller (Valve `28DE:1102`, kernel `hid-steam`) — dual trackpads, gyro,
/// two grip paddles, trackpad-only haptics. Needs Linux UHID. *(Reserved; its backend is not yet
/// built — currently folds to `Xbox360`; the Deck identity below is the implemented one.)*
SteamController,
/// UHID Steam Deck controller (Valve `28DE:1205`, kernel `hid-steam`) — full Deck gamepad incl.
/// the four back grips (L4/L5/R4/R5), a right trackpad, and the IMU; re-grabbed by Steam Input
/// with native glyphs when Steam runs on the host. Needs Linux UHID.
SteamDeck,
}
impl GamepadPref {
/// Wire byte. `0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`.
/// Wire byte. `0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`,
/// `5 = SteamController`, `6 = SteamDeck`.
pub const fn to_u8(self) -> u8 {
match self {
GamepadPref::Auto => 0,
@@ -166,6 +176,8 @@ impl GamepadPref {
GamepadPref::DualSense => 2,
GamepadPref::XboxOne => 3,
GamepadPref::DualShock4 => 4,
GamepadPref::SteamController => 5,
GamepadPref::SteamDeck => 6,
}
}
@@ -177,6 +189,8 @@ impl GamepadPref {
2 => GamepadPref::DualSense,
3 => GamepadPref::XboxOne,
4 => GamepadPref::DualShock4,
5 => GamepadPref::SteamController,
6 => GamepadPref::SteamDeck,
_ => GamepadPref::Auto,
}
}
@@ -192,12 +206,14 @@ impl GamepadPref {
GamepadPref::XboxOne
}
"dualshock4" | "dualshock" | "ds4" | "ps4" => GamepadPref::DualShock4,
"steamdeck" | "steam-deck" | "deck" => GamepadPref::SteamDeck,
"steamcontroller" | "steam-controller" | "steamcon" => GamepadPref::SteamController,
_ => return None,
})
}
/// Canonical lowercase identifier (`"auto"`, `"xbox360"`, `"dualsense"`, `"xboxone"`,
/// `"dualshock4"`).
/// `"dualshock4"`, `"steamcontroller"`, `"steamdeck"`).
pub fn as_str(self) -> &'static str {
match self {
GamepadPref::Auto => "auto",
@@ -205,6 +221,8 @@ impl GamepadPref {
GamepadPref::DualSense => "dualsense",
GamepadPref::XboxOne => "xboxone",
GamepadPref::DualShock4 => "dualshock4",
GamepadPref::SteamController => "steamcontroller",
GamepadPref::SteamDeck => "steamdeck",
}
}
}
@@ -381,4 +399,27 @@ mod tests {
c.fec.fec_percent = 15; // 250 + ceil(250*15/100)=288 > 255
assert!(c.validate().is_err());
}
#[test]
fn gamepad_pref_steam_roundtrip() {
use GamepadPref::*;
// Wire-byte round-trip for the Steam additions; an unknown byte still degrades to Auto.
for (p, b) in [(SteamController, 5u8), (SteamDeck, 6)] {
assert_eq!(p.to_u8(), b);
assert_eq!(GamepadPref::from_u8(b), p);
}
assert_eq!(GamepadPref::from_u8(99), Auto);
// Name parsing + canonical-name round-trip.
assert_eq!(GamepadPref::from_name("steamdeck"), Some(SteamDeck));
assert_eq!(GamepadPref::from_name("deck"), Some(SteamDeck));
assert_eq!(
GamepadPref::from_name("steamcontroller"),
Some(SteamController)
);
assert_eq!(SteamDeck.as_str(), "steamdeck");
assert_eq!(
GamepadPref::from_name(SteamController.as_str()),
Some(SteamController)
);
}
}
+14
View File
@@ -66,10 +66,24 @@ pub mod gamepad {
pub const BTN_B: u32 = 0x2000;
pub const BTN_X: u32 = 0x4000;
pub const BTN_Y: u32 = 0x8000;
// Extended buttons in Moonlight's `buttonFlags2 << 16` namespace (see `gamestream/gamepad.rs`),
// so the GameStream paddle path and the native path share one host injector map. The four Steam
// Deck back grips (L4/L5/R4/R5) reuse the four GameStream/Xbox-Elite paddle slots — a semantic
// 1:1 for binding (the device identity carries the glyph distinction).
/// Back grip R4 — SDL `RightPaddle1` / GameStream `PADDLE1`.
pub const BTN_PADDLE1: u32 = 0x0001_0000;
/// Back grip L4 — SDL `LeftPaddle1` / GameStream `PADDLE2`.
pub const BTN_PADDLE2: u32 = 0x0002_0000;
/// Back grip R5 — SDL `RightPaddle2` / GameStream `PADDLE3`.
pub const BTN_PADDLE3: u32 = 0x0004_0000;
/// Back grip L5 — SDL `LeftPaddle2` / GameStream `PADDLE4`.
pub const BTN_PADDLE4: u32 = 0x0008_0000;
/// DualSense touchpad click. Moonlight's extended-button position (`buttonFlags2`
/// merges in at `<< 16`, see `gamestream/gamepad.rs`), so GameStream clients land on
/// the same bit. Only the DualSense backend renders it; the xpad has no such button.
pub const BTN_TOUCHPAD: u32 = 0x10_0000;
/// Misc / capture button — the Deck `…`/quick-access, Share/Capture / GameStream `MISC`.
pub const BTN_MISC1: u32 = 0x0020_0000;
/// Axis ids for `InputKind::GamepadAxis`.
pub const AXIS_LS_X: u32 = 0;
+92 -1
View File
@@ -1218,6 +1218,7 @@ pub fn decode_mic_datagram(b: &[u8]) -> Option<(u32, u64, &[u8])> {
const RICH_TOUCHPAD: u8 = 0x01;
const RICH_MOTION: u8 = 0x02;
const RICH_TOUCHPAD_EX: u8 = 0x03;
/// A rich client→host controller input beyond the fixed [`InputEvent`](crate::input::InputEvent):
/// the DualSense touchpad and motion sensors. `pad` is the gamepad index. Wire form is
@@ -1241,6 +1242,22 @@ pub enum RichInput {
gyro: [i16; 3],
accel: [i16; 3],
},
/// A richer trackpad contact that also identifies *which* physical pad (Steam Controller / Deck
/// have two), carries a separate click vs touch state, and a pressure reading. `surface`:
/// `0` = the single / DualSense touchpad, `1` = the Steam left pad, `2` = the Steam right pad.
/// Coordinates are **signed** (centred at 0), matching the real Steam report; `pressure` is `0`
/// for a surface with no force sensor. New clients send this for every touch surface; the host
/// decodes both `Touchpad` (`0x01`) and `TouchpadEx` (`0x03`) indefinitely.
TouchpadEx {
pad: u8,
surface: u8,
finger: u8,
touch: bool,
click: bool,
x: i16,
y: i16,
pressure: u16,
},
}
impl RichInput {
@@ -1264,6 +1281,22 @@ impl RichInput {
out.extend_from_slice(&v.to_le_bytes());
}
}
RichInput::TouchpadEx {
pad,
surface,
finger,
touch,
click,
x,
y,
pressure,
} => {
let state = (touch as u8) | ((click as u8) << 1);
out.extend_from_slice(&[RICH_TOUCHPAD_EX, pad, surface, finger, state]);
out.extend_from_slice(&x.to_le_bytes());
out.extend_from_slice(&y.to_le_bytes());
out.extend_from_slice(&pressure.to_le_bytes());
}
}
out
}
@@ -1288,6 +1321,16 @@ impl RichInput {
accel: [i16at(9), i16at(11), i16at(13)],
})
}
RICH_TOUCHPAD_EX if b.len() >= 12 => Some(RichInput::TouchpadEx {
pad: b[2],
surface: b[3],
finger: b[4],
touch: b[5] & 0x01 != 0,
click: b[5] & 0x02 != 0,
x: i16::from_le_bytes([b[6], b[7]]),
y: i16::from_le_bytes([b[8], b[9]]),
pressure: u16::from_le_bytes([b[10], b[11]]),
}),
_ => None,
}
}
@@ -1296,6 +1339,7 @@ impl RichInput {
const HIDOUT_LED: u8 = 0x01;
const HIDOUT_PLAYER_LEDS: u8 = 0x02;
const HIDOUT_TRIGGER: u8 = 0x03;
const HIDOUT_TRACKPAD_HAPTIC: u8 = 0x04;
/// DualSense feedback flowing host → client (what a game wrote to the host's virtual pad).
/// Wire form `[0xCD][kind][pad][fields…]`. The rich analog of the fixed rumble datagram;
@@ -1309,6 +1353,16 @@ pub enum HidOutput {
/// One adaptive-trigger effect: `which` 0 = L2, 1 = R2; `effect` is the raw DualSense
/// trigger parameter block (mode + params) for the client to replay on a real controller.
Trigger { pad: u8, which: u8, effect: Vec<u8> },
/// A trackpad haptic pulse for a Steam Controller's voice-coil actuators (its only "rumble").
/// `side` 0 = right pad, 1 = left pad; `amplitude` + `period` (µs off-time) + `count` (pulses)
/// synthesize a buzz. A client without trackpad coils drops it (or maps it to ordinary rumble).
TrackpadHaptic {
pad: u8,
side: u8,
amplitude: u16,
period: u16,
count: u16,
},
}
impl HidOutput {
@@ -1325,6 +1379,18 @@ impl HidOutput {
out.extend_from_slice(&[HIDOUT_TRIGGER, *pad, *which]);
out.extend_from_slice(effect);
}
HidOutput::TrackpadHaptic {
pad,
side,
amplitude,
period,
count,
} => {
out.extend_from_slice(&[HIDOUT_TRACKPAD_HAPTIC, *pad, *side]);
out.extend_from_slice(&amplitude.to_le_bytes());
out.extend_from_slice(&period.to_le_bytes());
out.extend_from_slice(&count.to_le_bytes());
}
}
out
}
@@ -1349,6 +1415,13 @@ impl HidOutput {
which: b[3],
effect: b[4..].to_vec(),
}),
HIDOUT_TRACKPAD_HAPTIC if b.len() >= 10 => Some(HidOutput::TrackpadHaptic {
pad: b[2],
side: b[3],
amplitude: u16::from_le_bytes([b[4], b[5]]),
period: u16::from_le_bytes([b[6], b[7]]),
count: u16::from_le_bytes([b[8], b[9]]),
}),
_ => None,
}
}
@@ -2486,6 +2559,16 @@ mod tests {
gyro: [-100, 200, -300],
accel: [16384, -8192, 1],
},
RichInput::TouchpadEx {
pad: 2,
surface: 1,
finger: 1,
touch: true,
click: false,
x: -12345,
y: 30000,
pressure: 4000,
},
] {
let d = ev.encode();
assert_eq!(d[0], RICH_INPUT_MAGIC);
@@ -2494,7 +2577,8 @@ mod tests {
// Disjoint from the fixed input datagram (0xC8); unknown kind + truncation → None.
assert!(RichInput::decode(&[crate::input::INPUT_MAGIC; 18]).is_none());
assert!(RichInput::decode(&[RICH_INPUT_MAGIC, 0x7F]).is_none()); // unknown kind
assert!(RichInput::decode(&[RICH_INPUT_MAGIC, RICH_TOUCHPAD, 0]).is_none());
assert!(RichInput::decode(&[RICH_INPUT_MAGIC, RICH_TOUCHPAD, 0]).is_none()); // short
assert!(RichInput::decode(&[RICH_INPUT_MAGIC, RICH_TOUCHPAD_EX, 0, 0, 0, 0]).is_none());
// short
}
@@ -2516,6 +2600,13 @@ mod tests {
which: 1,
effect: vec![0x26, 0x90, 0xA0, 0xFF, 0x00, 0x00],
},
HidOutput::TrackpadHaptic {
pad: 0,
side: 1,
amplitude: 0x1234,
period: 0x5678,
count: 9,
},
];
for ev in &cases {
let d = ev.encode();
+7
View File
@@ -89,6 +89,9 @@ tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] }
wayland-client = "0.31"
wayland-protocols-wlr = { version = "0.3", features = ["client"] }
wayland-protocols-misc = { version = "0.3", features = ["client"] }
# `xdg-output` (zxdg_output_v1): the per-output *logical* geometry (post-scale size + global
# position), used by the KWin fake_input backend to map absolute coordinates under display scaling.
wayland-protocols = { version = "0.32", features = ["client"] }
# Codegen for KDE's `zkde_screencast_unstable_v1` (vendored in `protocols/`): create a KWin
# virtual output sized to the client's resolution and get its PipeWire node (KRdp's path).
# `wayland-backend` is referenced by the generated interface tables.
@@ -119,6 +122,10 @@ ash = "0.38"
# `libcuda.so.1` is dlopen'd at runtime (NOT link-time) so one Linux binary runs on NVIDIA
# (zero-copy via CUDA) AND on AMD/Intel (VAAPI, no NVIDIA driver present) — see `zerocopy::cuda`.
libloading = "0.8"
# Vendored + trimmed `usbip` server core (no libusb) — presents a virtual Steam Deck over USB/IP
# so the local `vhci_hcd` attaches it: the shippable, Secure-Boot-clean, Steam-Input-promotable
# virtual-Deck transport on non-SteamOS hosts (`inject/linux/steam_usbip.rs`). See the crate's NOTICE.
usbip-sim = { path = "vendor/usbip-sim" }
[target.'cfg(target_os = "windows")'.dependencies]
# Windows host backends. `windows` covers the Win32/CCD APIs the SudoVDA virtual-display backend
@@ -1,8 +1,8 @@
//! P2 direct frame push (kill DDA) — HOST side. The pf-vdisplay driver runs in a restricted WUDFHost
//! token that canNOT create named kernel objects, so — exactly like the gamepad UMDF drivers
//! (`inject/dualsense_windows.rs`) — the HOST (privileged) CREATES the shared header + frame-ready
//! event + ring of keyed-mutex textures (`Global\` names, permissive `D:(A;;GA;;;WD)` SDDL) on the
//! discrete render GPU, and the driver only OPENS them and copies frames in. We then consume the ring
//! P2 direct frame push (kill DDA) — HOST side. The pf-vdisplay driver's WUDFHost canNOT create named
//! kernel objects, so — exactly like the gamepad UMDF drivers (`inject/dualsense_windows.rs`) — the
//! HOST (privileged) CREATES the shared header + frame-ready event + ring of keyed-mutex textures
//! (`Global\` names, scoped `D:(A;;GA;;;SY)(A;;GA;;;LS)` to SYSTEM + the driver's LocalService host —
//! see `shared_object_sa`) on the discrete render GPU, and the driver only OPENS them and copies frames in. We then consume the ring
//! straight into the zero-copy NVENC path — no DXGI Desktop Duplication, no `win32u` hook. Gated by
//! `PUNKTFUNK_IDD_PUSH`. Driver counterpart: `packaging/windows/drivers/pf-vdisplay/src/
//! frame_transport.rs`. The shared `SharedHeader` layout, `MAGIC`/`VERSION`/`RING_LEN`, the
@@ -236,13 +236,17 @@ pub struct IddPushCapturer {
// ownership to one thread at a time with NO concurrent access; we do not (and must not) claim `Sync`.
unsafe impl Send for IddPushCapturer {}
/// Build a permissive (Everyone:GenericAll) `SECURITY_ATTRIBUTES` so the restricted WUDFHost driver
/// can OPEN the host-created objects — the same `D:(A;;GA;;;WD)` SDDL the gamepad shared section uses.
/// The returned `psd` backing must outlive `sa`; both are dropped when the process exits.
unsafe fn permissive_sa() -> Result<(SECURITY_ATTRIBUTES, PSECURITY_DESCRIPTOR)> {
/// Build a `SECURITY_ATTRIBUTES` granting GENERIC_ALL to **SYSTEM** (the host creates+publishes the
/// shared event + texture ring) and **LocalService** (the account the pf_vdisplay WUDFHost runs under,
/// which consumes them) — `D:(A;;GA;;;SY)(A;;GA;;;LS)`, the same scoping as the gamepad section. The
/// old SDDL granted **Everyone** (`WD`), which let any local user open the `Global\pfvd-*` objects and
/// read captured screen frames (security-review 2026-06-28 #5). Verified on the RTX box (2026-06-29):
/// the WUDFHost token is `S-1-5-19` (LocalService), SYSTEM integrity, zero restricted SIDs — so SY+LS
/// suffices for the driver and excludes normal user processes. `psd` must outlive `sa`.
unsafe fn shared_object_sa() -> Result<(SECURITY_ATTRIBUTES, PSECURITY_DESCRIPTOR)> {
let mut psd = PSECURITY_DESCRIPTOR::default();
ConvertStringSecurityDescriptorToSecurityDescriptorW(
w!("D:(A;;GA;;;WD)"),
w!("D:(A;;GA;;;SY)(A;;GA;;;LS)"),
SDDL_REVISION_1,
&mut psd,
None,
@@ -269,7 +273,7 @@ impl IddPushCapturer {
h: u32,
format: DXGI_FORMAT,
) -> Result<Vec<HostSlot>> {
let (sa, _psd) = permissive_sa()?;
let (sa, _psd) = shared_object_sa()?;
let mut slots = Vec::new();
for k in 0..RING_LEN {
let desc = D3D11_TEXTURE2D_DESC {
@@ -375,7 +379,7 @@ impl IddPushCapturer {
// SAFETY: one block over the whole ring setup; every operation in it is sound:
// - `set_advanced_color`/`advanced_color_enabled` are `unsafe fn`s taking only a copy of the plain
// `u32` target id; they read/flip CCD display config and return owned values, borrowing nothing.
// - `CreateDXGIFactory1`, `EnumAdapterByLuid`, `make_device`, `permissive_sa`, `CreateFileMappingW`,
// - `CreateDXGIFactory1`, `EnumAdapterByLuid`, `make_device`, `shared_object_sa`, `CreateFileMappingW`,
// `MapViewOfFile`, `CreateEventW`, and `create_ring_slots` are all `?`-checked, so every returned
// interface/handle/view is non-error before use; `&sa`/`&adapter`/`&device`/the `&HSTRING` names
// are live borrows that outlive each synchronous call, and `sa.lpSecurityDescriptor` stays valid
@@ -421,7 +425,7 @@ impl IddPushCapturer {
.context("EnumAdapterByLuid(render adapter) for IDD push")?;
let (device, context) = make_device(&adapter).context("make_device for IDD push")?;
let (sa, _psd) = permissive_sa()?;
let (sa, _psd) = shared_object_sa()?;
let bytes = std::mem::size_of::<SharedHeader>().max(64);
// Header.
@@ -66,6 +66,13 @@ pub const BTN_A: u32 = 0x1000;
pub const BTN_B: u32 = 0x2000;
pub const BTN_X: u32 = 0x4000;
pub const BTN_Y: u32 = 0x8000;
// Extended buttons in the `buttonFlags2 << 16` namespace (mirror `punktfunk_core::input::gamepad`):
// the four back-grip paddles. `decode` already merges `buttonFlags2 << 16` into `buttons`, but the
// injector map dropped these bits — Sunshine/Moonlight paddle clients were silently no-op'd.
pub const BTN_PADDLE1: u32 = 0x0001_0000;
pub const BTN_PADDLE2: u32 = 0x0002_0000;
pub const BTN_PADDLE3: u32 = 0x0004_0000;
pub const BTN_PADDLE4: u32 = 0x0008_0000;
/// Decode one decrypted control plaintext into a controller event, if it is one. Mouse,
/// keyboard, keepalives etc. yield `None` (they're handled by [`super::input::decode`]).
@@ -101,6 +101,10 @@ struct Session {
server_challenge: [u8; 16],
/// The client's phase-3 hash, recomputed + checked in phase 4.
client_hash: Vec<u8>,
/// Set once phase 3 has produced the RSA-signed serversecret. A repeated phase 3 is refused so a
/// peer past phase 1 can't loop phase2/phase3 to harvest many signing-time samples (a passive
/// timing-oracle amplifier vs. the rsa-crate Marvin side-channel; see `.cargo/audit.toml`).
responded: bool,
}
pub struct Pairing {
@@ -155,6 +159,7 @@ impl Pairing {
serversecret: [0; 16],
server_challenge: [0; 16],
client_hash: Vec::new(),
responded: false,
},
);
tracing::info!(
@@ -216,6 +221,14 @@ impl Pairing {
bail!("short challenge response");
}
s.client_hash = client_hash[..32].to_vec();
// Sign the serversecret exactly ONCE per ceremony: refuse a repeated phase 3 so a peer that
// cleared phase 1 (operator PIN) can't replay it to collect many RSA signing-time samples
// (timing-oracle amplifier vs. RUSTSEC-2023-0071; see `.cargo/audit.toml`). A legit client
// signs once. The session stays for phase 4 (the cert-pin step) but won't re-sign.
if s.responded {
bail!("serverchallengeresp already answered for this pairing session");
}
s.responded = true;
let sig: Signature = id.signing_key.sign(&s.serversecret);
let mut secret = Vec::with_capacity(16 + 256);
secret.extend_from_slice(&s.serversecret);
+25
View File
@@ -491,6 +491,31 @@ pub mod gamepad;
#[cfg(target_os = "windows")]
#[path = "inject/windows/gamepad_raii.rs"]
mod gamepad_raii;
/// Linux: virtual Steam Deck via UHID — the kernel `hid-steam` driver binds it as a real Deck.
#[cfg(target_os = "linux")]
#[path = "inject/linux/steam_controller.rs"]
pub mod steam_controller;
/// Linux: virtual Steam Deck via the USB gadget subsystem (`raw_gadget` + `dummy_hcd`) — the only
/// virtual-Deck transport Steam Input promotes (presents the controller on USB interface 2).
/// SteamOS-host only (needs `dummy_hcd` + `raw_gadget`).
#[cfg(target_os = "linux")]
#[path = "inject/linux/steam_gadget.rs"]
pub mod steam_gadget;
/// Transport-independent Steam Controller / Steam Deck HID contract (descriptor, byte-exact Deck
/// serializer, XInput/rich mappers, rumble parser), used by the Linux UHID backend ([`steam_controller`]).
#[cfg(target_os = "linux")]
#[path = "inject/proto/steam_proto.rs"]
pub mod steam_proto;
/// Pure fallback-remap policy (Steam-only inputs onto a non-Steam backend) + the Deck motion rescale.
#[cfg(target_os = "linux")]
#[path = "inject/proto/steam_remap.rs"]
pub mod steam_remap;
/// Linux: virtual Steam Deck over **USB/IP** (`vhci_hcd`) — the shippable, Secure-Boot-clean,
/// Steam-Input-promotable virtual-Deck transport on non-SteamOS hosts (Bazzite/generic), where
/// `dummy_hcd`/`raw_gadget` aren't built. In-tree + signed; no module build, no MOK.
#[cfg(target_os = "linux")]
#[path = "inject/linux/steam_usbip.rs"]
pub mod steam_usbip;
/// Stub — virtual gamepads need Linux uinput or the Windows UMDF drivers; events are dropped elsewhere.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub mod gamepad {
@@ -182,6 +182,9 @@ pub struct DualSenseManager {
last_write: Vec<Instant>,
/// Pad creation failed (e.g. /dev/uhid permissions) — warn once, drop events.
broken: bool,
/// Fallback policy for the Steam back grips a client may send (the DualSense has no back-button
/// HID slot). `PUNKTFUNK_STEAM_REMAP=paddles=…`; default drop.
remap: crate::inject::steam_remap::RemapConfig,
}
impl Default for DualSenseManager {
@@ -198,6 +201,7 @@ impl DualSenseManager {
last_rumble: vec![(0, 0); MAX_PADS],
last_write: vec![Instant::now(); MAX_PADS],
broken: false,
remap: crate::inject::steam_remap::RemapConfig::from_env(),
}
}
@@ -229,8 +233,12 @@ impl DualSenseManager {
// Merge buttons/sticks/triggers from the frame, preserving touch + motion (those
// come on the rich-input plane and must survive a button-only frame).
let prev = self.state[idx];
// Steam back grips have no DualSense slot — fold them onto standard buttons per the
// configured policy (default drop) so they aren't silently lost.
let buttons =
crate::inject::steam_remap::fold_paddles(f.buttons, self.remap.paddles);
let mut s = DsState::from_gamepad(
f.buttons,
buttons,
f.ls_x,
f.ls_y,
f.rs_x,
@@ -252,7 +260,9 @@ impl DualSenseManager {
/// arrived first); they're dropped if the pad isn't present.
pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich {
RichInput::Touchpad { pad, .. } | RichInput::Motion { pad, .. } => pad as usize,
RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
};
if idx >= MAX_PADS || self.pads[idx].is_none() {
return;
@@ -280,6 +290,26 @@ impl DualSenseManager {
self.state[idx].gyro = gyro;
self.state[idx].accel = accel;
}
RichInput::TouchpadEx {
surface,
finger,
touch,
x,
y,
..
} => {
// A Steam right/single pad maps onto the one DualSense touchpad (signed centre-0 →
// 0..=65535); surface 1 (the Steam left pad) has no DualSense equivalent.
if surface != 1 {
let slot = (finger as usize).min(1);
let n = |v: i16| ((v as i32) + 32768) as u32;
let t = &mut self.state[idx].touch[slot];
t.active = touch;
t.id = slot as u8;
t.x = (n(x) * (DS_TOUCH_W - 1) as u32 / u16::MAX as u32) as u16;
t.y = (n(y) * (DS_TOUCH_H - 1) as u32 / u16::MAX as u32) as u16;
}
}
}
self.write(idx);
}
@@ -367,6 +367,9 @@ pub struct DualShock4Manager {
last_write: Vec<Instant>,
/// Pad creation failed (e.g. /dev/uhid permissions) — warn once, drop events.
broken: bool,
/// Fallback policy for the Steam back grips a client may send (the DS4 has no back-button HID
/// slot). `PUNKTFUNK_STEAM_REMAP=paddles=…`; default drop.
remap: crate::inject::steam_remap::RemapConfig,
}
impl Default for DualShock4Manager {
@@ -384,6 +387,7 @@ impl DualShock4Manager {
last_led: vec![None; MAX_PADS],
last_write: vec![Instant::now(); MAX_PADS],
broken: false,
remap: crate::inject::steam_remap::RemapConfig::from_env(),
}
}
@@ -416,8 +420,12 @@ impl DualShock4Manager {
// Merge buttons/sticks/triggers, preserving touch + motion (those arrive on the
// rich-input plane and must survive a button-only frame).
let prev = self.state[idx];
// Steam back grips have no DS4 slot — fold them onto standard buttons per the
// configured policy (default drop) so they aren't silently lost.
let buttons =
crate::inject::steam_remap::fold_paddles(f.buttons, self.remap.paddles);
let mut s = DsState::from_gamepad(
f.buttons,
buttons,
f.ls_x,
f.ls_y,
f.rs_x,
@@ -439,7 +447,9 @@ impl DualShock4Manager {
/// pad isn't present.
pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich {
RichInput::Touchpad { pad, .. } | RichInput::Motion { pad, .. } => pad as usize,
RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
};
if idx >= MAX_PADS || self.pads[idx].is_none() {
return;
@@ -466,6 +476,26 @@ impl DualShock4Manager {
self.state[idx].gyro = gyro;
self.state[idx].accel = accel;
}
RichInput::TouchpadEx {
surface,
finger,
touch,
x,
y,
..
} => {
// A Steam right/single pad maps onto the one DS4 touchpad (signed centre-0 →
// 0..=65535); surface 1 (the Steam left pad) has no DS4 equivalent.
if surface != 1 {
let slot = (finger as usize).min(1);
let n = |v: i16| ((v as i32) + 32768) as u32;
let t = &mut self.state[idx].touch[slot];
t.active = touch;
t.id = slot as u8;
t.x = (n(x) * (DS4_TOUCH_W - 1) as u32 / u16::MAX as u32) as u16;
t.y = (n(y) * (DS4_TOUCH_H - 1) as u32 / u16::MAX as u32) as u16;
}
}
}
self.write(idx);
}
@@ -69,9 +69,16 @@ const BTN_START: u16 = 0x13b;
const BTN_MODE: u16 = 0x13c;
const BTN_THUMBL: u16 = 0x13d;
const BTN_THUMBR: u16 = 0x13e;
// Xbox-Elite paddle codes (the xpad convention SDL / Steam Input recognize). A client's back grips —
// and the GameStream `buttonFlags2` paddle bits, which were silently dropped before — land here, so
// the virtual X-Box pad exposes paddles like an Elite controller. PADDLE1/2/3/4 = R4/L4/R5/L5.
const BTN_TRIGGER_HAPPY5: u16 = 0x2c4;
const BTN_TRIGGER_HAPPY6: u16 = 0x2c5;
const BTN_TRIGGER_HAPPY7: u16 = 0x2c6;
const BTN_TRIGGER_HAPPY8: u16 = 0x2c7;
/// `(GameStream button bit, evdev key code)` — D-pad is emitted as HAT axes instead.
const BUTTON_MAP: [(u32, u16); 11] = [
const BUTTON_MAP: [(u32, u16); 15] = [
(gamepad::BTN_A, BTN_SOUTH),
(gamepad::BTN_B, BTN_EAST),
(gamepad::BTN_X, BTN_NORTH),
@@ -83,6 +90,10 @@ const BUTTON_MAP: [(u32, u16); 11] = [
(gamepad::BTN_GUIDE, BTN_MODE),
(gamepad::BTN_LS_CLK, BTN_THUMBL),
(gamepad::BTN_RS_CLK, BTN_THUMBR),
(gamepad::BTN_PADDLE1, BTN_TRIGGER_HAPPY5),
(gamepad::BTN_PADDLE2, BTN_TRIGGER_HAPPY6),
(gamepad::BTN_PADDLE3, BTN_TRIGGER_HAPPY7),
(gamepad::BTN_PADDLE4, BTN_TRIGGER_HAPPY8),
];
/// The USB identity a virtual uinput pad presents. SDL/Steam/Proton key their built-in mapping off
@@ -7,9 +7,14 @@
//! which the libei/portal path cannot. We connect as an ordinary Wayland client on the KWin session's
//! `$WAYLAND_DISPLAY` and translate events into fake-input requests; keyboard keys are raw Linux
//! evdev codes that KWin resolves through the session's own keymap (no keymap upload, unlike the wlr
//! virtual-keyboard path), and absolute pointer/touch coordinates are global compositor space — which
//! on a headless box (single per-session virtual output at the origin, scale 1) equals the streamed
//! output's pixels.
//! virtual-keyboard path), and absolute pointer/touch coordinates are global compositor space.
//!
//! Global compositor space is *logical* pixels (post display-scaling), which only equals the streamed
//! output's physical pixels at scale 1. Under a fractional/integer scale the logical edge sits at
//! `physical / scale`, so feeding the raw streamed pixel coordinate lands the cursor `scale×` too far
//! toward the bottom-right (top-left stays put). We therefore track each output's logical geometry
//! (position + size) via `xdg-output` and map the normalized client position into the matching
//! output's logical rectangle — the same shape the libei backend uses with its EI region.
#![allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
@@ -18,8 +23,14 @@
use super::{gs_button_to_evdev, vk_to_evdev, InputEvent, InputInjector};
use anyhow::{Context, Result};
use punktfunk_core::input::InputKind;
use std::time::{Duration, Instant};
use wayland_client::protocol::wl_output::{self, WlOutput};
use wayland_client::protocol::wl_registry::{self, WlRegistry};
use wayland_client::{Connection, Dispatch, EventQueue, Proxy, QueueHandle};
use wayland_client::{Connection, Dispatch, EventQueue, Proxy, QueueHandle, WEnum};
use wayland_protocols::xdg::xdg_output::zv1::client::{
zxdg_output_manager_v1::ZxdgOutputManagerV1,
zxdg_output_v1::{self, ZxdgOutputV1},
};
// Generate the client bindings for the vendored protocol XML inline (no build.rs), exactly like the
// KWin virtual-output backend. Path is relative to CARGO_MANIFEST_DIR.
@@ -48,10 +59,39 @@ const AXIS_HORIZONTAL: u32 = 1;
/// `code` value marking a horizontal scroll event (mirrors `gamestream::input` / the wlr backend).
const SCROLL_HORIZONTAL: u32 = 1;
/// One tracked output: its physical mode (to match the streamed resolution) and its logical geometry
/// (the global-compositor-space rectangle absolute coordinates are addressed in). `logical_w == 0`
/// means xdg-output hasn't reported its size yet.
struct OutputTrack {
/// Registry global id — also the dispatch user-data, so events route back to this entry.
name: u32,
wl_output: WlOutput,
xdg_output: Option<ZxdgOutputV1>,
/// Physical pixel mode from `wl_output.mode` (the `current` mode); matched against the streamed WxH.
mode_w: i32,
mode_h: i32,
/// Logical (post-scale) geometry from `xdg-output`.
logical_x: i32,
logical_y: i32,
logical_w: i32,
logical_h: i32,
}
/// Registry-bound globals (the Wayland dispatch state).
#[derive(Default)]
struct State {
fake: Option<FakeInput>,
xdg_mgr: Option<ZxdgOutputManagerV1>,
outputs: Vec<OutputTrack>,
}
impl State {
/// Create the `xdg_output` for a tracked output once both it and the manager exist.
fn ensure_xdg_output(o: &mut OutputTrack, mgr: &ZxdgOutputManagerV1, qh: &QueueHandle<State>) {
if o.xdg_output.is_none() {
o.xdg_output = Some(mgr.get_xdg_output(&o.wl_output, qh, o.name));
}
}
}
impl Dispatch<WlRegistry, ()> for State {
@@ -63,15 +103,57 @@ impl Dispatch<WlRegistry, ()> for State {
_: &Connection,
qh: &QueueHandle<Self>,
) {
if let wl_registry::Event::Global {
match event {
wl_registry::Event::Global {
name,
interface,
version,
} = event
{
if interface == "org_kde_kwin_fake_input" {
} => match interface.as_str() {
"org_kde_kwin_fake_input" => {
state.fake = Some(registry.bind(name, version.min(MAX_VERSION), qh, ()));
}
"wl_output" => {
// v1 carries `mode` (all we need); bind no higher than the proxy's max (4).
let wl_output: WlOutput = registry.bind(name, version.min(4), qh, name);
let mut o = OutputTrack {
name,
wl_output,
xdg_output: None,
mode_w: 0,
mode_h: 0,
logical_x: 0,
logical_y: 0,
logical_w: 0,
logical_h: 0,
};
if let Some(mgr) = state.xdg_mgr.clone() {
State::ensure_xdg_output(&mut o, &mgr, qh);
}
state.outputs.push(o);
}
"zxdg_output_manager_v1" => {
let mgr: ZxdgOutputManagerV1 = registry.bind(name, version.min(3), qh, ());
// Outputs bound before the manager have no xdg_output yet — create them now.
for o in state.outputs.iter_mut() {
State::ensure_xdg_output(o, &mgr, qh);
}
state.xdg_mgr = Some(mgr);
}
_ => {}
},
wl_registry::Event::GlobalRemove { name } => {
state.outputs.retain(|o| {
if o.name == name {
if let Some(x) = &o.xdg_output {
x.destroy();
}
false
} else {
true
}
});
}
_ => {}
}
}
}
@@ -89,13 +171,86 @@ impl Dispatch<FakeInput, ()> for State {
}
}
impl Dispatch<WlOutput, u32> for State {
fn event(
state: &mut Self,
_: &WlOutput,
event: wl_output::Event,
name: &u32,
_: &Connection,
_: &QueueHandle<Self>,
) {
// Only the *current* mode matters — a real monitor also advertises its other supported modes.
if let wl_output::Event::Mode {
flags: WEnum::Value(flags),
width,
height,
..
} = event
{
if flags.contains(wl_output::Mode::Current) {
if let Some(o) = state.outputs.iter_mut().find(|o| o.name == *name) {
o.mode_w = width;
o.mode_h = height;
}
}
}
}
}
impl Dispatch<ZxdgOutputV1, u32> for State {
fn event(
state: &mut Self,
_: &ZxdgOutputV1,
event: zxdg_output_v1::Event,
name: &u32,
_: &Connection,
_: &QueueHandle<Self>,
) {
if let Some(o) = state.outputs.iter_mut().find(|o| o.name == *name) {
match event {
zxdg_output_v1::Event::LogicalPosition { x, y } => {
o.logical_x = x;
o.logical_y = y;
}
zxdg_output_v1::Event::LogicalSize { width, height } => {
o.logical_w = width;
o.logical_h = height;
}
_ => {}
}
}
}
}
// The manager has no events.
impl Dispatch<ZxdgOutputManagerV1, ()> for State {
fn event(
_: &mut Self,
_: &ZxdgOutputManagerV1,
_: <ZxdgOutputManagerV1 as Proxy>::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
pub struct KwinFakeInjector {
conn: Connection,
queue: EventQueue<State>,
state: State,
fake: FakeInput,
/// When output geometry was last re-read; throttles the per-event roundtrip (see `refresh_geometry`).
last_refresh: Option<Instant>,
}
/// How often the fake_input backend re-reads output geometry from the compositor. Output add/remove
/// (a new session's virtual output) and live scale/resolution changes are infrequent, so a lazy
/// poll on the injector's own thread is plenty and adds at most one local-socket roundtrip twice a
/// second — versus a blocking roundtrip on every single mouse-move event.
const GEO_REFRESH: Duration = Duration::from_millis(500);
impl KwinFakeInjector {
pub fn open() -> Result<Self> {
let conn = Connection::connect_to_env()
@@ -122,13 +277,77 @@ impl KwinFakeInjector {
.context("fake_input authenticate roundtrip")?;
conn.flush().ok();
tracing::info!("KWin fake_input ready (headless keyboard/mouse/touch — no portal)");
Ok(Self {
// Settle output geometry (wl_output + xdg-output were bound during the registry roundtrip
// above; their logical_size arrives on a follow-up roundtrip). Best-effort — falls back to
// scale-1 mapping if xdg-output is absent.
let mut injector = Self {
conn,
queue,
state,
fake,
})
last_refresh: None,
};
injector.refresh_geometry();
tracing::info!(
outputs = injector.state.outputs.len(),
"KWin fake_input ready (headless keyboard/mouse/touch — no portal)"
);
Ok(injector)
}
/// Re-read output geometry, throttled to [`GEO_REFRESH`]. A `roundtrip` both flushes any pending
/// `get_xdg_output` requests and reads the geometry events back. A wl_output that *appeared* this
/// round only gets its xdg_output created mid-dispatch, so its `logical_size` lands on a later
/// roundtrip — keep going (bounded) until every output is settled.
fn refresh_geometry(&mut self) {
let now = Instant::now();
if let Some(t) = self.last_refresh {
if now.duration_since(t) < GEO_REFRESH {
return;
}
}
self.last_refresh = Some(now);
for _ in 0..3 {
if self.queue.roundtrip(&mut self.state).is_err() {
return;
}
let pending =
self.state.xdg_mgr.is_some() && self.state.outputs.iter().any(|o| o.logical_w == 0);
if !pending {
break;
}
}
}
/// Resolve the logical (global-compositor-space) rectangle to map a normalized client position
/// into. Prefer the output whose physical mode matches the streamed `phys_w`×`phys_h` (the
/// per-session virtual output); fall back to the sole output, then — if xdg-output is unavailable
/// — to the streamed pixels at the origin (the pre-scaling behavior, correct at scale 1).
fn logical_target(&self, phys_w: i32, phys_h: i32) -> (f64, f64, f64, f64) {
let usable = || {
self.state
.outputs
.iter()
.filter(|o| o.logical_w > 0 && o.logical_h > 0)
};
let chosen = usable()
.find(|o| o.mode_w == phys_w && o.mode_h == phys_h)
.or_else(|| {
let mut it = usable();
match (it.next(), it.next()) {
(Some(only), None) => Some(only),
_ => None,
}
});
match chosen {
Some(o) => (
o.logical_x as f64,
o.logical_y as f64,
o.logical_w as f64,
o.logical_h as f64,
),
None => (0.0, 0.0, phys_w as f64, phys_h as f64),
}
}
}
@@ -139,12 +358,17 @@ impl InputInjector for KwinFakeInjector {
self.fake.pointer_motion(event.x as f64, event.y as f64);
}
InputKind::MouseMoveAbs => {
let w = (event.flags >> 16) & 0xffff;
let h = event.flags & 0xffff;
let w = ((event.flags >> 16) & 0xffff) as i32;
let h = (event.flags & 0xffff) as i32;
if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64;
let y = event.y.clamp(0, h as i32) as f64;
self.fake.pointer_motion_absolute(x, y);
self.refresh_geometry();
let (lx, ly, lw, lh) = self.logical_target(w, h);
// Normalize in the streamed (physical) pixel space, then place inside the output's
// logical rectangle — so display scaling no longer offsets the cursor.
let nx = (event.x as f64 / w as f64).clamp(0.0, 1.0);
let ny = (event.y as f64 / h as f64).clamp(0.0, 1.0);
self.fake
.pointer_motion_absolute(lx + nx * lw, ly + ny * lh);
}
}
InputKind::MouseButtonDown | InputKind::MouseButtonUp => {
@@ -179,11 +403,15 @@ impl InputInjector for KwinFakeInjector {
// Touch: id = event.code, coords in the client surface w×h packed into flags (same
// absolute mapping as MouseMoveAbs). Each event is its own frame.
InputKind::TouchDown | InputKind::TouchMove => {
let w = (event.flags >> 16) & 0xffff;
let h = event.flags & 0xffff;
let w = ((event.flags >> 16) & 0xffff) as i32;
let h = (event.flags & 0xffff) as i32;
if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64;
let y = event.y.clamp(0, h as i32) as f64;
self.refresh_geometry();
let (lx, ly, lw, lh) = self.logical_target(w, h);
let nx = (event.x as f64 / w as f64).clamp(0.0, 1.0);
let ny = (event.y as f64 / h as f64).clamp(0.0, 1.0);
let x = lx + nx * lw;
let y = ly + ny * lh;
if event.kind == InputKind::TouchDown {
self.fake.touch_down(event.code, x, y);
} else {
@@ -0,0 +1,575 @@
//! Virtual Steam Deck controller via UHID — the Steam analogue of the virtual DualSense
//! ([`super::dualsense`]). A UHID device with Valve VID `28DE` / Deck PID `1205` is bound by the
//! kernel `hid-steam` driver, which exposes a full Steam Deck gamepad evdev (incl. the four back
//! grips) **plus** a separate IMU evdev, and — when Steam runs on the host — is re-grabbed by Steam
//! Input with native glyphs + trackpad/gyro/back-button bindings.
//!
//! The transport-independent contract (descriptor, byte-exact serializer, the `XInput`/rich
//! mappers, the rumble parser) lives in [`super::steam_proto`]; this module is the `/dev/uhid`
//! plumbing + the two Steam-specific lifecycle quirks the DualSense path lacks:
//!
//! 1. **`gamepad_mode` entry.** `steam_do_deck_input_event` early-returns under the default
//! `lizard_mode` until `gamepad_mode` is toggled on — which the kernel only does when the `b9.6`
//! Steam/menu-right button is held ~450 ms with no hidraw client open. So on the first pad we
//! best-effort clear `lizard_mode` via sysfs (needs root; bypasses the gate entirely) AND every
//! pad pulses `b9.6` for [`MODE_ENTER`] at creation. After that an **anti-toggle guard** caps any
//! continuous `b9.6` (a long in-game Start-hold) below the kernel's 450 ms threshold so play can
//! never accidentally flip `gamepad_mode` back off.
//! 2. **`UHID_SET_REPORT`.** Steam feedback (`0xEB` rumble) + the kernel's settings/serial writes
//! arrive as FEATURE set-reports that MUST be answered `err = 0`, or the kernel stalls ~5 s per
//! command (the DualSense backend only services GET_REPORT + OUTPUT).
use super::steam_proto::{
btn, parse_steam_output, serial_reply, serialize_deck_state, SteamState, STEAMDECK_PRODUCT,
STEAMDECK_RDESC, STEAM_REPORT_LEN, STEAM_VENDOR,
};
use crate::gamestream::gamepad::{GamepadEvent, MAX_PADS};
use anyhow::{Context, Result};
use punktfunk_core::quic::{HidOutput, RichInput};
use std::fs::{File, OpenOptions};
use std::io::{Read, Write};
use std::os::unix::fs::OpenOptionsExt;
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::{Duration, Instant};
// /dev/uhid event ABI — same layout as the DualSense backend.
const UHID_PATH: &str = "/dev/uhid";
const UHID_DESTROY: u32 = 1;
const UHID_OUTPUT: u32 = 6;
const UHID_GET_REPORT: u32 = 9;
const UHID_GET_REPORT_REPLY: u32 = 10;
const UHID_CREATE2: u32 = 11;
const UHID_INPUT2: u32 = 12;
const UHID_SET_REPORT: u32 = 13;
const UHID_SET_REPORT_REPLY: u32 = 14;
const HID_MAX_DESCRIPTOR_SIZE: usize = 4096;
const UHID_EVENT_SIZE: usize = 4 + 4372;
const BUS_USB: u16 = 0x03;
/// Hold the `b9.6` mode-switch this long at creation to toggle `gamepad_mode` on (the kernel needs
/// ~450 ms continuous; give margin).
const MODE_ENTER: Duration = Duration::from_millis(650);
/// Cap continuous `b9.6` (Start) below the kernel's 450 ms mode-switch threshold: after this long
/// we insert a one-frame release so an in-game long-Start-hold can't toggle `gamepad_mode` off.
const MENU_HOLD_CAP: Duration = Duration::from_millis(350);
fn put_cstr(ev: &mut [u8], off: usize, cap: usize, s: &str) {
let n = s.len().min(cap - 1);
ev[off..off + n].copy_from_slice(&s.as_bytes()[..n]);
}
/// Best-effort, once per process: clear `hid_steam`'s `lizard_mode` so `steam_do_deck_input_event`
/// stops gating on `gamepad_mode` (gamepad events then always flow). Needs root; on failure the
/// per-pad `b9.6` pulse + guard handle it instead.
fn try_clear_lizard_mode() {
static TRIED: AtomicBool = AtomicBool::new(false);
if TRIED.swap(true, Ordering::Relaxed) {
return;
}
match std::fs::write("/sys/module/hid_steam/parameters/lizard_mode", "N") {
Ok(()) => {
tracing::info!("cleared hid_steam lizard_mode (Steam Deck gamepad events always flow)")
}
Err(e) => tracing::debug!(
error = %e,
"could not clear hid_steam lizard_mode (no root?) — using the gamepad_mode pulse + guard"
),
}
}
/// A virtual Steam Deck backed by `/dev/uhid`. Dropping it destroys the device (the kernel tears
/// down the bound `hid-steam` interface + both evdevs).
pub struct SteamDeckPad {
fd: File,
seq: u32,
created: Instant,
/// When `b9.6` started being continuously held in our OUTPUT (anti-toggle guard); `None` = not.
menu_hold_since: Option<Instant>,
}
impl SteamDeckPad {
pub fn open(index: u8) -> Result<SteamDeckPad> {
try_clear_lizard_mode();
let fd = OpenOptions::new()
.read(true)
.write(true)
.custom_flags(libc::O_NONBLOCK)
.open(UHID_PATH)
.with_context(|| {
format!("open {UHID_PATH} (is the uhid udev rule installed + are you in 'input'?)")
})?;
let mut pad = SteamDeckPad {
fd,
seq: 0,
created: Instant::now(),
menu_hold_since: None,
};
pad.send_create2(index).context("UHID_CREATE2 Steam Deck")?;
Ok(pad)
}
fn send_create2(&mut self, index: u8) -> Result<()> {
let mut ev = [0u8; UHID_EVENT_SIZE];
ev[0..4].copy_from_slice(&UHID_CREATE2.to_ne_bytes());
put_cstr(&mut ev, 4, 128, &format!("Punktfunk Steam Deck {index}")); // name[128]
put_cstr(&mut ev, 132, 64, &format!("punktfunk/steam/{index}")); // phys[64]
put_cstr(&mut ev, 196, 64, &format!("punktfunk-steam-{index}")); // uniq[64]
ev[260..262].copy_from_slice(&(STEAMDECK_RDESC.len() as u16).to_ne_bytes()); // rd_size
ev[262..264].copy_from_slice(&BUS_USB.to_ne_bytes()); // bus
ev[264..268].copy_from_slice(&STEAM_VENDOR.to_ne_bytes());
ev[268..272].copy_from_slice(&STEAMDECK_PRODUCT.to_ne_bytes());
ev[272..276].copy_from_slice(&0x0100u32.to_ne_bytes()); // version
ev[276..280].copy_from_slice(&0u32.to_ne_bytes()); // country
ev[280..280 + STEAMDECK_RDESC.len()].copy_from_slice(STEAMDECK_RDESC);
self.fd.write_all(&ev).context("write UHID_CREATE2")?;
Ok(())
}
/// Serialize `st` (with the gamepad-mode entry overlay + anti-toggle guard applied) and write it.
pub fn write_state(&mut self, st: &SteamState) -> Result<()> {
self.seq = self.seq.wrapping_add(1);
let mut s = *st;
s.buttons = self.effective_buttons(st.buttons);
let mut r = [0u8; STEAM_REPORT_LEN];
serialize_deck_state(&mut r, &s, self.seq);
let mut ev = [0u8; UHID_EVENT_SIZE];
ev[0..4].copy_from_slice(&UHID_INPUT2.to_ne_bytes());
ev[4..6].copy_from_slice(&(r.len() as u16).to_ne_bytes()); // input2.size
ev[6..6 + r.len()].copy_from_slice(&r); // input2.data
self.fd.write_all(&ev).context("write UHID_INPUT2")?;
Ok(())
}
/// True while still pulsing the mode-switch at creation (the caller force-writes during this).
fn in_mode_entry(&self) -> bool {
self.created.elapsed() < MODE_ENTER
}
/// During mode entry, force `b9.6` held (override). Afterwards, pass the real buttons through but
/// drop `b9.6` for one frame whenever it's been continuously held past [`MENU_HOLD_CAP`].
fn effective_buttons(&mut self, mut buttons: u64) -> u64 {
if self.in_mode_entry() {
return btn::STEAM_MENU_RIGHT;
}
if buttons & btn::MENU != 0 {
let now = Instant::now();
match self.menu_hold_since {
None => self.menu_hold_since = Some(now),
Some(since) if now.duration_since(since) >= MENU_HOLD_CAP => {
buttons &= !btn::MENU; // one-frame release resets the kernel's mode-switch timer
self.menu_hold_since = None;
}
Some(_) => {}
}
} else {
self.menu_hold_since = None;
}
buttons
}
/// Service the device, non-blocking: answer the kernel's GET_REPORT (serial) + SET_REPORT
/// (settings / rumble — ack `err=0`) and parse any rumble feedback (`0xEB`, on either the
/// SET_REPORT or OUTPUT path) into `(low, high)` for the universal rumble plane.
pub fn service(&mut self) -> Option<(u16, u16)> {
let mut rumble = None;
let mut ev = [0u8; UHID_EVENT_SIZE];
while let Ok(n) = self.fd.read(&mut ev) {
if n < UHID_EVENT_SIZE {
break;
}
match u32::from_ne_bytes([ev[0], ev[1], ev[2], ev[3]]) {
UHID_OUTPUT => {
let size = u16::from_ne_bytes([ev[4100], ev[4101]]) as usize;
let end = 4 + size.min(HID_MAX_DESCRIPTOR_SIZE);
if let Some(r) = parse_steam_output(&ev[4..end]).rumble {
rumble = Some(r);
}
}
UHID_GET_REPORT => {
let id = u32::from_ne_bytes([ev[4], ev[5], ev[6], ev[7]]);
let _ = self.reply_get_report(id, &serial_reply("PUNKTFUNK01"));
}
UHID_SET_REPORT => {
let id = u32::from_ne_bytes([ev[4], ev[5], ev[6], ev[7]]);
// SET_REPORT data: [report-id 0, cmd, …] at ev[12..]. Surface rumble, then ack.
let end = (12 + 16).min(UHID_EVENT_SIZE);
if let Some(r) = parse_steam_output(&ev[12..end]).rumble {
rumble = Some(r);
}
let _ = self.reply_set_report(id);
}
_ => {} // Start/Stop/Open/Close — ignore
}
}
rumble
}
fn reply_get_report(&mut self, id: u32, data: &[u8]) -> Result<()> {
let mut ev = [0u8; UHID_EVENT_SIZE];
ev[0..4].copy_from_slice(&UHID_GET_REPORT_REPLY.to_ne_bytes());
ev[4..8].copy_from_slice(&id.to_ne_bytes());
ev[8..10].copy_from_slice(&0u16.to_ne_bytes()); // err 0
ev[10..12].copy_from_slice(&(data.len() as u16).to_ne_bytes());
ev[12..12 + data.len()].copy_from_slice(data);
self.fd.write_all(&ev).context("UHID_GET_REPORT_REPLY")?;
Ok(())
}
fn reply_set_report(&mut self, id: u32) -> Result<()> {
let mut ev = [0u8; UHID_EVENT_SIZE];
ev[0..4].copy_from_slice(&UHID_SET_REPORT_REPLY.to_ne_bytes());
ev[4..8].copy_from_slice(&id.to_ne_bytes());
ev[8..10].copy_from_slice(&0u16.to_ne_bytes()); // err 0 (ack)
self.fd.write_all(&ev).context("UHID_SET_REPORT_REPLY")?;
Ok(())
}
}
impl Drop for SteamDeckPad {
fn drop(&mut self) {
let mut ev = [0u8; UHID_EVENT_SIZE];
ev[0..4].copy_from_slice(&UHID_DESTROY.to_ne_bytes());
let _ = self.fd.write_all(&ev);
}
}
/// All virtual Steam Deck pads of a session — the Steam analogue of
/// [`DualSenseManager`](super::dualsense::DualSenseManager), selected with `PUNKTFUNK_GAMEPAD=steamdeck`.
/// Button/stick frames arrive via [`handle`](Self::handle); the right trackpad + motion via
/// [`apply_rich`](Self::apply_rich); [`pump`](Self::pump) services the kernel handshake + routes
/// rumble back; [`heartbeat`](Self::heartbeat) keeps the pad alive (and drives the mode-entry pulse).
/// The transport a manager pad drives. UHID is universal but Steam Input won't promote it (a UHID
/// device has no USB interface number, `Interface: -1`); the USB **gadget** (`raw_gadget`, SteamOS)
/// and **usbip** (`vhci_hcd`, universal) both present the controller on USB interface 2, which Steam
/// Input *does* promote. Selected per-pad by [`open_transport`].
enum DeckTransport {
Uhid(SteamDeckPad),
Gadget(crate::inject::steam_gadget::SteamDeckGadget),
Usbip(crate::inject::steam_usbip::SteamDeckUsbip),
}
impl DeckTransport {
fn write_state(&mut self, st: &SteamState) {
match self {
DeckTransport::Uhid(p) => {
let _ = p.write_state(st);
}
DeckTransport::Gadget(g) => g.write_state(st),
DeckTransport::Usbip(u) => u.write_state(st),
}
}
fn service(&mut self) -> Option<(u16, u16)> {
match self {
DeckTransport::Uhid(p) => p.service(),
DeckTransport::Gadget(g) => g.service().rumble,
DeckTransport::Usbip(u) => u.service().rumble,
}
}
fn in_mode_entry(&self) -> bool {
match self {
// Only the UHID pad needs the gamepad-mode entry pulse: the promoted transports are
// read raw via hidraw by Steam Input, which bypasses the kernel's evdev mode gate.
DeckTransport::Uhid(p) => p.in_mode_entry(),
DeckTransport::Gadget(_) | DeckTransport::Usbip(_) => false,
}
}
}
/// Open the best Steam-Input-promotable Deck transport available, in preference order:
/// **`raw_gadget` (SteamOS validated fast-path) → `usbip`/`vhci_hcd` (universal, Secure-Boot-clean)
/// → UHID (universal, but `Interface: -1` so Steam Input won't promote it).** Each rung degrades to
/// the next on failure, so a host lacking the gadget modules still gets a *promotable* Deck via
/// usbip, and one lacking both still gets a working (if non-promoted) UHID pad.
fn open_transport(idx: u8) -> Result<DeckTransport> {
use crate::inject::{steam_gadget, steam_usbip};
// 1. raw_gadget — the validated SteamOS fast-path (default on there).
if steam_gadget::gadget_preferred() {
steam_gadget::ensure_modules();
match steam_gadget::SteamDeckGadget::open(idx) {
Ok(g) => {
tracing::info!(
index = idx,
"virtual Steam Deck created (USB gadget — Steam Input recognizes it)"
);
return Ok(DeckTransport::Gadget(g));
}
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "USB-gadget Deck unavailable — trying usbip")
}
}
}
// 2. usbip/vhci_hcd — the universal, in-tree, Secure-Boot-clean transport (default on elsewhere).
if steam_usbip::usbip_preferred() {
match steam_usbip::SteamDeckUsbip::open(idx) {
Ok(u) => return Ok(DeckTransport::Usbip(u)),
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "usbip Deck unavailable — falling back to UHID")
}
}
}
// 3. UHID — universal fallback (works everywhere; Steam Input won't promote it).
let p = SteamDeckPad::open(idx)?;
tracing::info!(
index = idx,
"virtual Steam Deck created (UHID hid-steam — not Steam-Input-promoted)"
);
Ok(DeckTransport::Uhid(p))
}
pub struct SteamControllerManager {
pads: Vec<Option<DeckTransport>>,
state: Vec<SteamState>,
last_rumble: Vec<(u16, u16)>,
last_write: Vec<Instant>,
broken: bool,
}
impl Default for SteamControllerManager {
fn default() -> SteamControllerManager {
SteamControllerManager::new()
}
}
impl SteamControllerManager {
pub fn new() -> SteamControllerManager {
SteamControllerManager {
pads: (0..MAX_PADS).map(|_| None).collect(),
state: vec![SteamState::neutral(); MAX_PADS],
last_rumble: vec![(0, 0); MAX_PADS],
last_write: vec![Instant::now(); MAX_PADS],
broken: false,
}
}
pub fn handle(&mut self, ev: &GamepadEvent) {
match ev {
GamepadEvent::Arrival { index, kind, .. } => {
tracing::info!(index, kind, "controller arrival (Steam Deck)");
self.ensure(*index as usize);
}
GamepadEvent::State(f) => {
let idx = f.index as usize;
if idx >= MAX_PADS {
return;
}
for (i, slot) in self.pads.iter_mut().enumerate() {
if slot.is_some() && f.active_mask & (1 << i) == 0 {
tracing::info!(index = i, "controller unplugged (Steam Deck)");
*slot = None;
self.state[i] = SteamState::neutral();
self.last_rumble[i] = (0, 0);
}
}
if f.active_mask & (1 << idx) == 0 {
return;
}
self.ensure(idx);
// Merge buttons/sticks/triggers, preserving the rich-plane fields (trackpad + motion
// arrive separately and must survive a button-only frame).
let prev = self.state[idx];
let mut s = SteamState::from_gamepad(
f.buttons,
f.ls_x,
f.ls_y,
f.rs_x,
f.rs_y,
f.left_trigger,
f.right_trigger,
);
s.rpad_x = prev.rpad_x;
s.rpad_y = prev.rpad_y;
s.lpad_x = prev.lpad_x;
s.lpad_y = prev.lpad_y;
s.gyro = prev.gyro;
s.accel = prev.accel;
s.buttons |= prev.buttons & (btn::RPAD_TOUCH | btn::LPAD_TOUCH);
self.state[idx] = s;
self.write(idx);
}
}
}
/// Apply a rich client→host event (right trackpad / motion) to an existing pad.
pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich {
RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
};
if idx >= MAX_PADS || self.pads[idx].is_none() {
return;
}
self.state[idx].apply_rich(rich);
self.write(idx);
}
fn write(&mut self, idx: usize) {
let st = self.state[idx];
if let Some(pad) = self.pads[idx].as_mut() {
pad.write_state(&st);
}
self.last_write[idx] = Instant::now();
}
/// Re-emit each live pad's current report when silent past `max_gap`, and force a steady stream
/// while a pad is still pulsing its gamepad-mode entry (so the `b9.6` toggle completes even with
/// no game input).
pub fn heartbeat(&mut self, max_gap: Duration) {
let now = Instant::now();
for i in 0..self.pads.len() {
let Some(pad) = self.pads[i].as_ref() else {
continue;
};
if pad.in_mode_entry() || now.duration_since(self.last_write[i]) >= max_gap {
self.write(i);
}
}
}
fn ensure(&mut self, idx: usize) {
if idx >= MAX_PADS || self.pads[idx].is_some() || self.broken {
return;
}
match open_transport(idx as u8) {
Ok(t) => {
self.pads[idx] = Some(t);
self.state[idx] = SteamState::neutral();
self.last_rumble[idx] = (0, 0);
self.last_write[idx] = Instant::now();
}
Err(e) => {
tracing::error!(error = %format!("{e:#}"), "virtual Steam Deck creation failed — controller input disabled");
self.broken = true;
}
}
}
/// Service every pad: answer the kernel handshake and forward rumble on the universal plane.
/// `rumble` fires `(index, low, high)` only on a level change. The Steam Deck has no rich
/// host→client feedback plane (no lightbar / adaptive triggers), so `hidout` goes unused.
pub fn pump(&mut self, mut rumble: impl FnMut(u16, u16, u16), _hidout: impl FnMut(HidOutput)) {
for i in 0..self.pads.len() {
let Some(pad) = self.pads[i].as_mut() else {
continue;
};
if let Some(r) = pad.service() {
if self.last_rumble[i] != r {
self.last_rumble[i] = r;
rumble(i as u16, r.0, r.1);
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Find the evdev node for a kernel input device by exact name (e.g. `"Steam Deck"`).
fn find_node(name: &str) -> Option<String> {
let devs = std::fs::read_to_string("/proc/bus/input/devices").ok()?;
for block in devs.split("\n\n") {
if !block
.lines()
.any(|l| l.trim() == format!("N: Name=\"{name}\""))
{
continue;
}
for l in block.lines() {
if let Some(h) = l.strip_prefix("H: Handlers=") {
if let Some(ev) = h.split_whitespace().find(|t| t.starts_with("event")) {
return Some(format!("/dev/input/{ev}"));
}
}
}
}
None
}
/// Read the evdev's current key bitmap (`EVIOCGKEY`) and test whether `code` is down.
fn key_is_down(node: &str, code: u16) -> bool {
use std::os::unix::io::AsRawFd;
let Ok(f) = std::fs::File::open(node) else {
return false;
};
let mut bits = [0u8; 96];
const EVIOCGKEY: libc::c_ulong = (2 << 30) | (96 << 16) | (0x45 << 8) | 0x18;
// SAFETY: EVIOCGKEY copies the current key-state bitmap of the evdev behind the valid fd
// `f` into `bits`; 96 bytes covers KEY_MAX/8, so the kernel never writes past the buffer.
let rc = unsafe { libc::ioctl(f.as_raw_fd(), EVIOCGKEY, bits.as_mut_ptr()) };
rc >= 0 && (bits[(code / 8) as usize] >> (code % 8)) & 1 == 1
}
/// Read the current value of an absolute axis (`EVIOCGABS`) — the first `i32` of `input_absinfo`.
fn abs_value(node: &str, abs: u16) -> Option<i32> {
use std::os::unix::io::AsRawFd;
let f = std::fs::File::open(node).ok()?;
let mut info = [0u8; 24]; // struct input_absinfo { value, min, max, fuzz, flat, resolution }
let req: libc::c_ulong =
(2 << 30) | (24 << 16) | (0x45 << 8) | (0x40 + abs as libc::c_ulong);
// SAFETY: EVIOCGABS fills the 24-byte input_absinfo for the valid evdev fd `f`; we read only
// the leading i32 `value`. The buffer is exactly sizeof(input_absinfo), so no overflow.
let rc = unsafe { libc::ioctl(f.as_raw_fd(), req, info.as_mut_ptr()) };
(rc >= 0).then(|| i32::from_ne_bytes([info[0], info[1], info[2], info[3]]))
}
/// On-box smoke test for the real backend: a `SteamDeckPad` must bind `hid-steam` (creating both
/// the gamepad + IMU evdevs), enter `gamepad_mode` via the creation pulse, and land a held button
/// on the evdev (`BTN_A`, code 0x130) — proving the entry overlay + byte-exact serialize path —
/// then tear the device down on drop. Touches `/dev/uhid`, so it is `#[ignore]`d in CI; run on a
/// box with `hid-steam` + `input`-group access: `cargo test -p punktfunk-host -- --ignored`.
#[test]
#[ignore = "creates a real /dev/uhid device; needs hid-steam + the input group"]
fn backend_binds_and_input_flows() {
use punktfunk_core::input::gamepad as gs;
const BTN_A: u16 = 0x130;
const ABS_HAT0X: u16 = 0x10; // left trackpad X
let mut pad = SteamDeckPad::open(0).expect("open SteamDeckPad (/dev/uhid + input group?)");
// Drive the full M3 wire path: build state through `from_gamepad` (BTN_A + the L4 back grip)
// and `apply_rich` (a left-pad TouchpadEx contact), then hold it past MODE_ENTER (the b9.6
// pulse), servicing the handshake.
let mut st = SteamState::from_gamepad(gs::BTN_A | gs::BTN_PADDLE2, 0, 0, 0, 0, 0, 0);
st.apply_rich(RichInput::TouchpadEx {
pad: 0,
surface: 1,
finger: 0,
touch: true,
click: false,
x: -8000,
y: 9000,
pressure: 0,
});
let start = Instant::now();
while start.elapsed() < Duration::from_millis(1200) {
let _ = pad.service();
pad.write_state(&st).expect("write_state");
std::thread::sleep(Duration::from_millis(4));
}
let devs = std::fs::read_to_string("/proc/bus/input/devices").unwrap_or_default();
assert!(devs.contains("Steam Deck"), "gamepad evdev not created");
assert!(
devs.contains("Steam Deck Motion Sensors"),
"IMU evdev not created"
);
let node = find_node("Steam Deck").expect("gamepad evdev node");
assert!(
key_is_down(&node, BTN_A),
"BTN_A not down — gamepad_mode entry or serialize failed"
);
// The left trackpad contact (TouchpadEx surface 1, gated on LPAD_TOUCH) reaches ABS_HAT0X.
assert_eq!(
abs_value(&node, ABS_HAT0X),
Some(-8000),
"left trackpad (TouchpadEx surface 1) did not reach ABS_HAT0X"
);
drop(pad);
std::thread::sleep(Duration::from_millis(200));
let devs = std::fs::read_to_string("/proc/bus/input/devices").unwrap_or_default();
assert!(
!devs.contains("Steam Deck Motion Sensors"),
"device not torn down on drop"
);
}
}
@@ -0,0 +1,558 @@
//! Virtual Steam Deck via the USB **gadget** subsystem (`raw_gadget` + `dummy_hcd`) — the only
//! virtual-Deck transport Steam Input recognizes.
//!
//! The UHID [`super::steam_controller::SteamDeckPad`] binds the kernel `hid-steam` driver, but Steam's
//! own controller driver filters the Deck's controller to USB **interface 2**, and a UHID device has no
//! USB interface number (`Interface: -1`), so Steam enumerates it but never promotes it. This backend
//! instead presents a *real* 3-interface USB Deck (mouse = interface 0, keyboard = 1, **controller =
//! 2**) on a `dummy_hcd` loopback UDC, driven from userspace via `/dev/raw-gadget` so we can answer
//! every control transfer (including the HID feature reports `f_hid` can't). Proven on a real Deck:
//! hid-steam binds it, Steam reserves an XInput slot and emits an X-Box pad. Descriptors are captured
//! verbatim from a physical Deck; see `packaging/linux/steam-deck-gadget/` for the original PoC + the
//! USB-stack gotchas. **SteamOS-host only** (needs `dummy_hcd` + `raw_gadget`, which SteamOS ships).
//!
//! The transport here is self-contained (libc + std); the report bytes it streams are produced by
//! [`super::steam_proto`] in the wrapping backend.
use anyhow::{bail, Context, Result};
use std::mem::size_of;
use std::os::fd::RawFd;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::thread::JoinHandle;
// ---- raw_gadget UAPI (mirrors linux/usb/raw_gadget.h; inlined like the C PoC) ----
const UDC_NAME_MAX: usize = 128;
#[repr(C)]
struct UsbRawInit {
driver_name: [u8; UDC_NAME_MAX],
device_name: [u8; UDC_NAME_MAX],
speed: u8,
}
// usb_raw_event { u32 type; u32 length; u8 data[]; } — we read it into a fixed buffer.
const EVENT_HDR: usize = 8; // type + length
const EVENT_BUF: usize = EVENT_HDR + 64; // setup packet (8) fits easily
// usb_raw_ep_io { u16 ep; u16 flags; u32 length; u8 data[]; }
const EPIO_HDR: usize = 8;
// usb_endpoint_descriptor is 9 bytes in the kernel (audio bRefresh/bSynchAddress); EP_ENABLE wants it.
#[repr(C, packed)]
#[derive(Clone, Copy, Default)]
struct UsbEndpointDescriptor {
b_length: u8,
b_descriptor_type: u8,
b_endpoint_address: u8,
bm_attributes: u8,
w_max_packet_size: u16,
b_interval: u8,
b_refresh: u8,
b_synch_address: u8,
}
const fn ioc(dir: u64, nr: u64, size: usize) -> libc::c_ulong {
((dir << 30) | ((size as u64) << 16) | ((b'U' as u64) << 8) | nr) as libc::c_ulong
}
const IOCTL_INIT: libc::c_ulong = ioc(1, 0, size_of::<UsbRawInit>());
const IOCTL_RUN: libc::c_ulong = ioc(0, 1, 0);
const IOCTL_EVENT_FETCH: libc::c_ulong = ioc(2, 2, EVENT_HDR); // size is the header; kernel copies more
const IOCTL_EP0_WRITE: libc::c_ulong = ioc(1, 3, EPIO_HDR);
const IOCTL_EP0_READ: libc::c_ulong = ioc(2 | 1, 4, EPIO_HDR); // _IOWR
const IOCTL_EP_ENABLE: libc::c_ulong = ioc(1, 5, size_of::<UsbEndpointDescriptor>());
const IOCTL_EP_WRITE: libc::c_ulong = ioc(1, 7, EPIO_HDR);
const IOCTL_CONFIGURE: libc::c_ulong = ioc(0, 9, 0);
const IOCTL_VBUS_DRAW: libc::c_ulong = ioc(1, 10, 4);
const IOCTL_EP0_STALL: libc::c_ulong = ioc(0, 12, 0);
const USB_RAW_EVENT_CONNECT: u32 = 1;
const USB_RAW_EVENT_CONTROL: u32 = 2;
const USB_SPEED_HIGH: u8 = 3;
// Captured-from-hardware Deck descriptors + the `0x83`/`0xAE` feature contract live in the shared
// [`super::steam_proto`] module (single source of truth, also used by the usbip transport).
use super::steam_proto::{
deck_serial, deck_unit_id, feature_reply, neutral_deck_report, RDESC_DECK_CTRL as RDESC_CTRL,
RDESC_DECK_KBD as RDESC_KBD, RDESC_DECK_MOUSE as RDESC_MOUSE,
};
const DEV_DESC: [u8; 18] = [
18, 1, 0x00, 0x02, // bLength, DEVICE, bcdUSB 2.00
0, 0, 0, 64, // class/sub/proto, bMaxPacketSize0
0xDE, 0x28, 0x05, 0x12, // idVendor 28DE, idProduct 1205
0x00, 0x03, // bcdDevice 3.00
1, 2, 3, 1, // iManufacturer, iProduct, iSerial, bNumConfigurations
];
const HID_DT: u8 = 0x21;
const HID_RPT_DT: u8 = 0x22;
/// Assemble the 84-byte config descriptor: config + 3×(interface + HID + 7-byte endpoint).
fn build_config() -> Vec<u8> {
let mut c = Vec::with_capacity(84);
// config descriptor (wTotalLength patched after)
c.extend_from_slice(&[9, 2, 84, 0, 3, 1, 0, 0x80, 250]);
// helper closures
let iface = |n: u8, sub: u8, proto: u8| [9u8, 4, n, 0, 1, 3, sub, proto, 0];
let hid = |rlen: u16, country: u8| {
[
9u8,
HID_DT,
0x10,
0x01,
country,
1,
HID_RPT_DT,
(rlen & 0xff) as u8,
(rlen >> 8) as u8,
]
};
let ep = |addr: u8, mps: u16| [7u8, 5, addr, 0x03, (mps & 0xff) as u8, (mps >> 8) as u8, 4];
// interface 0: mouse, EP 0x81
c.extend_from_slice(&iface(0, 0, 2));
c.extend_from_slice(&hid(RDESC_MOUSE.len() as u16, 0));
c.extend_from_slice(&ep(0x81, 8));
// interface 1: keyboard (boot), EP 0x82
c.extend_from_slice(&iface(1, 1, 1));
c.extend_from_slice(&hid(RDESC_KBD.len() as u16, 0));
c.extend_from_slice(&ep(0x82, 8));
// interface 2: controller, EP 0x83, bCountryCode 33
c.extend_from_slice(&iface(2, 0, 0));
c.extend_from_slice(&hid(RDESC_CTRL.len() as u16, 33));
c.extend_from_slice(&ep(0x83, 64));
debug_assert_eq!(c.len(), 84);
c
}
fn string_desc(idx: u8, serial: &str) -> Vec<u8> {
if idx == 0 {
return vec![4, 3, 0x09, 0x04]; // LANGID en-US
}
let s: &str = match idx {
1 => "Valve Software",
2 => "Steam Deck Controller",
3 => serial,
_ => "",
};
let mut v = vec![(2 + s.len() * 2) as u8, 3];
for ch in s.encode_utf16() {
v.push((ch & 0xff) as u8);
v.push((ch >> 8) as u8);
}
v
}
// ---- ioctl wrappers (the only unsafe surface for the raw_gadget UAPI; documented once) ----
fn ioctl_ptr<T>(fd: RawFd, req: libc::c_ulong, arg: *const T) -> i32 {
// SAFETY: `fd` is our open /dev/raw-gadget descriptor; `arg` points to a correctly-sized,
// initialized argument for `req` (a raw_gadget UAPI struct or an owned usb_raw_ep_io buffer)
// that lives for the duration of the call. `ioctl` is variadic, so passing a thin pointer is ABI-correct.
unsafe { libc::ioctl(fd, req as _, arg) as i32 }
}
fn ioctl_mut<T>(fd: RawFd, req: libc::c_ulong, arg: *mut T) -> i32 {
// SAFETY: as `ioctl_ptr`, but `arg` is a writable buffer the kernel fills for `req` (EVENT_FETCH / EP0_READ).
unsafe { libc::ioctl(fd, req as _, arg) as i32 }
}
fn ioctl_val(fd: RawFd, req: libc::c_ulong, val: libc::c_ulong) -> i32 {
// SAFETY: `req` (VBUS_DRAW) takes an integer argument by value; `fd` is our descriptor.
unsafe { libc::ioctl(fd, req as _, val) as i32 }
}
fn ioctl_none(fd: RawFd, req: libc::c_ulong) -> i32 {
// SAFETY: `req` (RUN / CONFIGURE / EP0_STALL) takes no argument, but raw_gadget rejects a non-zero
// `value` with EINVAL — pass an explicit 0 (an omitted vararg would be an indeterminate register).
unsafe { libc::ioctl(fd, req as _, 0) as i32 }
}
// ---- low-level ep0 helpers (operate on the shared fd) ----
fn ep0_write(fd: RawFd, data: &[u8]) -> i32 {
let mut buf = vec![0u8; EPIO_HDR + data.len()];
buf[0..2].copy_from_slice(&0u16.to_ne_bytes()); // ep 0
buf[4..8].copy_from_slice(&(data.len() as u32).to_ne_bytes());
buf[EPIO_HDR..].copy_from_slice(data);
ioctl_ptr(fd, IOCTL_EP0_WRITE, buf.as_ptr())
}
fn ep0_read(fd: RawFd, len: usize) -> (i32, Vec<u8>) {
let mut buf = vec![0u8; EPIO_HDR + len.max(1)];
buf[4..8].copy_from_slice(&(len as u32).to_ne_bytes());
let r = ioctl_mut(fd, IOCTL_EP0_READ, buf.as_mut_ptr());
let n = if r > 0 { r as usize } else { 0 };
(r, buf[EPIO_HDR..EPIO_HDR + n.min(len.max(1))].to_vec())
}
/// Complete a no-data OUT control (status stage is an IN, handled by a zero-length read).
fn ep0_ack(fd: RawFd) {
ep0_read(fd, 0);
}
fn ep0_stall(fd: RawFd) {
ioctl_none(fd, IOCTL_EP0_STALL);
}
/// Owns the `/dev/raw-gadget` fd; closing it tears the device down.
struct GadgetFd(RawFd);
impl Drop for GadgetFd {
fn drop(&mut self) {
// SAFETY: `self.0` is the fd we opened in `SteamDeckGadget::open` and own uniquely here.
unsafe { libc::close(self.0) };
}
}
/// A virtual Steam Deck presented over the USB gadget subsystem. Dropping it stops the threads and
/// closes the gadget (the kernel tears down the device).
pub struct SteamDeckGadget {
report: Arc<Mutex<[u8; 64]>>,
feedback: Arc<Mutex<super::steam_proto::SteamFeedback>>,
running: Arc<AtomicBool>,
threads: Vec<JoinHandle<()>>,
_fd: Arc<GadgetFd>,
seq: u32,
}
impl SteamDeckGadget {
/// Bind a virtual Deck on a fresh `dummy_hcd` UDC. `index` only varies the serial. Requires
/// `dummy_hcd` + `raw_gadget` loaded and write access to `/dev/raw-gadget` (root on SteamOS).
pub fn open(index: u8) -> Result<SteamDeckGadget> {
// SAFETY: opening a constant NUL-terminated device path with O_RDWR; returns a fd or -1.
let fd = unsafe { libc::open(c"/dev/raw-gadget".as_ptr(), libc::O_RDWR) };
if fd < 0 {
bail!(
"open /dev/raw-gadget ({}) — is raw_gadget+dummy_hcd loaded and are we root?",
std::io::Error::last_os_error()
);
}
let fd = Arc::new(GadgetFd(fd));
let raw = fd.0;
// INIT against the dummy UDC, then RUN.
// SAFETY: `UsbRawInit` is a plain-old-data struct (byte arrays + u8); all-zero is a valid value.
let mut init: UsbRawInit = unsafe { std::mem::zeroed() };
copy_cstr(&mut init.driver_name, "dummy_udc");
copy_cstr(&mut init.device_name, "dummy_udc.0");
init.speed = USB_SPEED_HIGH;
if ioctl_ptr(raw, IOCTL_INIT, &init as *const _) < 0 {
bail!("raw_gadget INIT: {}", std::io::Error::last_os_error());
}
if ioctl_none(raw, IOCTL_RUN) < 0 {
bail!("raw_gadget RUN: {}", std::io::Error::last_os_error());
}
let serial = deck_serial(index);
let unit_id = deck_unit_id(index); // "PF" + index — a synthetic per-instance device id
let report = Arc::new(Mutex::new(neutral_deck_report()));
let feedback = Arc::new(Mutex::new(Default::default()));
let running = Arc::new(AtomicBool::new(true));
let ctrl_ep = Arc::new(std::sync::atomic::AtomicI32::new(-1));
let configured = Arc::new(AtomicBool::new(false));
// Control thread: enumerate + answer every control transfer.
let control = {
let fd = fd.clone();
let running = running.clone();
let ctrl_ep = ctrl_ep.clone();
let configured = configured.clone();
let feedback = feedback.clone();
std::thread::Builder::new()
.name("pf-deck-gadget-ctrl".into())
.spawn(move || {
control_loop(fd, running, ctrl_ep, configured, feedback, serial, unit_id)
})
.context("spawn gadget control thread")?
};
// Stream thread: push the current report on the controller interrupt-IN endpoint.
let stream = {
let fd = fd.clone();
let running = running.clone();
let ctrl_ep = ctrl_ep.clone();
let configured = configured.clone();
let report = report.clone();
std::thread::Builder::new()
.name("pf-deck-gadget-stream".into())
.spawn(move || stream_loop(fd, running, ctrl_ep, configured, report))
.context("spawn gadget stream thread")?
};
Ok(SteamDeckGadget {
report,
feedback,
running,
threads: vec![control, stream],
_fd: fd,
seq: 0,
})
}
/// Serialize `st` into the 64-byte Deck state report streamed to the kernel.
pub fn write_state(&mut self, st: &super::steam_proto::SteamState) {
self.seq = self.seq.wrapping_add(1);
let mut r = [0u8; 64];
super::steam_proto::serialize_deck_state(&mut r, st, self.seq);
if let Ok(mut g) = self.report.lock() {
*g = r;
}
}
/// Drain any feedback (rumble) the kernel/Steam wrote to the device.
pub fn service(&mut self) -> super::steam_proto::SteamFeedback {
self.feedback
.lock()
.map(|mut f| std::mem::take(&mut *f))
.unwrap_or_default()
}
}
impl Drop for SteamDeckGadget {
fn drop(&mut self) {
self.running.store(false, Ordering::SeqCst);
for t in self.threads.drain(..) {
let _ = t.join();
}
}
}
fn copy_cstr(dst: &mut [u8], s: &str) {
let n = s.len().min(dst.len() - 1);
dst[..n].copy_from_slice(&s.as_bytes()[..n]);
}
fn control_loop(
fd: Arc<GadgetFd>,
running: Arc<AtomicBool>,
ctrl_ep: Arc<std::sync::atomic::AtomicI32>,
configured: Arc<AtomicBool>,
feedback: Arc<Mutex<super::steam_proto::SteamFeedback>>,
serial: String,
unit_id: u32,
) {
let raw = fd.0;
let cfg = build_config();
let mut last_set: Vec<u8> = Vec::new();
let mut evbuf = [0u8; EVENT_BUF];
while running.load(Ordering::SeqCst) {
// EVENT_FETCH: type(4) length(4) data[].
evbuf[4..8].copy_from_slice(&(8u32).to_ne_bytes()); // request setup-sized payload
let r = ioctl_mut(raw, IOCTL_EVENT_FETCH, evbuf.as_mut_ptr());
if r < 0 {
if running.load(Ordering::SeqCst) {
// transient; brief backoff
std::thread::sleep(std::time::Duration::from_millis(2));
}
continue;
}
let etype = u32::from_ne_bytes([evbuf[0], evbuf[1], evbuf[2], evbuf[3]]);
match etype {
USB_RAW_EVENT_CONNECT => {}
USB_RAW_EVENT_CONTROL => {
let s = &evbuf[EVENT_HDR..EVENT_HDR + 8];
let ctrl = Setup {
bm_request_type: s[0],
b_request: s[1],
w_value: u16::from_le_bytes([s[2], s[3]]),
w_index: u16::from_le_bytes([s[4], s[5]]),
w_length: u16::from_le_bytes([s[6], s[7]]),
};
handle_control(
raw,
&ctrl,
&cfg,
&serial,
unit_id,
&ctrl_ep,
&configured,
&mut last_set,
&feedback,
);
}
_ => {}
}
}
}
struct Setup {
bm_request_type: u8,
b_request: u8,
w_value: u16,
w_index: u16,
w_length: u16,
}
#[allow(clippy::too_many_arguments)]
fn handle_control(
raw: RawFd,
ctrl: &Setup,
cfg: &[u8],
serial: &str,
unit_id: u32,
ctrl_ep: &std::sync::atomic::AtomicI32,
configured: &AtomicBool,
last_set: &mut Vec<u8>,
feedback: &Mutex<super::steam_proto::SteamFeedback>,
) {
let idx = (ctrl.w_index & 0xff) as u8;
let type_class = ctrl.bm_request_type & 0x60;
let wl = ctrl.w_length as usize;
if type_class == 0x00 {
// standard
match ctrl.b_request {
0x06 => {
// GET_DESCRIPTOR
let dt = (ctrl.w_value >> 8) as u8;
let di = (ctrl.w_value & 0xff) as u8;
let resp: Vec<u8> = match dt {
1 => DEV_DESC.to_vec(),
2 => cfg.to_vec(),
3 => string_desc(di, serial),
HID_RPT_DT => match idx {
0 => RDESC_MOUSE.to_vec(),
1 => RDESC_KBD.to_vec(),
_ => RDESC_CTRL.to_vec(),
},
HID_DT => {
// re-emit the interface's HID descriptor from the config blob (best effort)
hid_desc_for(cfg, idx)
}
_ => {
ep0_stall(raw);
return;
}
};
let n = resp.len().min(wl);
ep0_write(raw, &resp[..n]);
}
0x09 => {
// SET_CONFIGURATION
ioctl_val(raw, IOCTL_VBUS_DRAW, 0x32);
ioctl_none(raw, IOCTL_CONFIGURE);
enable_endpoints(raw, ctrl_ep);
ep0_ack(raw);
configured.store(true, Ordering::SeqCst);
}
0x0b => ep0_ack(raw), // SET_INTERFACE
0x00 => {
let st = 0u16;
ep0_write(raw, &st.to_le_bytes());
}
_ => ep0_stall(raw),
}
} else if type_class == 0x20 {
// HID class
match ctrl.b_request {
0x01 => {
// GET_REPORT — serve the Deck feature reply for the last requested command.
let resp = feature_reply(last_set, serial, unit_id);
let n = resp.len().min(wl);
ep0_write(raw, &resp[..n]);
}
0x09 => {
// SET_REPORT — read the host's data; remember it + extract feedback.
let (r, data) = ep0_read(raw, wl);
if r > 0 {
*last_set = data.clone();
// parse_steam_output expects [report-id(0), cmd, …]; EP0 OUT data is [cmd, …].
let mut framed = Vec::with_capacity(data.len() + 1);
framed.push(0);
framed.extend_from_slice(&data);
let fb = super::steam_proto::parse_steam_output(&framed);
if fb.rumble.is_some() {
if let Ok(mut g) = feedback.lock() {
*g = fb;
}
}
}
}
0x0a | 0x0b => ep0_ack(raw), // SET_IDLE / SET_PROTOCOL
0x03 => {
ep0_write(raw, &[0u8]);
} // GET_PROTOCOL
_ => ep0_stall(raw),
}
} else {
ep0_stall(raw);
}
}
fn hid_desc_for(cfg: &[u8], idx: u8) -> Vec<u8> {
// The HID descriptors live right after each interface descriptor in the config blob.
// Offsets: cfg(9) | i0(9) h0(9) e0(7) | i1(9) h1(9) e1(7) | i2(9) h2(9) e2(7)
let off = match idx {
0 => 9 + 9,
1 => 9 + 25 + 9,
_ => 9 + 50 + 9,
};
cfg.get(off..off + 9)
.map(|s| s.to_vec())
.unwrap_or_default()
}
fn enable_endpoints(raw: RawFd, ctrl_ep: &std::sync::atomic::AtomicI32) {
let mk = |addr: u8, mps: u16| UsbEndpointDescriptor {
b_length: 7,
b_descriptor_type: 5,
b_endpoint_address: addr,
bm_attributes: 0x03,
w_max_packet_size: mps,
b_interval: 4,
..Default::default()
};
let e0 = mk(0x81, 8);
let e1 = mk(0x82, 8);
let e2 = mk(0x83, 64);
ioctl_ptr(raw, IOCTL_EP_ENABLE, &e0 as *const _);
ioctl_ptr(raw, IOCTL_EP_ENABLE, &e1 as *const _);
let h2 = ioctl_ptr(raw, IOCTL_EP_ENABLE, &e2 as *const _);
ctrl_ep.store(h2, Ordering::SeqCst);
}
fn stream_loop(
fd: Arc<GadgetFd>,
running: Arc<AtomicBool>,
ctrl_ep: Arc<std::sync::atomic::AtomicI32>,
configured: Arc<AtomicBool>,
report: Arc<Mutex<[u8; 64]>>,
) {
let raw = fd.0;
while running.load(Ordering::SeqCst) {
let ep = ctrl_ep.load(Ordering::SeqCst);
if configured.load(Ordering::SeqCst) && ep >= 0 {
let r = report
.lock()
.map(|g| *g)
.unwrap_or_else(|_| neutral_deck_report());
let mut buf = [0u8; EPIO_HDR + 64];
buf[0..2].copy_from_slice(&(ep as u16).to_ne_bytes());
buf[4..8].copy_from_slice(&(64u32).to_ne_bytes());
buf[EPIO_HDR..].copy_from_slice(&r);
// Blocks until the host polls the interrupt-IN endpoint; that's fine on its own thread.
ioctl_ptr(raw, IOCTL_EP_WRITE, buf.as_ptr());
}
std::thread::sleep(std::time::Duration::from_millis(8));
}
}
/// Best-effort load of the gadget modules (SteamOS ships `dummy_hcd` + `raw_gadget`). Failures are
/// ignored — the caller falls back to UHID if `/dev/raw-gadget` is then still unusable.
pub fn ensure_modules() {
for m in ["dummy_hcd", "raw_gadget"] {
let _ = std::process::Command::new("modprobe").arg(m).status();
}
}
/// Whether to prefer the USB-gadget Deck over the UHID `SteamDeckPad` — the only transport Steam Input
/// promotes (validated glass-to-glass on a Deck). Defaults **on for SteamOS** hosts (which ship the
/// gadget modules + run Steam Input); off elsewhere, where the universal UHID path stays the default.
/// `PUNKTFUNK_STEAM_GADGET=1`/`0` forces it on/off. A Deck-as-host with a *physical* Deck never reaches
/// here: `resolve_gamepad`'s conflict gate degrades `SteamDeck` → DualSense before the manager is built.
pub fn gadget_preferred() -> bool {
if let Ok(v) = std::env::var("PUNKTFUNK_STEAM_GADGET") {
return v == "1" || v.eq_ignore_ascii_case("true");
}
is_steamos()
}
/// True on SteamOS-class hosts (`/etc/os-release` `ID=steamos`, or `ID_LIKE` naming it).
fn is_steamos() -> bool {
std::fs::read_to_string("/etc/os-release")
.map(|s| {
s.lines()
.any(|l| l == "ID=steamos" || (l.starts_with("ID_LIKE=") && l.contains("steamos")))
})
.unwrap_or(false)
}
@@ -0,0 +1,733 @@
//! Virtual Steam Deck over **USB/IP** (`vhci_hcd`) — the shippable, Secure-Boot-clean, universal
//! alternative to [`super::steam_gadget`] (`raw_gadget` + `dummy_hcd`, SteamOS-only).
//!
//! Like the gadget, this presents a *real* 3-interface USB Steam Deck (mouse = interface 0, keyboard
//! = 1, **controller = 2**) — the interface-2 layout Steam's own driver filters on, so Steam Input
//! promotes it (a UHID Deck, `Interface: -1`, never is). Unlike the gadget it needs no out-of-tree
//! module: `vhci_hcd` is in-tree + signed on SteamOS, Bazzite, and ~every distro, loads under Secure
//! Boot, and needs no MOK. A userspace [`usbip_sim`] server emulates the Deck; the local `vhci_hcd`
//! attaches it. **Validated on Bazzite**: `vhci_hcd` enumerates the 3-interface Deck, `hid-steam`
//! binds it, and Steam reserves an XInput slot — identical recognition to the gadget.
//!
//! The device model + the USB/IP protocol come from the vendored [`usbip_sim`] crate (the upstream
//! `usbip` crate trimmed of its libusb host mode); the captured descriptors + the `0x83`/`0xAE`
//! feature contract come from the shared [`super::steam_proto`] (one source of truth with the gadget).
//!
//! **Attach** is in-process by default (no external `usbip` CLI dependency — the production goal): we
//! run the emulation server on a loopback TCP port, connect to it ourselves, perform the
//! `OP_REQ_IMPORT` handshake, then hand the connected socket fd to `vhci_hcd` via its sysfs `attach`
//! file. If anything in that path fails we fall back to the widely-packaged `usbip` CLI; if *that*
//! also fails, [`open`](SteamDeckUsbip::open) returns `Err` and the caller degrades to UHID.
use super::steam_proto::{
deck_serial, deck_unit_id, feature_reply, neutral_deck_report, parse_steam_output,
SteamFeedback, SteamState, RDESC_DECK_CTRL, RDESC_DECK_KBD, RDESC_DECK_MOUSE,
};
use anyhow::{bail, Context, Result};
use std::any::Any;
use std::collections::HashSet;
use std::io::{Read, Write};
use std::net::TcpStream;
use std::os::fd::AsRawFd;
use std::path::{Path, PathBuf};
use std::process::Command;
use std::sync::{Arc, Mutex};
use std::thread::JoinHandle;
use std::time::{Duration, Instant};
use usbip_sim::{
Direction, SetupPacket, UsbDevice, UsbEndpoint, UsbInterface, UsbInterfaceHandler, UsbIpServer,
Version,
};
const STEAM_VENDOR: u16 = 0x28DE;
const STEAMDECK_PRODUCT: u16 = 0x1205;
/// The single device's USB/IP bus id (one device per server, so the fixed default is fine).
const BUS_ID: &str = "0-0-0";
/// The usbip default TCP port — the server must listen here for the `usbip` CLI fallback to attach.
const USBIP_TCP_PORT: u16 = 3240;
/// Build the 9-byte HID class descriptor inserted between the interface and endpoint descriptors.
fn hid_desc(report_len: usize, country: u8) -> Vec<u8> {
let l = report_len as u16;
#[rustfmt::skip]
let d = vec![0x09, 0x21, 0x10, 0x01, country, 1, 0x22, (l & 0xff) as u8, (l >> 8) as u8];
d
}
/// The Deck **controller** interface (vendor HID, interface 2): answers the HID feature reports
/// (descriptor / `0x83` attributes / `0xAE` serial), streams the current 64-byte state on the
/// interrupt-IN endpoint, and surfaces rumble written via SET_REPORT.
#[derive(Debug)]
struct ControllerHandler {
/// The current 64-byte Deck input report, shared with [`SteamDeckUsbip::write_state`].
report: Arc<Mutex<[u8; 64]>>,
/// Rumble extracted from the kernel's SET_REPORTs, drained by [`SteamDeckUsbip::service`].
feedback: Arc<Mutex<SteamFeedback>>,
/// The host's last SET_REPORT command (drives [`feature_reply`]).
last_set: Vec<u8>,
serial: String,
unit_id: u32,
}
impl UsbInterfaceHandler for ControllerHandler {
fn get_class_specific_descriptor(&self) -> Vec<u8> {
hid_desc(RDESC_DECK_CTRL.len(), 33)
}
fn handle_urb(
&mut self,
_interface: &UsbInterface,
ep: UsbEndpoint,
_len: u32,
setup: SetupPacket,
req: &[u8],
) -> std::io::Result<Vec<u8>> {
if ep.is_ep0() {
Ok(match (setup.request_type, setup.request) {
// GET report descriptor (standard, interface recipient).
(0x81, 0x06) if (setup.value >> 8) == 0x22 => RDESC_DECK_CTRL.to_vec(),
// HID GET_REPORT (feature) — the Deck `0x83`/`0xAE` contract.
(0xA1, 0x01) => feature_reply(&self.last_set, &self.serial, self.unit_id).to_vec(),
// HID SET_REPORT — remember the command (for the next feature reply) + surface rumble.
(0x21, 0x09) => {
self.last_set = req.to_vec();
// `parse_steam_output` expects `[report-id(0), cmd, …]`; EP0 OUT data is `[cmd, …]`.
let mut framed = Vec::with_capacity(req.len() + 1);
framed.push(0);
framed.extend_from_slice(req);
let fb = parse_steam_output(&framed);
if fb.rumble.is_some() {
if let Ok(mut g) = self.feedback.lock() {
*g = fb;
}
}
vec![]
}
(0x21, 0x0A) | (0x21, 0x0B) => vec![], // SET_IDLE / SET_PROTOCOL
_ => vec![],
})
} else if let Direction::In = ep.direction() {
// Interrupt-IN poll: return the current report. The vendored sim paces interrupt-IN by
// bInterval (vhci_hcd does NOT throttle the server side), so this isn't a busy spin.
let r = self
.report
.lock()
.map(|g| *g)
.unwrap_or_else(|_| neutral_deck_report());
Ok(r.to_vec())
} else {
Ok(vec![])
}
}
fn as_any(&mut self) -> &mut dyn Any {
self
}
}
/// A minimal idle HID interface (mouse / keyboard) — serves only its report descriptor.
#[derive(Debug)]
struct IdleHidHandler {
report_desc: Vec<u8>,
}
impl UsbInterfaceHandler for IdleHidHandler {
fn get_class_specific_descriptor(&self) -> Vec<u8> {
hid_desc(self.report_desc.len(), 0)
}
fn handle_urb(
&mut self,
_i: &UsbInterface,
ep: UsbEndpoint,
_l: u32,
setup: SetupPacket,
_req: &[u8],
) -> std::io::Result<Vec<u8>> {
if ep.is_ep0() && setup.request == 0x06 && (setup.value >> 8) == 0x22 {
Ok(self.report_desc.clone())
} else {
Ok(vec![])
}
}
fn as_any(&mut self) -> &mut dyn Any {
self
}
}
fn boxed(
h: impl UsbInterfaceHandler + Send + 'static,
) -> Arc<Mutex<Box<dyn UsbInterfaceHandler + Send>>> {
Arc::new(Mutex::new(Box::new(h)))
}
fn ep(addr: u8, mps: u16) -> UsbEndpoint {
UsbEndpoint {
address: addr,
attributes: 0x03, // interrupt
max_packet_size: mps,
interval: 4,
}
}
/// Assemble the simulated 3-interface USB Deck. The controller handler shares `report` + `feedback`
/// with the owning [`SteamDeckUsbip`].
fn build_device(
index: u8,
report: &Arc<Mutex<[u8; 64]>>,
feedback: &Arc<Mutex<SteamFeedback>>,
) -> UsbDevice {
let mut dev = UsbDevice::new(0); // one device per server; bus_id stays the default "0-0-0".
dev.vendor_id = STEAM_VENDOR;
dev.product_id = STEAMDECK_PRODUCT;
dev.usb_version = Version::from(0x0200u16); // bcdUSB 2.00
dev.device_bcd = Version::from(0x0300u16); // bcdDevice 3.00 (matches the gadget)
dev.set_manufacturer_name("Valve Software");
dev.set_product_name("Steam Deck Controller");
dev.set_serial_number(&deck_serial(index));
dev.with_interface(
0x03,
0x00,
0x02,
Some("mouse"),
vec![ep(0x81, 8)],
boxed(IdleHidHandler {
report_desc: RDESC_DECK_MOUSE.to_vec(),
}),
)
.with_interface(
0x03,
0x01,
0x01,
Some("keyboard"),
vec![ep(0x82, 8)],
boxed(IdleHidHandler {
report_desc: RDESC_DECK_KBD.to_vec(),
}),
)
.with_interface(
0x03,
0x00,
0x00,
Some("controller"),
vec![ep(0x83, 64)],
boxed(ControllerHandler {
report: report.clone(),
feedback: feedback.clone(),
last_set: vec![],
serial: deck_serial(index),
unit_id: deck_unit_id(index),
}),
)
}
/// Owns the emulation-server thread (a dedicated current-thread tokio runtime) and stops it on drop.
/// Run on its own thread so `SteamDeckUsbip::open` works whether or not the caller is inside a tokio
/// runtime (creating a runtime inside one would panic).
struct ServerThread {
stop: Arc<tokio::sync::Notify>,
join: Option<JoinHandle<()>>,
}
impl ServerThread {
/// Spawn the server on `listener`, serving exactly the one simulated `dev`.
fn spawn(listener: std::net::TcpListener, dev: UsbDevice) -> Result<ServerThread> {
let stop = Arc::new(tokio::sync::Notify::new());
let stop_t = stop.clone();
let join = std::thread::Builder::new()
.name("pf-deck-usbip".into())
.spawn(move || {
let rt = match tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
{
Ok(rt) => rt,
Err(e) => {
tracing::error!(error = %e, "usbip server runtime build failed");
return;
}
};
rt.block_on(run_server(
listener,
Arc::new(UsbIpServer::new_simulated(vec![dev])),
stop_t,
));
})
.context("spawn usbip server thread")?;
Ok(ServerThread {
stop,
join: Some(join),
})
}
}
impl Drop for ServerThread {
fn drop(&mut self) {
self.stop.notify_one();
if let Some(j) = self.join.take() {
let _ = j.join();
}
}
}
/// Accept loop: serve each USB/IP connection with the vendored `usbip_sim::handler` until stopped.
async fn run_server(
listener: std::net::TcpListener,
server: Arc<UsbIpServer>,
stop: Arc<tokio::sync::Notify>,
) {
let listener = match tokio::net::TcpListener::from_std(listener) {
Ok(l) => l,
Err(e) => {
tracing::error!(error = %e, "usbip TcpListener::from_std failed");
return;
}
};
loop {
tokio::select! {
_ = stop.notified() => break,
r = listener.accept() => match r {
Ok((mut sock, _)) => {
let server = server.clone();
tokio::spawn(async move {
let _ = usbip_sim::handler(&mut sock, server).await;
});
}
Err(e) => {
tracing::warn!(error = %e, "usbip accept error");
break;
}
}
}
}
}
/// A virtual Steam Deck presented over USB/IP. Dropping it detaches the `vhci_hcd` port (the device
/// disappears, Steam releases its slot) and stops the emulation server.
pub struct SteamDeckUsbip {
report: Arc<Mutex<[u8; 64]>>,
feedback: Arc<Mutex<SteamFeedback>>,
/// The `vhci_hcd` port we attached to — written to the sysfs `detach` file on drop.
vhci_port: u16,
/// Kept alive so the connected socket fd we handed to `vhci_hcd` stays valid (in-process attach
/// only; the CLI hands its own fd to the kernel and exits, so this is `None` there).
_client_sock: Option<TcpStream>,
/// Emulation-server thread; dropped (stopped) after the detach.
_server: ServerThread,
seq: u32,
}
impl SteamDeckUsbip {
/// Bind a virtual Deck and attach it locally via `vhci_hcd`. `index` varies only the serial.
/// Requires `vhci_hcd` loaded and root (the sysfs attach / the CLI both need it). Tries the
/// in-process sysfs attach first, then the `usbip` CLI; `PUNKTFUNK_USBIP_ATTACH=inproc|cli`
/// pins one path (for debugging).
pub fn open(index: u8) -> Result<SteamDeckUsbip> {
ensure_modules();
if vhci_base().is_none() {
bail!(
"vhci_hcd unavailable (no /sys/devices/platform/vhci_hcd*/status) — is it loaded?"
);
}
let mode = std::env::var("PUNKTFUNK_USBIP_ATTACH").ok();
if mode.as_deref() != Some("cli") {
match Self::open_in_process(index) {
Ok(d) => return Ok(d),
Err(e) if mode.as_deref() == Some("inproc") => return Err(e),
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "in-process vhci attach failed — trying the usbip CLI")
}
}
}
Self::open_via_cli(index)
}
/// In-process attach: emulate on a loopback port, do the import handshake ourselves, hand the
/// connected socket to `vhci_hcd` via sysfs. No external dependency.
fn open_in_process(index: u8) -> Result<SteamDeckUsbip> {
let report = Arc::new(Mutex::new(neutral_deck_report()));
let feedback = Arc::new(Mutex::new(SteamFeedback::default()));
let dev = build_device(index, &report, &feedback);
// An ephemeral loopback port (avoids contending the usbip default with another pad).
let listener =
std::net::TcpListener::bind(("127.0.0.1", 0)).context("bind loopback usbip server")?;
let port = listener
.local_addr()
.context("usbip server local_addr")?
.port();
listener
.set_nonblocking(true)
.context("usbip listener set_nonblocking")?;
let server = ServerThread::spawn(listener, dev)?;
// Connect to our own server and run the OP_REQ_IMPORT handshake.
let mut sock = connect_loopback(port).context("connect to usbip server")?;
let (devid, speed) = import_handshake(&mut sock).context("usbip import handshake")?;
// Hand the connected socket to vhci_hcd. Clear BOTH timeouts first: the kernel's vhci rx/tx
// threads honour SO_RCVTIMEO/SO_SNDTIMEO on this socket, so the 3s handshake timeouts would
// otherwise tear the device down after 3s idle (rx) or a 3s-blocked send (tx).
let vhci_port = vhci_find_free_port(speed).context("find a free vhci port")?;
sock.set_read_timeout(None).ok();
sock.set_write_timeout(None).ok();
vhci_attach(vhci_port, sock.as_raw_fd(), devid, speed).context("write vhci_hcd attach")?;
tracing::info!(
index,
vhci_port,
"virtual Steam Deck attached via usbip (in-process — Steam Input recognizes it)"
);
Ok(SteamDeckUsbip {
report,
feedback,
vhci_port,
_client_sock: Some(sock),
_server: server,
seq: 0,
})
}
/// Fallback: emulate on the usbip default port and let the `usbip` CLI attach (it picks the vhci
/// port itself; we recover it by diffing the sysfs status).
fn open_via_cli(index: u8) -> Result<SteamDeckUsbip> {
let report = Arc::new(Mutex::new(neutral_deck_report()));
let feedback = Arc::new(Mutex::new(SteamFeedback::default()));
let dev = build_device(index, &report, &feedback);
let listener = std::net::TcpListener::bind(("127.0.0.1", USBIP_TCP_PORT))
.with_context(|| format!("bind usbip default port {USBIP_TCP_PORT} for CLI attach"))?;
listener
.set_nonblocking(true)
.context("usbip listener set_nonblocking")?;
let server = ServerThread::spawn(listener, dev)?;
let before = vhci_used_ports();
usbip_attach_cli().context("usbip CLI attach")?;
let vhci_port = wait_for_new_port(&before)
.context("could not determine the vhci port the usbip CLI attached to")?;
tracing::info!(
index,
vhci_port,
"virtual Steam Deck attached via usbip (CLI — Steam Input recognizes it)"
);
Ok(SteamDeckUsbip {
report,
feedback,
vhci_port,
_client_sock: None,
_server: server,
seq: 0,
})
}
/// Serialize `st` into the 64-byte Deck report streamed on the controller interrupt-IN endpoint.
pub fn write_state(&mut self, st: &SteamState) {
self.seq = self.seq.wrapping_add(1);
let mut r = [0u8; 64];
super::steam_proto::serialize_deck_state(&mut r, st, self.seq);
if let Ok(mut g) = self.report.lock() {
*g = r;
}
}
/// Drain any rumble feedback the kernel/Steam wrote to the device.
pub fn service(&mut self) -> SteamFeedback {
self.feedback
.lock()
.map(|mut f| std::mem::take(&mut *f))
.unwrap_or_default()
}
}
impl Drop for SteamDeckUsbip {
fn drop(&mut self) {
// Detach the vhci port first (the kernel closes its end of the socket + tears down the
// device); `_client_sock` + `_server` then drop, closing our side + stopping the server.
if let Err(e) = vhci_detach(self.vhci_port) {
tracing::debug!(port = self.vhci_port, error = %e, "vhci detach failed (device may already be gone)");
}
}
}
// ---- USB/IP import handshake (we act as the usbip *client* before handing the fd to the kernel) ----
const USBIP_VERSION: u16 = 0x0111;
const OP_REQ_IMPORT: u16 = 0x8003;
/// Connect to our own loopback server, retrying briefly while the server thread comes up.
fn connect_loopback(port: u16) -> Result<TcpStream> {
let addr = ("127.0.0.1", port);
let mut last = None;
for _ in 0..50 {
match TcpStream::connect(addr) {
Ok(s) => {
s.set_nodelay(true).ok();
return Ok(s);
}
Err(e) => {
last = Some(e);
std::thread::sleep(Duration::from_millis(10));
}
}
}
Err(anyhow::anyhow!(
"connect 127.0.0.1:{port}: {}",
last.map(|e| e.to_string()).unwrap_or_default()
))
}
/// Send `OP_REQ_IMPORT` for [`BUS_ID`] and read `OP_REP_IMPORT`, returning `(devid, speed)` parsed
/// from the device record (the same `devid = bus_num<<16 | dev_num` + speed `vhci_hcd` wants). The
/// whole 320-byte reply MUST be consumed here so the socket starts clean at the kernel's first
/// `USBIP_CMD_SUBMIT`.
fn import_handshake(sock: &mut TcpStream) -> Result<(u32, u32)> {
// Bounded so a non-responsive server can't head-block the per-session input thread (this talks
// to our own in-process loopback server, so a working handshake completes in well under a ms).
sock.set_read_timeout(Some(Duration::from_secs(1))).ok();
sock.set_write_timeout(Some(Duration::from_secs(1))).ok();
let mut req = Vec::with_capacity(40);
req.extend_from_slice(&USBIP_VERSION.to_be_bytes());
req.extend_from_slice(&OP_REQ_IMPORT.to_be_bytes());
req.extend_from_slice(&0u32.to_be_bytes()); // status
let mut busid = [0u8; 32];
let b = BUS_ID.as_bytes();
busid[..b.len()].copy_from_slice(b);
req.extend_from_slice(&busid);
sock.write_all(&req).context("send OP_REQ_IMPORT")?;
// Reply: version(2) code(2) status(4), then the 312-byte device record on success.
let mut header = [0u8; 8];
sock.read_exact(&mut header)
.context("read OP_REP_IMPORT header")?;
let status = u32::from_be_bytes([header[4], header[5], header[6], header[7]]);
if status != 0 {
bail!("OP_REP_IMPORT refused (status={status}) — device {BUS_ID} not exported?");
}
let mut dev = [0u8; 312];
sock.read_exact(&mut dev)
.context("read OP_REP_IMPORT device record")?;
// Device record layout: path[256], bus_id[32], bus_num(4 BE)@288, dev_num(4 BE)@292, speed(4)@296.
let be = |o: usize| u32::from_be_bytes([dev[o], dev[o + 1], dev[o + 2], dev[o + 3]]);
let bus_num = be(288);
let dev_num = be(292);
let speed = be(296);
Ok(((bus_num << 16) | dev_num, speed))
}
// ---- vhci_hcd sysfs plumbing ----
/// Best-effort load of `vhci_hcd` (in-tree + signed on SteamOS/Bazzite/most distros).
pub fn ensure_modules() {
let _ = Command::new("modprobe").arg("vhci_hcd").status();
}
/// Run `usbip attach -r 127.0.0.1 -b 0-0-0`, bounded by a deadline so a hung CLI can't head-block
/// the per-session input thread indefinitely (the caller runs this inline on that thread).
fn usbip_attach_cli() -> Result<()> {
let mut child = Command::new("usbip")
.args(["attach", "-r", "127.0.0.1", "-b", BUS_ID])
.spawn()
.context("spawn `usbip attach` (is usbip-utils installed?)")?;
let deadline = Instant::now() + Duration::from_secs(6);
loop {
match child.try_wait().context("wait on `usbip attach`")? {
Some(st) if st.success() => return Ok(()),
Some(st) => bail!("`usbip attach` exited with {st}"),
None if Instant::now() >= deadline => {
let _ = child.kill();
let _ = child.wait();
bail!("`usbip attach` timed out (>6s) — killed");
}
None => std::thread::sleep(Duration::from_millis(20)),
}
}
}
/// Whether a usbip attach should be attempted at all. Default on (the universal Steam-promotable
/// transport on non-SteamOS hosts); `PUNKTFUNK_STEAM_USBIP=0` forces it off, `=1` forces it on.
/// [`open`](SteamDeckUsbip::open) still degrades gracefully if `vhci_hcd` turns out to be absent.
pub fn usbip_preferred() -> bool {
!matches!(
std::env::var("PUNKTFUNK_STEAM_USBIP").ok().as_deref(),
Some("0") | Some("false")
)
}
/// The `vhci_hcd.0` (or legacy `vhci_hcd`) platform sysfs directory, if present.
fn vhci_base() -> Option<PathBuf> {
for p in [
"/sys/devices/platform/vhci_hcd.0",
"/sys/devices/platform/vhci_hcd",
] {
let base = Path::new(p);
if base.join("status").exists() {
return Some(base.to_path_buf());
}
}
None
}
fn read_status() -> Result<String> {
let base = vhci_base().context("vhci_hcd sysfs not present")?;
std::fs::read_to_string(base.join("status")).context("read vhci_hcd status")
}
/// One parsed `status` row: `(port, hub_is_superspeed, sta)`. Handles both the modern
/// `hub port sta …` and the legacy `port sta …` column layouts; returns `None` for header/blank rows.
fn parse_status_row(line: &str) -> Option<(u16, bool, u32)> {
let t: Vec<&str> = line.split_whitespace().collect();
if t.is_empty() {
return None;
}
let (hub_ss, port_str, sta_str) = if t[0] == "hs" || t[0] == "ss" {
(Some(t[0] == "ss"), *t.get(1)?, *t.get(2)?)
} else if t[0].chars().all(|c| c.is_ascii_digit()) {
(None, t[0], *t.get(1)?) // legacy: port sta …
} else {
return None; // header ("hub"/"prt"/"port" …)
};
let port = port_str.parse::<u16>().ok()?;
let sta = sta_str.parse::<u32>().ok()?;
Some((port, hub_ss.unwrap_or(false), sta))
}
/// `sta == 4` is `VDEV_ST_NULL` (a free port).
const VDEV_ST_NULL: u32 = 4;
/// Pick a free `vhci_hcd` port matching the device speed (`usbip_speed >= 5` ⇒ SuperSpeed hub).
fn vhci_find_free_port(usbip_speed: u32) -> Result<u16> {
let want_ss = usbip_speed >= 5;
let status = read_status()?;
for line in status.lines() {
if let Some((port, is_ss, sta)) = parse_status_row(line) {
if sta == VDEV_ST_NULL && is_ss == want_ss {
return Ok(port);
}
}
}
// Speed-class match failed (legacy single-hub status): take any free port.
for line in status.lines() {
if let Some((port, _, sta)) = parse_status_row(line) {
if sta == VDEV_ST_NULL {
return Ok(port);
}
}
}
bail!("no free vhci_hcd port (all ports in use?)")
}
/// Ports currently in use (`sta != VDEV_ST_NULL`) — snapshotted around a CLI attach to recover its port.
fn vhci_used_ports() -> HashSet<u16> {
read_status()
.unwrap_or_default()
.lines()
.filter_map(parse_status_row)
.filter(|&(_, _, sta)| sta != VDEV_ST_NULL)
.map(|(port, _, _)| port)
.collect()
}
/// Poll the status file (briefly) for a port that became used since `before` — the one the CLI attached.
fn wait_for_new_port(before: &HashSet<u16>) -> Result<u16> {
let deadline = Instant::now() + Duration::from_secs(2);
loop {
if let Some(p) = vhci_used_ports().difference(before).copied().min() {
return Ok(p);
}
if Instant::now() >= deadline {
bail!("no newly-attached vhci port appeared after `usbip attach`");
}
std::thread::sleep(Duration::from_millis(50));
}
}
fn vhci_attach(port: u16, sockfd: i32, devid: u32, speed: u32) -> Result<()> {
let base = vhci_base().context("vhci_hcd sysfs not present")?;
let line = format!("{port} {sockfd} {devid} {speed}");
std::fs::write(base.join("attach"), line)
.with_context(|| format!("write vhci_hcd attach (port {port}) — root?"))
}
fn vhci_detach(port: u16) -> Result<()> {
let base = vhci_base().context("vhci_hcd sysfs not present")?;
std::fs::write(base.join("detach"), format!("{port}")).context("write vhci_hcd detach")
}
#[cfg(test)]
mod tests {
use super::*;
/// The `status` parser handles the modern `hub port sta …` layout, the legacy `port sta …`
/// layout, and skips header/blank lines — a slip here would mean attaching to a busy port.
#[test]
fn status_parser_handles_both_layouts() {
// modern
assert_eq!(
parse_status_row("hs 0000 004 000 00000000 000000 0-0"),
Some((0, false, 4))
);
assert_eq!(
parse_status_row("ss 0008 006 000 00000000 000000 0-0"),
Some((8, true, 6))
);
// legacy (no hub column)
assert_eq!(
parse_status_row("0001 004 000 00000000 000000 0-0"),
Some((1, false, 4))
);
// header / blank
assert_eq!(
parse_status_row("hub port sta spd dev sockfd local_busid"),
None
);
assert_eq!(parse_status_row(""), None);
}
/// A free HS port is preferred for an HS device; a free SS port for an SS device.
#[test]
fn free_port_selection_matches_speed() {
let status = "hub port sta spd dev sockfd local_busid\n\
hs 0000 006 000 00000000 000000 0-0\n\
hs 0001 004 000 00000000 000000 0-0\n\
ss 0008 004 000 00000000 000000 0-0\n";
// Reuse the parser directly (vhci_find_free_port reads sysfs; test the selection logic).
let hs = status
.lines()
.filter_map(parse_status_row)
.find(|&(_, is_ss, sta)| sta == VDEV_ST_NULL && !is_ss)
.map(|(p, _, _)| p);
let ss = status
.lines()
.filter_map(parse_status_row)
.find(|&(_, is_ss, sta)| sta == VDEV_ST_NULL && is_ss)
.map(|(p, _, _)| p);
assert_eq!(hs, Some(1));
assert_eq!(ss, Some(8));
}
/// On-box smoke test (needs root + `vhci_hcd`): attach a virtual Deck, confirm `hid-steam` binds
/// it (the `Steam Deck` evdev appears) and that it tears down on drop. `#[ignore]`d in CI.
#[test]
#[ignore = "attaches a real vhci_hcd device; needs root + vhci_hcd"]
fn usbip_deck_binds_and_tears_down() {
ensure_modules();
let mut pad = SteamDeckUsbip::open(0).expect("open SteamDeckUsbip (root + vhci_hcd?)");
let st = SteamState::from_gamepad(punktfunk_core::input::gamepad::BTN_A, 0, 0, 0, 0, 0, 0);
let start = Instant::now();
while start.elapsed() < Duration::from_millis(800) {
pad.write_state(&st);
let _ = pad.service();
std::thread::sleep(Duration::from_millis(8));
}
let devs = std::fs::read_to_string("/proc/bus/input/devices").unwrap_or_default();
assert!(
devs.contains("Steam Deck"),
"hid-steam did not bind the usbip Deck"
);
drop(pad);
std::thread::sleep(Duration::from_millis(300));
let devs = std::fs::read_to_string("/proc/bus/input/devices").unwrap_or_default();
assert!(
!devs.contains("Steam Deck Motion Sensors"),
"device not torn down on drop"
);
}
}
@@ -0,0 +1,684 @@
//! Transport-independent Steam Controller / Steam Deck HID contract — the Steam analogue of
//! [`super::dualsense_proto`]. The report descriptor, the command/feature IDs, the byte-exact
//! Deck input-report serializer, the `XInput`/rich-input → state mappers, and the rumble-feedback
//! parser. Pure logic, shared by the Linux UHID backend and (later) a Windows UMDF backend.
//!
//! **Layout source of truth:** the kernel `drivers/hid/hid-steam.c` `steam_do_deck_input_event`
//! (+ `steam_do_deck_sensors_event`) — every offset/bit/sign below is transcribed verbatim from
//! it and on-box-validated against kernel 7.0 (see `design/steam-controller-deck-support.md`).
//! M0 proved the device binds + parses; M1 (here) makes the serializer byte-exact.
//!
//! Three load-bearing details the DualSense path does NOT have:
//! * **report id 0 / unnumbered**: input reports are the raw 64 bytes starting `[0x01,0x00,0x09]`
//! (no report-id prefix); FEATURE get/set reports DO carry a leading `0x00` report-id byte
//! (`steam_send_report` does `memcpy(buf+1, cmd, …)`, `steam_recv_report` strips `buf[0]`).
//! * **`gamepad_mode` gate**: `steam_do_deck_input_event` early-returns when
//! `!gamepad_mode && lizard_mode` (the module param, default on). `gamepad_mode` starts false
//! and TOGGLES when [`btn::STEAM_MENU_RIGHT`] (`b9.6`, the mode-switch) is held ~450 ms while
//! no hidraw client is open. The backend enters gamepad mode at session start (pulse that bit,
//! or load `hid_steam lizard_mode=0`) — see the backend, not this module.
//! * **the `UHID_SET_REPORT` handshake** must be answered (DualSense omits it).
#![allow(dead_code)] // Some of the full model is consumed only once the M2 backend + M3 wire land.
use punktfunk_core::input::gamepad as gs;
use punktfunk_core::quic::RichInput;
/// Valve. `hid-steam` matches purely by VID/PID over `BUS_USB`.
pub const STEAM_VENDOR: u32 = 0x28DE;
/// Steam Deck built-in controller (same PID on LCD + OLED).
pub const STEAMDECK_PRODUCT: u32 = 0x1205;
/// Classic Steam Controller, wired (report id 1 / `ID_CONTROLLER_STATE`; a later model).
pub const STEAMCTRL_WIRED_PRODUCT: u32 = 0x1102;
/// The Steam HID state/command report is a fixed 64-byte, **unnumbered** (report-id-0) frame.
pub const STEAM_REPORT_LEN: usize = 64;
// Command IDs (drivers/hid/hid-steam.c), confirmed against the kernel source.
pub const ID_CLEAR_DIGITAL_MAPPINGS: u8 = 0x81;
pub const ID_GET_ATTRIBUTES_VALUES: u8 = 0x83;
pub const ID_SET_SETTINGS_VALUES: u8 = 0x87;
pub const ID_LOAD_DEFAULT_SETTINGS: u8 = 0x8E;
pub const ID_GET_DEVICE_INFO: u8 = 0xA1;
pub const ID_GET_STRING_ATTRIBUTE: u8 = 0xAE;
pub const ATTRIB_STR_UNIT_SERIAL: u8 = 0x01;
/// Host→client feedback: `steam_haptic_rumble` emits report `[0xEB, 9, …]` (FF_RUMBLE → trackpad
/// actuators / Deck motors). The Deck's rumble path; the classic SC also has `0x8F` pad pulses.
pub const ID_TRIGGER_RUMBLE_CMD: u8 = 0xEB;
pub const ID_TRIGGER_HAPTIC_PULSE: u8 = 0x8F;
/// Input report message types: SC = `ID_CONTROLLER_STATE`, Deck = `ID_CONTROLLER_DECK_STATE`.
pub const ID_CONTROLLER_STATE: u8 = 0x01;
pub const ID_CONTROLLER_DECK_STATE: u8 = 0x09;
/// Which Steam device identity to present. M1 implements the Deck fully; the classic Controller
/// (dual trackpads, report id 1, trackpad-only haptics) is a later identity behind the same path.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum SteamModel {
Deck,
Controller,
}
impl SteamModel {
pub fn product(self) -> u32 {
match self {
SteamModel::Deck => STEAMDECK_PRODUCT,
SteamModel::Controller => STEAMCTRL_WIRED_PRODUCT,
}
}
}
/// Minimal vendor-defined HID report descriptor: one application collection with a 64-byte input
/// report and a 64-byte feature report, both UNNUMBERED (report id 0). `hid-steam` is a raw-event
/// driver, so the field layout is cosmetic — but `steam_probe` requires `hid_parse` to succeed AND
/// a non-empty FEATURE report list (`steam_is_valve_interface`), so the feature item is mandatory.
#[rustfmt::skip]
pub const STEAMDECK_RDESC: &[u8] = &[
0x06, 0x00, 0xFF, // Usage Page (Vendor-Defined 0xFF00)
0x09, 0x01, // Usage (0x01)
0xA1, 0x01, // Collection (Application)
0x15, 0x00, // Logical Minimum (0)
0x26, 0xFF, 0x00, // Logical Maximum (255)
0x75, 0x08, // Report Size (8 bits)
0x95, 0x40, // Report Count (64)
0x09, 0x01, // Usage (0x01)
0x81, 0x02, // Input (Data,Var,Abs) — the 64-byte state report
0x09, 0x01, // Usage (0x01)
0x95, 0x40, // Report Count (64)
0xB1, 0x02, // Feature (Data,Var,Abs) — makes steam_is_valve_interface() true
0xC0, // End Collection
];
/// Deck button bits, indexed in the `u64` packed across report bytes 8..16 — bit `(byte-8)*8 + bit`,
/// transcribed verbatim from `steam_do_deck_input_event` (bytes 12 + 15 carry no buttons). Naming
/// follows the physical Deck control; the trailing comment is the kernel `BTN_*` it maps to.
pub mod btn {
// byte 8
pub const RT_FULL: u64 = 1 << 0; // BTN_TR2 — right trigger fully pressed
pub const LT_FULL: u64 = 1 << 1; // BTN_TL2 — left trigger fully pressed
pub const RB: u64 = 1 << 2; // BTN_TR — right shoulder
pub const LB: u64 = 1 << 3; // BTN_TL — left shoulder
pub const Y: u64 = 1 << 4;
pub const B: u64 = 1 << 5;
pub const X: u64 = 1 << 6;
pub const A: u64 = 1 << 7;
// byte 9
pub const DPAD_UP: u64 = 1 << 8;
pub const DPAD_RIGHT: u64 = 1 << 9;
pub const DPAD_LEFT: u64 = 1 << 10;
pub const DPAD_DOWN: u64 = 1 << 11;
pub const VIEW: u64 = 1 << 12; // BTN_SELECT — "menu left" (View / Back)
pub const STEAM: u64 = 1 << 13; // BTN_MODE — Steam logo button
pub const MENU: u64 = 1 << 14; // BTN_START — "menu right" (Start / Options)
pub const L5: u64 = 1 << 15; // BTN_GRIPL2 — left BOTTOM back grip
// byte 10
pub const R5: u64 = 1 << 16; // BTN_GRIPR2 — right BOTTOM back grip
pub const LPAD_CLICK: u64 = 1 << 17; // BTN_THUMB — left pad pressed (click)
pub const RPAD_CLICK: u64 = 1 << 18; // BTN_THUMB2 — right pad pressed (click)
pub const LPAD_TOUCH: u64 = 1 << 19; // gates ABS_HAT0 (left pad coords)
pub const RPAD_TOUCH: u64 = 1 << 20; // gates ABS_HAT1 (right pad coords)
pub const L3: u64 = 1 << 22; // BTN_THUMBL — left joystick click
// byte 11
pub const R3: u64 = 1 << 26; // BTN_THUMBR — right joystick click
// byte 13
pub const L4: u64 = 1 << 41; // BTN_GRIPL — left TOP back grip
pub const R4: u64 = 1 << 42; // BTN_GRIPR — right TOP back grip
pub const LJOY_TOUCH: u64 = 1 << 46;
pub const RJOY_TOUCH: u64 = 1 << 47;
// byte 14
pub const QAM: u64 = 1 << 50; // BTN_BASE — quick-access (…) button
/// `b9.6` doubles as the mode-switch: held ~450 ms (no hidraw client) it toggles `gamepad_mode`.
pub const STEAM_MENU_RIGHT: u64 = MENU;
}
/// Full virtual Steam Deck controller state. All analog fields are stored as the RAW little-endian
/// report values the kernel reads (so [`serialize_deck_state`] is a pure memcpy); the kernel applies
/// its own sign conventions on top (`ABS_Y = -raw`, etc.) — see [`SteamState::from_gamepad`].
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct SteamState {
/// Packed button bits (see [`btn`]); occupies report bytes 8..16.
pub buttons: u64,
/// Left / right joystick, raw s16 (report 48/50/52/54). The kernel negates the Y axes.
pub lx: i16,
pub ly: i16,
pub rx: i16,
pub ry: i16,
/// Left / right analog trigger, raw u16 (report 44/46 → ABS_HAT2Y/X).
pub lt: u16,
pub rt: u16,
/// Left / right trackpad position, raw s16, centred 0 (report 16/18/20/22). Only surfaced by
/// the kernel while the matching `*PAD_TOUCH` button bit is set.
pub lpad_x: i16,
pub lpad_y: i16,
pub rpad_x: i16,
pub rpad_y: i16,
pub lpad_pressure: u16,
pub rpad_pressure: u16,
/// IMU, raw s16. `accel`/`gyro` are `[X, Y, Z]`; the kernel maps them to ABS_X/Z/Y + ABS_RX/RZ/RY
/// (with Z/RZ negated) on the separate sensors evdev.
pub accel: [i16; 3],
pub gyro: [i16; 3],
}
impl SteamState {
pub fn neutral() -> SteamState {
SteamState::default()
}
/// Set/clear a button (or group) by its [`btn`] mask.
pub fn press(&mut self, mask: u64, down: bool) {
if down {
self.buttons |= mask;
} else {
self.buttons &= !mask;
}
}
/// Map an `XInput`/GameStream pad frame (button bitmask + i16 sticks + u8 triggers) into the Deck
/// state. Sticks pass through (the kernel negates Y, which yields the conventional direction —
/// validated on-box); triggers scale u8 0..255 → u16 0..32640 and set the full-pull bit when
/// pressed. Trackpad + motion + the back grips arrive separately ([`apply_rich`], the M3 wire).
pub fn from_gamepad(
buttons: u32,
lx: i16,
ly: i16,
rx: i16,
ry: i16,
lt: u8,
rt: u8,
) -> SteamState {
let on = |bit: u32| buttons & bit != 0;
let mut s = SteamState {
lx,
ly,
rx,
ry,
lt: (lt as u16) * 128,
rt: (rt as u16) * 128,
..SteamState::neutral()
};
let mut b = 0u64;
let set = |b: &mut u64, on: bool, m: u64| {
if on {
*b |= m;
}
};
set(&mut b, on(gs::BTN_A), btn::A);
set(&mut b, on(gs::BTN_B), btn::B);
set(&mut b, on(gs::BTN_X), btn::X);
set(&mut b, on(gs::BTN_Y), btn::Y);
set(&mut b, on(gs::BTN_LB), btn::LB);
set(&mut b, on(gs::BTN_RB), btn::RB);
set(&mut b, lt > 0, btn::LT_FULL);
set(&mut b, rt > 0, btn::RT_FULL);
set(&mut b, on(gs::BTN_BACK), btn::VIEW);
set(&mut b, on(gs::BTN_START), btn::MENU);
set(&mut b, on(gs::BTN_GUIDE), btn::STEAM);
set(&mut b, on(gs::BTN_LS_CLICK), btn::L3);
set(&mut b, on(gs::BTN_RS_CLICK), btn::R3);
set(&mut b, on(gs::BTN_DPAD_UP), btn::DPAD_UP);
set(&mut b, on(gs::BTN_DPAD_DOWN), btn::DPAD_DOWN);
set(&mut b, on(gs::BTN_DPAD_LEFT), btn::DPAD_LEFT);
set(&mut b, on(gs::BTN_DPAD_RIGHT), btn::DPAD_RIGHT);
// The DualSense touchpad-click wire bit maps to the Deck's RIGHT pad click (the pad that
// stands in for the DualSense touchpad — see apply_rich).
set(&mut b, on(gs::BTN_TOUCHPAD), btn::RPAD_CLICK);
// Back grips (the whole reason for the Deck identity): the wire paddle bits map to the four
// Deck grips — PADDLE1/2/3/4 = R4/L4/R5/L5 (see `input::gamepad`); MISC1 = the QAM '…' button.
set(&mut b, on(gs::BTN_PADDLE1), btn::R4);
set(&mut b, on(gs::BTN_PADDLE2), btn::L4);
set(&mut b, on(gs::BTN_PADDLE3), btn::R5);
set(&mut b, on(gs::BTN_PADDLE4), btn::L5);
set(&mut b, on(gs::BTN_MISC1), btn::QAM);
s.buttons = b;
s
}
/// Apply one rich client→host event into this state, preserving everything else. The single-pad
/// wire [`RichInput::Touchpad`] maps to the **right** trackpad (the Deck pad analogous to the
/// DualSense touchpad); the left pad arrives via the M3 `TouchpadEx` surface. [`RichInput::Motion`]
/// passes gyro/accel straight through (raw i16; cross-device unit scaling is M3).
pub fn apply_rich(&mut self, rich: RichInput) {
match rich {
RichInput::Touchpad { active, x, y, .. } => {
self.press(btn::RPAD_TOUCH, active);
// Normalized 0..=65535 (centre 32768) → the pad's centred s16 range.
self.rpad_x = ((x as i32) - 32768) as i16;
self.rpad_y = ((y as i32) - 32768) as i16;
}
RichInput::Motion { gyro, accel, .. } => {
// The wire carries DualSense-convention units (what every client capture emits); the
// Deck's hid-steam report wants 16 LSB/°·s + 16384 LSB/g, so rescale here.
let (g, a) = super::steam_remap::motion_wire_to_deck(gyro, accel);
self.gyro = g;
self.accel = a;
}
RichInput::TouchpadEx {
surface,
touch,
click,
x,
y,
..
} => {
// Steam pads are natively signed (centre 0), so x/y map straight in. surface 1 =
// left pad, anything else (0 single / 2 right) = right pad.
if surface == 1 {
self.press(btn::LPAD_TOUCH, touch);
self.press(btn::LPAD_CLICK, click);
self.lpad_x = x;
self.lpad_y = y;
} else {
self.press(btn::RPAD_TOUCH, touch);
self.press(btn::RPAD_CLICK, click);
self.rpad_x = x;
self.rpad_y = y;
}
}
}
}
}
/// Serialize the full Deck input report (`ID_CONTROLLER_DECK_STATE`) into the 64-byte unnumbered
/// frame `hid-steam` parses. Pure + byte-exact against `steam_do_deck_input_event`; the report-id
/// constant is `data[0]=0x01` (NOT a HID report id — this report is unnumbered).
pub fn serialize_deck_state(r: &mut [u8; STEAM_REPORT_LEN], st: &SteamState, seq: u32) {
r.fill(0);
r[0] = 0x01;
r[1] = 0x00;
r[2] = ID_CONTROLLER_DECK_STATE;
r[3] = 0x3C; // payload length; the kernel ignores it
r[4..8].copy_from_slice(&seq.to_le_bytes());
r[8..16].copy_from_slice(&st.buttons.to_le_bytes()); // bytes 8..16 (12+15 stay 0)
r[16..18].copy_from_slice(&st.lpad_x.to_le_bytes());
r[18..20].copy_from_slice(&st.lpad_y.to_le_bytes());
r[20..22].copy_from_slice(&st.rpad_x.to_le_bytes());
r[22..24].copy_from_slice(&st.rpad_y.to_le_bytes());
r[24..26].copy_from_slice(&st.accel[0].to_le_bytes()); // accel X → IMU ABS_X
r[26..28].copy_from_slice(&st.accel[1].to_le_bytes()); // accel Y → IMU ABS_Z (kernel negates)
r[28..30].copy_from_slice(&st.accel[2].to_le_bytes()); // accel Z → IMU ABS_Y
r[30..32].copy_from_slice(&st.gyro[0].to_le_bytes()); // gyro X → IMU ABS_RX
r[32..34].copy_from_slice(&st.gyro[1].to_le_bytes()); // gyro Y → IMU ABS_RZ (kernel negates)
r[34..36].copy_from_slice(&st.gyro[2].to_le_bytes()); // gyro Z → IMU ABS_RY
// 36..44 quaternion — left 0 (optional; the kernel does not surface it)
r[44..46].copy_from_slice(&st.lt.to_le_bytes()); // left trigger → ABS_HAT2Y
r[46..48].copy_from_slice(&st.rt.to_le_bytes()); // right trigger → ABS_HAT2X
r[48..50].copy_from_slice(&st.lx.to_le_bytes()); // left joystick X → ABS_X
r[50..52].copy_from_slice(&st.ly.to_le_bytes()); // left joystick Y → ABS_Y (kernel negates)
r[52..54].copy_from_slice(&st.rx.to_le_bytes()); // right joystick X → ABS_RX
r[54..56].copy_from_slice(&st.ry.to_le_bytes()); // right joystick Y → ABS_RY (kernel negates)
r[56..58].copy_from_slice(&st.lpad_pressure.to_le_bytes());
r[58..60].copy_from_slice(&st.rpad_pressure.to_le_bytes());
}
/// Build the `steam_get_serial` GET_REPORT reply. The Steam feature path is report-id-0 with a
/// leading report-id byte the kernel strips (`steam_recv_report` does `memcpy(data, buf+1, …)`), so
/// the wire is `[0x00, 0xAE, len, 0x01, ascii…]`; the kernel then validates `reply[0]==0xAE`,
/// `1<=reply[1]<=21`, `reply[2]==0x01`. Non-fatal (a bad reply → the `"XXXXXXXXXX"` fallback).
pub fn serial_reply(serial: &str) -> [u8; STEAM_REPORT_LEN] {
let mut buf = [0u8; STEAM_REPORT_LEN];
let bytes = serial.as_bytes();
let len = bytes.len().clamp(1, 21);
buf[0] = 0x00; // report id 0 — stripped by steam_recv_report
buf[1] = ID_GET_STRING_ATTRIBUTE;
buf[2] = len as u8;
buf[3] = ATTRIB_STR_UNIT_SERIAL;
buf[4..4 + len].copy_from_slice(&bytes[..len]);
buf
}
/// One service pass's extracted feedback. Rumble rides the universal 0xCA plane (so any client
/// feels it); the classic SC's trackpad-pulse haptics (`0x8F`) are a later, model-specific add.
#[derive(Default, Debug, PartialEq, Eq)]
pub struct SteamFeedback {
/// `(low, high)` motor levels (left/strong, right/weak), if a rumble report carried them.
pub rumble: Option<(u16, u16)>,
}
/// Parse a feature/output report the kernel wrote to our device. The Steam feedback path is a
/// FEATURE `SET_REPORT` whose wire data is `[0x00 report-id, cmd, len, …]`; `cmd == 0xEB`
/// (`steam_haptic_rumble`) carries `[…, 0, intensity(2), left_speed(2), right_speed(2), gains(2)]`.
/// We surface `(left_speed, right_speed)` as `(low, high)` for the 0xCA rumble plane.
pub fn parse_steam_output(data: &[u8]) -> SteamFeedback {
let mut fb = SteamFeedback::default();
// data[0] is the stripped report-id byte (0); the command id follows.
if data.len() >= 10 && data[1] == ID_TRIGGER_RUMBLE_CMD {
let le = |o: usize| u16::from_le_bytes([data[o], data[o + 1]]);
let left = le(6); // left_speed (report[5..7]) → low / strong motor
let right = le(8); // right_speed (report[7..9]) → high / weak motor
fb.rumble = Some((left, right));
}
fb
}
// ===========================================================================================
// Real-USB Deck device contract (the gadget + usbip transports present a *real* 3-interface USB
// Deck so Steam Input promotes it; the UHID path above uses the minimal [`STEAMDECK_RDESC`]).
//
// These descriptors are captured verbatim from a physical Steam Deck (28DE:1205): mouse =
// interface 0, keyboard = interface 1, **controller = interface 2** (the interface number Steam's
// own driver filters on — the reason a UHID Deck, `Interface: -1`, is never promoted). The
// `0x83`/`0xAE` feature contract is what stops Steam re-probing (the gamepad-evdev churn). Shared
// by [`super::super::steam_gadget`] (raw_gadget) and [`super::super::steam_usbip`] (usbip/vhci).
// ===========================================================================================
/// Captured Deck **mouse** report descriptor (interface 0, EP 0x81).
#[rustfmt::skip]
pub const RDESC_DECK_MOUSE: &[u8] = &[
0x05,0x01,0x09,0x02,0xa1,0x01,0x09,0x01,0xa1,0x00,0x05,0x09,0x19,0x01,0x29,0x02,
0x15,0x00,0x25,0x01,0x75,0x01,0x95,0x02,0x81,0x02,0x75,0x06,0x95,0x01,0x81,0x01,
0x05,0x01,0x09,0x30,0x09,0x31,0x15,0x81,0x25,0x7f,0x75,0x08,0x95,0x02,0x81,0x06,
0x95,0x01,0x09,0x38,0x81,0x06,0x05,0x0c,0x0a,0x38,0x02,0x95,0x01,0x81,0x06,0xc0,0xc0];
/// Captured Deck **keyboard** (boot) report descriptor (interface 1, EP 0x82).
#[rustfmt::skip]
pub const RDESC_DECK_KBD: &[u8] = &[
0x05,0x01,0x09,0x06,0xa1,0x01,0x05,0x07,0x19,0xe0,0x29,0xe7,0x15,0x00,0x25,0x01,
0x75,0x01,0x95,0x08,0x81,0x02,0x81,0x01,0x19,0x00,0x29,0x65,0x15,0x00,0x25,0x65,
0x75,0x08,0x95,0x06,0x81,0x00,0xc0];
/// Captured Deck **controller** report descriptor (interface 2, EP 0x83; Usage Page `0xFFFF`,
/// `bCountryCode 33`). The vendor-defined report the `hid-steam` driver binds.
#[rustfmt::skip]
pub const RDESC_DECK_CTRL: &[u8] = &[
0x06,0xff,0xff,0x09,0x01,0xa1,0x01,0x09,0x02,0x09,0x03,0x15,0x00,0x26,0xff,0x00,
0x75,0x08,0x95,0x40,0x81,0x02,0x09,0x06,0x09,0x07,0x15,0x00,0x26,0xff,0x00,0x75,
0x08,0x95,0x40,0xb1,0x02,0xc0];
/// Per-instance Deck unit id stamped into the `0x83` GET_ATTRIBUTES device-id attrs (`0x0a`/`0x04`)
/// so a virtual Deck never collides with a real one or another instance. `"PF"` high word + index.
pub fn deck_unit_id(index: u8) -> u32 {
0x5046_0000 | index as u32
}
/// A Steam-accepted alphanumeric unit serial (a real Deck's is e.g. `"FVZZ4200469B"`; Steam rejects
/// a too-short/oddly-formatted one as "Invalid or missing unit serial number" and substitutes its
/// own — benign, but we present a clean 12-char one). Derived from [`deck_unit_id`] so the `0xAE`
/// serial reply and the `0x83` unit-id attrs stay consistent.
pub fn deck_serial(index: u8) -> String {
format!("PFDK{:08X}", deck_unit_id(index))
}
/// The neutral 64-byte Deck input report (header only, all controls released) — the report the
/// real-USB transports stream until the first [`serialize_deck_state`] call updates it.
pub fn neutral_deck_report() -> [u8; STEAM_REPORT_LEN] {
let mut r = [0u8; STEAM_REPORT_LEN];
r[0] = 0x01;
r[2] = ID_CONTROLLER_DECK_STATE;
r[3] = 0x3C;
r
}
/// Build the HID feature GET_REPORT reply for the host's last SET_REPORT command, for the *real-USB*
/// Deck (gadget + usbip). Steam's `GetControllerInfo` reads the `0x83` attributes + the `0xAE`
/// serial; **serving the real `0x83` blob is what stops Steam re-probing** (the gamepad-evdev churn).
/// The 9-attribute `0x83` layout + the `0xAE` string format were captured from a physical Deck via
/// hidraw. `unit_id` (see [`deck_unit_id`]) stamps a per-instance value into the device-id attrs.
///
/// Note this is the raw 64-byte EP0 feature payload (command id first, no report-id prefix) — the USB
/// control path, distinct from [`serial_reply`] which carries the UHID report-id byte the kernel
/// strips.
pub fn feature_reply(last_set: &[u8], serial: &str, unit_id: u32) -> [u8; STEAM_REPORT_LEN] {
let cmd = last_set.first().copied().unwrap_or(ID_GET_STRING_ATTRIBUTE);
let mut r = [0u8; STEAM_REPORT_LEN];
match cmd {
ID_GET_ATTRIBUTES_VALUES => {
// GET_ATTRIBUTES_VALUES: [0x83, 0x2d, then 9× (attr-id, value u32-LE)].
r[0] = ID_GET_ATTRIBUTES_VALUES;
r[1] = 0x2d;
let attrs: [(u8, u32); 9] = [
(0x01, 0x1205), // product id
(0x02, 0),
(0x0a, unit_id), // unit serial number (per-instance)
(0x04, unit_id ^ 0x5555_5555),
(0x09, 0x2e),
(0x0b, 0x0fa0),
(0x0d, 0),
(0x0c, 0),
(0x0e, 0),
];
let mut o = 2;
for (id, val) in attrs {
r[o] = id;
r[o + 1..o + 5].copy_from_slice(&val.to_le_bytes());
o += 5;
}
}
ID_GET_STRING_ATTRIBUTE => {
// GET_STRING_ATTRIBUTE: [0xAE, len, attr, ascii…]. The kernel validates the serial (attr
// 0x01) wants reply[2]==0x01 and 1<=len<=21; for other attrs we echo the requested id.
let attr = last_set.get(2).copied().unwrap_or(ATTRIB_STR_UNIT_SERIAL);
let b = serial.as_bytes();
let len = b.len().clamp(1, 20);
r[0] = ID_GET_STRING_ATTRIBUTE;
r[1] = len as u8;
r[2] = attr;
r[3..3 + len].copy_from_slice(&b[..len]);
}
_ => {
// Settings read-back (e.g. 0x87): echo the host's last command + data.
let n = last_set.len().min(STEAM_REPORT_LEN);
r[..n].copy_from_slice(&last_set[..n]);
}
}
r
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn descriptor_declares_input_and_feature_reports() {
assert!(
STEAMDECK_RDESC.contains(&0xB1),
"missing Feature main item — steam_is_valve_interface() would fail"
);
assert!(STEAMDECK_RDESC.contains(&0x81), "missing Input main item");
assert_eq!(
*STEAMDECK_RDESC.last().unwrap(),
0xC0,
"unterminated collection"
);
}
/// Every analog field lands at the exact offset `steam_do_deck_input_event` reads, the header is
/// what `steam_raw_event` requires, and the buttons pack into bytes 8..16 (12+15 zero). A
/// one-byte slip here turns the whole controller into noise.
#[test]
fn serialize_is_byte_exact() {
let mut st = SteamState::neutral();
st.buttons = btn::A | btn::L4 | btn::R5 | btn::QAM;
st.lx = 0x1122;
st.ly = 0x3344;
st.rx = 0x5566;
st.ry = 0x778;
st.lt = 0xABCD;
st.rt = 0xEF01;
st.lpad_x = 0x0A0B;
st.lpad_y = 0x0C0D;
st.rpad_x = 0x0E0F;
st.rpad_y = 0x1011;
st.accel = [0x0102, 0x0304, 0x0506];
st.gyro = [0x0708, 0x090A, 0x0B0C];
st.lpad_pressure = 0x1314;
st.rpad_pressure = 0x1516;
let mut r = [0u8; STEAM_REPORT_LEN];
serialize_deck_state(&mut r, &st, 0xAABB_CCDD);
assert_eq!(&r[0..4], &[0x01, 0x00, 0x09, 0x3C]);
assert_eq!(&r[4..8], &[0xDD, 0xCC, 0xBB, 0xAA]); // seq LE
// buttons: A=bit7 (byte8), L4=bit41 (byte13.1), R5=bit16 (byte10.0), QAM=bit50 (byte14.2).
assert_eq!(r[8], 0x80); // A
assert_eq!(r[10], 0x01); // R5
assert_eq!(r[12], 0x00); // unused button byte
assert_eq!(r[13], 0x02); // L4 (bit 1)
assert_eq!(r[14], 0x04); // QAM (bit 2)
assert_eq!(r[15], 0x00); // unused button byte
assert_eq!(&r[16..18], &0x0A0Bi16.to_le_bytes()); // lpad X
assert_eq!(&r[20..22], &0x0E0Fi16.to_le_bytes()); // rpad X
assert_eq!(&r[24..26], &0x0102i16.to_le_bytes()); // accel X
assert_eq!(&r[26..28], &0x0304i16.to_le_bytes()); // accel Y
assert_eq!(&r[28..30], &0x0506i16.to_le_bytes()); // accel Z
assert_eq!(&r[30..32], &0x0708i16.to_le_bytes()); // gyro X
assert_eq!(&r[44..46], &0xABCDu16.to_le_bytes()); // left trigger
assert_eq!(&r[46..48], &0xEF01u16.to_le_bytes()); // right trigger
assert_eq!(&r[48..50], &0x1122i16.to_le_bytes()); // left joy X
assert_eq!(&r[50..52], &0x3344i16.to_le_bytes()); // left joy Y
assert_eq!(&r[52..54], &0x5566i16.to_le_bytes()); // right joy X
assert_eq!(&r[56..58], &0x1314u16.to_le_bytes()); // left pad pressure
assert_eq!(&r[58..60], &0x1516u16.to_le_bytes()); // right pad pressure
}
/// `from_gamepad` sets the right Deck bits + scales triggers, and a touched flag is merged when
/// a trackpad contact arrives via `apply_rich`.
#[test]
fn from_gamepad_and_rich_mapping() {
let s = SteamState::from_gamepad(
gs::BTN_A | gs::BTN_START | gs::BTN_GUIDE | gs::BTN_LB,
1000,
-2000,
0,
0,
255,
0,
);
assert_ne!(s.buttons & btn::A, 0);
assert_ne!(s.buttons & btn::MENU, 0);
assert_ne!(s.buttons & btn::STEAM, 0);
assert_ne!(s.buttons & btn::LB, 0);
assert_ne!(s.buttons & btn::LT_FULL, 0); // lt=255 → full-pull bit
assert_eq!(s.lt, 255 * 128);
assert_eq!(s.lx, 1000);
assert_eq!(s.ly, -2000);
let mut s = SteamState::neutral();
s.apply_rich(RichInput::Touchpad {
pad: 0,
finger: 0,
active: true,
x: 65535,
y: 0,
});
assert_ne!(s.buttons & btn::RPAD_TOUCH, 0);
assert_eq!(s.rpad_x, 32767); // 65535-32768
assert_eq!(s.rpad_y, -32768); // 0-32768
// Motion is rescaled from the wire (DualSense) convention into Deck units (gyro ×16/20,
// accel ×16384/10000) — see steam_remap::motion_wire_to_deck.
s.apply_rich(RichInput::Motion {
pad: 0,
gyro: [1000, -2000, 0],
accel: [10000, -5000, 0],
});
assert_eq!(s.gyro, [800, -1600, 0]);
assert_eq!(s.accel, [16384, -8192, 0]);
}
/// M3: the wire back-button bits map to the four Deck grips + QAM, and `TouchpadEx` routes the
/// left / right surfaces to the matching pad (signed coords pass straight through).
#[test]
fn back_buttons_and_dual_trackpad_mapping() {
let s = SteamState::from_gamepad(
gs::BTN_PADDLE1 | gs::BTN_PADDLE2 | gs::BTN_PADDLE3 | gs::BTN_PADDLE4 | gs::BTN_MISC1,
0,
0,
0,
0,
0,
0,
);
assert_ne!(s.buttons & btn::R4, 0); // PADDLE1 = R4
assert_ne!(s.buttons & btn::L4, 0); // PADDLE2 = L4
assert_ne!(s.buttons & btn::R5, 0); // PADDLE3 = R5
assert_ne!(s.buttons & btn::L5, 0); // PADDLE4 = L5
assert_ne!(s.buttons & btn::QAM, 0); // MISC1 = QAM
let mut s = SteamState::neutral();
s.apply_rich(RichInput::TouchpadEx {
pad: 0,
surface: 1,
finger: 0,
touch: true,
click: true,
x: -5000,
y: 6000,
pressure: 100,
});
assert_ne!(s.buttons & btn::LPAD_TOUCH, 0);
assert_ne!(s.buttons & btn::LPAD_CLICK, 0);
assert_eq!((s.lpad_x, s.lpad_y), (-5000, 6000));
s.apply_rich(RichInput::TouchpadEx {
pad: 0,
surface: 2,
finger: 0,
touch: true,
click: false,
x: 7000,
y: -8000,
pressure: 0,
});
assert_ne!(s.buttons & btn::RPAD_TOUCH, 0);
assert_eq!((s.rpad_x, s.rpad_y), (7000, -8000));
}
/// The serial reply carries the leading report-id byte the kernel strips, so the *stripped*
/// view (`reply[1..]`) is what `steam_get_serial` validates: `[0xAE, len, 0x01, ascii…]`.
#[test]
fn serial_reply_has_stripped_prefix() {
let r = serial_reply("PUNKTFUNK01");
assert_eq!(r[0], 0x00); // report id, stripped by steam_recv_report
assert_eq!(r[1], ID_GET_STRING_ATTRIBUTE); // becomes reply[0] after strip
assert!((1..=21).contains(&r[2]));
assert_eq!(r[3], ATTRIB_STR_UNIT_SERIAL);
assert_eq!(&r[4..4 + r[2] as usize], b"PUNKTFUNK01");
}
/// A `0xEB` rumble feature report parses to `(left_speed, right_speed)`; other commands don't.
#[test]
fn parse_rumble_feedback() {
// [report-id 0, 0xEB, len 9, 0, intensity(2), left(2), right(2), gains(2)]
let mut d = vec![0u8; 12];
d[1] = ID_TRIGGER_RUMBLE_CMD;
d[2] = 9;
d[6..8].copy_from_slice(&0x8000u16.to_le_bytes()); // left_speed
d[8..10].copy_from_slice(&0x4000u16.to_le_bytes()); // right_speed
assert_eq!(parse_steam_output(&d).rumble, Some((0x8000, 0x4000)));
let mut d = vec![0u8; 12];
d[1] = ID_SET_SETTINGS_VALUES; // a settings write — no rumble
assert_eq!(parse_steam_output(&d).rumble, None);
}
/// The shared real-USB Deck feature contract (gadget + usbip): the `0x83` GET_ATTRIBUTES reply
/// carries the 9-attribute blob with the per-instance unit id, and the `0xAE` reply carries the
/// Steam-accepted serial — both keyed off the host's last SET_REPORT command. A slip here is the
/// gamepad-evdev churn (Steam re-probing).
#[test]
fn deck_feature_reply_contract() {
let serial = deck_serial(0);
let unit_id = deck_unit_id(0);
assert_eq!(serial, "PFDK50460000"); // 12-char alphanumeric, derived from the unit id
assert_eq!(serial.len(), 12);
// 0x83 GET_ATTRIBUTES_VALUES: header + (0x0a, unit_id) at the 3rd attribute slot.
let r = feature_reply(&[ID_GET_ATTRIBUTES_VALUES], &serial, unit_id);
assert_eq!(r[0], ID_GET_ATTRIBUTES_VALUES);
assert_eq!(r[1], 0x2d);
assert_eq!(r[12], 0x0a); // 3rd attr id (slots at 2,7,12,…)
assert_eq!(
u32::from_le_bytes([r[13], r[14], r[15], r[16]]),
unit_id,
"unit serial attribute must carry the per-instance unit id"
);
// 0xAE GET_STRING_ATTRIBUTE: [0xAE, len, attr(0x01), ascii serial…].
let r = feature_reply(
&[ID_GET_STRING_ATTRIBUTE, 0, ATTRIB_STR_UNIT_SERIAL],
&serial,
unit_id,
);
assert_eq!(r[0], ID_GET_STRING_ATTRIBUTE);
assert_eq!(r[1] as usize, serial.len());
assert_eq!(r[2], ATTRIB_STR_UNIT_SERIAL);
assert_eq!(&r[3..3 + serial.len()], serial.as_bytes());
// Distinct pad indices get distinct unit ids + serials (no collision between virtual Decks).
assert_ne!(deck_unit_id(0), deck_unit_id(1));
assert_ne!(deck_serial(0), deck_serial(1));
}
}
@@ -0,0 +1,149 @@
//! Pure fallback-remap policy for the Steam Controller / Steam Deck rich inputs when the resolved
//! host backend is **not** the virtual `hid-steam` device (DualSense / DualShock 4 / Xbox), so a
//! client's Steam-only inputs aren't silently dropped — plus the cross-device motion rescale the
//! Deck backend itself needs.
//!
//! Driven by the host's `PUNKTFUNK_STEAM_REMAP` env (`key=value`, `,`/`;`-separated, e.g.
//! `paddles=stickclicks`). No I/O beyond [`RemapConfig::from_env`]; everything else is pure +
//! unit-testable. The uinput Xbox pad already exposes the back grips as Elite paddles
//! (`BTN_TRIGGER_HAPPY5-8`), so only the slot-less DualSense / DS4 backends fold them.
use punktfunk_core::input::gamepad as gs;
/// Where the four Steam back grips go on a backend with no native back-button HID slot.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)]
pub enum PaddleFallback {
/// Drop them — the back buttons are simply absent on this pad. The honest default: don't fire
/// buttons the user didn't ask for. Set the env to map them instead.
#[default]
Drop,
/// L4/L5 → left-stick click, R4/R5 → right-stick click.
StickClicks,
/// L4/L5 → left bumper, R4/R5 → right bumper.
Shoulders,
}
/// Fallback-remap knobs parsed from `PUNKTFUNK_STEAM_REMAP`.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct RemapConfig {
pub paddles: PaddleFallback,
}
impl RemapConfig {
/// Parse the host's `PUNKTFUNK_STEAM_REMAP` env (absent / unrecognized → defaults).
pub fn from_env() -> RemapConfig {
std::env::var("PUNKTFUNK_STEAM_REMAP")
.map(|s| RemapConfig::parse(&s))
.unwrap_or_default()
}
/// Pure parse of the `key=value[,key=value…]` string (the testable core of [`from_env`]).
pub fn parse(s: &str) -> RemapConfig {
let mut cfg = RemapConfig::default();
for kv in s.split([',', ';']) {
let mut it = kv.splitn(2, '=');
if let (Some(k), Some(v)) = (it.next(), it.next()) {
if k.trim().eq_ignore_ascii_case("paddles") {
cfg.paddles = match v.trim().to_ascii_lowercase().as_str() {
"stickclicks" | "l3r3" | "sticks" => PaddleFallback::StickClicks,
"shoulders" | "lbrb" | "bumpers" => PaddleFallback::Shoulders,
_ => PaddleFallback::Drop,
};
}
}
}
cfg
}
}
/// Fold the wire back-grip bits (`BTN_PADDLE1..4`) into standard buttons per `policy` for a pad with
/// no native back-button slot, clearing the paddle bits. Pure. PADDLE1/2/3/4 = R4/L4/R5/L5.
pub fn fold_paddles(mut buttons: u32, policy: PaddleFallback) -> u32 {
let left = buttons & (gs::BTN_PADDLE2 | gs::BTN_PADDLE4) != 0; // L4 | L5
let right = buttons & (gs::BTN_PADDLE1 | gs::BTN_PADDLE3) != 0; // R4 | R5
buttons &= !(gs::BTN_PADDLE1 | gs::BTN_PADDLE2 | gs::BTN_PADDLE3 | gs::BTN_PADDLE4);
let (lbit, rbit) = match policy {
PaddleFallback::Drop => return buttons,
PaddleFallback::StickClicks => (gs::BTN_LS_CLICK, gs::BTN_RS_CLICK),
PaddleFallback::Shoulders => (gs::BTN_LB, gs::BTN_RB),
};
if left {
buttons |= lbit;
}
if right {
buttons |= rbit;
}
buttons
}
// Motion rescale. The wire uses the DualSense convention (20 LSB/°·s gyro, 10000 LSB/g accel — the
// scale every client capture applies). The Steam Deck's `hid-steam` report wants 16 LSB/°·s and
// 16384 LSB/g, so the Deck backend rescales; the DualSense / DS4 backends consume the wire 1:1.
const GYRO_NUM: i32 = 16;
const GYRO_DEN: i32 = 20;
const ACCEL_NUM: i32 = 16384;
const ACCEL_DEN: i32 = 10000;
fn scale(v: i16, num: i32, den: i32) -> i16 {
((v as i32 * num) / den).clamp(i16::MIN as i32, i16::MAX as i32) as i16
}
/// Rescale a wire (DualSense-convention) motion sample into the Steam Deck's `hid-steam` units.
pub fn motion_wire_to_deck(gyro: [i16; 3], accel: [i16; 3]) -> ([i16; 3], [i16; 3]) {
(
gyro.map(|g| scale(g, GYRO_NUM, GYRO_DEN)),
accel.map(|a| scale(a, ACCEL_NUM, ACCEL_DEN)),
)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_paddle_policy() {
assert_eq!(RemapConfig::parse("").paddles, PaddleFallback::Drop);
assert_eq!(
RemapConfig::parse("paddles=stickclicks").paddles,
PaddleFallback::StickClicks
);
assert_eq!(
RemapConfig::parse("foo=bar; paddles = Shoulders").paddles,
PaddleFallback::Shoulders
);
assert_eq!(
RemapConfig::parse("paddles=nonsense").paddles,
PaddleFallback::Drop
);
}
#[test]
fn fold_paddles_maps_and_clears() {
// All four grips set + a real A button.
let b = gs::BTN_A | gs::BTN_PADDLE1 | gs::BTN_PADDLE2 | gs::BTN_PADDLE3 | gs::BTN_PADDLE4;
// Drop: paddle bits cleared, A preserved, nothing added.
assert_eq!(fold_paddles(b, PaddleFallback::Drop), gs::BTN_A);
// StickClicks: left grips → L3, right grips → R3.
assert_eq!(
fold_paddles(b, PaddleFallback::StickClicks),
gs::BTN_A | gs::BTN_LS_CLICK | gs::BTN_RS_CLICK
);
// Only a left grip (L4 = PADDLE2) → only the left bumper under Shoulders.
assert_eq!(
fold_paddles(gs::BTN_PADDLE2, PaddleFallback::Shoulders),
gs::BTN_LB
);
}
#[test]
fn motion_rescale_to_deck_units() {
// gyro × 16/20 = 0.8; accel × 16384/10000 = 1.6384.
let (g, a) = motion_wire_to_deck([1000, -2000, 0], [10000, -5000, 0]);
assert_eq!(g, [800, -1600, 0]);
assert_eq!(a, [16384, -8192, 0]);
// Saturates rather than wraps.
let (_, a) = motion_wire_to_deck([0; 3], [32767, i16::MIN, 0]);
assert_eq!(a[0], i16::MAX);
assert_eq!(a[1], i16::MIN);
}
}
@@ -385,7 +385,9 @@ impl DualSenseWindowsManager {
/// Apply one rich client→host event (touchpad contact / motion sample) to an existing pad.
pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich {
RichInput::Touchpad { pad, .. } | RichInput::Motion { pad, .. } => pad as usize,
RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
};
if idx >= MAX_PADS || self.pads[idx].is_none() {
return;
@@ -409,6 +411,26 @@ impl DualSenseWindowsManager {
self.state[idx].gyro = gyro;
self.state[idx].accel = accel;
}
RichInput::TouchpadEx {
surface,
finger,
touch,
x,
y,
..
} => {
// A Steam right/single pad maps onto the one DualSense touchpad (signed centre-0 →
// 0..=65535); surface 1 (the Steam left pad) has no DualSense equivalent.
if surface != 1 {
let slot = (finger as usize).min(1);
let n = |v: i16| ((v as i32) + 32768) as u32;
let t = &mut self.state[idx].touch[slot];
t.active = touch;
t.id = slot as u8;
t.x = (n(x) * (DS_TOUCH_W - 1) as u32 / u16::MAX as u32) as u16;
t.y = (n(y) * (DS_TOUCH_H - 1) as u32 / u16::MAX as u32) as u16;
}
}
}
self.write(idx);
}
@@ -186,7 +186,9 @@ impl DualShock4WindowsManager {
/// Apply one rich client→host event (touchpad contact / motion sample) to an existing pad.
pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich {
RichInput::Touchpad { pad, .. } | RichInput::Motion { pad, .. } => pad as usize,
RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
};
if idx >= MAX_PADS || self.pads[idx].is_none() {
return;
@@ -210,6 +212,26 @@ impl DualShock4WindowsManager {
self.state[idx].gyro = gyro;
self.state[idx].accel = accel;
}
RichInput::TouchpadEx {
surface,
finger,
touch,
x,
y,
..
} => {
// A Steam right/single pad maps onto the one DS4 touchpad (signed centre-0 →
// 0..=65535); surface 1 (the Steam left pad) has no DS4 equivalent.
if surface != 1 {
let slot = (finger as usize).min(1);
let n = |v: i16| ((v as i32) + 32768) as u32;
let t = &mut self.state[idx].touch[slot];
t.active = touch;
t.id = slot as u8;
t.x = (n(x) * (DS4_TOUCH_W - 1) as u32 / u16::MAX as u32) as u16;
t.y = (n(y) * (DS4_TOUCH_H - 1) as u32 / u16::MAX as u32) as u16;
}
}
}
self.write(idx);
}
@@ -21,10 +21,18 @@ use windows::Win32::System::Memory::{
MEMORY_MAPPED_VIEW_ADDRESS, PAGE_READWRITE,
};
/// A named, anonymous (pagefile-backed) shared section + its mapped read/write view, created with the
/// permissive `D:(A;;GA;;;WD)` SDDL the restricted-token driver needs to open it. RAII: drop unmaps the
/// view, then the [`OwnedHandle`] closes the section handle (in that order). Replaces the three backends'
/// hand-duplicated `CreateFileMappingW` + `MapViewOfFile` + manual `Drop`.
/// A named, anonymous (pagefile-backed) shared section + its mapped read/write view. RAII: drop unmaps
/// the view, then the [`OwnedHandle`] closes the section handle (in that order). Replaces the three
/// backends' hand-duplicated `CreateFileMappingW` + `MapViewOfFile` + manual `Drop`.
///
/// SDDL `D:(A;;GA;;;SY)(A;;GA;;;LS)`: GENERIC_ALL to **SYSTEM** (the host creates the section and
/// writes the live HID input report into it) and **LocalService** (the account the UMDF driver's
/// WUDFHost runs under, which reads it). The old SDDL granted **Everyone** (`WD`) — on the (mistaken)
/// assumption the driver needed a restricted token's broad access — letting any local user
/// `OpenFileMapping` the section to inject controller input or tamper the trusted channel
/// (security-review 2026-06-28 #5). Verified on the RTX box (2026-06-29): the WUDFHost token is
/// `S-1-5-19` (LocalService), SYSTEM integrity, with **zero restricted SIDs** — so scoping to SY+LS is
/// sufficient for the driver and excludes normal (medium-IL, non-service) user processes.
pub(super) struct Shm {
/// Owns the section handle (closed on drop). Held only for ownership — never read after construction.
_handle: OwnedHandle,
@@ -40,7 +48,7 @@ impl Shm {
// exit — acceptable for a host-lifetime object).
unsafe {
ConvertStringSecurityDescriptorToSecurityDescriptorW(
w!("D:(A;;GA;;;WD)"),
w!("D:(A;;GA;;;SY)(A;;GA;;;LS)"),
SDDL_REVISION_1,
&mut psd,
None,
+23 -4
View File
@@ -394,6 +394,12 @@ struct ArmNativePairing {
/// Window length in seconds (default 120; clamped to 15600).
#[schema(example = 120)]
ttl_secs: Option<u32>,
/// Optional: bind the window to ONE device fingerprint (hex SHA-256, e.g. from a pending knock).
/// When set, only a pairing attempt from that fingerprint consumes the window — so an unpaired
/// LAN peer can neither pair nor burn a window armed for a specific device (security-review #9).
/// Omit for an unbound window (any device may use the PIN — trusted-LAN only).
#[schema(example = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08")]
fingerprint: Option<String>,
}
/// A paired native (punktfunk/1) client.
@@ -879,8 +885,21 @@ async fn arm_native_pairing(
);
};
let ttl = req.ttl_secs.unwrap_or(120).clamp(15, 600);
let _pin = np.arm(std::time::Duration::from_secs(ttl as u64));
tracing::info!(ttl_secs = ttl, "management API: native pairing armed");
// A bound window (operator selected a specific device) is DoS-proof: only that fingerprint can
// consume it (#9). An unbound window (no fingerprint) keeps the legacy any-device behavior.
let bound = req
.fingerprint
.as_deref()
.map(str::trim)
.filter(|s| !s.is_empty())
.map(|s| s.to_ascii_lowercase());
let bound_to_device = bound.is_some();
let _pin = np.arm_for(std::time::Duration::from_secs(ttl as u64), bound);
tracing::info!(
ttl_secs = ttl,
bound_to_device,
"management API: native pairing armed"
);
Json(native_status(&st)).into_response()
}
@@ -1975,8 +1994,8 @@ mod tests {
assert_eq!(b.as_array().unwrap().len(), 0);
// Two devices knock (what the QUIC gate records); they appear in the list.
np.note_pending("Enrico's MacBook", "aa11");
np.note_pending("device bb22cc33", "bb22");
np.note_pending("Enrico's MacBook", "aa11", None);
np.note_pending("device bb22cc33", "bb22", None);
let (_, b) = send(&app, get_req("/api/v1/native/pending")).await;
assert_eq!(b.as_array().unwrap().len(), 2);
assert_eq!(b[0]["name"], "Enrico's MacBook");
+204 -21
View File
@@ -8,6 +8,7 @@
//! armed on demand for a short window — rather than accepting one.
use anyhow::Result;
use std::net::IpAddr;
use std::path::PathBuf;
use std::sync::Mutex;
use std::time::{Duration, Instant};
@@ -42,10 +43,29 @@ struct PairedState {
/// The current arming window. `pin == None` ⇒ disarmed. `expires_at == None` ⇒ armed with no
/// expiry (the CLI `--allow-pairing` flag); `Some(t)` ⇒ a web-armed window that auto-disarms.
///
/// `bound_fp == Some(fp)` ⇒ the window is **bound to one operator-selected device fingerprint**:
/// only a pairing attempt from that fingerprint may consume it (security-review 2026-06-28 #9). This
/// closes the window-burn DoS — an unpaired LAN peer cannot consume a window armed for a specific
/// device, because the QUIC client-auth proves cert possession (it can't forge the bound fingerprint).
/// `None` ⇒ unbound (the CLI flag / a console "arm open"): any well-formed attempt consumes it (the
/// legacy behavior, retaining the window-burn DoS — acceptable only on a trusted LAN).
#[derive(Default)]
struct Armed {
pin: Option<String>,
expires_at: Option<Instant>,
bound_fp: Option<String>,
}
/// The result of resolving the armed PIN for a specific client fingerprint ([`NativePairing::pin_for_attempt`]).
pub enum PinAttempt {
/// No window is armed (disarmed/expired) — reject; do not run the ceremony.
Disarmed,
/// A window IS armed but **bound to a different fingerprint** — reject WITHOUT consuming it, so
/// an unrelated (attacker) fingerprint can't burn the operator's armed window (#9).
BoundToOther,
/// Proceed: the PIN to run the ceremony with (the window is unbound, or bound to this fingerprint).
Pin(String),
}
/// An unpaired (but identified) device that knocked on a pairing-required host — held for
@@ -57,6 +77,13 @@ struct Pending {
name: String,
fp_hex: String,
requested_at: Instant,
/// QUIC-validated source address of the knock — used for the per-source cap (#13), so one host
/// can't fill the queue. `None` if unknown (e.g. tests / a caller that doesn't supply it).
src_ip: Option<IpAddr>,
/// True while a connection is held open in [`NativePairing::wait_for_decision`] for this knock.
/// A live parked knock is a genuine device waiting for the operator — eviction skips it unless
/// every entry is parked, so a cert-rotating flood can't evict the device being onboarded (#13).
parked: bool,
}
#[derive(Default)]
@@ -94,6 +121,10 @@ pub enum PairingDecision {
const PENDING_TTL: Duration = Duration::from_secs(10 * 60);
/// Cap on the pending list — a LAN scanner must not grow it unboundedly. Oldest entries drop.
const PENDING_CAP: usize = 32;
/// Max pending knocks one source IP may occupy, so a single host can't fill the whole queue and hide
/// / evict a genuine device's knock (security-review 2026-06-28 #13). The QUIC path is address-
/// validated, so the source IP isn't off-path spoofable; an attacker would need that many real hosts.
const MAX_PENDING_PER_IP: usize = 4;
/// Shared native-pairing state: the arming PIN window + the persistent trust store + the
/// pending-approval queue.
@@ -209,6 +240,7 @@ impl NativePairing {
Armed {
pin: Some(fixed_pin.unwrap_or_else(random_pin)),
expires_at: None,
bound_fp: None,
}
} else {
Armed::default()
@@ -221,16 +253,43 @@ impl NativePairing {
})
}
/// Arm pairing with a fresh random PIN, valid for `ttl`. Returns the PIN to display.
/// Arm pairing with a fresh random PIN, valid for `ttl`, **unbound** (any well-formed attempt
/// consumes it). Returns the PIN to display. Prefer [`Self::arm_for`] with a specific device
/// fingerprint on untrusted LANs — an unbound window is burnable by any peer (#9).
pub fn arm(&self, ttl: Duration) -> String {
self.arm_for(ttl, None)
}
/// Arm pairing with a fresh random PIN, valid for `ttl`. If `bound_fp` is `Some`, the window is
/// bound to that device fingerprint: only a pairing attempt from it consumes the window, so an
/// unrelated (attacker) fingerprint can neither pair nor burn the window (#9). Returns the PIN.
pub fn arm_for(&self, ttl: Duration, bound_fp: Option<String>) -> String {
let pin = random_pin();
*self.arm.lock().unwrap() = Armed {
pin: Some(pin.clone()),
expires_at: Some(Instant::now() + ttl),
bound_fp,
};
pin
}
/// Resolve the PIN for an attempt from `client_fp_hex`, honoring fingerprint binding (#9):
/// `Disarmed` if no window is armed; `BoundToOther` if a window is armed but bound to a different
/// fingerprint (the caller MUST reject without consuming it); else `Pin` to run the ceremony.
pub fn pin_for_attempt(&self, client_fp_hex: &str) -> PinAttempt {
let mut arm = self.arm.lock().unwrap();
Self::expire(&mut arm);
match &arm.pin {
None => PinAttempt::Disarmed,
Some(pin) => match &arm.bound_fp {
Some(bound) if !bound.eq_ignore_ascii_case(client_fp_hex) => {
PinAttempt::BoundToOther
}
_ => PinAttempt::Pin(pin.clone()),
},
}
}
/// Disarm pairing (no new ceremonies accepted).
pub fn disarm(&self) {
*self.arm.lock().unwrap() = Armed::default();
@@ -342,11 +401,30 @@ impl NativePairing {
.retain(|p| p.requested_at.elapsed() < PENDING_TTL);
}
/// Record an unpaired device's knock for delegated approval. Re-knocks from the same
/// fingerprint refresh the existing entry in place (same id; a connect-retry loop must not spam
/// the list); a fresh fingerprint gets a new id, evicting the **least-recently-active** entry
/// past [`PENDING_CAP`]. The name is sanitized (untrusted; see [`sanitize_device_name`]).
pub fn note_pending(&self, name: &str, fp_hex: &str) {
/// Pick the entry to evict, optionally restricted to a single source IP: the least-recently-active
/// **non-parked** entry (a live parked knock is a genuine device awaiting the operator — never
/// evict it under load); only if every candidate is parked does it fall back to the oldest of
/// those (#13). Returns the index, or `None` if there's nothing to evict.
fn evict_index(items: &[Pending], only_ip: Option<IpAddr>) -> Option<usize> {
let pick = |allow_parked: bool| {
items
.iter()
.enumerate()
.filter(|(_, p)| only_ip.is_none_or(|ip| p.src_ip == Some(ip)))
.filter(|(_, p)| allow_parked || !p.parked)
.min_by_key(|(_, p)| p.requested_at)
.map(|(i, _)| i)
};
pick(false).or_else(|| pick(true))
}
/// Record an unpaired device's knock for delegated approval. Re-knocks from the same fingerprint
/// refresh the existing entry in place (same id; a connect-retry loop must not spam the list). A
/// fresh fingerprint gets a new id; the queue is bounded two ways so a flood can't crowd out a
/// genuine knock (#13): a **per-source-IP cap** ([`MAX_PENDING_PER_IP`]) means one host can hold at
/// most a few slots, and the global [`PENDING_CAP`] evicts the least-recently-active **non-parked**
/// entry (never a live, held-open parked knock). The name is sanitized (untrusted).
pub fn note_pending(&self, name: &str, fp_hex: &str, src_ip: Option<IpAddr>) {
let name = sanitize_device_name(name, fp_hex);
let mut pending = self.pending.lock().unwrap();
Self::expire_pending(&mut pending);
@@ -357,19 +435,31 @@ impl NativePairing {
{
p.requested_at = Instant::now();
p.name = name;
if p.src_ip.is_none() {
p.src_ip = src_ip;
}
return;
}
if pending.items.len() >= PENDING_CAP {
// Evict the least-recently-active entry. NOT index 0: the in-place refresh above means
// Vec order no longer tracks recency, so pick the minimum `requested_at` explicitly.
if let Some(at) = pending
// Per-source-IP cap: a single host can't occupy more than MAX_PENDING_PER_IP slots — evict its
// own oldest entry first so it can't crowd out other devices' knocks (#13).
if let Some(ip) = src_ip {
if pending
.items
.iter()
.enumerate()
.min_by_key(|(_, p)| p.requested_at)
.map(|(i, _)| i)
.filter(|p| p.src_ip == Some(ip))
.count()
>= MAX_PENDING_PER_IP
{
pending.items.remove(at);
if let Some(i) = Self::evict_index(&pending.items, Some(ip)) {
pending.items.remove(i);
}
}
}
// Global cap: evict the least-recently-active non-parked entry (Vec order no longer tracks
// recency after in-place refreshes, so pick explicitly).
if pending.items.len() >= PENDING_CAP {
if let Some(i) = Self::evict_index(&pending.items, None) {
pending.items.remove(i);
}
}
let id = pending.next_id;
@@ -379,9 +469,24 @@ impl NativePairing {
name,
fp_hex: fp_hex.to_string(),
requested_at: Instant::now(),
src_ip,
parked: false,
});
}
/// Mark/unmark the pending entry for `fp_hex` as having a live parked waiter (no-op if it's gone).
/// A parked entry is protected from eviction under load (#13).
fn set_parked(&self, fp_hex: &str, parked: bool) {
let mut pending = self.pending.lock().unwrap();
if let Some(p) = pending
.items
.iter_mut()
.find(|p| p.fp_hex.eq_ignore_ascii_case(fp_hex))
{
p.parked = parked;
}
}
/// The devices currently awaiting approval (for the management API).
pub fn pending(&self) -> Vec<PendingRequest> {
let mut pending = self.pending.lock().unwrap();
@@ -462,6 +567,23 @@ impl NativePairing {
/// to keep the knocking connection open until a human clicks Approve — so the device pairs and
/// streams with no reconnect (delegated approval, roadmap §8b-1).
pub async fn wait_for_decision(&self, fp_hex: &str, timeout: Duration) -> PairingDecision {
// Mark this knock parked so a cert-rotating flood can't evict the genuine, held-open
// connection out of the pending queue while the operator decides (#13). Cleared on every
// exit path by the guard's Drop.
self.set_parked(fp_hex, true);
struct ParkGuard<'a> {
np: &'a NativePairing,
fp: &'a str,
}
impl Drop for ParkGuard<'_> {
fn drop(&mut self) {
self.np.set_parked(self.fp, false);
}
}
let _park = ParkGuard {
np: self,
fp: fp_hex,
};
let deadline = tokio::time::Instant::now() + timeout;
loop {
// Arm the wakeup BEFORE re-reading state, and `enable()` it, so an approve/deny that
@@ -548,8 +670,8 @@ mod tests {
// A knock appears; a re-knock from the same fingerprint refreshes (same id, new name)
// instead of duplicating.
np.note_pending("device aa11", "AA11");
np.note_pending("Bedroom TV", "aa11");
np.note_pending("device aa11", "AA11", None);
np.note_pending("Bedroom TV", "aa11", None);
let pend = np.pending();
assert_eq!(pend.len(), 1, "re-knock dedups by fingerprint");
assert_eq!(pend[0].name, "Bedroom TV");
@@ -562,7 +684,7 @@ mod tests {
assert!(!np.is_paired("aa11"));
// Approve pairs the fingerprint (operator label wins) and clears the entry.
np.note_pending("device bb22", "BB22");
np.note_pending("device bb22", "BB22", None);
let id = np.pending()[0].id;
assert!(
np.approve_pending(9999, None).unwrap().is_none(),
@@ -578,8 +700,11 @@ mod tests {
assert_eq!(np.list()[0].name, "Living Room");
// The cap evicts the oldest knock.
// Flood from many DISTINCT source IPs (so the per-IP cap doesn't kick in) → the global cap
// holds at PENDING_CAP, evicting the oldest non-parked entries first.
for i in 0..(PENDING_CAP + 3) {
np.note_pending("flood", &format!("f{i:03}"));
let ip = IpAddr::from([10, 0, (i / 256) as u8, (i % 256) as u8]);
np.note_pending("flood", &format!("f{i:03}"), Some(ip));
}
let pend = np.pending();
assert_eq!(pend.len(), PENDING_CAP);
@@ -610,7 +735,7 @@ mod tests {
let p = temp();
let _ = std::fs::remove_file(&p);
let np = NativePairing::load_with(Some(p.clone()), None, false).unwrap();
np.note_pending("Knocker", "cc44");
np.note_pending("Knocker", "cc44", None);
assert_eq!(np.pending().len(), 1);
// Pairing the same fingerprint (e.g. via the PIN ceremony) drops the stale pending entry.
np.add("Knocker", "CC44").unwrap();
@@ -656,7 +781,7 @@ mod tests {
let np = Arc::new(NativePairing::load_with(Some(p.clone()), None, false).unwrap());
// TimedOut: a parked knock with no decision returns TimedOut; the entry survives.
np.note_pending("Knocker", "ab01");
np.note_pending("Knocker", "ab01", None);
let d = np
.wait_for_decision("ab01", Duration::from_millis(80))
.await;
@@ -681,7 +806,7 @@ mod tests {
assert!(np.is_paired("ab01"));
// Denied: denying WHILE parked wakes the waiter with Denied (not held until timeout).
np.note_pending("Knock2", "cd02");
np.note_pending("Knock2", "cd02", None);
let np3 = np.clone();
let waiter =
tokio::spawn(
@@ -703,4 +828,62 @@ mod tests {
assert_eq!(d, PairingDecision::Approved);
let _ = std::fs::remove_file(&p);
}
/// #9: a window can be bound to one operator-selected fingerprint, so an unrelated (attacker)
/// fingerprint can neither pair nor BURN the window (it's rejected without a PIN).
#[test]
fn armed_pin_is_fingerprint_bindable() {
let p = temp();
let _ = std::fs::remove_file(&p);
let np = NativePairing::load_with(Some(p.clone()), None, false).unwrap();
// Unbound: any fingerprint resolves to the PIN (legacy behavior).
let pin = np.arm(Duration::from_secs(60));
assert!(matches!(np.pin_for_attempt("aa11"), PinAttempt::Pin(x) if x == pin));
assert!(matches!(np.pin_for_attempt("bb22"), PinAttempt::Pin(_)));
// Bound to AA11: only that fp (case-insensitive) gets the PIN; another fp is BoundToOther —
// the caller rejects it WITHOUT consuming the window.
let pin = np.arm_for(Duration::from_secs(60), Some("AA11".into()));
assert!(matches!(np.pin_for_attempt("aa11"), PinAttempt::Pin(x) if x == pin));
assert!(matches!(
np.pin_for_attempt("bb22"),
PinAttempt::BoundToOther
));
np.disarm();
assert!(matches!(np.pin_for_attempt("aa11"), PinAttempt::Disarmed));
let _ = std::fs::remove_file(&p);
}
/// #13: one source IP can't exceed the per-IP cap, and a parked (held-open) genuine knock is
/// never evicted by a flood — even one that fills the global cap from many distinct IPs.
#[test]
fn pending_per_ip_cap_and_parked_protection() {
let p = temp();
let _ = std::fs::remove_file(&p);
let np = NativePairing::load_with(Some(p.clone()), None, false).unwrap();
// Per-IP cap: one source flooding distinct fingerprints holds at most MAX_PENDING_PER_IP.
let attacker = IpAddr::from([192, 168, 1, 66]);
for i in 0..20 {
np.note_pending("flood", &format!("atk{i:03}"), Some(attacker));
}
assert_eq!(
np.pending().len(),
MAX_PENDING_PER_IP,
"one IP can't exceed the per-IP cap"
);
// A genuine knock from a different IP, parked (a live held-open connection), survives a flood
// from many distinct IPs that fills the global cap.
let legit = IpAddr::from([192, 168, 1, 50]);
np.note_pending("Living Room", "legit01", Some(legit));
np.set_parked("legit01", true);
for i in 0..(PENDING_CAP * 2) {
let ip = IpAddr::from([10, 0, (i / 256) as u8, (i % 256) as u8]);
np.note_pending("flood2", &format!("g{i:04}"), Some(ip));
}
assert!(
np.pending_contains("legit01"),
"a parked, held-open knock is never evicted by a flood"
);
assert!(np.pending().len() <= PENDING_CAP, "global cap still holds");
let _ = std::fs::remove_file(&p);
}
}
+130 -5
View File
@@ -532,10 +532,25 @@ async fn serve_session(
.await
.map_err(|_| anyhow!("first message timeout"))??;
if let Ok(req) = PairRequest::decode(&first) {
// Read the live arming PIN per attempt, so a window that lapsed no longer pairs.
let pin = np
.current_pin()
.context("pairing not armed (arm it in the console, or start with --allow-pairing)")?;
// The client fingerprint (cert possession is proven by the QUIC handshake) is needed to honor
// a fingerprint-bound PIN window (#9): a window the operator armed for a SPECIFIC device must
// not be consumable — or burnable — by any other fingerprint.
let client_fp = endpoint::peer_fingerprint(&conn)
.ok_or_else(|| anyhow!("pairing requires the client to present a certificate"))?;
let client_fp_hex = fingerprint_hex(&client_fp);
// Resolve the live arming PIN per attempt (so a lapsed window no longer pairs), honoring any
// fingerprint binding.
let pin = match np.pin_for_attempt(&client_fp_hex) {
crate::native_pairing::PinAttempt::Pin(pin) => pin,
crate::native_pairing::PinAttempt::Disarmed => anyhow::bail!(
"pairing not armed (arm it in the console, or start with --allow-pairing)"
),
// Armed for a DIFFERENT device — reject without running the ceremony, so this attempt does
// NOT consume (burn) the operator's window for the device they actually selected (#9).
crate::native_pairing::PinAttempt::BoundToOther => anyhow::bail!(
"pairing is armed for a different device — this attempt does not consume the window"
),
};
{
let mut last = last_pairing.lock().unwrap();
if let Some(t) = *last {
@@ -589,7 +604,9 @@ async fn serve_session(
);
tracing::info!(name = %label, fingerprint = %fp_hex,
"unpaired device knocked — parking connection for delegated approval in the console");
np.note_pending(&label, &fp_hex);
// Record the QUIC-validated source IP so the pending queue's per-source cap can stop one
// host from flooding/evicting genuine knocks (#13).
np.note_pending(&label, &fp_hex, Some(peer.ip()));
// Free the session slot while a human decides — a parked knock must not hold an NVENC
// permit (a handful of parked knocks would otherwise block every real session).
drop(permit);
@@ -1382,6 +1399,8 @@ enum PadBackend {
DualSense(crate::inject::dualsense::DualSenseManager),
#[cfg(target_os = "linux")]
DualShock4(crate::inject::dualshock4::DualShock4Manager),
#[cfg(target_os = "linux")]
SteamDeck(crate::inject::steam_controller::SteamControllerManager),
#[cfg(target_os = "windows")]
DualSenseWindows(crate::inject::dualsense_windows::DualSenseWindowsManager),
#[cfg(target_os = "windows")]
@@ -1403,6 +1422,12 @@ impl PadBackend {
tracing::info!("gamepad backend: virtual DualShock 4 (UHID hid-playstation)");
return PadBackend::DualShock4(crate::inject::dualshock4::DualShock4Manager::new());
}
GamepadPref::SteamDeck => {
tracing::info!("gamepad backend: virtual Steam Deck (UHID hid-steam)");
return PadBackend::SteamDeck(
crate::inject::steam_controller::SteamControllerManager::new(),
);
}
GamepadPref::XboxOne => {
tracing::info!("gamepad backend: uinput X-Box One/Series pad");
return PadBackend::Xbox360(crate::inject::gamepad::GamepadManager::with_identity(
@@ -1438,6 +1463,8 @@ impl PadBackend {
PadBackend::DualSense(m) => m.handle(ev),
#[cfg(target_os = "linux")]
PadBackend::DualShock4(m) => m.handle(ev),
#[cfg(target_os = "linux")]
PadBackend::SteamDeck(m) => m.handle(ev),
#[cfg(target_os = "windows")]
PadBackend::DualSenseWindows(m) => m.handle(ev),
#[cfg(target_os = "windows")]
@@ -1454,6 +1481,8 @@ impl PadBackend {
PadBackend::DualSense(m) => m.apply_rich(_rich),
#[cfg(target_os = "linux")]
PadBackend::DualShock4(m) => m.apply_rich(_rich),
#[cfg(target_os = "linux")]
PadBackend::SteamDeck(m) => m.apply_rich(_rich),
#[cfg(target_os = "windows")]
PadBackend::DualSenseWindows(m) => m.apply_rich(_rich),
#[cfg(target_os = "windows")]
@@ -1479,6 +1508,8 @@ impl PadBackend {
PadBackend::DualSense(m) => m.pump(rumble, hidout),
#[cfg(target_os = "linux")]
PadBackend::DualShock4(m) => m.pump(rumble, hidout),
#[cfg(target_os = "linux")]
PadBackend::SteamDeck(m) => m.pump(rumble, hidout),
#[cfg(target_os = "windows")]
PadBackend::DualSenseWindows(m) => m.pump(rumble, hidout),
#[cfg(target_os = "windows")]
@@ -1498,6 +1529,8 @@ impl PadBackend {
PadBackend::DualSense(m) => m.heartbeat(std::time::Duration::from_millis(8)),
#[cfg(target_os = "linux")]
PadBackend::DualShock4(m) => m.heartbeat(std::time::Duration::from_millis(8)),
#[cfg(target_os = "linux")]
PadBackend::SteamDeck(m) => m.heartbeat(std::time::Duration::from_millis(8)),
#[cfg(target_os = "windows")]
PadBackend::DualSenseWindows(m) => m.heartbeat(std::time::Duration::from_millis(8)),
#[cfg(target_os = "windows")]
@@ -1877,10 +1910,94 @@ fn pick_gamepad(pref: GamepadPref, env: Option<&str>, linux: bool, windows: bool
// One/Series: a real, distinct uinput identity on Linux; folded into the 360 backend on
// Windows (XInput can't tell them apart anyway).
GamepadPref::XboxOne if linux => GamepadPref::XboxOne,
// Steam Deck: Linux UHID hid-steam. The classic Steam Controller's backend isn't built yet,
// so it folds to Xbox360 for now (Windows Steam devices are M7).
GamepadPref::SteamDeck if linux => GamepadPref::SteamDeck,
_ => GamepadPref::Xbox360,
}
}
/// Runtime degrade for the Linux UHID backends (DualSense / DualShock 4 / Steam Deck): if
/// `/dev/uhid` can't be opened for write *now*, fall back to the uinput X-Box 360 pad rather than a
/// dead controller (the UHID device-create would just fail). Cheap — opens + drops the char device,
/// no `UHID_CREATE2`, so no device is created. A no-op on non-Linux (those backends are UMDF/uinput).
#[cfg(target_os = "linux")]
fn degrade_if_no_uhid(chosen: GamepadPref) -> GamepadPref {
let needs_uhid = matches!(
chosen,
GamepadPref::DualSense | GamepadPref::DualShock4 | GamepadPref::SteamDeck
);
if needs_uhid
&& std::fs::OpenOptions::new()
.write(true)
.open("/dev/uhid")
.is_err()
{
tracing::warn!(
wanted = chosen.as_str(),
"/dev/uhid not writable — falling back to the X-Box 360 pad"
);
return GamepadPref::Xbox360;
}
chosen
}
#[cfg(not(target_os = "linux"))]
fn degrade_if_no_uhid(chosen: GamepadPref) -> GamepadPref {
chosen
}
/// True if a **physical** Valve Steam controller (`28DE`) is already attached. The host's own Steam
/// Input is then managing a `28DE` device, and presenting a second (virtual) one makes Steam juggle
/// two Decks — confirmed conflict-prone on a Deck-as-host (the physical `28DE:1205` + Steam's
/// `28DE:11FF` XInput output pad are both live). HID device dirs are named `BUS:VID:PID.INST`
/// (uppercase); a UHID virtual device resolves through `/devices/virtual/…`, a real one does not.
#[cfg(target_os = "linux")]
fn physical_steam_controller_present() -> bool {
let Ok(entries) = std::fs::read_dir("/sys/bus/hid/devices") else {
return false;
};
entries.flatten().any(|e| {
if !e.file_name().to_string_lossy().contains(":28DE:") {
return false;
}
match std::fs::read_link(e.path()) {
Ok(target) => !target.to_string_lossy().contains("/virtual/"),
Err(_) => true,
}
})
}
/// Gate a virtual Steam pad off when a physical Steam controller is attached (§ conflict). Degrade to
/// DualSense (then the uhid ladder), which Steam treats as an ordinary, distinct pad. Override with
/// `PUNKTFUNK_STEAM_FORCE=1` when the host has no competing Steam Input (e.g. a remote-only box).
#[cfg(target_os = "linux")]
fn degrade_steam_on_conflict(chosen: GamepadPref) -> GamepadPref {
if !matches!(
chosen,
GamepadPref::SteamDeck | GamepadPref::SteamController
) {
return chosen;
}
let forced = std::env::var("PUNKTFUNK_STEAM_FORCE")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(false);
if !forced && physical_steam_controller_present() {
tracing::warn!(
wanted = chosen.as_str(),
"a physical Steam controller is attached — the host's Steam Input would manage two 28DE \
devices; falling back to DualSense (set PUNKTFUNK_STEAM_FORCE=1 to override)"
);
return degrade_if_no_uhid(GamepadPref::DualSense);
}
chosen
}
#[cfg(not(target_os = "linux"))]
fn degrade_steam_on_conflict(chosen: GamepadPref) -> GamepadPref {
chosen
}
/// Resolve the client's gamepad-backend preference (the env/logging shell around
/// [`pick_gamepad`]). Always concrete — the `Welcome` reports what the session will drive.
fn resolve_gamepad(pref: GamepadPref) -> GamepadPref {
@@ -1891,6 +2008,14 @@ fn resolve_gamepad(pref: GamepadPref) -> GamepadPref {
cfg!(target_os = "linux"),
cfg!(target_os = "windows"),
);
// Runtime degrade (separate from the compile-time platform check above): the Linux UHID
// backends need `/dev/uhid` usable *now*, else creating the device just fails and the controller
// goes dead — fall back to the always-available uinput X-Box 360 pad instead.
let chosen = degrade_if_no_uhid(chosen);
// Conflict gate: don't present a virtual Steam (28DE) pad when the host already has a physical
// Steam controller — its own Steam Input would then manage two Decks (confirmed conflict-prone on
// a Deck-as-host). `PUNKTFUNK_STEAM_FORCE=1` overrides.
let chosen = degrade_steam_on_conflict(chosen);
match pref {
GamepadPref::Auto => {
// The operator's env knob deserves a diagnostic when it didn't drive the
@@ -327,17 +327,17 @@ impl VirtualDisplayManager {
}
});
// Windows defaults a new IddCx monitor into CLONE mode when a physical display is already
// active (a laptop panel, an attached monitor): the cloned IDD shares that display's source, so
// the OS never commits a distinct path for it and capture sees no frames. Force EXTEND first so
// the IDD comes up as its OWN active path; the resolve loop below then finds it. Idempotent /
// no-op on a sole-display box, so it's safe on the headless single-GPU path too.
// SAFETY: `force_extend_topology` only calls `SetDisplayConfig` (a CCD topology apply) with no
// borrowed caller memory; it runs under the manager `state` lock, the sole topology mutator.
unsafe { force_extend_topology() };
// Resolve the capture target. May be None on a GPU-less box (target added but not WDDM-activated);
// Resolve the capture target — wait for Windows to auto-activate the freshly-ADDed IDD into its
// OWN display path (it comes up EXTENDED alongside any existing/basic display; `set_active_mode`
// below then promotes it to primary and `isolate_displays_ccd` makes it the sole composited
// desktop — the proven flow). May be None on a GPU-less box (target added but not WDDM-activated);
// the capture backend re-resolves once a GPU is present.
//
// We do NOT force a topology change FIRST: the bare `SDC_TOPOLOGY_EXTEND` preset is ACCESS_DENIED
// from our Session-0 service context on a headless box and BREAKS this auto-activate (it regressed
// the headless path — the IDD then never gets its own path → "not an active display path" → black).
// force-EXTEND is only the FALLBACK below, for an integrated-screen box where a fresh IDD is CLONED
// onto the panel (shares its source) instead of getting its own path.
let mut gdi_name = None;
for _ in 0..15 {
thread::sleep(Duration::from_millis(200));
@@ -349,6 +349,32 @@ impl VirtualDisplayManager {
break;
}
}
// Fallback for an integrated-screen box (e.g. a laptop panel): Windows CLONES a freshly-added
// IDD onto the existing display, sharing its source, so it never gets its own committed path. On
// the IddCx clone behaviour observed live (commit 8e87e61, an Intel-iGPU + NVIDIA-Optimus laptop)
// `resolve_gdi_name` then stays None — so this `is_none()` fallback fires, force-EXTENDs to
// de-clone, and the second resolve finds the now-committed path. Headless/extended boxes already
// resolved above (the IDD auto-activates with its OWN source) and skip this — which is the whole
// point, since force-EXTEND's bare preset is ACCESS_DENIED from our service context there.
//
// CAVEAT (unobserved for IddCx, untested across GPU/driver/OS): textbook CCD also lets a clone
// appear as a *shared-source ACTIVE* path (resolve → Some), which this `is_none()` gate would NOT
// catch. If that ever shows up, widen the gate to also fire when the IDD target's source is shared
// with another active path (a `target_is_cloned` helper) — needs on-laptop validation first.
if gdi_name.is_none() {
// SAFETY: as above — `force_extend_topology` only calls `SetDisplayConfig` (CCD) with no
// borrowed caller memory, under the `state` lock.
unsafe { force_extend_topology() };
for _ in 0..15 {
thread::sleep(Duration::from_millis(200));
// SAFETY: as the resolve loop above.
if let Some(n) = unsafe { resolve_gdi_name(added.target_id) } {
gdi_name = Some(n);
break;
}
}
}
let mut ccd_saved: Option<SavedConfig> = None;
match &gdi_name {
Some(n) => {
+33
View File
@@ -0,0 +1,33 @@
# Vendored + trimmed copy of the `usbip` crate (jiegec/usbip v0.8.0, MIT), reduced to the
# USB/IP *server simulation* path only: we present a virtual Steam Deck and let the local
# `vhci_hcd` attach it. The upstream crate hard-depends on `rusb`→`libusb1-sys` (for its USB
# *host* mode, which we do not use and which would add a libusb runtime dep + break `musl`),
# so the host modules (`host.rs`, the `rusb`/`nusb` device constructors) and the helper
# interface handlers (`cdc.rs`/`hid.rs`) are removed. What remains — the device model, the
# USB/IP protocol framing, and the `UsbInterfaceHandler` trait — is pure `std` + `tokio` and
# carries no libusb dependency. See `NOTICE` for upstream attribution.
[package]
name = "usbip-sim"
version = "0.8.0"
edition = "2021"
description = "Trimmed usbip server-simulation core (no libusb) — vendored for the virtual Steam Deck"
license = "MIT"
publish = false
[lib]
name = "usbip_sim"
path = "src/lib.rs"
[dependencies]
log = "0.4"
num-derive = "0.4"
num-traits = "0.2"
# `time` is for the interrupt-IN pacing added in device.rs (punktfunk modification — see NOTICE).
tokio = { version = "1", features = ["rt", "net", "io-util", "sync", "time"] }
# Upstream gated its struct derives behind a `serde` feature; kept (off by default) so the
# `#[cfg(feature = "serde")]` attributes stay valid and the vendored diff stays minimal.
serde = { version = "1", features = ["derive"], optional = true }
[features]
default = []
serde = ["dep:serde"]
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2020-2025 Jiajie Chen
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+23
View File
@@ -0,0 +1,23 @@
This crate (`usbip-sim`) is a vendored, trimmed copy of:
usbip v0.8.0
Copyright (c) Jiajie Chen <c@jia.je> and contributors
https://github.com/jiegec/usbip
Licensed under the MIT License.
Modifications by the punktfunk project:
- Removed the USB host modules (`src/host.rs`) and the `rusb`/`nusb` device
constructors in `src/lib.rs` (`with_rusb_*`, `with_nusb_*`, `new_from_host*`),
eliminating the libusb runtime dependency (which also broke `musl`).
- Removed the example helper interface handlers `src/cdc.rs` and `src/hid.rs`.
- Replaced the `rusb::Direction` re-export and `rusb::Version` conversions with
local definitions.
- Dropped the in-crate test modules (kept the library surface only).
- Paced interrupt/bulk IN endpoint transfers by bInterval in `device.rs`
`handle_urb` (so a simulated interrupt-IN mimics a real device's
NAK-until-bInterval behaviour rather than free-running over the loopback
link); added the tokio `time` feature for it.
Only the USB/IP server *simulation* path is retained: the device model, the
USB/IP wire protocol, and the `UsbInterfaceHandler` trait. The original MIT
license text is reproduced in LICENSE-MIT.
+122
View File
@@ -0,0 +1,122 @@
use super::*;
/// A list of known USB speeds
#[derive(Copy, Clone, Debug)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub enum UsbSpeed {
Unknown = 0x0,
Low,
Full,
High,
Wireless,
Super,
SuperPlus,
}
/// A list of defined USB class codes
// https://www.usb.org/defined-class-codes
#[derive(Copy, Clone, Debug)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub enum ClassCode {
SeeInterface = 0,
Audio,
CDC,
HID,
Physical = 0x05,
Image,
Printer,
MassStorage,
Hub,
CDCData,
SmartCard,
ContentSecurity = 0x0D,
Video,
PersonalHealthcare,
AudioVideo,
Billboard,
TypeCBridge,
Diagnostic = 0xDC,
WirelessController = 0xE0,
Misc = 0xEF,
ApplicationSpecific = 0xFE,
VendorSpecific = 0xFF,
}
/// A list of defined USB endpoint attributes
#[derive(Copy, Clone, Debug, FromPrimitive)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub enum EndpointAttributes {
Control = 0,
Isochronous,
Bulk,
Interrupt,
}
/// USB endpoint direction: IN or OUT.
///
/// Upstream re-exported `rusb::Direction`; vendored locally so this crate carries no libusb
/// dependency. `UsbEndpoint::direction()` returns this, and `device.rs` matches on the variants.
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub enum Direction {
/// Host → device (`bEndpointAddress` bit 7 clear).
Out,
/// Device → host (`bEndpointAddress` bit 7 set).
In,
}
/// Emulated max packet size of EP0
pub const EP0_MAX_PACKET_SIZE: u16 = 64;
/// A list of defined USB standard requests
/// from USB 2.0 standard Table 9.4. Standard Request Codes
#[derive(Copy, Clone, Debug, FromPrimitive)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub enum StandardRequest {
GetStatus = 0,
ClearFeature = 1,
SetFeature = 3,
SetAddress = 5,
GetDescriptor = 6,
SetDescriptor = 7,
GetConfiguration = 8,
SetConfiguration = 9,
GetInterface = 10,
SetInterface = 11,
SynchFrame = 12,
}
/// A list of defined USB descriptor types
/// from USB 2.0 standard Table 9.5. Descriptor Types
#[derive(Copy, Clone, Debug, FromPrimitive)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub enum DescriptorType {
/// DEVICE
Device = 1,
/// CONFIGURATION
Configuration = 2,
/// STRING
String = 3,
/// INTERFACE
Interface = 4,
/// ENDPOINT
Endpoint = 5,
/// DEVICE_QUALIFIER
DeviceQualifier = 6,
/// OTHER_SPEED_CONFIGURATION
OtherSpeedConfiguration = 7,
/// INTERFACE_POINTER
InterfacePower = 8,
/// OTG
OTG = 9,
/// DEBUG
Debug = 0xA,
/// INTERFACE_ASSOCIATION
InterfaceAssociation = 0xB,
/// BOS
BOS = 0xF,
// DEVICE CAPABILITY
DeviceCapability = 0x10,
/// SUPERSPEED_USB_ENDPOINT_COMPANION
SuperspeedUsbEndpointCompanion = 0x30,
}
+555
View File
@@ -0,0 +1,555 @@
use super::*;
#[derive(Clone, Default, Debug)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub struct Version {
pub major: u8,
pub minor: u8,
pub patch: u8,
}
// (Upstream's `From<rusb::Version>` conversions removed — this crate has no libusb dependency.)
/// bcdDevice
impl From<u16> for Version {
fn from(value: u16) -> Self {
Self {
major: (value >> 8) as u8,
minor: ((value >> 4) & 0xF) as u8,
patch: (value & 0xF) as u8,
}
}
}
/// Represent a USB device
#[derive(Clone, Default, Debug)]
#[cfg_attr(feature = "serde", derive(Serialize))]
pub struct UsbDevice {
pub path: String,
pub bus_id: String,
pub bus_num: u32,
pub dev_num: u32,
pub speed: u32,
pub vendor_id: u16,
pub product_id: u16,
pub device_bcd: Version,
pub device_class: u8,
pub device_subclass: u8,
pub device_protocol: u8,
pub configuration_value: u8,
pub num_configurations: u8,
pub interfaces: Vec<UsbInterface>,
#[cfg_attr(feature = "serde", serde(skip))]
pub device_handler: Option<Arc<Mutex<Box<dyn UsbDeviceHandler + Send>>>>,
pub usb_version: Version,
pub(crate) ep0_in: UsbEndpoint,
pub(crate) ep0_out: UsbEndpoint,
// strings
pub(crate) string_pool: HashMap<u8, String>,
pub(crate) string_configuration: u8,
pub(crate) string_manufacturer: u8,
pub(crate) string_product: u8,
pub(crate) string_serial: u8,
}
impl UsbDevice {
pub fn new(index: u32) -> Self {
let mut res = Self {
path: "/sys/bus/0/0/0".to_string(),
bus_id: "0-0-0".to_string(),
dev_num: index,
speed: UsbSpeed::High as u32,
ep0_in: UsbEndpoint {
address: 0x80,
attributes: EndpointAttributes::Control as u8,
max_packet_size: EP0_MAX_PACKET_SIZE,
interval: 0,
},
ep0_out: UsbEndpoint {
address: 0x00,
attributes: EndpointAttributes::Control as u8,
max_packet_size: EP0_MAX_PACKET_SIZE,
interval: 0,
},
// configured by default
configuration_value: 1,
num_configurations: 1,
..Self::default()
};
res.string_configuration = res.new_string("Default Configuration");
res.string_manufacturer = res.new_string("Manufacturer");
res.string_product = res.new_string("Product");
res.string_serial = res.new_string("Serial");
res
}
/// Returns the old value, if present.
pub fn set_configuration_name(&mut self, name: &str) -> Option<String> {
let old = (self.string_configuration != 0)
.then(|| self.string_pool.remove(&self.string_configuration))
.flatten();
self.string_configuration = self.new_string(name);
old
}
/// Unset configuration name and returns the old value, if present.
pub fn unset_configuration_name(&mut self) -> Option<String> {
let old = (self.string_configuration != 0)
.then(|| self.string_pool.remove(&self.string_configuration))
.flatten();
self.string_configuration = 0;
old
}
/// Returns the old value, if present.
pub fn set_serial_number(&mut self, name: &str) -> Option<String> {
let old = (self.string_serial != 0)
.then(|| self.string_pool.remove(&self.string_serial))
.flatten();
self.string_serial = self.new_string(name);
old
}
/// Unset serial number and returns the old value, if present.
pub fn unset_serial_number(&mut self) -> Option<String> {
let old = (self.string_serial != 0)
.then(|| self.string_pool.remove(&self.string_serial))
.flatten();
self.string_serial = 0;
old
}
/// Returns the old value, if present.
pub fn set_product_name(&mut self, name: &str) -> Option<String> {
let old = (self.string_product != 0)
.then(|| self.string_pool.remove(&self.string_product))
.flatten();
self.string_product = self.new_string(name);
old
}
/// Unset product name and returns the old value, if present.
pub fn unset_product_name(&mut self) -> Option<String> {
let old = (self.string_product != 0)
.then(|| self.string_pool.remove(&self.string_product))
.flatten();
self.string_product = 0;
old
}
/// Returns the old value, if present.
pub fn set_manufacturer_name(&mut self, name: &str) -> Option<String> {
let old = (self.string_manufacturer != 0)
.then(|| self.string_pool.remove(&self.string_manufacturer))
.flatten();
self.string_manufacturer = self.new_string(name);
old
}
/// Unset manufacturer name and returns the old value, if present.
pub fn unset_manufacturer_name(&mut self) -> Option<String> {
let old = (self.string_manufacturer != 0)
.then(|| self.string_pool.remove(&self.string_manufacturer))
.flatten();
self.string_manufacturer = 0;
old
}
pub fn with_interface(
mut self,
interface_class: u8,
interface_subclass: u8,
interface_protocol: u8,
name: Option<&str>,
endpoints: Vec<UsbEndpoint>,
handler: Arc<Mutex<Box<dyn UsbInterfaceHandler + Send>>>,
) -> Self {
let string_interface = name.map(|name| self.new_string(name)).unwrap_or(0);
let class_specific_descriptor = handler.lock().unwrap().get_class_specific_descriptor();
self.interfaces.push(UsbInterface {
interface_class,
interface_subclass,
interface_protocol,
endpoints,
string_interface,
class_specific_descriptor,
handler,
});
self
}
pub fn with_device_handler(
mut self,
handler: Arc<Mutex<Box<dyn UsbDeviceHandler + Send>>>,
) -> Self {
self.device_handler = Some(handler);
self
}
pub(crate) fn new_string(&mut self, s: &str) -> u8 {
for i in 1.. {
if let std::collections::hash_map::Entry::Vacant(e) = self.string_pool.entry(i) {
e.insert(s.to_string());
return i;
}
}
panic!("string poll exhausted")
}
pub(crate) fn find_ep(&self, ep: u8) -> Option<(UsbEndpoint, Option<&UsbInterface>)> {
if ep == self.ep0_in.address {
Some((self.ep0_in, None))
} else if ep == self.ep0_out.address {
Some((self.ep0_out, None))
} else {
for intf in &self.interfaces {
for endpoint in &intf.endpoints {
if endpoint.address == ep {
return Some((*endpoint, Some(intf)));
}
}
}
None
}
}
pub(crate) fn to_bytes(&self) -> Vec<u8> {
let mut result = Vec::with_capacity(312);
let mut path = self.path.as_bytes().to_vec();
debug_assert!(path.len() <= 256);
path.resize(256, 0);
result.extend_from_slice(path.as_slice());
let mut bus_id = self.bus_id.as_bytes().to_vec();
debug_assert!(bus_id.len() <= 32);
bus_id.resize(32, 0);
result.extend_from_slice(bus_id.as_slice());
result.extend_from_slice(&self.bus_num.to_be_bytes());
result.extend_from_slice(&self.dev_num.to_be_bytes());
result.extend_from_slice(&self.speed.to_be_bytes());
result.extend_from_slice(&self.vendor_id.to_be_bytes());
result.extend_from_slice(&self.product_id.to_be_bytes());
result.push(self.device_bcd.major);
result.push(self.device_bcd.minor);
result.push(self.device_class);
result.push(self.device_subclass);
result.push(self.device_protocol);
result.push(self.configuration_value);
result.push(self.num_configurations);
result.push(self.interfaces.len() as u8);
result
}
pub(crate) fn to_bytes_with_interfaces(&self) -> Vec<u8> {
let mut result = self.to_bytes();
result.reserve(4 * self.interfaces.len());
for intf in &self.interfaces {
result.push(intf.interface_class);
result.push(intf.interface_subclass);
result.push(intf.interface_protocol);
result.push(0); // padding
}
result
}
pub(crate) async fn handle_urb(
&self,
ep: UsbEndpoint,
intf: Option<&UsbInterface>,
transfer_buffer_length: u32,
setup_packet: SetupPacket,
out_data: &[u8],
) -> Result<Vec<u8>> {
use DescriptorType::*;
use Direction::*;
use EndpointAttributes::*;
use StandardRequest::*;
match (FromPrimitive::from_u8(ep.attributes), ep.direction()) {
(Some(Control), In) => {
// control in
debug!("Control IN setup={setup_packet:x?}");
match (
setup_packet.request_type,
FromPrimitive::from_u8(setup_packet.request),
) {
(0b10000000, Some(GetDescriptor)) => {
// high byte: type
match FromPrimitive::from_u16(setup_packet.value >> 8) {
Some(Device) => {
debug!("Get device descriptor");
// Standard Device Descriptor
let mut desc = vec![
0x12, // bLength
Device as u8, // bDescriptorType: Device
self.usb_version.minor,
self.usb_version.major, // bcdUSB: USB 2.0
self.device_class, // bDeviceClass
self.device_subclass, // bDeviceSubClass
self.device_protocol, // bDeviceProtocol
self.ep0_in.max_packet_size as u8, // bMaxPacketSize0
self.vendor_id as u8, // idVendor
(self.vendor_id >> 8) as u8,
self.product_id as u8, // idProduct
(self.product_id >> 8) as u8,
self.device_bcd.minor, // bcdDevice
self.device_bcd.major,
self.string_manufacturer, // iManufacturer
self.string_product, // iProduct
self.string_serial, // iSerial
self.num_configurations, // bNumConfigurations
];
// requested len too short: wLength < real length
if setup_packet.length < desc.len() as u16 {
desc.resize(setup_packet.length as usize, 0);
}
Ok(desc)
}
Some(BOS) => {
debug!("Get BOS descriptor");
let mut desc = vec![
0x05, // bLength
BOS as u8, // bDescriptorType: BOS
0x05, 0x00, // wTotalLength
0x00, // bNumCapabilities
];
// requested len too short: wLength < real length
if setup_packet.length < desc.len() as u16 {
desc.resize(setup_packet.length as usize, 0);
}
Ok(desc)
}
Some(Configuration) => {
debug!("Get configuration descriptor");
// Standard Configuration Descriptor
let mut desc = vec![
0x09, // bLength
Configuration as u8, // bDescriptorType: Configuration
0x00,
0x00, // wTotalLength: to be filled below
self.interfaces.len() as u8, // bNumInterfaces
self.configuration_value, // bConfigurationValue
self.string_configuration, // iConfiguration
0x80, // bmAttributes: Bus Powered
0x32, // bMaxPower: 100mA
];
for (i, intf) in self.interfaces.iter().enumerate() {
let mut intf_desc = vec![
0x09, // bLength
Interface as u8, // bDescriptorType: Interface
i as u8, // bInterfaceNum
0x00, // bAlternateSettings
intf.endpoints.len() as u8, // bNumEndpoints
intf.interface_class, // bInterfaceClass
intf.interface_subclass, // bInterfaceSubClass
intf.interface_protocol, // bInterfaceProtocol
intf.string_interface, //iInterface
];
// class specific endpoint
let mut specific = intf.class_specific_descriptor.clone();
intf_desc.append(&mut specific);
// endpoint descriptors
for endpoint in &intf.endpoints {
let mut ep_desc = vec![
0x07, // bLength
Endpoint as u8, // bDescriptorType: Endpoint
endpoint.address, // bEndpointAddress
endpoint.attributes, // bmAttributes
endpoint.max_packet_size as u8,
(endpoint.max_packet_size >> 8) as u8, // wMaxPacketSize
endpoint.interval, // bInterval
];
intf_desc.append(&mut ep_desc);
}
desc.append(&mut intf_desc);
}
// length
let len = desc.len() as u16;
desc[2] = len as u8;
desc[3] = (len >> 8) as u8;
// requested len too short: wLength < real length
if setup_packet.length < desc.len() as u16 {
desc.resize(setup_packet.length as usize, 0);
}
Ok(desc)
}
Some(String) => {
debug!("Get string descriptor");
let index = setup_packet.value as u8;
if index == 0 {
// String Descriptor Zero, Specifying Languages Supported by the Device
// language ids
let mut desc = vec![
4, // bLength
DescriptorType::String as u8, // bDescriptorType
0x09,
0x04, // wLANGID[0], en-US
];
// requested len too short: wLength < real length
if setup_packet.length < desc.len() as u16 {
desc.resize(setup_packet.length as usize, 0);
}
Ok(desc)
} else if let Some(s) = &self.string_pool.get(&index) {
// UNICODE String Descriptor
let bytes: Vec<u16> = s.encode_utf16().collect();
let mut desc = vec![
2 + bytes.len() as u8 * 2, // bLength
DescriptorType::String as u8, // bDescriptorType
];
for byte in bytes {
desc.push(byte as u8);
desc.push((byte >> 8) as u8);
}
// requested len too short: wLength < real length
if setup_packet.length < desc.len() as u16 {
desc.resize(setup_packet.length as usize, 0);
}
Ok(desc)
} else {
Err(std::io::Error::new(
std::io::ErrorKind::InvalidInput,
format!("Invalid string index: {index}"),
))
}
}
Some(DeviceQualifier) => {
debug!("Get device qualifier descriptor");
// Device_Qualifier Descriptor
let mut desc = vec![
0x0A, // bLength
DeviceQualifier as u8, // bDescriptorType: Device Qualifier
self.usb_version.minor,
self.usb_version.major, // bcdUSB
self.device_class, // bDeviceClass
self.device_subclass, // bDeviceSUbClass
self.device_protocol, // bDeviceProtocol
self.ep0_in.max_packet_size as u8, // bMaxPacketSize0
self.num_configurations, // bNumConfigurations
0x00, // bReserved
];
// requested len too short: wLength < real length
if setup_packet.length < desc.len() as u16 {
desc.resize(setup_packet.length as usize, 0);
}
Ok(desc)
}
_ => {
warn!("unknown desc type: {setup_packet:x?}");
Ok(vec![])
}
}
}
_ if setup_packet.request_type & 0xF == 1 => {
// to interface
// see https://www.beyondlogic.org/usbnutshell/usb6.shtml
// only low 8 bits are valid
let intf = &self.interfaces[setup_packet.index as usize & 0xFF];
let mut handler = intf.handler.lock().unwrap();
handler.handle_urb(intf, ep, transfer_buffer_length, setup_packet, out_data)
}
_ if setup_packet.request_type & 0xF == 0 && self.device_handler.is_some() => {
// to device
// see https://www.beyondlogic.org/usbnutshell/usb6.shtml
let lock = self.device_handler.as_ref().unwrap();
let mut handler = lock.lock().unwrap();
handler.handle_urb(transfer_buffer_length, setup_packet, out_data)
}
_ => unimplemented!("control in"),
}
}
(Some(Control), Out) => {
// control out
debug!("Control OUT setup={setup_packet:x?}");
match (
setup_packet.request_type,
FromPrimitive::from_u8(setup_packet.request),
) {
(0b00000000, Some(SetConfiguration)) => {
let mut desc = vec![
self.configuration_value, // bConfigurationValue
];
// requested len too short: wLength < real length
if setup_packet.length < desc.len() as u16 {
desc.resize(setup_packet.length as usize, 0);
}
Ok(desc)
}
_ if setup_packet.request_type & 0xF == 1 => {
// to interface
// see https://www.beyondlogic.org/usbnutshell/usb6.shtml
// only low 8 bits are valid
let intf = &self.interfaces[setup_packet.index as usize & 0xFF];
let mut handler = intf.handler.lock().unwrap();
handler.handle_urb(intf, ep, transfer_buffer_length, setup_packet, out_data)
}
_ if setup_packet.request_type & 0xF == 0 && self.device_handler.is_some() => {
// to device
// see https://www.beyondlogic.org/usbnutshell/usb6.shtml
let lock = self.device_handler.as_ref().unwrap();
let mut handler = lock.lock().unwrap();
handler.handle_urb(transfer_buffer_length, setup_packet, out_data)
}
_ => unimplemented!("control out"),
}
}
(Some(_), _) => {
// others (interrupt / bulk / iso transfers to an endpoint)
// punktfunk modification: pace IN transfers by bInterval so a virtual interrupt-IN
// endpoint mimics a real device's NAK-until-bInterval behaviour instead of
// free-running as fast as the transport allows (vhci_hcd does not throttle the
// server side, so an unpaced sim would spin the loopback link). HS bInterval N →
// 2^(N-1) microframes × 125µs.
if let In = ep.direction() {
let n = ep.interval.clamp(1, 16) as u32;
let period_us = (1u32 << (n - 1)) * 125;
tokio::time::sleep(std::time::Duration::from_micros(period_us as u64)).await;
}
let intf = intf.unwrap();
let mut handler = intf.handler.lock().unwrap();
handler.handle_urb(intf, ep, transfer_buffer_length, setup_packet, out_data)
}
_ => unimplemented!("transfer to {:?}", ep),
}
}
}
/// A handler for URB targeting the device
pub trait UsbDeviceHandler: std::fmt::Debug {
/// Handle a URB(USB Request Block) targeting at this device
///
/// When the lower 4 bits of `bmRequestType` is zero and the URB is not handled by the library, this function is called.
/// The resulting data should not exceed `transfer_buffer_length`
fn handle_urb(
&mut self,
transfer_buffer_length: u32,
setup: SetupPacket,
req: &[u8],
) -> Result<Vec<u8>>;
/// Helper to downcast to actual struct
///
/// Please implement it as:
/// ```ignore
/// fn as_any(&mut self) -> &mut dyn Any {
/// self
/// }
/// ```
fn as_any(&mut self) -> &mut dyn Any;
}
// (In-crate test module removed in the vendored copy — see NOTICE.)
+31
View File
@@ -0,0 +1,31 @@
use super::*;
/// Represent a USB endpoint
#[derive(Clone, Copy, Debug, Default)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub struct UsbEndpoint {
/// bEndpointAddress
pub address: u8,
/// bmAttributes
pub attributes: u8,
/// wMaxPacketSize
pub max_packet_size: u16,
/// bInterval
pub interval: u8,
}
impl UsbEndpoint {
/// Get direction from MSB of address
pub fn direction(&self) -> Direction {
if self.address & 0x80 != 0 {
Direction::In
} else {
Direction::Out
}
}
/// Whether this is endpoint zero
pub fn is_ep0(&self) -> bool {
self.address & 0x7F == 0
}
}
+45
View File
@@ -0,0 +1,45 @@
use super::*;
/// Represent a USB interface
#[derive(Clone, Debug)]
#[cfg_attr(feature = "serde", derive(Serialize))]
pub struct UsbInterface {
pub interface_class: u8,
pub interface_subclass: u8,
pub interface_protocol: u8,
pub endpoints: Vec<UsbEndpoint>,
pub string_interface: u8,
pub class_specific_descriptor: Vec<u8>,
#[cfg_attr(feature = "serde", serde(skip))]
pub handler: Arc<Mutex<Box<dyn UsbInterfaceHandler + Send>>>,
}
/// A handler of a custom usb interface
pub trait UsbInterfaceHandler: std::fmt::Debug {
/// Return the class specific descriptor which is inserted between interface descriptor and endpoint descriptor
fn get_class_specific_descriptor(&self) -> Vec<u8>;
/// Handle a URB(USB Request Block) targeting at this interface
///
/// Can be one of: control transfer to ep0 or other types of transfer to its endpoint.
/// The resulting data should not exceed `transfer_buffer_length`.
fn handle_urb(
&mut self,
interface: &UsbInterface,
ep: UsbEndpoint,
transfer_buffer_length: u32,
setup: SetupPacket,
req: &[u8],
) -> Result<Vec<u8>>;
/// Helper to downcast to actual struct
///
/// Please implement it as:
/// ```ignore
/// fn as_any(&mut self) -> &mut dyn Any {
/// self
/// }
/// ```
fn as_any(&mut self) -> &mut dyn Any;
}
+250
View File
@@ -0,0 +1,250 @@
//! A USB/IP server (simulation path only).
//!
//! Vendored + trimmed from `usbip` v0.8.0 (jiegec/usbip, MIT); the USB *host* modules and the
//! `rusb`/`nusb` device constructors are removed so this carries no libusb dependency. See `NOTICE`.
use log::*;
use num_derive::FromPrimitive;
use num_traits::FromPrimitive;
use std::any::Any;
use std::collections::HashMap;
use std::io::{ErrorKind, Result};
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
use tokio::io::AsyncReadExt;
use tokio::io::AsyncWriteExt;
use tokio::net::TcpListener;
use tokio::sync::RwLock;
use usbip_protocol::UsbIpCommand;
#[cfg(feature = "serde")]
use serde::{Deserialize, Serialize};
mod consts;
mod device;
mod endpoint;
mod interface;
mod setup;
pub mod usbip_protocol;
mod util;
pub use consts::*;
pub use device::*;
pub use endpoint::*;
pub use interface::*;
pub use setup::*;
pub use util::*;
use crate::usbip_protocol::{UsbIpResponse, USBIP_RET_SUBMIT, USBIP_RET_UNLINK};
/// Main struct of a USB/IP server
#[derive(Default, Debug)]
pub struct UsbIpServer {
available_devices: RwLock<Vec<UsbDevice>>,
used_devices: RwLock<HashMap<String, UsbDevice>>,
}
impl UsbIpServer {
/// Create a [UsbIpServer] with simulated devices
pub fn new_simulated(devices: Vec<UsbDevice>) -> Self {
Self {
available_devices: RwLock::new(devices),
used_devices: RwLock::new(HashMap::new()),
}
}
pub async fn add_device(&self, device: UsbDevice) {
self.available_devices.write().await.push(device);
}
pub async fn remove_device(&self, bus_id: &str) -> Result<()> {
let mut available_devices = self.available_devices.write().await;
if let Some(device) = available_devices.iter().position(|d| d.bus_id == bus_id) {
available_devices.remove(device);
Ok(())
} else if let Some(device) = self
.used_devices
.read()
.await
.values()
.find(|d| d.bus_id == bus_id)
{
Err(std::io::Error::other(format!(
"Device {} is in use",
device.bus_id
)))
} else {
Err(std::io::Error::new(
ErrorKind::NotFound,
format!("Device {bus_id} not found"),
))
}
}
}
pub async fn handler<T: AsyncReadExt + AsyncWriteExt + Unpin>(
mut socket: &mut T,
server: Arc<UsbIpServer>,
) -> Result<()> {
let mut current_import_device_id: Option<String> = None;
loop {
let command = UsbIpCommand::read_from_socket(&mut socket).await;
if let Err(err) = command {
if let Some(dev_id) = current_import_device_id {
let mut used_devices = server.used_devices.write().await;
let mut available_devices = server.available_devices.write().await;
match used_devices.remove(&dev_id) {
Some(dev) => available_devices.push(dev),
None => unreachable!(),
}
}
if err.kind() == ErrorKind::UnexpectedEof {
info!("Remote closed the connection");
return Ok(());
} else {
return Err(err);
}
}
let used_devices = server.used_devices.read().await;
let mut current_import_device = current_import_device_id
.clone()
.and_then(|ref id| used_devices.get(id));
match command.unwrap() {
UsbIpCommand::OpReqDevlist { .. } => {
trace!("Got OP_REQ_DEVLIST");
let devices = server.available_devices.read().await;
// OP_REP_DEVLIST
UsbIpResponse::op_rep_devlist(&devices)
.write_to_socket(socket)
.await?;
trace!("Sent OP_REP_DEVLIST");
}
UsbIpCommand::OpReqImport { busid, .. } => {
trace!("Got OP_REQ_IMPORT");
current_import_device_id = None;
current_import_device = None;
std::mem::drop(used_devices);
let mut used_devices = server.used_devices.write().await;
let mut available_devices = server.available_devices.write().await;
let busid_compare =
&busid[..busid.iter().position(|&x| x == 0).unwrap_or(busid.len())];
for (i, dev) in available_devices.iter().enumerate() {
if busid_compare == dev.bus_id.as_bytes() {
let dev = available_devices.remove(i);
let dev_id = dev.bus_id.clone();
used_devices.insert(dev.bus_id.clone(), dev);
current_import_device_id = dev_id.clone().into();
current_import_device = Some(used_devices.get(&dev_id).unwrap());
break;
}
}
let res = if let Some(dev) = current_import_device {
UsbIpResponse::op_rep_import_success(dev)
} else {
UsbIpResponse::op_rep_import_fail()
};
res.write_to_socket(socket).await?;
trace!("Sent OP_REP_IMPORT");
}
UsbIpCommand::UsbIpCmdSubmit {
mut header,
transfer_buffer_length,
setup,
data,
..
} => {
trace!("Got USBIP_CMD_SUBMIT");
let device = current_import_device.unwrap();
let out = header.direction == 0;
let real_ep = if out { header.ep } else { header.ep | 0x80 };
header.command = USBIP_RET_SUBMIT.into();
let res = match device.find_ep(real_ep as u8) {
None => {
warn!("Endpoint {real_ep:02x?} not found");
UsbIpResponse::usbip_ret_submit_fail(&header)
}
Some((ep, intf)) => {
trace!("->Endpoint {ep:02x?}");
trace!("->Setup {setup:02x?}");
trace!("->Request {data:02x?}");
let resp = device
.handle_urb(
ep,
intf,
transfer_buffer_length,
SetupPacket::parse(&setup),
&data,
)
.await;
match resp {
Ok(resp) => {
if out {
trace!("<-Wrote {}", data.len());
} else {
trace!("<-Resp {resp:02x?}");
}
UsbIpResponse::usbip_ret_submit_success(&header, 0, 0, resp, vec![])
}
Err(err) => {
warn!("Error handling URB: {err}");
UsbIpResponse::usbip_ret_submit_fail(&header)
}
}
}
};
res.write_to_socket(socket).await?;
trace!("Sent USBIP_RET_SUBMIT");
}
UsbIpCommand::UsbIpCmdUnlink {
mut header,
unlink_seqnum,
} => {
trace!("Got USBIP_CMD_UNLINK for {unlink_seqnum:10x?}");
header.command = USBIP_RET_UNLINK.into();
let res = UsbIpResponse::usbip_ret_unlink_success(&header);
res.write_to_socket(socket).await?;
trace!("Sent USBIP_RET_UNLINK");
}
}
}
}
/// Spawn a USB/IP server at `addr` using [TcpListener]
pub async fn server(addr: SocketAddr, server: Arc<UsbIpServer>) {
let listener = TcpListener::bind(addr).await.expect("bind to addr");
let server = async move {
loop {
match listener.accept().await {
Ok((mut socket, _addr)) => {
info!("Got connection from {:?}", socket.peer_addr());
let new_server = server.clone();
tokio::spawn(async move {
let res = handler(&mut socket, new_server).await;
info!("Handler ended with {res:?}");
});
}
Err(err) => {
warn!("Got error {err:?}");
}
}
}
};
server.await
}
// (Host-mode constructors and in-crate tests removed in the vendored copy — see NOTICE.)
+31
View File
@@ -0,0 +1,31 @@
#[cfg(feature = "serde")]
use serde::{Deserialize, Serialize};
/// Parse the SETUP packet of control transfers
#[derive(Clone, Copy, Debug, Default)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub struct SetupPacket {
/// bmRequestType
pub request_type: u8,
/// bRequest
pub request: u8,
/// wValue
pub value: u16,
/// wIndex
pub index: u16,
/// wLength
pub length: u16,
}
impl SetupPacket {
/// Parse a [SetupPacket] from raw setup packet
pub fn parse(setup: &[u8; 8]) -> SetupPacket {
SetupPacket {
request_type: setup[0],
request: setup[1],
value: ((setup[3] as u16) << 8) | (setup[2] as u16),
index: ((setup[5] as u16) << 8) | (setup[4] as u16),
length: ((setup[7] as u16) << 8) | (setup[6] as u16),
}
}
}
@@ -0,0 +1,498 @@
//! USB/IP protocol structs
//!
//! This module contains declarations of all structs used in the USB/IP protocol,
//! as well as functions to serialize and deserialize them to/from byte arrays,
//! and functions to send and receive them over a socket.
//!
//! They are based on the [Linux kernel documentation](https://docs.kernel.org/usb/usbip_protocol.html).
use log::trace;
use std::io::Result;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
#[cfg(feature = "serde")]
use serde::{Deserialize, Serialize};
use crate::UsbDevice;
/// USB/IP protocol version
///
/// This is currently the only supported version of USB/IP
/// for this library.
pub const USBIP_VERSION: u16 = 0x0111;
/// Command code: Retrieve the list of exported USB devices
pub const OP_REQ_DEVLIST: u16 = 0x8005;
/// Command code: import a remote USB device
pub const OP_REQ_IMPORT: u16 = 0x8003;
/// Reply code: The list of exported USB devices
pub const OP_REP_DEVLIST: u16 = 0x0005;
/// Reply code: Reply to import
pub const OP_REP_IMPORT: u16 = 0x0003;
/// Command code: Submit an URB
pub const USBIP_CMD_SUBMIT: u16 = 0x0001;
/// Command code: Unlink an URB
pub const USBIP_CMD_UNLINK: u16 = 0x0002;
/// Reply code: Reply for submitting an URB
pub const USBIP_RET_SUBMIT: u16 = 0x0003;
/// Reply code: Reply for URB unlink
pub const USBIP_RET_UNLINK: u16 = 0x0004;
/// USB/IP direction
///
/// NOTE: Must not be confused with rusb::Direction,
/// which has the opposite enum values. This is only for
/// internal use in the USB/IP protocol.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Direction {
Out = 0,
In = 1,
}
/// Common header for all context sensitive packets
///
/// All commands/responses which rely on a device being attached
/// to a client use this header.
#[derive(Debug, Clone, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub struct UsbIpHeaderBasic {
pub command: u32,
pub seqnum: u32,
pub devid: u32,
pub direction: u32,
pub ep: u32,
}
impl UsbIpHeaderBasic {
/// Converts a byte array into a [UsbIpHeaderBasic].
pub fn from_bytes(bytes: &[u8; 20]) -> Self {
let result = UsbIpHeaderBasic {
command: u32::from_be_bytes(bytes[0..4].try_into().unwrap()),
seqnum: u32::from_be_bytes(bytes[4..8].try_into().unwrap()),
devid: u32::from_be_bytes(bytes[8..12].try_into().unwrap()),
direction: u32::from_be_bytes(bytes[12..16].try_into().unwrap()),
ep: u32::from_be_bytes(bytes[16..20].try_into().unwrap()),
};
// The direction should be 0 or 1
debug_assert!(result.direction & 1 == result.direction);
result
}
/// Converts the [UsbIpHeaderBasic] into a byte array.
pub fn to_bytes(&self) -> [u8; 20] {
let mut result = [0u8; 20];
result[0..4].copy_from_slice(&self.command.to_be_bytes());
result[4..8].copy_from_slice(&self.seqnum.to_be_bytes());
result[8..12].copy_from_slice(&self.devid.to_be_bytes());
result[12..16].copy_from_slice(&self.direction.to_be_bytes());
result[16..20].copy_from_slice(&self.ep.to_be_bytes());
result
}
pub(crate) async fn read_from_socket_with_command<T: AsyncReadExt + Unpin>(
socket: &mut T,
command: u16,
) -> Result<Self> {
let seqnum = socket.read_u32().await?;
let devid = socket.read_u32().await?;
let direction = socket.read_u32().await?;
// The direction should be 0 or 1
debug_assert!(direction & 1 == direction);
let ep = socket.read_u32().await?;
Ok(UsbIpHeaderBasic {
command: command.into(),
seqnum,
devid,
direction,
ep,
})
}
}
/// Client side commands from the Virtual Host Controller
#[derive(Debug, Clone, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
pub enum UsbIpCommand {
OpReqDevlist {
status: u32,
},
OpReqImport {
status: u32,
busid: [u8; 32],
},
UsbIpCmdSubmit {
header: UsbIpHeaderBasic,
transfer_flags: u32,
transfer_buffer_length: u32,
start_frame: u32,
number_of_packets: u32,
interval: u32,
setup: [u8; 8],
data: Vec<u8>,
iso_packet_descriptor: Vec<u8>,
},
UsbIpCmdUnlink {
header: UsbIpHeaderBasic,
unlink_seqnum: u32,
},
}
impl UsbIpCommand {
/// Constructs a [UsbIpCommand] from a socket
///
/// This will consume a variable amount of bytes from the socket.
/// It might fail if the bytes does not follow the USB/IP protocol properly.
pub async fn read_from_socket<T: AsyncReadExt + Unpin>(socket: &mut T) -> Result<UsbIpCommand> {
let version: u16 = socket.read_u16().await?;
if version != 0 && version != USBIP_VERSION {
return Err(std::io::Error::other(format!(
"Unknown version: {version:#04X}"
)));
}
let command: u16 = socket.read_u16().await?;
trace!(
"Received command: {:#04X} ({}), parsing...",
command,
match command {
OP_REQ_DEVLIST => "OP_REQ_DEVLIST",
OP_REQ_IMPORT => "OP_REQ_IMPORT",
USBIP_CMD_SUBMIT => "USBIP_CMD_SUBMIT",
USBIP_CMD_UNLINK => "USBIP_CMD_UNLINK",
_ => "Unknown",
}
);
match command {
OP_REQ_DEVLIST => {
let status = socket.read_u32().await?;
debug_assert!(status == 0);
Ok(UsbIpCommand::OpReqDevlist { status })
}
OP_REQ_IMPORT => {
let status = socket.read_u32().await?;
debug_assert!(status == 0);
let mut busid = [0; 32];
socket.read_exact(&mut busid).await?;
Ok(UsbIpCommand::OpReqImport { status, busid })
}
USBIP_CMD_SUBMIT => {
let header =
UsbIpHeaderBasic::read_from_socket_with_command(socket, USBIP_CMD_SUBMIT)
.await?;
let transfer_flags = socket.read_u32().await?;
let transfer_buffer_length = socket.read_u32().await?;
let start_frame = socket.read_u32().await?;
let number_of_packets = socket.read_u32().await?;
let interval = socket.read_u32().await?;
let mut setup = [0; 8];
socket.read_exact(&mut setup).await?;
let data = if header.direction == Direction::In as u32 {
vec![]
} else {
let mut data = vec![0; transfer_buffer_length as usize];
socket.read_exact(&mut data).await?;
data
};
// The kernel docs specifies that this should be set to 0xFFFFFFFF for all
// non-ISO packets, however the actual implementation resorts to 0x00000000
// https://stackoverflow.com/questions/76899798/usb-ip-what-is-the-size-of-the-iso-packet-descriptor
let iso_packet_descriptor =
if number_of_packets != 0 && number_of_packets != 0xFFFFFFFF {
let mut result = vec![0; 16 * number_of_packets as usize];
socket.read_exact(&mut result).await?;
result
} else {
vec![]
};
Ok(UsbIpCommand::UsbIpCmdSubmit {
header,
transfer_flags,
transfer_buffer_length,
start_frame,
number_of_packets,
interval,
setup,
data,
iso_packet_descriptor,
})
}
USBIP_CMD_UNLINK => {
let header =
UsbIpHeaderBasic::read_from_socket_with_command(socket, USBIP_CMD_UNLINK)
.await?;
let unlink_seqnum = socket.read_u32().await?;
let mut _padding = [0; 24];
socket.read_exact(&mut _padding).await?;
Ok(UsbIpCommand::UsbIpCmdUnlink {
header,
unlink_seqnum,
})
}
_ => Err(std::io::Error::other(format!(
"Unknown command: {command:#04X}"
))),
}
}
/// Converts the [UsbIpCommand] into a byte vector
pub fn to_bytes(&self) -> Vec<u8> {
match *self {
UsbIpCommand::OpReqDevlist { status } => {
let mut result = Vec::with_capacity(8);
result.extend_from_slice(&USBIP_VERSION.to_be_bytes());
result.extend_from_slice(&OP_REQ_DEVLIST.to_be_bytes());
result.extend_from_slice(&status.to_be_bytes());
result
}
UsbIpCommand::OpReqImport { status, busid } => {
let mut result = Vec::with_capacity(40);
result.extend_from_slice(&USBIP_VERSION.to_be_bytes());
result.extend_from_slice(&OP_REQ_IMPORT.to_be_bytes());
result.extend_from_slice(&status.to_be_bytes());
result.extend_from_slice(&busid);
result
}
UsbIpCommand::UsbIpCmdSubmit {
ref header,
transfer_flags,
transfer_buffer_length,
start_frame,
number_of_packets,
interval,
setup,
ref data,
ref iso_packet_descriptor,
} => {
debug_assert!(
header.direction != Direction::Out as u32
|| transfer_buffer_length == data.len() as u32
);
let mut result = Vec::with_capacity(48 + data.len() + iso_packet_descriptor.len());
result.extend_from_slice(&header.to_bytes());
result.extend_from_slice(&transfer_flags.to_be_bytes());
result.extend_from_slice(&transfer_buffer_length.to_be_bytes());
result.extend_from_slice(&start_frame.to_be_bytes());
result.extend_from_slice(&number_of_packets.to_be_bytes());
result.extend_from_slice(&interval.to_be_bytes());
result.extend_from_slice(&setup);
result.extend_from_slice(data);
result.extend_from_slice(iso_packet_descriptor);
result
}
UsbIpCommand::UsbIpCmdUnlink {
ref header,
unlink_seqnum,
} => {
let mut result = Vec::with_capacity(48);
result.extend_from_slice(&header.to_bytes());
result.extend_from_slice(&unlink_seqnum.to_be_bytes());
result.extend_from_slice(&[0; 24]);
result
}
}
}
}
/// Server side responses from the USB Host
#[derive(Clone)]
#[cfg_attr(feature = "serde", derive(Serialize))]
pub enum UsbIpResponse {
OpRepDevlist {
status: u32,
device_count: u32,
devices: Vec<UsbDevice>,
},
OpRepImport {
status: u32,
device: Option<UsbDevice>,
},
UsbIpRetSubmit {
header: UsbIpHeaderBasic,
status: u32,
actual_length: u32,
start_frame: u32,
number_of_packets: u32,
error_count: u32,
transfer_buffer: Vec<u8>,
iso_packet_descriptor: Vec<u8>,
},
UsbIpRetUnlink {
header: UsbIpHeaderBasic,
status: u32,
},
}
impl UsbIpResponse {
/// Converts the [UsbIpResponse] into a byte vector
pub fn to_bytes(&self) -> Vec<u8> {
match *self {
Self::OpRepDevlist {
status,
device_count,
ref devices,
} => {
let mut result = Vec::with_capacity(
12 + devices.len() * 312
+ devices
.iter()
.map(|d| d.interfaces.len() * 4)
.sum::<usize>(),
);
result.extend_from_slice(&USBIP_VERSION.to_be_bytes());
result.extend_from_slice(&OP_REP_DEVLIST.to_be_bytes());
result.extend_from_slice(&status.to_be_bytes());
result.extend_from_slice(&device_count.to_be_bytes());
for dev in devices {
result.extend_from_slice(&dev.to_bytes_with_interfaces());
}
result
}
Self::OpRepImport { status, ref device } => {
let mut result = Vec::with_capacity(320);
result.extend_from_slice(&USBIP_VERSION.to_be_bytes());
result.extend_from_slice(&OP_REP_IMPORT.to_be_bytes());
result.extend_from_slice(&status.to_be_bytes());
if let Some(device) = device {
result.extend_from_slice(&device.to_bytes());
}
result
}
Self::UsbIpRetSubmit {
ref header,
status,
actual_length,
start_frame,
number_of_packets,
error_count,
ref transfer_buffer,
ref iso_packet_descriptor,
} => {
let mut result =
Vec::with_capacity(48 + transfer_buffer.len() + iso_packet_descriptor.len());
debug_assert!(header.command == USBIP_RET_SUBMIT.into());
debug_assert!(if header.direction == Direction::In as u32 {
actual_length == transfer_buffer.len() as u32
} else {
actual_length == 0
});
result.extend_from_slice(&header.to_bytes());
result.extend_from_slice(&status.to_be_bytes());
result.extend_from_slice(&actual_length.to_be_bytes());
result.extend_from_slice(&start_frame.to_be_bytes());
result.extend_from_slice(&number_of_packets.to_be_bytes());
result.extend_from_slice(&error_count.to_be_bytes());
result.extend_from_slice(&[0; 8]);
result.extend_from_slice(transfer_buffer);
result.extend_from_slice(iso_packet_descriptor);
result
}
Self::UsbIpRetUnlink { ref header, status } => {
let mut result = Vec::with_capacity(48);
debug_assert!(header.command == USBIP_RET_UNLINK.into());
result.extend_from_slice(&header.to_bytes());
result.extend_from_slice(&status.to_be_bytes());
result.extend_from_slice(&[0; 24]);
result
}
}
}
pub async fn write_to_socket<T: AsyncWriteExt + Unpin>(&self, socket: &mut T) -> Result<()> {
socket.write_all(&self.to_bytes()).await
}
/// Constructs a OP_REP_DEVLIST response
pub fn op_rep_devlist(devices: &[UsbDevice]) -> Self {
Self::OpRepDevlist {
status: 0,
device_count: devices.len() as u32,
devices: devices.to_vec(),
}
}
/// Constructs a successful OP_REP_IMPORT response
pub fn op_rep_import_success(device: &UsbDevice) -> Self {
Self::OpRepImport {
status: 0,
device: Some(device.clone()),
}
}
/// Constructs a failed OP_REP_IMPORT response
pub fn op_rep_import_fail() -> Self {
Self::OpRepImport {
status: 1,
device: None,
}
}
/// Constructs a successful OP_REP_IMPORT response
pub fn usbip_ret_submit_success(
header: &UsbIpHeaderBasic,
start_frame: u32,
number_of_packets: u32,
transfer_buffer: Vec<u8>,
iso_packet_descriptor: Vec<u8>,
) -> Self {
Self::UsbIpRetSubmit {
header: header.clone(),
status: 0,
actual_length: transfer_buffer.len() as u32,
start_frame,
number_of_packets,
error_count: 0,
transfer_buffer,
iso_packet_descriptor,
}
}
/// Constructs a failed OP_REP_IMPORT response
pub fn usbip_ret_submit_fail(header: &UsbIpHeaderBasic) -> Self {
Self::UsbIpRetSubmit {
header: header.clone(),
status: 1,
actual_length: 0,
start_frame: 0,
number_of_packets: 0,
error_count: 0,
transfer_buffer: vec![],
iso_packet_descriptor: vec![],
}
}
/// Constructs a successful OP_REP_IMPORT response
pub fn usbip_ret_unlink_success(header: &UsbIpHeaderBasic) -> Self {
Self::UsbIpRetUnlink {
header: header.clone(),
status: 0,
}
}
/// Constructs a failed OP_REP_IMPORT response.
pub fn usbip_ret_unlink_fail(header: &UsbIpHeaderBasic) -> Self {
Self::UsbIpRetUnlink {
header: header.clone(),
status: 1,
}
}
}
// (In-crate test module removed in the vendored copy — see NOTICE.)
+10
View File
@@ -0,0 +1,10 @@
/// Check validity of a USB descriptor
pub fn verify_descriptor(desc: &[u8]) {
let mut offset = 0;
while offset < desc.len() {
offset += desc[offset] as usize; // length
}
assert_eq!(offset, desc.len());
}
// (In-crate test module removed in the vendored copy — see NOTICE.)
+6
View File
@@ -34,6 +34,8 @@ holds the full originals.
| [`apple-stage2-presenter.md`](apple-stage2-presenter.md) | Apple stage-2 (VTDecompressionSession + CAMetalLayer) presenter | **Shipped (opt-in)** — make-default + iOS open |
| [`game-library-stores.md`](game-library-stores.md) | Multi-store game library | **Phases 14 shipped** — 6 providers + 8 Qs open |
| [`dualsense-haptics.md`](dualsense-haptics.md) | DualSense advanced-haptics feasibility | **HID shipped**; audio haptics deferred (3 walls) |
| [`steam-controller-deck-support.md`](steam-controller-deck-support.md) | Rich Steam Controller / Steam Deck **input fidelity** (paddles · trackpads · gyro → virtual `hid-steam`) | **Design + M0 GREEN** (Linux bind proven); M1+ open |
| [`controller-only-mode.md`](controller-only-mode.md) | Controller-only **session shape** — Deck/desktop as a remote gamepad, no video/audio (complements ↑) | **Design** — not yet implemented |
| [`archive/windows-secure-desktop.md`](archive/windows-secure-desktop.md) | Two-process WGC secure-desktop design | **Archived** — shipped but now a fallback (IDD-push primary) |
Plus `research/gamestream-protocol-research.json` — raw Moonlight/GameStream wire reference (data, not prose).
@@ -74,6 +76,10 @@ owning doc.)
**Game library**
- 6 remaining providers (Desktop/Flatpak, itch.io, Ubisoft Connect, Amazon Games, Battle.net, EA app); the `/library/art/<entryId>/<slot>` mgmt endpoint; refactor `library.rs` into a `library/` dir; 8 open design questions; optional SteamGridDB v2 enrichment. → `game-library-stores`
**Controllers / input**
- Rich Steam Controller / Steam Deck capture + virtual `hid-steam` inject (M1+ — Linux UHID, then clients, then deferred Windows UMDF). → `steam-controller-deck-support`
- Controller-only session shape (Deck/desktop as a remote gamepad, no video/audio) — `session_flags`/`SESSION_INPUT_ONLY` protocol bit + host skip-data-plane branch + client controller-only path. → `controller-only-mode`
**Multi-user / sessions**
- gamescope per-session input/audio isolation (independent desktops) — the 4 plumbing items, deferred. → `gamescope-multiuser`, `implementation-plan`
+79 -12
View File
@@ -3,12 +3,61 @@ title: "Apple Stage-2 Presenter (handoff)"
description: "Design rationale + open items for the explicit VTDecompressionSession → CAMetalLayer presenter. Implementation shipped; this page is trimmed to the why + what's left."
---
> **Status:** SHIPPED behind the opt-in `punktfunk.presenter` flag (`AVSampleBufferDisplayLayer`
> stage-1 remains the default known-good path). Live-validated ~11 ms p50 capture→present (commit
> `7b10714`). Code: `clients/apple/Sources/PunktfunkKit/{Stage2Pipeline,MetalVideoPresenter,VideoDecoder,LatencyMeter}.swift`;
> Settings has a presenter picker (`DefaultsKey.presenter`, `SettingsView.swift`). This doc is trimmed
> to design rationale + open items — the shipped `.swift` code is the source of truth for the
> decode/present/measurement walkthrough.
> **Status:** SHIPPED as the **default** presenter (stage-1 `AVSampleBufferDisplayLayer` is the
> Metal-unavailable / DEBUG fallback). HDR corrected and **4:4:4** added on top of the proven
> main-thread present path (the hosting view's `CADisplayLink` drives `render` per vsync). Code:
> `clients/apple/Sources/PunktfunkKit/{Stage2Pipeline,MetalVideoPresenter,VideoDecoder,Stage444Probe,LatencyMeter}.swift`.
> This doc is trimmed to design rationale + open items — the shipped `.swift` code is the source of
> truth for the decode/present/measurement walkthrough.
>
> **HDR (the "too bright" fix).** The presenter renders to a *separate* CAMetalLayer drawable, so the
> mastering metadata that was attached to the source `CVPixelBuffer` was never composited — and with no
> reference-white anchor the system rendered the PQ signal far too bright. The fix is to keep the
> PQ-passthrough shader (BT.2020 limited→full → PQ RGB as-is) and put the anchor **on the layer**:
> `colorspace = itur_2100_PQ`, `wantsExtendedDynamicRangeContent = true` (on **all** platforms — the old
> `#if os(macOS)` guard left iOS/tvOS EDR half-engaged), and
> `edrMetadata = CAEDRMetadata.hdr10(displayInfo:contentInfo:opticalOutputScale: 203)`. 203 nits =
> BT.2408 HDR reference white anchors diffuse white at EDR 1.0; a larger value renders dimmer. The
> mastering/CLL blobs (host `0xCE` datagram) now refine `edrMetadata` (drained by the pump,
> `setHdrMeta` hops the layer write to main) rather than being attached to a never-composited source
> buffer. **Needs on-glass validation on a real EDR panel.**
>
> **Mid-session SDR↔HDR.** The control-plane colour (`connection.isHDR`, from the Welcome) is fixed per
> session, but the host can re-init its encoder mid-session (a game entering HDR), so the HEVC VUI — and
> the decoder's `frame.isHDR` — flips. The presenter follows the **decoded frame**, not the latched
> session flag: `render` calls the idempotent `configure(hdr:)` every frame, so on a flip it
> reconfigures the layer (per-mode pixel format `bgra8Unorm` SDR / `rgba16Float` HDR, colorspace, EDR)
> and selects the matching shader — all synchronously on the main thread (the present path is
> main-thread, so no cross-thread hop is needed). The last `0xCE` grade is cached so an SDR→HDR
> reconfigure re-applies the real mastering metadata instead of the bare anchor. The pump drains `0xCE`
> **unconditionally** (not gated on the Welcome flag) so a session that starts SDR still gets mastering
> metadata when it goes HDR. A ≤2-frame transition flash on the rare flip is accepted.
>
> **Pacing.** The hosting view owns a **main-runloop `CADisplayLink`** (a weak `DisplayLinkProxy`
> breaks the retain cycle) that calls `renderTick` once per vsync. `renderTick` pops the **newest**
> ready frame from the 1-slot ring (older undisplayed frames dropped — lowest latency, no smoothing
> buffer) and, if there is one, draws it via **manual `layer.nextDrawable()`** and presents at the next
> vsync; on an idle vsync (no new frame) it does nothing and the compositor holds the last presented
> drawable (no idle re-render — matters at 5K). `drawableSize` is set **before** `nextDrawable` (it
> doesn't track bounds, defaults to 0), so allocation always uses the decoded size. `maximumDrawableCount
> = 3`. macOS `displaySyncEnabled = **false**`: the display link is the single pacing source, so leaving
> the layer's own vsync wait on would *also* block `present`/`nextDrawable` on the main thread and
> serialize it to the display — the cause of the fullscreen judder; disabling it lets present return
> promptly. Present is stamped at the display link's `targetTimestamp` projected to `CLOCK_REALTIME`
> (the actual on-glass instant, <1 vsync after the draw — accurate for the HUD).
>
> *(History: an off-main `CAMetalDisplayLink` variant and an off-main blocking-render present thread
> were both tried and **reverted** — both measured slower on macOS *and* iPad than this main-thread
> display-link path, whose real judder fix was simply `displaySyncEnabled = false`, not moving present
> off-thread. Measured ~11 ms p50 on the main-thread path.)*
>
> **4:4:4.** Chroma, bit-depth, and colorimetry are orthogonal: the decode pixel format is a 2×2 of
> `(chroma, HDR)``420v/x420/444v/x444` (all biplanar, so the existing shaders sample a full-size
> chroma plane unchanged); the shader is keyed only on HDR. The client advertises `VIDEO_CAP_444` only
> when `Stage444Probe` confirms **hardware** 4:4:4 decode (a hardware-required `VTDecompressionSession`
> over an embedded 256×256 4:4:4 keyframe — software 4:4:4 is too slow for real-time; validated on M3:
> `444v`/`x444` produced). A bounded pump backstop ends a 4:4:4 session that persistently fails to
> decode (gated to 4:4:4 sessions, so 4:2:0 loss-recovery is untouched).
## Why stage 2 (design rationale)
@@ -47,10 +96,28 @@ Async `VTDecompressionSession` callback → **1-slot newest-ready ring** → dis
## Open items
- **Make stage 2 the default** — after resolution / HDR edge-case checks (HDR = BT.2020/PQ, 10-bit
`…10BiPlanar` + EDR `CAMetalLayer.wantsExtendedDynamicRangeContent`; ties in with the HDR roadmap).
- **On-glass HDR validation** — eyeball `edrMetadata` + `opticalOutputScale: 203` on a real EDR panel
(XDR display) against stage-1 side-by-side: diffuse white should sit at SDR-white level with only
highlights climbing. The reference white is a single named constant (`hdrReferenceWhiteNits`) for easy
tuning. (Needs a Windows HDR host; the Linux host is 8-bit SDR only.)
- **On-glass 4:4:4 validation** — confirm a `PUNKTFUNK_444` host (RTX box) streams a 4:4:4 session the
client decodes in hardware (HUD shows the resolved chroma); verify the resolution-ceiling backstop by
forcing a too-large 4:4:4 mode.
- **Glass-to-glass numbers via `tools/latency-probe`** — close the still-unmeasured host render→capture
term.
- **Smoothing / pacing policy** — present newest-ready for lowest latency today; a pacing policy can come
later if frames look uneven.
- **iOS / iPadOS / tvOS stage-2 variants.**
term and confirm the main-thread display-link present p50 holds at ~11 ms (and isn't regressed by the
per-frame `configure` / HDR-anchor work).
- **Smoothing / pacing policy** — present newest-ready for lowest latency today; an optional even-pacing
policy (`present(_:afterMinimumDuration:)`) can come later if frames look uneven.
- **4:4:4 runtime downgrade-reconnect** — today a persistently-undecodable 4:4:4 session ends cleanly
(the live 4:4:4 decode requires hardware, so a resolution-ceiling miss fails the session create
*synchronously* and the pump backstop ends it — no black-screen loop); auto-reconnecting at 4:2:0
(dropping `VIDEO_CAP_444`) is a future refinement.
- **HLG**`isHDR`/`isHDRFormat` fold HLG (transfer 18) in with PQ, but the presenter is PQ-only
(`itur_2100_PQ` + `hdr10` EDR), so an HLG stream would be mis-toned. Latent — no host emits HLG
(the stack is BT.2020 **PQ** only). A real HLG path (`itur_2100_HLG`, no PQ reference-white anchor)
is future work; until then HLG should be treated as out of scope.
- **Full-range** — the shaders hardcode limited→full expansion and the decoder requests the
`*VideoRange` formats regardless of `connection.colorFullRange`; VideoToolbox range-converts a
full-range source to video range on decode, so it stays self-consistent (mild level compression on
genuinely full-range content, which no host emits). Pre-existing; wire `colorFullRange` into the
range constants eventually.

Some files were not shown because too many files have changed in this diff Show More