47 Commits

Author SHA1 Message Date
enricobuehler 1c04e77293 feat(apple): Improve presenter
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / web (push) Has been cancelled
ci / rust (push) Has been cancelled
android-screenshots / screenshots (push) Successful in 2m16s
deb / build-publish (push) Successful in 3m26s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
windows-host / package (push) Successful in 6m48s
release / apple (push) Successful in 7m45s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m22s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m37s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
android / android (push) Successful in 9m35s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m32s
linux-client-screenshots / screenshots (push) Successful in 2m31s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m53s
web-screenshots / screenshots (push) Successful in 2m32s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m37s
flatpak / build-publish (push) Failing after 3m47s
docker / deploy-docs (push) Failing after 1m9s
feat(apple): add cursor capture on iPad
2026-06-30 01:31:48 +02:00
enricobuehler e2d4c40167 feat(android): HDR toggle, video-feed stats, landscape lock
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 5m14s
ci / web (push) Successful in 49s
apple / screenshots (push) Successful in 5m19s
ci / docs-site (push) Successful in 1m0s
ci / bench (push) Successful in 4m36s
ci / rust (push) Successful in 13m31s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 40s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
deb / build-publish (push) Successful in 10m9s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m50s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m19s
docker / deploy-docs (push) Failing after 1m4s
- HDR toggle in Settings → Display. Persisted (hdr_enabled, default on); the
  host is advertised HDR only when the toggle is on AND the panel can present
  HDR10 (displaySupportsHdr), so SDR panels never get PQ they'd mis-tone-map.
  The toggle is disabled/greyed on non-HDR displays (ToggleRow gained `enabled`).
- Extend the stats HUD with a video-feed line, e.g.
  "HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0". nativeVideoStats now returns 14
  doubles (appends bitDepth, CICP primaries/transfer, chroma_format_idc from the
  negotiated Welcome); older/shorter layouts just omit the line.
- Lock the stream to landscape while streaming (SENSOR_LANDSCAPE), restoring the
  prior orientation on exit. The activity declares configChanges=orientation, so
  it re-lays-out in place with no stream restart; harmless no-op on TV.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 22:16:04 +02:00
enricobuehler 580b1ea7a7 feat(host/steam): shippable usbip/vhci_hcd virtual Deck + client leave-shortcuts
apple / screenshots (push) Has been cancelled
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
audit / cargo-audit (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
Steam Deck pass-through (design/steam-deck-passthrough-plan.md), code-complete +
all CI checks green on Linux + adversarially reviewed; on-glass validation pending:

- usbip/`vhci_hcd` virtual Deck transport (inject/linux/steam_usbip.rs) for
  non-SteamOS hosts (Bazzite/generic) — presents a real interface-2 USB Deck so
  Steam Input promotes it. In-process vhci attach (loopback OP_REQ_IMPORT handshake
  → sysfs attach) with a bounded `usbip`-CLI fallback; detach on drop.
- Backed by a vendored, libusb-free trim of the `usbip` crate
  (crates/punktfunk-host/vendor/usbip-sim, MIT + NOTICE; host/cdc/hid + rusb/nusb
  removed; interrupt-IN paced by bInterval).
- Selection ladder raw_gadget (SteamOS fast-path) → usbip (universal) → UHID,
  with PUNKTFUNK_STEAM_USBIP / PUNKTFUNK_USBIP_ATTACH knobs.
- Shared Deck descriptors + the 0x83/0xAE feature contract + a Steam-accepted
  serial consolidated into steam_proto.rs; the raw_gadget backend reuses them.
- Linux client leave-shortcuts: Ctrl+Alt+Shift+D + holding the escape chord
  (L1+R1+Start+Select) >=1.5s end the session (short press still exits
  fullscreen); the chord state resets across sessions.

Also bundles in-progress work already staged in the tree:
- host(kwin): xdg-output logical-geometry mapping so the KWin fake_input backend
  places absolute coordinates correctly under display scaling.
- docs: design/README index entries + design/controller-only-mode.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 831b37b4b7 build: exclude the usbip-poc from the workspace (standalone PoC, pulls libusb)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 4f0b4aa68f docs(steam): production plan for Deck client pass-through + shippable usbip host
Write design/steam-deck-passthrough-plan.md — the build plan to ship exact Steam
Deck pass-through from the Linux client (incl. the Steam + QAM buttons) plus a
virtual Deck on any Linux host. Key validated facts captured so the next session
doesn't re-investigate:

- Client capture is ALREADY correct: SDL3 maps Steam->Guide, QAM->Misc1; the
  client forwards BTN_GUIDE/BTN_MISC1; the host maps them to btn::STEAM/btn::QAM.
  Only precondition: Steam Input disabled on the client (the Decky UX).
- Shippable host transport = usbip + vhci_hcd (in-tree + signed everywhere, no
  module build, no MOK) — PROVEN on Bazzite: Steam promotes the usbip interface-2
  Deck (XInput slot + X-Box pad), identical to raw_gadget on SteamOS.
- Build steps: refactor steam_gadget.rs into shared Deck-logic + a transport
  trait; add the usbip transport (vendor-trim the usbip crate to drop rusb/libusb,
  in-process vhci attach); transport-select raw_gadget->usbip->UHID/DualSense;
  client leave-shortcut (controller chord + Ctrl+Alt+Shift+D); serial polish.

Also checks in the working usbip Deck PoC (packaging/linux/steam-deck-gadget/
usbip-poc/) for the next session to build on. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 963c406f33 feat(host/steam): default the gadget Deck on for SteamOS (glass-confirmed)
The virtual Steam Deck is validated glass-to-glass on a Deck: it appears as a
distinct second Steam controller, a held A drives Steam's overlay ("Resume
Game"), and a button press registers in a real game (confirmed in-game).

gadget_preferred() now defaults ON for SteamOS hosts (/etc/os-release ID=steamos
or ID_LIKE), OFF elsewhere where the universal UHID path stays the default;
PUNKTFUNK_STEAM_GADGET=1/0 forces it. A Deck-as-host with a physical Deck never
reaches this path — resolve_gamepad's conflict gate degrades SteamDeck → DualSense
first, so the two-Deck case never happens in production (it was only a test-rig
confound on the dev Deck).

The feature is complete: a virtual Steam Deck that Steam Input recognizes +
promotes, churn-free, with input flowing to games. Workspace clippy/fmt/test
green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 7ab8acaf55 feat(host/steam): harden the gadget feature contract — fixes the evdev churn
The virtual Deck's gamepad evdev was churning (destroyed + recreated) because
Steam kept re-probing: GetControllerInfo reads HID feature reports, and the gadget
served zeros for them. Captured the real contract off a physical Deck
(packaging/linux/steam-deck-gadget/get_deck_attrs.c, hidraw HIDIOCGFEATURE — usbmon
truncates to 32B) and implemented it in steam_gadget.rs::feature_reply:

- 0x83 GET_ATTRIBUTES_VALUES: [83, 2d, 9×(attr-id, u32-LE)] — product id 0x1205, a
  per-instance unit serial (0x0a/0x04, so a gadget never collides with a real Deck
  or another gadget), and the capability attrs (0x09=0x2e, 0x0b=0x0fa0, rest 0).
- 0xAE GET_STRING_ATTRIBUTE: [ae, len, attr, ascii] — serial (attr 1) / board
  serial (attr 0).
- other commands (0x87 settings): echo the last write.

Validated on the Deck: 1 connect / 0 disconnect / 1 gamepad evdev (was constant
churn), Steam activates the gadget cleanly (no GetControllerInfo failed, no zombie)
and emits its X-Box 360 pad. usbmon on the gadget's bus confirms our state reports
(pressed button at byte 8) are delivered on the interrupt-IN and consumed by
hid-steam — so with M1/M2's byte-8→BTN_SOUTH decode the input chain is proven
end-to-end. Remaining: a foreground-game confirmation of Steam Input's XInput
mapping, then default the gadget on for SteamOS.

Workspace clippy/fmt/test green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler c8e19396e4 feat(host/steam): raw_gadget Deck host backend (Steam-Input path, opt-in)
Port the proven raw_gadget virtual Deck to a Rust host gamepad backend, the
SteamOS-only transport that gets Steam Input to actually promote the Deck.

- inject/linux/steam_gadget.rs (new): SteamDeckGadget — a userspace raw_gadget
  emulator of the real 3-interface USB Deck (mouse=0/keyboard=1/controller=2,
  28DE:1205) on a dummy_hcd loopback UDC, descriptors captured from a physical
  Deck, answering every control transfer incl. the HID feature reports. Driven by
  the same steam_proto::serialize_deck_state as the UHID pad; rumble feedback via
  parse_steam_output. The raw_gadget UAPI is funneled through 4 documented ioctl
  wrappers (the crate denies undocumented unsafe).
- inject/linux/steam_controller.rs: the manager pad is now a DeckTransport enum
  (Uhid | Gadget); ensure() prefers the gadget when PUNKTFUNK_STEAM_GADGET=1
  (best-effort modprobe dummy_hcd+raw_gadget), gracefully falling back to the
  universal UHID SteamDeckPad. write/pump/heartbeat dispatch through the enum.

Validated on a real Deck via a static musl harness that #[path]-includes the
module: enumerates, hid-steam binds + reads our serial + creates the Steam Deck +
Motion Sensors evdevs — identical to the C PoC. Caught a real portability bug:
raw_gadget's no-arg ioctls (RUN/CONFIGURE/EP0_STALL) reject a non-zero `value`
with EINVAL, and on musl an omitted ioctl vararg is a garbage register — so they
must pass an explicit 0.

Opt-in (default off) while the Steam GetControllerInfo feature contract is
hardened (to stop the gamepad-evdev churn). Workspace clippy/fmt/test green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 78020cd66c docs(steam): gadget input-flow status — reports delivered + format-validated
On the Deck, a pressa build shows hid-steam polls our interface-2 interrupt-IN
endpoint and our 64-byte state reports are delivered ("STREAM: first input report
delivered"). The report format is already validated (M1 serializer on-box + M2's
EVIOCGKEY/EVIOCGABS test on the same hid-steam decode). The "Steam Deck" gamepad
evdev forms but is transient (hid-steam recreates it as gamepad_mode toggles —
Steam keeps re-probing because the PoC serves the serial but not Steam's full
GetControllerInfo attribute set, on a heavily-churned test Deck), so a stable live
EVIOCGKEY catch of the held A wasn't obtained. Delivery + format proven; the
evdev transience is a feature-report-completeness gap the host backend resolves.
Doc §11. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 8870e85233 feat(steam): raw_gadget virtual Deck — full Steam Input recognition (proven on Deck)
The interface-2 wall is climbed. packaging/linux/steam-deck-gadget/deck_raw_gadget.c
is a raw_gadget userspace emulator of a real 3-interface USB Steam Deck (28DE:1205,
mouse=0/keyboard=1/controller=2) on a dummy_hcd loopback UDC, with descriptors
captured verbatim from a physical Deck and full HID feature-report handling.

Live on a real Deck (SteamOS 3.8.11): hid-steam reads our serial (PFDECK000),
creates the Steam Deck + Motion Sensors evdevs, and Steam Input PROMOTES it —
controller.txt "Interface: 2 ... device opened ... reserving XInput slot 1" +
"input: Microsoft X-Box 360 pad 1". Stable (1 connect, 0 disconnects, no zombie);
the kernel Steam Deck evdev is then grabbed by Steam Input which exposes its own
X-Box pad, exactly like a real Deck. First time a virtual Deck is fully Steam-Input
promoted (UHID can't — it has no USB interface number, so Steam filters it).

Also includes the configfs f_hid variant (configfs_gadget_up/down.sh) — the minimal
reproducer that proved interface 2 makes Steam open+XInput-reserve the device, but
f_hid can't serve feature reports so Steam dropped it as a zombie.

Gotchas documented in the README: 7-byte vs 9-byte endpoint descriptor, no-data OUT
controls acked via zero-length EP0_READ (not WRITE, else error -110), streamer must
not start before SET_CONFIGURATION is acked. SteamOS-host only (needs dummy_hcd +
raw_gadget). Recognition proven; feeding real client reports + a host backend is next.
Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler a81f1304cd docs(steam): gadget PoC — interface 2 PROVEN (Steam opens + XInput-reserves it)
On the Deck (which ships dummy_hcd + raw_gadget + configfs f_hid), a pure-shell
configfs gadget stood up a real 3-interface USB Deck (kbd=0/mouse=1/controller=2,
28de:1205) on a dummy_hcd loopback UDC. hid-steam bound all 3 interfaces, and
crucially Steam PROMOTED the interface-2 controller: "Local Device Found ...
Interface: 2 ... Steam controller device opened for index 14 ... Steam Controller
reserving XInput slot 1" — exactly where the interface -1 UHID Deck was filtered.

It then failed only at feature-report exchange (f_hid can't serve HID GET_REPORT:
"steam_send_report: error -32", "couldn't get controller details ... zombie
controller"), and no gamepad evdev formed for the same reason. So interface 2 is
necessary AND sufficient for Steam to open+XInput-reserve the Deck; the remaining
piece is serving feature/output reports, which raw_gadget can (full control,
like UHID). Next: a raw_gadget 3-interface Deck emulator. Doc §11. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler c75f39fd8e docs(steam): VALIDATED — virtual DualSense IS Steam-Input recognized; isolates the Deck wall to interface 2
Definitive hardware test (Bazzite running Steam): a virtual DualSense (UHID,
054c:0ce6, Interface: -1) is FULLY promoted by Steam — controller.txt logs
"Local Device Found 054c 0ce6 DualSense Wireless Controller", then "Controller
using HIDAPI driver vid=0x054c pid=0x0ce6" and loads configset_controller_ps5.vdf
(our calibration/pairing/firmware feature blobs read back). The SAME Interface:
-1 that the Deck is rejected at is accepted for the DualSense.

So the wall is specifically the Deck's MULTI-INTERFACE requirement (Steam must
pick interface 2 among kbd/mouse/controller), NOT a UHID limitation. The
DualSense path delivers real Steam Input (gyro + touchpad + glyphs + bindings)
for a streamed Deck/SC client; it loses only Deck glyphs, the 2nd trackpad, and
the 4 back grips as distinct Steam-Input paddles (M5 folds them to buttons).

Full Deck-identity Steam Input would need interface 2 -> a USB gadget (dummy_hcd
+ configfs HID, controller on interface 2). Feasible but heavy/non-portable:
dummy_hcd isn't built on Bazzite/Deck/dev-box, so it'd be a per-kernel build +
(on immutable SteamOS/Bazzite) a package-layer + reboot per host.

Doc-only (design §11). Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 37c3e2bed2 docs(steam): criterion-4 RESOLVED — the interface-2 ceiling; recommend dropping M7
Hardware finding (a SteamOS Deck @ .253 + a Bazzite host @ .41, both running
Steam, via a minimal C UHID probe on Bazzite): a UHID virtual Steam Deck binds
the kernel hid-steam and creates the evdevs (so kernel-evdev + SDL-hidapi
consumers see the full grips/trackpads/IMU surface), but Steam Input will NOT
manage it. Steam's controller.txt enumerates it ("Local Device Found, 28de 1205,
Product Punktfunk Steam Deck") but logs Interface: -1 and never promotes it (no
28de:11ff XInput pad). The physical Deck on the same logs is Interface: 2 — a
real Deck is a 3-interface USB device (kbd 0 / mouse 1 / controller 2) and Steam
binds the controller on interface 2; a single UHID device has no USB interface
number, so Steam reads -1 and filters it out. (The feared 0x83/0xA1 attribute
probes never fired — it's an interface filter, not a probe-reject.)

Consequences (design §11):
- The virtual Deck's value is non-Steam / SDL games on Linux (grips + trackpads
  + gyro via evdev / SDL HIDAPI), NOT Steam Input.
- The virtual DualSense stays the Steam-Input path everywhere (Steam recognizes
  a single-interface DualSense); M5's paddle-fold carries the back grips.
- M7 (a Windows UMDF virtual Deck) is NOT recommended: same interface filter,
  and Windows has no kernel-hid-steam evdev fallback, so nothing would consume
  it; the existing Windows virtual DualSense already covers that case.
- M0-M6 is not wasted: the protocol/wire + client capture feed the DualSense
  path too, and the virtual Deck is the best option for non-Steam Linux games.

Doc-only (design/steam-controller-deck-support.md): added §11, updated the status
+ pending-validation. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 4f40fa3cb7 feat(host/steam): M6 — Steam-pad conflict gate (hardware-validated)
Don't present a virtual Steam (28DE) pad on a host that already has a physical
Steam controller — the host's own Steam Input would then manage two Decks and
confuse player assignment.

- physical_steam_controller_present(): scans /sys/bus/hid/devices for a 28DE HID
  device on a real (non-/virtual/) path.
- degrade_steam_on_conflict() in resolve_gamepad: a resolved SteamDeck /
  SteamController with a physical Steam controller attached degrades to DualSense
  (then the M5 uhid ladder); PUNKTFUNK_STEAM_FORCE=1 overrides (e.g. a remote-only
  box with no competing Steam Input).

Validated on real hardware (a SteamOS Steam Deck @ .253 + a Bazzite host @ .41,
both running Steam):
- Conflict confirmed: the Deck-as-host already has its physical 28DE:1205 AND
  Steam's 28DE:11FF XInput output pad live; a 2nd virtual 28DE = two Decks.
- Bind robustness: the virtual Deck binds hid-steam on a SECOND kernel (Bazzite
  6.17.7, vs the dev box 7.0) and the kernel accepted our serial (the M1 fix).
- Criterion-4 (running-Steam recognition) PARTIAL: a userspace consumer (Steam/
  SDL) engaged the virtual Deck (opened the hidraw, ran the lizard-disable +
  settings sequence the kernel's Deck path skips) but emitted NO 28DE:11FF XInput
  pad on the desktop — so Steam recognizes it enough to manage lizard mode but did
  not promote it to a managed XInput controller (likely needs a Big-Picture/game
  context, or a richer device; the 0x83/0xA1 attribute probes never fired, so it
  wasn't a probe-reject either).
- The heuristic itself checks TRUE on the Deck, FALSE on Bazzite.

Workspace clippy/fmt/test green. Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 486a292845 feat(host/steam): M5 — fallback remap, motion rescale, degrade ladder
Keep the rich Steam inputs from silently dropping when the resolved backend
isn't the virtual hid-steam device, and fix a cross-device motion-scale bug.

- inject/proto/steam_remap.rs (new, pure + unit-tested):
  * motion_wire_to_deck — the wire carries DualSense-convention units (20 LSB/
    deg.s gyro, 10000 LSB/g accel — what every client capture emits), but the
    Deck's hid-steam report wants 16 LSB/deg.s + 16384 LSB/g. The Deck backend
    now rescales (gyro x16/20, accel x16384/10000): a real Deck<->Deck gyro/
    accel correctness fix (the DualSense/DS4 backends consume the wire 1:1).
  * fold_paddles + RemapConfig (PUNKTFUNK_STEAM_REMAP=paddles=drop|stickclicks|
    shoulders, default drop) — the DualSense + DS4 managers fold a client's back
    grips onto standard buttons rather than dropping them (those pads have no
    back-button HID slot; the uinput Xbox pad already exposes them as Elite
    paddles BTN_TRIGGER_HAPPY5-8).

- resolve_gamepad: a runtime degrade ladder — a UHID backend (DualSense / DS4 /
  Steam Deck) on a host where /dev/uhid isn't writable now falls back to the
  uinput Xbox 360 pad instead of a dead controller (the device-create would
  just fail). Separate from pick_gamepad's compile-time platform check, so the
  existing pick_gamepad tests are untouched.

- Delete the throwaway M0/M1 spike (src/bin/steam_uhid_spike.rs) — M2's
  #[ignore]d backend test subsumes its validation, and removing it frees
  steam_proto to reference steam_remap cleanly.

On-box backend test still green; workspace clippy/fmt/test green (incl. the new
steam_remap tests). Deferred as optional RemapConfig growth: gyro->mouse /
trackpad->stick synthesis on an Xbox target (no slot — documented drop today).
Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler d8c254281e feat(steam): M4 complete — C-ABI send path, Decky UX, Apple/Android parity
Finish the client side of the Steam Controller / Steam Deck pipeline.

- C-ABI (core abi.rs): PunktfunkRichInputEx — a size-prefixed superset of
  PunktfunkRichInput that can express the second trackpad (surface), a distinct
  click vs touch, signed coords + pressure — plus
  punktfunk_connection_send_rich_input2 (the struct_size ABI-skew-guard
  precedent). The only way a C client (Apple/embedders) can emit a TouchpadEx;
  the legacy struct + send_rich_input stay byte-for-byte. punktfunk_core.h
  regenerated.

- Decky (clients/decky): a "Steam Deck" gamepad type in Settings + an unmissable
  Disable-Steam-Input instruction shown when it's selected (in Game Mode Steam
  Input holds 0x1205, so the SDL HIDAPI Steam driver can't open the Deck's
  controls until the user disables Steam Input for the shortcut). Plus a
  best-effort, feature-detected disableSteamInputForShortcut() in launchStream —
  never blocks/throws; the manual toggle is the documented source of truth.

- Apple parity (PunktfunkConnection.swift): GamepadType.steamController/steamDeck
  (wire 5/6) + name parsing, so the resolved type round-trips. Capture is blocked
  (GameController never surfaces a 0x28DE HID device).

- Android parity (Gamepad.kt): PREF_STEAMCONTROLLER/STEAMDECK + the Valve 0x28DE
  PIDs in prefFor(). Rich-input capture stays out of scope (no rich-input plane
  yet) — standard buttons/sticks resolve to the host's Steam Deck pad.

Rust workspace clippy/fmt/test green; Decky src/ typechecks clean (only a
pre-existing @decky/api dep resolution error remains); Swift/Kotlin compile on
their CI. The full pipeline is now BUILT; what remains is validation that needs
hardware we don't have (a running Steam on the host, a live Deck client, the
Moonlight paddle regression). Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler ae71e4628d feat(clients/steam): M4 — desktop SDL clients capture the rich Steam inputs
The Linux + Windows native clients (clients/{linux,windows}/src/gamepad.rs) now
capture and send the Steam Controller / Steam Deck rich inputs, so a real Deck
(off Steam Input) or a Steam Controller on a desktop client drives the host's
virtual hid-steam pad end-to-end:

- Set SDL's HIDAPI Steam hints (SDL_JOYSTICK_HIDAPI_STEAMDECK / _STEAM) before
  init so SDL opens Valve devices directly (paddles + both trackpads + gyro as
  first-class SDL gamepad inputs).
- Detect the Deck/SC by VID/PID (0x28DE + 0x1205 / 0x1102 / 0x1142) ->
  GamepadPref::SteamDeck (there is no SDL gamepad type for it), so the host
  builds the virtual Deck with the right identity.
- Map the SDL paddle + Misc1 buttons -> BTN_PADDLE1..4 / BTN_MISC1 (a free win
  for Xbox Elite paddles too).
- Route a SECOND touchpad -> RichInput::TouchpadEx (SDL touchpad 0 = left ->
  surface 1, 1 = right -> surface 2, signed coords); a single touchpad keeps the
  legacy Touchpad. New forward_touch() helper centralizes the choice.
- Track held touchpad contacts per (surface, finger) and lift them on pad
  switch/detach so a contact held at that moment can't stick.
- Sensor (gyro/accel) capture was already generic across pad types.

Linux client builds + clippy clean; the Windows client is a near-verbatim
mirror (windows CI compiles it). On a Deck in Game Mode, Steam Input still holds
the device — the user disables Steam Input for the client (the Decky UX, next);
on a desktop client (or a Deck with Steam Input off) the hints just work.

Remaining M4: Decky Disable-Steam-Input UX, Apple/Android parity, and the C-ABI
PunktfunkRichInputEx + send_rich_input2 (Apple/embedder send path). Not pushed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 01c55aed38 feat(proto/steam): M3 — rich Steam wire (back buttons + 2nd trackpad)
Carry the rich Steam Controller / Steam Deck inputs end-to-end on the wire —
strictly additive + forward-compatible (unknown kinds/bits drop on old peers).

Core (punktfunk-core):
- input.rs: BTN_PADDLE1..4 + BTN_MISC1 in Moonlight's buttonFlags2<<16 namespace
  (so the GameStream paddle path and native grips share one host injector map;
  Steam L4/L5/R4/R5 reuse the four Xbox-Elite paddle slots).
- quic.rs: RichInput::TouchpadEx (kind 0x03 — surface 0/1/2, touch+click, signed
  coords, pressure; the second trackpad the single Touchpad can't express) and
  HidOutput::TrackpadHaptic (kind 0x04 — the SC voice-coil pulse). Round-tripped.
- abi.rs: PUNKTFUNK_GAMEPAD_STEAMDECK=6 / _STEAMCONTROLLER=5, the paddle bits,
  RICH_TOUCHPAD_EX / HIDOUT_TRACKPAD_HAPTIC constants. from_hid packs
  TrackpadHaptic into the existing which + effect[0..6] — the legacy structs do
  NOT grow (guarded by new size_of==20/19 asserts); GamepadPref lockstep +
  paddle-bit lockstep asserts extended. include/punktfunk_core.h regenerated.

Host (punktfunk-host):
- steam_proto::from_gamepad maps the wire paddles -> the four Deck grips + QAM;
  apply_rich routes TouchpadEx left/right -> the matching pad.
- every DualSense/DS4 manager (Linux + Windows) gained a TouchpadEx arm
  (surface 0/2 -> its one touchpad; surface 1 ignored) so the variant compiles
  everywhere and a Steam client streaming to a DS host keeps its right pad.
- the xpad BUTTON_MAP finally consumes the GameStream paddle bits
  (BTN_TRIGGER_HAPPY5-8) — Sunshine/Moonlight paddle clients were silently
  no-op'd before (design §5.6).
- Android feedback: drop TrackpadHaptic (no coils; rumble rides 0xCA).

Validated on-box: the ignored backend test now drives the full wire path —
from_gamepad (BTN_A + the L4 grip) + apply_rich (a left-pad TouchpadEx) reach the
evdev as BTN_A + ABS_HAT0X=-8000. Wire round-trips + paddle/TouchpadEx mapping
unit-tested. Workspace clippy/fmt/test green. Not pushed.

Deferred to M4: the C-ABI PunktfunkRichInputEx + send_rich_input2 (only the
Apple/embedder *send* path needs it; the host decodes TouchpadEx today).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 95308d352b feat(host/steam): M2 — virtual Steam Deck as a wired PadBackend (Linux)
Make the virtual hid-steam device a selectable per-session host gamepad,
end-to-end on Linux: PUNKTFUNK_GAMEPAD=steamdeck now builds a
SteamControllerManager that creates a /dev/uhid 28DE:1205 Deck, enters
gamepad_mode, and feeds the byte-exact Deck report (M1).

- inject/linux/steam_controller.rs: SteamControllerManager / SteamDeckPad,
  mirroring dualsense.rs (open/create2, GET/SET_REPORT pump, heartbeat, RAII
  destroy). Two Steam-specific quirks beyond the DualSense path:
    * gamepad_mode entry — best-effort `lizard_mode=0` via sysfs, plus a b9.6
      creation pulse (MODE_ENTER) so steam_do_deck_input_event stops
      early-returning, plus an anti-toggle guard (MENU_HOLD_CAP) so a long
      in-game Start-hold can't flip gamepad_mode back off.
    * UHID_SET_REPORT answered err=0 (DualSense omits it; the kernel stalls
      ~5s/cmd otherwise); the 0xEB rumble report parsed onto the 0xCA plane.
- core config.rs: GamepadPref::SteamDeck (wire byte 6) + SteamController
  (byte 5, reserved — folds to Xbox360 until its backend lands); from_u8 /
  from_name / as_str. Forward-compatible (unknown byte -> Auto); the C-ABI
  PUNKTFUNK_GAMEPAD_* constants stay M3, so no generated-header drift.
- punktfunk1.rs: PadBackend::SteamDeck variant + select / handle / apply_rich
  / pump / heartbeat arms; pick_gamepad Linux arm.

On-box: an #[ignore]d backend test (backend_binds_and_input_flows) drives the
real SteamDeckPad — it binds hid-steam (gamepad + IMU evdevs), enters gamepad
mode, BTN_A reaches the evdev, and the device tears down on drop. Workspace
clippy/fmt/test green. Not pushed. Next: M3 (protocol/ABI wire) + M4 (client
capture).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 9ff7d41bfe feat(host/steam): M1 — byte-exact Deck input serializer, on-box validated
Flesh out inject/proto/steam_proto.rs into the full Steam Deck HID contract,
transcribed verbatim from the kernel steam_do_deck_input_event /
steam_do_deck_sensors_event and validated field-for-field against kernel 7.0:

- SteamState: the u64 button map (bytes 8..16), sticks/triggers/trackpads/IMU
  stored as raw little-endian report values; serialize_deck_state is a pure,
  byte-exact memcpy into the 64-byte unnumbered frame.
- from_gamepad (XInput frame -> Deck buttons/sticks/triggers) + apply_rich
  (RichInput touchpad -> right pad, motion -> IMU).
- parse_steam_output: the 0xEB ID_TRIGGER_RUMBLE_CMD feedback -> (low, high)
  for the universal rumble plane.
- serial_reply fixed: prepend the report-id-0 byte the kernel strips
  (steam_recv_report does memcpy(data, buf+1, ...)); M0's reply lacked it, so
  the kernel fell back to the "XXXXXXXXXX" serial.
- SteamModel (Deck now; classic Controller later), command/feature IDs.

The spike is repurposed as the M1 validator: it pulses the b9.6 mode-switch to
enter gamepad_mode (steam_do_deck_input_event early-returns under the default
lizard_mode otherwise), then holds a known test pattern. Reading both evdevs via
EVIOCGABS/EVIOCGKEY, every field matched: ABS_X/Y/RX/RY (incl. the kernel
Y-negation), both triggers, the touched right-pad HAT1X/Y, the IMU accel/gyro
(with ABS_Z/RZ negations), and the 6 expected buttons incl. the L4/R5 grips.

5 unit tests + workspace clippy/fmt/test green. Next: M2 (SteamControllerManager
UHID backend + PadBackend wiring). Not pushed — pipeline not yet shippable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 2b47d8cc28 feat(host/steam): M0 — virtual hid-steam UHID device binds + parses (Linux)
Greenfield virtual Steam Deck controller, the Steam analogue of the shipped
virtual DualSense. Proves the kernel hid-steam driver binds a /dev/uhid
28DE:1205 device, registers it as a real Steam Deck, and parses our input
reports — the go/no-go gate for the full Steam Controller/Deck pipeline.

- inject/proto/steam_proto.rs: keeper module — the vendor HID descriptor (one
  feature report, the sole thing steam_is_valve_interface() checks), the
  command/feature IDs, serialize_deck_state, and the serial GET_REPORT reply.
  Unit-tested.
- src/bin/steam_uhid_spike.rs: throwaway M0 spike (Linux-only) — opens
  /dev/uhid, creates the device, services the handshake including
  UHID_SET_REPORT (which the DualSense backend omits and which hid-steam
  stalls ~5s/cmd without), and heartbeats a neutral report.
- design/steam-controller-deck-support.md: full design + M0–M7 plan; the two
  walls (Steam Input capture ownership; virtual-Steam recognition) and the
  fidelity ceiling. Status: M0 GREEN.

On-box (headless Ubuntu 26.04, kernel 7.0, no Steam): journalctl -k shows
hid-steam binding the device (rebind off hid-generic), "Steam Controller
connected", and the kernel creating BOTH a "Steam Deck" gamepad evdev and a
"Steam Deck Motion Sensors" IMU evdev (INPUT_PROP_ACCELEROMETER). A
layout-agnostic mash-probe drove 23 distinct BTN_* codes through
hid-steam -> evdev, proving the input-report parse path. M1 line-checks the
exact per-bit report layout against the lab kernel.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 7cd9364c9e style(host): rustfmt the #9/#13 pairing edits
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 3e498cd40d docs(security): #9/#13 fixed, S7 rationale corrected (red-team follow-up)
17/18 now fixed. A red-team of the three accepted findings showed #9 and #13
rested on a circular premise (each was the other's "safe fallback") and S7's
written rationale was wrong (signing exercises the same modexp Marvin targets).
#9/#13 closed; S7 accept retained for the corrected reasons + amplifier hardened.
See f0574a5, f6c9576.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 60de506f66 fix(host/gamestream): correct the rsa-Marvin (S7) rationale + cap pairing signatures
Red-team found the .cargo/audit.toml justification for RUSTSEC-2023-0071 was
materially wrong: it claimed "Marvin targets decryption, so the vulnerable path
isn't exercised" — but the advisory is a variable-time modexp of the secret
exponent, which RSA *signing* (signing_key.sign) also runs. The accept is still
correct, for the RIGHT reasons (no decryption/padding oracle; the signed
serversecret is host-random not attacker-chosen; signing is operator-PIN-gated;
GameStream is off by default and the native QUIC plane uses rustls, not rsa;
Moonlight mandates RSA-2048 so the GameStream key can't move off it). Rewrite
the rationale accordingly.

Also shut the timing-sample amplifier the review surfaced: the pairing session
was never marked after phase 3, so a peer past phase 1 could loop phase2/phase3
to harvest many RSA signing-time samples. Sign exactly once per ceremony
(reject a repeated serverchallengeresp).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 2865368771 fix(host/pairing): close native-pairing DoS findings #9 + #13 (red-team follow-up)
The accepts for #9 (PIN-window burn) and #13 (knock-queue flood) rested on a
circular premise — each cited the other as the safe fallback — and a re-review
showed one LAN attacker could defeat BOTH, denying all onboarding. Close them:

- #13 per-source-IP cap on the pending-knock queue (MAX_PENDING_PER_IP) so one
  host can't fill/evict the 32-slot queue (QUIC validates the source address);
  and eviction now NEVER drops a live *parked* knock (a held-open connection
  awaiting operator approval), so a cert-rotating flood can't evict the genuine
  device being onboarded. This makes the delegated-approval path genuinely
  flood-resistant — restoring the validity of #9's "use delegated approval on
  hostile LANs" fallback.

- #9 fingerprint-bindable PIN window: `NativePairing::arm_for(ttl, Some(fp))`
  binds the window to one operator-selected device; `pin_for_attempt` returns
  `BoundToOther` for any other fingerprint, which the QUIC pair path rejects
  WITHOUT consuming the window — so an unpaired peer can neither pair nor BURN a
  window armed for a specific device (it can't forge the bound fingerprint). The
  mgmt `POST /native/pair/arm` gains an optional `fingerprint` (from a pending
  knock); unbound arming keeps the legacy any-device behavior (trusted-LAN).
  (Web-console "pair this pending device with a PIN" UX is a follow-up; the
  flood-resistant knock path above is the immediate hostile-LAN onboarding path.)

+ regression tests (armed_pin_is_fingerprint_bindable,
  pending_per_ip_cap_and_parked_protection); api/openapi.json regenerated.
110 host tests + clippy + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler 6e2e946bc9 docs(security): #5 fixed + on-box validated (RTX box, 2026-06-29)
15/18 now fixed; no finding remains open and actionable. SDDL scoped to
SYSTEM+LocalService, validated live (6943-frame DualSense+IDD session works;
non-SYSTEM OpenFileMapping now ACCESS_DENIED). See e59fa60.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler b5f02000d6 fix(host/security): scope Windows shared-section SDDL to SYSTEM+LocalService (close #5)
The gamepad host<->UMDF-driver shared sections (Global\pfds-shm-*, pfxusb-shm-*)
and the IDD-push frame ring/event (Global\pfvd-*) were created with
`D:(A;;GA;;;WD)` — GENERIC_ALL to **Everyone** — on the assumption the driver's
WUDFHost ran under a restricted token needing broad access. So any local
unprivileged user could OpenFileMapping the section to inject controller input,
tamper the trusted HID channel, or read captured screen frames
(security-review 2026-06-28 #5).

On-box validation (RTX box, 2026-06-29) disproved the restricted-token premise:
the WUDFHost token is NT AUTHORITY\LocalService (S-1-5-19), SYSTEM integrity,
with ZERO restricted SIDs. So the section only needs SYSTEM (the host creates +
writes it) and LocalService (the driver opens it). Scope both SDDL sites to
`D:(A;;GA;;;SY)(A;;GA;;;LS)`; rename the now-misnamed `permissive_sa` ->
`shared_object_sa`; correct the stale "restricted-token / Everyone" docs.

Validated live: a full DualSense + 1280x720x60 session — 6943 frames received,
HID output round-tripped, device status OK (pf_dualsense + pf_vdisplay WUDFHosts
both LocalService open the scoped sections fine), while OpenFileMapping from a
non-SYSTEM admin session now returns ACCESS_DENIED (was a granted handle under
WD). Host-only change (the SDDL is set when the host CREATES the section);
drivers unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00
enricobuehler fe562f0562 chore(apple): rename app display name to "Punktfunk"
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
release / apple (push) Has been cancelled
android / android (push) Has been cancelled
ci / rust (push) Has been cancelled
decky / build-publish (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
CFBundleDisplayName was "Punktfunkempfänger" across all targets/configs; the
in-app title is already "Punktfunk", so make the home-screen name match. Built
iOS app resolves CFBundleDisplayName = "Punktfunk".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:34:16 +02:00
enricobuehler 4e00037a89 feat(apple): stage-2 default + pixel-perfect, decode robustness, UI/rumble polish
apple / swift (push) Successful in 1m4s
android / android (push) Successful in 4m33s
ci / rust (push) Successful in 5m4s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 59s
deb / build-publish (push) Successful in 3m12s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
release / apple (push) Successful in 8m30s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m45s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m48s
apple / screenshots (push) Successful in 5m43s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m24s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m46s
Stream reliability
- Default to the stage-2 presenter (VTDecompressionSession + CAMetalLayer): it detects
  and recovers a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard
  on a lost HEVC reference frame with no app-side recovery (confirmed Apple limitation).
  Stage 1 is now a DEBUG-only presenter toggle, plus the automatic no-Metal fallback.
- Stage-2 pixel-perfect: render the drawable at the decoded size (shader stays 1:1 =
  identity) and let the layer's contentsGravity scale via the system compositor — the
  same path stage-1's videoGravity used — instead of scaling in-shader.
- Loss recovery in both pumps is now a persistent awaitingIDR want, retried until an IDR
  actually lands, so a keyframe request swallowed by the throttle can't strand a frozen
  frame; 100 ms keyframe throttle to match the Android path.
- Fix "Publishing changes from within view updates": defer the HostStore writes out of
  the .onChange(of: model.phase) callback.
- Move AVAudioSession setActive/setCategory off the main thread (async on a shared serial
  queue) to stop the UI-stall warning.

Controllers
- Rumble: capped-exponential backoff when the gamecontrollerd.haptics XPC breaks (-4811)
  so a transient server interruption self-heals instead of cascading; playsHapticsOnly so
  a controller engine doesn't join the always-active streaming audio session.
- Host cards: iPad pointer "magnet" hover effect; iPhone press scale + light haptic.

UI / design
- Ship Geist (SIL OFL 1.1) as the app font (bundled OTFs + registration), with the
  license surfaced in Acknowledgements.
- Restructure iOS/iPadOS Settings into a category NavigationSplitView; resolution wheel
  with custom-resolution entry; 10-bit HDR toggle in Display.
- Industrial host-card redesign (left-aligned, bold, brand monogram tiles).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:26:10 +02:00
enricobuehler 46b9aa8cf0 fix(windows-host): IDD activation — resolve-first, force-EXTEND only as fallback
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 4m23s
ci / rust (push) Successful in 5m9s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 57s
apple / screenshots (push) Successful in 5m33s
windows-host / package (push) Successful in 8m46s
deb / build-publish (push) Successful in 3m13s
decky / build-publish (push) Successful in 27s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m51s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m15s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m19s
docker / deploy-docs (push) Successful in 21s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m44s
force_extend_topology() was added before the resolve loop to de-clone a fresh IDD on
integrated-screen boxes (laptops), but its bare SDC_TOPOLOGY_EXTEND preset is
ACCESS_DENIED from the Session-0 service context on a HEADLESS box and broke the IDD
auto-activation there: resolve_gdi_name stayed None -> "not an active display path" ->
black screen. That regressed the headless/primary platform (live RTX box).

Revert to the proven e2c9bfd flow: resolve FIRST (Windows auto-activates the IDD as its
own extended path), and force-EXTEND only as the FALLBACK when resolve returns None (the
integrated-screen clone case, observed live to leave resolve None). The success path is
byte-identical to e2c9bfd (resolve -> set_active_mode -> isolate_displays_ccd).

Validated live: the headless RTX box streams again (probe: frames flow, driver attaches
to the ring, host/driver render LUIDs match).

Reviewed multi-agent + adversarial: no regression on the validated headless path or the
observed Optimus-laptop clone path (a cloned IddCx target resolves to None there, so the
is_none() fallback fires + de-clones). Known theoretical caveat, documented inline and
unobserved for IddCx but untested across GPU/driver/OS: a CCD clone that manifests as a
shared-source ACTIVE path would resolve to Some and bypass the is_none() gate. Follow-up:
widen the gate (a target_is_cloned helper) once an integrated-screen box is available to
validate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 16:15:48 +02:00
enricobuehler 372b27540b fix(apple): render Acknowledgements notices in lazy chunks
apple / swift (push) Successful in 1m11s
android / android (push) Successful in 4m19s
ci / web (push) Successful in 51s
ci / rust (push) Successful in 5m14s
ci / docs-site (push) Successful in 58s
release / apple (push) Successful in 7m54s
deb / build-publish (push) Successful in 4m0s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
ci / bench (push) Successful in 5m19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
apple / screenshots (push) Successful in 5m37s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m26s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m35s
docker / deploy-docs (push) Successful in 18s
THIRD-PARTY-NOTICES.txt is ~885 KB / 16k lines; rendering it in a single
SwiftUI Text overshot the text-rendering height limit — it laid out for ages
and drew blank below the cutoff (only the small punktfunk licenses above it
showed). Split the notices into ~80 line-chunks (<=200 lines / <=18 KB each,
computed once as Licenses.thirdPartyNoticesChunks) and render them in a
top-level LazyVStack so only on-screen chunks lay out and no chunk is tall
enough to clip. Chunking is lossless — rejoining the chunks reproduces the
original byte-for-byte, so no notice text is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:34:19 +02:00
enricobuehler db4d15bf8b fix(apple): stop the iOS/iPadOS Add Host sheet from scrolling
apple / swift (push) Successful in 1m5s
android / android (push) Successful in 4m15s
ci / rust (push) Successful in 4m56s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 58s
deb / build-publish (push) Successful in 3m11s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
release / apple (push) Successful in 8m32s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m40s
apple / screenshots (push) Successful in 5m44s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m29s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m31s
docker / deploy-docs (push) Successful in 17s
The add-host content is a SwiftUI Form (backed by a scrollable list), so it
bounced/scrolled inside the fixed .height(320) detent even though the three
rows + action button fit exactly. Lock it with .scrollDisabled(true) on iOS
(covers iPadOS); macOS (fixed-size panel) and tvOS (custom rows, no Form) are
untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:12:08 +02:00
enricobuehler 8e24ea9ed7 fix(ci): archive Apple release builds with Automatic signing
The in-app OSS license screens (7591425) added a `resources:` array to the
PunktfunkKit SwiftPM target, which makes SwiftPM emit a resource-bundle target
(PunktfunkKit_PunktfunkKit). A resource bundle is a product type that cannot
carry a provisioning profile, so the explicit PROVISIONING_PROFILE_SPECIFIER
each release.yml archive step set — global on macOS, sdk-scoped on iOS/tvOS —
now lands on it and fails the archive ("does not support provisioning profiles")
on all three platforms. (Before that commit there was no resource bundle, so the
profile was harmless.)

Switch all three archive steps to CODE_SIGN_STYLE=Automatic (development):
Automatic signing assigns a profile only to the app target and leaves the
resource bundle (and the macOS-host SwiftPM macro plugins) alone, and bakes the
sandbox entitlements in. No -allowProvisioningUpdates, so it stays offline and
never cloud-signs (the App-Manager ASC key can't). DISTRIBUTION signing is
unchanged — still manual, in the -exportArchive step (which maps the profile to
io.unom.punktfunk only). Drops the now-unneeded manual signing xcconfigs.

Requires the runner to have a development provisioning profile for
io.unom.punktfunk on each platform (now installed for macOS/iOS/tvOS).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:12:08 +02:00
enricobuehler 73c0125843 fix(mgmt): regenerate api/openapi.json for 0.3.0
apple / swift (push) Successful in 1m6s
android / android (push) Successful in 4m16s
ci / rust (push) Successful in 5m5s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 58s
apple / screenshots (push) Successful in 5m17s
deb / build-publish (push) Successful in 3m11s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
ci / bench (push) Successful in 4m43s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 26s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m28s
docker / deploy-docs (push) Successful in 17s
The OpenAPI 'info.version' tracks CARGO_PKG_VERSION; the 0.3.0 bump made the
checked-in spec stale (the openapi_document_is_complete_and_checked_in test).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 07:54:30 +00:00
enricobuehler ed54f22997 docs(design): add multi-user / profiles design (schema-of-record)
apple / swift (push) Successful in 1m10s
audit / cargo-audit (push) Successful in 1m16s
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m8s
release / apple (push) Successful in 4m28s
ci / bench (push) Successful in 4m51s
apple / screenshots (push) Successful in 5m45s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 3m0s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 3m4s
android-screenshots / screenshots (push) Successful in 2m22s
windows-host / package (push) Successful in 7m31s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m13s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m13s
android / android (push) Successful in 3m41s
deb / build-publish (push) Successful in 3m28s
decky / build-publish (push) Successful in 16s
linux-client-screenshots / screenshots (push) Successful in 2m20s
flatpak / build-publish (push) Successful in 4m13s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m4s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m43s
docker / deploy-docs (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
web-screenshots / screenshots (push) Successful in 2m29s
ci / rust (push) Failing after 4m8s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 06:52:43 +00:00
enricobuehler 031ee86ed5 chore(release): bump workspace version to 0.3.0
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 06:52:43 +00:00
enricobuehler 7591425f6f feat(clients): in-app OSS / third-party-license screens
Surface THIRD-PARTY-NOTICES.txt in every GUI client (the desktop packages already
ship it as a file; this adds the on-glass screen):

- Linux: Preferences -> About -> Third-party licenses (adw::AboutDialog with the app
  license + Legal sections; include_str! the root notices).
- Apple: macOS About tab / iOS+tvOS Acknowledgements link; notices bundled as
  PunktfunkKit SPM resources, read via Bundle.module (the Xcode app links the SPM
  product, so they ride along - no .pbxproj edit).
- Android: Settings -> About -> Open-source licenses (reads the bundled asset).
- (Windows landed earlier in d1d2ca2: Settings -> About -> Third-party licenses.)

gen-third-party-notices.sh now copies the generated file into the Apple Resources/
and Android assets/ trees so the in-tree copies never drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 06:52:43 +00:00
enricobuehler d1d2ca293d feat(pairing): seamless no-PIN delegated approval (host parks the knock, clients add "Request access")
Web-console "Approve" (delegated pairing, roadmap §8b-1) was unreachable: every
client routed a fresh pair=required host straight to the SPAKE2 PIN ceremony, so
no "knock" was ever recorded; and an unpaired connect was rejected+closed with no
way to resume after approval. The backend + console were complete but had no
client-side trigger and no post-approval admit path.

Host (native_pairing.rs, punktfunk1.rs): an unpaired identified knock is now
PARKED instead of rejected — it releases its NVENC session permit, awaits an
operator decision (NativePairing::wait_for_decision, woken by a Notify on
approve/deny), and on approval re-acquires a slot and admits the SAME connection
with no reconnect. QUIC keep-alive (4s/8s) holds the parked connection warm. The
pairing gate moves out of the HANDSHAKE_TIMEOUT-bounded handshake future;
approve_pending is reordered read-then-add and wait_for_decision double-checks
is_paired to close a "neither pending nor paired" race. New PENDING_APPROVAL_WAIT
(180s). Tests: delegated_approval_admits_after_knock now approves mid-park (no
reconnect) + new wait_for_decision_approve_deny_timeout unit test (108 host tests
green).

Clients (Linux/Apple/Windows/Android): a fresh pair=required host now offers
"Request access" alongside the PIN ceremony — a plain identified connect with a
~185s handshake budget and a cancelable "waiting for approval" UI; on success the
host is saved as paired, and cancel returns the UI immediately while a late-
resolving connect is torn down silently via a per-attempt flag. Apple reuses the
existing C-ABI timeout_ms (no ABI change); Windows adds SessionParams.connect_timeout
+ a RequestAccess screen; Android adds a timeoutMs arg to the nativeConnect JNI
seam (both sides + both callers). Linux built + clippy + fmt clean; Apple/Windows/
Android pending their CI/on-device compiles.

SPAKE2 ceremony reviewed end-to-end against the spake2 0.4 contract — correct, no
changes needed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 06:41:09 +00:00
enricobuehler 705a8fa94e chore(deps): drop unmaintained rustls-pemfile; axum-server 0.7 -> 0.8
axum-server was used only for the plain-HTTP nvhttp listener, but we enabled
its tls-rustls feature (HTTPS is hand-rolled over tokio-rustls) — and that
feature was what pulled the unmaintained rustls-pemfile (RUSTSEC-2025-0134).
Drop the feature, bump axum-server to 0.8 (0.8 also no longer pulls it), and
move our own PEM parsing in gamestream/tls.rs to rustls-pki-types' PemObject
(the same path punktfunk-core/quic.rs already uses), removing our direct
rustls-pemfile dep too.

Net: rustls-pemfile fully gone; dependency graph trimmed 547 -> 529 crates
(the tls-rustls feature also dragged in prettyplease + a wasm-tooling chain).
cargo audit now reports only audiopus_sys + paste (transitive, latest, no
successor). 108 host tests + clippy + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 06:32:58 +00:00
enricobuehler 4ba63b7da6 fix(deps): bump memmap2 0.9.10 -> 0.9.11 (RUSTSEC-2026-0186, unsound)
windows-drivers / probe-and-proto (push) Successful in 20s
apple / swift (push) Successful in 1m12s
windows-drivers / driver-build (push) Successful in 1m13s
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
audit / cargo-audit (push) Successful in 16s
release / apple (push) Successful in 8m15s
ci / web (push) Successful in 47s
ci / docs-site (push) Successful in 57s
windows-host / package (push) Successful in 9m9s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m46s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m13s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 58s
ci / rust (push) Successful in 8m24s
ci / bench (push) Successful in 4m53s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 58s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
deb / build-publish (push) Successful in 3m12s
flatpak / build-publish (push) Successful in 4m8s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m10s
docker / deploy-docs (push) Successful in 17s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m47s
memmap2 0.9.10 has an unchecked-pointer-offset unsoundness; 0.9.11 is the
patched release (pulled transitively via xkbcommon in the host). cargo audit
now reports only the 3 deliberately-visible `unmaintained` warnings
(audiopus_sys / paste / rustls-pemfile — all latest, transitive, warn-only,
do not fail CI per .cargo/audit.toml).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 06:20:55 +00:00
enricobuehler bee1f0416d chore(licensing): LGPL FFmpeg swap, third-party notices, attribution hygiene
The MIT OR Apache-2.0 SOURCE license is clean (audit found no copied copyleft); the
gaps were all binary-distribution (Layer-2). This makes the shipped artifacts honest:

- Windows host + client: bundled FFmpeg BtbN gpl-shared -> lgpl-shared (AMF/QSV/decode
  unaffected; the GPL-only x264/x265 were never used), and ship the FFmpeg LGPL notice
  + license text in the installer + MSIX (licenses/).
- THIRD-PARTY-NOTICES.txt generated + bundled into installer/MSIX/deb/rpm. Offline
  generator (scripts/gen-third-party-notices.{py,sh}) + cargo-about config (about.toml/
  .hbs) with a permissive-only accepted-license allow-list as a copyleft regression gate.
- Reword the win32u GPU-preference hook comments to reflect independent reimplementation
  (no Apollo/Sunshine GPL-3.0 source copied).
- README dual-license + inbound=outbound contributor clause + non-affiliation trademark
  disclaimer; new CONTRIBUTING.md.
- LICENSE files into the standalone driver + vk-layer workspaces; deb copyright holder
  aligned to "unom and the punktfunk contributors".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 06:20:38 +00:00
enricobuehler 54d9246ca7 fix(deps): bump quinn-proto 0.11.14 -> 0.11.15 (RUSTSEC-2026-0185)
apple / swift (push) Successful in 1m7s
audit / cargo-audit (push) Successful in 1m14s
android / android (push) Successful in 4m24s
ci / web (push) Successful in 46s
ci / docs-site (push) Successful in 57s
ci / rust (push) Successful in 7m32s
windows-host / package (push) Successful in 8m47s
release / apple (push) Successful in 8m42s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m26s
ci / bench (push) Successful in 4m40s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 7s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m23s
deb / build-publish (push) Successful in 3m6s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m28s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m34s
apple / screenshots (push) Successful in 5m28s
flatpak / build-publish (push) Successful in 4m21s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m39s
docker / deploy-docs (push) Successful in 17s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m43s
The 0.11.15 bump for S1 (pre-auth out-of-order STREAM reassembly memory
exhaustion on the default QUIC listener) was reverted before the original
fix commit, so Cargo.lock on main still pinned the vulnerable 0.11.14 and
the new cargo-audit CI gate failed. Re-apply and lock it in.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 06:05:10 +00:00
enricobuehler 91bb955d0c style(host): rustfmt the security-fix wrapping (cargo fmt --all --check)
apple / swift (push) Successful in 1m5s
ci / rust (push) Successful in 1m53s
ci / web (push) Successful in 57s
android / android (push) Successful in 3m47s
ci / docs-site (push) Successful in 1m2s
apple / screenshots (push) Successful in 5m35s
deb / build-publish (push) Successful in 2m52s
decky / build-publish (push) Successful in 22s
windows-host / package (push) Successful in 8m26s
ci / bench (push) Successful in 4m51s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m41s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m16s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 55s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m5s
docker / deploy-docs (push) Successful in 23s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m53s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 05:19:22 +00:00
enricobuehler 36259b264f docs(security): record remediation status for the 2026-06-28 host audit
apple / swift (push) Successful in 1m6s
ci / rust (push) Failing after 56s
ci / web (push) Successful in 52s
android / android (push) Successful in 3m24s
ci / docs-site (push) Successful in 1m4s
apple / screenshots (push) Successful in 5m23s
windows-host / package (push) Successful in 7m36s
deb / build-publish (push) Successful in 2m52s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m43s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m59s
docker / deploy-docs (push) Successful in 17s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m43s
14/18 fixed (3532e35 Linux-verified + 6f903f7 Windows DACL paths pending
CI/box); #5 deferred (needs on-box validation), #9/#13 accepted, S7
acknowledged (no upstream rsa fix).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 22:16:25 +00:00
enricobuehler 6f903f79bc fix(host/security): Windows DACL hardening — close audit #2, #3, #8, #11
Windows local-privilege findings from design/security-review-2026-06-28.md.
These are #[cfg(windows)] paths (verify in CI / on the box; this Linux dev
VM can't compile MSVC). They follow the existing write_secret_file/icacls
patterns; the cross-platform parts are cargo check/clippy/test green.

- #2 [HIGH]: route the mgmt bearer token write through the shared
  write_secret_file so it gets the SAME Windows DACL (SYSTEM/Administrators)
  as the host key — it was cfg(unix)-only and left Users-readable, leaking
  full mgmt admin authority to any local user.
- #3 [HIGH]: create_private_dir now applies a restrictive DACL to the
  %ProgramData%\punktfunk config directory (re-owns to Administrators to
  defeat a pre-creation, strips inheritance, SYSTEM/Admins/OWNER full +
  Users read-only) so a local user can't plant host.env/apps.json that the
  SYSTEM service trusts (env/arg-injection LPE). host.env is now written
  DACL-locked via write_secret_file; the config + logs dirs go through
  create_private_dir.
- #8 [LOW]: write the web-console password file empty, icacls-lock it, THEN
  write the secret — closes the brief write-then-icacls TOCTOU window.
- #11 [LOW]: the SYSTEM logs dir is DACL-locked (Users read-only, no
  create), so a local user can't pre-plant host.log as a reparse/hardlink to
  redirect SYSTEM's writes (subsumed by the #3 dir lockdown).

Deferred: #5 (host<->UMDF gamepad/IDD shared-section Everyone:GENERIC_ALL).
The section SDDL is intentionally permissive because the UMDF driver opens
it under a restricted token of unknown SID/integrity; scoping it blind would
likely break the live-validated gamepad/IDD pipeline, so it needs on-box
validation first. Tracked in the report.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 22:14:19 +00:00
enricobuehler 3532e35b75 fix(host/security): close audit findings S1,#1,#4,#10,#12,#7,#6,S2-S6 (Linux/cross-platform)
Remediations from design/security-review-2026-06-28.md verified on Linux
(cargo check/clippy/test green; Windows-gated paths verify in CI):

- S1 [HIGH]: bump quinn-proto 0.11.14 -> 0.11.15 (RUSTSEC-2026-0185,
  pre-auth out-of-order STREAM reassembly memory exhaustion on the
  always-on default QUIC listener).
- #1 [HIGH]: remove the unauthenticated nvhttp `GET /pin` endpoint; the
  GameStream PIN is delivered ONLY via the bearer-gated mgmt API, so a
  network client can no longer submit its own displayed PIN and self-pair.
- #4 [HIGH->MED]: gate the unauthenticated RTSP/UDP media plane on a paired
  `/launch` and bind it to the launching client's source IP (threaded
  through the HTTPS handler), so an unpaired peer can neither start capture
  on an idle host nor ride a paired client's active launch.
- #12: bound concurrent parked pairing waiters (MAX_PARKED_WAITERS) so a
  pre-auth peer can't pin unbounded 300s handshakes. +regression test.
- #10: throttle the per-packet ENet control GCM-decrypt-failed warn
  (exponential backoff) so a junk flood can't spam the log.
- #7 [MED->LOW]: serialize all process-global env mutation on the
  session-setup path under a new vdisplay::ENV_LOCK (apply_session_env /
  apply_input_env / the launch-cmd set_var / the gamescope env read), so
  concurrent native sessions can't race set_var/getenv (data-race UB ->
  host-wide DoS). Full per-session SessionContext threading remains a
  follow-up for cross-session value confusion.
- #6 [MED]: move the gamescope EIS socket relay from world-writable /tmp to
  $XDG_RUNTIME_DIR (per-user 0700) and reject a symlinked relay file, so a
  local user can't intercept (keylog) or deny the remote session's input.
- S2: a malformed client Opus mic frame now drops that frame instead of
  tearing down the shared host-lifetime virtual mic (cross-session DoS).
- S3: track held buttons/keys in capped HashSets (was unbounded Vec with
  O(n) scans) so a paired client can't grow per-session input state.
- S5: reject fps==0/absurd at the open_video chokepoint (covers Hello,
  ANNOUNCE, Reconfigure) so the encoder time_base/pts math can't div-by-0.
- S6: bound the shared mic mpsc (drop-newest when full).
- S4: cap Epic launcher-cache reads (catcache.bin/.item) so a planted giant
  can't OOM the host during library enumeration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 22:06:24 +00:00
enricobuehler 6b846913f5 docs(security): 2026-06-28 host security audit (follow-up) report
Multi-agent follow-up audit of the privileged streaming host: 18 attack
surfaces, every finding adversarially double-verified, plus a coverage
critic. Records 15 confirmed + 9 partial findings and a prior-fix
re-verification of the 2026-06-21 review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 22:05:58 +00:00
161 changed files with 64491 additions and 1333 deletions
+25 -8
View File
@@ -5,16 +5,33 @@
# means the audit job stops flagging it, so the reasoning must hold up.
#
# NOTE: `cargo audit` (no `--deny warnings`) fails only on *vulnerabilities*, not on the
# `unmaintained` warnings (audiopus_sys / paste / rustls-pemfile). Those are left visible on purpose
# so we keep getting the maintenance signal — they do not fail CI.
# `unmaintained` warnings (audiopus_sys via opus, paste via utoipa-axum). Both are transitive, at
# their latest published version with no successor, so there's nothing to bump — left visible on
# purpose so we keep getting the maintenance signal; they do not fail CI. (rustls-pemfile was dropped
# 2026-06-29 by removing axum-server's unused tls-rustls feature + moving our own PEM parsing to
# rustls-pki-types; memmap2's unsoundness was fixed by the 0.9.11 bump.)
[advisories]
ignore = [
# rsa "Marvin Attack" a timing sidechannel in RSA *decryption* (PKCS#1 v1.5 padding oracle).
# There is NO fixed rsa release (the constant-time rewrite is still unreleased upstream), and rsa
# is required for GameStream/Moonlight pairing. Crucially, the host uses rsa ONLY for PKCS#1 v1.5
# SIGNING / VERIFYING (gamestream/cert.rs + gamestream/pairing.rs: SigningKey / VerifyingKey /
# Signer / Verifier) — it never performs RSA decryption, which is the operation Marvin targets.
# So the vulnerable code path is not exercised. Revisit if a fixed rsa ships or we add RSA decrypt.
# rsa "Marvin Attack" (RUSTSEC-2023-0071): a timing side-channel in the rsa crate's variable-time
# modular exponentiation of the SECRET exponent. IMPORTANT — this affects the RSA private-key op in
# general, INCLUDING signing (m^d mod n), which the host DOES perform (gamestream/pairing.rs
# `signing_key.sign(&serversecret)`). It is NOT, as an earlier version of this note wrongly claimed,
# limited to decryption — so "the vulnerable path isn't exercised" is false; signing exercises it.
# We accept it because the attack is not practically reachable here, NOT because the path is unused:
# * No RSA decryption / PKCS#1v1.5 padding oracle exists anywhere (every `decrypt` in the tree is
# AES/AES-GCM), so the classic Bleichenbacher/Marvin chosen-ciphertext oracle is absent.
# * The only signed message (`serversecret`) is HOST-generated random, never attacker-chosen — so
# there's no adaptive chosen-input probing (the lever remote RSA-timing key recovery needs); and
# signing is gated behind the operator-entered pairing PIN, ONE signature per ceremony (a
# repeated phase-3 is rejected — gamestream/pairing.rs — to deny a passive timing-sample harvester).
# * GameStream is OFF by default (bare `serve` is native-only); the secure native QUIC plane uses
# rustls' constant-time backend, NOT the rsa crate. RSA is touched only on the opt-in,
# trusted-LAN GameStream/Moonlight pairing handshake. Moonlight mandates RSA-2048, so the
# GameStream identity cannot move to Ed25519/ECDSA (only the native identity could, and it
# already avoids the rsa crate).
# There is NO fixed rsa release (the constant-time rewrite is still unreleased upstream). Revisit if:
# a constant-time rsa ships (then drop this), the host ever signs an attacker-chosen message with
# this key, or any RSA decryption / key-transport using the private key is added.
"RUSTSEC-2023-0071",
]
+40 -48
View File
@@ -207,10 +207,20 @@ jobs:
# (Config/Punktfunk-macOS.entitlements) — mandatory for the Mac App Store.
continue-on-error: true
run: |
# Separate archive from the Developer ID one above: App Store needs a profile-signed
# archive (manual signing), not the unsigned-then-codesign DMG path. Same App-Manager
# ASC-key constraint as iOS/tvOS — MANUAL signing, NOT -allowProvisioningUpdates
# (cloud signing the key can't do). Quit Xcode so it can't prune the dropped profile.
# Separate archive from the Developer ID one above: App Store needs a signed, entitled
# archive that -exportArchive can re-sign for distribution, not the unsigned-then-codesign
# DMG path. Archive with AUTOMATIC signing (development). Why not a manually-specified
# profile (as this step used to do): the in-app license screens added a SwiftPM resource
# bundle (PunktfunkKit_PunktfunkKit), and a resource bundle is a product type that cannot
# carry a provisioning profile — a global PROVISIONING_PROFILE_SPECIFIER (here) or an
# sdk-scoped one (iOS/tvOS) lands on it and fails the archive ("does not support
# provisioning profiles"). Automatic signing assigns a profile only to the app and leaves
# the resource bundle (and the macOS-host macro plugins) alone, and bakes the sandbox
# entitlements in. No -allowProvisioningUpdates → it stays OFFLINE and never cloud-signs
# (the App-Manager ASC key can't), so the runner must have a macOS *development* profile
# for io.unom.punktfunk installed. DISTRIBUTION signing happens in the export step below
# (manual, via the plist). Quit Xcode so it can't prune the manually-installed App Store
# distribution profile that export needs.
osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true
pkill -x Xcode 2>/dev/null || true
PROFILE="Punktfunk macOS App Store Distribution"
@@ -218,11 +228,10 @@ jobs:
-project "$PROJECT" -scheme Punktfunk \
-destination 'generic/platform=macOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-macos-appstore.xcarchive" \
-skipMacroValidation -skipPackagePluginValidation \
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \
CODE_SIGN_STYLE=Manual \
CODE_SIGN_IDENTITY="Apple Distribution" \
DEVELOPMENT_TEAM="$TEAM_ID" \
PROVISIONING_PROFILE_SPECIFIER="$PROFILE"
CODE_SIGN_STYLE=Automatic \
DEVELOPMENT_TEAM="$TEAM_ID"
cat > "$RUNNER_TEMP/export-macos-appstore.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
@@ -252,35 +261,27 @@ jobs:
# Best-effort until the App Store Connect app record for io.unom.punktfunk exists.
continue-on-error: true
run: |
# MANUAL App Store signing: the local (valid) Apple Distribution identity + the App
# Store provisioning profile. NOT -allowProvisioningUpdates — with an App-Manager-role
# ASC key that forces Xcode's CLOUD-managed signing, which the role can't do ("Cloud
# signing permission error"). The profile must be installed on the runner under
# ~/Library/Developer/Xcode/UserData/Provisioning Profiles/ (install it once with
# Xcode.app quit, or it prunes the manually-dropped distribution profile).
# A running Xcode.app prunes unrecognized profiles from that dir — quit it so the App
# Store profile survives this build; headless xcodebuild doesn't need the GUI app.
# Archive with AUTOMATIC signing (development) — see the macOS App Store step for the full
# rationale. The SwiftPM resource bundle (PunktfunkKit_PunktfunkKit, added with the in-app
# license screens) builds for iphoneos, so even the sdk-scoped PROVISIONING_PROFILE_SPECIFIER
# this step used to set matched it and failed the archive ("does not support provisioning
# profiles"). Automatic signing profiles only the app and leaves the resource bundle (and
# the macOS-host macro plugins) alone. No -allowProvisioningUpdates → OFFLINE, never
# cloud-signs (the App-Manager ASC key can't), so the runner needs an iOS *development*
# profile for io.unom.punktfunk installed. DISTRIBUTION signing is the export step below
# (manual, via the plist). A running Xcode.app prunes unrecognized profiles — quit it so the
# manually-installed App Store distribution profile survives for export.
osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true
pkill -x Xcode 2>/dev/null || true
PROFILE="Punktfunk iOS App Store Distribution"
# Scope signing to the iOS device SDK via an xcconfig — see the tvOS step below for the
# full rationale. A global (CLI) profile specifier would also be forced onto the shared
# macOS-host SwiftPM macro plugins, which reject it and fail the archive; [sdk=iphoneos*]
# in an xcconfig lands it on the app/framework slices only.
SIGN_XCCONFIG="$RUNNER_TEMP/sign-ios.xcconfig"
cat > "$SIGN_XCCONFIG" <<XCCONF
CODE_SIGN_STYLE = Manual
DEVELOPMENT_TEAM = $TEAM_ID
CODE_SIGN_IDENTITY[sdk=iphoneos*] = Apple Distribution
PROVISIONING_PROFILE_SPECIFIER[sdk=iphoneos*] = $PROFILE
XCCONF
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild archive \
-project "$PROJECT" -scheme Punktfunk-iOS \
-destination 'generic/platform=iOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-ios.xcarchive" \
-skipMacroValidation -skipPackagePluginValidation \
-xcconfig "$SIGN_XCCONFIG" \
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM"
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \
CODE_SIGN_STYLE=Automatic \
DEVELOPMENT_TEAM="$TEAM_ID"
cat > "$RUNNER_TEMP/export-appstore.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
@@ -312,33 +313,24 @@ jobs:
# on the runner (xcodebuild -downloadPlatform tvOS).
continue-on-error: true
run: |
# Same manual App Store signing as iOS (the App-Manager ASC key can't cloud-sign).
# Archive with AUTOMATIC signing (development) — see the macOS App Store step. The SwiftPM
# resource bundle (PunktfunkKit_PunktfunkKit) builds for appletvos and rejected the
# sdk-scoped profile this step used to set; Automatic signing profiles only the app and
# leaves the resource bundle + the macOS-host macro plugins (OnceMacro/SwizzlingMacro/
# AssociationMacro) alone. No -allowProvisioningUpdates → OFFLINE, never cloud-signs (the
# App-Manager ASC key can't), so the runner needs a tvOS *development* profile for
# io.unom.punktfunk installed. DISTRIBUTION signing is the export step below (manual, plist).
osascript -e 'tell application "Xcode" to quit' >/dev/null 2>&1 || true
pkill -x Xcode 2>/dev/null || true
PROFILE="Punktfunk tvOS App Store Distribution"
# Scope signing to the tvOS device SDK via an xcconfig. A global (CLI) profile specifier
# hits EVERY target, including the shared SwiftPM macro plugins (OnceMacro/SwizzlingMacro/
# AssociationMacro) which build for the macOS host and reject a provisioning profile
# ("<macro> does not support provisioning profiles"), failing the archive. Conditionals
# work only in an xcconfig (xcodebuild mis-parses a CLI "SETTING[sdk=..]=val"), and a
# command-line -xcconfig outranks target settings, so [sdk=appletvos*] puts the profile on
# the app/framework slices only — the macosx-host macros get nothing. (The macOS archive
# above is immune: its host-SDK macros are CODE_SIGNING_ALLOWED=NO, so a global specifier
# is ignored there.)
SIGN_XCCONFIG="$RUNNER_TEMP/sign-tvos.xcconfig"
cat > "$SIGN_XCCONFIG" <<XCCONF
CODE_SIGN_STYLE = Manual
DEVELOPMENT_TEAM = $TEAM_ID
CODE_SIGN_IDENTITY[sdk=appletvos*] = Apple Distribution
PROVISIONING_PROFILE_SPECIFIER[sdk=appletvos*] = $PROFILE
XCCONF
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild archive \
-project "$PROJECT" -scheme Punktfunk-tvOS \
-destination 'generic/platform=tvOS' \
-archivePath "$RUNNER_TEMP/Punktfunk-tvos.xcarchive" \
-skipMacroValidation -skipPackagePluginValidation \
-xcconfig "$SIGN_XCCONFIG" \
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM"
MARKETING_VERSION="$VERSION" CURRENT_PROJECT_VERSION="$BUILD_NUM" \
CODE_SIGN_STYLE=Automatic \
DEVELOPMENT_TEAM="$TEAM_ID"
cat > "$RUNNER_TEMP/export-tvos.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+3 -2
View File
@@ -24,8 +24,9 @@
# GPU backends: the host builds with --features nvenc,amf-qsv = all three vendors in one installer.
# - NVENC (NVIDIA, direct SDK): the only link need is nvencodeapi.lib, synthesised from a 2-export
# .def with llvm-dlltool (no GPU/SDK at build time).
# - AMF/QSV (AMD/Intel, libavcodec): link-imports the FFmpeg libs from FFMPEG_DIR (the BtbN gpl-shared
# - AMF/QSV (AMD/Intel, libavcodec): link-imports the FFmpeg libs from FFMPEG_DIR (the BtbN lgpl-shared
# tree the client uses; includes the *_amf/*_qsv encoders) and bundles its DLLs into the installer.
# lgpl-shared (not gpl-shared) keeps those bundled DLLs LGPL (we never use the GPL-only x264/x265).
# CI never launches the exe, so no GPU is needed here — this is build + Windows clippy coverage only.
name: windows-host
@@ -80,7 +81,7 @@ jobs:
# (pwsh Out-File utf8 = no BOM, unlike Windows PowerShell 5.1 — keeps the first line clean).
"CARGO_TARGET_DIR=C:\t" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"CARGO_WORKSPACE_DIR=$env:GITHUB_WORKSPACE" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# FFMPEG_DIR: the same BtbN gpl-shared x64 tree the Windows CLIENT links against (provisioned
# FFMPEG_DIR: the same BtbN lgpl-shared x64 tree the Windows CLIENT links against (provisioned
# by scripts/ci/setup-windows-runner.ps1). The host's AMD/Intel AMF/QSV encode backend
# (--features amf-qsv) link-imports avcodec/avutil/swscale from it; pack-host-installer.ps1
# then bundles its bin\*.dll into the installer. LIBCLANG_PATH is in the runner daemon env.
+19 -5
View File
@@ -144,11 +144,25 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
`test-loopback.sh` (Swift client vs synthetic punktfunk1-hosts on loopback — runs on macOS;
includes the pairing ceremony + `--require-pairing` gate),
`RemoteFirstLightTests` (full pipeline over the LAN). See
[`clients/apple/README.md`](clients/apple/README.md). **Stage 2 presenter**
(`VTDecompressionSession` + `CAMetalLayer`) is built and live-validated on glass behind the opt-in
`punktfunk.presenter` flag (~11 ms p50 capture→present), to become the default after a few
resolution/HDR checks. Next: make stage 2 the default, glass-to-glass numbers via
`tools/latency-probe`, iOS/iPadOS/tvOS variants.
[`clients/apple/README.md`](clients/apple/README.md). **Stage 2 presenter is now the DEFAULT**
(stage-1 is the Metal-unavailable / DEBUG fallback): explicit `VTDecompressionSession` decode →
`CAMetalLayer`, presented from the hosting view's **main-runloop `CADisplayLink`** (`renderTick` pops
the newest ready frame per vsync; macOS `displaySyncEnabled = false` is the real fullscreen-judder fix,
~11 ms p50). *(An off-main `CAMetalDisplayLink` and an off-main blocking-render present thread were
both tried and reverted — both measured slower on macOS and iPad.)* **HDR fixed**
(`design/apple-stage2-presenter.md`): the "too bright" bug was a missing reference-white anchor — the
fix keeps the PQ-passthrough shader and adds `CAEDRMetadata.hdr10(…, opticalOutputScale: 203)` +
`wantsExtendedDynamicRangeContent` on the layer (on all platforms; the old `#if os(macOS)` guard left
iOS/tvOS EDR half-engaged), routing the 0xCE mastering metadata to the layer (via `setHdrMeta`) instead
of a never-composited source buffer. **Mid-session SDR↔HDR** is handled: `render` reconciles the layer
per-frame from the decoded `frame.isHDR` (per-mode pixel format `bgra8`/`rgba16Float`), so a game
entering HDR mid-stream just reconfigures (last 0xCE grade cached + re-applied; pump drains 0xCE
unconditionally). **4:4:4 added**: decode format is a 2×2 `(chroma, HDR)` matrix
(`420v/x420/444v/x444`, all biplanar so the shaders are unchanged), advertised (`VIDEO_CAP_444`) only
behind a **hardware-required `VTDecompressionSession` probe** (`Stage444Probe`, validated on M3) with a
Settings opt-out + a bounded pump backstop for an undecodable 4:4:4 session. *HDR brightness + 4:4:4
still need on-glass validation (Windows-HDR / `PUNKTFUNK_444` host).* Next: glass-to-glass numbers via
`tools/latency-probe`.
**Linux stage 1 done, first light 2026-06-12** (`clients/linux`, binary
`punktfunk-client`): GTK4/libadwaita shell linking `punktfunk-core` directly (no C ABI;
`NativeClient` is now `Sync` — mutexed plane receivers), mDNS host list, TOFU + SPAKE2
+43
View File
@@ -0,0 +1,43 @@
# Contributing to punktfunk
Thanks for your interest in contributing!
## Licensing of contributions (inbound = outbound)
punktfunk is dual-licensed under **MIT OR Apache-2.0**.
> Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in
> the work by you, as defined in the Apache-2.0 license, shall be dual licensed as **MIT OR
> Apache-2.0**, without any additional terms or conditions.
By opening a pull request you agree to license your contribution under these terms. This is the
standard Rust-ecosystem "inbound = outbound" model; it keeps the project's licensing unambiguous
(including the Apache-2.0 §5 contributor patent grant) and any future relicensing clean. You retain
the copyright to your contributions.
### Do not paste copyleft (or otherwise incompatibly-licensed) code
The single thing that could poison the permissive license is **copied source from a copyleft
project**. Several adjacent projects (Sunshine, Apollo, Moonlight) are GPL-3.0. You may study them
and reimplement a *technique*, protocol, or wire format — those are not copyrightable — but **never
paste their code**, and do not translate a GPL implementation line-by-line. When a comment credits
prior art, make clear it is an independent reimplementation, not a copy. The same applies to any
third party's code under a license incompatible with MIT/Apache.
If you add a new third-party dependency, it must be permissive (MIT / Apache-2.0 / BSD / ISC / Zlib /
Unicode-3.0 / etc.). `about.toml` holds the accepted-license allow-list; regenerate the attribution
file with `scripts/gen-third-party-notices.sh` when the dependency tree changes.
## Before you push
```sh
cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
```
Generated artifacts are checked in and CI fails on drift: `include/punktfunk_core.h` (cbindgen) and
`api/openapi.json` (`cargo run -p punktfunk-host -- openapi`). Match the surrounding code's comment
density and naming. Commit messages end with the `Co-Authored-By` trailer (see `git log`).
See [`CLAUDE.md`](CLAUDE.md) for the full build/test/run guide and design invariants.
Generated
+113 -293
View File
@@ -137,18 +137,9 @@ dependencies = [
[[package]]
name = "anyhow"
version = "1.0.102"
version = "1.0.103"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "arc-swap"
version = "1.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a3a1fd6f75306b68087b831f025c712524bcb19aad54e557b1129cfa0a2b207"
dependencies = [
"rustversion",
]
checksum = "2a4385e2e34eb35d6b3efe798b9eb88096925d87726c0798709bf56d9ed84af3"
[[package]]
name = "ash"
@@ -161,13 +152,13 @@ dependencies = [
[[package]]
name = "ashpd"
version = "0.13.11"
version = "0.13.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "340e0f6bf7f9ee78549c61454f1460a3ed97c011902ee76b58301bbc6d502a32"
checksum = "281e6645758940dee594495e28807a7672ce40f11ebf4df6c22c4fcd59e2689f"
dependencies = [
"enumflags2",
"futures-util",
"getrandom 0.4.2",
"getrandom 0.4.3",
"serde",
"serde_repr",
"tokio",
@@ -358,23 +349,18 @@ dependencies = [
[[package]]
name = "axum-server"
version = "0.7.3"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c1ab4a3ec9ea8a657c72d99a03a824af695bd0fb5ec639ccbd9cd3543b41a5f9"
checksum = "b1df331683d982a0b9492b38127151e6453639cd34926eb9c07d4cd8c6d22bfc"
dependencies = [
"arc-swap",
"bytes",
"either",
"fs-err",
"http",
"http-body",
"hyper",
"hyper-util",
"pin-project-lite",
"rustls",
"rustls-pemfile",
"rustls-pki-types",
"tokio",
"tokio-rustls",
"tower-service",
]
@@ -476,9 +462,9 @@ dependencies = [
[[package]]
name = "bytes"
version = "1.11.1"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33"
checksum = "8ae3f5d315924270530207e2a68396c3cc547f6dca3fbdca317cfb1a51edb593"
[[package]]
name = "cairo-rs"
@@ -520,9 +506,9 @@ dependencies = [
[[package]]
name = "cbindgen"
version = "0.29.3"
version = "0.29.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c95537b45400390270fae69ac098d057c8f5399001cde9d04f700c105ddfff2d"
checksum = "2ecb53484c9c167ba674026b656d8a27d7657a58e6066aa902bfb1a4aa00ae20"
dependencies = [
"clap",
"heck",
@@ -539,9 +525,9 @@ dependencies = [
[[package]]
name = "cc"
version = "1.2.63"
version = "1.2.65"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "556e016178bb5662a08681bbe0f00f8e17631781a4dfc8c45e466e4b185ec27f"
checksum = "e228eec9be7c17ccb640b59b36a5cd805ea2a564a4c5e162c2f659fea30d3b96"
dependencies = [
"find-msvc-tools",
"jobserver",
@@ -906,9 +892,6 @@ name = "deranged"
version = "0.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7cd812cc2bc1d69d4764bd80df88b4317eaef9e773c75226407d9bc0876b211c"
dependencies = [
"powerfmt",
]
[[package]]
name = "digest"
@@ -1127,9 +1110,9 @@ dependencies = [
[[package]]
name = "flume"
version = "0.11.1"
version = "0.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da0e4dd2a88388a1f4ccc7c9ce104604dab68d9f408dc34cd45823d5a9069095"
checksum = "5e139bc46ca777eb5efaf62df0ab8cc5fd400866427e56c68b22e414e53bd3be"
dependencies = [
"futures-core",
"futures-sink",
@@ -1142,12 +1125,6 @@ version = "1.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
[[package]]
name = "foldhash"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "foldhash"
version = "0.2.0"
@@ -1376,15 +1353,13 @@ dependencies = [
[[package]]
name = "getrandom"
version = "0.4.2"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555"
checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099"
dependencies = [
"cfg-if",
"libc",
"r-efi 6.0.0",
"wasip2",
"wasip3",
]
[[package]]
@@ -1595,9 +1570,9 @@ dependencies = [
[[package]]
name = "h2"
version = "0.4.14"
version = "0.4.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "171fefbc92fe4a4de27e0698d6a5b392d6a0e333506bc49133760b3bcf948733"
checksum = "6cb093c84e8bd9b188d4c4a8cb6579fc016968d14c99882163cd3ff402a4f155"
dependencies = [
"atomic-waker",
"bytes",
@@ -1623,22 +1598,13 @@ dependencies = [
"zerocopy",
]
[[package]]
name = "hashbrown"
version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"foldhash 0.1.5",
]
[[package]]
name = "hashbrown"
version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
dependencies = [
"foldhash 0.2.0",
"foldhash",
]
[[package]]
@@ -1647,7 +1613,7 @@ version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
dependencies = [
"foldhash 0.2.0",
"foldhash",
]
[[package]]
@@ -1858,12 +1824,6 @@ dependencies = [
"zerovec",
]
[[package]]
name = "id-arena"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "idna"
version = "1.1.0"
@@ -2014,9 +1974,9 @@ dependencies = [
[[package]]
name = "js-sys"
version = "0.3.100"
version = "0.3.103"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2025f20d7a4fa7785846e7b63d10a76d3f1cee98ee5cb79ea59703f95e42162"
checksum = "53b44bfcdb3f8d5837a46dae1ca9660a837176eee74a28b229bc626816589102"
dependencies = [
"cfg-if",
"futures-util",
@@ -2035,7 +1995,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.0.1"
version = "0.3.0"
[[package]]
name = "lazy_static"
@@ -2046,12 +2006,6 @@ dependencies = [
"spin",
]
[[package]]
name = "leb128fmt"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
[[package]]
name = "libadwaita"
version = "0.9.1"
@@ -2167,13 +2121,13 @@ dependencies = [
[[package]]
name = "log"
version = "0.4.32"
version = "0.4.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "953f07c43838f8e6f9758cab68bf5bed85465e7587ebe0b823f1bcd81978ad3a"
checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.0.1"
version = "0.3.0"
dependencies = [
"punktfunk-core",
]
@@ -2201,9 +2155,9 @@ checksum = "47e1ffaa40ddd1f3ed91f717a33c8c0ee23fff369e3aa8772b9605cc1d22f4c3"
[[package]]
name = "mdns-sd"
version = "0.20.0"
version = "0.20.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "892f96f6d2ebe1ea641279f986ac52a2a6bac71e8f743bb258315cfe2bd7e88e"
checksum = "fb75febbe5fa1837a52fdbd1c735e168286c5c645fc2ddd31526f65c49941c2e"
dependencies = [
"fastrand",
"flume",
@@ -2216,15 +2170,15 @@ dependencies = [
[[package]]
name = "memchr"
version = "2.8.1"
version = "2.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b947ae49db0d222b1dbc6b113ce7248a3fc3a6ca21b696717bfc000ba4484d8"
checksum = "88904434abc2901f197fe8cc55f0445e7ded921dba5911dad2e2b39b48e663c4"
[[package]]
name = "memmap2"
version = "0.9.10"
version = "0.9.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "714098028fe011992e1c3962653c96b2d578c4b4bce9036e15ff220319b1e0e3"
checksum = "d1219ed1b7f229ee7104d281dd01d6802fe28bb6e95d292942c4daacdeb798c0"
dependencies = [
"libc",
]
@@ -2377,6 +2331,17 @@ version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441"
[[package]]
name = "num-derive"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed3955f1a9c7c0c15e092f9c887db08b1fc683305fdf6eb6684f22555355e202"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "num-integer"
version = "0.1.46"
@@ -2716,16 +2681,6 @@ dependencies = [
"zerocopy",
]
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn",
]
[[package]]
name = "proc-macro-crate"
version = "3.5.0"
@@ -2765,7 +2720,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.0.1"
version = "0.3.0"
dependencies = [
"android_logger",
"jni",
@@ -2779,7 +2734,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.0.1"
version = "0.3.0"
dependencies = [
"anyhow",
"async-channel",
@@ -2799,7 +2754,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.0.1"
version = "0.3.0"
dependencies = [
"anyhow",
"async-channel",
@@ -2819,7 +2774,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.0.1"
version = "0.3.0"
dependencies = [
"aes-gcm",
"bytes",
@@ -2849,7 +2804,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.0.1"
version = "0.3.0"
dependencies = [
"aes",
"aes-gcm",
@@ -2885,7 +2840,6 @@ dependencies = [
"rsa",
"rusqlite",
"rustls",
"rustls-pemfile",
"rusty_enet",
"serde",
"serde_json",
@@ -2896,12 +2850,14 @@ dependencies = [
"tracing",
"tracing-subscriber",
"ureq",
"usbip-sim",
"utoipa",
"utoipa-axum",
"utoipa-scalar",
"wasapi",
"wayland-backend",
"wayland-client",
"wayland-protocols",
"wayland-protocols-misc",
"wayland-protocols-wlr",
"wayland-scanner",
@@ -2914,7 +2870,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.0.1"
version = "0.3.0"
dependencies = [
"anyhow",
"mdns-sd",
@@ -2943,9 +2899,9 @@ dependencies = [
[[package]]
name = "quinn"
version = "0.11.9"
version = "0.11.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9e20a958963c291dc322d98411f541009df2ced7b5a4f2bd52337638cfccf20"
checksum = "0c1a41e437b6bbd489372cd4971de128e85c855f56c57f283d20ff016cf7c0a8"
dependencies = [
"bytes",
"cfg_aliases",
@@ -2963,9 +2919,9 @@ dependencies = [
[[package]]
name = "quinn-proto"
version = "0.11.14"
version = "0.11.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "434b42fec591c96ef50e21e886936e66d3cc3f737104fdb9b737c40ffb94c098"
checksum = "4fcb935c5bec503c2f0e306bdd3e58bb9029dcb14fa8d9ac76e3a5256ac0763e"
dependencies = [
"bytes",
"fastbloom",
@@ -3000,9 +2956,9 @@ dependencies = [
[[package]]
name = "quote"
version = "1.0.45"
version = "1.0.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
checksum = "dfbc457d0c7a0759a614551b11a6409e5951f6c7537be1f1b7682b9ae9230368"
dependencies = [
"proc-macro2",
]
@@ -3156,9 +3112,9 @@ dependencies = [
[[package]]
name = "regex"
version = "1.12.3"
version = "1.12.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e10754a14b9137dd7b1e3e5b0493cc9171fdd105e0ab477f51b72e7f3ac0e276"
checksum = "f1292b7759ae1cb9ec195452d1390a074f0cd8541ab7a5a8c31cd6db45d4a6ba"
dependencies = [
"aho-corasick",
"memchr",
@@ -3179,9 +3135,9 @@ dependencies = [
[[package]]
name = "regex-syntax"
version = "0.8.10"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a"
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]]
name = "reis"
@@ -3309,9 +3265,9 @@ dependencies = [
[[package]]
name = "rustls"
version = "0.23.40"
version = "0.23.41"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ef86cd5876211988985292b91c96a8f2d298df24e75989a43a3c73f2d4d8168b"
checksum = "6b92b125634d9b795e7beca796cc790df15a7fb38323bf3196fda83292d06b1f"
dependencies = [
"aws-lc-rs",
"log",
@@ -3335,15 +3291,6 @@ dependencies = [
"security-framework",
]
[[package]]
name = "rustls-pemfile"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dce314e5fee3f39953d46bb63bb8a46d40c2f8fb7cc5a3b6cab2bde9721d6e50"
dependencies = [
"rustls-pki-types",
]
[[package]]
name = "rustls-pki-types"
version = "1.14.1"
@@ -3740,19 +3687,19 @@ checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "smallvec"
version = "1.15.1"
version = "1.15.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03"
checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90"
[[package]]
name = "socket-pktinfo"
version = "0.3.2"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "927136cc2ae6a1b0e66ac6b1210902b75c3f726db004a73bc18686dcd0dcd22f"
checksum = "3e8e43b4bdce7cff8a4d3f8025ee38fce5ca138fab868ebbf9529c81328fbf9d"
dependencies = [
"libc",
"socket2",
"windows-sys 0.60.2",
"windows-sys 0.61.2",
]
[[package]]
@@ -3828,9 +3775,9 @@ checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "syn"
version = "2.0.117"
version = "2.0.118"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
checksum = "1b9ae57f904213ebb649ce6895b8a66c66f0203b9319718f69a5612a065b1422"
dependencies = [
"proc-macro2",
"quote",
@@ -3880,7 +3827,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.4.2",
"getrandom 0.4.3",
"once_cell",
"rustix",
"windows-sys 0.61.2",
@@ -3937,12 +3884,11 @@ dependencies = [
[[package]]
name = "time"
version = "0.3.47"
version = "0.3.51"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "743bd48c283afc0388f9b8827b976905fb217ad9e647fae3a379a9283c4def2c"
checksum = "85c17d80feb7334b40c484e45ed1a5273dfd8bfda537c3be2e74a06a6686f327"
dependencies = [
"deranged",
"itoa",
"num-conv",
"powerfmt",
"serde_core",
@@ -3952,15 +3898,15 @@ dependencies = [
[[package]]
name = "time-core"
version = "0.1.8"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7694e1cfe791f8d31026952abf09c69ca6f6fa4e1a1229e18988f06a04a12dca"
checksum = "9e1c906769ad99c88eaa54e728060edef082f8e358ff32030cb7c7d315e81109"
[[package]]
name = "time-macros"
version = "0.2.27"
version = "0.2.30"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2e70e4c5a0e0a8a4823ad65dfe1a6930e4f4d756dcd9dd7939022b5e8c501215"
checksum = "dcef1a61bdb119096e153208ec5cbec23944ce8bca13be5c7f60c634f7403935"
dependencies = [
"num-conv",
"time-core",
@@ -4259,12 +4205,6 @@ version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254"
[[package]]
name = "unicode-xid"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
[[package]]
name = "universal-hash"
version = "0.5.1"
@@ -4309,6 +4249,17 @@ dependencies = [
"serde",
]
[[package]]
name = "usbip-sim"
version = "0.8.0"
dependencies = [
"log",
"num-derive",
"num-traits",
"serde",
"tokio",
]
[[package]]
name = "utf8_iter"
version = "1.0.4"
@@ -4372,9 +4323,9 @@ dependencies = [
[[package]]
name = "uuid"
version = "1.23.2"
version = "1.23.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d258b83ceec21034727ecee8c382cfa6c3e133699b0742c64571814fb420c9f7"
checksum = "bf80a72845275afea99e7f2b434723d3bc7e38470fcd1c7ed39a599c73319a53"
dependencies = [
"js-sys",
"serde_core",
@@ -4445,27 +4396,18 @@ checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
[[package]]
name = "wasip2"
version = "1.0.3+wasi-0.2.9"
version = "1.0.4+wasi-0.2.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20064672db26d7cdc89c7798c48a0fdfac8213434a1186e5ef29fd560ae223d6"
checksum = "b67efb37e106e55ce722a510d6b5f9c17f083e5fc79afc2badeb12cc313d9487"
dependencies = [
"wit-bindgen 0.57.1",
]
[[package]]
name = "wasip3"
version = "0.4.0+wasi-0.3.0-rc-2026-01-06"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5428f8bf88ea5ddc08faddef2ac4a67e390b88186c703ce6dbd955e1c145aca5"
dependencies = [
"wit-bindgen 0.51.0",
"wit-bindgen",
]
[[package]]
name = "wasm-bindgen"
version = "0.2.123"
version = "0.2.126"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a254a4b10c19a76f09a27640e7ffbf9bc30bf67e16a3bf28aaefa4920fe81563"
checksum = "4b067c0c11094aef6b7a801c1e34a26affafdf3d051dba08456b868789aaf9a4"
dependencies = [
"cfg-if",
"once_cell",
@@ -4476,9 +4418,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.123"
version = "0.2.126"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "24a40fc75b0ec6f3746ceb10d36f53a93dcd68a93b11b6445983945d79eba0dc"
checksum = "167ce5e579f6bcf889c4f7175a8a5a585de84e8ff93976ce393efa5f2837aab1"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
@@ -4486,9 +4428,9 @@ dependencies = [
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.123"
version = "0.2.126"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "908f34bd9b9ce3d4caf07b72dfab63d61504d156856c6bd3cd87fa350cf3985b"
checksum = "f3997c7839262f4ef12cf90b818d6340c18e80f263f1a94bf157d0ec4420380e"
dependencies = [
"bumpalo",
"proc-macro2",
@@ -4499,47 +4441,13 @@ dependencies = [
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.123"
version = "0.2.126"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7acbf7616c27b194bbb550bf77ed0c2c3e5b7fd1260a93082b95fb7f47959b92"
checksum = "dc1b4cb0cc549fcf58d7dfc081778139b3d283a081644e833e84682ad71cea24"
dependencies = [
"unicode-ident",
]
[[package]]
name = "wasm-encoder"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "990065f2fe63003fe337b932cfb5e3b80e0b4d0f5ff650e6985b1048f62c8319"
dependencies = [
"leb128fmt",
"wasmparser",
]
[[package]]
name = "wasm-metadata"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb0e353e6a2fbdc176932bbaab493762eb1255a7900fe0fea1a2f96c296cc909"
dependencies = [
"anyhow",
"indexmap",
"wasm-encoder",
"wasmparser",
]
[[package]]
name = "wasmparser"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "47b807c72e1bac69382b3a6fb3dbe8ea4c0ed87ff5629b8685ae6b9a611028fe"
dependencies = [
"bitflags",
"hashbrown 0.15.5",
"indexmap",
"semver",
]
[[package]]
name = "wayland-backend"
version = "0.3.15"
@@ -4567,9 +4475,9 @@ dependencies = [
[[package]]
name = "wayland-protocols"
version = "0.32.12"
version = "0.32.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "563a85523cade2429938e790815fd7319062103b9f4a2dc806e9b53b95982d8f"
checksum = "23d0c813de3daa2ed6520af85a3bd49b0e722a3078506899aa9686fea58dc4b6"
dependencies = [
"bitflags",
"wayland-backend",
@@ -4635,9 +4543,9 @@ dependencies = [
[[package]]
name = "webpki-root-certs"
version = "1.0.7"
version = "1.0.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f31141ce3fc3e300ae89b78c0dd67f9708061d1d2eda54b8209346fd6be9a92c"
checksum = "0d46a5a140e6f7afeccd8eae97eff335163939eac8b929834875168b29b3d267"
dependencies = [
"rustls-pki-types",
]
@@ -5195,100 +5103,12 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "wit-bindgen"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7249219f66ced02969388cf2bb044a09756a083d0fab1e566056b04d9fbcaa5"
dependencies = [
"wit-bindgen-rust-macro",
]
[[package]]
name = "wit-bindgen"
version = "0.57.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e"
[[package]]
name = "wit-bindgen-core"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea61de684c3ea68cb082b7a88508a8b27fcc8b797d738bfc99a82facf1d752dc"
dependencies = [
"anyhow",
"heck",
"wit-parser",
]
[[package]]
name = "wit-bindgen-rust"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b7c566e0f4b284dd6561c786d9cb0142da491f46a9fbed79ea69cdad5db17f21"
dependencies = [
"anyhow",
"heck",
"indexmap",
"prettyplease",
"syn",
"wasm-metadata",
"wit-bindgen-core",
"wit-component",
]
[[package]]
name = "wit-bindgen-rust-macro"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c0f9bfd77e6a48eccf51359e3ae77140a7f50b1e2ebfe62422d8afdaffab17a"
dependencies = [
"anyhow",
"prettyplease",
"proc-macro2",
"quote",
"syn",
"wit-bindgen-core",
"wit-bindgen-rust",
]
[[package]]
name = "wit-component"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d66ea20e9553b30172b5e831994e35fbde2d165325bec84fc43dbf6f4eb9cb2"
dependencies = [
"anyhow",
"bitflags",
"indexmap",
"log",
"serde",
"serde_derive",
"serde_json",
"wasm-encoder",
"wasm-metadata",
"wasmparser",
"wit-parser",
]
[[package]]
name = "wit-parser"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ecc8ac4bc1dc3381b7f59c34f00b67e18f910c2c0f50015669dde7def656a736"
dependencies = [
"anyhow",
"id-arena",
"indexmap",
"log",
"semver",
"serde",
"serde_derive",
"serde_json",
"unicode-xid",
"wasmparser",
]
[[package]]
name = "writeable"
version = "0.6.3"
@@ -5419,18 +5239,18 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.50"
version = "0.8.52"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3b065d4f0e55f82fae73202e189638116a87c55ab6b8e6c2721e13dd9d854ad1"
checksum = "ce1022995ff5ff5d841ad7d994facc23098cd40152f2c1d11cd607c6f530653f"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.50"
version = "0.8.52"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b631b19d36a892ab55420c92dbc83ccd79274f25be714855d3074aa71cab639"
checksum = "1ae7f38b72ec2a254e2b87ef277cf2cd4fb97cbebf944faa6f33354da0867930"
dependencies = [
"proc-macro2",
"quote",
@@ -5460,9 +5280,9 @@ dependencies = [
[[package]]
name = "zeroize"
version = "1.8.2"
version = "1.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0"
checksum = "e13c156562582aa81c60cb29407084cdb54c4164760106ab78e6c5b0858cf64e"
[[package]]
name = "zerotrie"
+4 -1
View File
@@ -3,6 +3,7 @@ resolver = "2"
members = [
"crates/punktfunk-core",
"crates/punktfunk-host",
"crates/punktfunk-host/vendor/usbip-sim",
"crates/pf-driver-proto",
"clients/probe",
"clients/linux",
@@ -11,9 +12,11 @@ members = [
"tools/latency-probe",
"tools/loss-harness",
]
# Standalone PoC (built on its own; pulls usbip/tokio/libusb we don't want in the workspace).
exclude = ["packaging/linux/steam-deck-gadget/usbip-poc"]
[workspace.package]
version = "0.0.1"
version = "0.3.0"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+28 -1
View File
@@ -155,4 +155,31 @@ tools/ latency-probe · loss-harness (measurement)
## License
MIT OR Apache-2.0.
Licensed under either of
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or
<https://www.apache.org/licenses/LICENSE-2.0>)
- MIT license ([LICENSE-MIT](LICENSE-MIT) or <https://opensource.org/licenses/MIT>)
at your option — `SPDX-License-Identifier: MIT OR Apache-2.0`.
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in
the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any
additional terms or conditions. See [CONTRIBUTING.md](CONTRIBUTING.md).
### Third-party components
punktfunk's own source is MIT/Apache-2.0. Shipped binaries additionally link third-party components
under their own (permissive) licenses — see [`THIRD-PARTY-NOTICES.txt`](THIRD-PARTY-NOTICES.txt)
(regenerate with `scripts/gen-third-party-notices.sh`). The Windows host and client builds also
bundle FFmpeg under the **LGPL v2.1+** (dynamically linked, replaceable DLLs; the license text and
notice ship in the installed `licenses/` folder).
### Trademarks
punktfunk is an independent project and is **not affiliated with, endorsed by, or sponsored by**
NVIDIA, Microsoft, Sony, Valve, or the Moonlight project. "GameStream", "Moonlight", "Xbox",
"DualSense", "DualShock", and "PlayStation" are trademarks of their respective owners and are used
here only to describe interoperability.
File diff suppressed because it is too large Load Diff
+23
View File
@@ -0,0 +1,23 @@
THIRD-PARTY SOFTWARE NOTICES
============================================================================
punktfunk (https://git.unom.io/unom/punktfunk) is licensed under MIT OR Apache-2.0.
The binaries it ships statically/dynamically link the third-party Rust crates below.
Each is distributed under its own permissive license; full texts follow.
Generated by `cargo about generate about.hbs` (see about.toml) — do not edit by hand.
Overview:
{{#each overview}}
{{name}} ({{id}}): {{count}} crate(s)
{{/each}}
{{#each licenses}}
----------------------------------------------------------------------------
{{name}} ({{id}})
Used by:
{{#each used_by}} - {{crate.name}} {{crate.version}}{{#if crate.repository}} ({{crate.repository}}){{/if}}
{{/each}}
----------------------------------------------------------------------------
{{text}}
{{/each}}
+49
View File
@@ -0,0 +1,49 @@
# cargo-about config — full-fidelity third-party license harvest for CI.
#
# cargo install cargo-about
# cargo about generate about.hbs > THIRD-PARTY-NOTICES.txt # (or use scripts/gen-third-party-notices.sh)
#
# `accepted` is the allow-list of SPDX licenses permitted in the dependency tree. CI fails if a crate
# carries anything not listed here — which is exactly the regression guard we want against a copyleft
# dependency silently entering the linked set. All entries
# below are permissive / attribution-only; deliberately NO GPL/LGPL/AGPL/MPL-link/SSPL/EPL.
#
# The dependency-free fallback is scripts/gen-third-party-notices.py (reads the cargo registry cache),
# which is what produced the committed baseline when cargo-about is unavailable offline.
accepted = [
"MIT",
"MIT-0",
"Apache-2.0",
"Apache-2.0 WITH LLVM-exception",
"BSD-2-Clause",
"BSD-3-Clause",
"ISC",
"Zlib",
"0BSD",
"BSL-1.0",
"Unicode-3.0",
"Unicode-DFS-2016",
"CDLA-Permissive-2.0",
"CC0-1.0",
"Unlicense",
"WTFPL",
"OpenSSL",
]
# cbindgen is MPL-2.0 but it is a BUILD-ONLY codegen tool that never links into a shipped artifact
# (its generated header is not a derivative work), so it is excluded from the notices rather than
# accepted as a linked license.
ignore-build-dependencies = true
ignore-dev-dependencies = true
# r-efi offers an LGPL-2.1-or-later arm but is tri-licensed; take a permissive arm. (It is also
# UEFI-target-gated out of every shipped build.)
[r-efi.clarify]
license = "MIT OR Apache-2.0"
[ring.clarify]
license = "MIT AND ISC AND OpenSSL"
[aws-lc-sys.clarify]
license = "ISC AND Apache-2.0 AND MIT AND BSD-3-Clause AND OpenSSL"
+9 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0"
},
"version": "0.0.1"
"version": "0.3.0"
},
"paths": {
"/api/v1/clients": {
@@ -1354,6 +1354,14 @@
"type": "object",
"description": "Arm-native-pairing request body.",
"properties": {
"fingerprint": {
"type": [
"string",
"null"
],
"description": "Optional: bind the window to ONE device fingerprint (hex SHA-256, e.g. from a pending knock).\nWhen set, only a pairing attempt from that fingerprint consumes the window — so an unpaired\nLAN peer can neither pair nor burn a window armed for a specific device (security-review #9).\nOmit for an unbound window (any device may use the PIN — trusted-LAN only).",
"example": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
},
"ttl_secs": {
"type": [
"integer",
File diff suppressed because it is too large Load Diff
@@ -74,10 +74,31 @@ import io.unom.punktfunk.kit.security.KnownHostStore
import io.unom.punktfunk.kit.security.obtainIdentity
import io.unom.punktfunk.models.HostStatus
import io.unom.punktfunk.models.PendingTrust
import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
/** Handshake budget for a normal connect (the prior hardcoded value, now passed explicitly). */
private const val CONNECT_TIMEOUT_MS = 10_000
/**
* Handshake budget for the no-PIN "request access" connect. Must exceed the host's approval-park
* window (~180 s) so a slow operator approval still lands on this same parked connection rather than
* timing the client out first. Mirrors the Linux client's 185 s.
*/
private const val REQUEST_ACCESS_TIMEOUT_MS = 185_000
/**
* A no-PIN "request access" connect in flight — the host being requested (drives the cancelable
* "Waiting for approval…" dialog) and a per-attempt flag the Cancel button trips. The connect is a
* blocking call with no abort, so Cancel returns the UI immediately and a late result checks
* [cancelled] and tears the (possibly just-approved) session down silently rather than navigating.
*/
private class RequestAccessState(val target: PendingTrust) {
val cancelled = AtomicBoolean(false)
}
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
@@ -128,8 +149,11 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
.onSuccess { identity = it }
.onFailure { status = "Identity unavailable: ${it.message} — re-pair may be required" }
}
// A trust decision awaiting the user (first-connect TOFU / fp changed / PIN pairing).
// A trust decision awaiting the user (first-connect TOFU / fp changed / PIN pairing / the
// request-access-or-PIN choice).
var pendingTrust by remember { mutableStateOf<PendingTrust?>(null) }
// A no-PIN "request access" connect in flight (the cancelable "Waiting for approval…" dialog).
var awaiting by remember { mutableStateOf<RequestAccessState?>(null) }
// A saved host whose label is being edited (the Rename dialog).
var renameTarget by remember { mutableStateOf<KnownHost?>(null) }
@@ -151,9 +175,9 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
status = "Connecting to $targetHost:$targetPort"
discovery.stop() // free the Wi-Fi radio before the stream session
scope.launch {
// Advertise HDR only when this device's display can present it (else the host sends a
// proper SDR stream rather than PQ the panel would mis-tone-map).
val hdrEnabled = displaySupportsHdr(context)
// Advertise HDR only when the user enabled it AND this device's display can present it
// (else the host sends a proper SDR stream rather than PQ the panel would mis-tone-map).
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
// "Automatic" resolves to a concrete pad type from the connected controller's VID/PID
// (Android exposes no controller-type enum) — parity with the Linux/Apple clients. An
// explicit choice is passed through unchanged.
@@ -163,7 +187,7 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
targetHost, targetPort, w, h, hz,
id.certPem, id.privateKeyPem, pinHex ?: "",
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels,
hdrEnabled, settings.audioChannels, CONNECT_TIMEOUT_MS,
)
}
connecting = false
@@ -182,10 +206,66 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
}
}
// Decide pinned-reconnect vs fp-changed vs TOFU vs PIN pairing before connecting. Trust state is
// The no-PIN "request access" path (delegated approval): open a normal identified connect that
// the host PARKS until the operator clicks Approve in its console/web UI, showing a cancelable
// "Waiting for approval…" dialog meanwhile. The SAME connection is admitted on approval (no
// reconnect), so on success we record the host as PAIRED — the operator's approval IS the pairing.
// The connect can't be aborted, so Cancel returns the UI immediately and a late result is torn
// down silently via the per-attempt flag (mirrors the Linux client's request-access flow).
fun requestAccess(target: PendingTrust) {
val id = identity
if (id == null) {
status = "Identity not ready yet — try again in a moment"
return
}
val req = RequestAccessState(target)
awaiting = req
connecting = true
status = null
discovery.stop() // free the Wi-Fi radio before the (parked) stream session
scope.launch {
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
val gamepadPref = Gamepad.resolvePref(settings.gamepad)
// Pin the advertised fingerprint for a discovered host (defence against an impostor while
// we wait); a manually-typed host has none, so trust-on-first-use.
val pinHex = target.advertisedFp ?: ""
val handle = withContext(Dispatchers.IO) {
NativeBridge.nativeConnect(
target.host, target.port, w, h, hz,
id.certPem, id.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels, REQUEST_ACCESS_TIMEOUT_MS,
)
}
// Cancelled while we were parked: tear the (possibly just-approved) session down and
// don't touch UI a fresh action may now own.
if (req.cancelled.get()) {
if (handle != 0L) withContext(Dispatchers.IO) { NativeBridge.nativeClose(handle) }
return@launch
}
awaiting = null
connecting = false
if (handle != 0L) {
// Approved — save the host as PAIRED, pinning the fingerprint it presented, so
// future connects are silent (exactly like after a PIN ceremony).
val fp = NativeBridge.nativeHostFingerprint(handle)
if (fp.isNotEmpty()) {
knownHostStore.save(KnownHost(target.host, target.port, target.name, fp, paired = true))
savedHosts = knownHostStore.all()
}
onConnected(handle)
} else {
status = "Request timed out — approve this device in the host's console, then retry."
discovery.start()
}
}
}
// Decide pinned-reconnect vs fp-changed vs TOFU vs pairing before connecting. Trust state is
// keyed by address:port, so a discovered and a manually-typed connection to the same host share
// one record. Trust-on-first-use is permitted ONLY when the host advertised pair=optional; a
// pair=required host, or a manual/unknown-policy host, must pair by PIN.
// pair=required host, or a manual/unknown-policy host, must pair — either by no-PIN request
// access (approve in the console) or by the SPAKE2 PIN ceremony.
fun connect(
targetHost: String,
targetPort: Int,
@@ -208,9 +288,10 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
// clearly labeled, alongside PIN pairing). Smart-cast: this branch ⇒ dh != null.
dh?.pairingRequired == false -> pendingTrust =
PendingTrust(targetHost, targetPort, name, dh.fingerprint, PendingTrust.Kind.TRUST_NEW)
// pair=required, or a manual/unknown-policy host → PIN pairing is mandatory.
// pair=required, or a manual/unknown-policy host → offer the two ways in: a no-PIN
// "request access" (approve in the console) or the SPAKE2 PIN ceremony.
else -> pendingTrust =
PendingTrust(targetHost, targetPort, name, adv, PendingTrust.Kind.PAIR)
PendingTrust(targetHost, targetPort, name, adv, PendingTrust.Kind.REQUEST_ACCESS)
}
}
@@ -471,6 +552,33 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
TextButton({ pendingTrust = null }) { Text("Cancel") }
},
)
// A fresh pair=required (or manual/unknown-policy) host: offer the two ways in. "Request
// access" is the no-PIN path — connect and wait for the operator to click Approve in the
// host's console; "Use a PIN…" switches to the SPAKE2 ceremony.
PendingTrust.Kind.REQUEST_ACCESS -> AlertDialog(
onDismissRequest = { pendingTrust = null },
title = { Text("Pairing required") },
text = {
Column {
Text("${pt.host}:${pt.port} requires pairing before it will stream.")
Text(
"Request access and approve this device in the host's console (or web " +
"UI) — no PIN needed. Or pair with the 4-digit PIN the host displays.",
)
}
},
confirmButton = {
TextButton({ pendingTrust = null; requestAccess(pt) }) { Text("Request access") }
},
dismissButton = {
Row {
TextButton({ pendingTrust = pt.copy(kind = PendingTrust.Kind.PAIR) }) {
Text("Use a PIN…")
}
TextButton({ pendingTrust = null }) { Text("Cancel") }
}
},
)
PendingTrust.Kind.PAIR -> {
var pin by remember(pt) { mutableStateOf("") }
var name by remember(pt) { mutableStateOf(Build.MODEL ?: "Android") }
@@ -537,6 +645,44 @@ fun ConnectScreen(settings: Settings, onConnected: (Long) -> Unit) {
}
}
// The no-PIN "request access" wait: the connect is parked on the host until the operator
// approves this device. Cancel returns the UI immediately — it trips the per-attempt flag so a
// late approval is torn down silently (see requestAccess) and resumes discovery.
awaiting?.let { req ->
fun cancel() {
req.cancelled.set(true)
awaiting = null
connecting = false
discovery.start() // the request may still be pending on the host; keep scanning
}
AlertDialog(
onDismissRequest = { cancel() },
title = { Text("Waiting for approval") },
text = {
val deviceName = Build.MODEL ?: "this device"
Column(verticalArrangement = Arrangement.spacedBy(12.dp)) {
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
CircularProgressIndicator(modifier = Modifier.size(20.dp), strokeWidth = 2.dp)
Text("Approve this device on ${req.target.name}.")
}
Text(
"Open the host's console (or web UI) and approve “$deviceName”. It connects " +
"automatically once you approve — no PIN needed.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
},
confirmButton = {},
dismissButton = {
TextButton(onClick = { cancel() }) { Text("Cancel") }
},
)
}
// Rename a saved host's label (discovered hosts are named by mDNS; this is how you give one a
// friendly name like "Living Room" after pairing). Keyed by the host so reopening resets the field.
renameTarget?.let { kh ->
@@ -0,0 +1,66 @@
package io.unom.punktfunk
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.verticalScroll
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.remember
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.unit.dp
/**
* Open-source licenses: punktfunk's own license (MIT OR Apache-2.0) plus the third-party software
* notices, read from the bundled `THIRD-PARTY-NOTICES.txt` asset (generated by
* scripts/gen-third-party-notices.sh). Reached from [SettingsScreen]; Back returns there.
*/
@Composable
fun LicensesScreen(onBack: () -> Unit) {
val context = LocalContext.current
BackHandler(onBack = onBack)
val notices = remember {
runCatching {
context.assets.open("THIRD-PARTY-NOTICES.txt").bufferedReader().use { it.readText() }
}.getOrDefault("Third-party notices unavailable.")
}
val version = remember {
runCatching {
@Suppress("DEPRECATION")
context.packageManager.getPackageInfo(context.packageName, 0).versionName
}.getOrNull()
}
Column(
modifier = Modifier
.fillMaxSize()
.verticalScroll(rememberScrollState())
.padding(horizontal = 20.dp, vertical = 24.dp),
verticalArrangement = Arrangement.spacedBy(16.dp),
) {
Text("Open-source licenses", style = MaterialTheme.typography.headlineMedium)
if (version != null) {
Text(
"punktfunk $version",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
Text(
"punktfunk is licensed under MIT OR Apache-2.0, at your option. It uses the open-source " +
"components below, each under its own license.",
style = MaterialTheme.typography.bodyMedium,
)
Text(
notices,
style = MaterialTheme.typography.bodySmall.copy(fontFamily = FontFamily.Monospace),
)
}
}
@@ -14,6 +14,13 @@ data class Settings(
val height: Int = 0,
val hz: Int = 0,
val bitrateKbps: Int = 0,
/**
* Advertise HDR (10-bit BT.2020 PQ) to the host. Default on, but only *effective* on a panel that
* can actually present HDR10 (see [displaySupportsHdr]) — on an SDR display HDR is never
* advertised regardless, so the host sends a proper 8-bit BT.709 stream rather than PQ the panel
* would mis-tone-map. Turning this off forces SDR even on a capable panel.
*/
val hdrEnabled: Boolean = true,
val compositor: Int = 0,
val gamepad: Int = 0,
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
@@ -40,6 +47,7 @@ class SettingsStore(context: Context) {
height = prefs.getInt(K_H, 0),
hz = prefs.getInt(K_HZ, 0),
bitrateKbps = prefs.getInt(K_BITRATE, 0),
hdrEnabled = prefs.getBoolean(K_HDR, true),
compositor = prefs.getInt(K_COMPOSITOR, 0),
gamepad = prefs.getInt(K_GAMEPAD, 0),
audioChannels = prefs.getInt(K_AUDIO_CH, 2),
@@ -54,6 +62,7 @@ class SettingsStore(context: Context) {
.putInt(K_H, s.height)
.putInt(K_HZ, s.hz)
.putInt(K_BITRATE, s.bitrateKbps)
.putBoolean(K_HDR, s.hdrEnabled)
.putInt(K_COMPOSITOR, s.compositor)
.putInt(K_GAMEPAD, s.gamepad)
.putInt(K_AUDIO_CH, s.audioChannels)
@@ -68,6 +77,7 @@ class SettingsStore(context: Context) {
const val K_H = "height"
const val K_HZ = "hz"
const val K_BITRATE = "bitrate_kbps"
const val K_HDR = "hdr_enabled"
const val K_COMPOSITOR = "compositor"
const val K_GAMEPAD = "gamepad"
const val K_AUDIO_CH = "audio_channels"
@@ -5,6 +5,7 @@ import android.content.pm.PackageManager
import androidx.activity.compose.BackHandler
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.ColumnScope
@@ -44,6 +45,7 @@ import androidx.core.content.ContextCompat
fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -> Unit) {
var s by remember { mutableStateOf(initial) }
val context = LocalContext.current
var showLicenses by remember { mutableStateOf(false) }
fun update(next: Settings) {
s = next
onChange(next)
@@ -56,6 +58,11 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
ActivityResultContracts.RequestPermission(),
) { granted -> update(s.copy(micEnabled = granted)) }
if (showLicenses) {
LicensesScreen(onBack = { showLicenses = false })
return
}
Column(
modifier = Modifier
.fillMaxSize()
@@ -87,6 +94,22 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
options = BITRATE_OPTIONS,
selected = s.bitrateKbps,
) { kbps -> update(s.copy(bitrateKbps = kbps)) }
// HDR is only meaningful on a panel that can present HDR10; on an SDR display the toggle
// is disabled (and HDR is never advertised regardless) so the host doesn't send PQ the
// panel would mis-tone-map. The capability is fixed for the device, so read it once.
val hdrCapable = remember { displaySupportsHdr(context) }
ToggleRow(
title = "HDR",
subtitle = if (hdrCapable) {
"Stream 10-bit HDR (BT.2020 PQ) when the host supports it"
} else {
"This display can't present HDR10 — streams stay SDR"
},
checked = s.hdrEnabled && hdrCapable,
enabled = hdrCapable,
onCheckedChange = { on -> update(s.copy(hdrEnabled = on)) },
)
}
SettingsGroup("Host") {
@@ -143,6 +166,14 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
onCheckedChange = { on -> update(s.copy(statsHudEnabled = on)) },
)
}
SettingsGroup("About") {
ClickableRow(
title = "Open-source licenses",
subtitle = "Third-party notices and credits",
onClick = { showLicenses = true },
)
}
}
}
@@ -166,15 +197,41 @@ private fun SettingsGroup(title: String, content: @Composable ColumnScope.() ->
}
}
/** A title + subtitle on the left, a Switch on the right. */
/** A title + subtitle on the left, a Switch on the right. [enabled] greys out the whole row. */
@Composable
private fun ToggleRow(
title: String,
subtitle: String,
checked: Boolean,
onCheckedChange: (Boolean) -> Unit,
enabled: Boolean = true,
) {
// Dim the labels when disabled so the row reads as inactive (the Switch dims itself).
val labelAlpha = if (enabled) 1f else 0.38f
Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Column(Modifier.weight(1f)) {
Text(
title,
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurface.copy(alpha = labelAlpha),
)
Text(
subtitle,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = labelAlpha),
)
}
Switch(checked = checked, onCheckedChange = onCheckedChange, enabled = enabled)
}
}
/** A title + subtitle on the left; the whole row is clickable (opens a sub-screen). */
@Composable
private fun ClickableRow(title: String, subtitle: String, onClick: () -> Unit) {
Row(
modifier = Modifier.fillMaxWidth().clickable(onClick = onClick),
verticalAlignment = Alignment.CenterVertically,
) {
Column(Modifier.weight(1f)) {
Text(title, style = MaterialTheme.typography.bodyLarge)
Text(
@@ -183,7 +240,6 @@ private fun ToggleRow(
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
Switch(checked = checked, onCheckedChange = onCheckedChange)
}
}
@@ -1,6 +1,7 @@
package io.unom.punktfunk
import android.Manifest
import android.content.pm.ActivityInfo
import android.content.pm.PackageManager
import android.view.SurfaceHolder
import android.view.SurfaceView
@@ -102,6 +103,13 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
it.systemBarsBehavior = WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
it.hide(WindowInsetsCompat.Type.systemBars())
}
// Lock to landscape while streaming — the host streams a landscape desktop, so pin the device
// there (either landscape direction is fine) and stop it rotating to portrait mid-session. The
// activity declares configChanges=orientation, so this re-lays out the surface in place without
// recreating the activity (no stream restart). On TV (fixed landscape) it's a harmless no-op.
// The prior request is captured and restored on the way out.
val priorOrientation = activity?.requestedOrientation
activity?.requestedOrientation = ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE
activity?.streamHandle = handle // route hardware keys to this session
activity?.axisMapper = Gamepad.AxisMapper(handle) // route joystick axes
// Host→client feedback (rumble + DualSense lightbar/LEDs); poll threads stopped before close.
@@ -114,6 +122,9 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
activity?.streamHandle = 0L
controller?.show(WindowInsetsCompat.Type.systemBars())
window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
// Release the landscape lock so the rest of the app follows the device/system again.
activity?.requestedOrientation =
priorOrientation ?: ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED
// Leaving the stream: stop the mic + audio + decode threads and tear down the session.
NativeBridge.nativeStopMic(handle)
NativeBridge.nativeStopAudio(handle)
@@ -314,9 +325,11 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
}
/**
* The live stats overlay — mirrors the Apple client's HUD. Reads the 10-double layout from
* The live stats overlay — mirrors the Apple client's HUD. Reads the 14-double layout from
* [NativeBridge.nativeVideoStats]:
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skew, w, h, hz, dropped]`.
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skew, w, h, hz, dropped, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc]`. The trailing four (present on a current native lib) describe the
* negotiated video feed and render as a codec/depth/colour/chroma line; older layouts just omit it.
*/
@Composable
internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
@@ -338,6 +351,14 @@ internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
videoFeedLine(s)?.let { feed ->
Text(
feed,
color = Color.White,
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
if (latValid) {
val tag = if (skew) "" else " (same-host)"
Text(
@@ -357,3 +378,31 @@ internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
}
}
}
/**
* Format the negotiated video-feed descriptor from the trailing four stats doubles
* `[bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`, e.g.
* `HEVC · 10-bit · HDR (BT.2020 PQ) · 4:2:0`. Returns `null` on a pre-video-feed layout (< 14 doubles)
* so the overlay simply omits the line. The codes are CICP / H.273: transfer 16 = PQ, 18 = HLG (else
* SDR); primaries 9 = BT.2020, 1 = BT.709; chroma_format_idc 1 = 4:2:0, 2 = 4:2:2, 3 = 4:4:4. The
* Android decoder is always HEVC (`video/hevc`).
*/
private fun videoFeedLine(s: DoubleArray): String? {
if (s.size < 14) return null
val bitDepth = s[10].toInt()
val primaries = s[11].toInt()
val transfer = s[12].toInt()
val chromaIdc = s[13].toInt()
val depthLabel = if (bitDepth > 0) "$bitDepth-bit" else "8-bit"
val (dynamicRange, colorSpace) = when (transfer) {
16 -> "HDR" to "BT.2020 PQ"
18 -> "HDR" to "BT.2020 HLG"
else -> "SDR" to if (primaries == 9) "BT.2020" else "BT.709"
}
val chromaLabel = when (chromaIdc) {
3 -> "4:4:4"
2 -> "4:2:2"
else -> "4:2:0"
}
return "HEVC · $depthLabel · $dynamicRange ($colorSpace) · $chromaLabel"
}
@@ -14,8 +14,10 @@ enum class Tab(val label: String, val icon: ImageVector) {
/**
* A trust decision awaiting the user before a connect proceeds. [name] is the label to save the
* host under. Trust-on-first-use ([Kind.TRUST_NEW]) is only ever offered when the host ADVERTISED
* pair=optional; a pair=required host or a manually-typed/unknown-policy host goes straight to PIN
* pairing ([Kind.PAIR]), and a changed fingerprint forces re-pairing — never a silent re-trust.
* pair=optional; a pair=required host or a manually-typed/unknown-policy host is offered the
* two ways in ([Kind.REQUEST_ACCESS]): a no-PIN "request access" connect the operator approves in
* the host's console, or the SPAKE2 PIN ceremony ([Kind.PAIR]). A changed fingerprint forces
* re-pairing by PIN ([Kind.FP_CHANGED]) — never a silent re-trust.
*/
data class PendingTrust(
val host: String,
@@ -24,7 +26,7 @@ data class PendingTrust(
val advertisedFp: String?,
val kind: Kind,
) {
enum class Kind { TRUST_NEW, FP_CHANGED, PAIR }
enum class Kind { TRUST_NEW, FP_CHANGED, PAIR, REQUEST_ACCESS }
}
/** Trust state of a host, shown as a colored pill on its card. */
@@ -186,9 +186,11 @@ internal fun StreamScene() {
Brush.linearGradient(listOf(Color(0xFF2A1E5C), Color(0xFF0E1B3D), Color(0xFF06122B))),
),
) {
// [fps, mbps, latP50, latP95, latValid, skew, w, h, hz, dropped]
// [fps, mbps, latP50, latP95, latValid, skew, w, h, hz, dropped,
// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc] — the last four = a 10-bit
// BT.2020 PQ (HDR) 4:2:0 feed, so the HUD renders its video-feed line.
StatsOverlay(
doubleArrayOf(238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 0.0),
doubleArrayOf(238.0, 921.4, 1.3, 2.1, 1.0, 1.0, 5120.0, 1440.0, 240.0, 0.0, 10.0, 9.0, 16.0, 1.0),
Modifier.align(Alignment.TopStart).padding(12.dp),
)
}
@@ -50,15 +50,25 @@ object Gamepad {
const val PREF_DUALSENSE = 2
const val PREF_XBOXONE = 3
const val PREF_DUALSHOCK4 = 4
const val PREF_STEAMCONTROLLER = 5
const val PREF_STEAMDECK = 6
// USB vendor ids of the controllers we can identify by VID/PID.
private const val VID_SONY = 0x054C
private const val VID_MICROSOFT = 0x045E
private const val VID_VALVE = 0x28DE
// Sony product ids. DualSense (PS5) and DualShock 4 (PS4) map to distinct host pad types.
private val PID_DUALSENSE = setOf(0x0CE6, 0x0DF2)
private val PID_DUALSHOCK4 = setOf(0x05C4, 0x09CC)
// Valve: Steam Deck built-in controller (0x1205); classic Steam Controller wired (0x1102) /
// dongle (0x1142). The host builds the virtual hid-steam pad; rich-input capture (paddles /
// trackpads / gyro) is out of scope on Android (no rich-input plane yet), so only the standard
// buttons + sticks reach the host for now — parity with the desktop type resolution.
private val PID_STEAMDECK = setOf(0x1205)
private val PID_STEAMCONTROLLER = setOf(0x1102, 0x1142)
// Microsoft Xbox One / Series product ids (wired + the common Bluetooth/dongle revisions). All
// behave like Xbox 360 on the host minus the glyph identity, so they share one pref byte.
private val PID_XBOXONE = setOf(
@@ -82,6 +92,8 @@ object Gamepad {
vid == VID_SONY && pid in PID_DUALSENSE -> PREF_DUALSENSE
vid == VID_SONY && pid in PID_DUALSHOCK4 -> PREF_DUALSHOCK4
vid == VID_MICROSOFT && pid in PID_XBOXONE -> PREF_XBOXONE
vid == VID_VALVE && pid in PID_STEAMDECK -> PREF_STEAMDECK
vid == VID_VALVE && pid in PID_STEAMCONTROLLER -> PREF_STEAMCONTROLLER
else -> PREF_XBOX360
}
}
@@ -29,8 +29,10 @@ object NativeBridge {
* trust-on-first-use — read [nativeHostFingerprint] after; else 64-hex host SHA-256, mismatch →
* `0`). [width]/[height]/[refreshHz] are the requested virtual-output mode (the host streams at
* exactly this); [bitrateKbps] 0 = host default; [compositorPref]/[gamepadPref] are the
* `CompositorPref`/`GamepadPref` wire bytes (0 = Auto). Returns an opaque session handle, or `0`
* on failure. Pair with exactly one [nativeClose].
* `CompositorPref`/`GamepadPref` wire bytes (0 = Auto). [timeoutMs] is the handshake budget — the
* normal path passes a short value, the no-PIN "request access" path a long one (≥ the host's
* approval-park window) so a slow operator approval lands on this same parked connection. Returns
* an opaque session handle, or `0` on failure. Pair with exactly one [nativeClose].
*/
external fun nativeConnect(
host: String,
@@ -46,6 +48,7 @@ object NativeBridge {
gamepadPref: Int,
hdrEnabled: Boolean,
audioChannels: Int,
timeoutMs: Int,
): Long
/** 64-hex SHA-256 of the cert the host presented on [handle]; valid after a successful connect. */
@@ -100,9 +103,12 @@ object NativeBridge {
/**
* Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs.
* Returns 10 doubles:
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped]`
* (the two flags are 1.0/0.0). Poll ~1 Hz; each call resets the measurement window.
* Returns 14 doubles:
* `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped,
* bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`
* (the two flags are 1.0/0.0; the trailing four describe the negotiated video feed — bit depth
* 8/10, CICP primaries/transfer, and the HEVC chroma_format_idc 1=4:2:0 / 3=4:4:4). Poll ~1 Hz;
* each call resets the measurement window.
*/
external fun nativeVideoStats(handle: Long): DoubleArray?
+5
View File
@@ -114,6 +114,11 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextHidout(
out[2..n].copy_from_slice(&effect);
n
}
HidOutput::TrackpadHaptic { .. } => {
// Steam Controller trackpad-coil haptics — no Android equivalent; drop it (motor
// rumble already rides the universal 0xCA plane).
return -1;
}
};
n as jint
})
+27 -11
View File
@@ -140,13 +140,15 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeGenerateIde
}
/// `NativeBridge.nativeConnect(host, port, w, h, hz, certPem, keyPem, pinHex, bitrateKbps,
/// compositorPref, gamepadPref, hdrEnabled, audioChannels): Long`. `certPem`/`keyPem` empty =
/// anonymous, else presented as the persistent identity. `pinHex` empty = TOFU (read
/// compositorPref, gamepadPref, hdrEnabled, audioChannels, timeoutMs): Long`. `certPem`/`keyPem`
/// empty = anonymous, else presented as the persistent identity. `pinHex` empty = TOFU (read
/// `nativeHostFingerprint` after), else 64-hex SHA-256 to pin the host (mismatch → 0). `bitrateKbps`
/// 0 = host default. `compositorPref`/`gamepadPref` are `CompositorPref`/`GamepadPref` wire bytes
/// (0 = Auto; unknown → Auto). `audioChannels` is the requested surround layout (2/6/8; normalized,
/// anything else → stereo) — the host clamps it and the resolved count drives playback.
/// Returns an opaque handle, or 0 on failure (logged).
/// anything else → stereo) — the host clamps it and the resolved count drives playback. `timeoutMs`
/// is the handshake budget: the normal path passes a short value, the no-PIN "request access" path a
/// long one (≥ the host's approval-park window) so a slow operator approval lands on this same parked
/// connection rather than timing the client out first. Returns an opaque handle, or 0 on failure.
#[no_mangle]
#[allow(clippy::too_many_arguments)]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'local>(
@@ -165,6 +167,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
gamepad_pref: jint,
hdr_enabled: jboolean,
audio_channels: jint,
timeout_ms: jint,
) -> jlong {
let host: String = match env.get_string(&host) {
Ok(s) => s.into(),
@@ -224,7 +227,9 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
None, // launch: default app
pin, // Some → Crypto on host-fp mismatch
identity, // owned (cert, key) PEM, or None (anonymous)
Duration::from_secs(10),
// Handshake budget from Kotlin: ~10 s for a normal connect, ~185 s for "request access"
// (the host parks the connection until the operator approves the device — see ConnectScreen).
Duration::from_millis(timeout_ms.max(0) as u64),
) {
Ok(client) => {
let handle = SessionHandle {
@@ -404,11 +409,13 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
}
/// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD.
/// Returns 10 doubles
/// `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped]`
/// (the two flags are 1.0/0.0), or `null` when no decode thread is running. Poll ~1 Hz from the UI;
/// each call resets the measurement window. Not android-gated — pure `jni` + connector reads, so it
/// links on the host build too (Kotlin only ever calls it on device).
/// Returns 14 doubles
/// `[fps, mbps, latP50Ms, latP95Ms, latValid, skewCorrected, width, height, refreshHz, framesDropped,
/// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc]`
/// (the two flags are 1.0/0.0; the trailing four describe the negotiated video feed — see below), or
/// `null` when no decode thread is running. Poll ~1 Hz from the UI; each call resets the measurement
/// window. Not android-gated — pure `jni` + connector reads, so it links on the host build too
/// (Kotlin only ever calls it on device).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
env: JNIEnv,
@@ -426,7 +433,8 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
None => return std::ptr::null_mut(), // not streaming → no stats
};
let mode = h.client.mode();
let buf: [f64; 10] = [
let color = h.client.color;
let buf: [f64; 14] = [
snap.fps,
snap.mbps,
snap.lat_p50_ms,
@@ -437,6 +445,14 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
mode.height as f64,
mode.refresh_hz as f64,
h.client.frames_dropped() as f64,
// Video-feed properties the host resolved at the handshake (Welcome): encode bit depth
// (8 / 10), the CICP colour primaries + transfer code points (Kotlin maps these to a
// colour-space / HDR label — transfer 16 = PQ, 18 = HLG ⇒ HDR), and the HEVC
// chroma_format_idc (1 = 4:2:0, 3 = 4:4:4). Static for the session unless renegotiated.
h.client.bit_depth as f64,
color.primaries as f64,
color.transfer as f64,
h.client.chroma_format as f64,
];
let arr = match env.new_double_array(buf.len() as jsize) {
Ok(a) => a,
+12
View File
@@ -16,6 +16,18 @@ let package = Package(
.target(
name: "PunktfunkKit",
dependencies: ["PunktfunkCore"],
// OSS attribution shown by the app's Acknowledgements screen. Bundled here (not in the
// app target) so it rides along via Bundle.module in both `swift build` and the Xcode
// app, which links the PunktfunkKit product. Refresh with
// scripts/gen-third-party-notices.sh (it copies the generated file into Resources/).
resources: [
.copy("Resources/THIRD-PARTY-NOTICES.txt"),
.copy("Resources/LICENSE-MIT.txt"),
.copy("Resources/LICENSE-APACHE.txt"),
// Geist (SIL OFL 1.1) the brand typeface, shared with punktfunk-website.
// Registered with Core Text at first use; see BrandFont.swift.
.copy("Resources/Fonts"),
],
linkerSettings: [
// Rust staticlib system deps.
.linkedFramework("Security"),
@@ -364,7 +364,7 @@
ENABLE_HARDENED_RUNTIME = YES;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
@@ -398,7 +398,7 @@
ENABLE_HARDENED_RUNTIME = YES;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSHumanReadableCopyright = "";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
@@ -429,7 +429,7 @@
CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -468,7 +468,7 @@
CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -506,7 +506,7 @@
CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -536,7 +536,7 @@
CURRENT_PROJECT_VERSION = 1;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunkempfänger";
INFOPLIST_KEY_CFBundleDisplayName = "Punktfunk";
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
@@ -0,0 +1,87 @@
import PunktfunkKit
import SwiftUI
/// Open-source acknowledgements: punktfunk's own license (MIT OR Apache-2.0) followed by the
/// third-party software notices. Used as a pushed view on iOS/tvOS and a preferences tab on macOS.
struct AcknowledgementsView: View {
private var version: String? {
Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String
}
var body: some View {
ScrollView {
// Top-level LazyVStack so the third-party-notices chunks (Licenses.thirdPartyNoticesChunks,
// ~885 KB total) load lazily as they scroll into view a single Text that large overshoots
// the text-rendering height limit (blank below the limit + very slow). spacing 0 keeps the
// notice chunks visually continuous; the header block carries its own spacing + bottom pad.
LazyVStack(alignment: .leading, spacing: 0) {
VStack(alignment: .leading, spacing: 18) {
Text("punktfunk")
.font(.geist(22, .bold, relativeTo: .title2))
if let version {
Text("Version \(version)")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
Text(Licenses.appLicense)
.font(.caption.monospaced())
.modifier(SelectableText())
Divider()
Text("Bundled font")
.font(.geist(17, .semibold, relativeTo: .headline))
Text("punktfunk ships the Geist typeface (Geist Sans), "
+ "© The Geist Project Authors / Vercel, used under the SIL Open Font "
+ "License 1.1.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
if !Licenses.fontLicense.isEmpty {
Text(Licenses.fontLicense)
.font(.caption2.monospaced())
.modifier(SelectableText())
}
Divider()
Text("Third-party software")
.font(.geist(17, .semibold, relativeTo: .headline))
Text(
"punktfunk uses the open-source components below, each under its own license. "
+ "On some platforms FFmpeg is additionally bundled under the LGPL v2.1+ "
+ "(dynamically linked, replaceable)."
)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
.frame(maxWidth: .infinity, alignment: .leading)
.padding(.bottom, 18)
ForEach(Licenses.thirdPartyNoticesChunks.indices, id: \.self) { i in
Text(Licenses.thirdPartyNoticesChunks[i])
.font(.caption2.monospaced())
.frame(maxWidth: .infinity, alignment: .leading)
.modifier(SelectableText())
}
}
.frame(maxWidth: 900, alignment: .leading)
.frame(maxWidth: .infinity, alignment: .leading)
.padding()
#if os(tvOS)
.padding(40)
#endif
}
.navigationTitle("Acknowledgements")
}
}
/// `textSelection(.enabled)` is unavailable on tvOS, so apply it only where it exists.
private struct SelectableText: ViewModifier {
func body(content: Content) -> some View {
#if os(tvOS)
content
#else
content.textSelection(.enabled)
#endif
}
}
@@ -80,6 +80,11 @@ struct AddHostSheet: View {
}
#if !os(tvOS)
.formStyle(.grouped)
#endif
#if os(iOS)
// The detent below is sized to fit all 3 rows + the action button exactly, so the
// Form must NOT scroll/bounce inside it lock it. (iOS 16+; safe at iOS 17.)
.scrollDisabled(true)
#endif
#if os(macOS)
// macOS: UNCHANGED Cancel + Spacer + Add in an HStack, both wired to the
@@ -120,8 +125,8 @@ struct AddHostSheet: View {
// Form + the full-width action row, instead of the half-screen .medium it used to rest
// at. A single fixed detent is enough: the system keeps the content above the keyboard
// when Address/Port is focused, and on iPadOS this renders as a short bottom sheet (not a
// centered formSheet card). If Dynamic Type grows the rows past this height the Form just
// scrolls inside the detent nothing is clipped. (.height(_:) is iOS 16+, safe at iOS 17.)
// centered formSheet card). The Form itself is .scrollDisabled (above) so it can't
// bounce/scroll inside this fixed detent. (.height(_:) is iOS 16+, safe at iOS 17.)
.presentationDetents([.height(320)])
.presentationDragIndicator(.visible)
#endif
@@ -0,0 +1,39 @@
// App-wide brand chrome. SwiftUI has no single switch to put a custom font on every navigation
// title, so the iOS large/inline nav titles are themed through UINavigationBar's appearance proxy
// (set once at launch). Backgrounds are left at the system defaults transparent at the scroll
// edge (the large title floats on the content), blurred once scrolled so only the typeface
// changes: Geist, matching the cards and the website.
#if os(iOS)
import PunktfunkKit
import UIKit
enum BrandTheme {
static func apply() {
BrandFont.registerIfNeeded()
let scrollEdge = UINavigationBarAppearance()
scrollEdge.configureWithTransparentBackground()
applyFonts(to: scrollEdge)
let standard = UINavigationBarAppearance()
standard.configureWithDefaultBackground()
applyFonts(to: standard)
let proxy = UINavigationBar.appearance()
proxy.scrollEdgeAppearance = scrollEdge
proxy.standardAppearance = standard
proxy.compactAppearance = standard
}
/// Override only the title fonts; leave colors/backgrounds at the configured defaults.
private static func applyFonts(to appearance: UINavigationBarAppearance) {
if let large = UIFont(name: "Geist-Bold", size: 34) {
appearance.largeTitleTextAttributes[.font] = large
}
if let inline = UIFont(name: "Geist-SemiBold", size: 17) {
appearance.titleTextAttributes[.font] = inline
}
}
}
#endif
@@ -4,10 +4,12 @@
// (HomeView/HostCards), the trust prompt (TrustCardView), and the HUD (StreamHUDView) live in
// their own files.
//
// Two ways to establish trust on first contact: the TOFU prompt (host fingerprint over the
// live-but-blurred stream, compared with the host's log) or the PIN pairing ceremony pairing
// verifies both sides at once and is the only way into hosts running --require-pairing. Once
// pinned, reconnects are silent and a changed host identity refuses to connect.
// Ways to establish trust on first contact: the TOFU prompt (host fingerprint over the
// live-but-blurred stream, compared with the host's log; only for a host advertising pair=optional),
// the PIN pairing ceremony (verifies both sides at once), or for a host that requires pairing
// delegated approval ("Request Access": a plain identified connect the host parks until the operator
// approves this device in its console, no PIN). Once pinned, reconnects are silent and a changed
// host identity refuses to connect.
#if os(macOS)
import AppKit
@@ -26,11 +28,18 @@ struct ContentView: View {
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.audioChannels) private var audioChannels = 2
@AppStorage(DefaultsKey.hdrEnabled) private var hdrEnabled = true
@AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.hudEnabled) private var hudEnabled = true
@AppStorage(DefaultsKey.hudPlacement) private var hudPlacement = HUDPlacement.topTrailing.rawValue
@State private var showAddHost = false
@State private var pairingTarget: StoredHost?
/// A fresh `pair=required`/unknown host the user tapped: drives the choice between no-PIN
/// delegated approval ("Request Access") and the SPAKE2 PIN ceremony (rule 3b).
@State private var approvalChoice: ApprovalRequest?
/// A delegated-approval connect is in flight (host parks it until the operator approves):
/// drives the cancelable "Waiting for approval" prompt and the pin-as-paired on success.
@State private var awaitingApproval: ApprovalRequest?
@State private var speedTestTarget: StoredHost?
@State private var libraryTarget: StoredHost?
#if !os(macOS)
@@ -55,10 +64,31 @@ struct ContentView: View {
autoConnectIfAsked()
}
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
// A session actually started remember it on the card ("Connected ago"
// plus the accent ring on the most recent host).
if case .streaming = phase, let host = model.activeHost {
guard let host = model.activeHost else { break }
// Delegated approval just succeeded: the operator let this device in, so pin the
// host's observed fingerprint and remember it as paired future connects are then
// silent (rule 1), exactly like after a PIN/TOFU success. Dismisses the wait prompt.
let approvedFingerprint = awaitingApproval?.host.id == host.id
? model.connection?.hostFingerprint : nil
if awaitingApproval?.host.id == host.id { awaitingApproval = nil }
// Persist on the next runloop tick: HostStore is an ObservableObject, and mutating
// its @Published from inside .onChange (a view-update callback) trips SwiftUI's
// "Publishing changes from within view updates". A one-tick delay is imperceptible.
let store = store
DispatchQueue.main.async {
store.markConnected(host.id)
if let approvedFingerprint { store.pin(host.id, fingerprint: approvedFingerprint) }
}
case .idle:
// The delegated-approval connect failed, timed out, or was cancelled drop the
// wait prompt (SessionModel surfaces any error via `errorMessage`).
if awaitingApproval != nil { awaitingApproval = nil }
default:
break
}
}
.onDisappear { model.disconnect() } // window closed mid-session (Cmd+N spawns more)
@@ -90,6 +120,47 @@ struct ContentView: View {
}
}
#endif
// Fresh pair=required / unknown host: offer the two ways in. An action sheet (not an
// alert) so it never collides with the wait alert below. "Request Access" is the no-PIN
// delegated-approval path; "Pair with PIN" runs the SPAKE2 ceremony. The follow-on
// presentation is deferred a tick so this dialog is fully dismissed first.
.confirmationDialog(
"Pairing required",
isPresented: Binding(
get: { approvalChoice != nil },
set: { if !$0 { approvalChoice = nil } }),
titleVisibility: .visible,
presenting: approvalChoice
) { req in
Button("Request Access") {
DispatchQueue.main.async { requestAccess(req) }
}
Button("Pair with PIN…") {
DispatchQueue.main.async { pairingTarget = req.host }
}
Button("Cancel", role: .cancel) {}
} message: { req in
Text("\(req.host.displayName) requires pairing. Request access and approve this "
+ "device in the host's web console (port 3000 → Pairing) — no PIN needed. Or "
+ "pair with the 4-digit PIN it can display.")
}
// The delegated-approval wait: the host holds the connection open until the operator
// approves it. Cancel returns the UI at once; the in-flight connect is left to time out
// and its late result is discarded by SessionModel's connect guard (disconnect resets the
// phase/host it checks).
.alert(
"Waiting for approval",
isPresented: Binding(
get: { awaitingApproval != nil },
set: { if !$0 { awaitingApproval = nil } }),
presenting: awaitingApproval
) { _ in
Button("Cancel", role: .cancel) { model.disconnect() }
} message: { req in
Text("Approve \u{201C}\(localDeviceName)\u{201D} in \(req.host.displayName)'s web "
+ "console (port 3000 → Pairing). This device connects automatically once you "
+ "approve it — no need to reconnect.")
}
}
private var home: some View {
@@ -230,19 +301,32 @@ struct ContentView: View {
// A pinned host connects on its stored fingerprint; an unpinned host may only TOFU when
// the host's LIVE advert says `pair=optional` (rule 3a). When the caller doesn't already
// know the policy (a saved-card tap / manual entry), resolve it from the current mDNS set:
// an unpinned host with no matching `pair=optional` advert routes to PIN pairing instead
// of silently entering the trust prompt (rules 3b + 4). A pinned host ignores all of this.
// an unpinned host with no matching `pair=optional` advert routes to the approval choice
// (request access / pair with PIN) instead of silently entering the trust prompt (rules
// 3b + 4). A pinned host ignores all of this.
if host.pinnedSHA256 == nil {
let tofuOK = allowTofu ?? discovery.hosts.contains {
host.matches($0) && $0.allowsTofu
}
if !tofuOK {
pairingTarget = host
// pair=required / unknown policy / manual entry (rule 3b): never a silent
// connect offer no-PIN delegated approval or the PIN ceremony.
approvalChoice = ApprovalRequest(
host: host, advertisedFingerprint: advertisedFingerprint(for: host))
return
}
}
// The gamepad-type setting resolves NOW (Automatic match the active physical
// controller): the host's virtual pad backend is fixed per session.
startSession(host, launchID: launchID, allowTofu: host.pinnedSHA256 == nil)
}
/// Resolve the @AppStorage stream mode + input prefs and hand off to the session model. The
/// gamepad-type setting resolves NOW (Automatic match the active physical controller): the
/// host's virtual pad backend is fixed per session. `requestAccess` opens the no-PIN
/// delegated-approval connect (host parks it until the operator approves).
private func startSession(
_ host: StoredHost, launchID: String? = nil,
allowTofu: Bool, requestAccess: Bool = false
) {
model.connect(
to: host,
width: UInt32(clamping: width), height: UInt32(clamping: height),
@@ -254,8 +338,24 @@ struct ContentView: View {
rawValue: UInt32(clamping: gamepadType)) ?? .auto),
bitrateKbps: UInt32(clamping: bitrateKbps),
audioChannels: UInt8(clamping: audioChannels),
hdrEnabled: hdrEnabled,
launchID: launchID,
allowTofu: host.pinnedSHA256 == nil)
allowTofu: allowTofu,
requestAccess: requestAccess)
}
/// The no-PIN delegated-approval flow: open an identified connect the host parks until the
/// operator approves it in the console, showing the cancelable "Waiting for approval" prompt
/// meanwhile. On success the SAME connection is admitted (no reconnect) and the host is pinned
/// as paired (see the `.streaming` branch of `onChange`).
private func requestAccess(_ req: ApprovalRequest) {
guard !model.isBusy else { return }
awaitingApproval = req
// Pin the advertised certificate for a discovered host (impostor defence during the long
// wait); a manually-typed host has no advertised fingerprint, so trust-on-first-use.
var host = req.host
host.pinnedSHA256 = req.advertisedFingerprint
startSession(host, allowTofu: false, requestAccess: true)
}
/// Picked a title in the (experimental) library: dismiss the browser and start a session that
@@ -268,8 +368,9 @@ struct ContentView: View {
/// Tap a discovered host: save it (so the session has a stored identity and the trust pin
/// persists), then connect or pair per the host's advertised policy. The host is the policy
/// authority TOFU is offered ONLY when it explicitly advertised `pair=optional` (rule 3a);
/// a `pair=required` host, or one with no/unknown `pair` field, goes straight to the PIN
/// pairing ceremony (rule 3b). (A pinned discovered host connects silently inside `connect`.)
/// a `pair=required` host, or one with no/unknown `pair` field, gets the approval choice
/// (request access / pair with PIN) (rule 3b). (A pinned discovered host connects silently
/// inside `connect`.)
private func connectDiscovered(_ d: DiscoveredHost) {
guard !model.isBusy else { return }
let host = StoredHost(name: d.name, address: d.host, port: d.port)
@@ -277,7 +378,9 @@ struct ContentView: View {
if d.allowsTofu {
connect(host, allowTofu: true)
} else {
pairingTarget = host
// pair=required / unknown policy (rule 3b): offer no-PIN delegated approval or PIN.
approvalChoice = ApprovalRequest(
host: host, advertisedFingerprint: pinFingerprint(d.fingerprintHex))
}
}
@@ -291,6 +394,30 @@ struct ContentView: View {
connect(pinned)
}
/// The certificate fingerprint a live mDNS advert carries for this saved host (advisory see
/// `HostDiscovery`), to pin during a delegated-approval wait. nil if the host isn't currently
/// advertising or advertised no/invalid `fp`.
private func advertisedFingerprint(for host: StoredHost) -> Data? {
pinFingerprint(discovery.hosts.first { host.matches($0) }?.fingerprintHex)
}
/// Parse an advertised cert fingerprint (lowercase hex) into the 32-byte pin the connect
/// expects; nil unless it's exactly a 32-byte (SHA-256) value, so a malformed advert falls
/// back to trust-on-first-use rather than failing the connect closed.
private func pinFingerprint(_ hex: String?) -> Data? {
guard let hex, let data = Data(hexString: hex), data.count == 32 else { return nil }
return data
}
/// How the host lists this device in its approval prompt (matches PairSheet's client name).
private var localDeviceName: String {
#if os(macOS)
Host.current().localizedName ?? "Mac"
#else
UIDevice.current.name
#endif
}
// MARK: - First-run + dev hooks
/// First run on iOS: default the stream mode to this device's native screen so the
@@ -354,6 +481,7 @@ struct ContentView: View {
gamepad: pad,
bitrateKbps: bitrate,
audioChannels: UInt8(clamping: audioChannels),
hdrEnabled: hdrEnabled,
autoTrust: true)
}
}
@@ -378,3 +506,31 @@ private struct FullscreenController: NSViewRepresentable {
}
}
#endif
/// A fresh `pair=required`/unknown host pending a trust decision: drives both the "request access
/// vs. pair with PIN" choice and the subsequent approval wait. `advertisedFingerprint` is the
/// discovered host's advertised cert (nil for a manually-typed host trust-on-first-use).
private struct ApprovalRequest {
let host: StoredHost
let advertisedFingerprint: Data?
}
private extension Data {
/// Parse an even-length hex string into bytes; nil on any non-hex character or odd length.
/// Used to turn an mDNS-advertised cert fingerprint into a connect pin.
init?(hexString: String) {
let chars = Array(hexString)
guard chars.count.isMultiple(of: 2) else { return nil }
var bytes = [UInt8]()
bytes.reserveCapacity(chars.count / 2)
var i = 0
while i < chars.count {
guard let hi = chars[i].hexDigitValue, let lo = chars[i + 1].hexDigitValue else {
return nil
}
bytes.append(UInt8(hi << 4 | lo))
i += 2
}
self = Data(bytes)
}
}
@@ -54,7 +54,7 @@ struct ControllerTestView: View {
var body: some View {
VStack(spacing: 0) {
HStack {
Text("Test Controller").font(.headline)
Text("Test Controller").font(.geist(17, .semibold, relativeTo: .headline))
Spacer()
Button("Done") { dismiss() }.keyboardShortcut(.cancelAction)
}
@@ -99,8 +99,8 @@ struct ControllerTestView: View {
.font(.title2)
.foregroundStyle(.secondary)
VStack(alignment: .leading, spacing: 2) {
Text(c.name).font(.headline)
Text(c.productCategory).font(.caption).foregroundStyle(.secondary)
Text(c.name).font(.geist(17, .semibold, relativeTo: .headline))
Text(c.productCategory).font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
}
Spacer()
}
@@ -209,7 +209,7 @@ struct ControllerTestView: View {
) -> some View {
VStack(alignment: .leading, spacing: 4) {
Text("Touchpad\(tp.button.isPressed ? " — click" : "")")
.font(.caption2).foregroundStyle(.secondary)
.font(.geist(11, relativeTo: .caption2)).foregroundStyle(.secondary)
ZStack {
RoundedRectangle(cornerRadius: 8).stroke(Color.secondary.opacity(0.3))
fingerDot(tp.primary, color: .accentColor)
@@ -230,7 +230,7 @@ struct ControllerTestView: View {
private func motionReadout(_ m: GCMotion) -> some View {
let a = Self.totalAccel(m)
return VStack(alignment: .leading, spacing: 2) {
Text("Motion").font(.caption2).foregroundStyle(.secondary)
Text("Motion").font(.geist(11, relativeTo: .caption2)).foregroundStyle(.secondary)
Text(String(format: "gyro %+.2f %+.2f %+.2f",
m.rotationRate.x, m.rotationRate.y, m.rotationRate.z))
.font(.caption2.monospaced())
@@ -254,11 +254,11 @@ struct ControllerTestView: View {
Toggle("Heavy motor (left)", isOn: $heavyOn)
Toggle("Light motor (right)", isOn: $lightOn)
Label("Backend: \(tester.rumbleBackend)", systemImage: "waveform")
.font(.caption).foregroundStyle(.secondary)
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
Text("Toggle a motor to feel it. The host maps a game's low/high-frequency "
+ "rumble onto these two. A DualSense is driven over raw HID (CoreHaptics "
+ "can't reach its motors on macOS).")
.font(.caption).foregroundStyle(.secondary)
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
}
.onChange(of: heavyOn) { _, _ in applyRumble() }
.onChange(of: lightOn) { _, _ in applyRumble() }
@@ -289,11 +289,11 @@ struct ControllerTestView: View {
}
}
Text("Pick an effect, then pull L2/R2 to feel the resistance.")
.font(.caption).foregroundStyle(.secondary)
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
}
} else {
Text("Adaptive triggers need a DualSense.")
.font(.caption).foregroundStyle(.secondary)
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
}
}
}
@@ -348,7 +348,7 @@ struct ControllerTestView: View {
_ title: String, @ViewBuilder _ content: () -> Content
) -> some View {
VStack(alignment: .leading, spacing: 10) {
Text(title).font(.subheadline.weight(.semibold))
Text(title).font(.geist(15, .semibold, relativeTo: .subheadline))
content()
}
.frame(maxWidth: .infinity, alignment: .leading)
@@ -127,14 +127,13 @@ struct HomeView: View {
AddHostSheet { store.add($0) }
}
#if os(iOS)
// SettingsView owns its own NavigationSplitView (sidebar + detail) and Done button, so it
// is presented directly wrapping it in a NavigationStack here would nest a split view in
// a stack (double title bars). `settingsSheetSizing()` widens the sheet on iPad for the
// two-column layout.
.sheet(isPresented: $showSettings) {
NavigationStack {
SettingsView()
.navigationTitle("Settings")
.toolbar {
Button("Done") { showSettings = false }
}
}
.settingsSheetSizing()
}
#endif
#endif
@@ -172,7 +171,7 @@ struct HomeView: View {
private var discoveredSection: some View {
VStack(alignment: .leading, spacing: 10) {
Label("On this network", systemImage: "antenna.radiowaves.left.and.right")
.font(.headline)
.font(.geist(15, .semibold, relativeTo: .headline))
.foregroundStyle(.secondary)
.padding(.horizontal)
LazyVGrid(columns: gridColumns, spacing: gridSpacing) {
@@ -249,8 +248,10 @@ struct HomeView: View {
/// the width so the cards stay edge-aligned with the title and bars sized touch-first: one
/// column on iPhone portrait, 34 generous cards on iPad.
private var gridColumns: [GridItem] {
// Wider than before: the monogram card is a horizontal module (tile + address line), so
// it needs room for a monospaced "IP:port" without truncating.
#if os(macOS)
[GridItem(.adaptive(minimum: 180, maximum: 240), spacing: 16)]
[GridItem(.adaptive(minimum: 250, maximum: 320), spacing: 16)]
#elseif os(tvOS)
[GridItem(.adaptive(minimum: 320), spacing: 48)]
#else
@@ -1,26 +1,75 @@
// The host grid's cards: a saved host (tap to connect, context menu) and an mDNS-discovered
// host (tap to save + connect). Both share the same platform-tuned sizing.
// host (tap to save + connect). Both share the "monogram module" look a squared brand-purple
// monogram tile + a left-aligned bold Geist name over monospaced technical metadata
// (address, status), framed by a hairline panel border. Industrial, not soft.
import PunktfunkKit
import SwiftUI
/// Shared host-card sizing touch-first on iOS, compact on macOS/tvOS.
/// Shared host-card sizing touch-first on iOS, compact on macOS, roomy on tvOS.
private struct CardMetrics {
let iconSize: CGFloat
let iconBox: CGFloat
let cardPadding: CGFloat
let nameFont: Font
let tile: CGFloat // monogram tile side
let monogram: CGFloat // monogram letter point size
let name: CGFloat // host-name point size
let meta: CGFloat // address (mono) point size
let status: CGFloat // status-label (mono) point size
let padding: CGFloat
let spacing: CGFloat // tile text gap
let radius: CGFloat
static var current: CardMetrics {
#if os(iOS)
CardMetrics(iconSize: 56, iconBox: 76, cardPadding: 28, nameFont: .title3.weight(.semibold))
CardMetrics(tile: 54, monogram: 26, name: 19, meta: 13, status: 11,
padding: 16, spacing: 14, radius: 12)
#elseif os(tvOS)
CardMetrics(tile: 64, monogram: 32, name: 24, meta: 16, status: 14,
padding: 18, spacing: 18, radius: 14)
#else
CardMetrics(iconSize: 42, iconBox: 56, cardPadding: 18, nameFont: .headline)
CardMetrics(tile: 44, monogram: 21, name: 15, meta: 12, status: 10.5,
padding: 13, spacing: 12, radius: 10)
#endif
}
}
/// A saved host. The accent ring marks the most-recently-connected one; the context menu
/// First letter of a host name, uppercased the monogram glyph. Falls back to a bullet.
private func monogram(_ name: String) -> String {
guard let first = name.trimmingCharacters(in: .whitespacesAndNewlines).first else { return "" }
return String(first).uppercased()
}
/// The squared monogram tile. `filled` = a solid brand-purple chip (saved hosts); otherwise a
/// tinted outline (discovered hosts). Shows a spinner in place of the glyph while connecting.
private func monogramTile(_ letter: String, m: CardMetrics, connecting: Bool, filled: Bool) -> some View {
let shape = RoundedRectangle(cornerRadius: m.radius - 3, style: .continuous)
return ZStack {
shape.fill(filled
? AnyShapeStyle(LinearGradient(
colors: [Color.brand, Color.brand.opacity(0.72)],
startPoint: .top, endPoint: .bottom))
: AnyShapeStyle(Color.brand.opacity(0.14)))
if connecting {
ProgressView().tint(filled ? .white : Color.brand)
} else {
// Fixed size (not Dynamic Type): the glyph is pinned inside a fixed tile, so it must
// not scale up and spill out at large accessibility text sizes. minimumScaleFactor +
// the clip below are belt-and-suspenders for an unusually wide glyph.
Text(letter)
.font(.geistFixed(m.monogram, .bold))
.minimumScaleFactor(0.5)
.lineLimit(1)
.foregroundStyle(filled ? Color.white : Color.brand)
}
}
.frame(width: m.tile, height: m.tile)
.clipShape(shape)
.overlay {
if !filled {
shape.strokeBorder(Color.brand.opacity(0.45), lineWidth: 1)
}
}
}
/// A saved host. A left accent bar marks the most-recently-connected one; the context menu
/// pairs / speed-tests / forgets / removes. Disabled while a session is busy.
struct HostCardView: View {
let host: StoredHost
@@ -41,66 +90,44 @@ struct HostCardView: View {
var body: some View {
let m = CardMetrics.current
return Button(action: onConnect) {
VStack(spacing: 10) {
ZStack {
Image(systemName: "play.display")
.font(.system(size: m.iconSize, weight: .light))
.foregroundStyle(.tint)
.opacity(isConnecting ? 0.3 : 1)
if isConnecting {
ProgressView()
}
}
.frame(height: m.iconBox)
VStack(spacing: 2) {
HStack(spacing: 6) {
// Presence dot: green = advertising on the LAN now; grey = not seen.
Circle()
.fill(isOnline ? Color.green : Color.secondary.opacity(0.35))
.frame(width: 7, height: 7)
.accessibilityLabel(isOnline ? "Online" : "Offline")
HStack(spacing: m.spacing) {
monogramTile(monogram(host.displayName), m: m, connecting: isConnecting, filled: true)
VStack(alignment: .leading, spacing: 4) {
Text(host.displayName)
.font(m.nameFont)
.font(.geist(m.name, .bold, relativeTo: .title3))
.foregroundStyle(.primary)
.lineLimit(1)
}
HStack(spacing: 4) {
if host.pinnedSHA256 != nil {
Image(systemName: "lock.fill")
.font(.system(size: 9))
.foregroundStyle(.secondary)
}
Text("\(host.address):\(String(host.port))")
.font(.caption)
.font(.geist(m.meta, relativeTo: .caption))
.foregroundStyle(.secondary)
.lineLimit(1)
statusRow(m)
}
if let last = host.lastConnected {
Text("Connected \(last, format: .relative(presentation: .named))")
.font(.caption2)
.foregroundStyle(.tertiary)
.lineLimit(1)
Spacer(minLength: 0)
}
}
}
.frame(maxWidth: .infinity)
.padding(.vertical, m.cardPadding)
.padding(.horizontal, 12)
.padding(m.padding)
.frame(maxWidth: .infinity, alignment: .leading)
#if !os(tvOS)
// tvOS: the .card button style owns platter + focus motion extra chrome
// inside it mutes the grow/tilt. Material + accent ring are for pointer UIs.
// Deliberately .regularMaterial, not Liquid Glass: HIG keeps glass off content
// tiles (it flattens hierarchy over an opaque grid) see GlassStyle.swift.
.background(.regularMaterial, in: RoundedRectangle(cornerRadius: 14))
.overlay {
// tvOS: the .card button style owns platter + focus motion; extra chrome mutes it.
// Elsewhere: a flat material panel with a hairline border (industrial, not a soft blob),
// and a brand accent bar down the leading edge for the most-recent host.
.background(.regularMaterial)
.overlay(alignment: .leading) {
if isMostRecent {
RoundedRectangle(cornerRadius: 14)
.strokeBorder(Color.accentColor.opacity(0.35), lineWidth: 1.5)
Rectangle().fill(Color.brand).frame(width: 3)
}
}
.clipShape(RoundedRectangle(cornerRadius: m.radius, style: .continuous))
.overlay {
RoundedRectangle(cornerRadius: m.radius, style: .continuous)
.strokeBorder(.quaternary, lineWidth: 1)
}
#endif
}
#if os(tvOS)
.buttonStyle(.card)
#elseif os(iOS)
.buttonStyle(HostCardButtonStyle(cornerRadius: m.radius))
#else
.buttonStyle(.plain)
#endif
@@ -119,10 +146,31 @@ struct HostCardView: View {
Button("Remove", role: .destructive, action: onRemove)
}
}
/// Technical status line: a square presence pip + monospaced ONLINE/OFFLINE, and PAIRED when a
/// certificate is pinned (the lock state, spelled out).
@ViewBuilder private func statusRow(_ m: CardMetrics) -> some View {
HStack(spacing: 6) {
RoundedRectangle(cornerRadius: 1.5)
.fill(isOnline ? Color.green : Color.secondary.opacity(0.4))
.frame(width: 6, height: 6)
// The state is spelled out in the adjacent text, so the pip is decorative
// otherwise VoiceOver reads the status twice ("Online, ONLINE ").
.accessibilityHidden(true)
Text(isOnline ? "ONLINE" : "OFFLINE")
if host.pinnedSHA256 != nil {
Text("· PAIRED")
}
}
.font(.geist(m.status, .medium, relativeTo: .caption2))
.tracking(0.8)
.foregroundStyle(.secondary)
.lineLimit(1)
}
}
/// A host found on the LAN but not yet saved. A dashed ring distinguishes it from saved cards;
/// tapping saves it and connects (or pairs, if the host requires it).
/// A host found on the LAN but not yet saved. A tinted-outline monogram + dashed panel border
/// distinguish it from saved cards; tapping saves it and connects (or pairs, if required).
struct DiscoveredCardView: View {
let discovered: DiscoveredHost
let isBusy: Bool
@@ -131,47 +179,77 @@ struct DiscoveredCardView: View {
var body: some View {
let m = CardMetrics.current
return Button(action: onConnect) {
VStack(spacing: 10) {
Image(systemName: "play.display")
.font(.system(size: m.iconSize, weight: .light))
.foregroundStyle(.tint)
.frame(height: m.iconBox)
VStack(spacing: 2) {
HStack(spacing: m.spacing) {
monogramTile(monogram(discovered.name), m: m, connecting: false, filled: false)
VStack(alignment: .leading, spacing: 4) {
Text(discovered.name)
.font(m.nameFont)
.font(.geist(m.name, .bold, relativeTo: .title3))
.foregroundStyle(.primary)
.lineLimit(1)
HStack(spacing: 4) {
Image(systemName: discovered.requiresPairing ? "lock.fill" : "wifi")
.font(.system(size: 9))
.foregroundStyle(.secondary)
Text("\(discovered.host):\(String(discovered.port))")
.font(.caption)
.font(.geist(m.meta, relativeTo: .caption))
.foregroundStyle(.secondary)
.lineLimit(1)
HStack(spacing: 6) {
Image(systemName: discovered.requiresPairing
? "lock.fill" : "antenna.radiowaves.left.and.right")
.font(.system(size: m.status))
.accessibilityHidden(true) // decorative; the adjacent text says the state
Text(discovered.requiresPairing ? "PAIRING REQUIRED" : "DISCOVERED")
}
.font(.geist(m.status, .medium, relativeTo: .caption2))
.tracking(0.8)
.foregroundStyle(.secondary)
.lineLimit(1)
}
Text(discovered.requiresPairing ? "Pairing required" : "Discovered")
.font(.caption2)
.foregroundStyle(.tertiary)
Spacer(minLength: 0)
}
}
.frame(maxWidth: .infinity)
.padding(.vertical, m.cardPadding)
.padding(.horizontal, 12)
.padding(m.padding)
.frame(maxWidth: .infinity, alignment: .leading)
#if !os(tvOS)
.background(.regularMaterial, in: RoundedRectangle(cornerRadius: 14))
.background(.regularMaterial)
.clipShape(RoundedRectangle(cornerRadius: m.radius, style: .continuous))
.overlay {
RoundedRectangle(cornerRadius: 14)
RoundedRectangle(cornerRadius: m.radius, style: .continuous)
.strokeBorder(
Color.secondary.opacity(0.25),
Color.secondary.opacity(0.3),
style: StrokeStyle(lineWidth: 1, dash: [4, 3]))
}
#endif
}
#if os(tvOS)
.buttonStyle(.card)
#elseif os(iOS)
.buttonStyle(HostCardButtonStyle(cornerRadius: m.radius))
#else
.buttonStyle(.plain)
#endif
.disabled(isBusy)
}
}
#if os(iOS)
/// The iOS host-card press/hover treatment, one style for both idioms:
/// - iPhone: a subtle scale-down on press + a light impact haptic on press-down. (`hoverEffect` is
/// inert without a pointer.)
/// - iPad: the system pointer "magnet" the cursor morphs into a highlight that conforms to the
/// card's rounded rect on hover. (`sensoryFeedback` is inert without a Taptic Engine, and the
/// press scale doubles as click feedback.)
struct HostCardButtonStyle: ButtonStyle {
var cornerRadius: CGFloat
func makeBody(configuration: Configuration) -> some View {
configuration.label
.scaleEffect(configuration.isPressed ? 0.96 : 1)
.animation(.spring(response: 0.3, dampingFraction: 0.65), value: configuration.isPressed)
// Conform the pointer highlight to the card's rounded rect, not its square bounds.
.contentShape(.hoverEffect, RoundedRectangle(cornerRadius: cornerRadius, style: .continuous))
.hoverEffect(.highlight)
// Light tap on press-down (nil on release so it fires once, on touch). No haptic
// hardware on iPad silently ignored there.
.sensoryFeedback(trigger: configuration.isPressed) { _, pressed in
pressed ? .impact(weight: .light) : nil
}
}
}
#endif
@@ -146,7 +146,7 @@ private struct GameCard: View {
.clipShape(RoundedRectangle(cornerRadius: 10, style: .continuous))
.overlay(alignment: .topLeading) { storeBadge }
Text(game.title)
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.lineLimit(2)
.foregroundStyle(.secondary)
}
@@ -154,7 +154,7 @@ private struct GameCard: View {
private var storeBadge: some View {
Text(game.isCustom ? "Custom" : "Steam")
.font(.caption2.weight(.semibold))
.font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 6)
.padding(.vertical, 3)
.background(.ultraThinMaterial, in: Capsule())
@@ -193,7 +193,7 @@ private struct PosterImage: View {
ZStack {
Rectangle().fill(.quaternary)
Text(title)
.font(.headline)
.font(.geist(17, .semibold, relativeTo: .headline))
.multilineTextAlignment(.center)
.foregroundStyle(.secondary)
.padding(8)
@@ -48,7 +48,7 @@ struct PairSheet: View {
+ "(http://<host>:3000 → Pairing). "
+ "Pairing verifies both sides at once — no fingerprint comparison "
+ "needed.")
.font(.callout)
.font(.geist(16, relativeTo: .callout))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
TVFieldRow(
@@ -59,7 +59,7 @@ struct PairSheet: View {
) { editing = .clientName }
if let errorText {
Text(errorText)
.font(.callout)
.font(.geist(16, relativeTo: .callout))
.foregroundStyle(.red)
}
HStack(spacing: 32) {
@@ -121,13 +121,13 @@ struct PairSheet: View {
+ "(http://<host>:3000 → Pairing). "
+ "Pairing verifies both sides at once — no fingerprint "
+ "comparison needed.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
if let errorText {
Section {
Text(errorText)
.font(.callout)
.font(.geist(16, relativeTo: .callout))
.foregroundStyle(.red)
}
}
@@ -12,8 +12,19 @@ struct PunktfunkClientApp: App {
@NSApplicationDelegateAdaptor(AppDelegate.self) private var appDelegate
#endif
init() {
#if os(iOS)
// Put Geist on the navigation titles before any bar is built.
BrandTheme.apply()
#endif
}
var body: some Scene {
WindowGroup("Punktfunk") {
// Pin the whole app's tint to the brand purple explicitly the asset-catalog accent
// resolution is environment/timing-sensitive and can fall back to system blue. Wraps the
// screenshot harness too, so captured screens are on-brand.
Group {
#if DEBUG
// PUNKTFUNK_SHOT_SCENE=<name> show that single mock-populated screen full-bleed for
// the App Store screenshot capture (tools/screenshots.sh). Normal launch otherwise;
@@ -27,6 +38,11 @@ struct PunktfunkClientApp: App {
ContentView()
#endif
}
.tint(.brand)
// Geist Sans is the app's typeface. This sets the default for unstyled text and the
// form row labels; views that pick an explicit size/weight use `.geist()` directly.
.font(.geist(17, relativeTo: .body))
}
// The Stream menu (Disconnect D, Show/Hide Statistics S) a real menu bar on
// macOS, hardware-keyboard shortcuts on iPad. tvOS has neither.
#if !os(tvOS)
@@ -34,7 +50,10 @@ struct PunktfunkClientApp: App {
#endif
#if os(macOS)
Settings {
// A separate scene `.tint` does not cross scene boundaries, so re-apply the brand
// tint here or the Preferences window falls back to the (unreliable) asset accent.
SettingsView()
.tint(.brand)
}
#endif
}
@@ -103,11 +103,11 @@ private struct ShotSettings: View {
.shadow(radius: 40, y: 16)
}
#elseif os(iOS)
NavigationStack {
SettingsView()
.navigationTitle("Settings")
.navigationBarTitleDisplayMode(.inline)
}
// SettingsView owns its NavigationSplitView (sidebar + detail) and Done button, so it is
// rendered directly a wrapping NavigationStack would nest a split view in a stack. Open
// on General so the shot lands on real controls (iPad: sidebar + General detail; iPhone:
// the General page) instead of the bare category list.
SettingsView(initialCategory: .general)
#else
NavigationStack { SettingsView() }
#endif
@@ -175,10 +175,10 @@ private struct ShotHUD: View {
.foregroundStyle(.secondary)
#if os(macOS)
Text("⌘⎋ releases the mouse")
.font(.caption2).foregroundStyle(.secondary)
.font(.geist(11, relativeTo: .caption2)).foregroundStyle(.secondary)
#elseif os(tvOS)
Text("Press Menu to disconnect")
.font(.caption).foregroundStyle(.secondary)
.font(.geist(12, relativeTo: .caption)).foregroundStyle(.secondary)
#endif
}
.padding(10)
@@ -259,7 +259,7 @@ private struct ShotDesktopFrame: View {
HStack(spacing: 8) {
Image(systemName: "gamecontroller.fill")
Text("Streaming from Battlestation")
.font(.system(.callout, weight: .semibold))
.font(.geist(16, .semibold, relativeTo: .callout))
}
.padding(.horizontal, 14).padding(.vertical, 9)
.glassBackground(Capsule())
@@ -95,6 +95,13 @@ final class SessionModel: ObservableObject {
/// field TOFU is forbidden (rule 3b): the connect refuses rather than offering trust, and
/// the user is routed to PIN pairing by the caller. (A pinned host connects regardless: its
/// stored fingerprint is the trust decision.)
///
/// `requestAccess` is the no-PIN delegated-approval path: open an identified connect the host
/// PARKS until the operator clicks Approve in its console, then admits the SAME connection (no
/// reconnect). The handshake budget is widened to exceed the host's park window, and a
/// successful connect streams directly (the approval IS the trust decision) the caller pins
/// the observed fingerprint as paired. `host.pinnedSHA256`, when set, pins the advertised cert
/// for the wait; nil = trust-on-first-use.
func connect(to host: StoredHost, width: UInt32, height: UInt32, hz: UInt32,
compositor: PunktfunkConnection.Compositor = .auto,
gamepad: PunktfunkConnection.GamepadType = .auto,
@@ -103,7 +110,8 @@ final class SessionModel: ObservableObject {
hdrEnabled: Bool = true,
launchID: String? = nil,
allowTofu: Bool = false,
autoTrust: Bool = false) {
autoTrust: Bool = false,
requestAccess: Bool = false) {
guard phase == .idle else { return }
phase = .connecting
activeHost = host
@@ -121,6 +129,8 @@ final class SessionModel: ObservableObject {
#endif
}()
let hdrCapable = hdrEnabled && displayHDR
// 4:4:4 opt-out (default on); the hardware-decode probe below is the real gate.
let want444 = (UserDefaults.standard.object(forKey: DefaultsKey.enable444) as? Bool) ?? true
Task.detached(priority: .userInitiated) {
// PunktfunkConnection.init blocks on the QUIC handshake keep it off the main
// actor. The persistent identity is presented on every connect so a paired
@@ -130,15 +140,31 @@ final class SessionModel: ObservableObject {
// Advertise 10-bit + HDR10 when enabled: the host upgrades to a BT.2020 PQ Main10 stream
// only for actual HDR content (its own gate); the VideoToolbox/Metal present path is
// HDR-capable (P010 + itur_2100_PQ + EDR). 0 keeps the 8-bit BT.709 SDR stream.
let videoCaps: UInt8 = hdrCapable
var videoCaps: UInt8 = hdrCapable
? (PunktfunkConnection.videoCap10Bit | PunktfunkConnection.videoCapHDR)
: 0
// Advertise full-chroma 4:4:4 only when allowed AND this device can HARDWARE-decode it
// (software 4:4:4 is too slow for real-time). The host content-gates depth, so an
// HDR-advertised session can still receive an 8-bit 4:4:4 stream (SDR content) require
// BOTH depths there. Otherwise a no-op (the host emits 4:4:4 only if it too opted in);
// `chromaFormat` on the connection reflects what was actually resolved.
let canDecode444 =
hdrCapable
? (Stage444Probe.hwDecode444_8bit && Stage444Probe.hwDecode444_10bit)
: Stage444Probe.hwDecode444_8bit
if want444, canDecode444 {
videoCaps |= PunktfunkConnection.videoCap444
}
let result = Result { try PunktfunkConnection(
host: host.address, port: host.port,
width: width, height: height, refreshHz: hz,
pinSHA256: pin, identity: identity, compositor: compositor,
gamepad: gamepad, bitrateKbps: bitrateKbps, videoCaps: videoCaps,
audioChannels: audioChannels, launchID: launchID) }
audioChannels: audioChannels, launchID: launchID,
// Delegated approval: the host holds this connect open until the operator approves
// it (~180 s) outwait that window so a slow approval still lands here. Normal
// connects keep the snappy default.
timeoutMs: requestAccess ? 185_000 : 10_000) }
await MainActor.run { [weak self] in
guard let self else { return }
// The user may have abandoned this attempt (window closed, another host
@@ -152,7 +178,9 @@ final class SessionModel: ObservableObject {
}
switch result {
case .success(let conn):
if pin != nil || autoTrust {
if pin != nil || autoTrust || requestAccess {
// requestAccess: the operator approved this device on the host, so the
// session is trusted stream directly (the caller pins it as paired).
self.connection = conn
self.startStatsTimer()
self.beginStreaming()
@@ -174,6 +202,14 @@ final class SessionModel: ObservableObject {
case .failure:
self.phase = .idle
self.activeHost = nil
if requestAccess {
// The delegated-approval connect ended without being admitted: the
// operator didn't approve it before the host's park window elapsed (or
// the host was unreachable).
self.errorMessage = "\(host.displayName) didn't let this device in. "
+ "Approve it in the host's web console (port 3000 → Pairing), then "
+ "request access again — the request expires after a few minutes."
} else {
self.errorMessage = pin != nil
? "Could not connect to \(host.displayName) — host unreachable, "
+ "not running, its identity no longer matches the pinned "
@@ -188,6 +224,7 @@ final class SessionModel: ObservableObject {
}
}
}
}
/// The user confirmed the fingerprint: returns it for pinning and enters streaming.
func confirmTrust() -> Data? {
@@ -1,10 +1,12 @@
// App settings. The host creates a native virtual output at exactly the chosen size/refresh
// there is no scaling anywhere in the pipeline.
//
// Navigation differs per platform: macOS uses a tabbed preferences window (the sections had
// outgrown one scrolling pane); iOS uses a single grouped Form; tvOS uses a focus-native
// pushed-picker layout. The individual sections (`streamModeSection`, `audioSection`, ) are
// shared across all three so a setting is defined exactly once.
// Navigation differs per platform, but all three group the same categories (General, Display,
// Audio, Controllers, Advanced, About): macOS uses a tabbed preferences window; iOS/iPadOS uses
// an adaptive NavigationSplitView a category sidebar + detail pane on iPad, auto-collapsing to
// a hierarchical push list on iPhone (the system Settings idiom on each); tvOS uses a
// focus-native pushed-picker layout. The individual sections (`streamModeSection`,
// `audioSection`, ) are shared across all three so a setting is defined exactly once.
#if os(macOS)
import AppKit
@@ -21,7 +23,9 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.compositor) private var compositor = 0
@AppStorage(DefaultsKey.gamepadType) private var gamepadType = 0
@AppStorage(DefaultsKey.bitrateKbps) private var bitrateKbps = 0
@AppStorage(DefaultsKey.presenter) private var presenter = "stage1"
@AppStorage(DefaultsKey.presenter) private var presenter = "stage2"
@AppStorage(DefaultsKey.hdrEnabled) private var hdrEnabled = true
@AppStorage(DefaultsKey.enable444) private var enable444 = true
@AppStorage(DefaultsKey.libraryEnabled) private var libraryEnabled = false
@AppStorage(DefaultsKey.fullscreenWhileStreaming) private var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.micEnabled) private var micEnabled = true
@@ -32,6 +36,22 @@ struct SettingsView: View {
#if DEBUG && !os(tvOS)
@State private var showControllerTest = false
#endif
#if os(iOS)
@AppStorage(DefaultsKey.pointerCapture) private var pointerCapture = true
// The sidebar selection drives the detail pane on iPad and the pushed sub-page on iPhone.
// Width class decides the initial value: nil on iPhone (show the category list first),
// General on iPad (a two-column layout should never open with an empty detail).
@Environment(\.horizontalSizeClass) private var horizontalSizeClass
@State private var settingsSelection: SettingsCategory?
// Tracked so the detail can show its own Done whenever the sidebar (and its Done) is off screen
// not just on iPhone, but on any iPad layout that collapses the sidebar to an overlay. Starts
// .doubleColumn so iPad reliably opens with the sidebar (and its Done) visible.
@State private var columnVisibility: NavigationSplitViewVisibility = .doubleColumn
// Sticky once the wheel lands on "Custom", so editing a width/height that briefly equals a
// preset doesn't snap the wheel back off Custom. A stored non-preset value reads as custom even
// when this is false (see `isCustomResolution`), so it survives relaunches without persisting.
@State private var customMode = false
#endif
#if os(macOS)
@AppStorage(DefaultsKey.speakerUID) private var speakerUID = ""
@AppStorage(DefaultsKey.micUID) private var micUID = ""
@@ -39,6 +59,15 @@ struct SettingsView: View {
@State private var inputDevices: [AudioDevice] = []
#endif
#if os(iOS)
/// `initialCategory` is nil in the app (the list opens un-selected on iPhone; iPad lands on
/// General via `onAppear`). The screenshot harness passes an explicit category so the captured
/// shot opens on a real settings page (a populated detail) rather than the bare category list.
init(initialCategory: SettingsCategory? = nil) {
_settingsSelection = State(initialValue: initialCategory)
}
#endif
var body: some View {
#if os(tvOS)
// Native tv pattern: no inline text entry (typing numbers with a remote is
@@ -66,6 +95,7 @@ struct SettingsView: View {
Form {
presenterSection
hdrSection
windowSection
statisticsSection
}
@@ -98,31 +128,124 @@ struct SettingsView: View {
}
.formStyle(.grouped)
.tabItem { Label("Advanced", systemImage: "slider.horizontal.3") }
AcknowledgementsView()
.tabItem { Label("About", systemImage: "info.circle") }
}
.frame(width: 480, height: 460)
}
#endif
// MARK: - iOS: one grouped Form
// MARK: - iOS / iPadOS: adaptive split view
#if os(iOS)
private var iosBody: some View {
Form {
streamModeSection
audioSection
compositorSection
presenterSection
statisticsSection
experimentalSection
controllersSection
NavigationSplitView(columnVisibility: $columnVisibility) {
List(selection: $settingsSelection) {
ForEach(SettingsCategory.allCases) { category in
// On iPhone the split view collapses to a push list, but a selection List
// draws no disclosure indicator of its own add one in compact width for the
// expected drill-in affordance. On iPad the selected row highlights instead, so
// the chevron is omitted there.
HStack {
Label(category.title, systemImage: category.symbol)
if horizontalSizeClass == .compact {
Spacer()
Image(systemName: "chevron.forward")
.font(.footnote.weight(.semibold))
.foregroundStyle(.tertiary)
// Purely a drill-in affordance the row's button trait already
// conveys "opens"; keep it out of the VoiceOver announcement.
.accessibilityHidden(true)
}
}
.tag(category)
}
}
.navigationTitle("Settings")
.toolbar {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
}
}
} detail: {
// NavigationSplitView hosts the detail in its own navigation context (its title bar),
// so no inner NavigationStack that would double the bar on iPad. On iPhone the split
// view collapses to one stack and pushes this when a row is tapped. `?? .general` only
// backs the brief pre-selection window; the list never auto-pushes on a nil selection.
settingsDetail(settingsSelection ?? .general)
// Keep a Done on the detail whenever the sidebar (and its Done) isn't on screen: the
// iPhone push, or any iPad layout that collapsed the sidebar to an overlay. When the
// sidebar is showing, its Done is the only one so this stays hidden to avoid two.
.toolbar {
if horizontalSizeClass == .compact || columnVisibility == .detailOnly {
ToolbarItem(placement: .confirmationAction) {
Button("Done") { dismiss() }
}
}
}
}
.formStyle(.grouped)
.onAppear {
if horizontalSizeClass == .regular, settingsSelection == nil {
settingsSelection = .general
}
gamepads.refresh()
gamepads.startDiscovery()
}
// A regularregular launch sets the default above; this catches a compactregular change
// (e.g. an iPad leaving narrow split-screen multitasking) so the detail pane fills in.
.onChange(of: horizontalSizeClass) { _, newValue in
if newValue == .regular, settingsSelection == nil {
settingsSelection = .general
}
}
.onDisappear { gamepads.stopDiscovery() }
}
@ViewBuilder
private func settingsDetail(_ category: SettingsCategory) -> some View {
switch category {
case .general:
Form {
streamModeSection
pointerSection
compositorSection
}
.formStyle(.grouped)
.navigationTitle("General")
.navigationBarTitleDisplayMode(.inline)
case .display:
Form {
presenterSection
hdrSection
statisticsSection
}
.formStyle(.grouped)
.navigationTitle("Display")
.navigationBarTitleDisplayMode(.inline)
case .audio:
Form { audioSection }
.formStyle(.grouped)
.navigationTitle("Audio")
.navigationBarTitleDisplayMode(.inline)
case .controllers:
Form { controllersSection }
.formStyle(.grouped)
.navigationTitle("Controllers")
.navigationBarTitleDisplayMode(.inline)
case .advanced:
Form { experimentalSection }
.formStyle(.grouped)
.navigationTitle("Advanced")
.navigationBarTitleDisplayMode(.inline)
case .about:
// Already a full scrollable view that sets its own "Acknowledgements" title; pin the
// display mode inline to match the five sibling detail pages (it would otherwise inherit
// the large title from the "Settings" sidebar root).
AcknowledgementsView()
.navigationBarTitleDisplayMode(.inline)
}
}
#endif
// MARK: - tvOS
@@ -150,6 +273,10 @@ struct SettingsView: View {
Binding(get: { hudEnabled ? "on" : "off" }, set: { hudEnabled = $0 == "on" })
}
private var hdrEnabledTag: Binding<String> {
Binding(get: { hdrEnabled ? "on" : "off" }, set: { hdrEnabled = $0 == "on" })
}
private var tvBody: some View {
let currentTag = "\(width)x\(height)x\(hz)"
let bounds = UIScreen.main.nativeBounds
@@ -180,20 +307,25 @@ struct SettingsView: View {
selection: $audioChannels)
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange)
.multilineTextAlignment(.center)
}
TVSelectionRow(
title: "Compositor", options: compositors, selection: $compositor)
#if DEBUG
TVSelectionRow(
title: "Presenter",
options: [("Stage 1 (default)", "stage1"), ("Stage 2 (experimental)", "stage2")],
title: "Presenter (debug)",
options: [("Stage 2 (default)", "stage2"), ("Stage 1 (debug)", "stage1")],
selection: $presenter)
#endif
TVSelectionRow(
title: "10-bit HDR",
options: [("On", "on"), ("Off", "off")], selection: hdrEnabledTag)
Text("The host creates a virtual output at exactly this mode — native "
+ "resolution, no scaling. \(Self.bitrateFooter) A specific compositor "
+ "is honored only if available on the host.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
.padding(.top, 8)
@@ -213,10 +345,12 @@ struct SettingsView: View {
TVSelectionRow(
title: "Controller type", options: Self.padTypes, selection: $gamepadType)
Text(Self.controllersFooter)
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
.padding(.top, 8)
NavigationLink("Acknowledgements") { AcknowledgementsView() }
.padding(.top, 8)
}
.frame(maxWidth: 1000)
.frame(maxWidth: .infinity)
@@ -235,6 +369,63 @@ struct SettingsView: View {
@ViewBuilder private var streamModeSection: some View {
Section {
#if os(iOS)
// Touch-first: a rotating wheel of common resolutions (this device's own mode first) and
// a segmented refresh-rate control the same family as the Clock/Timer pickers. The host
// renders a virtual output at exactly the chosen mode, so these are real pixel sizes. The
// last wheel row, "Custom", reveals width/height/refresh fields for an arbitrary mode.
VStack(alignment: .leading, spacing: 4) {
Text("Resolution")
.font(.geist(15, relativeTo: .subheadline))
.foregroundStyle(.secondary)
Picker("Resolution", selection: resolutionSelection) {
ForEach(resolutionChoices, id: \.tag) { choice in
Text(choice.label).tag(choice.tag)
}
}
.labelsHidden()
.pickerStyle(.wheel)
.frame(maxHeight: 140)
}
if isCustomResolution {
// Arbitrary entry: type the exact width × height (and refresh) the host should drive.
HStack {
TextField("Width", value: $width, format: .number.grouping(.never))
.keyboardType(.numberPad)
Text("×")
TextField("Height", value: $height, format: .number.grouping(.never))
.labelsHidden()
.keyboardType(.numberPad)
}
// A row built from an HStack of TextFields otherwise insets its bottom separator to
// the inner content, clipping the hairline under "Width"; pin it to the cell edge.
.alignmentGuide(.listRowSeparatorLeading) { _ in 0 }
LabeledContent("Refresh rate") {
TextField("Hz", value: $hz, format: .number.grouping(.never))
.keyboardType(.numberPad)
.multilineTextAlignment(.trailing)
}
} else if refreshChoices.count > 1 {
VStack(alignment: .leading, spacing: 6) {
Text("Refresh rate")
.font(.geist(15, relativeTo: .subheadline))
.foregroundStyle(.secondary)
Picker("Refresh rate", selection: $hz) {
ForEach(refreshChoices, id: \.self) { rate in
Text("\(rate) Hz").tag(rate)
}
}
.labelsHidden()
.pickerStyle(.segmented)
}
} else {
// A device with a single supported rate (e.g. 60 Hz) has nothing to pick.
LabeledContent("Refresh rate") {
Text("\(hz) Hz").foregroundStyle(.secondary)
}
}
Button("Use this display's mode") { fillFromMainScreen() }
#elseif os(macOS)
HStack {
TextField("Resolution", value: $width, format: .number.grouping(.never))
Text("×")
@@ -245,6 +436,7 @@ struct SettingsView: View {
LabeledContent("") {
Button("Use this display's mode") { fillFromMainScreen() }
}
#endif
#if !os(tvOS)
Toggle("Automatic bitrate", isOn: automaticBitrate)
if bitrateKbps != 0 {
@@ -259,7 +451,7 @@ struct SettingsView: View {
}
if bitrateKbps > 1_000_000 {
Label(Self.gigabitWarning, systemImage: "exclamationmark.triangle.fill")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.orange)
}
}
@@ -269,11 +461,85 @@ struct SettingsView: View {
} footer: {
Text("The host creates a virtual output at exactly this mode — "
+ "native resolution, no scaling. \(Self.bitrateFooter)")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
#if os(iOS)
// MARK: - Stream mode (iOS wheel)
/// Sentinel wheel tag for the "Custom" row. Real tags are "WxH" (digits + "x"), so this can't
/// collide with a resolution.
private static let customResolutionTag = "custom"
/// 16:9 then ultrawide presets; the device's native mode is prepended at runtime.
private static let resolutionPresets: [(name: String, w: Int, h: Int)] = [
("720p", 1280, 720),
("1080p", 1920, 1080),
("1440p", 2560, 1440),
("4K", 3840, 2160),
("Ultrawide 1080p", 2560, 1080),
("Ultrawide 1440p", 3440, 1440),
("Super ultrawide", 5120, 1440),
]
/// The non-custom wheel rows: this device's native mode first, then the presets, deduped by
/// dimensions (native wins a tie).
private var resolutionModes: [(name: String, w: Int, h: Int)] {
let bounds = UIScreen.main.nativeBounds // portrait-oriented pixels
let native = (w: Int(max(bounds.width, bounds.height)), h: Int(min(bounds.width, bounds.height)))
let all = [(name: "This device", w: native.w, h: native.h)] + Self.resolutionPresets
var seen = Set<String>()
return all.filter { seen.insert("\($0.w)x\($0.h)").inserted }
}
/// Wheel rows: the resolution modes, then a "Custom" row that reveals the numeric fields.
private var resolutionChoices: [(label: String, tag: String)] {
resolutionModes.map { (label: "\($0.name) · \($0.w) × \($0.h)", tag: "\($0.w)x\($0.h)") }
+ [(label: "Custom…", tag: Self.customResolutionTag)]
}
private var presetResolutionTags: Set<String> {
Set(resolutionModes.map { "\($0.w)x\($0.h)" })
}
/// True when the editable custom fields should show: the wheel is parked on "Custom" (sticky),
/// or the stored size simply isn't one of the presets (e.g. a value synced from a Mac) so a
/// non-preset mode stays editable across relaunches without a persisted flag.
private var isCustomResolution: Bool {
customMode || !presetResolutionTags.contains("\(width)x\(height)")
}
/// The wheel works in "WxH" tags so one selection drives both width and height; the custom
/// sentinel toggles `customMode` instead of writing a size.
private var resolutionSelection: Binding<String> {
Binding(
get: { isCustomResolution ? Self.customResolutionTag : "\(width)x\(height)" },
set: { tag in
if tag == Self.customResolutionTag {
customMode = true
return
}
customMode = false
let parts = tag.split(separator: "x").compactMap { Int($0) }
guard parts.count == 2 else { return }
width = parts[0]
height = parts[1]
})
}
/// Refresh rates the device can actually display (no point asking the host to render frames the
/// screen can't show), plus any stored custom value so it stays selectable.
private var refreshChoices: [Int] {
let maxHz = UIScreen.main.maximumFramesPerSecond
var rates = [60, 120, 240].filter { $0 <= maxHz }
if rates.isEmpty { rates = [maxHz] }
if !rates.contains(hz) { rates.append(hz) }
return rates.sorted()
}
#endif
@ViewBuilder private var audioSection: some View {
Section {
Picker("Audio channels", selection: $audioChannels) {
@@ -313,11 +579,35 @@ struct SettingsView: View {
Text("Host audio plays through the speaker; the microphone feeds the "
+ "host's virtual mic. System default follows macOS device changes. "
+ "Applies from the next session.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
#if os(iOS)
/// iPad-only pointer-capture toggle: lock the mouse/trackpad for relative movement (games) vs
/// forward an absolute cursor position (desktop). Empty on iPhone (no hardware-pointer lock
/// the mouse path there is always the absolute fallback).
@ViewBuilder private var pointerSection: some View {
if UIDevice.current.userInterfaceIdiom == .pad {
Section {
Toggle("Capture pointer for games", isOn: $pointerCapture)
} header: {
Text("Pointer")
} footer: {
Text("With a mouse or trackpad connected, lock the pointer and send relative "
+ "movement — the expected behavior for games (mouse-look). Turn this off for "
+ "desktop use to keep the pointer free and send its absolute position instead. "
+ "The lock needs the stream full-screen and frontmost; it falls back to the "
+ "absolute pointer automatically (Stage Manager, Slide Over). Finger touch is "
+ "unaffected. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
}
#endif
@ViewBuilder private var compositorSection: some View {
Section {
Picker("Compositor", selection: $compositor) {
@@ -333,7 +623,7 @@ struct SettingsView: View {
Text("Which compositor drives the virtual output on the host. A specific "
+ "choice is honored only if that backend is available there — "
+ "otherwise the host falls back to auto-detection.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@@ -347,26 +637,50 @@ struct SettingsView: View {
} footer: {
Text("Take the window fullscreen when a session starts and restore it on the host "
+ "list, so only the stream is fullscreen — not the picker.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
// Stage-2 (Metal/VTDecompressionSession) is the default and only user-visible presenter it
// recovers from a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard on a
// lost HEVC reference. Stage-1 is kept reachable as a DEBUG-only override for diagnostics, like
// the controller test. Empty in release builds (no presenter UI; stage-2 always).
@ViewBuilder private var presenterSection: some View {
#if DEBUG
Section {
Picker("Presenter", selection: $presenter) {
Text("Stage 1 (default)").tag("stage1")
Text("Stage 2 (experimental)").tag("stage2")
Text("Stage 2 (default)").tag("stage2")
Text("Stage 1 (debug)").tag("stage1")
}
} header: {
Text("Video presenter")
Text("Video presenter · debug")
} footer: {
Text("Stage 1 feeds compressed video to the system display layer (known-good). "
+ "Stage 2 decodes explicitly and presents through Metal with a display "
+ "link — it adds a capture→present (glass-to-glass) latency line in the HUD "
+ "and shortens the present tail. Applies from the next session.")
.font(.caption)
Text("Stage 2 (default) decodes explicitly and presents through Metal with a display "
+ "link — it adds a capture→present (glass-to-glass) latency line in the HUD and "
+ "self-recovers from decode stalls. Stage 1 feeds compressed video straight to the "
+ "system display layer; it freezes on a lost HEVC reference frame, so it's a debug "
+ "fallback only. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
#endif
}
@ViewBuilder private var hdrSection: some View {
Section {
Toggle("10-bit HDR", isOn: $hdrEnabled)
Toggle("Full chroma (4:4:4)", isOn: $enable444)
} header: {
Text("Video quality")
} footer: {
Text("HDR requests a 10-bit BT.2020 PQ (HDR10) stream — it only engages when the host is "
+ "sending HDR content AND this display supports HDR. 4:4:4 requests full chroma "
+ "(sharper text/UI, more bandwidth) — it only engages when this device can "
+ "hardware-decode it AND the host opted in. Otherwise the stream stays 8-bit "
+ "4:2:0 SDR. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@@ -384,7 +698,7 @@ struct SettingsView: View {
Text("Statistics")
} footer: {
Text(Self.statisticsFooter)
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@@ -399,7 +713,7 @@ struct SettingsView: View {
+ "(Steam + custom) via the host's management API; tap a title to launch it. "
+ "The host must expose that API on the LAN with a token "
+ "(serve --mgmt-bind 0.0.0.0 --mgmt-token …).")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@@ -433,7 +747,7 @@ struct SettingsView: View {
Text("Controllers")
} footer: {
Text(Self.controllersFooter)
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
}
}
@@ -585,13 +899,13 @@ struct SettingsView: View {
}
}
}
.font(.caption2)
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
}
Spacer()
if gamepads.active?.id == controller.id {
Text("In use")
.font(.caption2.weight(.semibold))
.font(.geist(11, .semibold, relativeTo: .caption2))
.padding(.horizontal, 8)
.padding(.vertical, 3)
.background(Capsule().fill(.green.opacity(0.2)))
@@ -613,6 +927,10 @@ struct SettingsView: View {
width = Int(max(bounds.width, bounds.height))
height = Int(min(bounds.width, bounds.height))
hz = UIScreen.main.maximumFramesPerSecond
#if os(iOS)
// The native mode is the "This device" wheel row, so leave Custom mode if it was on.
customMode = false
#endif
#endif
}
}
@@ -623,3 +941,52 @@ extension Double {
Swift.min(Swift.max(self, lo), hi)
}
}
#if os(iOS)
/// The settings groups, mirroring the macOS preference tabs. On iPad each is a sidebar row that
/// drives the detail pane; on iPhone the same list collapses to pushed sub-pages. Internal (not
/// private) so the screenshot harness can open SettingsView on a specific category.
enum SettingsCategory: String, CaseIterable, Identifiable {
case general, display, audio, controllers, advanced, about
var id: Self { self }
var title: String {
switch self {
case .general: return "General"
case .display: return "Display"
case .audio: return "Audio"
case .controllers: return "Controllers"
case .advanced: return "Advanced"
case .about: return "About"
}
}
var symbol: String {
switch self {
case .general: return "gearshape"
case .display: return "display"
case .audio: return "speaker.wave.2"
case .controllers: return "gamecontroller"
case .advanced: return "slider.horizontal.3"
case .about: return "info.circle"
}
}
}
extension View {
/// Present the settings sheet large on iPad so the NavigationSplitView has room for its
/// sidebar + detail a default form sheet is too narrow and the split view would collapse to
/// the iPhone push list. No-op on iPhone (the standard sheet is already right) and on iOS 17
/// (no `presentationSizing` it falls back to the default sheet, which still degrades cleanly
/// to the push list).
@ViewBuilder
func settingsSheetSizing() -> some View {
if UIDevice.current.userInterfaceIdiom == .pad, #available(iOS 18, *) {
presentationSizing(.page)
} else {
self
}
}
}
#endif
@@ -52,7 +52,7 @@ struct SpeedTestSheet: View {
var body: some View {
VStack(spacing: 20) {
Label("Speed test — \(host.displayName)", systemImage: "gauge.with.needle")
.font(.headline)
.font(.geist(17, .semibold, relativeTo: .headline))
.foregroundStyle(.tint)
switch phase {
@@ -73,7 +73,7 @@ struct SpeedTestSheet: View {
resultView(result)
case .failed(let message):
Text(message)
.font(.callout)
.font(.geist(16, relativeTo: .callout))
.foregroundStyle(.red)
.multilineTextAlignment(.center)
}
@@ -149,13 +149,13 @@ struct SpeedTestSheet: View {
if let rec = Self.recommendedKbps(result) {
Text("Recommended bitrate: \(Self.mbpsLabel(kbps: rec)) "
+ "(~70% of measured, headroom for encoder bursts).")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
} else {
Text("Too little data made it through to recommend a bitrate — "
+ "check the network and retry.")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
}
@@ -69,19 +69,19 @@ struct StreamHUDView: View {
Text(model.mouseCaptured
? "⌘⎋ releases the mouse"
: "Click the stream to capture input")
.font(.caption2)
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
// The client-side cursor (C) draws the local cursor over the stream instead of
// capturing it the only accurate cursor for gamescope, whose capture has none.
Text("⌘⇧C toggles the on-screen cursor")
.font(.caption2)
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
#elseif os(iOS)
// Touch always plays directly; (hardware keyboard) toggles kb/mouse.
Text(model.mouseCaptured
? "⌘⎋ releases keyboard & mouse"
: "⌘⎋ captures keyboard & mouse")
.font(.caption2)
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
#endif
#if os(tvOS)
@@ -89,13 +89,13 @@ struct StreamHUDView: View {
// A press (the focus engine consumes it before the host sees it). Disconnect is
// the Siri Remote's Menu button (.onExitCommand on the stream) just hint it.
Text("Press Menu to disconnect")
.font(.caption)
.font(.geist(12, relativeTo: .caption))
.foregroundStyle(.secondary)
#else
// D lives on the app's Stream menu (so it still works when the HUD is hidden);
// this button is the in-overlay, click-to-disconnect affordance.
Button("Disconnect (⌘D)") { model.disconnect() }
.font(.caption)
.font(.geist(12, relativeTo: .caption))
#endif
}
.padding(10)
@@ -3,6 +3,7 @@
// or drops this and runs the PIN pairing ceremony instead.
import Foundation
import PunktfunkKit
import SwiftUI
struct TrustCardView: View {
@@ -18,11 +19,11 @@ struct TrustCardView: View {
.font(.system(size: 36, weight: .light))
.foregroundStyle(.tint)
Text("Verify \(hostName)")
.font(.title3.weight(.semibold))
.font(.geist(20, .semibold, relativeTo: .title3))
Text("First connection. Compare this fingerprint with the one "
+ "punktfunk-host logged at startup (\u{201C}clients pin this "
+ "fingerprint\u{201D}):")
.font(.callout)
.font(.geist(16, relativeTo: .callout))
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
Text(Self.format(fingerprint: fingerprint))
@@ -58,7 +59,7 @@ struct TrustCardView: View {
#else
.buttonStyle(.borderless)
#endif
.font(.callout)
.font(.geist(16, relativeTo: .callout))
}
.padding(28)
.frame(maxWidth: 440)
@@ -0,0 +1,101 @@
// Geist the punktfunk brand typeface (the same family the website ships). Bundled as static
// OTF weights in this kit's resources and registered with Core Text at first use, so it works
// identically in the Xcode app and the `swift run` dev shell (Bundle.module resolves to the
// package resource bundle in both). Geist Sans carries titles/UI; Geist Mono carries the technical
// readouts host addresses, status labels, the stream-stats HUD for the industrial look.
//
// Licensed under the SIL Open Font License 1.1 (Resources/Fonts/Geist-OFL.txt).
import CoreText
import SwiftUI
#if canImport(UIKit)
import UIKit
#elseif canImport(AppKit)
import AppKit
#endif
public enum BrandFont {
public enum Weight {
case regular, medium, semibold, bold
}
/// PostScript names of the bundled faces (verified from each OTF's name table). Geist Sans only
/// Geist Mono is intentionally not shipped; the app's typeface is Geist Sans throughout.
private static let sansFaces = ["Geist-Regular", "Geist-Medium", "Geist-SemiBold", "Geist-Bold"]
/// Registered exactly once per process a static `let` initializer is run lazily and is
/// guaranteed thread-safe + run-at-most-once by the runtime.
private static let registered: Void = {
for face in sansFaces {
guard let url = Bundle.module.url(
forResource: face, withExtension: "otf", subdirectory: "Fonts") else {
#if DEBUG
print("BrandFont: bundled face \(face).otf not found — text will fall back to system")
#endif
continue
}
var error: Unmanaged<CFError>?
if !CTFontManagerRegisterFontsForURL(url as CFURL, .process, &error) {
#if DEBUG
let message = error?.takeRetainedValue().localizedDescription ?? "unknown error"
print("BrandFont: failed to register \(face): \(message)")
#endif
}
}
}()
/// Force registration before the first `Font.custom` lookup. Cheap to call repeatedly.
public static func registerIfNeeded() { _ = registered }
fileprivate static func sansFace(_ weight: Weight) -> String {
switch weight {
case .regular: return "Geist-Regular"
case .medium: return "Geist-Medium"
case .semibold: return "Geist-SemiBold"
case .bold: return "Geist-Bold"
}
}
}
public extension Color {
/// The punktfunk brand purple (the app-icon lens / website `--brand`). Defined explicitly,
/// independent of the asset-catalog accent `Color.accentColor` resolution is environment- and
/// timing-sensitive (it can fall back to system blue), and the brand mark must never drift.
/// Light: #6656F2, Dark: #8678F5 (the lighter violet reads better on dark surfaces).
static let brand: Color = {
#if canImport(UIKit)
return Color(UIColor { traits in
traits.userInterfaceStyle == .dark
? UIColor(red: 0x86 / 255, green: 0x78 / 255, blue: 0xF5 / 255, alpha: 1)
: UIColor(red: 0x66 / 255, green: 0x56 / 255, blue: 0xF2 / 255, alpha: 1)
})
#elseif canImport(AppKit)
return Color(NSColor(name: nil) { appearance in
appearance.bestMatch(from: [.aqua, .darkAqua]) == .darkAqua
? NSColor(red: 0x86 / 255, green: 0x78 / 255, blue: 0xF5 / 255, alpha: 1)
: NSColor(red: 0x66 / 255, green: 0x56 / 255, blue: 0xF2 / 255, alpha: 1)
})
#else
// Non-Apple fallback: the light brand value, so all branches agree on a canonical color.
return Color(red: 0x66 / 255, green: 0x56 / 255, blue: 0xF2 / 255)
#endif
}()
}
public extension Font {
/// Geist Sans at an explicit point size, scaling with Dynamic Type relative to `textStyle`.
static func geist(
_ size: CGFloat, _ weight: BrandFont.Weight = .regular,
relativeTo textStyle: TextStyle = .body
) -> Font {
BrandFont.registerIfNeeded()
return .custom(BrandFont.sansFace(weight), size: size, relativeTo: textStyle)
}
/// Geist Sans at a FIXED point size that does not scale with Dynamic Type for glyphs pinned
/// inside a fixed-size container (e.g. the monogram tile), where a scaled letter would overflow.
static func geistFixed(_ size: CGFloat, _ weight: BrandFont.Weight = .regular) -> Font {
BrandFont.registerIfNeeded()
return .custom(BrandFont.sansFace(weight), fixedSize: size)
}
}
@@ -22,9 +22,22 @@ public enum DefaultsKey {
public static let speakerUID = "punktfunk.speakerUID"
public static let micUID = "punktfunk.micUID"
public static let presenter = "punktfunk.presenter"
/// Request a 10-bit BT.2020 PQ (HDR10) stream. On by default; only takes effect when the host
/// has HDR content AND this display supports HDR otherwise the stream stays 8-bit SDR.
public static let hdrEnabled = "punktfunk.hdrEnabled"
/// Request a full-chroma 4:4:4 stream when this device can HARDWARE-decode it (`Stage444Probe`).
/// On by default; only takes effect when the host also opted in to 4:4:4 (otherwise the stream
/// stays 4:2:0). Sharper text/UI at the cost of more bandwidth.
public static let enable444 = "punktfunk.enable444"
public static let hosts = "punktfunk.hosts"
/// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never".
public static let cursorMode = "punktfunk.cursorMode"
/// iPad: capture the mouse/trackpad pointer (pointer lock relative movement) for games,
/// rather than forwarding an absolute cursor position. On by default. Only meaningful on iPad
/// with a hardware mouse/trackpad; the system grants the lock only to a full-screen, frontmost
/// scene and silently falls back to the absolute pointer when it can't (Stage Manager / Slide
/// Over). Read by `StreamViewController.prefersPointerLocked`.
public static let pointerCapture = "punktfunk.pointerCapture"
/// Experimental: show the host's game library (browsed over the management API). Off by default.
public static let libraryEnabled = "punktfunk.libraryEnabled"
/// macOS: take the window fullscreen while streaming and restore it on the host list. On by default.
@@ -68,6 +68,14 @@ private final class RumbleRenderer: @unchecked Sendable {
private var broken = false
/// Last logged active/silent state for a one-line transition log, not per-event spam.
private var wasActive = false
// Backoff after an engine failure. A broken `gamecontrollerd.haptics` XPC connection (CoreHaptics
// -4811 "server connection broke") fails EVERY rebuild until the service relaunches and that
// break fires neither stoppedHandler nor resetHandler, so without a cooldown the next rumble
// update immediately rebuilds into the same dead connection, flooding the log and never
// recovering. Delay the next setup() growing 0.5124 s on repeated failure and clear it
// the moment a player runs cleanly (or the controller changes).
private var retryAfter = Date.distantPast
private var consecutiveFailures = 0
/// CHHapticEvent sharpness = actuator frequency. A DualSense's voice-coil motors need a
/// defined frequency to move at all an intensity-only event (no sharpness) left them
@@ -91,6 +99,8 @@ private final class RumbleRenderer: @unchecked Sendable {
self.closeHID()
self.controller = c
self.broken = false
self.consecutiveFailures = 0
self.retryAfter = .distantPast
_ = self.openHIDIfDualSense(c)
onBackend?(self.backendNote(for: c))
}
@@ -108,7 +118,7 @@ private final class RumbleRenderer: @unchecked Sendable {
// other pad (and for a DualSense whose HID device could not be opened).
if self.hidRumble(low: lowAmp, high: highAmp) { return }
guard !self.broken else { return }
if active, self.low == nil, self.high == nil {
if active, self.low == nil, self.high == nil, Date() >= self.retryAfter {
self.setup()
}
let ok: Bool
@@ -124,8 +134,15 @@ private final class RumbleRenderer: @unchecked Sendable {
}
// Rebuild on the next nonzero amplitude if an engine errored and tear down OUTSIDE
// the `inout` accesses above, so teardown() never mutates a motor that a `drive` call
// still holds an exclusive reference to.
if !ok { self.teardown() }
// still holds an exclusive reference to. Back off so a broken XPC isn't re-hit every
// update; once a player is actually running the path has recovered, so clear the backoff.
if !ok {
self.teardown()
self.scheduleRetryBackoff()
} else if self.low?.player != nil || self.high?.player != nil {
self.consecutiveFailures = 0
self.retryAfter = .distantPast
}
}
}
@@ -157,14 +174,29 @@ private final class RumbleRenderer: @unchecked Sendable {
low = makeMotor(haptics, .default)
}
if low == nil, high == nil {
// Haptics present but no engine could be built right now (server busy / a transient
// error). Do NOT latch broken the next nonzero amplitude retries setup().
log.warning("rumble: haptics present but engine setup failed — will retry on next rumble")
// Haptics present but no engine could be built right now (server busy / XPC broken). Do
// NOT latch broken back off and the next nonzero amplitude past the cooldown retries.
log.warning("rumble: haptics present but engine setup failed — backing off, will retry")
scheduleRetryBackoff()
}
}
/// Push the next engine-build attempt out after a failure (capped exponential backoff), so a
/// broken `gamecontrollerd.haptics` connection gets time to relaunch instead of being re-hit on
/// every rumble update.
private func scheduleRetryBackoff() {
consecutiveFailures += 1
let shift = min(consecutiveFailures - 1, 4)
retryAfter = Date().addingTimeInterval(min(0.5 * Double(1 << shift), 4))
}
private func makeMotor(_ haptics: GCDeviceHaptics, _ locality: GCHapticsLocality) -> Motor? {
guard let engine = haptics.createEngine(withLocality: locality) else { return nil }
// A controller's motors carry no audio, so keep this engine OUT of the app's audio session
// (the default is to join it). Streaming keeps an AVAudioSession active the whole time;
// letting a haptics-only engine join it is a needless coupling that can get its
// gamecontrollerd XPC connection interrupted (the repeated -4811 server-connection breaks).
engine.playsHapticsOnly = true
// The haptic server can stop or reset the engine out from under us app backgrounding, an
// audio-session interruption (a call, Siri, another audio app), or a server crash. Left
// unhandled the players go dead and every later rumble throws, latching rumble off for the
@@ -338,29 +370,32 @@ public final class GamepadFeedback {
// Hidout traffic (lightbar / player LEDs / triggers) only exists on a PlayStation-pad
// session a DualSense or a DualShock 4 (lightbar only). Block briefly on it there and
// let rumble own the wait elsewhere; on an Xbox session it stays nonblocking.
let hasHidout = connection.resolvedGamepad == .dualSense
|| connection.resolvedGamepad == .dualShock4
let hidTimeout: UInt32 = hasHidout ? 10 : 0
let thread = Thread { [connection, flag, drainDone, weak self] in
while !flag.isStopped {
do {
if let r = try connection.nextRumble(timeoutMs: 10), r.pad == 0 {
// Poll the feedback planes NON-BLOCKING. A blocking poll (timeoutMs > 0) holds
// the connection's shared feedback lock for its whole wait; the video pump drains
// HDR mastering metadata (nextHdrMeta) on the SAME lock every frame, so a blocking
// poll here starved it and throttled HDR to ~1 fps (SDR, which never drains HDR
// meta, was unaffected). Pacing with a short sleep OUTSIDE the lock (below) keeps
// rumble/HID latency low while leaving the lock free between polls.
if let r = try connection.nextRumble(timeoutMs: 0), r.pad == 0 {
self?.rumble.apply(low: r.low, high: r.high)
}
// Drain a BOUNDED burst of hidout events: only the first poll waits,
// and the cap + stop check keep sustained 0xCD traffic (a game writing
// per-frame LED/trigger reports) from starving the rumble poll above
// or blocking stop() past one cycle.
// Drain a BOUNDED burst of hidout events so sustained 0xCD traffic (a game writing
// per-frame LED/trigger reports) can't spin here or block stop() past one cycle.
var burst = 0
while burst < 64, !flag.isStopped,
let ev = try connection.nextHidOutput(
timeoutMs: burst == 0 ? hidTimeout : 0) {
let ev = try connection.nextHidOutput(timeoutMs: 0) {
self?.render(ev)
burst += 1
}
} catch {
break // .closed (or fatal) the session is over
}
// ~8 ms poll cadence (125 Hz), slept OUTSIDE the feedback lock low rumble/HID
// latency without holding the lock the HDR-meta drain needs.
if !flag.isStopped { Thread.sleep(forTimeInterval: 0.008) }
}
drainDone.signal()
}
@@ -107,6 +107,23 @@ public final class InputCapture {
/// macOS (no GCMouse handlers installed; `sendMouseAbs` is never called there). Main-queue.
public var gcMouseForwarding = false
#if os(iOS)
/// Whether any device is attached as a `GCMouse` right now. The Magic Keyboard TRACKPAD does
/// not always register as a GCMouse on iPadOS (only a standalone mouse does) when no GCMouse
/// is present the relative GCMouse path can't carry pointer motion. Main-queue.
public var hasGCMouse: Bool { !mice.isEmpty }
/// Diagnostic: a one-line description of every attached GCMouse (count + GCDevice identity), so
/// PUNKTFUNK_INPUT_DEBUG can reveal whether the trackpad showed up as a mouse at all.
public var attachedMiceSummary: String {
guard !mice.isEmpty else { return "0 mice" }
let parts = mice.map { mouse -> String in
"\(mouse.productCategory)/\(mouse.vendorName ?? "?")"
}
return "\(mice.count) mice: \(parts.joined(separator: ", "))"
}
#endif
/// Fired on (the capture toggle detected here so it works in both states; the
/// event itself is swallowed). Main queue.
public var onToggleCapture: (() -> Void)?
@@ -394,6 +411,12 @@ public final class InputCapture {
!mice.contains(where: { $0 === mouse }) // re-delivered on wake attach once
else { return }
mice.append(mouse)
#if os(iOS)
if inputDebug {
inputLog.debug(
"GCMouse attached: \(mouse.productCategory, privacy: .public)/\(mouse.vendorName ?? "?", privacy: .public) — now \(self.attachedMiceSummary, privacy: .public)")
}
#endif
// macOS drives motion + buttons from NSEvent (StreamLayerView's local monitor
// sendMotion/sendMouseButton) because GCMouse's handlers proved unreliable there;
// installing them too would double-send. iOS keeps GCMouse (raw deltas under
@@ -0,0 +1,61 @@
import Foundation
/// Open-source license / attribution text bundled with PunktfunkKit (see `Resources/`).
///
/// Exposed from the kit so the app shell can show an Acknowledgements screen. The text files are
/// bundled as SwiftPM resources and read via `Bundle.module`, which works both for `swift build`
/// and for the Xcode app (it links the PunktfunkKit product, so the resource bundle rides along).
public enum Licenses {
private static func resource(_ name: String) -> String {
guard let url = Bundle.module.url(forResource: name, withExtension: "txt"),
let text = try? String(contentsOf: url, encoding: .utf8)
else { return "" }
return text
}
/// punktfunk's own license MIT OR Apache-2.0, at your option.
public static var appLicense: String {
let mit = resource("LICENSE-MIT")
let apache = resource("LICENSE-APACHE")
if mit.isEmpty && apache.isEmpty {
return "punktfunk is licensed under MIT OR Apache-2.0, at your option."
}
return "punktfunk is licensed under MIT OR Apache-2.0, at your option.\n\n"
+ "================================ MIT ================================\n\n"
+ mit
+ "\n\n============================== Apache-2.0 ==============================\n\n"
+ apache
}
/// The bundled brand typeface (Geist Sans + Geist Mono) SIL Open Font License 1.1. The
/// license file ships alongside the OTFs in `Resources/Fonts/`, satisfying the OFL's
/// distribution requirement; this surfaces it in the Acknowledgements screen too.
public static var fontLicense: String {
guard let url = Bundle.module.url(
forResource: "Geist-OFL", withExtension: "txt", subdirectory: "Fonts"),
let text = try? String(contentsOf: url, encoding: .utf8)
else { return "" }
return text
}
/// Third-party software notices for the linked Rust crates (generated by
/// `scripts/gen-third-party-notices.sh`).
public static var thirdPartyNotices: String {
let text = resource("THIRD-PARTY-NOTICES")
return text.isEmpty ? "Third-party notices unavailable." : text
}
/// `thirdPartyNotices` pre-split into render-sized line chunks. The full notices are ~885 KB /
/// 16k lines; a single SwiftUI `Text` that large overshoots CoreText/CoreAnimation's max
/// renderable height it lays out for ages and draws blank past the limit so the
/// Acknowledgements screen renders these chunks in a `LazyVStack` (only on-screen chunks lay
/// out, and no chunk is tall enough to clip). Split at line boundaries and joined with "\n";
/// the inter-chunk break is the `LazyVStack` row boundary, so no text is lost. Computed once.
public static let thirdPartyNoticesChunks: [String] = {
let lines = thirdPartyNotices.split(separator: "\n", omittingEmptySubsequences: false)
let chunkSize = 200
return stride(from: 0, to: lines.count, by: chunkSize).map { start in
lines[start..<min(start + chunkSize, lines.count)].joined(separator: "\n")
}
}()
}
@@ -1,21 +1,35 @@
// Stage-2 presenter, present half: draw a decoded NV12 CVPixelBuffer into a CAMetalLayer
// drawable with a BT.709 YUVRGB shader. The display link (owned by the hosting view) drives
// `render` once per vsync with the target present time, so a present can finally be stamped and
// the present tail hand-paced. See docs apple-stage2-presenter.md.
// Stage-2 presenter, present half: draw a decoded NV12 / P010 / 4:4:4 CVPixelBuffer into a CAMetalLayer
// drawable with a YCbCrRGB shader. The hosting view's CADisplayLink drives `render` once per vsync
// (via Stage2Pipeline.renderTick) with the target present time, so a present can be stamped and the
// present tail hand-paced. See docs apple-stage2-presenter.md.
//
// Main-thread only: created during view setup, `render` called from the view's CADisplayLink
// (which fires on the main runloop). The Metal objects + texture cache are touched only here.
// Main-thread only: created during view setup, `render`/`configure` called from the view's CADisplayLink
// (which fires on the main runloop). The Metal objects + texture cache are touched only here. The one
// exception is `setHdrMeta`, called from the pump thread it hops the layer write to main so every
// CALayer mutation stays on one thread.
#if canImport(Metal) && canImport(QuartzCore)
import CoreGraphics
import CoreVideo
import Metal
import QuartzCore
import os
/// Runtime-compiled (no metallib build step needed in SwiftPM): a fullscreen triangle and a
/// BT.709 limited-range NV12RGB fragment shader. uv.y is flipped (1 - p.y) so the top-left-
/// origin texture presents upright (NDC y is up), not upside down. (Colorspace is BT.709 SDR
/// for now matches the host; 10-bit/HDR + other matrices are a later tie-in.)
private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter")
/// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the
/// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors
/// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the
/// brighter highlights into the panel's headroom. This is the missing anchor that made the old HDR path
/// render "way too bright" (no `edrMetadata` no reference-white anchoring); a LARGER value renders
/// dimmer. Matches the host's standard PQ reference white.
private let hdrReferenceWhiteNits: Float = 203.0
/// Runtime-compiled (no metallib build step needed in SwiftPM): a fullscreen triangle and BT.709 SDR
/// and BT.2020-PQ HDR YCbCrRGB fragment shaders. uv.y is flipped (1 - p.y) so the top-left-origin
/// texture presents upright (NDC y is up). The HDR shader outputs PQ-encoded RGB as-is the
/// CAMetalLayer's `itur_2100_PQ` colour space + `edrMetadata` tell the system compositor the samples
/// are PQ and how to tone-map them (no EOTF here, matching the host's BT.2020 PQ emission).
private let shaderSource = """
#include <metal_stdlib>
using namespace metal;
@@ -30,11 +44,46 @@ vertex VOut pf_vtx(uint vid [[vertex_id]]) {
return o;
}
// Bicubic (Catmull-Rom) sampling of the single-channel luma plane. When the drawable is larger
// than the decoded frame (a window/view bigger than the host's fixed mode), a bilinear upscale
// looks soft; Catmull-Rom keeps edges crisp — matching AVSampleBufferDisplayLayer's (stage-1)
// scaler — and reduces to the exact texel at 1:1, so a native-resolution present stays pixel-exact.
// Nine bilinear taps (TheRealMJP's optimisation of the 16-tap kernel); `s` MUST be a linear
// sampler. Luma carries the perceived detail, so only it gets bicubic; chroma stays bilinear.
float catmullRomLuma(texture2d<float> tex, sampler s, float2 uv) {
float2 texSize = float2(tex.get_width(), tex.get_height());
float2 samplePos = uv * texSize;
float2 tc1 = floor(samplePos - 0.5) + 0.5;
float2 f = samplePos - tc1;
float2 w0 = f * (-0.5 + f * (1.0 - 0.5 * f));
float2 w1 = 1.0 + f * f * (-2.5 + 1.5 * f);
float2 w2 = f * (0.5 + f * (2.0 - 1.5 * f));
float2 w3 = f * f * (-0.5 + 0.5 * f);
float2 w12 = w1 + w2;
float2 off12 = w2 / w12;
float2 tc0 = (tc1 - 1.0) / texSize;
float2 tc3 = (tc1 + 2.0) / texSize;
float2 tc12 = (tc1 + off12) / texSize;
float r = 0.0;
r += tex.sample(s, float2(tc0.x, tc0.y)).r * (w0.x * w0.y);
r += tex.sample(s, float2(tc12.x, tc0.y)).r * (w12.x * w0.y);
r += tex.sample(s, float2(tc3.x, tc0.y)).r * (w3.x * w0.y);
r += tex.sample(s, float2(tc0.x, tc12.y)).r * (w0.x * w12.y);
r += tex.sample(s, float2(tc12.x, tc12.y)).r * (w12.x * w12.y);
r += tex.sample(s, float2(tc3.x, tc12.y)).r * (w3.x * w12.y);
r += tex.sample(s, float2(tc0.x, tc3.y)).r * (w0.x * w3.y);
r += tex.sample(s, float2(tc12.x, tc3.y)).r * (w12.x * w3.y);
r += tex.sample(s, float2(tc3.x, tc3.y)).r * (w3.x * w3.y);
return r;
}
// SDR: 8-bit NV12 / 4:4:4 (BT.709, limited/video range) → full-range RGB. Chroma is sampled at the
// same UV as luma, so a full-size 4:4:4 chroma plane needs no shader change vs 4:2:0.
fragment float4 pf_frag(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> chromaTex [[texture(1)]]) {
constexpr sampler s(filter::linear, address::clamp_to_edge);
float y = lumaTex.sample(s, in.uv).r;
float y = catmullRomLuma(lumaTex, s, in.uv);
float2 c = chromaTex.sample(s, in.uv).rg;
// BT.709, 8-bit limited (video) range → full-range RGB.
y = (y - 16.0/255.0) * (255.0/219.0);
@@ -46,18 +95,18 @@ fragment float4 pf_frag(VOut in [[stage_in]],
return float4(saturate(float3(r, g, b)), 1.0);
}
// HDR: 10-bit P010 (BT.2020, limited range), Y'CbCr that is PQ-encoded. We apply the BT.2020
// matrix to get PQ-encoded R'G'B' and output it as-is — the CAMetalLayer's itur_2100_PQ colour
// space + EDR tells the compositor the samples are PQ, so it does the PQ→display mapping. No EOTF
// here (matching the host, which emitted BT.2020 PQ). P010 stores the 10-bit code in the high bits
// of each 16-bit sample, so an .r16Unorm sample reads ~code/1023 (the /1024 vs /1023 error is < 0.1%).
// HDR: 10-bit P010 / 4:4:4 (BT.2020, limited range), YCbCr that is PQ-encoded. We apply the BT.2020
// matrix to get PQ-encoded RGB and output it as-is — the CAMetalLayer's itur_2100_PQ colour space
// + edrMetadata tell the compositor the samples are PQ, so it does the PQ→display tone-map. No EOTF
// here. P010/x444 store the 10-bit code in the high bits of each 16-bit sample, so an .r16Unorm sample
// reads ~code/1023 (the /1024 vs /1023 error is < 0.1%).
fragment float4 pf_frag_hdr(VOut in [[stage_in]],
texture2d<float> lumaTex [[texture(0)]],
texture2d<float> chromaTex [[texture(1)]]) {
constexpr sampler s(filter::linear, address::clamp_to_edge);
float y = lumaTex.sample(s, in.uv).r;
float y = catmullRomLuma(lumaTex, s, in.uv);
float2 c = chromaTex.sample(s, in.uv).rg;
// BT.2020 10-bit limited (video) range → full-range PQ R'G'B'.
// BT.2020 10-bit limited (video) range → full-range PQ RGB.
y = (y - 64.0/1023.0) * (1023.0/876.0);
float u = (c.x - 512.0/1023.0) * (1023.0/896.0);
float v = (c.y - 512.0/1023.0) * (1023.0/896.0);
@@ -74,21 +123,34 @@ public final class MetalVideoPresenter {
private let device: MTLDevice
private let queue: MTLCommandQueue
/// SDR (BT.709 8-bit NV12 bgra8) and HDR (BT.2020 PQ 10-bit P010 rgba16Float) pipelines.
/// Selected per frame by `render`; the layer is reconfigured when the mode flips (HDR toggle).
/// SDR (BT.709 8-bit bgra8) and HDR (BT.2020 PQ 10-bit rgba16Float) pipelines. Selected per
/// frame in `render`; the layer is reconfigured to match when the session flips (HDR toggle).
private let pipelineSDR: MTLRenderPipelineState
private let pipelineHDR: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache?
/// Current layer configuration switched lazily in `configure(hdr:)` when a frame's mode differs.
private var hdrActive = false
/// nil if Metal is unavailable (no GPU / a headless CI) the caller falls back to stage-1.
public init?() {
/// Current layer configuration switched in `configure(hdr:)` when a frame's HDR-ness differs.
/// Main-thread only (read + written from `render`/`configure`, all on the display-link runloop).
private var hdrActive = false
/// Last HDR mastering grade received via `setHdrMeta` (the host's 0xCE). Cached so a mid-session
/// SDRHDR flip's `configureColor` re-applies the real grade instead of clobbering it back to the
/// bare reference-white anchor (an out-of-order race otherwise: `setHdrMeta` and the flip both write
/// `edrMetadata`). Main-thread only.
private var lastHdrMeta: PunktfunkConnection.HdrMeta?
#if DEBUG
/// Last logged "decodeddrawable" signature, so the diagnostic logs only on a size/HDR change.
private var lastSizeSig = ""
#endif
/// nil if Metal is unavailable (no GPU / a headless CI) or a shader fails to compile the caller
/// falls back to stage-1.
public static func make() -> MetalVideoPresenter? {
guard let device = MTLCreateSystemDefaultDevice(),
let queue = device.makeCommandQueue()
else { return nil }
self.device = device
self.queue = queue
let pipelineSDR: MTLRenderPipelineState
let pipelineHDR: MTLRenderPipelineState
do {
let library = try device.makeLibrary(source: shaderSource, options: nil)
let vtx = library.makeFunction(name: "pf_vtx")
@@ -105,76 +167,137 @@ public final class MetalVideoPresenter {
} catch {
return nil
}
CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, device, nil, &textureCache)
guard textureCache != nil else { return nil }
var cache: CVMetalTextureCache?
CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, device, nil, &cache)
guard let textureCache = cache else { return nil }
let layer = CAMetalLayer()
layer.device = device
layer.pixelFormat = .bgra8Unorm
layer.framebufferOnly = true
layer.isOpaque = true
// Triple-buffer: more in-flight drawables before `nextDrawable()` (called on the
// display-link / MAIN thread) has to block waiting for one to free.
layer.maximumDrawableCount = 3
#if os(macOS)
// The display link already paces exactly one present per vsync. Leaving the layer's
// own vsync wait on means `commandBuffer.present` ALSO blocks for the hardware vsync,
// so `nextDrawable()` stalls the MAIN thread until a drawable frees windowed, the
// WindowServer's looser compositing hides it; FULLSCREEN's tighter, more-direct path
// serializes the main thread to the display and the stall surfaces as bad judder.
// Disabling the layer-level sync lets present return promptly (the display link is the
// pacing source), which is what fixes the fullscreen stutter. macOS-only property.
// The display link already paces exactly one present per vsync. Leaving the layer's own vsync
// wait on means `commandBuffer.present` ALSO blocks for the hardware vsync, so `nextDrawable()`
// stalls the MAIN thread until a drawable frees windowed, the WindowServer's looser
// compositing hides it; FULLSCREEN's tighter path serializes the main thread to the display and
// the stall surfaces as bad judder. Disabling the layer-level sync lets present return promptly
// (the display link is the pacing source) the fix for the fullscreen stutter. macOS-only.
layer.displaySyncEnabled = false
#endif
// Render the drawable at the DECODED frame's resolution (set per-frame in `render`) and let the
// system compositor scale it to the layer's bounds the same `.resizeAspect` path stage-1's
// AVSampleBufferDisplayLayer uses. A native-resolution present is then pixel-exact (1:1, no
// shader scaling); a resized window rescales via the system's scaler.
layer.contentsGravity = .resizeAspect
// Triple-buffer: more in-flight drawables before `nextDrawable()` (called on the display-link /
// MAIN thread) has to block waiting for one to free.
layer.maximumDrawableCount = 3
return MetalVideoPresenter(
device: device, queue: queue, pipelineSDR: pipelineSDR, pipelineHDR: pipelineHDR,
textureCache: textureCache, layer: layer)
}
private init(
device: MTLDevice, queue: MTLCommandQueue, pipelineSDR: MTLRenderPipelineState,
pipelineHDR: MTLRenderPipelineState, textureCache: CVMetalTextureCache, layer: CAMetalLayer
) {
self.device = device
self.queue = queue
self.pipelineSDR = pipelineSDR
self.pipelineHDR = pipelineHDR
self.textureCache = textureCache
self.layer = layer
}
/// Track the stream mode (the host can Reconfigure mid-stream). Size is in pixels.
public func setDrawableSize(_ size: CGSize) {
guard size.width > 0, size.height > 0 else { return }
if layer.drawableSize != size { layer.drawableSize = size }
}
/// Reconfigure the layer for SDR or HDR when the stream mode flips (HDR toggle). HDR uses an
/// rgba16Float drawable + a BT.2020 PQ colour space + EDR, so the compositor PQ-maps to the
/// display; SDR uses the plain 8-bit sRGB path. Main-thread only (called from `render`).
private func configure(hdr: Bool) {
/// Configure the layer + active pipeline for an SDR or HDR session. MAIN THREAD ONLY. Called once at
/// session start and again per-frame from `render` (idempotent the guard makes a same-state call a
/// no-op), so a mid-session HDR toggle (the host re-inits its encoder; the decoded `frame.isHDR`
/// flips) reconfigures here automatically. HDR uses an rgba16Float drawable + BT.2020 PQ colour space
/// + EDR with a 203-nit reference-white anchor; SDR uses the plain 8-bit sRGB path.
public func configure(hdr: Bool) {
guard hdr != hdrActive else { return }
hdrActive = hdr
configureColor(hdr: hdr)
}
/// Set the layer's pixel format + colour config for SDR or HDR. MAIN THREAD ONLY. EDR is requested
/// on ALL platforms the property is available on macOS/iOS/tvOS at our deployment floor, and the
/// old `#if os(macOS)` guard left iOS/tvOS EDR half-engaged.
private func configureColor(hdr: Bool) {
if hdr {
layer.pixelFormat = .rgba16Float
layer.colorspace = CGColorSpace(name: CGColorSpace.itur_2100_PQ)
#if os(macOS)
layer.wantsExtendedDynamicRangeContent = true
#endif
// Anchor reference white. Re-apply the real grade if one already arrived (0xCE before the
// flip); otherwise the bare 203-nit anchor. Without this anchor the PQ signal is too bright.
layer.edrMetadata = makeEDR(lastHdrMeta)
} else {
// SDR: gamma-encoded BT.709 [0,1] in an 8-bit drawable; a nil colorspace tags it device/sRGB
// (the proven SDR path never showed the "too bright" issue, which was HDR-only).
layer.pixelFormat = .bgra8Unorm
layer.colorspace = nil
#if os(macOS)
layer.wantsExtendedDynamicRangeContent = false
#endif
layer.edrMetadata = nil
}
}
/// Draw one decoded frame to the next drawable and present it. `isHDR` selects the 10-bit
/// BT.2020 PQ path (P010 input) vs the 8-bit BT.709 path (NV12 input). Returns true on success;
/// false when there's no drawable yet, a texture couldn't be made, or Metal errored the
/// caller then doesn't stamp a present for this frame.
private func makeEDR(_ meta: PunktfunkConnection.HdrMeta?) -> CAEDRMetadata {
CAEDRMetadata.hdr10(
displayInfo: meta?.masteringDisplayColorVolume(),
contentInfo: meta?.contentLightLevelInfo(),
opticalOutputScale: hdrReferenceWhiteNits)
}
/// Update the HDR mastering metadata (drained from the host's 0xCE datagram) to refine the system
/// tone-map from the real grade. Called from the PUMP thread, so the layer write is hopped to MAIN
/// (every CALayer mutation stays on one thread). The grade is cached so a later SDRHDR
/// `configureColor` re-applies it; the `edrMetadata` write is gated on `hdrActive` (setting it on an
/// SDR layer is harmless but pointless, and the flip will apply it anyway).
public func setHdrMeta(_ meta: PunktfunkConnection.HdrMeta) {
DispatchQueue.main.async { [weak self] in
guard let self else { return }
self.lastHdrMeta = meta
if self.hdrActive { self.layer.edrMetadata = self.makeEDR(meta) }
}
}
/// Draw one decoded frame to the next drawable and present it. MAIN THREAD (the display link).
/// `isHDR` selects the 10-bit BT.2020 PQ path vs the 8-bit BT.709 path and is reconciled with the
/// layer config via `configure`. Returns true on success; false when there's no drawable yet, a
/// texture couldn't be made, or Metal errored the caller then doesn't stamp a present.
@discardableResult
public func render(_ pixelBuffer: CVPixelBuffer, isHDR: Bool = false) -> Bool {
// Reconcile the layer with the decoded frame's HDR-ness (handles a mid-session SDRHDR flip).
configure(hdr: isHDR)
// P010 stores 10-bit luma/chroma in 16-bit samples R16/RG16; NV12 is 8-bit R8/RG8.
let lumaFmt: MTLPixelFormat = isHDR ? .r16Unorm : .r8Unorm
let chromaFmt: MTLPixelFormat = isHDR ? .rg16Unorm : .rg8Unorm
// P010/x444 store 10-bit luma/chroma in 16-bit samples R16/RG16; NV12/444v is 8-bit R8/RG8.
// Derived from the actual decoded buffer so a 4:4:4 (full chroma plane) frame just works.
let pf = CVPixelBufferGetPixelFormatType(pixelBuffer)
let tenBit =
pf == kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange
|| pf == kCVPixelFormatType_420YpCbCr10BiPlanarFullRange
|| pf == kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange
|| pf == kCVPixelFormatType_444YpCbCr10BiPlanarFullRange
guard let textureCache,
let luma = makeTexture(pixelBuffer, plane: 0, format: lumaFmt, cache: textureCache),
let chroma = makeTexture(pixelBuffer, plane: 1, format: chromaFmt, cache: textureCache)
let luma = makeTexture(
pixelBuffer, plane: 0, format: tenBit ? .r16Unorm : .r8Unorm, cache: textureCache),
let chroma = makeTexture(
pixelBuffer, plane: 1, format: tenBit ? .rg16Unorm : .rg8Unorm, cache: textureCache)
else { return false }
// The hosting view owns drawableSize (aspect-fit to its bounds); skip until it's laid
// out. The fullscreen triangle scales the decoded texture to fill the drawable.
guard layer.drawableSize.width > 0, layer.drawableSize.height > 0,
let drawable = layer.nextDrawable(),
// Size the drawable to the decoded frame so the fullscreen triangle samples 1:1 (pixel-exact);
// the layer's contentsGravity then scales it to the on-screen bounds via the system compositor
// (matching stage-1). drawableSize does NOT track bounds (defaults to 0), so set it BEFORE
// nextDrawable; re-set only on a change (first frame / Reconfigure / HDR flip).
let decodedSize = CGSize(
width: CVPixelBufferGetWidth(pixelBuffer), height: CVPixelBufferGetHeight(pixelBuffer))
if layer.drawableSize != decodedSize { layer.drawableSize = decodedSize }
#if DEBUG
logSizeIfChanged(decoded: decodedSize)
#endif
guard let drawable = layer.nextDrawable(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
@@ -186,24 +309,23 @@ public final class MetalVideoPresenter {
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(isHDR ? pipelineHDR : pipelineSDR)
encoder.setRenderPipelineState(hdrActive ? pipelineHDR : pipelineSDR)
encoder.setFragmentTexture(CVMetalTextureGetTexture(luma), index: 0)
encoder.setFragmentTexture(CVMetalTextureGetTexture(chroma), index: 1)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
commandBuffer.present(drawable) // present at the next vsync lowest latency
// Hold the CVMetalTextures + the source pixel buffer (its IOSurface) alive until the GPU
// finishes sampling releasing them at scope exit could free the backing mid-read.
// Hold the CVMetalTextures + source pixel buffer (its IOSurface) alive until the GPU finishes
// sampling releasing them at scope exit could free the backing mid-read.
commandBuffer.addCompletedHandler { _ in _ = (luma, chroma, pixelBuffer) }
commandBuffer.commit()
return true
}
/// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past
/// the draw the MTLTexture is only valid while its CVMetalTexture is retained.
/// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the
/// draw the MTLTexture is only valid while its CVMetalTexture is retained.
private func makeTexture(
_ pixelBuffer: CVPixelBuffer, plane: Int, format: MTLPixelFormat,
cache: CVMetalTextureCache
_ pixelBuffer: CVPixelBuffer, plane: Int, format: MTLPixelFormat, cache: CVMetalTextureCache
) -> CVMetalTexture? {
let w = CVPixelBufferGetWidthOfPlane(pixelBuffer, plane)
let h = CVPixelBufferGetHeightOfPlane(pixelBuffer, plane)
@@ -215,5 +337,16 @@ public final class MetalVideoPresenter {
else { return nil }
return cvTexture
}
#if DEBUG
private func logSizeIfChanged(decoded: CGSize) {
let sig = "\(Int(decoded.width))x\(Int(decoded.height))|hdr\(hdrActive ? 1 : 0)"
if sig != lastSizeSig {
lastSizeSig = sig
let msg = "stage2: decoded \(Int(decoded.width))x\(Int(decoded.height)) hdr=\(hdrActive)"
presenterLog.info("\(msg, privacy: .public)")
}
}
#endif
}
#endif
@@ -0,0 +1,94 @@
// Steers the system's iPad pointer-lock resolution down to a chosen "anchor" view controller.
//
// `UIViewController.prefersPointerLocked` is resolved the same way as the status bar: the system
// walks DOWN from the window's root view controller through `childViewControllerForPointerLock`.
// SwiftUI's hosting / container view controllers do NOT forward that query to their children, so a
// `UIViewControllerRepresentable` controller buried in the SwiftUI tree (our StreamViewController)
// is never consulted its `prefersPointerLocked = true` is silently ignored and a Magic Keyboard
// trackpad / mouse falls through to the absolute-pointer path instead of being captured.
//
// Swizzling the DEFAULT implementation isn't enough: the controllers that break the chain
// (UIHostingController and SwiftUI's internal containers) provide their OWN implementation of the
// property, so a base-class swizzle never runs for them. Instead we walk UP the LIVE `parent`
// chain from the anchor to the window root and, on each real ancestor, force
// `childViewControllerForPointerLock` to return the next controller toward the anchor. Each forced
// value is a genuine direct child (we follow the actual containment chain), so the system's
// downward walk reaches the anchor and reads its `prefersPointerLocked`.
//
// The forcing is per-INSTANCE an associated object gated behind a one-time per-CLASS IMP
// swizzle. Only the specific controllers in the anchor's chain are affected; every other instance
// of those classes keeps its original behavior (associated object nil original IMP). The forced
// values are cleared on disengage so the long-lived SwiftUI parents don't retain a stale controller
// across sessions. Only the PUBLIC `childViewControllerForPointerLock` selector is touched
// (App-Store-safe; no private API).
#if os(iOS)
import ObjectiveC
import UIKit
enum PointerLockChain {
private static var forcedChildKey: UInt8 = 0
/// Classes whose `childViewControllerForPointerLock` we've already IMP-swizzled (keyed by the
/// class object). Main-thread only pointer-lock resolution and capture toggles are all main.
private static var swizzledClasses = Set<ObjectIdentifier>()
/// Ancestors we've stamped with a forced child this engagement, held weakly so a deallocated
/// SwiftUI controller drops out on its own (no dangling). disengage() clears every one even
/// if the live `parent` chain has since broken so a stamped parent can never retain a stale
/// controller subtree across sessions. One anchor is ever engaged at a time.
private static let stampedParents = NSHashTable<UIViewController>.weakObjects()
private static func forcedChild(of vc: UIViewController) -> UIViewController? {
objc_getAssociatedObject(vc, &forcedChildKey) as? UIViewController
}
private static func setForcedChild(_ child: UIViewController?, on vc: UIViewController) {
// RETAIN: while steering, the parent must keep the toward-anchor child alive. It's also
// already a strong child of `vc` via UIKit containment, so this adds no cycle (the reverse
// `.parent` link is weak), and disengage() always clears it so it can't outlive a session.
objc_setAssociatedObject(vc, &forcedChildKey, child, .OBJC_ASSOCIATION_RETAIN_NONATOMIC)
}
/// Ensure `cls`'s `childViewControllerForPointerLock` getter consults the per-instance forced
/// child first, falling back to the class's original implementation. Idempotent per class.
private static func ensureSwizzled(_ cls: AnyClass) {
let id = ObjectIdentifier(cls)
guard !swizzledClasses.contains(id) else { return }
swizzledClasses.insert(id)
let selector = #selector(getter: UIViewController.childViewControllerForPointerLock)
guard let method = class_getInstanceMethod(cls, selector) else { return }
let originalIMP = method_getImplementation(method)
typealias OriginalFn = @convention(c) (AnyObject, Selector) -> UIViewController?
let original = unsafeBitCast(originalIMP, to: OriginalFn.self)
let forwarding: @convention(block) (UIViewController) -> UIViewController? = { vc in
if let forced = forcedChild(of: vc) { return forced }
return original(vc, selector)
}
method_setImplementation(method, imp_implementationWithBlock(forwarding))
}
/// Force every ancestor of `anchor` to forward pointer-lock resolution toward it, then ask the
/// system to re-resolve. No-op when `anchor` isn't in a view-controller hierarchy yet (it
/// re-runs from the anchor's appearance/parent callbacks once it is).
static func engage(_ anchor: UIViewController) {
disengage(anchor) // clear any prior engagement first (reparent / re-anchor)
var child = anchor
while let parent = child.parent {
ensureSwizzled(object_getClass(parent)!)
setForcedChild(child, on: parent)
stampedParents.add(parent)
child = parent
}
anchor.setNeedsUpdateOfPrefersPointerLocked()
}
/// Clear the forced forwarding on every stamped ancestor (so the SwiftUI parents stop retaining
/// the anchor's subtree) and re-resolve to drop the lock.
static func disengage(_ anchor: UIViewController) {
for parent in stampedParents.allObjects {
setForcedChild(nil, on: parent)
}
stampedParents.removeAllObjects()
anchor.setNeedsUpdateOfPrefersPointerLocked()
}
}
#endif
@@ -0,0 +1,36 @@
// Synthetic 4:4:4 HEVC keyframes used only by `Stage444Probe` to probe decode capability.
//
// Each is the first IDR access unit (VPS + SPS + PPS + IDR slice, Annex-B) of a 256×256 HEVC
// Range-Extensions clip `chroma_format_idc = 3` generated offline with libx265:
// ffmpeg -f lavfi -i color=c=gray:s=256x256:r=30:d=0.1 -frames:v 3 \
// -pix_fmt yuv444p[10le] -c:v libx265 \
// -x265-params keyint=1:min-keyint=1:no-info=1:repeat-headers=1:aud=0 -f hevc out.hevc
// 256×256 clears the hardware decoder's minimum-dimension floor (a 16×16 clip is rejected for every
// chroma format). Validated to hardware-decode to `444v`/`x444` on Apple Silicon (M3).
enum Probe444Blobs {
/// 256×256 HEVC Range-Extensions 4:4:4 keyframe (Annex-B): 134 bytes.
static let au444_8bit: [UInt8] = [
0x00, 0x00, 0x00, 0x01, 0x40, 0x01, 0x0c, 0x01, 0xff, 0xff, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00,
0x9e, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c, 0xba, 0x02, 0x40, 0x00, 0x00, 0x00, 0x01, 0x42,
0x01, 0x01, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00, 0x9e, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c,
0x90, 0x01, 0x01, 0x00, 0x80, 0xb2, 0xdd, 0x49, 0x26, 0x57, 0x80, 0xb4, 0x04, 0x00, 0x00, 0x03,
0x00, 0x04, 0x00, 0x00, 0x03, 0x00, 0x78, 0x20, 0x00, 0x00, 0x00, 0x01, 0x44, 0x01, 0xc1, 0x72,
0x86, 0x0c, 0x06, 0x24, 0x00, 0x00, 0x00, 0x01, 0x28, 0x01, 0xaf, 0x72, 0x15, 0xe8, 0x34, 0xeb,
0xae, 0xfb, 0xfe, 0x75, 0x57, 0xca, 0xc1, 0x71, 0x43, 0x16, 0xf5, 0xc2, 0x40, 0xbd, 0x80, 0xa6,
0x65, 0x35, 0x20, 0x28, 0x81, 0xa2, 0x5e, 0xc0, 0x93, 0x04, 0x10, 0x9b, 0x00, 0x34, 0xe0, 0x87,
0x00, 0x00, 0x03, 0x00, 0x5b, 0x40,
]
/// 256×256 HEVC Range-Extensions 4:4:4 10-bit keyframe (Annex-B): 133 bytes.
static let au444_10bit: [UInt8] = [
0x00, 0x00, 0x00, 0x01, 0x40, 0x01, 0x0c, 0x01, 0xff, 0xff, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00,
0x9c, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c, 0xba, 0x02, 0x40, 0x00, 0x00, 0x00, 0x01, 0x42,
0x01, 0x01, 0x04, 0x08, 0x00, 0x00, 0x03, 0x00, 0x9c, 0x28, 0x00, 0x00, 0x03, 0x00, 0x00, 0x3c,
0x90, 0x01, 0x01, 0x00, 0x80, 0x9b, 0x2d, 0xd4, 0x92, 0x65, 0x78, 0x0b, 0x40, 0x40, 0x00, 0x00,
0x03, 0x00, 0x40, 0x00, 0x00, 0x07, 0x82, 0x00, 0x00, 0x00, 0x01, 0x44, 0x01, 0xc1, 0x72, 0x86,
0x0c, 0x06, 0x24, 0x00, 0x00, 0x00, 0x01, 0x28, 0x01, 0xaf, 0x72, 0x15, 0xe8, 0x34, 0xeb, 0xae,
0xfb, 0xfe, 0x75, 0x57, 0xca, 0xc1, 0x71, 0x43, 0x16, 0xf5, 0xc2, 0x40, 0xbd, 0x80, 0xa6, 0x65,
0x35, 0x20, 0x28, 0x81, 0xa2, 0x5e, 0xc0, 0x93, 0x04, 0x10, 0x9b, 0x00, 0x34, 0xe0, 0x87, 0x00,
0x00, 0x03, 0x00, 0x5b, 0x40,
]
}
@@ -182,6 +182,11 @@ public final class PunktfunkConnection {
case dualSense = 2
case xboxOne = 3
case dualShock4 = 4
// Valve Steam Controller / Steam Deck (Linux UHID hid-steam hosts). Parity only on Apple
// GameController never surfaces a 0x28DE HID device, so the client can't capture one; these
// exist so the resolved type round-trips and name parsing matches the host.
case steamController = 5
case steamDeck = 6
/// Loose name parsing for env/dev hooks, mirroring the host's
/// `GamepadPref::from_name`.
@@ -192,6 +197,8 @@ public final class PunktfunkConnection {
case "dualsense", "ds", "ds5", "ps5": self = .dualSense
case "xboxone", "xbox-one", "xboxseries", "series": self = .xboxOne
case "dualshock4", "dualshock", "ds4", "ps4": self = .dualShock4
case "steamdeck", "steam-deck", "deck": self = .steamDeck
case "steamcontroller", "steam-controller", "steamcon": self = .steamController
default: return nil
}
}
@@ -231,6 +238,13 @@ public final class PunktfunkConnection {
public private(set) var colorFullRange: Bool = false
/// Encoded bit depth (8 or 10).
public private(set) var bitDepth: UInt8 = 8
/// The chroma subsampling the host resolved for this session, as the HEVC `chroma_format_idc`:
/// `1` = 4:2:0 (every pre-4:4:4 host, and the back-compat default) or `3` = full-chroma 4:4:4
/// (only when this client advertised `videoCap444` *and* the host could open a real 4:4:4
/// encoder). Drive the decoder's requested pixel format from this. See `isChroma444`.
public private(set) var chromaFormat: UInt8 = 1
/// Convenience: the resolved stream is full-chroma 4:4:4 (`chroma_format_idc == 3`).
public var isChroma444: Bool { chromaFormat == 3 }
/// True when the negotiated stream is HDR (PQ or HLG transfer) drive an HDR present path and
/// drain `nextHdrMeta`.
public var isHDR: Bool { colorTransfer == 16 || colorTransfer == 18 }
@@ -327,6 +341,9 @@ public final class PunktfunkConnection {
colorMatrix = mtx
colorFullRange = fullRange != 0
bitDepth = depth
var cf: UInt8 = 1
_ = punktfunk_connection_chroma_format(handle, &cf)
chromaFormat = cf
var ac: UInt8 = 2
_ = punktfunk_connection_audio_channels(handle, &ac)
resolvedAudioChannels = ac
@@ -598,6 +615,10 @@ public final class PunktfunkConnection {
public static let videoCap10Bit: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_10BIT)
/// Video-capability bit: the client can present BT.2020 PQ HDR10 (implies 10-bit).
public static let videoCapHDR: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_HDR)
/// Video-capability bit: the client can decode a full-chroma 4:4:4 HEVC stream (Range
/// Extensions). Advertise only when the device can *hardware*-decode it (`Stage444Probe`);
/// the host then emits 4:4:4 only if it too opted in. `chromaFormat` reflects the real value.
public static let videoCap444: UInt8 = UInt8(PUNKTFUNK_VIDEO_CAP_444)
/// Static HDR mastering metadata (SMPTE ST.2086 + content light level) the host sent for an HDR
/// session. Mirrors the wire/ABI `PunktfunkHdrMeta`; primaries are in ST.2086 **G, B, R** order,
@@ -0,0 +1,93 @@
Copyright 2024 The Geist Project Authors (https://github.com/vercel/geist-font)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
https://openfontlicense.org
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or Derivative
Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2026 unom
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 unom
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
File diff suppressed because it is too large Load Diff
@@ -177,6 +177,16 @@ public final class SessionAudio {
private var playbackEngine: AVAudioEngine?
private var captureEngine: AVAudioEngine?
private var drainStarted = false
#if !os(macOS)
/// AVAudioSession `setCategory`/`setActive` are synchronous and block on the audio server, so
/// they must not run on the main thread (UI stall AVFoundation warns about it). PROCESS-WIDE
/// (static) so every SessionAudio shares one serial queue: the AVAudioSession is a process
/// singleton, and across a reconnect the old session's deactivate must be ordered before the
/// new session's activate (a per-instance queue would let them race and leave the new session's
/// audio deactivated). stop() enqueues its deactivate promptly so it lands before the next
/// session's activate.
private static let sessionQueue = DispatchQueue(label: "io.unom.punktfunk.audio.session")
#endif
public init(connection: PunktfunkConnection) {
self.connection = connection
@@ -189,37 +199,60 @@ public final class SessionAudio {
flag.stop()
}
/// Start playback (and, if enabled+authorized, the mic uplink). Empty UIDs = system
/// default device; on iOS the UIDs are ignored entirely (routes are
/// AVAudioSession-managed). Main thread (engine setup); returns after the engines
/// start the mic may start slightly later if the permission prompt is pending.
/// Start playback (and, if enabled+authorized, the mic uplink). Empty UIDs = system default
/// device; on iOS the UIDs are ignored entirely (routes are AVAudioSession-managed). On macOS
/// the engines start synchronously on the caller's (main) thread. On iOS/tvOS start() is
/// ASYNCHRONOUS: it activates the AVAudioSession off the main thread, then starts the engines on
/// a later main-queue hop (gated by `!flag.isStopped`) so playback is live shortly after, not
/// on return. The mic may start later still if the permission prompt is pending.
public func start(speakerUID: String, micUID: String, micEnabled: Bool) {
#if os(iOS)
// Route + policy live in the session, not per-engine: stereo playback, mic
// capture when enabled, Bluetooth allowed. Failure is non-fatal (defaults).
#if os(macOS)
// No AVAudioSession on macOS start the engines directly (caller's thread, as before).
startEngines(speakerUID: speakerUID, micUID: micUID, micEnabled: micEnabled)
#else
// Configure + activate the session OFF the main thread (it blocks on the audio server),
// then start the engines back on the main thread once it's active engine routing/format
// depend on the active session. A stop() racing in between is caught by the flag guard.
Self.sessionQueue.async { [weak self] in
guard let self else { return }
self.activateAudioSession(micEnabled: micEnabled)
DispatchQueue.main.async { [weak self] in
guard let self, !self.flag.isStopped else { return }
self.startEngines(speakerUID: speakerUID, micUID: micUID, micEnabled: micEnabled)
}
}
#endif
}
#if !os(macOS)
/// Route + policy live in the session, not per-engine: stereo playback, mic capture when
/// enabled, Bluetooth allowed. Failure is non-fatal (defaults). Runs on `sessionQueue`.
private func activateAudioSession(micEnabled: Bool) {
let session = AVAudioSession.sharedInstance()
do {
#if os(iOS)
if micEnabled {
// .defaultToSpeaker: .playAndRecord otherwise routes to the iPhone
// EARPIECE; only affects the built-in route (headphones/BT still win).
// .defaultToSpeaker: .playAndRecord otherwise routes to the iPhone EARPIECE; only
// affects the built-in route (headphones/BT still win).
try session.setCategory(
.playAndRecord, mode: .default,
options: [.allowBluetoothA2DP, .defaultToSpeaker])
} else {
try session.setCategory(.playback, mode: .default)
}
#else // tvOS no app-accessible mic
try session.setCategory(.playback, mode: .default)
#endif
try session.setActive(true)
} catch {
log.warning("AVAudioSession setup failed: \(error.localizedDescription)")
}
#elseif os(tvOS)
do {
try AVAudioSession.sharedInstance().setCategory(.playback, mode: .default)
try AVAudioSession.sharedInstance().setActive(true)
} catch {
log.warning("AVAudioSession setup failed: \(error.localizedDescription)")
}
#endif
/// Build + start the playback engine (and the mic uplink when enabled + authorized). Main
/// thread (engine setup); on iOS/tvOS the session is already active by the time this runs.
private func startEngines(speakerUID: String, micUID: String, micEnabled: Bool) {
startPlayback(speakerUID: speakerUID)
#if os(tvOS)
// No app-accessible microphone input on tvOS playback only.
@@ -258,19 +291,24 @@ public final class SessionAudio {
capture.stop()
}
playback?.stop()
if wasDraining {
_ = drainDone.wait(timeout: .now() + .milliseconds(400))
}
#if !os(macOS)
// Release the session so audio we interrupted (Music, podcasts) gets its
// resume cue.
// Release the session so audio we interrupted (Music, podcasts) gets its resume cue. Like
// activation, setActive is synchronous/blocking run it on the shared serial session queue
// (off the main thread). Enqueued HERE engines already stopped, and BEFORE the drain wait
// below so across a reconnect it lands ahead of the next session's activate on the shared
// queue (otherwise a deferred deactivate could deactivate the new session). Fire-and-forget.
Self.sessionQueue.async {
do {
try AVAudioSession.sharedInstance().setActive(
false, options: .notifyOthersOnDeactivation)
} catch {
log.warning("AVAudioSession deactivation failed: \(error.localizedDescription)")
}
}
#endif
if wasDraining {
_ = drainDone.wait(timeout: .now() + .milliseconds(400))
}
}
// MARK: - Playback (host speaker)
@@ -1,21 +1,21 @@
// Stage-2 presenter orchestrator: a pump thread pulls AUs VideoDecoder; the decoder's async
// output drops the newest decoded frame into a 1-slot ring; the hosting view's display link
// calls `renderTick` once per vsync to draw + present the newest ready frame and stamp
// capturepresent. Mirrors StreamPump's lifecycle (one per start; cancel is permanent).
// Stage-2 presenter orchestrator: a pump thread pulls AUs VideoDecoder; the decoder's async output
// drops the newest decoded frame into a 1-slot ring; the hosting view's display link calls `renderTick`
// once per vsync to draw + present the newest ready frame and stamp capturepresent. Mirrors
// StreamPump's lifecycle (one per start; cancel is permanent).
//
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; `renderTick`
// + `setDrawableSize` + `start`/`stop` on the MAIN thread (the view's CADisplayLink fires there).
// Only the ring + decoder cross threads and both are internally locked.
// Threading: the pump runs on its own thread; the decoder callback on a VT thread; `renderTick` +
// `start`/`stop` on the MAIN thread (the view's CADisplayLink fires there). Only the ring (lock-guarded)
// and the decoder/presenter (internally locked / main-hopped) cross threads.
#if canImport(Metal) && canImport(QuartzCore)
import AVFoundation
import Foundation
import QuartzCore
/// Weak-target wrapper for CADisplayLink. The link retains its target, so targeting a view
/// directly makes a `view link view` cycle that only `invalidate()` breaks if a teardown
/// is ever missed the view leaks and keeps ticking. This proxy holds the handler weakly, so the
/// view can deallocate and its `deinit` invalidate the link.
/// Weak-target wrapper for CADisplayLink. The link retains its target, so targeting a view directly
/// makes a `view link view` cycle that only `invalidate()` breaks if a teardown is ever missed
/// the view leaks and keeps ticking. This proxy holds the handler weakly, so the view can deallocate
/// and its `deinit` invalidate the link.
public final class DisplayLinkProxy: NSObject {
private let onTick: (CADisplayLink) -> Void
public init(_ onTick: @escaping (CADisplayLink) -> Void) { self.onTick = onTick }
@@ -44,10 +44,10 @@ private final class PumpToken: @unchecked Sendable {
func cancel() { lock.lock(); live = false; lock.unlock() }
}
/// Throttled host keyframe requests for decode recovery. The decoder's async error callback
/// (a VT thread) and the pump thread (a submit failure) both signal a wedge; this coalesces
/// them so the control stream isn't flooded while the decode stays stalled for several frames
/// until the requested IDR lands. Bound to the live connection in `start`, unbound in `stop`.
/// Throttled host keyframe requests for decode recovery. The decoder's async error callback (a VT
/// thread) and the pump thread (a submit failure) both signal a wedge; this coalesces them so the
/// control stream isn't flooded while the decode stays stalled for several frames until the requested
/// IDR lands. Bound to the live connection in `start`, unbound in `stop`.
private final class KeyframeRecovery: @unchecked Sendable {
private let lock = NSLock()
private var connection: PunktfunkConnection?
@@ -60,7 +60,7 @@ private final class KeyframeRecovery: @unchecked Sendable {
func request() {
lock.lock()
let now = DispatchTime.now().uptimeNanoseconds
let due = lastNs == 0 || now &- lastNs > 250_000_000 // 250 ms since the last request
let due = lastNs == 0 || now &- lastNs > 100_000_000 // 100 ms since the last request
if due { lastNs = now }
let conn = due ? connection : nil
lock.unlock()
@@ -76,30 +76,36 @@ public final class Stage2Pipeline {
private let recovery = KeyframeRecovery()
private var token = PumpToken()
private var offsetNs: Int64 = 0
/// Signalled when the pump thread exits, so `stop()` can join it (bounded) before `decoder.reset()`
/// otherwise a pump iteration already past its `token.isLive` check can rebuild a decode session
/// right after the reset (a brief orphan session). `pumpJoinable` is armed by `start`, consumed by
/// the first `stop` (so the idempotent second `stop`/deinit doesn't block on an already-drained
/// semaphore). start/stop are sequential lifecycle calls, so the plain flag is safe.
private let pumpStopped = DispatchSemaphore(value: 0)
private var pumpJoinable = false
/// The Metal layer the hosting view installs + sizes. nil-init fails when Metal is
/// unavailable so the caller can fall back to stage-1.
/// The Metal layer the hosting view installs + sizes.
public var layer: CAMetalLayer { presenter.layer }
/// `presentMeter` records capturepresent (the glass-to-glass term). Returns nil if Metal
/// can't be set up (headless / no GPU) caller falls back to the stage-1 presenter.
/// `presentMeter` records capturepresent (the glass-to-glass term). Returns nil if Metal can't be
/// set up (headless / no GPU) caller falls back to the stage-1 presenter.
public init?(presentMeter: LatencyMeter) {
guard let presenter = MetalVideoPresenter() else { return nil }
guard let presenter = MetalVideoPresenter.make() else { return nil }
self.presenter = presenter
self.presentMeter = presentMeter
let ring = ring
let recovery = recovery
self.decoder = VideoDecoder(
onDecoded: { ring.submit($0) },
// Async decode failure (a bad P-frame referencing a lost/corrupt IDR): the pump
// resets to re-gate on the next IDR, and we ask the host to send one now (infinite
// GOP it wouldn't otherwise come soon). Throttled in KeyframeRecovery.
// Async decode failure (a bad P-frame referencing a lost/corrupt IDR): the pump resets to
// re-gate on the next IDR, and we ask the host to send one now (infinite GOP it wouldn't
// otherwise come soon). Throttled in KeyframeRecovery.
onDecodeError: { _ in recovery.request() })
}
/// Start pulling AUs into the decoder. `onFrame` fires per AU at receipt (captureclient
/// meter, exactly as stage-1); `onSessionEnd` on close. `clockOffsetNs` (host minus client)
/// makes the present stamp cross-machine valid.
/// Start pulling AUs into the decoder. MAIN THREAD. `onFrame` fires per AU at receipt (captureclient
/// meter, exactly as stage-1); `onSessionEnd` on close. `clockOffsetNs` (host minus client) makes the
/// present stamp cross-machine valid.
public func start(
connection: PunktfunkConnection,
onFrame: (@Sendable (AccessUnit) -> Void)?,
@@ -108,43 +114,70 @@ public final class Stage2Pipeline {
offsetNs = connection.clockOffsetNs
recovery.bind(connection) // arm host-keyframe recovery for this session
token = PumpToken() // fresh token per start cancel is permanent (like StreamPump)
// Configure the decoder's chroma + the layer's initial colorimetry before the first frame. The
// chroma subsampling drives only the decode pixel format (orthogonal to HDR/depth); the HDR
// config is the Welcome's latched value, which a mid-session flip then overrides per-frame.
decoder.setChroma444(connection.isChroma444)
presenter.configure(hdr: connection.isHDR)
let token = token
let decoder = decoder
let recovery = recovery
let presenter = presenter
let pumpStopped = pumpStopped
let thread = Thread {
defer { pumpStopped.signal() } // let stop() join the pump (bounded) before decoder.reset()
var format: CMVideoFormatDescription?
var lastFramesDropped = connection.framesDropped()
// Persistent recovery WANT, not a one-shot edge (see StreamPump for the full rationale):
// keep asking until an IDR lands so a request swallowed by the throttle is re-sent.
var awaitingIDR = false
// 4:4:4 backstop: a run of decode/create failures in a 4:4:4 session means this device can't
// decode 4:4:4 at the negotiated resolution (the HW probe clears the common case but not a
// resolution-ceiling miss). End cleanly instead of looping on a black screen.
var decodeFailRun = 0
while token.isLive {
do {
// Loss recovery (the primary recovery path). The reassembler drops unrecoverable
// AUs (framesDropped) and the decoder then conceals the reference-missing delta
// frames that follow often rendering them WITHOUT an error callback so the
// onDecodeError trigger rarely fires after a real network blip. Ask the host for
// a fresh IDR whenever the drop count climbs (throttled in KeyframeRecovery).
// Polled every iteration so a total-loss drought recovers the moment packets
// resume and the reassembler counts the gap.
// Loss recovery (the primary path). The reassembler drops unrecoverable AUs and the
// decoder conceals the reference-missing deltas often WITHOUT an error callback
// so key off the drop count climbing, then keep asking (awaitingIDR) until a fresh
// IDR re-anchors decode.
let dropped = connection.framesDropped()
if dropped > lastFramesDropped {
lastFramesDropped = dropped
recovery.request()
awaitingIDR = true
}
// Drain any HDR mastering-metadata update (0xCE) and hand it to the decoder, which
// attaches it to subsequent HDR frames. Non-blocking; only HDR sessions emit these.
if connection.isHDR, let meta = try? connection.nextHdrMeta(timeoutMs: 0) {
decoder.setHdrMeta(meta)
if awaitingIDR { recovery.request() }
// Drain HDR mastering metadata (0xCE) and hand it to the PRESENTER ( CAEDRMetadata).
// Polled UNCONDITIONALLY (not gated on connection.isHDR, the fixed Welcome flag): the
// host sends 0xCE only for HDR, INCLUDING a mid-session SDRHDR transition (a game
// entering HDR the host re-inits its encoder) the Welcome flag would never reflect.
// Non-blocking; nil for an SDR stream.
if let meta = try? connection.nextHdrMeta(timeoutMs: 0) {
presenter.setHdrMeta(meta)
}
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
if let f = AnnexB.formatDescription(fromIDR: au.data) {
format = f // refreshed on every IDR (mode changes included)
awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
}
guard let f = format, token.isLive else { continue }
if !decoder.decode(au: au, format: f) {
// Submit/decoder error: drop the session and re-gate on the next IDR's
// in-band parameter sets (a delta frame can't recover) stage-1's policy
// and ask the host for that IDR now (infinite GOP; throttled).
if decoder.decode(au: au, format: f) {
decodeFailRun = 0
} else {
// Submit/decoder error: drop the session and re-gate on the next IDR's in-band
// parameter sets (a delta frame can't recover) and keep asking for that IDR.
decoder.reset()
recovery.request()
awaitingIDR = true
decodeFailRun += 1
// ~3 s of solid failure in a 4:4:4 session (and only there a 4:2:0 loss
// recovers within a GOP) 4:4:4 isn't decodable here; end the session.
if connection.isChroma444, decodeFailRun >= 180 {
if token.isLive { onSessionEnd?() }
break
}
}
} catch {
if token.isLive { onSessionEnd?() }
@@ -154,27 +187,30 @@ public final class Stage2Pipeline {
}
thread.name = "punktfunk-stage2-pump"
thread.qualityOfService = .userInteractive
pumpJoinable = true
thread.start()
}
/// MAIN thread, once per vsync. Present the newest ready frame (if any) and stamp
/// capturepresent at `targetPresentNs` the display link's target present instant, already
/// converted to `CLOCK_REALTIME` (see `realtimeNs(forDisplayLinkTimestamp:)`).
/// MAIN thread, once per vsync. Present the newest ready frame (if any) and stamp capturepresent at
/// `targetPresentNs` the display link's target present instant, already converted to
/// `CLOCK_REALTIME` (see `realtimeNs(forDisplayLinkTimestamp:)`).
public func renderTick(targetPresentNs: Int64) {
guard let frame = ring.take() else { return }
guard presenter.render(frame.pixelBuffer, isHDR: frame.isHDR) else { return }
presentMeter.record(ptsNs: frame.ptsNs, atNs: targetPresentNs, offsetNs: offsetNs)
}
/// MAIN thread. Keep the drawable matched to the negotiated mode (host can Reconfigure).
public func setDrawableSize(_ size: CGSize) {
presenter.setDrawableSize(size)
}
/// Stop the pump ( one poll timeout) and drop the decode session. Does not close the
/// connection. A restart needs a fresh Stage2Pipeline (cancel is permanent).
/// Stop the pump ( one poll timeout) and drop the decode session. MAIN THREAD; idempotent. Does not
/// close the connection. A restart needs a fresh Stage2Pipeline (cancel is permanent).
public func stop() {
token.cancel()
// Join the pump (bounded: one nextAU poll + an in-flight decode) before resetting the decoder,
// so the pump can't rebuild a session right after the reset. Only the first stop joins; a
// repeat/deinit stop skips the already-drained semaphore.
if pumpJoinable {
pumpJoinable = false
_ = pumpStopped.wait(timeout: .now() + 0.5)
}
decoder.reset()
recovery.bind(nil) // stop requesting keyframes once the session is torn down
}
@@ -182,8 +218,8 @@ public final class Stage2Pipeline {
deinit { token.cancel() }
/// Convert a `CADisplayLink.targetTimestamp` (CACurrentMediaTime basis) to a `CLOCK_REALTIME`
/// nanosecond instant the present clock the AU pts + skew offset live in. Projects to the
/// target present time (when the frame is actually on glass), not the moment we drew.
/// nanosecond instant the present clock the AU pts + skew offset live in. Projects to the target
/// present time (when the frame is actually on glass), not the moment we drew.
public static func realtimeNs(forDisplayLinkTimestamp t: CFTimeInterval) -> Int64 {
let caNow = CACurrentMediaTime()
var ts = timespec()
@@ -0,0 +1,83 @@
// Runtime 4:4:4 HEVC decode-capability probe.
//
// We advertise `VIDEO_CAP_444` (so the host upgrades to a full-chroma 4:4:4 stream) ONLY when this
// device can decode 4:4:4 HEVC *in hardware* software 4:4:4 decode works but is far too slow for a
// real-time stream at the negotiated resolution, so a software-only device must keep 4:2:0.
//
// `VTIsHardwareDecodeSupported(HEVC)` and the HEVC-decoder-capabilities dictionary report HEVC HW
// decode but expose nothing about `chroma_format_idc`, so the only reliable signal is to actually
// create a *hardware-required* `VTDecompressionSession` for a tiny synthetic 4:4:4 keyframe and
// confirm it both creates and decodes to the expected biplanar 4:4:4 pixel format. Validated on an
// Apple M3 (HW 4:4:4 8- and 10-bit decode to `444v`/`x444`); a software-only decoder fails the
// hardware-required create and we fall back to 4:2:0.
//
// The probe blobs are 256×256 (above the hardware decoder's minimum-dimension floor a 16×16 clip
// is rejected for ALL chroma formats, including 4:2:0) HEVC Range-Extensions keyframes generated
// offline with libx265; see scripts notes. Results are cached (device-static) in lazy statics.
import CoreMedia
import CoreVideo
import Foundation
import VideoToolbox
public enum Stage444Probe {
/// True iff this device hardware-decodes 8-bit 4:4:4 HEVC (the host's current 4:4:4 path
/// BT.709 limited `yuv444p`). Cached after first evaluation.
public static let hwDecode444_8bit: Bool = probeHardware444(
au: Probe444Blobs.au444_8bit,
want: kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange,
fullRangeSibling: kCVPixelFormatType_444YpCbCr8BiPlanarFullRange)
/// True iff this device hardware-decodes 10-bit 4:4:4 HEVC (the 4:4:4 HDR/10-bit intersection).
/// Cached after first evaluation.
public static let hwDecode444_10bit: Bool = probeHardware444(
au: Probe444Blobs.au444_10bit,
want: kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange,
fullRangeSibling: kCVPixelFormatType_444YpCbCr10BiPlanarFullRange)
/// Create a hardware-REQUIRED `VTDecompressionSession` for the synthetic 4:4:4 keyframe and
/// decode it, returning true only when the decoder produces the expected (video- or full-range)
/// biplanar 4:4:4 pixel format. Any failure (no hardware path, wrong output format, decode error)
/// false we keep 4:2:0.
private static func probeHardware444(
au auBytes: [UInt8], want: OSType, fullRangeSibling: OSType
) -> Bool {
let data = Data(auBytes)
guard let format = AnnexB.formatDescription(fromIDR: data) else { return false }
// Require a hardware decoder a software false-positive would make us advertise 4:4:4 and
// then decode every real frame on the CPU, blowing the latency budget.
let spec: [CFString: Any] = [
kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder: true,
]
let attrs: [CFString: Any] = [
kCVPixelBufferPixelFormatTypeKey: want,
kCVPixelBufferMetalCompatibilityKey: true,
]
var session: VTDecompressionSession?
let created = VTDecompressionSessionCreate(
allocator: kCFAllocatorDefault, formatDescription: format,
decoderSpecification: spec as CFDictionary, imageBufferAttributes: attrs as CFDictionary,
outputCallback: nil, decompressionSessionOut: &session)
guard created == noErr, let session else { return false }
defer { VTDecompressionSessionInvalidate(session) }
let au = AccessUnit(data: data, ptsNs: 0, frameIndex: 0, flags: 0)
guard let sample = AnnexB.sampleBuffer(au: au, format: format) else { return false }
var produced: OSType = 0
let done = DispatchSemaphore(value: 0)
let status = VTDecompressionSessionDecodeFrame(
session, sampleBuffer: sample,
flags: [._EnableAsynchronousDecompression], infoFlagsOut: nil
) { status, _, imageBuffer, _, _ in
if status == noErr, let imageBuffer {
produced = CVPixelBufferGetPixelFormatType(imageBuffer)
}
done.signal()
}
guard status == noErr else { return false }
VTDecompressionSessionWaitForAsynchronousFrames(session)
_ = done.wait(timeout: .now() + 1.0)
return produced == want || produced == fullRangeSibling
}
}
@@ -6,6 +6,9 @@
import AVFoundation
import Foundation
import os
private let pumpLog = Logger(subsystem: "io.unom.punktfunk", category: "video")
/// Cancellation handle owned by exactly one pump thread a restart hands the old pump
/// its own token, so it can never be revived by a newer start().
@@ -47,44 +50,74 @@ final class StreamPump {
var format: CMVideoFormatDescription?
var lastKeyframeRequest = Date.distantPast
var lastFramesDropped = connection.framesDropped()
// Coalesced host keyframe request: the decode stays wedged for several frames until
// the IDR lands, so requesting on every frame would flood the control stream.
// Recovery is a persistent WANT, not a one-shot edge: set it on detected loss (or a
// decoder reset), retry the throttled request EVERY iteration, and clear it only when a
// fresh IDR actually re-anchors decode. The old code advanced `lastFramesDropped` on the
// same edge it fired the throttled request so a request swallowed by the throttle (a
// second drop within the window, e.g. the lost recovery IDR itself being pruned) was
// never re-sent: the counter went flat, the climb never re-fired, and the picture stayed
// frozen for good while audio kept playing. The iPhone's lossy Wi-Fi hits this where the
// Mac's Ethernet never does.
var awaitingIDR = false
var awaitingSince = Date.distantPast // when the current recovery began (for the resume log)
var wasFailed = false
// Coalesced host keyframe request. 100 ms throttle (matches the working Android path):
// fast enough that a lost recovery IDR is re-requested promptly, bounded so a sustained
// freeze can't flood the control stream.
func requestKeyframeThrottled() {
let now = Date()
if now.timeIntervalSince(lastKeyframeRequest) > 0.25 {
if now.timeIntervalSince(lastKeyframeRequest) > 0.1 {
connection.requestKeyframe()
lastKeyframeRequest = now
}
}
while token.isLive {
do {
// Loss recovery (the primary recovery path). Under the host's infinite GOP the
// only recovery keyframe is one we request. The reassembler drops unrecoverable
// AUs (framesDropped); the decoder then *conceals* the reference-missing delta
// frames that follow a frozen / garbage picture, WITHOUT flipping the layer to
// .failed so the .failed check below rarely fires after a real network blip.
// Ask the host for a fresh IDR whenever the drop count climbs. Polled every
// iteration (not just per AU) so a total-loss drought still recovers the moment
// packets resume and the reassembler counts the gap.
// Loss recovery (the primary path). Under the host's infinite GOP the only
// recovery keyframe is one we request. The reassembler drops unrecoverable AUs
// (framesDropped); the decoder then *conceals* the reference-missing deltas a
// frozen / garbage picture that never flips the layer to .failed so key off the
// drop count climbing, then keep asking (awaitingIDR) until an IDR lands. Polled
// every iteration so a total-loss drought still recovers when packets resume.
let dropped = connection.framesDropped()
if dropped > lastFramesDropped {
lastFramesDropped = dropped
requestKeyframeThrottled()
// Log only on the falsetrue transition (once per recovery cycle), not per
// dropped AU, so heavy loss doesn't spam the log.
if !awaitingIDR {
awaitingSince = Date()
pumpLog.notice(
"video: unrecoverable drop (framesDropped=\(dropped, privacy: .public)) — requesting recovery IDR")
}
lastFramesDropped = dropped
awaitingIDR = true
}
if awaitingIDR { requestKeyframeThrottled() }
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
onFrame?(au)
if let f = AnnexB.formatDescription(fromIDR: au.data) {
let idrFormat = AnnexB.formatDescription(fromIDR: au.data)
if let f = idrFormat {
format = f // refreshed on every IDR (mode changes included)
if awaitingIDR {
let ms = Int(Date().timeIntervalSince(awaitingSince) * 1000)
pumpLog.notice("video: recovery IDR received — resumed after \(ms, privacy: .public) ms")
}
if layer.status == .failed {
awaitingIDR = false // a fresh IDR re-anchored decode recovery complete
}
let failed = layer.status == .failed
if failed {
// Decode wedged hard (the cold-first-connect case a lost/corrupt opening
// IDR): flush and re-gate on the next in-band parameter sets (resuming with
// a delta frame can't recover), AND ask the host for a fresh IDR. Throttled:
// the layer stays .failed across several polls until the IDR lands.
// IDR): flush and, unless THIS AU is the recovering IDR (re-anchored above),
// re-gate on the next in-band parameter sets and keep asking enqueuing a
// delta into a failed layer can't recover it.
if !wasFailed { pumpLog.warning("video: display layer .failed — flushing + re-anchoring") }
layer.flush()
format = AnnexB.formatDescription(fromIDR: au.data)
requestKeyframeThrottled()
if idrFormat == nil {
format = nil
awaitingIDR = true
}
}
wasFailed = failed
guard let f = format,
let sample = AnnexB.sampleBuffer(au: au, format: f),
token.isLive // don't enqueue a stale frame after a restart
@@ -137,8 +137,8 @@ public struct StreamView: NSViewRepresentable {
public final class StreamLayerView: NSView {
private let displayLayer = AVSampleBufferDisplayLayer()
private var pump: StreamPump?
/// Stage-2 presenter (opt-in via `punktfunk.presenter`): a CAMetalLayer sublayer driven by a
/// display link instead of the StreamPump displayLayer path. nil = stage-1 (default).
/// Stage-2 presenter (default): a CAMetalLayer sublayer driven by a display link instead of the
/// StreamPump displayLayer path. nil = stage-1 (Metal-unavailable fallback / DEBUG toggle).
var presentMeter: LatencyMeter?
private var stage2: Stage2Pipeline?
private var stage2Link: CADisplayLink?
@@ -245,6 +245,15 @@ public final class StreamLayerView: NSView {
layoutMetalLayer() // keep the stage-2 sublayer aspect-fit to the view
}
public override func setFrameSize(_ newSize: NSSize) {
super.setFrameSize(newSize)
// `layout()` isn't guaranteed on a manual-frame (no-Auto-Layout) live resize, so the
// stage-2 metal sublayer's drawableSize could stay at the old size while the view grows
// the compositor then upscales a too-small drawable and the video turns blocky. Resize the
// drawable here too so it always tracks the window's pixel size (no stale upscale).
layoutMetalLayer()
}
// MARK: - Capture state machine
/// Clicking into the video engages capture; that click is local (engagement), so
@@ -549,10 +558,17 @@ public final class StreamLayerView: NSView {
cursorVisible = false
_ = connection.resolvedCompositor // (was: Auto gamescope; kept to document intent)
// Presenter choice default stage-1 (the known-good AVSampleBufferDisplayLayer). Stage-2
// (`punktfunk.presenter == "stage2"`) takes explicit VTDecompressionSession decode + a
// CAMetalLayer/display-link present; it falls back here if Metal can't be set up.
if UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage2",
// Presenter choice stage-2 is the DEFAULT (explicit VTDecompressionSession decode + a
// CAMetalLayer/display-link present): it can detect + recover a wedged decoder where
// stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference. Stage-1 is
// reachable only via the DEBUG presenter toggle; release always takes stage-2 (the stage-1
// pump below stays the automatic fallback if Metal is missing).
#if DEBUG
let forceStage1 = UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage1"
#else
let forceStage1 = false
#endif
if !forceStage1,
let meter = presentMeter,
let pipeline = Stage2Pipeline(presentMeter: meter) {
startStage2(pipeline, connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
@@ -593,9 +609,11 @@ public final class StreamLayerView: NSView {
targetPresentNs: Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: link.targetTimestamp))
}
/// Aspect-fit the metal sublayer in the view (the host streams at the client's native mode,
/// so this is usually the full bounds; it letterboxes a resized window). drawableSize is the
/// layer's pixel size the fullscreen-triangle shader scales the decoded texture to fill it.
/// Position the metal sublayer aspect-fit in the view (the host streams at the client's native
/// mode, so this is usually the full bounds; it letterboxes a resized window). Only the layer
/// FRAME is set here the presenter sizes the drawable to the decoded frame and the layer's
/// contentsGravity (.resizeAspect) scales it to this frame via the system compositor, so a
/// resized window rescales through the system's filter (matching stage-1) instead of the shader.
private func layoutMetalLayer() {
guard let metalLayer, let connection else { return }
let mode = connection.currentMode()
@@ -604,14 +622,12 @@ public final class StreamLayerView: NSView {
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds)
: bounds
let scale = window?.backingScaleFactor ?? 1
// No implicit resize animation; refresh contentsScale on a retinanon-retina move.
CATransaction.begin()
CATransaction.setDisableActions(true)
metalLayer.contentsScale = scale
metalLayer.contentsScale = window?.backingScaleFactor ?? 1
metalLayer.frame = fit
CATransaction.commit()
stage2?.setDrawableSize(CGSize(width: fit.width * scale, height: fit.height * scale))
}
public override func viewDidChangeBackingProperties() {
@@ -622,7 +638,7 @@ public final class StreamLayerView: NSView {
private func teardownStage2() {
stage2Link?.invalidate()
stage2Link = nil
stage2?.stop()
stage2?.stop() // stops the pump (synchronous join) + drops the decode session
stage2 = nil
metalLayer?.removeFromSuperlayer()
metalLayer = nil
@@ -92,8 +92,8 @@ public final class StreamViewController: UIViewController {
public private(set) var connection: PunktfunkConnection?
private var pump: StreamPump?
private var observers: [NSObjectProtocol] = []
/// Stage-2 presenter (opt-in via `punktfunk.presenter`): a CAMetalLayer sublayer driven by a
/// CADisplayLink instead of the StreamPump displayLayer path. nil = stage-1 (default).
/// Stage-2 presenter (default): a CAMetalLayer sublayer driven by a CADisplayLink instead of the
/// StreamPump displayLayer path. nil = stage-1 (Metal-unavailable fallback / DEBUG toggle).
var presentMeter: LatencyMeter?
private var stage2: Stage2Pipeline?
private var stage2Link: CADisplayLink?
@@ -136,6 +136,13 @@ public final class StreamViewController: UIViewController {
public override func loadView() {
view = StreamLayerUIView()
// Re-size the stage-2 drawable if the display scale changes without a bounds change (e.g.
// moving to an external display at a different scale) the iOS analogue of macOS's
// viewDidChangeBackingProperties relayout. The handler takes the VC as its argument, so it
// doesn't capture self (no retain cycle with the registration).
registerForTraitChanges([UITraitDisplayScale.self]) { (vc: StreamViewController, _) in
vc.layoutMetalLayer()
}
#if os(iOS)
// Hide the iPadOS cursor while it hovers the video: the host renders its own
// cursor from our deltas, so the local one only diverges from it. This hides the
@@ -148,19 +155,58 @@ public final class StreamViewController: UIViewController {
}
#if os(iOS)
// Pointer lock is only meaningful on iPad (iPhone has no hardware-pointer lock) and
// only when capture is engaged. The system additionally requires full-screen + frontmost
// and may drop it (Slide Over/Stage Manager/backgrounding) verified in setCaptured().
public override var prefersPointerLocked: Bool {
captured && UIDevice.current.userInterfaceIdiom == .pad
/// Whether the user wants the mouse/trackpad pointer CAPTURED (pointer lock relative
/// movement, the gaming default) rather than forwarded as an absolute position (desktop
/// use). Read live from UserDefaults so it tracks the Settings toggle; defaults to on when
/// unset. iPad-only gated again in `prefersPointerLocked`.
private var pointerCaptureEnabled: Bool {
UserDefaults.standard.object(forKey: DefaultsKey.pointerCapture) as? Bool ?? true
}
/// Whether the pointer should be CAPTURED right now: iPad, capture engaged, and the user
/// hasn't opted into the absolute (desktop) pointer. The system additionally requires
/// full-screen + frontmost and may drop the lock (Slide Over/Stage Manager/backgrounding)
/// syncPointerLock() handles the actual grant/drop and falls back to absolute when unlocked.
private var wantsPointerLock: Bool {
captured && pointerCaptureEnabled && UIDevice.current.userInterfaceIdiom == .pad
}
public override var prefersPointerLocked: Bool { wantsPointerLock }
public override var prefersHomeIndicatorAutoHidden: Bool { true }
// If SwiftUI's UIHostingController reparents us, a plain container parent that forwards
// its pointer-lock decision to its children will then reach this VC. (UIHostingController
// itself does not consult children, which is why GCMouse deltas can never arrive there
// the touch path, always forwarded, is the unconditional fallback.)
public override var childViewControllerForPointerLock: UIViewController? { self }
// NOTE: we deliberately do NOT override `childViewControllerForPointerLock`. The default
// returns nil, which tells the system to use THIS controller's own `prefersPointerLocked`
// exactly what we want, since `PointerLockChain` forces our SwiftUI ancestors to forward the
// downward walk to us and we are the terminal anchor. Returning `self` here would make the
// system ask the same controller forever (it keeps delegating to the returned child)
// unbounded recursion stack overflow once the chain actually reaches us.
/// (Re)build or tear down the forced pointer-lock forwarding chain from this controller to the
/// window root so the system actually resolves our `prefersPointerLocked`. Safe to call
/// repeatedly it no-ops until the view is in a window with a parent chain, and re-runs from
/// the appearance/parent callbacks once SwiftUI has placed us.
private func updatePointerLockChain() {
// Engaging needs a live parent chain to the window root; disengaging is always safe and
// must run even after the view has left the window (session teardown) so the stamped
// SwiftUI ancestors are cleared.
if wantsPointerLock, view.window != nil {
PointerLockChain.engage(self)
} else {
PointerLockChain.disengage(self)
}
}
public override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
// SwiftUI places us in the hierarchy AFTER start()'s setCaptured(true), and may reparent us
// later re-anchor the chain here so a lock requested before we had a parent still lands.
updatePointerLockChain()
}
public override func didMove(toParent parent: UIViewController?) {
super.didMove(toParent: parent)
updatePointerLockChain() // chain shape changed re-anchor (or no-op if not yet in a window)
}
#endif
func start(
@@ -193,7 +239,14 @@ public final class StreamViewController: UIViewController {
// Indirect pointer (mouse/trackpad with no lock) absolute cursor + buttons, routed
// through InputCapture so the forwarding gate and release-on-blur apply uniformly.
streamView.onPointerMoveAbs = { [weak self] p in
self?.inputCapture?.sendMouseAbs(
guard let self else { return }
if iosInputDebug {
// Whether ANY UIKit pointer movement reaches us while the scene is LOCKED tells us
// if the trackpad (which may not be a GCMouse) can still be captured via UIKit.
iosInputLog.debug(
"UIKit pointer move x=\(p.x, privacy: .public) y=\(p.y, privacy: .public) locked=\(self.pointerLockEngaged() == true, privacy: .public) gcFwd=\(self.inputCapture?.gcMouseForwarding == true, privacy: .public)")
}
self.inputCapture?.sendMouseAbs(
x: p.x, y: p.y, surfaceWidth: p.w, surfaceHeight: p.h)
}
streamView.onPointerButton = { [weak self] button, down in
@@ -203,7 +256,12 @@ public final class StreamViewController: UIViewController {
// UNLOCKED regime; while locked, GCMouse's scroll handler owns it mirror the
// sendMouseAbs !gcMouseForwarding gate so the two can't double-send.
streamView.onScroll = { [weak self] dx, dy in
guard let self, self.inputCapture?.gcMouseForwarding == false else { return }
guard let self else { return }
if iosInputDebug {
iosInputLog.debug(
"UIKit scroll dx=\(dx, privacy: .public) dy=\(dy, privacy: .public) locked=\(self.pointerLockEngaged() == true, privacy: .public)")
}
guard self.inputCapture?.gcMouseForwarding == false else { return }
self.inputCapture?.sendScroll(dx: dx, dy: dy)
}
@@ -219,10 +277,17 @@ public final class StreamViewController: UIViewController {
inputCapture = capture
#endif
// Presenter choice default stage-1 (the known-good AVSampleBufferDisplayLayer). Stage-2
// (`punktfunk.presenter == "stage2"`) takes VTDecompressionSession decode + a
// CAMetalLayer/display-link present; falls back here if Metal can't be set up.
if UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage2",
// Presenter choice stage-2 is the DEFAULT (VTDecompressionSession decode + a
// CAMetalLayer/display-link present): it can detect + recover a wedged decoder, where
// stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference frame with no
// way to recover. Stage-1 is reachable only via the DEBUG presenter toggle; release always
// takes stage-2 (the stage-1 pump below stays the automatic fallback if Metal is missing).
#if DEBUG
let forceStage1 = UserDefaults.standard.string(forKey: DefaultsKey.presenter) == "stage1"
#else
let forceStage1 = false
#endif
if !forceStage1,
let meter = presentMeter,
let pipeline = Stage2Pipeline(presentMeter: meter) {
startStage2(pipeline, connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
@@ -300,8 +365,8 @@ public final class StreamViewController: UIViewController {
onFrame: (@Sendable (AccessUnit) -> Void)?, onSessionEnd: (@Sendable () -> Void)?
) {
let metal = pipeline.layer
metal.contentsScale = streamView.contentScaleFactor
// Composites OVER the idle (un-enqueued in stage-2) AVSampleBufferDisplayLayer base.
// (contentsScale + frame are set by layoutMetalLayer() just below.)
streamView.layer.addSublayer(metal)
metalLayer = metal
stage2 = pipeline
@@ -325,9 +390,20 @@ public final class StreamViewController: UIViewController {
layoutMetalLayer()
}
/// Aspect-fit the metal sublayer in the view (the host streams at the client's native mode,
/// so this is usually the full bounds). drawableSize is the layer's pixel size; the shader's
/// fullscreen triangle scales the decoded texture to fill it.
/// The display scale to render the metal drawable at. `traitCollection.displayScale` is the
/// canonical render scale and is reliable once the controller is in the hierarchy;
/// `view.contentScaleFactor` can read 1.0 before the view attaches to a window/screen, which
/// would size the drawable at point resolution a pixelated, upscaled mess. Falls back to the
/// main screen scale if the trait is still unspecified.
private var renderScale: CGFloat {
let s = traitCollection.displayScale
return s > 0 ? s : UIScreen.main.scale
}
/// Position the metal sublayer aspect-fit in the view (the host streams at the client's native
/// mode, so this is usually the full bounds). Only the layer FRAME is set here the presenter
/// sizes the drawable to the decoded frame and the layer's contentsGravity (.resizeAspect)
/// scales it to this frame via the system compositor (matching stage-1's videoGravity).
private func layoutMetalLayer() {
guard let metalLayer, let connection else { return }
let mode = connection.currentMode()
@@ -337,19 +413,17 @@ public final class StreamViewController: UIViewController {
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds)
: bounds
let scale = streamView.contentScaleFactor
CATransaction.begin()
CATransaction.setDisableActions(true) // don't animate the resize
metalLayer.contentsScale = scale
metalLayer.contentsScale = renderScale
metalLayer.frame = fit
CATransaction.commit()
stage2?.setDrawableSize(CGSize(width: fit.width * scale, height: fit.height * scale))
}
private func teardownStage2() {
stage2Link?.invalidate()
stage2Link = nil
stage2?.stop()
stage2?.stop() // stops the pump (synchronous join) + drops the decode session
stage2 = nil
metalLayer?.removeFromSuperlayer()
metalLayer = nil
@@ -369,6 +443,7 @@ public final class StreamViewController: UIViewController {
captured = false
}
setNeedsUpdateOfPrefersPointerLocked()
updatePointerLockChain() // (re)anchor the SwiftUI ancestors so the lock actually resolves
syncPointerLock() // resolve cursor + GCMouse/absolute routing for the current state
let onCaptureChange = onCaptureChange
let captured = captured
@@ -397,7 +472,7 @@ public final class StreamViewController: UIViewController {
pointerInteraction?.invalidate() // re-resolve the hidden/visible cursor for the state
if iosInputDebug {
iosInputLog.debug(
"pointer lock isLocked=\(locked, privacy: .public) captured=\(self.captured, privacy: .public)")
"pointer lock isLocked=\(locked, privacy: .public) captured=\(self.captured, privacy: .public) useGCMouse=\(useGCMouse, privacy: .public) [\(self.inputCapture?.attachedMiceSummary ?? "n/a", privacy: .public)]")
}
}
#endif
@@ -49,11 +49,10 @@ public final class VideoDecoder: @unchecked Sendable {
/// pump can re-gate on the next IDR.
private let onDecodeError: @Sendable (OSStatus) -> Void
/// Latest source HDR mastering metadata (from `PunktfunkConnection.nextHdrMeta`), attached to
/// each decoded HDR pixel buffer so the compositor tone-maps from the real grade. Guarded by its
/// own lock written by the pump thread, read on the VT decode callback.
private let metaLock = NSLock()
private var hdrMeta: PunktfunkConnection.HdrMeta?
/// Whether the negotiated stream is full-chroma 4:4:4 (`connection.isChroma444`), set once at
/// session start before any decode. Selects the 4:4:4 decode pixel format (orthogonal to bit
/// depth / HDR). Read inside `createSessionLocked` under `lock`.
private var chroma444 = false
public init(
onDecoded: @escaping @Sendable (ReadyFrame) -> Void,
@@ -65,12 +64,13 @@ public final class VideoDecoder: @unchecked Sendable {
deinit { teardown() }
/// Set the source HDR mastering metadata (drained from `PunktfunkConnection.nextHdrMeta`). It's
/// attached to subsequent decoded HDR pixel buffers. Thread-safe; cheap to call on each update.
public func setHdrMeta(_ meta: PunktfunkConnection.HdrMeta) {
metaLock.lock()
hdrMeta = meta
metaLock.unlock()
/// Select the chroma subsampling of the decode output (4:2:0 vs full-chroma 4:4:4). Call once at
/// session start, before decoding, from `connection.isChroma444`. Takes effect on the next
/// session (re)build. Thread-safe.
public func setChroma444(_ on: Bool) {
lock.lock()
chroma444 = on
lock.unlock()
}
/// Submit one AU for asynchronous decode, (re)creating the session if `format` changed. The
@@ -135,8 +135,10 @@ public final class VideoDecoder: @unchecked Sendable {
/// True when `newFormat` carries a PQ (SMPTE ST 2084) or HLG transfer function i.e. the host
/// is sending HDR (BT.2020). VideoToolbox populates the transfer-function extension from the
/// HEVC VUI, so this tracks the *stream*, switching dynamically when the user toggles HDR
/// (the host re-emits parameter sets with the new VUI a new format desc session rebuild).
/// HEVC VUI, so this picks the decode bit depth (10-bit P010/x444 vs 8-bit NV12/444v) from the
/// stream. The present-side HDR config (colorspace/EDR/shader) is latched once per session from
/// the Welcome (`connection.isHDR`), which the host does NOT flip mid-session so this predicate
/// and that config agree for the session (a `#if DEBUG` assert in the presenter guards it).
static func isHDRFormat(_ format: CMVideoFormatDescription) -> Bool {
guard
let tf = CMFormatDescriptionGetExtension(
@@ -157,11 +159,18 @@ public final class VideoDecoder: @unchecked Sendable {
session = nil
format = nil
// Decode pixel format is a 2×2 of (chroma, depth/HDR), both biplanar so the presenter binds
// plane 0 = luma, plane 1 = interleaved chroma uniformly 4:4:4 just delivers a full-size
// chroma plane. 10-bit (P010 / `x444`) for HDR (PQ/HLG), 8-bit (NV12 / `444v`) otherwise.
let hdr = Self.isHDRFormat(newFormat)
let pixelFormat =
hdr
? kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange // P010 (10-bit)
: kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange // NV12 (8-bit)
let pixelFormat: OSType = {
switch (chroma444, hdr) {
case (false, false): return kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange // NV12
case (false, true): return kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange // P010
case (true, false): return kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange // 444v
case (true, true): return kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange // x444
}
}()
let imageAttrs: [CFString: Any] = [
kCVPixelBufferMetalCompatibilityKey: true,
kCVPixelBufferPixelFormatTypeKey: pixelFormat,
@@ -169,11 +178,20 @@ public final class VideoDecoder: @unchecked Sendable {
var callback = VTDecompressionOutputCallbackRecord(
decompressionOutputCallback: decoderOutputCallback,
decompressionOutputRefCon: Unmanaged.passUnretained(self).toOpaque())
// 4:4:4 sessions REQUIRE a hardware decoder: we only advertise 4:4:4 when the hardware probe
// passed, so a hardware-incapable mode (e.g. a resolution past the HW 4:4:4 ceiling) must fail
// HERE, synchronously, letting the pump's backstop end the session rather than silently
// falling back to a software 4:4:4 decoder far too slow for a real-time stream. 4:2:0 keeps the
// software fallback (nil spec) as a robustness net.
let spec: CFDictionary? =
chroma444
? [kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder: true] as CFDictionary
: nil
var newSession: VTDecompressionSession?
let status = VTDecompressionSessionCreate(
allocator: kCFAllocatorDefault,
formatDescription: newFormat,
decoderSpecification: nil, // hardware by default
decoderSpecification: spec,
imageBufferAttributes: imageAttrs as CFDictionary,
outputCallback: &callback,
decompressionSessionOut: &newSession)
@@ -195,26 +213,17 @@ public final class VideoDecoder: @unchecked Sendable {
// pts was stamped at timescale 1e9 (AnnexB.sampleBuffer); normalize defensively.
let p = CMTimeConvertScale(pts, timescale: 1_000_000_000, method: .default)
let ptsNs = p.value > 0 ? UInt64(p.value) : 0
// HDR iff the decoder produced a 10-bit P010 buffer (we only request P010 for PQ streams).
// HDR iff the decoder produced a 10-bit buffer (we only request a 10-bit format for PQ/HLG
// streams). Covers 4:2:0 (P010) and 4:4:4 (`x444`), video- and full-range, so a 10-bit 4:4:4
// HDR frame isn't misclassified as SDR. (The mastering metadata is applied to the presenter's
// CAMetalLayer via CAEDRMetadata, not to this source buffer a separate-drawable presenter
// never composites the source buffer's attachments, so attaching them here would be dead.)
let fmt = CVPixelBufferGetPixelFormatType(imageBuffer)
let isHDR =
CVPixelBufferGetPixelFormatType(imageBuffer)
== kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange
// Attach the source's mastering display + content light level (ST.2086 / CEA-861.3) so the
// compositor tone-maps from the real grade rather than inferring from the PQ colourspace
// alone. The SEI byte payloads map 1:1 to these CVImageBuffer attachment keys.
if isHDR {
metaLock.lock()
let meta = hdrMeta
metaLock.unlock()
if let meta {
CVBufferSetAttachment(
imageBuffer, kCVImageBufferMasteringDisplayColorVolumeKey,
meta.masteringDisplayColorVolume() as CFData, .shouldPropagate)
CVBufferSetAttachment(
imageBuffer, kCVImageBufferContentLightLevelInfoKey,
meta.contentLightLevelInfo() as CFData, .shouldPropagate)
}
}
fmt == kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange
|| fmt == kCVPixelFormatType_420YpCbCr10BiPlanarFullRange
|| fmt == kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange
|| fmt == kCVPixelFormatType_444YpCbCr10BiPlanarFullRange
onDecoded(
ReadyFrame(ptsNs: ptsNs, decodedNs: decodedNs, pixelBuffer: imageBuffer, isHDR: isHDR))
}
@@ -0,0 +1,69 @@
import XCTest
#if canImport(Metal)
import CoreVideo
import Metal
import QuartzCore
@testable import PunktfunkKit
final class MetalPresenterTests: XCTestCase {
/// `MetalVideoPresenter.make()` compiles the runtime Metal shaders (the BT.709/BT.2020 YUVRGB
/// fragment shaders plus the Catmull-Rom luma sampler). A `nil` result on a GPU-equipped host
/// means a shader failed to compile this catches a malformed shader before it silently
/// degrades stage-2 to a stage-1 fallback on device.
func testPresenterInitCompilesShaders() throws {
guard MTLCreateSystemDefaultDevice() != nil else {
throw XCTSkip("no Metal device available in this environment")
}
XCTAssertNotNil(
MetalVideoPresenter.make(),
"stage-2 Metal shaders failed to compile (presenter init returned nil)")
}
/// The HDR fix: `configure(hdr:)` must put the layer into the BT.2020-PQ EDR configuration with a
/// reference-white anchor (`edrMetadata`) the missing anchor was what made HDR render "too
/// bright". SDR must use the plain 8-bit path with EDR off and no metadata. A mid-session flip is a
/// per-mode reconfigure, so the round trip back to SDR must fully restore the SDR config.
func testConfigureHDRSetsEDRAnchor() throws {
guard let presenter = MetalVideoPresenter.make() else {
throw XCTSkip("no Metal device available in this environment")
}
presenter.configure(hdr: true)
XCTAssertEqual(presenter.layer.pixelFormat, .rgba16Float, "HDR uses an EDR-capable drawable")
XCTAssertNotNil(presenter.layer.colorspace, "HDR layer must be tagged (itur_2100_PQ)")
XCTAssertTrue(
presenter.layer.wantsExtendedDynamicRangeContent, "EDR must be requested on all platforms")
XCTAssertNotNil(
presenter.layer.edrMetadata,
"HDR must anchor reference white via edrMetadata (the fix for 'too bright')")
// Mid-session HDRSDR flip: the 8-bit path, EDR off, no metadata.
presenter.configure(hdr: false)
XCTAssertEqual(presenter.layer.pixelFormat, .bgra8Unorm, "SDR uses the plain 8-bit drawable")
XCTAssertFalse(presenter.layer.wantsExtendedDynamicRangeContent)
XCTAssertNil(presenter.layer.edrMetadata)
}
/// `render` with a freshly-allocated NV12 buffer must present without crashing or hanging the
/// main-thread present path is the highest-risk part of the stage-2 rewrite. (A headless CI with no
/// display can still allocate a drawable from a CAMetalLayer; if it can't, render returns false,
/// which is also a valid non-crashing outcome.)
func testRenderDoesNotCrashOnNV12Frame() throws {
guard let presenter = MetalVideoPresenter.make() else {
throw XCTSkip("no Metal device available in this environment")
}
presenter.configure(hdr: false)
var pb: CVPixelBuffer?
let attrs: [CFString: Any] = [kCVPixelBufferMetalCompatibilityKey: true]
let status = CVPixelBufferCreate(
kCFAllocatorDefault, 256, 256, kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange,
attrs as CFDictionary, &pb)
guard status == kCVReturnSuccess, let pixelBuffer = pb else {
throw XCTSkip("could not allocate a test pixel buffer")
}
// Just asserting it returns (true or false) without trapping the layer may have no drawable
// source headless, so a false return is acceptable.
_ = presenter.render(pixelBuffer, isHDR: false)
}
}
#endif
@@ -0,0 +1,68 @@
// 4:4:4 decode-path coverage: the hardware-capability probe is stable/cached, and a real 4:4:4 HEVC
// keyframe decodes through VideoDecoder to a biplanar 4:4:4 pixel buffer. Reuses the same synthetic
// 4:4:4 blobs the runtime probe ships with.
import CoreVideo
import VideoToolbox
import XCTest
@testable import PunktfunkKit
private final class FrameBox: @unchecked Sendable {
let lock = NSLock()
var frame: ReadyFrame?
var error: OSStatus?
}
final class Stage444Tests: XCTestCase {
/// The capability probe is device-static and cached reading it twice must return the same value
/// (and must never crash, including where 4:4:4 is unsupported false).
func testProbeIsStableAndCached() {
XCTAssertEqual(Stage444Probe.hwDecode444_8bit, Stage444Probe.hwDecode444_8bit)
XCTAssertEqual(Stage444Probe.hwDecode444_10bit, Stage444Probe.hwDecode444_10bit)
}
/// A real 8-bit 4:4:4 HEVC keyframe (the embedded probe blob) decodes through `VideoDecoder` with
/// `setChroma444(true)` to a 256×256 biplanar 4:4:4 (`444v`/`444f`) buffer classified SDR.
/// (4:4:4 sessions require a hardware decoder skip where there isn't one, which is exactly where
/// the client wouldn't advertise 4:4:4 anyway.)
func testVideoDecoderDecodes444() throws {
try XCTSkipUnless(
Stage444Probe.hwDecode444_8bit, "no hardware 4:4:4 decode on this device")
let data = Data(Probe444Blobs.au444_8bit)
let format = try XCTUnwrap(
AnnexB.formatDescription(fromIDR: data), "the 4:4:4 blob must yield a format description")
let au = AccessUnit(data: data, ptsNs: 7_000_000, frameIndex: 0, flags: 0)
let box = FrameBox()
let done = DispatchSemaphore(value: 0)
let decoder = VideoDecoder(
onDecoded: { f in box.lock.lock(); box.frame = f; box.lock.unlock(); done.signal() },
onDecodeError: { s in box.lock.lock(); box.error = s; box.lock.unlock(); done.signal() })
decoder.setChroma444(true)
XCTAssertTrue(decoder.decode(au: au, format: format), "4:4:4 frame submit should succeed")
XCTAssertEqual(done.wait(timeout: .now() + 10), .success, "the decode callback must fire")
decoder.reset()
box.lock.lock(); let frame = box.frame; let error = box.error; box.lock.unlock()
XCTAssertNil(error.map { "decode error \($0)" })
let ready = try XCTUnwrap(frame, "a 4:4:4 ReadyFrame must be delivered")
XCTAssertEqual(CVPixelBufferGetWidth(ready.pixelBuffer), 256)
XCTAssertEqual(CVPixelBufferGetHeight(ready.pixelBuffer), 256)
let pf = CVPixelBufferGetPixelFormatType(ready.pixelBuffer)
XCTAssertTrue(
pf == kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange
|| pf == kCVPixelFormatType_444YpCbCr8BiPlanarFullRange,
"expected a biplanar 4:4:4 8-bit buffer, got \(fourCCString(pf))")
XCTAssertFalse(ready.isHDR, "an 8-bit BT.709 4:4:4 stream is SDR")
// The chroma plane (plane 1) must be FULL resolution for 4:4:4 (vs half for 4:2:0) this is
// what lets the unchanged shader sample chroma at the luma UV.
XCTAssertEqual(CVPixelBufferGetWidthOfPlane(ready.pixelBuffer, 1), 256)
XCTAssertEqual(CVPixelBufferGetHeightOfPlane(ready.pixelBuffer, 1), 256)
}
private func fourCCString(_ t: OSType) -> String {
let b = [UInt8(t >> 24 & 0xff), UInt8(t >> 16 & 0xff), UInt8(t >> 8 & 0xff), UInt8(t & 0xff)]
return String(bytes: b, encoding: .ascii) ?? "\(t)"
}
}
+14 -5
View File
@@ -294,7 +294,13 @@ const RESOLUTIONS: [number, number, string][] = [
[2560, 1440, "2560 × 1440"],
];
const REFRESH = [0, 30, 60, 90, 120];
const GAMEPADS = ["auto", "xbox360", "dualsense"];
const GAMEPADS = ["auto", "xbox360", "dualsense", "steamdeck"];
const GAMEPAD_LABELS: Record<string, string> = {
auto: "Automatic",
xbox360: "Xbox 360",
dualsense: "DualSense",
steamdeck: "Steam Deck",
};
const SettingsSection: FC = () => {
const [s, setS] = useState<StreamSettings | null>(null);
@@ -355,14 +361,17 @@ const SettingsSection: FC = () => {
/>
<Field label="Gamepad type" childrenContainerWidth="max">
<Dropdown
rgOptions={GAMEPADS.map((g) => ({
data: g,
label: g === "auto" ? "Automatic" : g === "xbox360" ? "Xbox 360" : "DualSense",
}))}
rgOptions={GAMEPADS.map((g) => ({ data: g, label: GAMEPAD_LABELS[g] ?? g }))}
selectedOption={s.gamepad}
onChange={(o) => patch({ gamepad: o.data as string })}
/>
</Field>
{s.gamepad === "steamdeck" && (
<Field
label="⚠ Disable Steam Input"
description="Steam Deck mode forwards the paddles, both trackpads, and gyro to the host. For that, Steam Input must be OFF for punktfunk: on the game page tap ⚙ → Controller Settings → set Steam Input to Off. Otherwise Steam keeps the Deck's controls and only the sticks + buttons reach the host."
/>
)}
<ToggleField
label="Stream microphone"
checked={s.mic_enabled}
+23
View File
@@ -113,12 +113,35 @@ async function ensureShortcut(): Promise<number> {
return appId;
}
/**
* Best-effort: turn Steam Input OFF for our shortcut so SDL's HIDAPI Steam Deck driver can open the
* Deck's controls (paddles · trackpads · gyro) directly. There is no confirmed-stable SteamClient
* API for this, so it is feature-detected and MUST never block or throw into the launch — the manual
* toggle (game page → ⚙ → Controller Settings → Steam Input Off, surfaced in the plugin Settings) is
* the documented source of truth. No-op when the optional API is absent.
*/
function disableSteamInputForShortcut(appId: number): void {
try {
const input = (
SteamClient as unknown as {
Input?: { SetSteamInputEnabledForApp?: (appId: number, enabled: boolean) => void };
}
).Input;
input?.SetSteamInputEnabledForApp?.(appId, false);
} catch {
/* a controller tweak must never break the launch */
}
}
/**
* Launch a stream to `host:port` fullscreen in Gaming Mode. Encodes the target into the
* shortcut's launch options (so one generic shortcut serves every host), then RunGame.
*/
export async function launchStream(host: string, port: number): Promise<void> {
const appId = await ensureShortcut();
// Best-effort so the Deck's rich controls reach the client; no-op if the API is absent (the user
// disables Steam Input manually — see the Settings instruction).
disableSteamInputForShortcut(appId);
const target = port && port !== 9777 ? `${host}:${port}` : host;
// KEY=value ... %command% — the wrapper reads PF_HOST from the environment.
SteamClient.Apps.SetAppLaunchOptions(appId, `PF_HOST=${target} %command%`);
+157 -5
View File
@@ -295,19 +295,21 @@ fn initiate_connect(app: Rc<App>, req: ConnectRequest) {
// Rule 3a: the host opted into reduced-security TOFU; offer it alongside PIN.
tofu_dialog(app, req);
} else {
// Rule 3b: pair=required or unknown policy — PIN pairing is mandatory.
pin_dialog(app, req);
// Rule 3b: pair=required or unknown policy — offer no-PIN delegated approval
// (request access → approve in the console) or the PIN ceremony.
approval_dialog(app, req);
}
}
None => {
// Manual entry (no advertised fingerprint). A known address connects silently
// on its stored pin (rule 1); an unknown one must pair — never silent TOFU.
// on its stored pin (rule 1); an unknown one must pair — request access (approve in
// the console) or use a PIN; never silent TOFU.
match known
.find_by_addr(&req.addr, req.port)
.and_then(|k| crate::trust::parse_hex32(&k.fp_hex))
{
Some(pin) => start_session(app, req, Some(pin)),
None => pin_dialog(app, req), // rule 3b
None => approval_dialog(app, req), // rule 3b
}
}
}
@@ -418,6 +420,83 @@ fn pin_dialog(app: Rc<App>, req: ConnectRequest) {
dialog.present(Some(&parent));
}
/// A fresh host that requires pairing: offer the two ways in. "Request access" is the no-PIN
/// path — connect and wait for the operator to click Approve in the host's console/web UI
/// (delegated approval); "Use a PIN instead…" runs the SPAKE2 ceremony.
fn approval_dialog(app: Rc<App>, req: ConnectRequest) {
let dialog = adw::AlertDialog::new(
Some("Pairing Required"),
Some(&format!(
"{} requires pairing.\n\nRequest access and approve this device in the host's console \
(or web UI) — no PIN needed. Or pair with the 4-digit PIN it can display.",
req.name
)),
);
dialog.add_responses(&[
("cancel", "Cancel"),
("pin", "Use a PIN instead…"),
("request", "Request Access"),
]);
dialog.set_response_appearance("request", adw::ResponseAppearance::Suggested);
dialog.set_default_response(Some("request"));
dialog.set_close_response("cancel");
let parent = app.window.clone();
dialog.connect_response(None, move |_, response| match response {
"request" => request_access(app.clone(), req.clone()),
"pin" => pin_dialog(app.clone(), req.clone()),
_ => {}
});
dialog.present(Some(&parent));
}
/// The no-PIN "request access" flow: open an identified connect that the host PARKS until the
/// operator approves it in the console, showing a cancelable "waiting" dialog meanwhile. On
/// approval the same connection is admitted (no reconnect) and the host is saved as paired.
fn request_access(app: Rc<App>, req: ConnectRequest) {
// Pin the advertised certificate for a discovered host (defence against a host impostor while
// we wait); a manually-typed host has no advertised fingerprint, so trust-on-first-use.
let pin = req.fp_hex.as_deref().and_then(crate::trust::parse_hex32);
let cancel = Rc::new(std::cell::Cell::new(false));
let waiting = adw::AlertDialog::new(
Some("Waiting for Approval"),
Some(&format!(
"Approve “{}” in {}s console or web UI.\n\nThis device is waiting to be let in — it \
connects automatically once you approve it.",
glib::host_name(),
req.name
)),
);
waiting.add_responses(&[("cancel", "Cancel")]);
waiting.set_close_response("cancel");
{
let app = app.clone();
let cancel = cancel.clone();
waiting.connect_response(Some("cancel"), move |_, _| {
// Return the UI immediately; the in-flight connect is left to time out and is torn
// down silently by the event loop (see StartOpts::cancel).
cancel.set(true);
app.busy.set(false);
app.toast("Cancelled — the request may still be pending on the host.");
});
}
waiting.present(Some(&app.window));
start_session_with(
app,
req,
pin,
StartOpts {
// Must exceed the host's approval window (PENDING_APPROVAL_WAIT) so a slow operator
// approval still lands on this connection rather than timing the client out first.
connect_timeout: std::time::Duration::from_secs(185),
persist_paired: true,
waiting: Some(waiting),
cancel: Some(cancel),
},
);
}
/// Measure the path to a host over the real data plane (Swift's "Test Network Speed…"):
/// connect, have the host burst probe filler for 2 s up to its 3 Gbps ceiling, report
/// goodput · loss · a recommended bitrate (≈70 % of measured), and apply it in one tap.
@@ -556,7 +635,42 @@ fn resolve_mode(app: &App) -> punktfunk_core::config::Mode {
mode
}
/// Tunables for a session start that differ between the normal connect and the "request access"
/// (delegated-approval) flow. `Default` is the normal connect.
struct StartOpts {
/// Handshake budget. The request-access flow uses a long one because the host PARKS the
/// connection until the operator clicks Approve (see the host's `PENDING_APPROVAL_WAIT`).
connect_timeout: std::time::Duration,
/// Persist the host as *paired* on a successful connect. Set for request-access, where the
/// operator's approval IS the pairing, so future connects are silent (rule 1). Normal TOFU
/// persists the host *unpaired* (pinned, but not PIN/approval-verified).
persist_paired: bool,
/// A "waiting for approval" dialog to dismiss on the first session event (request-access only).
waiting: Option<adw::AlertDialog>,
/// Set by the waiting dialog's Cancel button. `NativeClient::connect` is a blocking call with
/// no abort, so Cancel returns the UI immediately (clears busy, closes the dialog) and leaves
/// the in-flight connect to time out; when it finally resolves, the event loop sees this flag
/// and tears down silently (drops the connector → closes the connection) without touching the
/// UI a new session may already own.
cancel: Option<Rc<std::cell::Cell<bool>>>,
}
impl Default for StartOpts {
fn default() -> Self {
Self {
connect_timeout: std::time::Duration::from_secs(15),
persist_paired: false,
waiting: None,
cancel: None,
}
}
}
fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
start_session_with(app, req, pin, StartOpts::default());
}
fn start_session_with(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>, opts: StartOpts) {
if app.busy.replace(true) {
return;
}
@@ -577,10 +691,14 @@ fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
audio_channels: s.audio_channels,
pin,
identity: app.identity.clone(),
connect_timeout: opts.connect_timeout,
};
let inhibit = s.inhibit_shortcuts;
drop(s);
let tofu = pin.is_none();
let persist_paired = opts.persist_paired;
let mut waiting = opts.waiting;
let cancel = opts.cancel;
let mut handle = crate::session::start(params);
let frames = std::mem::replace(&mut handle.frames, async_channel::bounded(1).1);
@@ -588,14 +706,41 @@ fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
let mut frames = Some(frames);
let mut page: Option<crate::ui_stream::StreamPage> = None;
while let Ok(event) = handle.events.recv().await {
// A cancelled request-access connect resolved late: tear down silently. Don't touch
// app.busy — Cancel already cleared it, and a fresh session may now own it.
if cancel.as_ref().is_some_and(|c| c.get()) {
if let Some(w) = waiting.take() {
w.close();
}
break;
}
match event {
SessionEvent::Connected {
connector,
mode,
fingerprint,
} => {
// Dismiss the "waiting for approval" dialog (request-access flow), if any.
if let Some(w) = waiting.take() {
w.close();
}
if persist_paired {
// Request-access: the operator approved this device, so record the host as
// a trusted PAIRED host (pinning the fingerprint we observed) — future
// connects are then silent (rule 1), exactly like after a PIN ceremony.
let fp_hex = crate::trust::hex(&fingerprint);
let mut known = KnownHosts::load();
known.upsert(KnownHost {
name: req.name.clone(),
addr: req.addr.clone(),
port: req.port,
fp_hex,
paired: true,
});
let _ = known.save();
app.toast("Approved — connecting…");
} else if tofu {
// A TOFU connect just observed the real fingerprint — pin it from now on.
if tofu {
let fp_hex = crate::trust::hex(&fingerprint);
let mut known = KnownHosts::load();
known.upsert(KnownHost {
@@ -622,6 +767,7 @@ fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
connector,
frames.take().expect("Connected delivered once"),
app.gamepad.escape_events(),
app.gamepad.disconnect_events(),
handle.stop.clone(),
inhibit,
&title,
@@ -644,6 +790,9 @@ fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
msg,
trust_rejected,
} => {
if let Some(w) = waiting.take() {
w.close();
}
tracing::warn!(%msg, trust_rejected, "connect failed");
app.busy.set(false);
// A pinned connect rejected on trust grounds means the host's cert no
@@ -658,6 +807,9 @@ fn start_session(app: Rc<App>, req: ConnectRequest, pin: Option<[u8; 32]>) {
break;
}
SessionEvent::Ended(err) => {
if let Some(w) = waiting.take() {
w.close();
}
app.gamepad.detach();
app.nav.pop_to_tag("hosts");
if let Some(e) = err {
+186 -32
View File
@@ -18,7 +18,7 @@ use punktfunk_core::quic::{HidOutput, RichInput};
use std::collections::HashMap;
use std::sync::mpsc::{Receiver, Sender};
use std::sync::{Arc, Mutex};
use std::time::Duration;
use std::time::{Duration, Instant};
/// Motion scale constants, shared convention with the Swift client (`GamepadWire`):
/// derived from hid-playstation's math over the host's fixed calibration blob. SDL hands
@@ -33,8 +33,15 @@ const G: f32 = 9.80665;
/// is the only way out. Four simultaneous buttons that no game uses as a deliberate
/// combo, so it can't be triggered by normal play. Still forwarded to the host (the user
/// is leaving anyway); we only also raise the escape signal.
///
/// **Escalation:** a quick press leaves fullscreen / releases capture; *holding* the same
/// chord for [`DISCONNECT_HOLD`] ends the session. Deliberately NOT the Steam / QAM buttons —
/// those are the marquee pass-through controls that now reach the host's game-mode UI.
const ESCAPE_CHORD: [u32; 4] = [wire::BTN_LB, wire::BTN_RB, wire::BTN_START, wire::BTN_BACK];
/// Hold the [`ESCAPE_CHORD`] at least this long to disconnect (escalates the leave-fullscreen press).
const DISCONNECT_HOLD: Duration = Duration::from_millis(1500);
#[derive(Clone, Debug)]
pub struct PadInfo {
pub id: u32,
@@ -58,6 +65,7 @@ impl PadInfo {
GamepadPref::DualSense => "DualSense",
GamepadPref::DualShock4 => "DualShock 4",
GamepadPref::XboxOne => "Xbox One",
GamepadPref::SteamDeck => "Steam Deck",
_ => "",
}
}
@@ -89,6 +97,9 @@ pub struct GamepadService {
/// Fires once per press of the [`ESCAPE_CHORD`]; the stream page consumes it to leave
/// fullscreen + release capture.
escape_rx: async_channel::Receiver<()>,
/// Fires once when the [`ESCAPE_CHORD`] is held past [`DISCONNECT_HOLD`]; the stream page
/// consumes it to end the session (the controller equivalent of Ctrl+Alt+Shift+D).
disconnect_rx: async_channel::Receiver<()>,
}
impl GamepadService {
@@ -98,11 +109,12 @@ impl GamepadService {
let pinned = Arc::new(Mutex::new(None));
let (ctl, ctl_rx) = std::sync::mpsc::channel();
let (escape_tx, escape_rx) = async_channel::unbounded();
let (disconnect_tx, disconnect_rx) = async_channel::unbounded();
let (p, a, pin) = (pads.clone(), active.clone(), pinned.clone());
if let Err(e) = std::thread::Builder::new()
.name("punktfunk-gamepad".into())
.spawn(move || {
if let Err(e) = run(&p, &a, &pin, &ctl_rx, &escape_tx) {
if let Err(e) = run(&p, &a, &pin, &ctl_rx, &escape_tx, &disconnect_tx) {
tracing::warn!(error = %e, "gamepad service ended — pads disabled");
}
})
@@ -115,6 +127,7 @@ impl GamepadService {
pinned,
ctl,
escape_rx,
disconnect_rx,
}
}
@@ -124,6 +137,12 @@ impl GamepadService {
self.escape_rx.clone()
}
/// A receiver that yields one `()` when the escape chord is held past [`DISCONNECT_HOLD`]
/// (controller disconnect). A fresh clone per call; the stream page spawns a future on it.
pub fn disconnect_events(&self) -> async_channel::Receiver<()> {
self.disconnect_rx.clone()
}
pub fn pads(&self) -> Vec<PadInfo> {
self.pads.lock().unwrap().clone()
}
@@ -188,6 +207,13 @@ fn button_bit(b: sdl3::gamepad::Button) -> Option<u32> {
Button::DPadLeft => wire::BTN_DPAD_LEFT,
Button::DPadRight => wire::BTN_DPAD_RIGHT,
Button::Touchpad => wire::BTN_TOUCHPAD,
// Back grips / paddles (Steam Deck L4/L5/R4/R5, Xbox Elite P1P4) + the misc/Share button.
// PADDLE1/2/3/4 = R4/L4/R5/L5 (see the host `input::gamepad`).
Button::RightPaddle1 => wire::BTN_PADDLE1,
Button::LeftPaddle1 => wire::BTN_PADDLE2,
Button::RightPaddle2 => wire::BTN_PADDLE3,
Button::LeftPaddle2 => wire::BTN_PADDLE4,
Button::Misc1 => wire::BTN_MISC1,
_ => return None,
})
}
@@ -259,11 +285,22 @@ struct Worker {
/// Wire state of the active pad — zeroed on the wire at switch/detach.
last_axis: [i32; 6],
held_buttons: Vec<u32>,
/// Touchpad contacts the host believes are down, keyed by `(surface, finger)` — lifted on pad
/// switch / detach so a contact held at that moment doesn't stick. surface 0 = the legacy single
/// touchpad, 1/2 = a Steam left/right pad.
held_touches: std::collections::HashSet<(u8, u8)>,
last_accel: [i16; 3],
/// Raises the UI escape signal; the escape chord fires it once per press.
escape_tx: async_channel::Sender<()>,
/// Raises the UI disconnect signal when the escape chord is held past [`DISCONNECT_HOLD`].
disconnect_tx: async_channel::Sender<()>,
/// The escape chord is fully held — latched so it fires once, not every poll.
chord_armed: bool,
/// When the escape chord became fully held (drives the hold-to-disconnect escalation); `None`
/// when the chord is broken.
chord_since: Option<Instant>,
/// The disconnect signal already fired for the current hold — latched so it fires once.
disconnect_fired: bool,
}
impl Worker {
@@ -275,13 +312,22 @@ impl Worker {
fn pad_info(&self, id: u32) -> Option<PadInfo> {
let pad = self.opened.get(&id)?;
let mut pref = pref_for_type(
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
);
// There is no SDL gamepad type for the Steam Deck / Steam Controller, so detect Valve by
// VID/PID (Deck 0x1205, SC wired 0x1102, SC dongle 0x1142) — the host then builds the virtual
// hid-steam pad with the back grips + dual trackpads and the right glyph identity.
if pad.vendor_id() == Some(0x28DE)
&& matches!(pad.product_id(), Some(0x1205 | 0x1102 | 0x1142))
{
pref = GamepadPref::SteamDeck;
}
Some(PadInfo {
id,
name: pad.name().unwrap_or_else(|| "Controller".into()),
pref: pref_for_type(
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
),
pref,
})
}
@@ -297,32 +343,90 @@ impl Worker {
}
*v = i32::MIN;
}
// Lift any touchpad contact the host still believes is down (surface 0 = legacy pad).
for (surface, finger) in self.held_touches.drain() {
let rich = if surface == 0 {
RichInput::Touchpad {
pad: 0,
finger,
active: false,
x: 0,
y: 0,
}
} else {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: false,
click: false,
x: 0,
y: 0,
pressure: 0,
}
};
let _ = c.send_rich_input(rich);
}
} else {
self.held_buttons.clear();
self.last_axis = [i32::MIN; 6];
self.held_touches.clear();
}
// A held chord doesn't survive a flush (detach / pad-switch) — clear its latches too.
self.reset_chord();
}
/// Raise the UI escape signal when the [`ESCAPE_CHORD`] just completed (latched so it
/// fires once per press). Called after each button-down updates `held_buttons`.
/// fires once per press) and start the hold-to-disconnect timer. Called after each
/// button-down updates `held_buttons`.
fn maybe_fire_escape(&mut self) {
if self.chord_armed {
return;
}
if ESCAPE_CHORD.iter().all(|b| self.held_buttons.contains(b)) {
self.chord_armed = true;
self.chord_since = Some(Instant::now());
let _ = self.escape_tx.try_send(());
tracing::info!("gamepad escape chord (L1+R1+Start+Select) — leaving fullscreen");
tracing::info!(
"gamepad escape chord (L1+R1+Start+Select) — leaving fullscreen (hold to disconnect)"
);
}
}
/// Fire the disconnect signal once the escape chord has been continuously held past
/// [`DISCONNECT_HOLD`]. Polled from the main loop so the hold completes without new events.
fn maybe_fire_disconnect(&mut self) {
if self.disconnect_fired {
return;
}
if let Some(since) = self.chord_since {
if since.elapsed() >= DISCONNECT_HOLD {
self.disconnect_fired = true;
let _ = self.disconnect_tx.try_send(());
tracing::info!("gamepad escape chord held — disconnecting");
}
}
}
/// Re-arm once the chord is broken (any of its buttons released).
fn rearm_escape(&mut self) {
if self.chord_armed && !ESCAPE_CHORD.iter().all(|b| self.held_buttons.contains(b)) {
self.chord_armed = false;
self.reset_chord();
}
}
/// Clear the escape/disconnect chord latches. Called at every session boundary
/// ([`flush_held`](Self::flush_held) on detach/pad-switch + on attach): the hold-to-disconnect
/// path *always* ends the session while the chord is still physically held, so the matching
/// button-up events arrive after detach (dropped by the `attached` guard) and `rearm_escape`
/// never runs — without this the latched state would leak into the next session and either
/// swallow its first chord press or fire a stale disconnect on connect.
fn reset_chord(&mut self) {
self.chord_armed = false;
self.chord_since = None;
self.disconnect_fired = false;
}
/// Sensors stream only while a session wants them (they cost USB/BT bandwidth).
fn set_sensors(&mut self, enabled: bool) {
let Some(id) = self.active_id() else { return };
@@ -335,6 +439,56 @@ impl Worker {
}
}
}
/// Forward one touchpad contact on the rich-input plane. A multi-touchpad pad (Steam Deck / Steam
/// Controller) sends `TouchpadEx` with the surface (SDL touchpad 0 = left → 1, 1 = right → 2) and
/// signed coordinates; a single-touchpad pad (DualSense) keeps the legacy `Touchpad` (unsigned).
fn forward_touch(
&mut self,
which: u32,
touchpad: u32,
finger: u8,
x: f32,
y: f32,
active: bool,
) {
let Some(c) = self.attached.as_ref() else {
return;
};
let multi = self
.opened
.get(&which)
.map(|p| p.touchpads_count() >= 2)
.unwrap_or(false);
let (cx, cy) = (x.clamp(0.0, 1.0), y.clamp(0.0, 1.0));
let surface = if multi { (touchpad as u8) + 1 } else { 0 };
let rich = if multi {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: active,
click: false,
x: (cx * 65535.0 - 32768.0) as i16,
y: (cy * 65535.0 - 32768.0) as i16,
pressure: 0,
}
} else {
RichInput::Touchpad {
pad: 0,
finger,
active,
x: (cx * 65535.0) as u16,
y: (cy * 65535.0) as u16,
}
};
let _ = c.send_rich_input(rich);
if active {
self.held_touches.insert((surface, finger));
} else {
self.held_touches.remove(&(surface, finger));
}
}
}
#[allow(clippy::too_many_lines)]
@@ -344,11 +498,18 @@ fn run(
pinned_out: &Mutex<Option<u32>>,
ctl: &Receiver<Ctl>,
escape_tx: &async_channel::Sender<()>,
disconnect_tx: &async_channel::Sender<()>,
) -> Result<(), String> {
// Off-main-thread + no video subsystem: keep SDL away from signals, poll pads on its
// own thread.
sdl3::hint::set("SDL_NO_SIGNAL_HANDLERS", "1");
sdl3::hint::set("SDL_JOYSTICK_THREAD", "1");
// Let SDL's HIDAPI drivers open Valve Steam Controller / Steam Deck devices directly, so the
// paddles, both trackpads, and gyro arrive as first-class SDL gamepad inputs. On a Deck in Game
// Mode, Steam Input still holds the device — the user must disable Steam Input for this app (see
// the Decky UX); on a desktop client (or a Deck with Steam Input off) the hints just work.
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAMDECK", "1");
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAM", "1");
let sdl = sdl3::init().map_err(|e| e.to_string())?;
let subsystem = sdl.gamepad().map_err(|e| e.to_string())?;
let mut pump = sdl.event_pump().map_err(|e| e.to_string())?;
@@ -361,9 +522,13 @@ fn run(
attached: None,
last_axis: [i32::MIN; 6],
held_buttons: Vec::new(),
held_touches: std::collections::HashSet::new(),
last_accel: [0; 3],
escape_tx: escape_tx.clone(),
disconnect_tx: disconnect_tx.clone(),
chord_armed: false,
chord_since: None,
disconnect_fired: false,
};
let publish = |w: &Worker| {
@@ -381,6 +546,7 @@ fn run(
Ok(Ctl::Attach(c)) => {
w.attached = Some(c);
w.last_axis = [i32::MIN; 6];
w.reset_chord(); // every session starts un-latched (Attach doesn't flush)
w.set_sensors(true);
}
Ok(Ctl::Detach) => {
@@ -474,9 +640,11 @@ fn run(
send(w.attached.as_ref().unwrap(), InputKind::GamepadAxis, id, v);
}
}
// DualSense touchpad → the rich-input plane, normalized 0..=65535.
// Touchpad contacts → the rich-input plane. One pad (DualSense) keeps the legacy
// `Touchpad`; two pads (Steam Deck / Steam Controller) send `TouchpadEx` per surface.
Event::ControllerTouchpadDown {
which,
touchpad,
finger,
x,
y,
@@ -484,41 +652,23 @@ fn run(
}
| Event::ControllerTouchpadMotion {
which,
touchpad,
finger,
x,
y,
..
} if active == Some(which) && w.attached.is_some() => {
let _ = w
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: true,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
w.forward_touch(which, touchpad as u32, finger as u8, x, y, true);
}
Event::ControllerTouchpadUp {
which,
touchpad,
finger,
x,
y,
..
} if active == Some(which) && w.attached.is_some() => {
let _ = w
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: false,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
w.forward_touch(which, touchpad as u32, finger as u8, x, y, false);
}
// Motion: accel events update the cache; each gyro event ships a sample
// (the DualSense reports both at ~250 Hz). Scale convention shared with
@@ -559,6 +709,10 @@ fn run(
}
}
// Escalate a held escape chord to a disconnect (polled — the hold completes with no
// new button events; the chord itself is only detected while a session is attached).
w.maybe_fire_disconnect();
// Feedback planes (this thread is their single consumer). The host re-sends
// rumble state periodically, so a generous duration with refresh-on-update is
// safe — a dropped stop heals within ~500 ms.
+6 -1
View File
@@ -27,6 +27,11 @@ pub struct SessionParams {
/// Pinned host fingerprint; `None` = trust on first use (caller persists the observed one).
pub pin: Option<[u8; 32]>,
pub identity: (String, String),
/// How long to wait for the handshake. The normal path uses a short budget; the
/// "request access" (delegated-approval) path uses a long one, because the host PARKS the
/// connection until the operator clicks Approve in its console (so this must exceed the
/// host's approval window — see `PENDING_APPROVAL_WAIT`).
pub connect_timeout: Duration,
}
#[derive(Clone, Copy, Default)]
@@ -139,7 +144,7 @@ fn pump(
None, // launch: the Linux client has no library picker yet
params.pin,
Some(params.identity),
Duration::from_secs(15),
params.connect_timeout,
) {
Ok(c) => Arc::new(c),
Err(e) => {
+57
View File
@@ -19,6 +19,49 @@ const REFRESH: &[u32] = &[0, 30, 60, 90, 120, 144, 165, 240];
const GAMEPADS: &[&str] = &["auto", "xbox360", "dualsense", "xboxone", "dualshock4"];
const COMPOSITORS: &[&str] = &["auto", "kwin", "wlroots", "mutter", "gamescope"];
/// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page.
const APP_LICENSE: &str = concat!(
"punktfunk is licensed under MIT OR Apache-2.0, at your option.\n\n",
"================================ MIT ================================\n\n",
include_str!("../../../LICENSE-MIT"),
"\n\n=============================== Apache-2.0 ===============================\n\n",
include_str!("../../../LICENSE-APACHE"),
);
/// Third-party software notices for the linked Rust crates (generated by
/// scripts/gen-third-party-notices.sh; shown as a Legal section in the About dialog).
const THIRD_PARTY_NOTICES: &str = include_str!("../../../THIRD-PARTY-NOTICES.txt");
/// Show the About dialog (app license + the third-party-software Legal section).
fn show_about(parent: &impl IsA<gtk::Widget>) {
let about = adw::AboutDialog::builder()
.application_name("punktfunk")
.developer_name("unom")
.version(env!("CARGO_PKG_VERSION"))
.website("https://git.unom.io/unom/punktfunk")
.license_type(gtk::License::Custom)
.license(APP_LICENSE)
.build();
// The native (FFmpeg/GTK/PipeWire/SDL3) components are dynamically linked under their own
// (LGPL/Zlib/MIT) licenses; the Rust crate notices are the substantive attribution set.
about.add_legal_section(
"Third-party software (Rust crates)",
None,
gtk::License::Custom,
Some(THIRD_PARTY_NOTICES),
);
about.add_legal_section(
"Third-party software (system libraries)",
None,
gtk::License::Custom,
Some(
"This application dynamically links system libraries under their own licenses, \
including FFmpeg (LGPL v2.1+), GTK 4 and libadwaita (LGPL v2.1+), PipeWire (MIT), \
and SDL 3 (Zlib). Their full license texts are available from each project.",
),
);
about.present(Some(parent));
}
pub fn show(
parent: &impl IsA<gtk::Widget>,
settings: Rc<RefCell<Settings>>,
@@ -156,9 +199,23 @@ pub fn show(
.build();
audio.add(&mic_row);
let about = adw::PreferencesGroup::builder().title("About").build();
let licenses_row = adw::ActionRow::builder()
.title("Third-party licenses")
.subtitle("Open-source software used by punktfunk")
.activatable(true)
.build();
licenses_row.add_suffix(&gtk::Image::from_icon_name("go-next-symbolic"));
{
let about_parent: gtk::Widget = parent.clone().upcast();
licenses_row.connect_activated(move |_| show_about(&about_parent));
}
about.add(&licenses_row);
page.add(&stream);
page.add(&input);
page.add(&audio);
page.add(&about);
// Seed from the current settings.
{
+36 -3
View File
@@ -124,12 +124,13 @@ impl Capture {
}
}
#[allow(clippy::too_many_lines)]
#[allow(clippy::too_many_lines, clippy::too_many_arguments)]
pub fn new(
window: &adw::ApplicationWindow,
connector: Arc<NativeClient>,
frames: async_channel::Receiver<DecodedFrame>,
escape_rx: async_channel::Receiver<()>,
disconnect_rx: async_channel::Receiver<()>,
stop: Arc<AtomicBool>,
inhibit_shortcuts: bool,
title: &str,
@@ -152,7 +153,7 @@ pub fn new(
stats_label.set_margin_top(12);
let hint = gtk::Label::new(Some(
"Click the stream to capture input · Ctrl+Alt+Shift+Q releases",
"Click the stream to capture input · Ctrl+Alt+Shift+Q releases · Ctrl+Alt+Shift+D disconnects",
));
hint.add_css_class("osd");
hint.set_halign(gtk::Align::Center);
@@ -163,7 +164,9 @@ pub fn new(
// Flashed when entering fullscreen — the only exit affordances once the header bar is
// hidden (F11 on a keyboard; the L1+R1+Start+Select chord on a controller, which is the
// only way out on a Steam Deck).
let fs_hint = gtk::Label::new(Some("F11 · L1 + R1 + Start + Select — exit fullscreen"));
let fs_hint = gtk::Label::new(Some(
"F11 · L1 + R1 + Start + Select — exit fullscreen (hold to disconnect)",
));
fs_hint.add_css_class("osd");
fs_hint.set_halign(gtk::Align::Center);
fs_hint.set_valign(gtk::Align::Start);
@@ -297,6 +300,7 @@ pub fn new(
key.set_propagation_phase(gtk::PropagationPhase::Capture);
let cap = capture.clone();
let window_k = window.clone();
let stop_kb = stop.clone();
key.connect_key_pressed(move |_, keyval, keycode, state| {
let chord = gdk::ModifierType::CONTROL_MASK
| gdk::ModifierType::ALT_MASK
@@ -309,6 +313,13 @@ pub fn new(
}
return glib::Propagation::Stop;
}
// Ctrl+Alt+Shift+D — leave the session. Now that Steam / QAM pass through to the host,
// the capture toggle alone can't end a stream, so this is the keyboard's explicit exit.
if state.contains(chord) && keyval.to_lower() == gdk::Key::d {
cap.release();
stop_kb.store(true, Ordering::SeqCst);
return glib::Propagation::Stop;
}
if keyval == gdk::Key::F11 {
if window_k.is_fullscreen() {
window_k.unfullscreen();
@@ -442,6 +453,24 @@ pub fn new(
})
};
// Controller disconnect (escape chord held past the hold threshold) → end the session, the
// controller equivalent of Ctrl+Alt+Shift+D. Setting `stop` ends the session pump, which pops
// this page (and fires `hidden` below). One-shot — the session is going away.
let disconnect_future = {
let window = window.clone();
let cap = capture.clone();
let stop_d = stop.clone();
glib::spawn_future_local(async move {
if disconnect_rx.recv().await.is_ok() {
cap.release();
if window.is_fullscreen() {
window.unfullscreen();
}
stop_d.store(true, Ordering::SeqCst);
}
})
};
// The page's `hidden` fires once navigation away completes (back button, pop on
// session end) — NOT on the transient unmap/map cycle a NavigationView push performs.
{
@@ -449,6 +478,7 @@ pub fn new(
let stop_h = stop.clone();
let handlers = RefCell::new(Some((fs_handler, active_handler)));
let escape_future = RefCell::new(Some(escape_future));
let disconnect_future = RefCell::new(Some(disconnect_future));
page.connect_hidden(move |_| {
tracing::debug!("stream page hidden — ending session");
if let Some((fs, active)) = handlers.borrow_mut().take() {
@@ -458,6 +488,9 @@ pub fn new(
if let Some(f) = escape_future.borrow_mut().take() {
f.abort();
}
if let Some(f) = disconnect_future.borrow_mut().take() {
f.abort();
}
if window.is_fullscreen() {
window.unfullscreen();
}
+19 -1
View File
@@ -76,11 +76,29 @@ foreach ($f in $required) {
Copy-Item $src (Join-Path $layout $f) -Force
}
# FFmpeg runtime DLLs (the exe link-imports the decode set; copy them all — small and correct)
# FFmpeg runtime DLLs (the exe link-imports the decode set; copy them all — small and correct).
# These are unmodified BtbN *lgpl-shared* builds, linked dynamically (replaceable DLLs) — FFmpeg is
# used under the LGPL v2.1+; the license text + notice ship in licenses\ below.
$ff = Get-ChildItem -Path $FfmpegBin -Filter *.dll -ErrorAction SilentlyContinue
if (-not $ff) { throw "no FFmpeg DLLs in $FfmpegBin" }
$ff | ForEach-Object { Copy-Item $_.FullName (Join-Path $layout $_.Name) -Force }
# license/attribution payload (MSIX has no installer EULA page, so ship them as files): FFmpeg's LGPL
# notice + license text, the project's own MIT/Apache texts, and the generated third-party notices.
$licDir = Join-Path $layout 'licenses'
New-Item -ItemType Directory -Force -Path $licDir | Out-Null
$repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..\..')).Path
Copy-Item (Join-Path $repoRoot 'packaging\windows\licenses\FFmpeg-LGPL-NOTICE.txt') $licDir -Force -ErrorAction SilentlyContinue
foreach ($n in @('THIRD-PARTY-NOTICES.txt', 'LICENSE-MIT', 'LICENSE-APACHE')) {
$p = Join-Path $repoRoot $n
if (Test-Path $p) { Copy-Item $p $licDir -Force }
}
$ffRoot = Split-Path $FfmpegBin -Parent
foreach ($lic in @('LICENSE.txt', 'LICENSE', 'COPYING.LGPLv2.1', 'COPYING.LGPLv3', 'COPYING.txt')) {
$p = Join-Path $ffRoot $lic
if (Test-Path $p) { Copy-Item $p $licDir -Force }
}
# tile/store assets
Copy-Item (Join-Path $assets '*') (Join-Path $layout 'Assets') -Force
+301 -15
View File
@@ -20,7 +20,9 @@ use crate::video::{DecodedFrame, DecoderPref};
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use std::cell::RefCell;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;
use windows_reactor::*;
const RESOLUTIONS: &[(u32, u32)] = &[
@@ -43,12 +45,27 @@ const BITRATES_MBPS: &[u32] = &[0, 10, 20, 30, 50, 80, 150];
/// capture; the resolved count drives the decoder + WASAPI render layout.
const AUDIO_CHANNELS: &[(u8, &str)] = &[(2, "Stereo"), (6, "5.1 Surround"), (8, "7.1 Surround")];
/// punktfunk's own license (MIT OR Apache-2.0), shown on the Licenses screen.
const APP_LICENSE: &str = concat!(
include_str!("../../../LICENSE-MIT"),
"\n\n================================ Apache-2.0 ================================\n\n",
include_str!("../../../LICENSE-APACHE"),
);
/// Third-party software notices for the linked Rust crates (generated by
/// scripts/gen-third-party-notices.sh; the MSIX also ships this under licenses/).
const THIRD_PARTY_NOTICES: &str = include_str!("../../../THIRD-PARTY-NOTICES.txt");
#[derive(Clone, PartialEq)]
enum Screen {
Hosts,
Connecting,
/// The no-PIN "request access" wait: an identified connect is in flight, parked by the host
/// until the operator approves this device in its console. Cancelable.
RequestAccess,
Stream,
Settings,
/// Open-source / third-party license notices (reached from Settings).
Licenses,
Pair,
}
@@ -132,6 +149,11 @@ struct Shared {
/// Latest stream stats, written by the session's event loop and mirrored into reactor state
/// by the stream page's HUD poll thread to drive the overlay.
stats: Mutex<Stats>,
/// Cancel flag for the in-flight "request access" connect. A FRESH flag is installed per
/// request: the waiting screen's Cancel button reads it back from here and sets it, and that
/// request's event loop (which captured the same `Arc` at spawn) then tears down silently when
/// the parked connect finally resolves. `None` outside a request-access flow.
cancel: Mutex<Option<Arc<AtomicBool>>>,
}
pub struct AppCtx {
@@ -376,8 +398,13 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
.vertical_alignment(VerticalAlignment::Center)
.into()
}
// request_access_page (like settings_page/Connecting) uses no hooks, so calling it inline
// is sound — it only wires a Cancel button to the shared cancel flag + navigation.
Screen::RequestAccess => request_access_page(ctx, &set_screen),
// settings_page uses no hooks (it never touches `cx`), so calling it inline is sound.
Screen::Settings => settings_page(ctx, &set_screen),
// licenses_page is a static text screen (no hooks), so inline is sound.
Screen::Licenses => licenses_page(&set_screen),
Screen::Pair => component(pair_page, svc),
Screen::Stream => component(stream_page, StreamProps { svc, stats }),
}
@@ -569,12 +596,61 @@ fn initiate(
}
}
/// Tunables that differ between the normal connect and the no-PIN "request access" flow.
/// `Default` is the normal connect: short handshake budget, persist *unpaired* on TOFU, and the
/// plain "Connecting" screen.
struct ConnectOpts {
/// Handshake budget. Request-access uses a long one because the host PARKS the connection
/// until the operator clicks Approve in its console (see the host's `PENDING_APPROVAL_WAIT`).
connect_timeout: Duration,
/// Persist the host as *paired* on a successful connect. Set for request-access, where the
/// operator's approval IS the pairing, so future connects are silent (rule 1). Normal TOFU
/// persists the host *unpaired* (pinned, but not PIN/approval-verified).
persist_paired: bool,
/// Show the cancelable "waiting for approval" screen instead of "Connecting" (request-access).
awaiting_approval: bool,
/// Set by the waiting screen's Cancel button. `NativeClient::connect` is blocking with no
/// abort, so Cancel returns the UI immediately and leaves the parked connect to resolve/time
/// out; this request's event loop then sees the flag and tears down silently (drops the
/// connector → closes the connection) without touching a screen a new session may already own.
cancel: Option<Arc<AtomicBool>>,
}
impl Default for ConnectOpts {
fn default() -> Self {
Self {
connect_timeout: Duration::from_secs(15),
persist_paired: false,
awaiting_approval: false,
cancel: None,
}
}
}
fn connect(
ctx: &Arc<AppCtx>,
target: &Target,
pin: Option<[u8; 32]>,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
connect_with(
ctx,
target,
pin,
set_screen,
set_status,
ConnectOpts::default(),
);
}
fn connect_with(
ctx: &Arc<AppCtx>,
target: &Target,
pin: Option<[u8; 32]>,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
opts: ConnectOpts,
) {
let s = ctx.settings.lock().unwrap().clone();
let mode = if s.width != 0 && s.refresh_hz != 0 {
@@ -607,29 +683,54 @@ fn connect(
decoder: DecoderPref::from_name(&s.decoder),
pin,
identity: ctx.identity.clone(),
connect_timeout: opts.connect_timeout,
});
set_status.call(String::new());
set_screen.call(Screen::Connecting);
set_screen.call(if opts.awaiting_approval {
Screen::RequestAccess
} else {
Screen::Connecting
});
let tofu = pin.is_none();
let persist_paired = opts.persist_paired;
let cancel = opts.cancel;
let (shared, gamepad) = (ctx.shared.clone(), ctx.gamepad.clone());
let (ss, st) = (set_screen.clone(), set_status.clone());
let target = target.clone();
std::thread::spawn(move || loop {
match handle.events.recv_blocking() {
Ok(SessionEvent::Connected {
let event = match handle.events.recv_blocking() {
Ok(e) => e,
Err(_) => {
gamepad.detach();
ss.call(Screen::Hosts);
break;
}
};
// A cancelled request-access connect that resolved late (the host approved or the park
// timed out after the user walked away): tear down silently. Cancel already returned the
// UI to the host list; dropping `event` (and with it any connector) closes the connection
// without popping a stream or a stray error over the screen a new session may own.
if cancel.as_ref().is_some_and(|c| c.load(Ordering::SeqCst)) {
break;
}
match event {
SessionEvent::Connected {
connector,
fingerprint,
..
}) => {
if tofu {
} => {
if persist_paired || tofu {
// Request-access: the operator approved this device, so record the host as a
// trusted PAIRED host — future connects are then silent (rule 1), exactly like
// after a PIN ceremony. A plain TOFU connect persists it *unpaired* (pinned).
let mut k = KnownHosts::load();
k.upsert(KnownHost {
name: target.name.clone(),
addr: target.addr.clone(),
port: target.port,
fp_hex: trust::hex(&fingerprint),
paired: false,
paired: persist_paired,
});
let _ = k.save();
}
@@ -638,10 +739,10 @@ fn connect(
*shared.handoff.lock().unwrap() = Some((connector, handle.frames.clone()));
ss.call(Screen::Stream);
}
Ok(SessionEvent::Failed {
SessionEvent::Failed {
msg,
trust_rejected,
}) => {
} => {
st.call(msg);
gamepad.detach();
if trust_rejected {
@@ -653,22 +754,100 @@ fn connect(
}
break;
}
Ok(SessionEvent::Ended(err)) => {
SessionEvent::Ended(err) => {
st.call(err.unwrap_or_else(|| "Session ended".into()));
gamepad.detach();
ss.call(Screen::Hosts);
break;
}
Ok(SessionEvent::Stats(s)) => *shared.stats.lock().unwrap() = s,
Err(_) => {
gamepad.detach();
ss.call(Screen::Hosts);
break;
}
SessionEvent::Stats(s) => *shared.stats.lock().unwrap() = s,
}
});
}
/// The no-PIN "request access" flow: open an identified connect that the host PARKS until the
/// operator approves this device in its console (or web UI), showing a cancelable "waiting"
/// screen meanwhile. On approval the SAME connection is admitted (no reconnect) and the host is
/// saved as paired, so later connects are silent.
fn request_access(
ctx: &Arc<AppCtx>,
target: &Target,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
// Pin the advertised certificate for a discovered host (defence against a host impostor while
// we wait); a manually-typed host has no advertised fingerprint, so trust-on-first-use.
let pin = target.fp_hex.as_deref().and_then(trust::parse_hex32);
// A fresh cancel flag per request, installed where the waiting screen's Cancel button can read
// it back; this request's event loop captures the same `Arc` (via ConnectOpts) below.
let cancel = Arc::new(AtomicBool::new(false));
*ctx.shared.cancel.lock().unwrap() = Some(cancel.clone());
connect_with(
ctx,
target,
pin,
set_screen,
set_status,
ConnectOpts {
// Must exceed the host's approval window (PENDING_APPROVAL_WAIT) so a slow operator
// approval still lands on this connection rather than timing the client out first.
connect_timeout: Duration::from_secs(185),
persist_paired: true,
awaiting_approval: true,
cancel: Some(cancel),
},
);
}
/// The cancelable "waiting for approval" screen (request-access flow): a spinner + guidance while
/// the identified connect sits parked on the host, plus a Cancel that returns to the host list and
/// trips the shared cancel flag so the parked connect tears down silently if it resolves after the
/// user has walked away. Mirrors the inline `Connecting` screen; uses no hooks.
fn request_access_page(ctx: &Arc<AppCtx>, set_screen: &AsyncSetState<Screen>) -> Element {
let target_name = ctx.shared.target.lock().unwrap().name.clone();
let headline = if target_name.is_empty() {
"Waiting for approval\u{2026}".to_string()
} else {
format!("Waiting for {target_name} to approve\u{2026}")
};
let cancel_btn = {
let (ctx, ss) = (ctx.clone(), set_screen.clone());
button("Cancel")
.icon(SymbolGlyph::Cancel)
.on_click(move || {
// Return the UI immediately; the parked connect is blocking with no abort, so trip
// the flag this request's event loop captured — it then tears down silently when
// the connect finally resolves (see ConnectOpts::cancel).
if let Some(c) = ctx.shared.cancel.lock().unwrap().as_ref() {
c.store(true, Ordering::SeqCst);
}
ss.call(Screen::Hosts);
})
.horizontal_alignment(HorizontalAlignment::Center)
};
vstack((
ProgressRing::indeterminate()
.width(48.0)
.height(48.0)
.horizontal_alignment(HorizontalAlignment::Center),
text_block(headline)
.font_size(18.0)
.semibold()
.horizontal_alignment(HorizontalAlignment::Center),
text_block(
"Approve this device in the host's console or web UI \u{2014} it connects automatically \
once you approve it. No PIN needed.",
)
.foreground(ThemeRef::SecondaryText)
.horizontal_alignment(HorizontalAlignment::Center),
cancel_btn,
))
.spacing(16.0)
.horizontal_alignment(HorizontalAlignment::Center)
.vertical_alignment(VerticalAlignment::Center)
.into()
}
fn pair_page(props: &Svc, cx: &mut RenderCx) -> Element {
let ctx = &props.ctx;
let set_screen = &props.set_screen;
@@ -728,6 +907,20 @@ fn pair_page(props: &Svc, cx: &mut RenderCx) -> Element {
.icon(SymbolGlyph::Cancel)
.on_click(move || ss.call(Screen::Hosts))
};
// The no-PIN alternative offered alongside the PIN ceremony: open an identified connect that
// the host parks until the operator approves this device in its console (delegated approval).
let request_btn = {
let (ctx2, ss, st, target2) = (
ctx.clone(),
set_screen.clone(),
set_status.clone(),
target.clone(),
);
button("Request access without a PIN")
.icon(SymbolGlyph::Send)
.on_click(move || request_access(&ctx2, &target2, &ss, &st))
.horizontal_alignment(HorizontalAlignment::Stretch)
};
let content = card(vstack((
grid((
@@ -760,6 +953,13 @@ fn pair_page(props: &Svc, cx: &mut RenderCx) -> Element {
.font_size(28.0)
.on_changed(move |s| set_code.call(s)),
hstack((pair_btn, cancel_btn)).spacing(8.0),
text_block(
"Don\u{2019}t have a PIN? Request access instead and approve this device on the host \
(its console or web UI) \u{2014} no PIN needed.",
)
.font_size(12.0)
.foreground(ThemeRef::SecondaryText),
request_btn,
))
.spacing(16.0))
.max_width(480.0)
@@ -967,6 +1167,21 @@ fn settings_page(ctx: &Arc<AppCtx>, set_screen: &AsyncSetState<Screen>) -> Eleme
.spacing(10.0),
);
let licenses_button = {
let ss = set_screen.clone();
button("Third-party licenses").on_click(move || ss.call(Screen::Licenses))
};
let about_card = card(
vstack((
text_block("About").font_size(15.0).semibold(),
text_block("punktfunk is licensed under MIT OR Apache-2.0.")
.font_size(12.0)
.foreground(ThemeRef::SecondaryText),
licenses_button,
))
.spacing(10.0),
);
page(vec![
header.into(),
section("DISPLAY"),
@@ -975,6 +1190,77 @@ fn settings_page(ctx: &Arc<AppCtx>, set_screen: &AsyncSetState<Screen>) -> Eleme
video_card.into(),
section("AUDIO"),
audio_card.into(),
section("ABOUT"),
about_card.into(),
])
}
/// Static screen: the app's own license + the third-party software notices (reached from Settings).
fn licenses_page(set_screen: &AsyncSetState<Screen>) -> Element {
let header = grid((
text_block("Third-party licenses")
.font_size(30.0)
.bold()
.grid_column(0)
.vertical_alignment(VerticalAlignment::Center),
button("Back")
.accent()
.icon(SymbolGlyph::Back)
.on_click({
let ss = set_screen.clone();
move || ss.call(Screen::Settings)
})
.grid_column(1)
.vertical_alignment(VerticalAlignment::Center),
))
.columns([GridLength::Star(1.0), GridLength::Auto])
.margin(edges(0.0, 0.0, 0.0, 6.0));
let app_card = card(
vstack((
text_block("punktfunk").font_size(15.0).semibold(),
text_block("Licensed under MIT OR Apache-2.0, at your option.")
.font_size(12.0)
.foreground(ThemeRef::SecondaryText),
text_block(APP_LICENSE)
.font_size(11.0)
.foreground(ThemeRef::SecondaryText),
))
.spacing(8.0),
);
let natives_card = card(
vstack((
text_block("Bundled components").font_size(15.0).semibold(),
text_block(
"FFmpeg is bundled under the LGPL v2.1+ (dynamically linked, replaceable DLLs); its \
license and notice ship in the installed licenses\\ folder. SDL 3 (Zlib) and the \
Windows App SDK (Microsoft) are also linked.",
)
.font_size(12.0)
.foreground(ThemeRef::SecondaryText),
))
.spacing(8.0),
);
let notices_card = card(
vstack((
text_block("Rust crates").font_size(15.0).semibold(),
text_block(THIRD_PARTY_NOTICES)
.font_size(11.0)
.foreground(ThemeRef::SecondaryText),
))
.spacing(8.0),
);
page(vec![
header.into(),
section("PUNKTFUNK"),
app_card.into(),
section("BUNDLED"),
natives_card.into(),
section("OPEN SOURCE"),
notices_card.into(),
])
}
+108 -27
View File
@@ -169,6 +169,13 @@ fn button_bit(b: sdl3::gamepad::Button) -> Option<u32> {
Button::DPadLeft => wire::BTN_DPAD_LEFT,
Button::DPadRight => wire::BTN_DPAD_RIGHT,
Button::Touchpad => wire::BTN_TOUCHPAD,
// Back grips / paddles (Steam Deck L4/L5/R4/R5, Xbox Elite P1P4) + the misc/Share button.
// PADDLE1/2/3/4 = R4/L4/R5/L5 (see the host `input::gamepad`).
Button::RightPaddle1 => wire::BTN_PADDLE1,
Button::LeftPaddle1 => wire::BTN_PADDLE2,
Button::RightPaddle2 => wire::BTN_PADDLE3,
Button::LeftPaddle2 => wire::BTN_PADDLE4,
Button::Misc1 => wire::BTN_MISC1,
_ => return None,
})
}
@@ -240,6 +247,9 @@ struct Worker {
/// Wire state of the active pad — zeroed on the wire at switch/detach.
last_axis: [i32; 6],
held_buttons: Vec<u32>,
/// Touchpad contacts the host believes are down, keyed by `(surface, finger)` — lifted on pad
/// switch / detach. surface 0 = the legacy single touchpad, 1/2 = a Steam left/right pad.
held_touches: std::collections::HashSet<(u8, u8)>,
last_accel: [i16; 3],
}
@@ -252,13 +262,21 @@ impl Worker {
fn pad_info(&self, id: u32) -> Option<PadInfo> {
let pad = self.opened.get(&id)?;
let mut pref = pref_for_type(
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
);
// No SDL type for the Steam Deck / Steam Controller — detect Valve by VID/PID (Deck 0x1205,
// SC wired 0x1102, SC dongle 0x1142) so the host builds the virtual hid-steam pad.
if pad.vendor_id() == Some(0x28DE)
&& matches!(pad.product_id(), Some(0x1205 | 0x1102 | 0x1142))
{
pref = GamepadPref::SteamDeck;
}
Some(PadInfo {
id,
name: pad.name().unwrap_or_else(|| "Controller".into()),
pref: pref_for_type(
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
),
pref,
})
}
@@ -274,9 +292,33 @@ impl Worker {
}
*v = i32::MIN;
}
for (surface, finger) in self.held_touches.drain() {
let rich = if surface == 0 {
RichInput::Touchpad {
pad: 0,
finger,
active: false,
x: 0,
y: 0,
}
} else {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: false,
click: false,
x: 0,
y: 0,
pressure: 0,
}
};
let _ = c.send_rich_input(rich);
}
} else {
self.held_buttons.clear();
self.last_axis = [i32::MIN; 6];
self.held_touches.clear();
}
}
@@ -292,6 +334,56 @@ impl Worker {
}
}
}
/// Forward one touchpad contact on the rich-input plane. A multi-touchpad pad (Steam Deck / Steam
/// Controller) sends `TouchpadEx` with the surface (SDL touchpad 0 = left → 1, 1 = right → 2) and
/// signed coordinates; a single-touchpad pad (DualSense) keeps the legacy `Touchpad` (unsigned).
fn forward_touch(
&mut self,
which: u32,
touchpad: u32,
finger: u8,
x: f32,
y: f32,
active: bool,
) {
let Some(c) = self.attached.as_ref() else {
return;
};
let multi = self
.opened
.get(&which)
.map(|p| p.touchpads_count() >= 2)
.unwrap_or(false);
let (cx, cy) = (x.clamp(0.0, 1.0), y.clamp(0.0, 1.0));
let surface = if multi { (touchpad as u8) + 1 } else { 0 };
let rich = if multi {
RichInput::TouchpadEx {
pad: 0,
surface,
finger,
touch: active,
click: false,
x: (cx * 65535.0 - 32768.0) as i16,
y: (cy * 65535.0 - 32768.0) as i16,
pressure: 0,
}
} else {
RichInput::Touchpad {
pad: 0,
finger,
active,
x: (cx * 65535.0) as u16,
y: (cy * 65535.0) as u16,
}
};
let _ = c.send_rich_input(rich);
if active {
self.held_touches.insert((surface, finger));
} else {
self.held_touches.remove(&(surface, finger));
}
}
}
#[allow(clippy::too_many_lines)]
@@ -305,6 +397,10 @@ fn run(
// thread.
sdl3::hint::set("SDL_NO_SIGNAL_HANDLERS", "1");
sdl3::hint::set("SDL_JOYSTICK_THREAD", "1");
// Let SDL's HIDAPI drivers open Valve Steam Controller / Steam Deck devices directly, so the
// paddles, both trackpads, and gyro arrive as first-class SDL gamepad inputs.
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAMDECK", "1");
sdl3::hint::set("SDL_JOYSTICK_HIDAPI_STEAM", "1");
let sdl = sdl3::init().map_err(|e| e.to_string())?;
let subsystem = sdl.gamepad().map_err(|e| e.to_string())?;
let mut pump = sdl.event_pump().map_err(|e| e.to_string())?;
@@ -317,6 +413,7 @@ fn run(
attached: None,
last_axis: [i32::MIN; 6],
held_buttons: Vec::new(),
held_touches: std::collections::HashSet::new(),
last_accel: [0; 3],
};
@@ -426,9 +523,11 @@ fn run(
send(w.attached.as_ref().unwrap(), InputKind::GamepadAxis, id, v);
}
}
// DualSense touchpad → the rich-input plane, normalized 0..=65535.
// Touchpad contacts → the rich-input plane. One pad (DualSense) keeps the legacy
// `Touchpad`; two pads (Steam Deck / Steam Controller) send `TouchpadEx` per surface.
Event::ControllerTouchpadDown {
which,
touchpad,
finger,
x,
y,
@@ -436,41 +535,23 @@ fn run(
}
| Event::ControllerTouchpadMotion {
which,
touchpad,
finger,
x,
y,
..
} if active == Some(which) && w.attached.is_some() => {
let _ = w
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: true,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
w.forward_touch(which, touchpad as u32, finger as u8, x, y, true);
}
Event::ControllerTouchpadUp {
which,
touchpad,
finger,
x,
y,
..
} if active == Some(which) && w.attached.is_some() => {
let _ = w
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: false,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
w.forward_touch(which, touchpad as u32, finger as u8, x, y, false);
}
// Motion: accel events update the cache; each gyro event ships a sample (the
// DualSense reports both at ~250 Hz). Scale convention shared with the other
+3
View File
@@ -184,6 +184,9 @@ fn run_headless_cli(args: &[String], identity: (String, String)) {
decoder,
pin,
identity,
// Headless CLI uses the normal (short) handshake budget; the long request-access wait is a
// GUI-only flow.
connect_timeout: Duration::from_secs(15),
});
let deadline = Instant::now() + Duration::from_secs(60);
+6 -1
View File
@@ -34,6 +34,11 @@ pub struct SessionParams {
/// Pinned host fingerprint; `None` = trust on first use (caller persists the observed one).
pub pin: Option<[u8; 32]>,
pub identity: (String, String),
/// How long to wait for the handshake. The normal path uses a short budget; the
/// "request access" (delegated-approval) path uses a long one, because the host PARKS the
/// connection until the operator clicks Approve in its console (so this must exceed the
/// host's approval window — see `PENDING_APPROVAL_WAIT`).
pub connect_timeout: Duration,
}
#[derive(Clone, Copy, Default, PartialEq)]
@@ -164,7 +169,7 @@ fn pump(
None, // launch: the Windows client has no library picker yet
params.pin,
Some(params.identity),
Duration::from_secs(15),
params.connect_timeout,
) {
Ok(c) => Arc::new(c),
Err(e) => {
+167
View File
@@ -492,6 +492,10 @@ pub const PUNKTFUNK_HIDOUT_LED: u8 = 1;
pub const PUNKTFUNK_HIDOUT_PLAYER_LEDS: u8 = 2;
/// `PunktfunkHidOutput::kind` — one adaptive-trigger effect (`which` + `effect`/`effect_len` valid).
pub const PUNKTFUNK_HIDOUT_TRIGGER: u8 = 3;
/// `PunktfunkHidOutput::kind` — a trackpad haptic pulse (Steam Controller voice-coils). `which` =
/// side (0 = right pad, 1 = left pad); `effect[0..6]` packs `amplitude` / `period` / `count` as
/// little-endian `u16`s with `effect_len = 6`. Clients without trackpad coils drop it.
pub const PUNKTFUNK_HIDOUT_TRACKPAD_HAPTIC: u8 = 4;
/// Capacity of `PunktfunkHidOutput::effect` (the DualSense trigger parameter block).
pub const PUNKTFUNK_HID_EFFECT_MAX: u8 = 11;
@@ -559,6 +563,23 @@ impl PunktfunkHidOutput {
out.effect[..n].copy_from_slice(&effect[..n]);
out.effect_len = n as u8;
}
HidOutput::TrackpadHaptic {
pad,
side,
amplitude,
period,
count,
} => {
// No new struct (PunktfunkHidOutput has no size guard): pack into the existing
// `which` (side) + `effect[0..6]` (amplitude/period/count LE), `effect_len = 6`.
out.kind = PUNKTFUNK_HIDOUT_TRACKPAD_HAPTIC;
out.pad = *pad;
out.which = *side;
out.effect[0..2].copy_from_slice(&amplitude.to_le_bytes());
out.effect[2..4].copy_from_slice(&period.to_le_bytes());
out.effect[4..6].copy_from_slice(&count.to_le_bytes());
out.effect_len = 6;
}
}
out
}
@@ -618,6 +639,11 @@ impl PunktfunkHdrMeta {
pub const PUNKTFUNK_RICH_TOUCHPAD: u8 = 1;
/// `PunktfunkRichInput::kind` — a motion sample (`gyro`/`accel` valid).
pub const PUNKTFUNK_RICH_MOTION: u8 = 2;
/// `RichInput::TouchpadEx` kind on the wire — an extended trackpad contact that identifies the
/// surface (0 single / 1 Steam-left / 2 Steam-right) and carries click + pressure. The host decodes
/// it today; *sending* it from a C client needs the size-prefixed `PunktfunkRichInputEx` +
/// `punktfunk_connection_send_rich_input2` (added with client capture).
pub const PUNKTFUNK_RICH_TOUCHPAD_EX: u8 = 3;
/// One rich client→host input for the host's virtual DualSense
/// ([`punktfunk_connection_send_rich_input`]): a touchpad contact or a motion sample. Set `kind`
@@ -666,6 +692,77 @@ impl PunktfunkRichInput {
}
}
/// Forward-compatible superset of [`PunktfunkRichInput`] that can also express the rich Steam
/// surfaces: a *second* trackpad (`surface`), a distinct `click` vs touch, signed coordinates, and
/// pressure. Sent via [`punktfunk_connection_send_rich_input2`] — the only way a C client can emit a
/// `TouchpadEx`. The caller MUST set `struct_size = sizeof(PunktfunkRichInputEx)` (the ABI-skew
/// guard, like [`PunktfunkConfig`]); the legacy [`PunktfunkRichInput`] +
/// [`punktfunk_connection_send_rich_input`] stay byte-for-byte for existing callers.
#[cfg(feature = "quic")]
#[repr(C)]
#[derive(Clone, Copy)]
pub struct PunktfunkRichInputEx {
/// MUST equal `sizeof(PunktfunkRichInputEx)`.
pub struct_size: u32,
/// One of `PUNKTFUNK_RICH_*` (`TOUCHPAD` / `MOTION` / `TOUCHPAD_EX`).
pub kind: u8,
/// Gamepad index.
pub pad: u8,
/// Touchpad/TouchpadEx: contact id.
pub finger: u8,
/// Touchpad/TouchpadEx: 1 = finger down / touching, 0 = lifted.
pub active: u8,
/// TouchpadEx: which surface — 0 = single/DualSense, 1 = Steam left pad, 2 = Steam right pad.
pub surface: u8,
/// TouchpadEx: 1 = the pad is physically clicked (depressed), distinct from a touch contact.
pub click: u8,
/// Reserved for alignment; set to 0.
pub _reserved: [u8; 2],
/// TouchpadEx: x coordinate — **signed**, centred at 0 (the real Steam report convention). For a
/// legacy `TOUCHPAD` kind sent through this struct, store the unsigned `0..=65535` value's bits.
pub x: i16,
/// TouchpadEx: y coordinate — signed, centred at 0.
pub y: i16,
/// TouchpadEx: contact pressure (`0` if the surface has no force sensor).
pub pressure: u16,
/// Motion: gyro (pitch, yaw, roll), raw signed-16.
pub gyro: [i16; 3],
/// Motion: accelerometer (x, y, z), raw signed-16.
pub accel: [i16; 3],
}
#[cfg(feature = "quic")]
impl PunktfunkRichInputEx {
fn to_rich(self) -> Option<crate::quic::RichInput> {
use crate::quic::RichInput;
match self.kind {
PUNKTFUNK_RICH_TOUCHPAD_EX => Some(RichInput::TouchpadEx {
pad: self.pad,
surface: self.surface,
finger: self.finger,
touch: self.active != 0,
click: self.click != 0,
x: self.x,
y: self.y,
pressure: self.pressure,
}),
PUNKTFUNK_RICH_MOTION => Some(RichInput::Motion {
pad: self.pad,
gyro: self.gyro,
accel: self.accel,
}),
PUNKTFUNK_RICH_TOUCHPAD => Some(RichInput::Touchpad {
pad: self.pad,
finger: self.finger,
active: self.active != 0,
x: self.x as u16,
y: self.y as u16,
}),
_ => None,
}
}
}
/// Read an optional NUL-terminated UTF-8 string parameter; `Err` = invalid pointer/UTF-8.
#[cfg(feature = "quic")]
unsafe fn opt_cstr<'a>(p: *const std::os::raw::c_char) -> std::result::Result<Option<&'a str>, ()> {
@@ -714,6 +811,22 @@ pub const PUNKTFUNK_GAMEPAD_XBOXONE: u32 = 3;
/// DualSense (minus adaptive triggers / player LEDs / mute). Honored only where available (Linux
/// hosts); otherwise the host falls back to X-Box 360.
pub const PUNKTFUNK_GAMEPAD_DUALSHOCK4: u32 = 4;
/// UHID classic Steam Controller (Valve `28DE:1102`, kernel `hid-steam`): dual trackpads, gyro,
/// two grip paddles. Reserved — currently folds to `XBOX360` until its backend lands.
pub const PUNKTFUNK_GAMEPAD_STEAMCONTROLLER: u32 = 5;
/// UHID Steam Deck controller (Valve `28DE:1205`, kernel `hid-steam`): full Deck gamepad incl. the
/// four back grips, a right trackpad, and the IMU; re-grabbed by Steam Input with native glyphs when
/// Steam runs on the host. Honored only where available (Linux hosts); else folds to X-Box 360.
pub const PUNKTFUNK_GAMEPAD_STEAMDECK: u32 = 6;
/// Extended `InputEvent` gamepad button bits for embedders building raw events: the four back grips
/// (Steam L4/L5/R4/R5 ≙ Xbox-Elite P1P4) + the misc/capture button, in Moonlight's
/// `buttonFlags2 << 16` namespace. Mirror `input::gamepad::BTN_PADDLE1..4` / `BTN_MISC1`.
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE1: u32 = 0x0001_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE2: u32 = 0x0002_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE3: u32 = 0x0004_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_PADDLE4: u32 = 0x0008_0000;
pub const PUNKTFUNK_GAMEPAD_BTN_MISC1: u32 = 0x0020_0000;
/// Connect to a `punktfunk/1` host and start a session at `width`x`height`@`refresh_hz`.
/// Blocks up to `timeout_ms` for the handshake. Returns NULL on failure. Equivalent to
@@ -742,11 +855,28 @@ const _: () = {
// Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift).
const _: () = {
use crate::config::GamepadPref;
use crate::input::gamepad as g;
assert!(PUNKTFUNK_GAMEPAD_AUTO == GamepadPref::Auto.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_XBOX360 == GamepadPref::Xbox360.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_DUALSENSE == GamepadPref::DualSense.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_XBOXONE == GamepadPref::XboxOne.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_DUALSHOCK4 == GamepadPref::DualShock4.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_STEAMCONTROLLER == GamepadPref::SteamController.to_u8() as u32);
assert!(PUNKTFUNK_GAMEPAD_STEAMDECK == GamepadPref::SteamDeck.to_u8() as u32);
// Extended button bits mirror the wire `input::gamepad` constants.
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE1 == g::BTN_PADDLE1);
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE2 == g::BTN_PADDLE2);
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE3 == g::BTN_PADDLE3);
assert!(PUNKTFUNK_GAMEPAD_BTN_PADDLE4 == g::BTN_PADDLE4);
assert!(PUNKTFUNK_GAMEPAD_BTN_MISC1 == g::BTN_MISC1);
};
// The additive M3 kinds (TouchpadEx / TrackpadHaptic) must never grow the legacy ABI structs —
// they have no `struct_size` guard, so a layout change would corrupt old-built callers' buffers.
#[cfg(feature = "quic")]
const _: () = {
assert!(core::mem::size_of::<PunktfunkRichInput>() == 20);
assert!(core::mem::size_of::<PunktfunkHidOutput>() == 19);
};
/// Trust: `pin_sha256` (NULL or 32 bytes) is the expected SHA-256 fingerprint of the host's
@@ -1727,6 +1857,43 @@ pub unsafe extern "C" fn punktfunk_connection_send_rich_input(
})
}
/// Send a rich client→host input via the forward-compatible [`PunktfunkRichInputEx`] — the only way
/// a C client can emit a `TouchpadEx` (a second trackpad / signed coords / pressure). Set
/// `rich->struct_size = sizeof(PunktfunkRichInputEx)`; a smaller (older-layout) value is rejected.
///
/// # Safety
/// `c` is a valid connection handle; `rich` is null or points to at least its declared
/// `struct_size` bytes.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_send_rich_input2(
c: *mut PunktfunkConnection,
rich: *const PunktfunkRichInputEx,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if rich.is_null() {
return PunktfunkStatus::NullPointer;
}
// Read only the 4-byte size prefix first to bound the subsequent full read (the
// `PunktfunkConfig` ABI-skew precedent).
let declared = unsafe { std::ptr::addr_of!((*rich).struct_size).read_unaligned() } as usize;
if declared < std::mem::size_of::<PunktfunkRichInputEx>() {
return PunktfunkStatus::InvalidArg;
}
match unsafe { *rich }.to_rich() {
Some(r) => match c.inner.send_rich_input(r) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
},
None => PunktfunkStatus::InvalidArg,
}
})
}
/// The currently active session mode — the Welcome's, until an accepted
/// [`punktfunk_connection_request_mode`] switches it. Safe any time after connect.
///
+45 -4
View File
@@ -137,8 +137,9 @@ impl CompositorPref {
/// host decide (its `PUNKTFUNK_GAMEPAD` env var, else X-Box 360). A concrete preference is
/// honored only if that backend is available on the host (DualSense / DualShock 4 need Linux UHID);
/// otherwise the host falls back and reports the real choice in `Welcome`. The wire form is a single
/// byte (`0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`), appended to
/// `Hello`/`Welcome` — older peers simply omit/ignore it (an unknown byte degrades to `Auto`).
/// byte (`0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`,
/// `5 = SteamController`, `6 = SteamDeck`), appended to `Hello`/`Welcome` — older peers simply
/// omit/ignore it (an unknown byte degrades to `Auto`).
#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)]
pub enum GamepadPref {
/// Let the host pick (its `PUNKTFUNK_GAMEPAD` env var, else X-Box 360).
@@ -155,10 +156,19 @@ pub enum GamepadPref {
/// UHID DualShock 4 (kernel `hid-playstation`, ≥ 6.2) — lightbar, touchpad, motion, rumble. Like
/// `DualSense` minus adaptive triggers / player LEDs / mute. Needs Linux UHID on the host.
DualShock4,
/// UHID classic Steam Controller (Valve `28DE:1102`, kernel `hid-steam`) — dual trackpads, gyro,
/// two grip paddles, trackpad-only haptics. Needs Linux UHID. *(Reserved; its backend is not yet
/// built — currently folds to `Xbox360`; the Deck identity below is the implemented one.)*
SteamController,
/// UHID Steam Deck controller (Valve `28DE:1205`, kernel `hid-steam`) — full Deck gamepad incl.
/// the four back grips (L4/L5/R4/R5), a right trackpad, and the IMU; re-grabbed by Steam Input
/// with native glyphs when Steam runs on the host. Needs Linux UHID.
SteamDeck,
}
impl GamepadPref {
/// Wire byte. `0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`.
/// Wire byte. `0 = Auto`, `1 = Xbox360`, `2 = DualSense`, `3 = XboxOne`, `4 = DualShock4`,
/// `5 = SteamController`, `6 = SteamDeck`.
pub const fn to_u8(self) -> u8 {
match self {
GamepadPref::Auto => 0,
@@ -166,6 +176,8 @@ impl GamepadPref {
GamepadPref::DualSense => 2,
GamepadPref::XboxOne => 3,
GamepadPref::DualShock4 => 4,
GamepadPref::SteamController => 5,
GamepadPref::SteamDeck => 6,
}
}
@@ -177,6 +189,8 @@ impl GamepadPref {
2 => GamepadPref::DualSense,
3 => GamepadPref::XboxOne,
4 => GamepadPref::DualShock4,
5 => GamepadPref::SteamController,
6 => GamepadPref::SteamDeck,
_ => GamepadPref::Auto,
}
}
@@ -192,12 +206,14 @@ impl GamepadPref {
GamepadPref::XboxOne
}
"dualshock4" | "dualshock" | "ds4" | "ps4" => GamepadPref::DualShock4,
"steamdeck" | "steam-deck" | "deck" => GamepadPref::SteamDeck,
"steamcontroller" | "steam-controller" | "steamcon" => GamepadPref::SteamController,
_ => return None,
})
}
/// Canonical lowercase identifier (`"auto"`, `"xbox360"`, `"dualsense"`, `"xboxone"`,
/// `"dualshock4"`).
/// `"dualshock4"`, `"steamcontroller"`, `"steamdeck"`).
pub fn as_str(self) -> &'static str {
match self {
GamepadPref::Auto => "auto",
@@ -205,6 +221,8 @@ impl GamepadPref {
GamepadPref::DualSense => "dualsense",
GamepadPref::XboxOne => "xboxone",
GamepadPref::DualShock4 => "dualshock4",
GamepadPref::SteamController => "steamcontroller",
GamepadPref::SteamDeck => "steamdeck",
}
}
}
@@ -381,4 +399,27 @@ mod tests {
c.fec.fec_percent = 15; // 250 + ceil(250*15/100)=288 > 255
assert!(c.validate().is_err());
}
#[test]
fn gamepad_pref_steam_roundtrip() {
use GamepadPref::*;
// Wire-byte round-trip for the Steam additions; an unknown byte still degrades to Auto.
for (p, b) in [(SteamController, 5u8), (SteamDeck, 6)] {
assert_eq!(p.to_u8(), b);
assert_eq!(GamepadPref::from_u8(b), p);
}
assert_eq!(GamepadPref::from_u8(99), Auto);
// Name parsing + canonical-name round-trip.
assert_eq!(GamepadPref::from_name("steamdeck"), Some(SteamDeck));
assert_eq!(GamepadPref::from_name("deck"), Some(SteamDeck));
assert_eq!(
GamepadPref::from_name("steamcontroller"),
Some(SteamController)
);
assert_eq!(SteamDeck.as_str(), "steamdeck");
assert_eq!(
GamepadPref::from_name(SteamController.as_str()),
Some(SteamController)
);
}
}
+14
View File
@@ -66,10 +66,24 @@ pub mod gamepad {
pub const BTN_B: u32 = 0x2000;
pub const BTN_X: u32 = 0x4000;
pub const BTN_Y: u32 = 0x8000;
// Extended buttons in Moonlight's `buttonFlags2 << 16` namespace (see `gamestream/gamepad.rs`),
// so the GameStream paddle path and the native path share one host injector map. The four Steam
// Deck back grips (L4/L5/R4/R5) reuse the four GameStream/Xbox-Elite paddle slots — a semantic
// 1:1 for binding (the device identity carries the glyph distinction).
/// Back grip R4 — SDL `RightPaddle1` / GameStream `PADDLE1`.
pub const BTN_PADDLE1: u32 = 0x0001_0000;
/// Back grip L4 — SDL `LeftPaddle1` / GameStream `PADDLE2`.
pub const BTN_PADDLE2: u32 = 0x0002_0000;
/// Back grip R5 — SDL `RightPaddle2` / GameStream `PADDLE3`.
pub const BTN_PADDLE3: u32 = 0x0004_0000;
/// Back grip L5 — SDL `LeftPaddle2` / GameStream `PADDLE4`.
pub const BTN_PADDLE4: u32 = 0x0008_0000;
/// DualSense touchpad click. Moonlight's extended-button position (`buttonFlags2`
/// merges in at `<< 16`, see `gamestream/gamepad.rs`), so GameStream clients land on
/// the same bit. Only the DualSense backend renders it; the xpad has no such button.
pub const BTN_TOUCHPAD: u32 = 0x10_0000;
/// Misc / capture button — the Deck `…`/quick-access, Share/Capture / GameStream `MISC`.
pub const BTN_MISC1: u32 = 0x0020_0000;
/// Axis ids for `InputKind::GamepadAxis`.
pub const AXIS_LS_X: u32 = 0;
+92 -1
View File
@@ -1218,6 +1218,7 @@ pub fn decode_mic_datagram(b: &[u8]) -> Option<(u32, u64, &[u8])> {
const RICH_TOUCHPAD: u8 = 0x01;
const RICH_MOTION: u8 = 0x02;
const RICH_TOUCHPAD_EX: u8 = 0x03;
/// A rich client→host controller input beyond the fixed [`InputEvent`](crate::input::InputEvent):
/// the DualSense touchpad and motion sensors. `pad` is the gamepad index. Wire form is
@@ -1241,6 +1242,22 @@ pub enum RichInput {
gyro: [i16; 3],
accel: [i16; 3],
},
/// A richer trackpad contact that also identifies *which* physical pad (Steam Controller / Deck
/// have two), carries a separate click vs touch state, and a pressure reading. `surface`:
/// `0` = the single / DualSense touchpad, `1` = the Steam left pad, `2` = the Steam right pad.
/// Coordinates are **signed** (centred at 0), matching the real Steam report; `pressure` is `0`
/// for a surface with no force sensor. New clients send this for every touch surface; the host
/// decodes both `Touchpad` (`0x01`) and `TouchpadEx` (`0x03`) indefinitely.
TouchpadEx {
pad: u8,
surface: u8,
finger: u8,
touch: bool,
click: bool,
x: i16,
y: i16,
pressure: u16,
},
}
impl RichInput {
@@ -1264,6 +1281,22 @@ impl RichInput {
out.extend_from_slice(&v.to_le_bytes());
}
}
RichInput::TouchpadEx {
pad,
surface,
finger,
touch,
click,
x,
y,
pressure,
} => {
let state = (touch as u8) | ((click as u8) << 1);
out.extend_from_slice(&[RICH_TOUCHPAD_EX, pad, surface, finger, state]);
out.extend_from_slice(&x.to_le_bytes());
out.extend_from_slice(&y.to_le_bytes());
out.extend_from_slice(&pressure.to_le_bytes());
}
}
out
}
@@ -1288,6 +1321,16 @@ impl RichInput {
accel: [i16at(9), i16at(11), i16at(13)],
})
}
RICH_TOUCHPAD_EX if b.len() >= 12 => Some(RichInput::TouchpadEx {
pad: b[2],
surface: b[3],
finger: b[4],
touch: b[5] & 0x01 != 0,
click: b[5] & 0x02 != 0,
x: i16::from_le_bytes([b[6], b[7]]),
y: i16::from_le_bytes([b[8], b[9]]),
pressure: u16::from_le_bytes([b[10], b[11]]),
}),
_ => None,
}
}
@@ -1296,6 +1339,7 @@ impl RichInput {
const HIDOUT_LED: u8 = 0x01;
const HIDOUT_PLAYER_LEDS: u8 = 0x02;
const HIDOUT_TRIGGER: u8 = 0x03;
const HIDOUT_TRACKPAD_HAPTIC: u8 = 0x04;
/// DualSense feedback flowing host → client (what a game wrote to the host's virtual pad).
/// Wire form `[0xCD][kind][pad][fields…]`. The rich analog of the fixed rumble datagram;
@@ -1309,6 +1353,16 @@ pub enum HidOutput {
/// One adaptive-trigger effect: `which` 0 = L2, 1 = R2; `effect` is the raw DualSense
/// trigger parameter block (mode + params) for the client to replay on a real controller.
Trigger { pad: u8, which: u8, effect: Vec<u8> },
/// A trackpad haptic pulse for a Steam Controller's voice-coil actuators (its only "rumble").
/// `side` 0 = right pad, 1 = left pad; `amplitude` + `period` (µs off-time) + `count` (pulses)
/// synthesize a buzz. A client without trackpad coils drops it (or maps it to ordinary rumble).
TrackpadHaptic {
pad: u8,
side: u8,
amplitude: u16,
period: u16,
count: u16,
},
}
impl HidOutput {
@@ -1325,6 +1379,18 @@ impl HidOutput {
out.extend_from_slice(&[HIDOUT_TRIGGER, *pad, *which]);
out.extend_from_slice(effect);
}
HidOutput::TrackpadHaptic {
pad,
side,
amplitude,
period,
count,
} => {
out.extend_from_slice(&[HIDOUT_TRACKPAD_HAPTIC, *pad, *side]);
out.extend_from_slice(&amplitude.to_le_bytes());
out.extend_from_slice(&period.to_le_bytes());
out.extend_from_slice(&count.to_le_bytes());
}
}
out
}
@@ -1349,6 +1415,13 @@ impl HidOutput {
which: b[3],
effect: b[4..].to_vec(),
}),
HIDOUT_TRACKPAD_HAPTIC if b.len() >= 10 => Some(HidOutput::TrackpadHaptic {
pad: b[2],
side: b[3],
amplitude: u16::from_le_bytes([b[4], b[5]]),
period: u16::from_le_bytes([b[6], b[7]]),
count: u16::from_le_bytes([b[8], b[9]]),
}),
_ => None,
}
}
@@ -2486,6 +2559,16 @@ mod tests {
gyro: [-100, 200, -300],
accel: [16384, -8192, 1],
},
RichInput::TouchpadEx {
pad: 2,
surface: 1,
finger: 1,
touch: true,
click: false,
x: -12345,
y: 30000,
pressure: 4000,
},
] {
let d = ev.encode();
assert_eq!(d[0], RICH_INPUT_MAGIC);
@@ -2494,7 +2577,8 @@ mod tests {
// Disjoint from the fixed input datagram (0xC8); unknown kind + truncation → None.
assert!(RichInput::decode(&[crate::input::INPUT_MAGIC; 18]).is_none());
assert!(RichInput::decode(&[RICH_INPUT_MAGIC, 0x7F]).is_none()); // unknown kind
assert!(RichInput::decode(&[RICH_INPUT_MAGIC, RICH_TOUCHPAD, 0]).is_none());
assert!(RichInput::decode(&[RICH_INPUT_MAGIC, RICH_TOUCHPAD, 0]).is_none()); // short
assert!(RichInput::decode(&[RICH_INPUT_MAGIC, RICH_TOUCHPAD_EX, 0, 0, 0, 0]).is_none());
// short
}
@@ -2516,6 +2600,13 @@ mod tests {
which: 1,
effect: vec![0x26, 0x90, 0xA0, 0xFF, 0x00, 0x00],
},
HidOutput::TrackpadHaptic {
pad: 0,
side: 1,
amplitude: 0x1234,
period: 0x5678,
count: 9,
},
];
for ev in &cases {
let d = ev.encode();
+14 -4
View File
@@ -35,9 +35,11 @@ base64 = "0.22"
ureq = "2"
rcgen = { version = "0.13", default-features = false, features = ["aws_lc_rs", "pem"] }
x509-parser = "0.16"
axum-server = { version = "0.7", features = ["tls-rustls"] }
# Only used for the plain-HTTP nvhttp listener (`bind().serve()`); HTTPS/mTLS is hand-rolled over
# tokio-rustls (axum-server can't surface the peer cert), so we do NOT enable `tls-rustls` — that
# feature is what pulled the unmaintained `rustls-pemfile` (security-review dep hygiene).
axum-server = "0.8"
rustls = "0.23"
rustls-pemfile = "2"
# Manual HTTPS+mTLS serve loop for the mgmt API (axum-server can't surface the peer cert): a
# tokio-rustls handshake exposes the client cert, then hyper serves the axum Router with the
# verified fingerprint injected as a request extension. Versions match the workspace lock.
@@ -87,6 +89,9 @@ tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] }
wayland-client = "0.31"
wayland-protocols-wlr = { version = "0.3", features = ["client"] }
wayland-protocols-misc = { version = "0.3", features = ["client"] }
# `xdg-output` (zxdg_output_v1): the per-output *logical* geometry (post-scale size + global
# position), used by the KWin fake_input backend to map absolute coordinates under display scaling.
wayland-protocols = { version = "0.32", features = ["client"] }
# Codegen for KDE's `zkde_screencast_unstable_v1` (vendored in `protocols/`): create a KWin
# virtual output sized to the client's resolution and get its PipeWire node (KRdp's path).
# `wayland-backend` is referenced by the generated interface tables.
@@ -117,6 +122,10 @@ ash = "0.38"
# `libcuda.so.1` is dlopen'd at runtime (NOT link-time) so one Linux binary runs on NVIDIA
# (zero-copy via CUDA) AND on AMD/Intel (VAAPI, no NVIDIA driver present) — see `zerocopy::cuda`.
libloading = "0.8"
# Vendored + trimmed `usbip` server core (no libusb) — presents a virtual Steam Deck over USB/IP
# so the local `vhci_hcd` attaches it: the shippable, Secure-Boot-clean, Steam-Input-promotable
# virtual-Deck transport on non-SteamOS hosts (`inject/linux/steam_usbip.rs`). See the crate's NOTICE.
usbip-sim = { path = "vendor/usbip-sim" }
[target.'cfg(target_os = "windows")'.dependencies]
# Windows host backends. `windows` covers the Win32/CCD APIs the SudoVDA virtual-display backend
@@ -217,6 +226,7 @@ bytemuck = { version = "1.19", features = ["derive"] }
# nvEncodeAPI64.dll) on the linker path. Build the GPU host with `--features nvenc`.
nvenc = ["dep:nvidia-video-codec-sdk"]
# AMD/Intel hardware encode on Windows (AMF/QSV via ffmpeg-next). OFF by default: it needs a
# `FFMPEG_DIR` (BtbN gpl-shared, includes `*_amf`/`*_qsv`) at build time and bundles the FFmpeg
# DLLs at runtime. Build the all-vendor GPU host with `--features nvenc,amf-qsv`.
# `FFMPEG_DIR` (BtbN lgpl-shared includes `*_amf`/`*_qsv`; the GPL-only x264/x265 are never used,
# so the LGPL build suffices and keeps the bundled DLLs LGPL, not GPL) at build time and bundles the
# FFmpeg DLLs at runtime. Build the all-vendor GPU host with `--features nvenc,amf-qsv`.
amf-qsv = ["dep:ffmpeg-next"]
@@ -188,7 +188,8 @@ pub(crate) unsafe fn make_device(
let device = device.context("null D3D11 device")?;
let context = context.context("null D3D11 context")?;
// Apollo-style GPU scheduling hardening (Sunshine display_base.cpp:599-709). Our capture+encode
// GPU scheduling hardening — the same approach Sunshine/Apollo use, reimplemented here via the
// documented D3DKMT/DXGI APIs (no GPL source copied). Our capture+encode
// shares the GPU with the streamed game; when the game saturates the GPU our process is starved of
// GPU time slices, so NVENC sits near-idle yet `lock_bitstream` waits ~20 ms for our context to be
// scheduled — capping the stream (~47 fps measured at 5K@240) and stuttering. Per-frame copy/convert
@@ -197,7 +198,7 @@ pub(crate) unsafe fn make_device(
// GPU thread priority and a 1-frame latency cap.
elevate_process_gpu_priority();
if let Ok(dxgi_dev) = device.cast::<IDXGIDevice>() {
// Apollo's absolute max GPU thread priority (0x4000001E); fall back to relative +7.
// The absolute max GPU thread priority (0x4000001E; the same value Sunshine/Apollo use); fall back to relative +7.
if dxgi_dev.SetGPUThreadPriority(0x4000_001E).is_err()
&& dxgi_dev.SetGPUThreadPriority(7).is_err()
{
@@ -291,7 +292,8 @@ unsafe fn d3dkmt_set_scheduling_priority_class(
Some(f(process, prio))
}
/// Apollo-style GPU scheduling-priority hardening (Sunshine `display_base.cpp:599-709`). On a
/// GPU scheduling-priority hardening — the same approach as Sunshine/Apollo, independently
/// implemented via the documented D3DKMT APIs (no GPL source copied). On a
/// GPU-saturated game our capture+encode process is starved of GPU time slices — NVENC sits ~idle but
/// `lock_bitstream` waits ~20 ms for our context to be scheduled. Elevating the PROCESS GPU scheduling
/// priority class (the strong cross-process lever — far more effective than `SetGPUThreadPriority`
@@ -532,7 +534,9 @@ const ES_DISPLAY_REQUIRED: u32 = 0x0000_0002;
/// Replacement for `win32u.dll!NtGdiDdDDIGetCachedHybridQueryValue`: always report
/// `D3DKMT_GPU_PREFERENCE_STATE_UNSPECIFIED` (3). We fully replace the function (never call the
/// original), so no trampoline is needed. (Ported verbatim from Apollo's MinHook hook.)
/// original), so no trampoline is needed. (Independent reimplementation of the same technique Apollo
/// uses: Apollo installs its hook via the MinHook library; this is an original inline byte-patch and
/// copies no Apollo/GPL source.)
unsafe extern "system" fn hybrid_query_hook(gpu_preference: *mut u32) -> i32 {
HYBRID_HOOK_HITS.fetch_add(1, Ordering::Relaxed);
if gpu_preference.is_null() {
@@ -542,7 +546,8 @@ unsafe extern "system" fn hybrid_query_hook(gpu_preference: *mut u32) -> i32 {
0 // STATUS_SUCCESS
}
/// Apollo's win32u GPU-preference hook, ported. On a HYBRID-GPU box DXGI resolves a GPU preference
/// The win32u GPU-preference hook (the same technique Apollo applies, reimplemented here from the
/// documented DDI — no GPL source copied). On a HYBRID-GPU box DXGI resolves a GPU preference
/// (registry + power settings + the hybrid-adapter DDI) and REPARENTS outputs onto the chosen render
/// GPU — which constantly invalidates Desktop Duplication (DXGI_ERROR_ACCESS_LOST 0x887A0026, the
/// freeze/churn observed on the RTX 4090 + AMD iGPU box; `SET_RENDER_ADAPTER` is ignored there). Faking
@@ -555,7 +560,7 @@ pub(crate) fn install_gpu_pref_hook() {
// SAFETY: this one-time hook install only touches a region it has just validated.
// `LoadLibraryA("win32u.dll")` + `GetProcAddress("NtGdiDdDDIGetCachedHybridQueryValue")` yield the
// live base of the real exported function, so `target` is a valid executable code pointer to at
// least the 12 bytes the patch overwrites (an x64 prologue, per Apollo's verified hook). The two
// least the 12 bytes the patch overwrites (an x64 prologue). The two
// `ptr::copy_nonoverlapping`s each move exactly 12 bytes between the 12-byte stack arrays
// (`patch`/`readback`) and `target`, which `VirtualProtect(target, 12, PAGE_EXECUTE_READWRITE, …)`
// has just made writable (and is restored to `old` after) — source and dest never overlap (stack
@@ -1,8 +1,8 @@
//! P2 direct frame push (kill DDA) — HOST side. The pf-vdisplay driver runs in a restricted WUDFHost
//! token that canNOT create named kernel objects, so — exactly like the gamepad UMDF drivers
//! (`inject/dualsense_windows.rs`) — the HOST (privileged) CREATES the shared header + frame-ready
//! event + ring of keyed-mutex textures (`Global\` names, permissive `D:(A;;GA;;;WD)` SDDL) on the
//! discrete render GPU, and the driver only OPENS them and copies frames in. We then consume the ring
//! P2 direct frame push (kill DDA) — HOST side. The pf-vdisplay driver's WUDFHost canNOT create named
//! kernel objects, so — exactly like the gamepad UMDF drivers (`inject/dualsense_windows.rs`) — the
//! HOST (privileged) CREATES the shared header + frame-ready event + ring of keyed-mutex textures
//! (`Global\` names, scoped `D:(A;;GA;;;SY)(A;;GA;;;LS)` to SYSTEM + the driver's LocalService host —
//! see `shared_object_sa`) on the discrete render GPU, and the driver only OPENS them and copies frames in. We then consume the ring
//! straight into the zero-copy NVENC path — no DXGI Desktop Duplication, no `win32u` hook. Gated by
//! `PUNKTFUNK_IDD_PUSH`. Driver counterpart: `packaging/windows/drivers/pf-vdisplay/src/
//! frame_transport.rs`. The shared `SharedHeader` layout, `MAGIC`/`VERSION`/`RING_LEN`, the
@@ -236,13 +236,17 @@ pub struct IddPushCapturer {
// ownership to one thread at a time with NO concurrent access; we do not (and must not) claim `Sync`.
unsafe impl Send for IddPushCapturer {}
/// Build a permissive (Everyone:GenericAll) `SECURITY_ATTRIBUTES` so the restricted WUDFHost driver
/// can OPEN the host-created objects — the same `D:(A;;GA;;;WD)` SDDL the gamepad shared section uses.
/// The returned `psd` backing must outlive `sa`; both are dropped when the process exits.
unsafe fn permissive_sa() -> Result<(SECURITY_ATTRIBUTES, PSECURITY_DESCRIPTOR)> {
/// Build a `SECURITY_ATTRIBUTES` granting GENERIC_ALL to **SYSTEM** (the host creates+publishes the
/// shared event + texture ring) and **LocalService** (the account the pf_vdisplay WUDFHost runs under,
/// which consumes them) — `D:(A;;GA;;;SY)(A;;GA;;;LS)`, the same scoping as the gamepad section. The
/// old SDDL granted **Everyone** (`WD`), which let any local user open the `Global\pfvd-*` objects and
/// read captured screen frames (security-review 2026-06-28 #5). Verified on the RTX box (2026-06-29):
/// the WUDFHost token is `S-1-5-19` (LocalService), SYSTEM integrity, zero restricted SIDs — so SY+LS
/// suffices for the driver and excludes normal user processes. `psd` must outlive `sa`.
unsafe fn shared_object_sa() -> Result<(SECURITY_ATTRIBUTES, PSECURITY_DESCRIPTOR)> {
let mut psd = PSECURITY_DESCRIPTOR::default();
ConvertStringSecurityDescriptorToSecurityDescriptorW(
w!("D:(A;;GA;;;WD)"),
w!("D:(A;;GA;;;SY)(A;;GA;;;LS)"),
SDDL_REVISION_1,
&mut psd,
None,
@@ -269,7 +273,7 @@ impl IddPushCapturer {
h: u32,
format: DXGI_FORMAT,
) -> Result<Vec<HostSlot>> {
let (sa, _psd) = permissive_sa()?;
let (sa, _psd) = shared_object_sa()?;
let mut slots = Vec::new();
for k in 0..RING_LEN {
let desc = D3D11_TEXTURE2D_DESC {
@@ -375,7 +379,7 @@ impl IddPushCapturer {
// SAFETY: one block over the whole ring setup; every operation in it is sound:
// - `set_advanced_color`/`advanced_color_enabled` are `unsafe fn`s taking only a copy of the plain
// `u32` target id; they read/flip CCD display config and return owned values, borrowing nothing.
// - `CreateDXGIFactory1`, `EnumAdapterByLuid`, `make_device`, `permissive_sa`, `CreateFileMappingW`,
// - `CreateDXGIFactory1`, `EnumAdapterByLuid`, `make_device`, `shared_object_sa`, `CreateFileMappingW`,
// `MapViewOfFile`, `CreateEventW`, and `create_ring_slots` are all `?`-checked, so every returned
// interface/handle/view is non-error before use; `&sa`/`&adapter`/`&device`/the `&HSTRING` names
// are live borrows that outlive each synchronous call, and `sa.lpSecurityDescriptor` stays valid
@@ -421,7 +425,7 @@ impl IddPushCapturer {
.context("EnumAdapterByLuid(render adapter) for IDD push")?;
let (device, context) = make_device(&adapter).context("make_device for IDD push")?;
let (sa, _psd) = permissive_sa()?;
let (sa, _psd) = shared_object_sa()?;
let bytes = std::mem::size_of::<SharedHeader>().max(64);
// Header.
+8
View File
@@ -230,6 +230,14 @@ pub fn open_video(
chroma: ChromaFormat,
) -> Result<Box<dyn Encoder>> {
validate_dimensions(codec, width, height)?;
// Refresh/fps must be positive and sane: fps feeds the encoder time_base (`Rational(1, fps)`)
// and the pts→ns conversion (`pts * 1e9 / fps`), so 0 builds a 1/0 rational / divides by zero.
// The mid-stream Reconfigure path already guards `refresh_hz > 0`; enforcing it at this single
// open chokepoint makes EVERY path (initial Hello, GameStream ANNOUNCE, Reconfigure) safe
// regardless of which backend opens (security-review 2026-06-28 S5).
if fps == 0 || fps > 1000 {
anyhow::bail!("invalid refresh/fps {fps}: must be 1..=1000 Hz");
}
// 4:4:4 is HEVC-only. The negotiator should never pass `Yuv444` for another codec (it gates on
// `codec == H265`), but defend the contract here so a future caller can't silently emit a stream
// no decoder expects: a non-HEVC 4:4:4 request degrades to 4:2:0 with a warning.
@@ -56,6 +56,9 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
.spawn(move || {
// GCM scheme detected from the first authenticating packet; reused thereafter.
let mut detected: Option<Scheme> = None;
// Consecutive control-decrypt failures for this peer — throttles the warn log so a
// junk-packet flood can't spam unbounded lines (security-review 2026-06-28 #10).
let mut decrypt_fails: u64 = 0;
// Decoded keyboard/mouse is forwarded to a dedicated host-lifetime injector thread —
// NEVER injected inline, so a slow Wayland/libei/SendInput call can't head-block ENet
// keepalive/retransmit servicing on this thread. The injector owns non-Send compositor
@@ -77,6 +80,7 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
Event::Disconnect { .. } => {
tracing::info!("control: client disconnected");
detected = None;
decrypt_fails = 0;
peer = None;
// Unplug the session's virtual pads.
pads = GamepadManager::new();
@@ -89,6 +93,7 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
channel_id,
packet.data(),
&mut detected,
&mut decrypt_fails,
&inj_tx,
&mut pads,
);
@@ -163,6 +168,7 @@ fn on_receive(
_channel_id: u8,
d: &[u8],
detected: &mut Option<Scheme>,
decrypt_fails: &mut u64,
inj_tx: &Sender<InputEvent>,
pads: &mut GamepadManager,
) {
@@ -180,10 +186,20 @@ fn on_receive(
tracing::info!(?scheme, "control: GCM scheme locked in");
}
*detected = Some(scheme);
*decrypt_fails = 0;
pt
}
None => {
tracing::warn!(len = d.len(), "control: GCM decrypt failed");
// Throttle: a junk-packet flood must not spam one warn line per packet. Log the first
// failure, then only at exponentially-spaced counts (1, 2, 4, 8, …).
*decrypt_fails += 1;
if decrypt_fails.is_power_of_two() {
tracing::warn!(
len = d.len(),
fails = *decrypt_fails,
"control: GCM decrypt failed"
);
}
return;
}
};
@@ -66,6 +66,13 @@ pub const BTN_A: u32 = 0x1000;
pub const BTN_B: u32 = 0x2000;
pub const BTN_X: u32 = 0x4000;
pub const BTN_Y: u32 = 0x8000;
// Extended buttons in the `buttonFlags2 << 16` namespace (mirror `punktfunk_core::input::gamepad`):
// the four back-grip paddles. `decode` already merges `buttonFlags2 << 16` into `buttons`, but the
// injector map dropped these bits — Sunshine/Moonlight paddle clients were silently no-op'd.
pub const BTN_PADDLE1: u32 = 0x0001_0000;
pub const BTN_PADDLE2: u32 = 0x0002_0000;
pub const BTN_PADDLE3: u32 = 0x0004_0000;
pub const BTN_PADDLE4: u32 = 0x0008_0000;
/// Decode one decrypted control plaintext into a controller event, if it is one. Mouse,
/// keyboard, keepalives etc. yield `None` (they're handled by [`super::input::decode`]).
+63 -4
View File
@@ -90,6 +90,11 @@ pub struct LaunchSession {
pub fps: u32,
/// `/launch?appid=N` — selects the app-catalog entry (session recipe).
pub appid: u32,
/// Source IP of the paired HTTPS client that issued `/launch`. The unauthenticated RTSP/UDP
/// media plane binds to this so only the launching peer can start/own the stream — an
/// unpaired RTSP peer cannot ride a paired client's launch (security-review 2026-06-28 #4).
/// `None` if the address could not be captured (then RTSP falls back to launch-present only).
pub peer_ip: Option<std::net::IpAddr>,
}
/// Shared control-plane state used as the axum app state.
@@ -262,9 +267,10 @@ pub(crate) fn config_dir() -> PathBuf {
}
/// Create `dir` (and parents) owner-private — **0700** on Unix (so the host's secrets aren't readable
/// by other local users via a traversable config path). Best-effort on Windows: the dir inherits the
/// (Users-readable) `%ProgramData%` ACL, so secret *files* are individually locked down by
/// [`write_secret_file`]. Tightens an already-existing dir too.
/// by other local users via a traversable config path). On Windows, applies a restrictive DACL
/// ([`restrict_dir_to_system_admins`]) so a local unprivileged user can't pre-create / plant files in
/// the config tree (the default `%ProgramData%` ACL grants Users *create*; security-review
/// 2026-06-28 #3/#11). Tightens (and re-owns) an already-existing dir too.
pub(crate) fn create_private_dir(dir: &std::path::Path) -> std::io::Result<()> {
#[cfg(unix)]
{
@@ -281,7 +287,60 @@ pub(crate) fn create_private_dir(dir: &std::path::Path) -> std::io::Result<()> {
}
#[cfg(not(unix))]
{
std::fs::create_dir_all(dir)
let r = std::fs::create_dir_all(dir);
#[cfg(windows)]
restrict_dir_to_system_admins(dir);
r
}
}
/// Best-effort Windows DACL lockdown of the config *directory* (the companion to
/// [`restrict_to_system_admins`] for files). The default `%ProgramData%` ACL lets `BUILTIN\Users`
/// create subfolders/files (and become `CREATOR OWNER`), so a non-admin could pre-create the
/// `punktfunk` dir or plant a `host.env`/`apps.json` that the privileged SYSTEM service then trusts
/// (LPE; security-review 2026-06-28 #3). This re-owns the dir to Administrators (defeating a
/// pre-creation), strips inheritance, and sets an explicit DACL: SYSTEM/Administrators/OWNER full
/// (object+container inherit so child files/dirs inherit it), and Users **read-only** (so existing
/// reads of non-secret config keep working but a local user can no longer write/plant). Secret files
/// are additionally locked to SYSTEM/Admins by [`write_secret_file`]. Hard-coded SIDs
/// (locale-independent) via the absolute `%SystemRoot%` path; never fatal.
#[cfg(windows)]
fn restrict_dir_to_system_admins(dir: &std::path::Path) {
let icacls = std::env::var("SystemRoot")
.map(|r| format!("{r}\\System32\\icacls.exe"))
.unwrap_or_else(|_| "icacls".to_string());
// Reset ownership of the directory object to Administrators first, so a dir a non-admin may have
// pre-created can't keep OWNER control (an owner can always rewrite the DACL). No `/T` — re-owning
// the dir itself is what defeats the pre-creation; recursing a large captures tree each call is
// needless churn (secret files are individually owner-locked by `write_secret_file`).
let _ = std::process::Command::new(&icacls)
.arg(dir.as_os_str())
.args(["/setowner", "*S-1-5-32-544"]) // BUILTIN\Administrators
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status();
let status = std::process::Command::new(&icacls)
.arg(dir.as_os_str())
.args([
"/inheritance:r",
"/grant:r",
"*S-1-5-18:(OI)(CI)(F)", // NT AUTHORITY\SYSTEM
"/grant:r",
"*S-1-5-32-544:(OI)(CI)(F)", // BUILTIN\Administrators
"/grant:r",
"*S-1-3-4:(OI)(CI)(F)", // OWNER RIGHTS
"/grant:r",
"*S-1-5-32-545:(OI)(CI)(RX)", // BUILTIN\Users — read-only (no create/write → no plant)
])
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status();
match status {
Ok(s) if s.success() => {}
_ => tracing::warn!(
dir = %dir.display(),
"config-dir DACL hardening did not fully succeed — a local user may be able to plant config files"
),
}
}
+13 -18
View File
@@ -1,9 +1,14 @@
//! The nvhttp servers: plain HTTP on 47989 and mutual-TLS on 47984. Serves `/serverinfo`,
//! the `/pair` flow, `/applist`, and `/launch`/`/resume`/`/cancel`, plus a punktfunk-only
//! `/pin` endpoint to deliver the Moonlight-displayed PIN. Over HTTPS the client is
//! the `/pair` flow, `/applist`, and `/launch`/`/resume`/`/cancel`. Over HTTPS the client is
//! mutual-TLS-authenticated, so `/serverinfo` reports `PairStatus=1` there.
//!
//! The pairing PIN is delivered out-of-band ONLY through the bearer-authenticated management
//! API (`POST /api/v1/pair/pin`): the operator reads the PIN off the Moonlight client and
//! types it into the host console. There is deliberately NO unauthenticated nvhttp PIN
//! endpoint — one would let a network client submit its own displayed PIN and drive the whole
//! ceremony to a pinned cert with no operator consent (security-review 2026-06-28 #1).
use super::tls::PeerCertFingerprint;
use super::tls::{PeerAddr, PeerCertFingerprint};
use super::{serverinfo, AppState, LaunchSession, HTTPS_PORT, HTTP_PORT, RTSP_PORT};
use anyhow::{anyhow, Context, Result};
use axum::{
@@ -58,7 +63,6 @@ fn router(state: Arc<AppState>, https: bool) -> Router {
Router::new()
.route("/serverinfo", get(h_serverinfo))
.route("/pair", get(h_pair))
.route("/pin", get(h_pin))
.route("/applist", get(h_applist))
.route("/launch", get(h_launch))
.route("/resume", get(h_resume))
@@ -82,19 +86,6 @@ async fn h_serverinfo(
xml(serverinfo::serverinfo_xml(&st.host, https, paired))
}
async fn h_pin(
State(st): State<Arc<AppState>>,
Query(q): Query<HashMap<String, String>>,
) -> impl IntoResponse {
match q.get("pin").filter(|p| !p.is_empty()) {
Some(pin) => {
st.pairing.pin.submit(pin.clone());
"PIN accepted\n".to_string()
}
None => "usage: GET /pin?pin=NNNN\n".to_string(),
}
}
async fn h_applist(
State(st): State<Arc<AppState>>,
peer: Option<Extension<PeerCertFingerprint>>,
@@ -110,6 +101,7 @@ async fn h_applist(
async fn h_launch(
State(st): State<Arc<AppState>>,
peer: Option<Extension<PeerCertFingerprint>>,
addr: Option<Extension<PeerAddr>>,
Query(q): Query<HashMap<String, String>>,
) -> impl IntoResponse {
if !peer_is_paired(&peer, &st) {
@@ -117,7 +109,9 @@ async fn h_launch(
return xml(error_xml());
}
match launch(&st, &q) {
Ok(session) => {
Ok(mut session) => {
// Bind the (unauthenticated) RTSP/UDP media plane to this paired client's source IP.
session.peer_ip = addr.map(|Extension(PeerAddr(a))| a.ip());
*st.launch.lock().unwrap() = Some(session);
tracing::info!(
w = session.width,
@@ -193,6 +187,7 @@ fn launch(_st: &AppState, q: &HashMap<String, String>) -> Result<LaunchSession>
height,
fps,
appid,
peer_ip: None, // set by `h_launch` from the verified HTTPS peer address
})
}
@@ -17,9 +17,14 @@ use std::sync::Mutex;
use std::time::Duration;
use tokio::sync::Notify;
/// Out-of-band PIN delivery. Moonlight generates + displays a PIN; the user submits it
/// (via the management API's `POST /api/v1/pair/pin` or nvhttp's `GET /pin?pin=NNNN`).
/// `getservercert` parks until a PIN arrives.
/// Out-of-band PIN delivery. Moonlight generates + displays a PIN; the operator submits it
/// via the bearer-authenticated management API (`POST /api/v1/pair/pin`) only — there is no
/// unauthenticated nvhttp delivery path (a network client must never be able to submit its
/// own PIN; security-review 2026-06-28 #1). `getservercert` parks until a PIN arrives.
/// Max pairing handshakes parked in [`PinGate::take`] at once (each holds a slot for up to
/// 300s), bounding a pre-auth waiter flood. Real pairing is one operator-driven client at a time.
const MAX_PARKED_WAITERS: usize = 4;
pub struct PinGate {
pin: Mutex<Option<String>>,
notify: Notify,
@@ -48,7 +53,20 @@ impl PinGate {
}
async fn take(&self, timeout: Duration) -> Option<String> {
self.waiters.fetch_add(1, Ordering::SeqCst);
// Bound the number of pairing handshakes parked at once: each `getservercert` is
// pre-auth and parks for up to 300s, so without a cap an unpaired LAN peer could pin
// unbounded tasks + keep `awaiting_pin` asserted (security-review 2026-06-28 #12).
// Reserve a slot atomically; refuse (treated as "no PIN") once the cap is reached.
if self
.waiters
.fetch_update(Ordering::SeqCst, Ordering::SeqCst, |n| {
(n < MAX_PARKED_WAITERS).then_some(n + 1)
})
.is_err()
{
tracing::warn!("pairing: too many handshakes awaiting a PIN — refusing");
return None;
}
// Decrement on every exit path (PIN delivered, timeout, or future cancellation).
struct WaiterGuard<'a>(&'a AtomicUsize);
impl Drop for WaiterGuard<'_> {
@@ -83,6 +101,10 @@ struct Session {
server_challenge: [u8; 16],
/// The client's phase-3 hash, recomputed + checked in phase 4.
client_hash: Vec<u8>,
/// Set once phase 3 has produced the RSA-signed serversecret. A repeated phase 3 is refused so a
/// peer past phase 1 can't loop phase2/phase3 to harvest many signing-time samples (a passive
/// timing-oracle amplifier vs. the rsa-crate Marvin side-channel; see `.cargo/audit.toml`).
responded: bool,
}
pub struct Pairing {
@@ -117,7 +139,8 @@ impl Pairing {
tracing::info!(
uniqueid,
"pairing phase 1 (getservercert) — awaiting PIN: submit `GET /pin?pin=NNNN`"
"pairing phase 1 (getservercert) — awaiting PIN: deliver it via the management \
API `POST /api/v1/pair/pin` (operator reads the PIN off the Moonlight client)"
);
let pin = self
.pin
@@ -136,6 +159,7 @@ impl Pairing {
serversecret: [0; 16],
server_challenge: [0; 16],
client_hash: Vec::new(),
responded: false,
},
);
tracing::info!(
@@ -197,6 +221,14 @@ impl Pairing {
bail!("short challenge response");
}
s.client_hash = client_hash[..32].to_vec();
// Sign the serversecret exactly ONCE per ceremony: refuse a repeated phase 3 so a peer that
// cleared phase 1 (operator PIN) can't replay it to collect many RSA signing-time samples
// (timing-oracle amplifier vs. RUSTSEC-2023-0071; see `.cargo/audit.toml`). A legit client
// signs once. The session stays for phase 4 (the cert-pin step) but won't re-sign.
if s.responded {
bail!("serverchallengeresp already answered for this pairing session");
}
s.responded = true;
let sig: Signature = id.signing_key.sign(&s.serversecret);
let mut secret = Vec::with_capacity(16 + 256);
secret.extend_from_slice(&s.serversecret);
@@ -304,4 +336,28 @@ mod tests {
assert_eq!(pairing.pin.take(Duration::from_millis(10)).await, None);
assert!(!pairing.pin.awaiting_pin());
}
/// A pre-auth peer flood can park at most `MAX_PARKED_WAITERS` pairing handshakes; the next
/// `take` is refused immediately (returns `None` without parking), bounding the 300s-waiter DoS
/// (security-review 2026-06-28 #12).
#[tokio::test]
async fn pin_gate_caps_parked_waiters() {
let pairing = Arc::new(Pairing::new());
let mut handles = Vec::new();
for _ in 0..MAX_PARKED_WAITERS {
let p = pairing.clone();
handles.push(tokio::spawn(async move {
p.pin.take(Duration::from_secs(5)).await
}));
}
// Wait until all the slots are taken.
while pairing.pin.waiters.load(Ordering::SeqCst) < MAX_PARKED_WAITERS {
tokio::time::sleep(Duration::from_millis(2)).await;
}
// One more is refused right away (no parking), even with a long timeout.
assert_eq!(pairing.pin.take(Duration::from_secs(5)).await, None);
for h in handles {
h.abort();
}
}
}
+23 -13
View File
@@ -14,7 +14,7 @@ use crate::encode::Codec;
use anyhow::{Context, Result};
use std::collections::HashMap;
use std::io::{Read, Write};
use std::net::{TcpListener, TcpStream};
use std::net::{SocketAddr, TcpListener, TcpStream};
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
use std::time::Duration;
@@ -102,13 +102,12 @@ fn handle_conn(mut stream: TcpStream, state: Arc<AppState>) -> Result<()> {
"RTSP {} | {}", req.head.replace("\r\n", " | "),
if req.body.is_empty() { String::new() } else { format!("body: {}", req.body.replace("\r\n", " | ")) }
);
let resp = handle_request(&req, &state);
let resp = handle_request(&req, &state, peer);
stream.write_all(resp.as_bytes()).context("RTSP write")?;
stream.flush().ok();
// Close (FIN after the flushed response) so the client detects end-of-response.
let _ = stream.shutdown(std::net::Shutdown::Both);
}
let _ = peer;
Ok(())
}
@@ -171,7 +170,7 @@ fn parse_request(head: &str, body: String) -> Request {
}
}
fn handle_request(req: &Request, state: &AppState) -> String {
fn handle_request(req: &Request, state: &AppState, peer: Option<SocketAddr>) -> String {
match req.method.as_str() {
"OPTIONS" => response(
&req.cseq,
@@ -216,16 +215,30 @@ fn handle_request(req: &Request, state: &AppState) -> String {
response(&req.cseq, &[], None)
}
"PLAY" => {
// The RTSP/UDP media plane is UNAUTHENTICATED. A stream may start only for the paired
// client that completed the pairing-gated `/launch` (which set `state.launch`), and —
// when the launching IP is known — only from that same source IP. So an unpaired RTSP
// peer can neither start a stream on an idle host nor ride a paired client's active
// launch (security-review 2026-06-28 #4). `nvhttp` gates `/launch` on a pinned cert.
let launch = *state.launch.lock().unwrap();
let Some(ls) = launch else {
tracing::warn!(?peer, "RTSP PLAY — refused: no paired `/launch` session");
return response_status("401 Unauthorized", &req.cseq, &[], None);
};
if let (Some(want), Some(got)) = (ls.peer_ip, peer.map(|p| p.ip())) {
if want != got {
tracing::warn!(
%want, %got,
"RTSP PLAY — refused: peer IP does not match the launching client"
);
return response_status("401 Unauthorized", &req.cseq, &[], None);
}
}
let cfg = *state.stream.lock().unwrap();
match cfg {
Some(cfg) if !state.streaming.swap(true, Ordering::SeqCst) => {
// Resolve the launched catalog entry (session recipe) for the stream.
let app = state
.launch
.lock()
.unwrap()
.map(|l| l.appid)
.and_then(super::apps::by_id);
let app = super::apps::by_id(ls.appid);
tracing::info!(app = ?app.as_ref().map(|a| &a.title), "RTSP PLAY — starting video stream");
stream::start(
cfg,
@@ -243,8 +256,6 @@ fn handle_request(req: &Request, state: &AppState) -> String {
// Audio runs independently (Opus on UDP 48000, stereo or 5.1/7.1 multistream per
// the ANNOUNCE); it needs the launch key for the AES-CBC payload encryption the
// client expects.
let launch = *state.launch.lock().unwrap();
if let Some(ls) = launch {
if !state.audio_streaming.swap(true, Ordering::SeqCst) {
tracing::info!("RTSP PLAY — starting audio stream");
audio::start(
@@ -255,7 +266,6 @@ fn handle_request(req: &Request, state: &AppState) -> String {
state.audio_cap.clone(),
);
}
}
response(&req.cseq, &[("Session", "DEADBEEFCAFE;timeout = 90")], None)
}
"TEARDOWN" => {
+19 -10
View File
@@ -7,11 +7,12 @@
//! fingerprint ([`PeerCertFingerprint`]) to each request, and the nvhttp/mgmt handlers reject
//! callers whose fingerprint is not pinned (mirroring Apollo's post-handshake `get_verified_cert`).
use anyhow::{anyhow, Context, Result};
use anyhow::{Context, Result};
use axum::Router;
use rustls::client::danger::HandshakeSignatureValid;
use rustls::crypto::{verify_tls12_signature, verify_tls13_signature, CryptoProvider};
use rustls::pki_types::{CertificateDer, UnixTime};
use rustls::pki_types::pem::PemObject;
use rustls::pki_types::{CertificateDer, PrivateKeyDer, UnixTime};
use rustls::server::danger::{ClientCertVerified, ClientCertVerifier};
use rustls::{DigitallySignedStruct, DistinguishedName, ServerConfig, SignatureScheme};
use std::net::SocketAddr;
@@ -24,6 +25,12 @@ use std::sync::Arc;
#[derive(Clone)]
pub(crate) struct PeerCertFingerprint(pub Option<String>);
/// The TCP source address of an HTTPS request, injected per-connection by [`serve_https`]. Used by
/// `/launch` to record which paired client owns the session so the unauthenticated RTSP/UDP media
/// plane can bind to that peer's IP (security-review 2026-06-28 #4).
#[derive(Clone, Copy)]
pub(crate) struct PeerAddr(pub SocketAddr);
/// HTTPS server that surfaces the verified client cert to handlers. `axum_server` can't expose the
/// peer cert, so this runs the rustls handshake itself (tokio-rustls), reads the peer certificate,
/// and serves the axum `Router` over hyper with the peer's fingerprint attached to every request as
@@ -39,7 +46,7 @@ pub(crate) async fn serve_https(
.await
.with_context(|| format!("bind HTTPS {bind}"))?;
loop {
let (tcp, _peer) = match listener.accept().await {
let (tcp, peer) = match listener.accept().await {
Ok(v) => v,
Err(e) => {
tracing::warn!(error = %e, "HTTPS accept failed");
@@ -63,14 +70,16 @@ pub(crate) async fn serve_https(
.peer_certificates()
.and_then(|c| c.first())
.map(|c| hex::encode(punktfunk_core::quic::endpoint::cert_fingerprint(c.as_ref())));
let peer = PeerCertFingerprint(fp);
let fp = PeerCertFingerprint(fp);
let addr = PeerAddr(peer);
let svc =
hyper::service::service_fn(move |req: hyper::Request<hyper::body::Incoming>| {
let app = app.clone();
let peer = peer.clone();
let fp = fp.clone();
async move {
let mut req = req.map(axum::body::Body::new);
req.extensions_mut().insert(peer);
req.extensions_mut().insert(fp);
req.extensions_mut().insert(addr);
app.oneshot(req).await // Router error is Infallible
}
});
@@ -169,12 +178,12 @@ fn build_server_config(
mandatory: bool,
) -> Result<Arc<ServerConfig>> {
let provider = Arc::new(rustls::crypto::aws_lc_rs::default_provider());
let certs = rustls_pemfile::certs(&mut cert_pem.as_bytes())
// PEM parsing via rustls-pki-types (the same `PemObject` path punktfunk-core/quic.rs uses),
// so we don't pull the unmaintained `rustls-pemfile`.
let certs = CertificateDer::pem_slice_iter(cert_pem.as_bytes())
.collect::<std::result::Result<Vec<_>, _>>()
.context("parse host cert PEM")?;
let key = rustls_pemfile::private_key(&mut key_pem.as_bytes())
.context("parse host key PEM")?
.ok_or_else(|| anyhow!("no private key in host key PEM"))?;
let key = PrivateKeyDer::from_pem_slice(key_pem.as_bytes()).context("parse host key PEM")?;
let verifier = Arc::new(AcceptAnyClientCert {
provider: provider.clone(),
+26 -3
View File
@@ -76,9 +76,7 @@ pub fn open(backend: Backend) -> Result<Box<dyn InputInjector>> {
#[cfg(target_os = "linux")]
{
Ok(Box::new(libei::LibeiInjector::open_with(
libei::EiSource::SocketPathFile(
crate::vdisplay::gamescope_ei_socket_file().into(),
),
libei::EiSource::SocketPathFile(crate::vdisplay::gamescope_ei_socket_file()),
)?))
}
#[cfg(not(target_os = "linux"))]
@@ -493,6 +491,31 @@ pub mod gamepad;
#[cfg(target_os = "windows")]
#[path = "inject/windows/gamepad_raii.rs"]
mod gamepad_raii;
/// Linux: virtual Steam Deck via UHID — the kernel `hid-steam` driver binds it as a real Deck.
#[cfg(target_os = "linux")]
#[path = "inject/linux/steam_controller.rs"]
pub mod steam_controller;
/// Linux: virtual Steam Deck via the USB gadget subsystem (`raw_gadget` + `dummy_hcd`) — the only
/// virtual-Deck transport Steam Input promotes (presents the controller on USB interface 2).
/// SteamOS-host only (needs `dummy_hcd` + `raw_gadget`).
#[cfg(target_os = "linux")]
#[path = "inject/linux/steam_gadget.rs"]
pub mod steam_gadget;
/// Transport-independent Steam Controller / Steam Deck HID contract (descriptor, byte-exact Deck
/// serializer, XInput/rich mappers, rumble parser), used by the Linux UHID backend ([`steam_controller`]).
#[cfg(target_os = "linux")]
#[path = "inject/proto/steam_proto.rs"]
pub mod steam_proto;
/// Pure fallback-remap policy (Steam-only inputs onto a non-Steam backend) + the Deck motion rescale.
#[cfg(target_os = "linux")]
#[path = "inject/proto/steam_remap.rs"]
pub mod steam_remap;
/// Linux: virtual Steam Deck over **USB/IP** (`vhci_hcd`) — the shippable, Secure-Boot-clean,
/// Steam-Input-promotable virtual-Deck transport on non-SteamOS hosts (Bazzite/generic), where
/// `dummy_hcd`/`raw_gadget` aren't built. In-tree + signed; no module build, no MOK.
#[cfg(target_os = "linux")]
#[path = "inject/linux/steam_usbip.rs"]
pub mod steam_usbip;
/// Stub — virtual gamepads need Linux uinput or the Windows UMDF drivers; events are dropped elsewhere.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub mod gamepad {
@@ -182,6 +182,9 @@ pub struct DualSenseManager {
last_write: Vec<Instant>,
/// Pad creation failed (e.g. /dev/uhid permissions) — warn once, drop events.
broken: bool,
/// Fallback policy for the Steam back grips a client may send (the DualSense has no back-button
/// HID slot). `PUNKTFUNK_STEAM_REMAP=paddles=…`; default drop.
remap: crate::inject::steam_remap::RemapConfig,
}
impl Default for DualSenseManager {
@@ -198,6 +201,7 @@ impl DualSenseManager {
last_rumble: vec![(0, 0); MAX_PADS],
last_write: vec![Instant::now(); MAX_PADS],
broken: false,
remap: crate::inject::steam_remap::RemapConfig::from_env(),
}
}
@@ -229,8 +233,12 @@ impl DualSenseManager {
// Merge buttons/sticks/triggers from the frame, preserving touch + motion (those
// come on the rich-input plane and must survive a button-only frame).
let prev = self.state[idx];
// Steam back grips have no DualSense slot — fold them onto standard buttons per the
// configured policy (default drop) so they aren't silently lost.
let buttons =
crate::inject::steam_remap::fold_paddles(f.buttons, self.remap.paddles);
let mut s = DsState::from_gamepad(
f.buttons,
buttons,
f.ls_x,
f.ls_y,
f.rs_x,
@@ -252,7 +260,9 @@ impl DualSenseManager {
/// arrived first); they're dropped if the pad isn't present.
pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich {
RichInput::Touchpad { pad, .. } | RichInput::Motion { pad, .. } => pad as usize,
RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
};
if idx >= MAX_PADS || self.pads[idx].is_none() {
return;
@@ -280,6 +290,26 @@ impl DualSenseManager {
self.state[idx].gyro = gyro;
self.state[idx].accel = accel;
}
RichInput::TouchpadEx {
surface,
finger,
touch,
x,
y,
..
} => {
// A Steam right/single pad maps onto the one DualSense touchpad (signed centre-0 →
// 0..=65535); surface 1 (the Steam left pad) has no DualSense equivalent.
if surface != 1 {
let slot = (finger as usize).min(1);
let n = |v: i16| ((v as i32) + 32768) as u32;
let t = &mut self.state[idx].touch[slot];
t.active = touch;
t.id = slot as u8;
t.x = (n(x) * (DS_TOUCH_W - 1) as u32 / u16::MAX as u32) as u16;
t.y = (n(y) * (DS_TOUCH_H - 1) as u32 / u16::MAX as u32) as u16;
}
}
}
self.write(idx);
}
@@ -367,6 +367,9 @@ pub struct DualShock4Manager {
last_write: Vec<Instant>,
/// Pad creation failed (e.g. /dev/uhid permissions) — warn once, drop events.
broken: bool,
/// Fallback policy for the Steam back grips a client may send (the DS4 has no back-button HID
/// slot). `PUNKTFUNK_STEAM_REMAP=paddles=…`; default drop.
remap: crate::inject::steam_remap::RemapConfig,
}
impl Default for DualShock4Manager {
@@ -384,6 +387,7 @@ impl DualShock4Manager {
last_led: vec![None; MAX_PADS],
last_write: vec![Instant::now(); MAX_PADS],
broken: false,
remap: crate::inject::steam_remap::RemapConfig::from_env(),
}
}
@@ -416,8 +420,12 @@ impl DualShock4Manager {
// Merge buttons/sticks/triggers, preserving touch + motion (those arrive on the
// rich-input plane and must survive a button-only frame).
let prev = self.state[idx];
// Steam back grips have no DS4 slot — fold them onto standard buttons per the
// configured policy (default drop) so they aren't silently lost.
let buttons =
crate::inject::steam_remap::fold_paddles(f.buttons, self.remap.paddles);
let mut s = DsState::from_gamepad(
f.buttons,
buttons,
f.ls_x,
f.ls_y,
f.rs_x,
@@ -439,7 +447,9 @@ impl DualShock4Manager {
/// pad isn't present.
pub fn apply_rich(&mut self, rich: RichInput) {
let idx = match rich {
RichInput::Touchpad { pad, .. } | RichInput::Motion { pad, .. } => pad as usize,
RichInput::Touchpad { pad, .. }
| RichInput::Motion { pad, .. }
| RichInput::TouchpadEx { pad, .. } => pad as usize,
};
if idx >= MAX_PADS || self.pads[idx].is_none() {
return;
@@ -466,6 +476,26 @@ impl DualShock4Manager {
self.state[idx].gyro = gyro;
self.state[idx].accel = accel;
}
RichInput::TouchpadEx {
surface,
finger,
touch,
x,
y,
..
} => {
// A Steam right/single pad maps onto the one DS4 touchpad (signed centre-0 →
// 0..=65535); surface 1 (the Steam left pad) has no DS4 equivalent.
if surface != 1 {
let slot = (finger as usize).min(1);
let n = |v: i16| ((v as i32) + 32768) as u32;
let t = &mut self.state[idx].touch[slot];
t.active = touch;
t.id = slot as u8;
t.x = (n(x) * (DS4_TOUCH_W - 1) as u32 / u16::MAX as u32) as u16;
t.y = (n(y) * (DS4_TOUCH_H - 1) as u32 / u16::MAX as u32) as u16;
}
}
}
self.write(idx);
}
@@ -69,9 +69,16 @@ const BTN_START: u16 = 0x13b;
const BTN_MODE: u16 = 0x13c;
const BTN_THUMBL: u16 = 0x13d;
const BTN_THUMBR: u16 = 0x13e;
// Xbox-Elite paddle codes (the xpad convention SDL / Steam Input recognize). A client's back grips —
// and the GameStream `buttonFlags2` paddle bits, which were silently dropped before — land here, so
// the virtual X-Box pad exposes paddles like an Elite controller. PADDLE1/2/3/4 = R4/L4/R5/L5.
const BTN_TRIGGER_HAPPY5: u16 = 0x2c4;
const BTN_TRIGGER_HAPPY6: u16 = 0x2c5;
const BTN_TRIGGER_HAPPY7: u16 = 0x2c6;
const BTN_TRIGGER_HAPPY8: u16 = 0x2c7;
/// `(GameStream button bit, evdev key code)` — D-pad is emitted as HAT axes instead.
const BUTTON_MAP: [(u32, u16); 11] = [
const BUTTON_MAP: [(u32, u16); 15] = [
(gamepad::BTN_A, BTN_SOUTH),
(gamepad::BTN_B, BTN_EAST),
(gamepad::BTN_X, BTN_NORTH),
@@ -83,6 +90,10 @@ const BUTTON_MAP: [(u32, u16); 11] = [
(gamepad::BTN_GUIDE, BTN_MODE),
(gamepad::BTN_LS_CLK, BTN_THUMBL),
(gamepad::BTN_RS_CLK, BTN_THUMBR),
(gamepad::BTN_PADDLE1, BTN_TRIGGER_HAPPY5),
(gamepad::BTN_PADDLE2, BTN_TRIGGER_HAPPY6),
(gamepad::BTN_PADDLE3, BTN_TRIGGER_HAPPY7),
(gamepad::BTN_PADDLE4, BTN_TRIGGER_HAPPY8),
];
/// The USB identity a virtual uinput pad presents. SDL/Steam/Proton key their built-in mapping off
@@ -7,9 +7,14 @@
//! which the libei/portal path cannot. We connect as an ordinary Wayland client on the KWin session's
//! `$WAYLAND_DISPLAY` and translate events into fake-input requests; keyboard keys are raw Linux
//! evdev codes that KWin resolves through the session's own keymap (no keymap upload, unlike the wlr
//! virtual-keyboard path), and absolute pointer/touch coordinates are global compositor space — which
//! on a headless box (single per-session virtual output at the origin, scale 1) equals the streamed
//! output's pixels.
//! virtual-keyboard path), and absolute pointer/touch coordinates are global compositor space.
//!
//! Global compositor space is *logical* pixels (post display-scaling), which only equals the streamed
//! output's physical pixels at scale 1. Under a fractional/integer scale the logical edge sits at
//! `physical / scale`, so feeding the raw streamed pixel coordinate lands the cursor `scale×` too far
//! toward the bottom-right (top-left stays put). We therefore track each output's logical geometry
//! (position + size) via `xdg-output` and map the normalized client position into the matching
//! output's logical rectangle — the same shape the libei backend uses with its EI region.
#![allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
@@ -18,8 +23,14 @@
use super::{gs_button_to_evdev, vk_to_evdev, InputEvent, InputInjector};
use anyhow::{Context, Result};
use punktfunk_core::input::InputKind;
use std::time::{Duration, Instant};
use wayland_client::protocol::wl_output::{self, WlOutput};
use wayland_client::protocol::wl_registry::{self, WlRegistry};
use wayland_client::{Connection, Dispatch, EventQueue, Proxy, QueueHandle};
use wayland_client::{Connection, Dispatch, EventQueue, Proxy, QueueHandle, WEnum};
use wayland_protocols::xdg::xdg_output::zv1::client::{
zxdg_output_manager_v1::ZxdgOutputManagerV1,
zxdg_output_v1::{self, ZxdgOutputV1},
};
// Generate the client bindings for the vendored protocol XML inline (no build.rs), exactly like the
// KWin virtual-output backend. Path is relative to CARGO_MANIFEST_DIR.
@@ -48,10 +59,39 @@ const AXIS_HORIZONTAL: u32 = 1;
/// `code` value marking a horizontal scroll event (mirrors `gamestream::input` / the wlr backend).
const SCROLL_HORIZONTAL: u32 = 1;
/// One tracked output: its physical mode (to match the streamed resolution) and its logical geometry
/// (the global-compositor-space rectangle absolute coordinates are addressed in). `logical_w == 0`
/// means xdg-output hasn't reported its size yet.
struct OutputTrack {
/// Registry global id — also the dispatch user-data, so events route back to this entry.
name: u32,
wl_output: WlOutput,
xdg_output: Option<ZxdgOutputV1>,
/// Physical pixel mode from `wl_output.mode` (the `current` mode); matched against the streamed WxH.
mode_w: i32,
mode_h: i32,
/// Logical (post-scale) geometry from `xdg-output`.
logical_x: i32,
logical_y: i32,
logical_w: i32,
logical_h: i32,
}
/// Registry-bound globals (the Wayland dispatch state).
#[derive(Default)]
struct State {
fake: Option<FakeInput>,
xdg_mgr: Option<ZxdgOutputManagerV1>,
outputs: Vec<OutputTrack>,
}
impl State {
/// Create the `xdg_output` for a tracked output once both it and the manager exist.
fn ensure_xdg_output(o: &mut OutputTrack, mgr: &ZxdgOutputManagerV1, qh: &QueueHandle<State>) {
if o.xdg_output.is_none() {
o.xdg_output = Some(mgr.get_xdg_output(&o.wl_output, qh, o.name));
}
}
}
impl Dispatch<WlRegistry, ()> for State {
@@ -63,15 +103,57 @@ impl Dispatch<WlRegistry, ()> for State {
_: &Connection,
qh: &QueueHandle<Self>,
) {
if let wl_registry::Event::Global {
match event {
wl_registry::Event::Global {
name,
interface,
version,
} = event
{
if interface == "org_kde_kwin_fake_input" {
} => match interface.as_str() {
"org_kde_kwin_fake_input" => {
state.fake = Some(registry.bind(name, version.min(MAX_VERSION), qh, ()));
}
"wl_output" => {
// v1 carries `mode` (all we need); bind no higher than the proxy's max (4).
let wl_output: WlOutput = registry.bind(name, version.min(4), qh, name);
let mut o = OutputTrack {
name,
wl_output,
xdg_output: None,
mode_w: 0,
mode_h: 0,
logical_x: 0,
logical_y: 0,
logical_w: 0,
logical_h: 0,
};
if let Some(mgr) = state.xdg_mgr.clone() {
State::ensure_xdg_output(&mut o, &mgr, qh);
}
state.outputs.push(o);
}
"zxdg_output_manager_v1" => {
let mgr: ZxdgOutputManagerV1 = registry.bind(name, version.min(3), qh, ());
// Outputs bound before the manager have no xdg_output yet — create them now.
for o in state.outputs.iter_mut() {
State::ensure_xdg_output(o, &mgr, qh);
}
state.xdg_mgr = Some(mgr);
}
_ => {}
},
wl_registry::Event::GlobalRemove { name } => {
state.outputs.retain(|o| {
if o.name == name {
if let Some(x) = &o.xdg_output {
x.destroy();
}
false
} else {
true
}
});
}
_ => {}
}
}
}
@@ -89,13 +171,86 @@ impl Dispatch<FakeInput, ()> for State {
}
}
impl Dispatch<WlOutput, u32> for State {
fn event(
state: &mut Self,
_: &WlOutput,
event: wl_output::Event,
name: &u32,
_: &Connection,
_: &QueueHandle<Self>,
) {
// Only the *current* mode matters — a real monitor also advertises its other supported modes.
if let wl_output::Event::Mode {
flags: WEnum::Value(flags),
width,
height,
..
} = event
{
if flags.contains(wl_output::Mode::Current) {
if let Some(o) = state.outputs.iter_mut().find(|o| o.name == *name) {
o.mode_w = width;
o.mode_h = height;
}
}
}
}
}
impl Dispatch<ZxdgOutputV1, u32> for State {
fn event(
state: &mut Self,
_: &ZxdgOutputV1,
event: zxdg_output_v1::Event,
name: &u32,
_: &Connection,
_: &QueueHandle<Self>,
) {
if let Some(o) = state.outputs.iter_mut().find(|o| o.name == *name) {
match event {
zxdg_output_v1::Event::LogicalPosition { x, y } => {
o.logical_x = x;
o.logical_y = y;
}
zxdg_output_v1::Event::LogicalSize { width, height } => {
o.logical_w = width;
o.logical_h = height;
}
_ => {}
}
}
}
}
// The manager has no events.
impl Dispatch<ZxdgOutputManagerV1, ()> for State {
fn event(
_: &mut Self,
_: &ZxdgOutputManagerV1,
_: <ZxdgOutputManagerV1 as Proxy>::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
pub struct KwinFakeInjector {
conn: Connection,
queue: EventQueue<State>,
state: State,
fake: FakeInput,
/// When output geometry was last re-read; throttles the per-event roundtrip (see `refresh_geometry`).
last_refresh: Option<Instant>,
}
/// How often the fake_input backend re-reads output geometry from the compositor. Output add/remove
/// (a new session's virtual output) and live scale/resolution changes are infrequent, so a lazy
/// poll on the injector's own thread is plenty and adds at most one local-socket roundtrip twice a
/// second — versus a blocking roundtrip on every single mouse-move event.
const GEO_REFRESH: Duration = Duration::from_millis(500);
impl KwinFakeInjector {
pub fn open() -> Result<Self> {
let conn = Connection::connect_to_env()
@@ -122,13 +277,77 @@ impl KwinFakeInjector {
.context("fake_input authenticate roundtrip")?;
conn.flush().ok();
tracing::info!("KWin fake_input ready (headless keyboard/mouse/touch — no portal)");
Ok(Self {
// Settle output geometry (wl_output + xdg-output were bound during the registry roundtrip
// above; their logical_size arrives on a follow-up roundtrip). Best-effort — falls back to
// scale-1 mapping if xdg-output is absent.
let mut injector = Self {
conn,
queue,
state,
fake,
})
last_refresh: None,
};
injector.refresh_geometry();
tracing::info!(
outputs = injector.state.outputs.len(),
"KWin fake_input ready (headless keyboard/mouse/touch — no portal)"
);
Ok(injector)
}
/// Re-read output geometry, throttled to [`GEO_REFRESH`]. A `roundtrip` both flushes any pending
/// `get_xdg_output` requests and reads the geometry events back. A wl_output that *appeared* this
/// round only gets its xdg_output created mid-dispatch, so its `logical_size` lands on a later
/// roundtrip — keep going (bounded) until every output is settled.
fn refresh_geometry(&mut self) {
let now = Instant::now();
if let Some(t) = self.last_refresh {
if now.duration_since(t) < GEO_REFRESH {
return;
}
}
self.last_refresh = Some(now);
for _ in 0..3 {
if self.queue.roundtrip(&mut self.state).is_err() {
return;
}
let pending =
self.state.xdg_mgr.is_some() && self.state.outputs.iter().any(|o| o.logical_w == 0);
if !pending {
break;
}
}
}
/// Resolve the logical (global-compositor-space) rectangle to map a normalized client position
/// into. Prefer the output whose physical mode matches the streamed `phys_w`×`phys_h` (the
/// per-session virtual output); fall back to the sole output, then — if xdg-output is unavailable
/// — to the streamed pixels at the origin (the pre-scaling behavior, correct at scale 1).
fn logical_target(&self, phys_w: i32, phys_h: i32) -> (f64, f64, f64, f64) {
let usable = || {
self.state
.outputs
.iter()
.filter(|o| o.logical_w > 0 && o.logical_h > 0)
};
let chosen = usable()
.find(|o| o.mode_w == phys_w && o.mode_h == phys_h)
.or_else(|| {
let mut it = usable();
match (it.next(), it.next()) {
(Some(only), None) => Some(only),
_ => None,
}
});
match chosen {
Some(o) => (
o.logical_x as f64,
o.logical_y as f64,
o.logical_w as f64,
o.logical_h as f64,
),
None => (0.0, 0.0, phys_w as f64, phys_h as f64),
}
}
}
@@ -139,12 +358,17 @@ impl InputInjector for KwinFakeInjector {
self.fake.pointer_motion(event.x as f64, event.y as f64);
}
InputKind::MouseMoveAbs => {
let w = (event.flags >> 16) & 0xffff;
let h = event.flags & 0xffff;
let w = ((event.flags >> 16) & 0xffff) as i32;
let h = (event.flags & 0xffff) as i32;
if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64;
let y = event.y.clamp(0, h as i32) as f64;
self.fake.pointer_motion_absolute(x, y);
self.refresh_geometry();
let (lx, ly, lw, lh) = self.logical_target(w, h);
// Normalize in the streamed (physical) pixel space, then place inside the output's
// logical rectangle — so display scaling no longer offsets the cursor.
let nx = (event.x as f64 / w as f64).clamp(0.0, 1.0);
let ny = (event.y as f64 / h as f64).clamp(0.0, 1.0);
self.fake
.pointer_motion_absolute(lx + nx * lw, ly + ny * lh);
}
}
InputKind::MouseButtonDown | InputKind::MouseButtonUp => {
@@ -179,11 +403,15 @@ impl InputInjector for KwinFakeInjector {
// Touch: id = event.code, coords in the client surface w×h packed into flags (same
// absolute mapping as MouseMoveAbs). Each event is its own frame.
InputKind::TouchDown | InputKind::TouchMove => {
let w = (event.flags >> 16) & 0xffff;
let h = event.flags & 0xffff;
let w = ((event.flags >> 16) & 0xffff) as i32;
let h = (event.flags & 0xffff) as i32;
if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64;
let y = event.y.clamp(0, h as i32) as f64;
self.refresh_geometry();
let (lx, ly, lw, lh) = self.logical_target(w, h);
let nx = (event.x as f64 / w as f64).clamp(0.0, 1.0);
let ny = (event.y as f64 / h as f64).clamp(0.0, 1.0);
let x = lx + nx * lw;
let y = ly + ny * lh;
if event.kind == InputKind::TouchDown {
self.fake.touch_down(event.code, x, y);
} else {

Some files were not shown because too many files have changed in this diff Show More