enricobuehler
e19f11bb0d
windows-drivers / probe-and-proto (pull_request) Successful in 25s
apple / swift (pull_request) Successful in 1m37s
apple / screenshots (pull_request) Skipped
windows-drivers / driver-build (pull_request) Failing after 1m43s
ci / web (pull_request) Successful in 2m57s
ci / bun-nix (pull_request) Successful in 18s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m36s
android / android (pull_request) Successful in 4m0s
ci / rust-arm64 (pull_request) Successful in 4m19s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 2m32s
ci / docs-site (pull_request) Successful in 4m36s
ci / rust (pull_request) Failing after 9m15s
nix / flake (pull_request) Successful in 15m3s
The `0xCA` wire already carries the two Xbox impulse-trigger motors (v3), and the Rust decode path
already parses them; `datagram_task.rs` dropped them on the floor with a comment naming exactly this
work as what remained. The blocker was the C ABI: every non-Rust client pulls rumble through
`punktfunk_connection_next_rumble_cmd`, whose out-params cannot carry two more channels.
PunktfunkStatus punktfunk_connection_next_rumble_cmd2(
PunktfunkConnection *c, uint16_t *pad, uint16_t *low, uint16_t *high,
uint16_t *left_trigger, uint16_t *right_trigger,
uint32_t *backstop_ms, uint32_t timeout_ms);
⚠️ ADDED, not widened. `_cmd` keeps its signature and its values bit-identical for handle-only
traffic — out-of-tree embedders depend on it and `docs/embedding-the-c-abi.md` documents it, so
silently changing an exported symbol would break every consumer at once. `nm` on the staticlib shows
all four rumble entry points still exported. `ABI_VERSION` 17 → 18; every other site reads it
dynamically, so there are no hardcoded mirrors to drift.
⚠️ ONE HONEST BEHAVIOURAL DELTA, documented in `abi.rs` and pinned by a test: against a
trigger-driving host a `_cmd` caller now receives commands with `low == high == 0` where the demux
previously dropped the update entirely. They are idempotent handle stops, and the redundant-stop
suppression cannot fold them because the command as a whole is not silent. Zero cost today —
nothing sources non-zero triggers.
The dedupe-jitter proof was RE-DERIVED rather than widened, which is the kind of thing that quietly
rots when a tuple grows: the nudge touches only `low` by ±1 LSB and `emit` is only reached with a
non-silent level, so the nudged tuple can collide with the four-field stop sentinel only at
`(1,0,0,0)`. A test pins both directions — refuse at `(1,0,0,0)`, flip freely at `(1,0,lt,0)`.
Apple renders them: `RumbleRenderer` gains `Motor?` slots at `GCHapticsLocality.leftTrigger` /
`.rightTrigger` beside the existing handles. A controller without trigger actuators degrades
silently — a nil engine yields a nil slot and `reconcile` no-ops — and absent localities are never
logged, because on most pads that is the normal case rather than a fault. The macOS DualSense
raw-HID branch stays a deliberate no-op: a DualSense has ADAPTIVE triggers, not trigger rumble
motors, and inventing a mapping there would buzz the wrong thing.
🛑 BUILT AHEAD OF A PRODUCER, DELIBERATELY, AND NOTHING HERE CLAIMS OTHERWISE. Nothing can currently
source trigger rumble on Windows and that is measured, not assumed: `XINPUT_VIBRATION` has two
members, and GameInput — the only four-motor API — does not enumerate an xinputhid-promoted Xbox pad
at all, verified against a REAL Microsoft Elite which is equally invisible to it while classic
XInput reads it live. So this path has never been exercised end to end and the comments say so.
VERIFIED
* `cargo test -p punktfunk-core --features quic --lib` 378 passed on macOS, 203 on Windows;
clippy `-D warnings` clean with and without default features; `cargo fmt --all --check` clean.
* The generated header is regenerated and idempotent on re-run (CI diffs it).
* SWIFT ACTUALLY COMPILES AND RUNS: `swift build` clean and `swift test` 262 passed / 0 failures
in `clients/apple`, against a locally built xcframework. (Editor SourceKit errors about
`PunktfunkCore`/`DualSenseHID` are index noise from that gitignored artifact — a real build
resolves both, and the `DualSenseHID` references are untouched by this change.)
* `cargo build -p punktfunk-host` clean on Windows.
NOT VERIFIED
* End to end — see above; there is no producer.
* Whether a real Xbox pad on Apple actually reports the two trigger localities. The degrade needs
no code, but the positive case is untested.
* `pf-client-core` (the SDL renderer) does not build on macOS at baseline and is unbuilt here. It
only reads `RumbleCommand` fields and never constructs one, so added fields cannot break it, but
it still calls `_cmd`; wiring `SDL_RumbleGamepadTriggers` is separate work.
ANDROID: NOT DONE, and it should stay that way for now. `pack_rumble` packs pad/backstop/low/high
into bits 0..52 of a `jlong` with `-1` reserved as a sentinel — two more `u16` do not fit. The right
fix if ever wanted is the direct-`ByteBuffer` shape `nativeNextHidout` already uses in the same file
(zero-allocation, caller-owned, the established idiom), not a second `jlong` (racy across two calls)
nor `long[]` (an allocation per pull). But no Android device exposes trigger actuators at all, so
there is nothing to render. Separately stale and also not fixed: `NativeBridge.kt`'s KDoc still
documents the v2 `ttl_ms` layout rather than `backstop_ms`.
punktfunk — Apple client (macOS · iOS · iPadOS · tvOS)
The native Apple app for streaming a punktfunk host to your Mac, iPhone, iPad, or Apple TV. A SwiftUI app that finds hosts on your network, pairs with a PIN, and streams at your display's own resolution and refresh rate — with VideoToolbox hardware decode and full controller support.
All the networking and protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM,
Opus audio, cert pinning — lives in the shared Rust punktfunk-core (statically linked as
PunktfunkCore.xcframework). This package is the Swift shell: decode, present, input, and UI.
Features
- Hardware decode — VideoToolbox H.264/HEVC (plus AV1 on devices with an AV1 hardware
decoder — M3-class Macs, A17 Pro-class iPhones), with a low-latency stage-2 presenter
(
VTDecompressionSession→CAMetalLayer, presented off aCADisplayLink, ~11 ms p50) as the default and anAVSampleBufferDisplayLayerfallback. - HDR & 4:4:4 — PQ passthrough with a correct reference-white anchor, mid-session SDR↔HDR reconfiguration, and hardware-probed 4:4:4 support.
- Your display's native mode — the host builds a virtual output at exactly your WxH@Hz; mid-stream resize renegotiates without reconnecting.
- Audio both ways — Opus playback (CoreAudio, no bundled libopus) with a jitter ring, plus mic uplink; speaker/mic selectable in Settings.
- Full controller support — one selected controller forwarded as pad 0, including DualSense feedback (rumble → CoreHaptics, lightbar, player LEDs, adaptive triggers) and touchpad/motion. The virtual pad type auto-resolves from your physical controller.
- Mouse & keyboard —
GCMouse/GCKeyboardcapture with click-to-capture and a ⌃⌥⇧Q release (the cross-client Ctrl+Alt+Shift+Q; ⌘⎋ still works as the macOS/iPad toggle), plus iPad pointer lock and touch input. - Find hosts automatically — mDNS discovery (
NWBrowserover_punktfunk._udp); first connect does a one-time SPAKE2 PIN pairing (or TOFU on trusted LANs), then reconnects on a pinned, Keychain-stored identity. - Tune the stream — a fps / Mb·s / latency HUD (skew-corrected across machines), a bitrate control, a per-host network speed test with a recommended bitrate, and a host-compositor picker.
Runs from one shared codebase across macOS, iOS, iPadOS, and tvOS.
Get it
Install from the App Store / TestFlight, or build from source below. Per-device install steps and the pairing walkthrough: docs.punktfunk.unom.io/docs/install-client.
Build / run / test (on a Mac)
Requires Xcode 26.5 / Swift 6.3. First build the Rust core into an xcframework, then build the app:
rustup target add aarch64-apple-darwin x86_64-apple-darwin
bash scripts/build-xcframework.sh # → clients/apple/PunktfunkCore.xcframework
# BUILD_IOS=1 also builds the iOS slices (add the ios rustup targets)
# BUILD_TVOS=1 also builds tvOS (tier-3 targets, built from source — see below)
cd clients/apple
open Punktfunk.xcodeproj # the real app: ⌘R builds + runs Punktfunk.app
swift run PunktfunkClient # or the unbundled dev shell (CLI)
swift build && swift test # unit + loopback/remote tests (self-skip w/o a host)
tvOS slices are tier-3 Rust targets, built from source:
rustup toolchain install nightly && rustup component add rust-src --toolchain nightly.
Test against a host
# full loopback proof — builds punktfunk-host (synthetic source, runs on macOS) and streams
# byte-verified frames into the Swift client, incl. the PIN pairing ceremony:
bash test-loopback.sh
# against a real Linux host on the LAN (see the repo README "Running on this box"):
PUNKTFUNK_REMOTE_HOST=<box-ip> swift test --filter RemoteFirstLightTests # headless
PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkClient # on glass
Project layout
PunktfunkKit(library) — the reusable pieces:PunktfunkConnection— the wrapper over the C ABI (thread-safeclose(), per-plane locks, pinning + TOFU).AnnexB/StreamView/VideoDecoder/MetalVideoPresenter— format handling, the stage-1 (AVSampleBufferDisplayLayer) and stage-2 (VTDecompressionSession→CAMetalLayer) presenters.InputCapture—GCMouse/GCKeyboard→ host VK/mouse, with fractional-delta accumulation.GamepadManager/GamepadCapture/GamepadFeedback/DualSenseTriggerEffect— controller discovery + selection, capture (buttons/axes/touchpad/motion), and host-feedback rendering.HostDiscovery—NWBrowserover_punktfunk._udp.
PunktfunkClient(the app) — hosts grid with an On this network section, add-host sheet, the two trust flows (TOFU prompt + SPAKE2PairSheet), the stream view with the HUD, a tabbed Settings pane (General / Display / Audio / Controllers / Advanced), and the network speed test. A Scene-level Stream menu carries the cross-client shortcut set: Release Mouse (⌃⌥⇧Q), Disconnect (⌃⌥⇧D) and the HUD toggle (⌃⌥⇧S) — the same Ctrl+Alt+Shift combos the Windows and Linux clients reserve, also shown on a 6-second banner at stream start. On iOS/iPadOS and macOS a connected controller swaps the whole home for the gamepad UI (Home/Gamepad*,Settings/GamepadSettingsView): a console-style host carousel (A connect · Y library · X settings), a controller-navigable settings screen, an add-host flow with an on-screen controller keyboard (no touch required anywhere), and the coverflow library browser — all driven by the sharedGamepadMenuInputpoller +GamepadCarousel/GamepadMenuListfocus machinery, with dual-channel haptics (device Taptic + controllerMenuHaptics), over an animated "aurora" backdrop (GamepadScreenBackground— TimelineView-driven drifting color blobs; deliberately pure SwiftUI, since a .metal library only reliably bundles in one of the two build systems these sources compile under). macOS presents the settings/add-host screens as sheets (nofullScreenCoverthere);PUNKTFUNK_FORCE_GAMEPAD_UI=1forces the mode without a physical pad (dev/screenshots).- Tests (
swift test) — Annex-B units, a real-codec VideoToolbox round trip, DualSense trigger-effect and gamepad-wire conversions, loopback integration against real local hosts, and the remote first-light test.
Notes for contributors
- Xcode project (
Punktfunk.xcodeproj) wraps the same sources as theswift runshell (a synchronized folder — no duplication). The macOS target is App-Sandboxed (needsnetwork.server— the raw-UDP plane and quinn bothbind()); iOS/tvOS use the shared entitlements file (keepapp-sandboxout of it). Verify withcodesign -d --entitlements :- <built .app>. - Decode flow: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band, and recovery keyframes re-send them — refresh the format description on every IDR; there is no out-of-band extradata, ever.
- ABI threading: one video pump thread per connection, one optional audio drain thread, and one
optional feedback drain thread (rumble + HID-output).
send()is enqueue-only and safe alongside all of them. The wrapper's per-plane locks makeclose()safe from anywhere. - DualSense motion scale (
GamepadWire) is derived from hid-playstation's math, not yet live-verified — if gyro/accel feel wrong in a game, correct sign/scale there andevtestthe host's virtual pad. - App Store screenshots are automated —
tools/screenshots.sh allrenders the real UI at the required pixel sizes via a DEBUG-only shot mode; theappleCI workflow captures the iOS sizes on every main push. See the script header for details. The script'sSCENESarray is the listing set, in listing order; override it (SCENES="06-gamepad-home 10-edithost" tools/screenshots.sh ios) to capture any of the other scenes inShotScenes.all. Mock data — hosts, adverts, profiles — is seeded inShotMockso a capture is byte-for-byte deterministic and never browses the real LAN (a stranger's hostname reached the live listing that way once). - Deeper design notes live in the internal planning repo (punktfunk-planning:
apple-stage2-presenter.md).
Related
- Documentation — quick start, pairing, troubleshooting
- Project README — the host, the other clients, and how it all fits together