forked from unom/punktfunk
enricobuehler
08a06ff295
WP6 of design/presenter-cadence-rework-implementation-plan.md — the Apple
binding of the CadenceClock landed in e207d0d7.
Apple's video pipeline is Swift and does not link the Rust core, so this is a
hand-written twin the way AudioRing is of JitterPolicy. That kind of port is only
as good as its evidence of agreement, so beyond the ten mirrored unit tests the
two implementations were run against each other numerically: disposable
harnesses on both sides, importing the REAL Rust type, printing eight vectors —
LCG output, the 400-frame settle state, head and tail due times, the ramp's
type-2 versus type-1 error, a lumpy source's dues, cushion and jitter under wide
jitter, and an outlier + gap + regression + off-cadence event sequence. All eight
lines diffed identical, down to `offset=-1784999999987888109 skew=946
jitter=514645`. Both scaffolds are deleted; the doc comments name the Rust type
and those vectors as the contract.
The clock-domain trap is the whole risk here and it is worth stating plainly:
`presentAtMediaTime` consumes `CACurrentMediaTime`, while `decodedNs` is
CLOCK_REALTIME, and §2.2's rule is one domain in, same domain out. So
`mediaTimeNs(forRealtimeNs:)` is written as the exact inverse of the existing
media→realtime bridge, reading the two clocks in the same order so the sub-µs
skew between them cancels on a round trip. The conversion happens ONCE, on the
way in; the due time comes back in media time and is consumed by `nextVsync` and
the store predicate with no second conversion — which is precisely what
`domainOffsetIsAbsorbed` licenses. It is per frame rather than per session
because the two clocks diverge across device sleep, the one case where the offset
is not constant.
The deadline presenter (iPhone/iPad) is included even though WP6 names only
`presentAt`, which that loop has no equivalent of: its link vend IS the grid
snap, so the due-gated `take` alone makes it cadence-driven. Leaving it out would
have made WP8's "Mac + iPhone" leg test nothing on half its hardware.
Preroll retires only on the due-gated path; plain `take()` and its tests are
untouched. On the cadence path `underflows` is deliberately not counted — an
empty store is the normal steady state once frames are held until due, and
`CadenceHealth.late` is the honest starvation signal instead.
Smoothness only: the clock exists iff the store policy is FIFO, so under
`latency` `takeReady` is the old `ring.take()`, `dueMediaTime` is nil and
`max(now, now)` is the old expression, and the stats line has no cadence segment.
Verified: `swift test` 319 passing, 0 failures, including the ten new ones.
⚠ iOS/tvOS could not be typechecked — the checked-in xcframework's Info.plist
declares only `macos-arm64`, which is pre-existing and unrelated. Confirmed
instead that no added line sits inside any `#if os(...)`, so the macOS build
typechecks the entire diff.
punktfunk — Apple client (macOS · iOS · iPadOS · tvOS)
The native Apple app for streaming a punktfunk host to your Mac, iPhone, iPad, or Apple TV. A SwiftUI app that finds hosts on your network, pairs with a PIN, and streams at your display's own resolution and refresh rate — with VideoToolbox hardware decode and full controller support.
All the networking and protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM,
Opus audio, cert pinning — lives in the shared Rust punktfunk-core (statically linked as
PunktfunkCore.xcframework). This package is the Swift shell: decode, present, input, and UI.
Features
- Hardware decode — VideoToolbox H.264/HEVC (plus AV1 on devices with an AV1 hardware
decoder — M3-class Macs, A17 Pro-class iPhones), with a low-latency stage-2 presenter
(
VTDecompressionSession→CAMetalLayer, presented off aCADisplayLink, ~11 ms p50) as the default and anAVSampleBufferDisplayLayerfallback. - HDR & 4:4:4 — PQ passthrough with a correct reference-white anchor, mid-session SDR↔HDR reconfiguration, and hardware-probed 4:4:4 support.
- Your display's native mode — the host builds a virtual output at exactly your WxH@Hz; mid-stream resize renegotiates without reconnecting.
- Audio both ways — Opus playback (CoreAudio, no bundled libopus) with a jitter ring, plus mic uplink; speaker/mic selectable in Settings.
- Full controller support — one selected controller forwarded as pad 0, including DualSense feedback (rumble → CoreHaptics, lightbar, player LEDs, adaptive triggers) and touchpad/motion. The virtual pad type auto-resolves from your physical controller.
- Mouse & keyboard —
GCMouse/GCKeyboardcapture with click-to-capture and a ⌃⌥⇧Q release (the cross-client Ctrl+Alt+Shift+Q; ⌘⎋ still works as the macOS/iPad toggle), plus iPad pointer lock and touch input. - Find hosts automatically — mDNS discovery (
NWBrowserover_punktfunk._udp); first connect does a one-time SPAKE2 PIN pairing (or TOFU on trusted LANs), then reconnects on a pinned, Keychain-stored identity. - Tune the stream — a fps / Mb·s / latency HUD (skew-corrected across machines), a bitrate control, a per-host network speed test with a recommended bitrate, and a host-compositor picker.
Runs from one shared codebase across macOS, iOS, iPadOS, and tvOS.
Get it
Install from the App Store / TestFlight, or build from source below. Per-device install steps and the pairing walkthrough: docs.punktfunk.unom.io/docs/install-client.
Build / run / test (on a Mac)
Requires Xcode 26.5 / Swift 6.3. First build the Rust core into an xcframework, then build the app:
rustup target add aarch64-apple-darwin x86_64-apple-darwin
bash scripts/build-xcframework.sh # → clients/apple/PunktfunkCore.xcframework
# BUILD_IOS=1 also builds the iOS slices (add the ios rustup targets)
# BUILD_TVOS=1 also builds tvOS (tier-3 targets, built from source — see below)
cd clients/apple
open Punktfunk.xcodeproj # the real app: ⌘R builds + runs Punktfunk.app
swift run PunktfunkClient # or the unbundled dev shell (CLI)
swift build && swift test # unit + loopback/remote tests (self-skip w/o a host)
tvOS slices are tier-3 Rust targets, built from source:
rustup toolchain install nightly && rustup component add rust-src --toolchain nightly.
Test against a host
# full loopback proof — builds punktfunk-host (synthetic source, runs on macOS) and streams
# byte-verified frames into the Swift client, incl. the PIN pairing ceremony:
bash test-loopback.sh
# against a real Linux host on the LAN (see the repo README "Running on this box"):
PUNKTFUNK_REMOTE_HOST=<box-ip> swift test --filter RemoteFirstLightTests # headless
PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkClient # on glass
Project layout
PunktfunkKit(library) — the reusable pieces:PunktfunkConnection— the wrapper over the C ABI (thread-safeclose(), per-plane locks, pinning + TOFU).AnnexB/StreamView/VideoDecoder/MetalVideoPresenter— format handling, the stage-1 (AVSampleBufferDisplayLayer) and stage-2 (VTDecompressionSession→CAMetalLayer) presenters.InputCapture—GCMouse/GCKeyboard→ host VK/mouse, with fractional-delta accumulation.GamepadManager/GamepadCapture/GamepadFeedback/DualSenseTriggerEffect— controller discovery + selection, capture (buttons/axes/touchpad/motion), and host-feedback rendering.HostDiscovery—NWBrowserover_punktfunk._udp.
PunktfunkClient(the app) — hosts grid with an On this network section, add-host sheet, the two trust flows (TOFU prompt + SPAKE2PairSheet), the stream view with the HUD, a tabbed Settings pane (General / Display / Audio / Controllers / Advanced), and the network speed test. A Scene-level Stream menu carries the cross-client shortcut set: Release Mouse (⌃⌥⇧Q), Disconnect (⌃⌥⇧D) and the HUD toggle (⌃⌥⇧S) — the same Ctrl+Alt+Shift combos the Windows and Linux clients reserve, also shown on a 6-second banner at stream start. On iOS/iPadOS and macOS a connected controller swaps the whole home for the gamepad UI (Home/Gamepad*,Settings/GamepadSettingsView): a console-style host carousel (A connect · Y library · X settings), a controller-navigable settings screen, an add-host flow with an on-screen controller keyboard (no touch required anywhere), and the coverflow library browser — all driven by the sharedGamepadMenuInputpoller +GamepadCarousel/GamepadMenuListfocus machinery, with dual-channel haptics (device Taptic + controllerMenuHaptics), over an animated "aurora" backdrop (GamepadScreenBackground— TimelineView-driven drifting color blobs; deliberately pure SwiftUI, since a .metal library only reliably bundles in one of the two build systems these sources compile under). macOS presents the settings/add-host screens as sheets (nofullScreenCoverthere);PUNKTFUNK_FORCE_GAMEPAD_UI=1forces the mode without a physical pad (dev/screenshots).- Tests (
swift test) — Annex-B units, a real-codec VideoToolbox round trip, DualSense trigger-effect and gamepad-wire conversions, loopback integration against real local hosts, and the remote first-light test.
Notes for contributors
- Xcode project (
Punktfunk.xcodeproj) wraps the same sources as theswift runshell (a synchronized folder — no duplication). The macOS target is App-Sandboxed (needsnetwork.server— the raw-UDP plane and quinn bothbind()); iOS/tvOS use the shared entitlements file (keepapp-sandboxout of it). Verify withcodesign -d --entitlements :- <built .app>. - Decode flow: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band, and recovery keyframes re-send them — refresh the format description on every IDR; there is no out-of-band extradata, ever.
- ABI threading: one video pump thread per connection, one optional audio drain thread, and one
optional feedback drain thread (rumble + HID-output).
send()is enqueue-only and safe alongside all of them. The wrapper's per-plane locks makeclose()safe from anywhere. - DualSense motion scale (
GamepadWire) is derived from hid-playstation's math, not yet live-verified — if gyro/accel feel wrong in a game, correct sign/scale there andevtestthe host's virtual pad. - App Store screenshots are automated —
tools/screenshots.sh allrenders the real UI at the required pixel sizes via a DEBUG-only shot mode; theappleCI workflow captures the iOS sizes on every main push. See the script header for details. The script'sSCENESarray is the listing set, in listing order; override it (SCENES="06-gamepad-home 10-edithost" tools/screenshots.sh ios) to capture any of the other scenes inShotScenes.all. Mock data — hosts, adverts, profiles — is seeded inShotMockso a capture is byte-for-byte deterministic and never browses the real LAN (a stranger's hostname reached the live listing that way once). - Deeper design notes live in the internal planning repo (punktfunk-planning:
apple-stage2-presenter.md).
Related
- Documentation — quick start, pairing, troubleshooting
- Project README — the host, the other clients, and how it all fits together