enricobuehlerandClaude Fable 5
ac3dc4323f
Native sessions could survive long after the client was gone, in two
independent ways.
HOST: `stop` was only advisory. The one thing that ends a session is the
stream thread returning — every teardown (conn.close, the joins, and the
RAII drops of the session permit, admission entry and stream marker)
sits after that await, and nothing forced the issue. The encode loop
checks `stop` between iterations, so any unbounded call INSIDE one never
reaches the check, and one stuck syscall became a permanent zombie: it
held its semaphore slot (four of those and the host stops accepting
QUIC entirely), its admission entry (a later client gets "host busy"
forever), and not even the console's Stop button could clear it — that
button sets this same flag.
* Bound the wait: once the session has been told to stop, the thread
gets STREAM_STOP_GRACE (90 s, well past the 40 s capture-rebuild
budget) to return, then teardown runs anyway. The thread is detached,
not killed — Rust can't cancel a blocking thread — so it keeps its
capturer/encoder until the stuck call returns, but the session's slot
and admission entry come back and the host keeps serving. It logs at
ERROR as the host wedge it is.
* Bound the audio/input joins too — the last unbounded await in
teardown.
* Take the session permit AFTER the QUIC handshake instead of before
`accept()`, so a host at its concurrency cap still accepts and the
waiting client sees a live path instead of a silent dial timeout.
* Bound the compositor helpers that caused the wedge in the first
place: new pf-vdisplay `proc::{status_within, output_within}` kill a
child that outlives its budget. `kscreen-doctor` is a Wayland client
of the very compositor it configures, so against a wedged KWin it
never returned; same for systemctl/dbus against a stuck session bus.
CLIENTS: the connection was never closed, so the host was right to keep
the session — it still had a live, keep-alive-answering peer.
* Android: backgrounding did no teardown at all, and Android doesn't
suspend the process, so the worker kept answering keep-alives until
the OS reclaimed it (on a TV box, never). End the session on ON_STOP,
via the existing onDispose path; a plain close, not a quit, so the
host lingers the display for a fast return.
* Apple: the .background arm was iOS-only AND gated on an opt-in that
defaults off, so backgrounding did nothing — while the `audio`
background mode kept the app (and its connection) alive indefinitely.
Act unconditionally, and cover tvOS.
* Core: `conn.close()` only queues the frame, and run_pump is the body
of a block_on whose runtime is dropped the instant it returns, so the
driver could never put it on the wire — a deliberate quit reached the
host as silence (8 s idle timeout, no quit code, and the linger meant
for an unwanted disconnect). Carry the endpoint out of the handshake
and flush with wait_idle(), the same discipline the pairing and probe
paths already use.
Linux check/clippy/tests green: 262 host, 71 pf-vdisplay (incl. new
bounded-process tests), 231 core.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
punktfunk — Apple client (macOS · iOS · iPadOS · tvOS)
The native Apple app for streaming a punktfunk host to your Mac, iPhone, iPad, or Apple TV. A SwiftUI app that finds hosts on your network, pairs with a PIN, and streams at your display's own resolution and refresh rate — with VideoToolbox hardware decode and full controller support.
All the networking and protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM,
Opus audio, cert pinning — lives in the shared Rust punktfunk-core (statically linked as
PunktfunkCore.xcframework). This package is the Swift shell: decode, present, input, and UI.
Features
- Hardware decode — VideoToolbox H.264/HEVC (plus AV1 on devices with an AV1 hardware
decoder — M3-class Macs, A17 Pro-class iPhones), with a low-latency stage-2 presenter
(
VTDecompressionSession→CAMetalLayer, presented off aCADisplayLink, ~11 ms p50) as the default and anAVSampleBufferDisplayLayerfallback. - HDR & 4:4:4 — PQ passthrough with a correct reference-white anchor, mid-session SDR↔HDR reconfiguration, and hardware-probed 4:4:4 support.
- Your display's native mode — the host builds a virtual output at exactly your WxH@Hz; mid-stream resize renegotiates without reconnecting.
- Audio both ways — Opus playback (CoreAudio, no bundled libopus) with a jitter ring, plus mic uplink; speaker/mic selectable in Settings.
- Full controller support — one selected controller forwarded as pad 0, including DualSense feedback (rumble → CoreHaptics, lightbar, player LEDs, adaptive triggers) and touchpad/motion. The virtual pad type auto-resolves from your physical controller.
- Mouse & keyboard —
GCMouse/GCKeyboardcapture with click-to-capture and a ⌃⌥⇧Q release (the cross-client Ctrl+Alt+Shift+Q; ⌘⎋ still works as the macOS/iPad toggle), plus iPad pointer lock and touch input. - Find hosts automatically — mDNS discovery (
NWBrowserover_punktfunk._udp); first connect does a one-time SPAKE2 PIN pairing (or TOFU on trusted LANs), then reconnects on a pinned, Keychain-stored identity. - Tune the stream — a fps / Mb·s / latency HUD (skew-corrected across machines), a bitrate control, a per-host network speed test with a recommended bitrate, and a host-compositor picker.
Runs from one shared codebase across macOS, iOS, iPadOS, and tvOS.
Get it
Install from the App Store / TestFlight, or build from source below. Per-device install steps and the pairing walkthrough: docs.punktfunk.unom.io/docs/install-client.
Build / run / test (on a Mac)
Requires Xcode 26.5 / Swift 6.3. First build the Rust core into an xcframework, then build the app:
rustup target add aarch64-apple-darwin x86_64-apple-darwin
bash scripts/build-xcframework.sh # → clients/apple/PunktfunkCore.xcframework
# BUILD_IOS=1 also builds the iOS slices (add the ios rustup targets)
# BUILD_TVOS=1 also builds tvOS (tier-3 targets, built from source — see below)
cd clients/apple
open Punktfunk.xcodeproj # the real app: ⌘R builds + runs Punktfunk.app
swift run PunktfunkClient # or the unbundled dev shell (CLI)
swift build && swift test # unit + loopback/remote tests (self-skip w/o a host)
tvOS slices are tier-3 Rust targets, built from source:
rustup toolchain install nightly && rustup component add rust-src --toolchain nightly.
Test against a host
# full loopback proof — builds punktfunk-host (synthetic source, runs on macOS) and streams
# byte-verified frames into the Swift client, incl. the PIN pairing ceremony:
bash test-loopback.sh
# against a real Linux host on the LAN (see the repo README "Running on this box"):
PUNKTFUNK_REMOTE_HOST=<box-ip> swift test --filter RemoteFirstLightTests # headless
PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkClient # on glass
Project layout
PunktfunkKit(library) — the reusable pieces:PunktfunkConnection— the wrapper over the C ABI (thread-safeclose(), per-plane locks, pinning + TOFU).AnnexB/StreamView/VideoDecoder/MetalVideoPresenter— format handling, the stage-1 (AVSampleBufferDisplayLayer) and stage-2 (VTDecompressionSession→CAMetalLayer) presenters.InputCapture—GCMouse/GCKeyboard→ host VK/mouse, with fractional-delta accumulation.GamepadManager/GamepadCapture/GamepadFeedback/DualSenseTriggerEffect— controller discovery + selection, capture (buttons/axes/touchpad/motion), and host-feedback rendering.HostDiscovery—NWBrowserover_punktfunk._udp.
PunktfunkClient(the app) — hosts grid with an On this network section, add-host sheet, the two trust flows (TOFU prompt + SPAKE2PairSheet), the stream view with the HUD, a tabbed Settings pane (General / Display / Audio / Controllers / Advanced), and the network speed test. A Scene-level Stream menu carries the cross-client shortcut set: Release Mouse (⌃⌥⇧Q), Disconnect (⌃⌥⇧D) and the HUD toggle (⌃⌥⇧S) — the same Ctrl+Alt+Shift combos the Windows and Linux clients reserve, also shown on a 6-second banner at stream start. On iOS/iPadOS and macOS a connected controller swaps the whole home for the gamepad UI (Home/Gamepad*,Settings/GamepadSettingsView): a console-style host carousel (A connect · Y library · X settings), a controller-navigable settings screen, an add-host flow with an on-screen controller keyboard (no touch required anywhere), and the coverflow library browser — all driven by the sharedGamepadMenuInputpoller +GamepadCarousel/GamepadMenuListfocus machinery, with dual-channel haptics (device Taptic + controllerMenuHaptics), over an animated "aurora" backdrop (GamepadScreenBackground— TimelineView-driven drifting color blobs; deliberately pure SwiftUI, since a .metal library only reliably bundles in one of the two build systems these sources compile under). macOS presents the settings/add-host screens as sheets (nofullScreenCoverthere);PUNKTFUNK_FORCE_GAMEPAD_UI=1forces the mode without a physical pad (dev/screenshots).- Tests (
swift test) — Annex-B units, a real-codec VideoToolbox round trip, DualSense trigger-effect and gamepad-wire conversions, loopback integration against real local hosts, and the remote first-light test.
Notes for contributors
- Xcode project (
Punktfunk.xcodeproj) wraps the same sources as theswift runshell (a synchronized folder — no duplication). The macOS target is App-Sandboxed (needsnetwork.server— the raw-UDP plane and quinn bothbind()); iOS/tvOS use the shared entitlements file (keepapp-sandboxout of it). Verify withcodesign -d --entitlements :- <built .app>. - Decode flow: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band, and recovery keyframes re-send them — refresh the format description on every IDR; there is no out-of-band extradata, ever.
- ABI threading: one video pump thread per connection, one optional audio drain thread, and one
optional feedback drain thread (rumble + HID-output).
send()is enqueue-only and safe alongside all of them. The wrapper's per-plane locks makeclose()safe from anywhere. - DualSense motion scale (
GamepadWire) is derived from hid-playstation's math, not yet live-verified — if gyro/accel feel wrong in a game, correct sign/scale there andevtestthe host's virtual pad. - App Store screenshots are automated —
tools/screenshots.sh allrenders the real UI at the required pixel sizes via a DEBUG-only shot mode; theappleCI workflow captures the iOS sizes on every main push. See the script header for details. - Deeper design notes live in the internal planning repo (punktfunk-planning:
apple-stage2-presenter.md).
Related
- Documentation — quick start, pairing, troubleshooting
- Project README — the host, the other clients, and how it all fits together