The whole client now runs on iPadOS/iOS from the same sources, first-lit live in the
iPad simulator against the real host at 1280x720@60 (60 fps on the HUD, capture state
machine active, mic permission flow shown).
- PunktfunkCore.xcframework grows iOS device + universal-simulator slices
(BUILD_IOS=1; rustup targets aarch64-apple-ios{,-sim} + x86_64-apple-ios).
- The decode pump is extracted into a shared StreamPump (identical IDR re-gate logic on
both platforms); the iOS StreamView (StreamViewIOS.swift) has the same name/signature
as the macOS one, so ContentView & co. are byte-identical across platforms — hosted
in a UIViewController for prefersPointerLocked (the iPadOS cursor capture; see README
note 9 for the UIHostingController forwarding caveat).
- Touch is always forwarded: per-finger wire ids, coordinates mapped through the
aspect-fit letterbox into LIVE host-mode pixels (surface == host mode, identity
rescale host-side; follows mid-stream requestMode switches).
- InputCapture is cross-platform: GC works the same on iPadOS, ⌘⎋ is detected from the
HID stream there; stale-⌘ tracking after focus loss fixed on both platforms
(releaseAll now drops the modifier/latch state — a ⌘ released in another app
otherwise hijacked Esc forever).
- SessionAudio: AVAudioSession on iOS (.playAndRecord + .defaultToSpeaker — without it
iPhones route host audio to the EARPIECE; deactivated with
notifyOthersOnDeactivation on stop so interrupted background audio resumes); HAL
device pinning + the Settings pickers stay macOS-only.
- New Punktfunk-iOS app target (shared synchronized sources, generated Info.plist with
mic + local-network usage descriptions — QUIC to a LAN host trips local network
privacy on real devices — scene manifest + indirect input events for Stage Manager /
external displays), shared scheme, macOS min-window frames gated off iOS.
For the iPad-on-an-external-screen idea: with multiple scenes + indirect input enabled,
Stage Manager iPads can drag the punktfunk window onto the external display and drive
the PC with keyboard/mouse/touch. Known gaps (README note 9): the pointer-lock
preference isn't consulted through UIHostingController (relative mouse works, the local
cursor just stays visible) and AVAudioSession interruptions don't auto-restart audio.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both directions of the audio plane, on CoreAudio's built-in Opus codec
(kAudioFormatOpus — no bundled libopus; OpusCodec.swift, round trip unit-tested):
- Playback: a drain thread pulls nextAudio() packets, decodes, and writes a priming
jitter ring feeding an AVAudioSourceNode (~20 ms prefill, adaptive to the device's
render quantum so large-buffer devices don't oscillate prime/dropout; a high-water
clamp sheds stall backlog so one network hiccup can't permanently lag audio behind
video; underrun re-primes — one dip, not sustained crackle).
- Mic: a second engine taps the input device, resamples to 48 kHz stereo, Opus-encodes
20 ms chunks and sendMic()s them into the host's virtual PipeWire source. Permission
via AVCaptureDevice (NSMicrophoneUsageDescription added to the Xcode target).
- Settings: Speaker + Microphone pickers (CoreAudio HAL enumeration, persisted by
device UID — "System default" leaves the engine unpinned so it follows macOS device
changes) and a "Send microphone" toggle (default on). Applies from the next session.
- Audio starts with streaming, never during the trust prompt (no host sound — and no
mic uplink — before the user trusted the host); teardown stops audio before close().
Adversarial-review fixes baked in: stop() and the dangling mic-permission callback
share one lock+flag protocol (no hot mic with no owner), the connect-success handler
bails when the attempt was abandoned mid-handshake (no session/mic for a dead window),
SessionAudio gets a deinit backstop (a dropped instance can't pin the connection via
its drain thread), and the render scratch buffer is block-owned (was leaked per
session).
Verified live against the box: remote test decodes 100 host Opus packets to PCM and
the host opens its virtual mic on the first uplinked frame ("punktfunk/1 virtual mic
ready"); on-glass session runs with both engines up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adopts punktfunk_connect_ex from the compositor-selection batch: a Compositor enum on
PunktfunkConnection (auto/kwin/wlroots/mutter/gamescope, with the host's name aliases
for env parsing), a "Host compositor" picker in Settings (default Automatic — a
concrete choice is honored only if that backend is available host-side), and
PUNKTFUNK_COMPOSITOR / PUNKTFUNK_REMOTE_COMPOSITOR pass-throughs for the autoconnect
dev hook and the remote first-light test. The wire change is backward-compatible
(optional trailing byte), so no behavior changes at the default.
Validated live against the box: host with no compositor env (auto-detect = KWin)
logged "honoring client compositor request compositor=gamescope" and streamed 60/60
decoded frames from the spawned gamescope.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Capture used to engage whenever the app became active, so the click that activates the
window — on the title bar (a drag) or a resize edge — got the cursor warped away
mid-gesture, and raw deltas kept streaming to the host while the user fought the window.
Reworked Moonlight-style, with capture as a deliberate, reversible state owned by
StreamLayerView:
- Engage: automatically once when the stream starts / trust is confirmed (one-shot, can
never fire surprisingly later), or by clicking into the video (that click's
press/release are suppressed toward the host; acceptsFirstMouse makes it one click
from another app). NEVER on app re-activation.
- Release: ⌘⎋ (toggles, key-window-scoped), focus loss — now including same-app window
switches (⌘, / ⌘N / ⌘M resign key without resigning the app; previously the new
window inherited a hidden frozen cursor and its typing was double-delivered to the
host) — and disconnect.
- While released: nothing is forwarded (InputCapture.forwarding gates the GC handlers;
held keys/buttons are flushed host-side so nothing sticks), the cursor is free, and
the HUD (now showing the capture state) is clickable.
- The no-beep behavior moved from the NSEvent monitor to first-responder key
consumption — swallowing at the monitor risked starving GC's own delivery (the
"input broken altogether" report). The monitor now only intercepts ⌘⎋.
- Adversarial-review fixes: a second session preempts the previous one cleanly instead
of leaving it captured with dead GC handlers (onPreempted); the engage click's
suppression latch can't outlive the click (mouseUp backstop); ⌘⎋'s physical Esc can't
type into the host in either toggle direction (suppressedVK latch + Esc-while-⌘
guard); capture callbacks defer out of the SwiftUI update pass.
Validated live against the box: 16185 input datagrams injected during a captured
session (gamescope EIS), title-bar drag/resize free while released, and visible
cursor + typing on a streamed KWin desktop, all user-confirmed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pairing/renegotiation batch bumped the punktfunk/1 ABI to v2 and the host now
hard-rejects v1 Hellos (m3.rs), so streaming from the Mac was dead until the bundled
PunktfunkCore.xcframework is rebuilt — it is gitignored, so that is a per-checkout step:
bash scripts/build-xcframework.sh. The Swift wrapper itself was already adapted upstream;
this lands the app on top of it.
- ClientIdentityStore: persistent client identity in the login Keychain, presented on
every connect so paired hosts recognize this Mac. Keychain access failure throws
instead of regenerating (a fresh identity would silently un-pair this Mac from every
--require-pairing host); a lost first-run race resolves toward the stored identity;
pairing uses the strict loadForPairing() so a memory-only identity can't strand a
ceremony.
- PairSheet: the SPAKE2 PIN ceremony, reachable from a host card's context menu and from
the trust prompt's "Pair with PIN instead…" (which drops the live session first — the
host's accept loop is sequential). Success pins the verified fingerprint and connects;
an in-flight ceremony self-discards when the sheet is dismissed, so a late success
can't pin + auto-connect behind the user's back. Wrong PIN and Keychain failures get
distinct, actionable error text.
- Tests: identity unit tests; the full pairing ceremony + --require-pairing gate on
loopback (test-loopback.sh arms a second host, parses its PIN from the log, and gives
both hosts throwaway config homes — no more writes to the real ~/.config/punktfunk);
remote pairing + pinned stream over the LAN (PUNKTFUNK_REMOTE_PIN, _PORT).
Validated live against the box: SPAKE2 ceremony with the host's arming PIN → verified
fingerprint → pinned + identified 720p60 session (host persisted the client identity);
first light 60/60 AUs decoded to pixels; vkcube on glass through the app.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The awaiting-trust and streaming phases rendered StreamView in different switch
branches, so confirming trust dismantled and recreated the NSView — the fresh pump had
already missed the opening IDR (infinite GOP: no other keyframe ever comes) and decoded
nothing. One session branch now hosts a single StreamView; the trust card is an overlay
on the blurred stream and only the capturesCursor flag flips on confirmation.
Verified live against the box (gamescope+vkcube at 720p60, 11.7 Mb/s on glass). Note for
host runs: without PUNKTFUNK_COMPOSITOR=gamescope + PUNKTFUNK_GAMESCOPE_APP, m3-host
auto-picks KWin and streams its (black, empty) session — looks identical to a client
bug but isn't one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The app grows from a dev connect form into a real client shell:
- Home is a grid of saved hosts (UserDefaults-persisted; context menu: Remove / Forget
Identity), "+" in the toolbar opens the add-host sheet, the stream mode moved into
Settings (⌘, / gear) — native resolution stays the only mode, no scaling.
- Trust is now explicit: the protocol always supported certificate pinning, but the app
passed no pin and discarded the observed fingerprint — silently trusting any host.
First connect now shows the host's SHA-256 fingerprint (compare with the "clients pin
this fingerprint" line in the host log) over the live-but-blurred stream; the stream
must pump immediately (the opening IDR is the only guaranteed one), so StreamView gains
a capturesCursor switch to keep the cursor free while the prompt needs clicking, and
input capture starts only after confirmation. Trusting pins the fingerprint per host;
a changed host identity then refuses to connect.
- PUNKTFUNK_AUTOCONNECT keeps working (auto-trusts, doesn't touch the saved hosts).
Host→client authorization (pairing PIN) remains a punktfunk-core roadmap item — the host
still accepts any client that can reach its port.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The host renders its own cursor from our raw deltas, so the local macOS cursor both
stays visible and drifts away from the remote one — and it can wander out of the window,
where a click focuses another app. While the stream has focus, do what Moonlight does:
warp the cursor mid-view, disconnect it from mouse movement
(CGAssociateMouseAndMouseCursorPosition(false) — GCMouse still delivers raw HID deltas),
and hide it. Released on app deactivation (Cmd+Tab is the escape hatch), view teardown,
and disconnect; re-captured when the stream regains focus. The HUD's Disconnect gains ⌘D
since a hidden, frozen cursor can't click it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Full project rename, decided 2026-06-10:
- Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs.
- C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h,
PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl.
PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants).
- Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1.
WIRE BREAK: clients must be rebuilt from this revision.
- Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / ….
- Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the
persistent identity is unchanged, pinned fingerprints stay valid).
- Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection
(Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated.
- scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated.
Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of
"desktop but no apps/settings" over the stream: plasmashell launched without
XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and
rendered an empty menu. The script sets the complete KDE session env (menu prefix,
KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell.
Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS,
zero lumen references left outside .git.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler