macOS GCKeyboard delivery is flaky — the same GameController quirk that
killed GCMouse motion (e414ec0). Keyboard input intermittently failed to
reach the host (e.g. typing in a gamescope game). Switch the macOS key
source to NSEvent, mirroring the mouse fix:
- StreamLayerView.keyDown/keyUp map NSEvent.keyCode (Carbon virtual
keycode) → Windows VK via the new InputCapture.keyCodeToVK table and
forward through InputCapture.sendKey, then consume the event (no beep).
- flagsChanged drives InputCapture.handleFlagsChanged, which diffs the raw
modifier flags to recover each L/R modifier down/up (modifiers never fire
keyDown/keyUp on macOS) and emits the same L/R VKs hidToVK already does.
- The macOS GCKeyboard keyChangedHandler is disabled (#if !os(macOS)) so it
can't double-send; iOS keeps the GCKeyboard path unchanged.
sendKey honors the ⌘⎋ capture-toggle suppressedVK latch and tracks into
pressedVKs so releaseAll()/blur flushes anything still held. The emitted
VKs are identical to the existing HID path, so the host (vk_to_evdev)
needs no change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Host-side logs proved the macOS client sent keyboard + scroll but ZERO relative
mouse-motion and ZERO button events for an entire session — the user was moving
the mouse the whole time. Root cause is client-side: GCMouse's
mouseMovedHandler/pressedChangedHandler silently never fired on the live Mac
(a documented GameController quirk) while GCKeyboard worked and scroll already
rode NSEvent. So motion/buttons were the only input on a GCMouse-only path, and
that path was dead.
macOS: stop relying on GCMouse for motion/buttons (compiled out with
#if !os(macOS)); drive them from a local NSEvent monitor installed only while
captured — the same channel scrollWheel already uses successfully. Under
CGAssociateMouseAndMouseCursorPosition(false) the mouseMoved/dragged deltaX/deltaY
ARE the relative motion (OS-acceleration-applied, exactly what Moonlight's macOS
client ships). All four motion event types are covered so motion keeps flowing
during a button-held drag; buttons map left/right/middle/X1/X2 through the
existing engage-click-suppression + release-on-blur logic. NSEvent deltaY is
already screen-space (+y down) so, unlike the GCMouse path, it is NOT negated.
iPad: the input failure there was a different cause — GCMouse only delivers
relative deltas while the scene holds a true pointer LOCK, which the system grants
only to a full-screen, frontmost iPad scene and which UIHostingController doesn't
consult for children. Gate prefersPointerLocked to iPad + captured, add
childViewControllerForPointerLock so a reparenting container forwards the lock
decision to this VC, and log the resolved lock state. Touch remains the
unconditional fallback.
Adds a PUNKTFUNK_INPUT_DEBUG=1 switch (os.Logger, throttled) so motion/buttons
being SENT is verifiable on-device without host-side logs. iOS GCMouse path
otherwise unchanged; GCKeyboard unchanged on both. Researched + adversarially
reviewed; Swift builds only on a Mac, so this is unverified-compiled here.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The same app now runs on tvOS (target Punktfunk-tvOS, bundle io.unom.punktfunk.tvos),
validated live against the box: vkcube at 1280x720@60, 60 fps in the Apple TV 4K
simulator, glass HUD with a focusable Disconnect button.
- PunktfunkCore.xcframework grows tvOS device + universal-simulator slices. These are
TIER-3 Rust targets (no prebuilt std): BUILD_TVOS=1 builds them with nightly and
-Zbuild-std from rust-src — the full quic stack (quinn/rustls-ring/tokio) compiles
for tvOS unchanged.
- The UIKit stream view covers iOS AND tvOS, with pointer interaction, pointer lock,
touch forwarding and InputCapture gated to iOS — tvOS is view-only until gamepad
capture lands (the natural tvOS input).
- SessionAudio on tvOS: .playback session, no mic (no app-accessible microphone).
- App chrome gates: keyboardShortcut/textSelection/controlSize/statusBarHidden are
iOS/macOS-only; host cards use the focus-native .card button style on tvOS; the
Audio settings section hides (system-routed); mode seeding works from the TV screen
(1920x1080@60).
- Package platforms += .tvOS(.v17); new Xcode target + shared scheme
(TARGETED_DEVICE_FAMILY 3, local-network usage description included).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The whole client now runs on iPadOS/iOS from the same sources, first-lit live in the
iPad simulator against the real host at 1280x720@60 (60 fps on the HUD, capture state
machine active, mic permission flow shown).
- PunktfunkCore.xcframework grows iOS device + universal-simulator slices
(BUILD_IOS=1; rustup targets aarch64-apple-ios{,-sim} + x86_64-apple-ios).
- The decode pump is extracted into a shared StreamPump (identical IDR re-gate logic on
both platforms); the iOS StreamView (StreamViewIOS.swift) has the same name/signature
as the macOS one, so ContentView & co. are byte-identical across platforms — hosted
in a UIViewController for prefersPointerLocked (the iPadOS cursor capture; see README
note 9 for the UIHostingController forwarding caveat).
- Touch is always forwarded: per-finger wire ids, coordinates mapped through the
aspect-fit letterbox into LIVE host-mode pixels (surface == host mode, identity
rescale host-side; follows mid-stream requestMode switches).
- InputCapture is cross-platform: GC works the same on iPadOS, ⌘⎋ is detected from the
HID stream there; stale-⌘ tracking after focus loss fixed on both platforms
(releaseAll now drops the modifier/latch state — a ⌘ released in another app
otherwise hijacked Esc forever).
- SessionAudio: AVAudioSession on iOS (.playAndRecord + .defaultToSpeaker — without it
iPhones route host audio to the EARPIECE; deactivated with
notifyOthersOnDeactivation on stop so interrupted background audio resumes); HAL
device pinning + the Settings pickers stay macOS-only.
- New Punktfunk-iOS app target (shared synchronized sources, generated Info.plist with
mic + local-network usage descriptions — QUIC to a LAN host trips local network
privacy on real devices — scene manifest + indirect input events for Stage Manager /
external displays), shared scheme, macOS min-window frames gated off iOS.
For the iPad-on-an-external-screen idea: with multiple scenes + indirect input enabled,
Stage Manager iPads can drag the punktfunk window onto the external display and drive
the PC with keyboard/mouse/touch. Known gaps (README note 9): the pointer-lock
preference isn't consulted through UIHostingController (relative mouse works, the local
cursor just stays visible) and AVAudioSession interruptions don't auto-restart audio.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Scroll was wired to GCMouse's scroll dpad, which only fires for plain HID wheel
deltas — trackpad and Magic Mouse scrolling are gesture events that never reach
GameController, so scrolling was dead on the default Mac setups. The stream view now
overrides scrollWheel (while captured the cursor is parked mid-view, so it receives
every scroll event) and feeds InputCapture.sendScroll: precise gesture deltas are
pixels (~0.1 notch/px, SDL's factor → ×12 for WHEEL_DELTA(120)), classic wheels are
lines (×120), fractional remainders accumulate, and the GC scroll handler is gone so
wheel mice can't double-deliver. Signs pass through as-is, preserving the local
(natural-)scrolling preference.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Capture used to engage whenever the app became active, so the click that activates the
window — on the title bar (a drag) or a resize edge — got the cursor warped away
mid-gesture, and raw deltas kept streaming to the host while the user fought the window.
Reworked Moonlight-style, with capture as a deliberate, reversible state owned by
StreamLayerView:
- Engage: automatically once when the stream starts / trust is confirmed (one-shot, can
never fire surprisingly later), or by clicking into the video (that click's
press/release are suppressed toward the host; acceptsFirstMouse makes it one click
from another app). NEVER on app re-activation.
- Release: ⌘⎋ (toggles, key-window-scoped), focus loss — now including same-app window
switches (⌘, / ⌘N / ⌘M resign key without resigning the app; previously the new
window inherited a hidden frozen cursor and its typing was double-delivered to the
host) — and disconnect.
- While released: nothing is forwarded (InputCapture.forwarding gates the GC handlers;
held keys/buttons are flushed host-side so nothing sticks), the cursor is free, and
the HUD (now showing the capture state) is clickable.
- The no-beep behavior moved from the NSEvent monitor to first-responder key
consumption — swallowing at the monitor risked starving GC's own delivery (the
"input broken altogether" report). The monitor now only intercepts ⌘⎋.
- Adversarial-review fixes: a second session preempts the previous one cleanly instead
of leaving it captured with dead GC handlers (onPreempted); the engage click's
suppression latch can't outlive the click (mouseUp backstop); ⌘⎋'s physical Esc can't
type into the host in either toggle direction (suppressedVK latch + Esc-while-⌘
guard); capture callbacks defer out of the SwiftUI update pass.
Validated live against the box: 16185 input datagrams injected during a captured
session (gamescope EIS), title-bar drag/resize free while released, and visible
cursor + typing on a streamed KWin desktop, all user-confirmed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GCKeyboard reads the HID state directly, so the key NSEvents kept traveling the
responder chain unhandled — and an unhandled keyDown makes NSWindow play the
"invalid input" sound on every keystroke. InputCapture now installs a local event
monitor for its lifetime that swallows key events, except ⌘-combos, which still
reach the local app (the HUD's ⌘D disconnect, ⌘Q) in addition to the host.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Full project rename, decided 2026-06-10:
- Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs.
- C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h,
PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl.
PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants).
- Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1.
WIRE BREAK: clients must be rebuilt from this revision.
- Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / ….
- Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the
persistent identity is unchanged, pinned fingerprints stay valid).
- Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection
(Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated.
- scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated.
Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of
"desktop but no apps/settings" over the stream: plasmashell launched without
XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and
rendered an empty menu. The script sets the complete KDE session env (menu prefix,
KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell.
Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS,
zero lumen references left outside .git.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler