15 Commits

Author	SHA1	Message	Date
enricobuehler	4e00037a89	feat(apple): stage-2 default + pixel-perfect, decode robustness, UI/rumble polish ... Stream reliability - Default to the stage-2 presenter (VTDecompressionSession + CAMetalLayer): it detects and recovers a wedged decoder, where stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference frame with no app-side recovery (confirmed Apple limitation). Stage 1 is now a DEBUG-only presenter toggle, plus the automatic no-Metal fallback. - Stage-2 pixel-perfect: render the drawable at the decoded size (shader stays 1:1 = identity) and let the layer's contentsGravity scale via the system compositor — the same path stage-1's videoGravity used — instead of scaling in-shader. - Loss recovery in both pumps is now a persistent awaitingIDR want, retried until an IDR actually lands, so a keyframe request swallowed by the throttle can't strand a frozen frame; 100 ms keyframe throttle to match the Android path. - Fix "Publishing changes from within view updates": defer the HostStore writes out of the .onChange(of: model.phase) callback. - Move AVAudioSession setActive/setCategory off the main thread (async on a shared serial queue) to stop the UI-stall warning. Controllers - Rumble: capped-exponential backoff when the gamecontrollerd.haptics XPC breaks (-4811) so a transient server interruption self-heals instead of cascading; playsHapticsOnly so a controller engine doesn't join the always-active streaming audio session. - Host cards: iPad pointer "magnet" hover effect; iPhone press scale + light haptic. UI / design - Ship Geist (SIL OFL 1.1) as the app font (bundled OTFs + registration), with the license surfaced in Acknowledgements. - Restructure iOS/iPadOS Settings into a category NavigationSplitView; resolution wheel with custom-resolution entry; 10-bit HDR toggle in Display. - Industrial host-card redesign (left-aligned, bold, brand monogram tiles). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-29 20:26:10 +02:00
enricobuehler	8c2e245c8b	fix(apple/cursor): disable the client-side cursor (gamescope traps input) ... The client-side cursor positions the host pointer with ABSOLUTE events, but gamescope's input socket (EIS) grants only a relative pointer — the host drops the absolute events (libei.rs: no PointerAbsolute → not emitted), so the pointer never moves and clicks/scroll land on the stuck position. Auto-mode enabled exactly this on gamescope, making all input appear dead until toggled off. Force `cursorVisible = false`, neuter the ⌘⇧C toggle, and hide the now-inert Settings picker. The resolution logic + handlers are kept (commented) for when per-compositor gating (KWin/GNOME/Sway have an absolute pointer) or a synthetic-cursor-over-relative path lands. Relative capture (the working path) is now always used. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 17:14:57 +00:00
enricobuehler	c64816c70a	feat(apple): client-side cursor for gamescope sessions (toggle + shortcut) ... gamescope's PipeWire capture carries no cursor (verified upstream — it never composites the cursor or adds SPA_META_Cursor), so the cursor must be drawn on the client. New macOS "cursor-visible" capture mode: instead of disassociating+hiding the system cursor and sending relative deltas (the game path, unchanged), it keeps the system cursor visible over the stream and sends ABSOLUTE positions (MouseMoveAbs), mapped through the video's aspect-fit (AVMakeRect) to host pixels with the letterbox bars dropped. The visible system cursor IS the client cursor — zero added latency, no double cursor (gamescope draws none), accurate (the client drives the host's absolute mouse). - Default: on iff the session's resolved compositor is gamescope (via the new punktfunk_connection_compositor getter, fc30307). - Settings: "Cursor in stream" → Auto (gamescope) / Always / Never. - Shortcut: ⌘⇧C toggles it live mid-session (re-engages capture so disassociation + abs/rel forwarding swap atomically); shown in the HUD. macOS-only (the visible-cursor mode lives in the macOS StreamView). Verified to compile + link via xcodebuild Release on the Mac; runtime behavior (cursor landing, hover forwarding) to be confirmed live. Rust ABI side committed separately. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 12:07:15 +00:00
enricobuehler	dea749186d	fix(quic/apple): QUIC keep-alive + reconnect input re-engage ... Three native-client bugs isolated against a stock Moonlight client (which stays connected / keeps input working under the same actions): - Connection drops mid-stream: the quinn endpoints (host + client) ran with default transport config, so keep_alive_interval was OFF. Any quiet stretch (no input, audio muted/stalled, a capture hiccup, a mode change) let the idle timer expire and quinn closed the session -> next_au=Closed -> "Session ended". Moonlight's ENet sends keepalive pings; we sent nothing. Add a shared TransportConfig (keep-alive 4s under an explicit 20s idle timeout) to both endpoint::server_from_der and endpoint::client_pinned_with_identity. - Reconnect input dead (macOS): the session-start auto-capture one-shot was consumed even when engageCapture(fromClick:false) was refused (window not key yet at the instant of reconnect), with no retry -> capture stayed off and input never forwarded. Clear the one-shot only on a successful engage, and retry on NSWindow.didBecomeKey. Stays scoped to session start, so it does not resurrect the rejected auto-grab-on-activation behavior. - Reconnect input dead (iOS): wasCapturedOnResign leaked stale state across sessions and the foreground-restore could fire before this session's InputCapture was wired (setForwarding no-ops on nil). Reset it per session in start() and guard the didBecomeActive restore on inputCapture != nil. Validated: cargo build -p punktfunk-core --features quic; swift build; swift test (39 passed, 0 failures); xcframework rebuilt (all 5 slices), no ABI/header drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 23:07:54 +02:00
enricobuehler	9e8135ccec	refactor(apple): code-quality pass — audit fixes + centralized defaults keys ... A 6-agent adversarial audit of the client (11 confirmed of 39 findings, the rest filtered) drove these: - fix: SessionAudio ring buffer — guard a write larger than the ring (would push readIdx past writeIdx and corrupt the buffer; never happens, but guard not corrupt). - fix: CADisplayLink retain cycle (stage-2 presenter) — a weak-target DisplayLinkProxy so the view can deallocate (the link retains its target); stage-2 teardown added to both StreamView/StreamViewController deinits as a safety net. - fix: GamepadFeedback deinit { flag.stop() } — the drain thread holds the connection strongly and self weakly, so an abrupt teardown without stop() would leak it. - refactor: centralize the 12 UserDefaults/@AppStorage key literals (scattered across 8 files) into one DefaultsKey enum — a typo silently splits a setting's reader from its writer. - docs: RumbleRenderer @unchecked Sendable invariant; the HID digit-row table; the stage-2 layer compositing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 16:30:34 +02:00
enricobuehler	7b10714b62	feat(apple): stage-2 presenter — explicit decode + Metal present + glass-to-glass ... Opt-in (Settings -> Presenter; `punktfunk.presenter`, default stage-1). Stage-1's AVSampleBufferDisplayLayer decodes AND presents internally with no per-frame callback, so neither decode nor present can be stamped or hand-paced. Stage-2 takes explicit control: - VideoDecoder: VTDecompressionSession, async output callback stamps decode-completion, session rebuilt on every IDR / format change. Unit-tested (testVideoDecoderAsyncCallbackDeliversPixels). - MetalVideoPresenter: CAMetalLayer + CVMetalTextureCache + a runtime-compiled BT.709 limited-range NV12->RGB shader, present at the next vsync. The CVMetalTextures + pixel buffer are held until the GPU completes. - Stage2Pipeline: pump thread -> decoder -> newest-ready 1-slot ring; the hosting view's display link drains it once per vsync and stamps capture->present (the display-link target time projected into CLOCK_REALTIME). - LatencyMeter gains record(ptsNs:atNs:offsetNs:); the HUD shows a capture->present (glass-to-glass, modulo host render->capture) line, skew-corrected via clockOffsetNs. Measured live ~11 ms p50 vs ~2.2 ms capture->client. - StreamView / StreamViewIOS host the CAMetalLayer as a sublayer + a CADisplayLink (NSView.displayLink on macOS) when stage-2; input capture + HUD unchanged. The session-active gates switch from `pump != nil` to `connection != nil` so capture engages without a StreamPump. Validated: builds macOS/iOS/tvOS; the decode half is unit-tested; the Metal present is live-validated on glass (correct image + the capture->present number). Colorspace is BT.709 SDR for now; 10-bit/HDR + a pacing policy are later. Plan: docs-site/content/docs/apple-stage2-presenter.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 15:29:23 +02:00
enricobuehler	6d3ff37d9e	feat(client): cross-target input handling + LAN mDNS discovery ... Input handling, building on macOS/iOS/tvOS: - macOS recapture after navigating out: engageCapture no longer latches captured=true when the cursor grab is refused mid app-activation (which left a free cursor that no later click could re-grab); cursorCapture.capture() now reports success. + canBecomeKeyView. - iOS/iPadOS recapture: restore the prior capture on didBecomeActive (nothing re-grabbed mouse/keyboard on return before). - iPad indirect pointer (no lock) is forwarded as an absolute MOUSE (move + buttons + scroll via hover / UITouch.indirectPointer), not as touch, with the local cursor visible; GCMouse owns the locked regime, gated so the two never double-send. Adds the MouseMoveAbs wire helper. - Trackpad scroll on iOS (was entirely missing): GCMouse scroll dpad when locked + a scroll-only UIPanGestureRecognizer otherwise. - tvOS: no focusable control during play (a focusable Disconnect button ate the controller's A in the focus engine); Siri Remote Menu disconnects. - Don't leak touch to the host under the TOFU trust prompt (gate on captureEnabled). LAN discovery: HostDiscovery (NWBrowser over _punktfunk._udp, the host's crate::discovery advert) resolves each service to IP:port and parses the TXT (fp advisory, pair, id); an "On this network" section in the grid (tap to save + connect, or pair if required). iOS/tvOS get NSBonjourServices via a merged Config/Info.plist. Integration-tested end to end against a fake NWListener advert. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 14:08:19 +02:00
enricobuehler	dcb2850c7c	fix(apple): drive macOS keyboard from NSEvent (GCKeyboard unreliable) ... macOS GCKeyboard delivery is flaky — the same GameController quirk that killed GCMouse motion (e414ec0). Keyboard input intermittently failed to reach the host (e.g. typing in a gamescope game). Switch the macOS key source to NSEvent, mirroring the mouse fix: - StreamLayerView.keyDown/keyUp map NSEvent.keyCode (Carbon virtual keycode) → Windows VK via the new InputCapture.keyCodeToVK table and forward through InputCapture.sendKey, then consume the event (no beep). - flagsChanged drives InputCapture.handleFlagsChanged, which diffs the raw modifier flags to recover each L/R modifier down/up (modifiers never fire keyDown/keyUp on macOS) and emits the same L/R VKs hidToVK already does. - The macOS GCKeyboard keyChangedHandler is disabled (#if !os(macOS)) so it can't double-send; iOS keeps the GCKeyboard path unchanged. sendKey honors the ⌘⎋ capture-toggle suppressedVK latch and tracks into pressedVKs so releaseAll()/blur flushes anything still held. The emitted VKs are identical to the existing HID path, so the host (vk_to_evdev) needs no change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 18:10:13 +00:00
enricobuehler	e414ec0895	fix(apple): drive macOS mouse motion/buttons from NSEvent; fix iPad pointer lock ... Host-side logs proved the macOS client sent keyboard + scroll but ZERO relative mouse-motion and ZERO button events for an entire session — the user was moving the mouse the whole time. Root cause is client-side: GCMouse's mouseMovedHandler/pressedChangedHandler silently never fired on the live Mac (a documented GameController quirk) while GCKeyboard worked and scroll already rode NSEvent. So motion/buttons were the only input on a GCMouse-only path, and that path was dead. macOS: stop relying on GCMouse for motion/buttons (compiled out with #if !os(macOS)); drive them from a local NSEvent monitor installed only while captured — the same channel scrollWheel already uses successfully. Under CGAssociateMouseAndMouseCursorPosition(false) the mouseMoved/dragged deltaX/deltaY ARE the relative motion (OS-acceleration-applied, exactly what Moonlight's macOS client ships). All four motion event types are covered so motion keeps flowing during a button-held drag; buttons map left/right/middle/X1/X2 through the existing engage-click-suppression + release-on-blur logic. NSEvent deltaY is already screen-space (+y down) so, unlike the GCMouse path, it is NOT negated. iPad: the input failure there was a different cause — GCMouse only delivers relative deltas while the scene holds a true pointer LOCK, which the system grants only to a full-screen, frontmost iPad scene and which UIHostingController doesn't consult for children. Gate prefersPointerLocked to iPad + captured, add childViewControllerForPointerLock so a reparenting container forwards the lock decision to this VC, and log the resolved lock state. Touch remains the unconditional fallback. Adds a PUNKTFUNK_INPUT_DEBUG=1 switch (os.Logger, throttled) so motion/buttons being SENT is verifiable on-device without host-side logs. iOS GCMouse path otherwise unchanged; GCKeyboard unchanged on both. Researched + adversarially reviewed; Swift builds only on a Mac, so this is unverified-compiled here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 12:30:19 +00:00
enricobuehler	e1af4d57c6	feat(apple): iOS/iPadOS client — touch, pointer lock, shared SwiftUI shell ... The whole client now runs on iPadOS/iOS from the same sources, first-lit live in the iPad simulator against the real host at 1280x720@60 (60 fps on the HUD, capture state machine active, mic permission flow shown). - PunktfunkCore.xcframework grows iOS device + universal-simulator slices (BUILD_IOS=1; rustup targets aarch64-apple-ios{,-sim} + x86_64-apple-ios). - The decode pump is extracted into a shared StreamPump (identical IDR re-gate logic on both platforms); the iOS StreamView (StreamViewIOS.swift) has the same name/signature as the macOS one, so ContentView & co. are byte-identical across platforms — hosted in a UIViewController for prefersPointerLocked (the iPadOS cursor capture; see README note 9 for the UIHostingController forwarding caveat). - Touch is always forwarded: per-finger wire ids, coordinates mapped through the aspect-fit letterbox into LIVE host-mode pixels (surface == host mode, identity rescale host-side; follows mid-stream requestMode switches). - InputCapture is cross-platform: GC works the same on iPadOS, ⌘⎋ is detected from the HID stream there; stale-⌘ tracking after focus loss fixed on both platforms (releaseAll now drops the modifier/latch state — a ⌘ released in another app otherwise hijacked Esc forever). - SessionAudio: AVAudioSession on iOS (.playAndRecord + .defaultToSpeaker — without it iPhones route host audio to the EARPIECE; deactivated with notifyOthersOnDeactivation on stop so interrupted background audio resumes); HAL device pinning + the Settings pickers stay macOS-only. - New Punktfunk-iOS app target (shared synchronized sources, generated Info.plist with mic + local-network usage descriptions — QUIC to a LAN host trips local network privacy on real devices — scene manifest + indirect input events for Stage Manager / external displays), shared scheme, macOS min-window frames gated off iOS. For the iPad-on-an-external-screen idea: with multiple scenes + indirect input enabled, Stage Manager iPads can drag the punktfunk window onto the external display and drive the PC with keyboard/mouse/touch. Known gaps (README note 9): the pointer-lock preference isn't consulted through UIHostingController (relative mouse works, the local cursor just stays visible) and AVAudioSession interruptions don't auto-restart audio. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 11:18:25 +02:00
enricobuehler	a730ca8557	fix(apple): scroll from trackpads/Magic Mouse — forward NSEvent scrollWheel, drop GC scroll ... Scroll was wired to GCMouse's scroll dpad, which only fires for plain HID wheel deltas — trackpad and Magic Mouse scrolling are gesture events that never reach GameController, so scrolling was dead on the default Mac setups. The stream view now overrides scrollWheel (while captured the cursor is parked mid-view, so it receives every scroll event) and feeds InputCapture.sendScroll: precise gesture deltas are pixels (~0.1 notch/px, SDL's factor → ×12 for WHEEL_DELTA(120)), classic wheels are lines (×120), fractional remainders accumulate, and the GC scroll handler is gone so wheel mice can't double-deliver. Signs pass through as-is, preserving the local (natural-)scrolling preference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 23:17:23 +02:00
enricobuehler	a4eacabecd	feat(apple): explicit input-capture state machine — no more cursor grabs on window chrome ... Capture used to engage whenever the app became active, so the click that activates the window — on the title bar (a drag) or a resize edge — got the cursor warped away mid-gesture, and raw deltas kept streaming to the host while the user fought the window. Reworked Moonlight-style, with capture as a deliberate, reversible state owned by StreamLayerView: - Engage: automatically once when the stream starts / trust is confirmed (one-shot, can never fire surprisingly later), or by clicking into the video (that click's press/release are suppressed toward the host; acceptsFirstMouse makes it one click from another app). NEVER on app re-activation. - Release: ⌘⎋ (toggles, key-window-scoped), focus loss — now including same-app window switches (⌘, / ⌘N / ⌘M resign key without resigning the app; previously the new window inherited a hidden frozen cursor and its typing was double-delivered to the host) — and disconnect. - While released: nothing is forwarded (InputCapture.forwarding gates the GC handlers; held keys/buttons are flushed host-side so nothing sticks), the cursor is free, and the HUD (now showing the capture state) is clickable. - The no-beep behavior moved from the NSEvent monitor to first-responder key consumption — swallowing at the monitor risked starving GC's own delivery (the "input broken altogether" report). The monitor now only intercepts ⌘⎋. - Adversarial-review fixes: a second session preempts the previous one cleanly instead of leaving it captured with dead GC handlers (onPreempted); the engage click's suppression latch can't outlive the click (mouseUp backstop); ⌘⎋'s physical Esc can't type into the host in either toggle direction (suppressedVK latch + Esc-while-⌘ guard); capture callbacks defer out of the SwiftUI update pass. Validated live against the box: 16185 input datagrams injected during a captured session (gamescope EIS), title-bar drag/resize free while released, and visible cursor + typing on a streamed KWin desktop, all user-confirmed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 22:42:44 +02:00
enricobuehler	5e77731da0	feat: hosts grid + trust-on-first-use UX + settings pane ... The app grows from a dev connect form into a real client shell: - Home is a grid of saved hosts (UserDefaults-persisted; context menu: Remove / Forget Identity), "+" in the toolbar opens the add-host sheet, the stream mode moved into Settings (⌘, / gear) — native resolution stays the only mode, no scaling. - Trust is now explicit: the protocol always supported certificate pinning, but the app passed no pin and discarded the observed fingerprint — silently trusting any host. First connect now shows the host's SHA-256 fingerprint (compare with the "clients pin this fingerprint" line in the host log) over the live-but-blurred stream; the stream must pump immediately (the opening IDR is the only guaranteed one), so StreamView gains a capturesCursor switch to keep the cursor free while the prompt needs clicking, and input capture starts only after confirmation. Trusting pins the fingerprint per host; a changed host identity then refuses to connect. - PUNKTFUNK_AUTOCONNECT keeps working (auto-trusts, doesn't touch the saved hosts). Host→client authorization (pairing PIN) remains a punktfunk-core roadmap item — the host still accepts any client that can reach its port. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 16:15:37 +02:00
enricobuehler	9aa2d71f49	fix: hide + freeze the local cursor while streaming ... The host renders its own cursor from our raw deltas, so the local macOS cursor both stays visible and drifts away from the remote one — and it can wander out of the window, where a click focuses another app. While the stream has focus, do what Moonlight does: warp the cursor mid-view, disconnect it from mouse movement (CGAssociateMouseAndMouseCursorPosition(false) — GCMouse still delivers raw HID deltas), and hide it. Released on app deactivation (Cmd+Tab is the escape hatch), view teardown, and disconnect; re-captured when the stream regains focus. The HUD's Disconnect gains ⌘D since a hidden, frozen cursor can't click it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 15:38:03 +02:00
enricobuehler	bfd64ce871	rename: lumen → punktfunk, everywhere ... Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 13:11:59 +00:00