25 Commits

Author	SHA1	Message	Date
enricobuehler	4be993df87	fix(apple/stage2): disable layer vsync wait to kill fullscreen stutter ... The experimental stage-2 presenter (CAMetalLayer + display link) stuttered badly in fullscreen but ran fine windowed. render() runs on the display-link / MAIN thread and calls layer.nextDrawable(), which blocks that thread until a drawable frees. With the layer's own displaySyncEnabled left on (default), present also waits for the hardware vsync, so the block serializes the main thread to the display — windowed, the WindowServer's looser compositing hides it; fullscreen's tighter, more-direct path exposes it as judder. (Apple dev-forum guidance: displaySync off measurably reduces nextDrawable() blocking.) - displaySyncEnabled = false (macOS-only): the display link is already the per- vsync pacing source, so the layer's redundant vsync wait only adds the stall. - maximumDrawableCount = 3 (explicit): more in-flight headroom before nextDrawable() has to block on the main thread. Swift-only (no core/ABI change → no xcframework rebuild). Validated: swift build; swift test (39 passed, 0 failures). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-13 01:07:57 +02:00
enricobuehler	c56b1b455a	feat(punktfunk/1): request-IDR recovery for a wedged client decode ... Fixes the intermittent first-connect freeze. The host streams infinite GOP — one opening IDR, then P-frames only (recovery keyframes just on loss) — so when the client's decoder wedges on the cold first session (a lost/corrupt opening IDR, a bad early P-frame) the picture stays frozen until the far-off next keyframe. The client had no way to ask for one; now it does. Add a RequestKeyframe control message (client -> host, reliable control stream), mirroring Reconfigure: - core: quic.rs RequestKeyframe (type 0x03) + roundtrip test; client.rs CtrlRequest::Keyframe + NativeClient::request_keyframe; abi.rs punktfunk_connection_request_keyframe (header regenerated). - host: m3.rs decodes it in the control loop and signals the encode loop, which coalesces a burst and calls enc.request_keyframe() — wiring the existing NvencEncoder hook (force_kf -> next frame pict_type=I), the same recovery the GameStream path already had via force_idr. - apple: PunktfunkConnection.requestKeyframe(); StreamPump (stage-1) requests on layer.status==.failed; Stage2Pipeline (stage-2) on a sync submit failure and on the async decode-error callback via a thread-safe KeyframeRecovery. All throttled to <=1/250ms (the decode stays wedged for several frames until the IDR lands, so per-frame requests would flood the control stream). Self-healing: a lost recovery IDR is re-requested after the throttle; the host coalesces bursts into a single IDR. Validated: cargo fmt + clippy clean; core + host test suites green (incl. new request_keyframe_roundtrip); swift build + test (39 passed); xcframework rebuilt (all 5 slices), header regenerated with no unrelated drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-13 00:48:24 +02:00
enricobuehler	dea749186d	fix(quic/apple): QUIC keep-alive + reconnect input re-engage ... Three native-client bugs isolated against a stock Moonlight client (which stays connected / keeps input working under the same actions): - Connection drops mid-stream: the quinn endpoints (host + client) ran with default transport config, so keep_alive_interval was OFF. Any quiet stretch (no input, audio muted/stalled, a capture hiccup, a mode change) let the idle timer expire and quinn closed the session -> next_au=Closed -> "Session ended". Moonlight's ENet sends keepalive pings; we sent nothing. Add a shared TransportConfig (keep-alive 4s under an explicit 20s idle timeout) to both endpoint::server_from_der and endpoint::client_pinned_with_identity. - Reconnect input dead (macOS): the session-start auto-capture one-shot was consumed even when engageCapture(fromClick:false) was refused (window not key yet at the instant of reconnect), with no retry -> capture stayed off and input never forwarded. Clear the one-shot only on a successful engage, and retry on NSWindow.didBecomeKey. Stays scoped to session start, so it does not resurrect the rejected auto-grab-on-activation behavior. - Reconnect input dead (iOS): wasCapturedOnResign leaked stale state across sessions and the foreground-restore could fire before this session's InputCapture was wired (setForwarding no-ops on nil). Reset it per session in start() and guard the didBecomeActive restore on inputCapture != nil. Validated: cargo build -p punktfunk-core --features quic; swift build; swift test (39 passed, 0 failures); xcframework rebuilt (all 5 slices), no ABI/header drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 23:07:54 +02:00
enricobuehler	9e8135ccec	refactor(apple): code-quality pass — audit fixes + centralized defaults keys ... A 6-agent adversarial audit of the client (11 confirmed of 39 findings, the rest filtered) drove these: - fix: SessionAudio ring buffer — guard a write larger than the ring (would push readIdx past writeIdx and corrupt the buffer; never happens, but guard not corrupt). - fix: CADisplayLink retain cycle (stage-2 presenter) — a weak-target DisplayLinkProxy so the view can deallocate (the link retains its target); stage-2 teardown added to both StreamView/StreamViewController deinits as a safety net. - fix: GamepadFeedback deinit { flag.stop() } — the drain thread holds the connection strongly and self weakly, so an abrupt teardown without stop() would leak it. - refactor: centralize the 12 UserDefaults/@AppStorage key literals (scattered across 8 files) into one DefaultsKey enum — a typo silently splits a setting's reader from its writer. - docs: RumbleRenderer @unchecked Sendable invariant; the HID digit-row table; the stage-2 layer compositing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 16:30:34 +02:00
enricobuehler	7b10714b62	feat(apple): stage-2 presenter — explicit decode + Metal present + glass-to-glass ... Opt-in (Settings -> Presenter; `punktfunk.presenter`, default stage-1). Stage-1's AVSampleBufferDisplayLayer decodes AND presents internally with no per-frame callback, so neither decode nor present can be stamped or hand-paced. Stage-2 takes explicit control: - VideoDecoder: VTDecompressionSession, async output callback stamps decode-completion, session rebuilt on every IDR / format change. Unit-tested (testVideoDecoderAsyncCallbackDeliversPixels). - MetalVideoPresenter: CAMetalLayer + CVMetalTextureCache + a runtime-compiled BT.709 limited-range NV12->RGB shader, present at the next vsync. The CVMetalTextures + pixel buffer are held until the GPU completes. - Stage2Pipeline: pump thread -> decoder -> newest-ready 1-slot ring; the hosting view's display link drains it once per vsync and stamps capture->present (the display-link target time projected into CLOCK_REALTIME). - LatencyMeter gains record(ptsNs:atNs:offsetNs:); the HUD shows a capture->present (glass-to-glass, modulo host render->capture) line, skew-corrected via clockOffsetNs. Measured live ~11 ms p50 vs ~2.2 ms capture->client. - StreamView / StreamViewIOS host the CAMetalLayer as a sublayer + a CADisplayLink (NSView.displayLink on macOS) when stage-2; input capture + HUD unchanged. The session-active gates switch from `pump != nil` to `connection != nil` so capture engages without a StreamPump. Validated: builds macOS/iOS/tvOS; the decode half is unit-tested; the Metal present is live-validated on glass (correct image + the capture->present number). Colorspace is BT.709 SDR for now; 10-bit/HDR + a pacing policy are later. Plan: docs-site/content/docs/apple-stage2-presenter.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 15:29:23 +02:00
enricobuehler	6d3ff37d9e	feat(client): cross-target input handling + LAN mDNS discovery ... Input handling, building on macOS/iOS/tvOS: - macOS recapture after navigating out: engageCapture no longer latches captured=true when the cursor grab is refused mid app-activation (which left a free cursor that no later click could re-grab); cursorCapture.capture() now reports success. + canBecomeKeyView. - iOS/iPadOS recapture: restore the prior capture on didBecomeActive (nothing re-grabbed mouse/keyboard on return before). - iPad indirect pointer (no lock) is forwarded as an absolute MOUSE (move + buttons + scroll via hover / UITouch.indirectPointer), not as touch, with the local cursor visible; GCMouse owns the locked regime, gated so the two never double-send. Adds the MouseMoveAbs wire helper. - Trackpad scroll on iOS (was entirely missing): GCMouse scroll dpad when locked + a scroll-only UIPanGestureRecognizer otherwise. - tvOS: no focusable control during play (a focusable Disconnect button ate the controller's A in the focus engine); Siri Remote Menu disconnects. - Don't leak touch to the host under the TOFU trust prompt (gate on captureEnabled). LAN discovery: HostDiscovery (NWBrowser over _punktfunk._udp, the host's crate::discovery advert) resolves each service to IP:port and parses the TXT (fp advisory, pair, id); an "On this network" section in the grid (tap to save + connect, or pair if required). iOS/tvOS get NSBonjourServices via a merged Config/Info.plist. Integration-tested end to end against a fake NWListener advert. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 14:08:19 +02:00
enricobuehler	6b4de5d738	feat(client/speedtest): request the host's full 3 Gbps probe ceiling ... The Apple speed test asked for only 400 Mbps, capping the measured throughput there and hiding the link's real headroom. Request the host's full MAX_PROBE_KBPS (3 Gbps) instead, and raise the recommended-bitrate clamp from 500 Mbps to the host's 2 Gbps session ceiling so a fast measurement yields a usable recommendation. Also fix the stale caps left when the host clamps were raised (b8a33e2): the resolved-bitrate range and the probe doc comments (abi.rs, client.rs, regenerated header), plus the section 9 roadmap copy, now read 3 Gbps probe / 2 Gbps session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 14:08:19 +02:00
enricobuehler	e04328f086	feat(apple): capture->client latency HUD (skew-corrected) via the connect offset ... The Apple client now consumes the connector's clock offset. PunktfunkConnection reads punktfunk_connection_clock_offset_ns into clockOffsetNs at connect; a new LatencyMeter (PunktfunkKit, NSLock + percentiles, mirrors FrameMeter) records each AU's capture->client-receipt latency = now(CLOCK_REALTIME) + offset - pts_ns, and SessionModel drains p50/p95 into the macOS HUD ("capture->client N/N ms p50/p95", "(same-host)" when the host didn't answer the skew handshake). Wired at the existing onFrame hook in ContentView — additive, no change to the decode/present path. Unit test for the meter (percentiles, skew flag, absurd-value guard). This is the first cross-machine latency the real Apple client reports. SCOPE: stage-1 AVSampleBufferDisplayLayer decodes+presents compressed samples internally with no per-frame callback, so this excludes decode+present; true decode->present needs the stage-2 presenter (VTDecompressionSession + CAMetalLayer). Rebuild PunktfunkCore.xcframework (for the new C getter) before swift build/test on a Mac. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 11:58:54 +00:00
enricobuehler	dcb2850c7c	fix(apple): drive macOS keyboard from NSEvent (GCKeyboard unreliable) ... macOS GCKeyboard delivery is flaky — the same GameController quirk that killed GCMouse motion (e414ec0). Keyboard input intermittently failed to reach the host (e.g. typing in a gamescope game). Switch the macOS key source to NSEvent, mirroring the mouse fix: - StreamLayerView.keyDown/keyUp map NSEvent.keyCode (Carbon virtual keycode) → Windows VK via the new InputCapture.keyCodeToVK table and forward through InputCapture.sendKey, then consume the event (no beep). - flagsChanged drives InputCapture.handleFlagsChanged, which diffs the raw modifier flags to recover each L/R modifier down/up (modifiers never fire keyDown/keyUp on macOS) and emits the same L/R VKs hidToVK already does. - The macOS GCKeyboard keyChangedHandler is disabled (#if !os(macOS)) so it can't double-send; iOS keeps the GCKeyboard path unchanged. sendKey honors the ⌘⎋ capture-toggle suppressedVK latch and tracks into pressedVKs so releaseAll()/blur flushes anything still held. The emitted VKs are identical to the existing HID path, so the host (vk_to_evdev) needs no change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 18:10:13 +00:00
enricobuehler	1d605fb781	feat(gamepad): controller discovery + client-negotiated pad type + rich DualSense end to end ... The Apple client grows full gamepad support and punktfunk/1 learns to negotiate the virtual pad type: - Protocol: Hello carries a GamepadPref byte (offset 21, the same trailing-byte back-compat pattern as the compositor; echoed resolved in Welcome at 54). Host precedence: explicit client choice > PUNKTFUNK_GAMEPAD env > Xbox 360, DualSense (UHID) only where available. ABI: punktfunk_connect_ex2 + punktfunk_connection_gamepad (connect_ex delegates; ABI_VERSION stays 2 — the trailing byte IS the compat mechanism). punktfunk-client-rs gets --gamepad. - Swift client: GamepadManager (app-lifetime discovery + selection — Settings lists every controller with capabilities/battery/"In use"; exactly ONE pad forwards as pad 0, auto = most recently connected, or pinned), GamepadCapture (snapshot-diff button/axis events, DualSense touchpad + ~250 Hz motion on the rich-input plane, held state released on switch/deactivate/stop), GamepadFeedback (rumble → CoreHaptics per-handle engines; lightbar → GCDeviceLight; player LEDs → playerIndex; adaptive-trigger blocks → the table-driven DualSenseTriggerEffect parser → GCDualSenseAdaptiveTrigger, exact for the 10-zone positional modes). The pad type auto-resolves from the physical controller at connect time, user-overridable in Settings. - Host DualSense fixes surfaced by adversarial review against hid-playstation / SDL / Nielk1 ground truth: input-report sensor/touch offsets were off by one (the kernel read garbage motion + phantom touches), the L2/R2 trigger blocks were swapped (the report is right-trigger-first), feedback now gates on the report's valid-flags (a plain rumble write no longer blanks lightbar/ triggers), and the touchpad rescale clamps to the advertised ABS_MT extents. - Tests: Hello/Welcome trailing-byte back-compat, pick_gamepad precedence, byte-exact input-report layout, valid-flag gating, per-mode trigger-parser table (incl. packed 3-bit zones), wire conversions, and a scripted loopback feedback burst (PUNKTFUNK_TEST_FEEDBACK=1) asserted through the xcframework on the rumble + HID-output planes. Validated: cargo test/clippy/fmt green on macOS + Linux (61 host tests), swift build/test green, test-loopback.sh green, tvOS/iOS targets compile. DualSense motion sign/scale is derived from the calibration blob, not yet live-verified (constants isolated in GamepadWire). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 16:28:33 +02:00
enricobuehler	e414ec0895	fix(apple): drive macOS mouse motion/buttons from NSEvent; fix iPad pointer lock ... Host-side logs proved the macOS client sent keyboard + scroll but ZERO relative mouse-motion and ZERO button events for an entire session — the user was moving the mouse the whole time. Root cause is client-side: GCMouse's mouseMovedHandler/pressedChangedHandler silently never fired on the live Mac (a documented GameController quirk) while GCKeyboard worked and scroll already rode NSEvent. So motion/buttons were the only input on a GCMouse-only path, and that path was dead. macOS: stop relying on GCMouse for motion/buttons (compiled out with #if !os(macOS)); drive them from a local NSEvent monitor installed only while captured — the same channel scrollWheel already uses successfully. Under CGAssociateMouseAndMouseCursorPosition(false) the mouseMoved/dragged deltaX/deltaY ARE the relative motion (OS-acceleration-applied, exactly what Moonlight's macOS client ships). All four motion event types are covered so motion keeps flowing during a button-held drag; buttons map left/right/middle/X1/X2 through the existing engage-click-suppression + release-on-blur logic. NSEvent deltaY is already screen-space (+y down) so, unlike the GCMouse path, it is NOT negated. iPad: the input failure there was a different cause — GCMouse only delivers relative deltas while the scene holds a true pointer LOCK, which the system grants only to a full-screen, frontmost iPad scene and which UIHostingController doesn't consult for children. Gate prefersPointerLocked to iPad + captured, add childViewControllerForPointerLock so a reparenting container forwards the lock decision to this VC, and log the resolved lock state. Touch remains the unconditional fallback. Adds a PUNKTFUNK_INPUT_DEBUG=1 switch (os.Logger, throttled) so motion/buttons being SENT is verifiable on-device without host-side logs. iOS GCMouse path otherwise unchanged; GCKeyboard unchanged on both. Researched + adversarially reviewed; Swift builds only on a Mac, so this is unverified-compiled here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 12:30:19 +00:00
enricobuehler	bfd8c7be93	feat(apple): tvOS client — third app target, first-lit in the Apple TV simulator ... The same app now runs on tvOS (target Punktfunk-tvOS, bundle io.unom.punktfunk.tvos), validated live against the box: vkcube at 1280x720@60, 60 fps in the Apple TV 4K simulator, glass HUD with a focusable Disconnect button. - PunktfunkCore.xcframework grows tvOS device + universal-simulator slices. These are TIER-3 Rust targets (no prebuilt std): BUILD_TVOS=1 builds them with nightly and -Zbuild-std from rust-src — the full quic stack (quinn/rustls-ring/tokio) compiles for tvOS unchanged. - The UIKit stream view covers iOS AND tvOS, with pointer interaction, pointer lock, touch forwarding and InputCapture gated to iOS — tvOS is view-only until gamepad capture lands (the natural tvOS input). - SessionAudio on tvOS: .playback session, no mic (no app-accessible microphone). - App chrome gates: keyboardShortcut/textSelection/controlSize/statusBarHidden are iOS/macOS-only; host cards use the focus-native .card button style on tvOS; the Audio settings section hides (system-routed); mode seeding works from the TV screen (1920x1080@60). - Package platforms += .tvOS(.v17); new Xcode target + shared scheme (TARGETED_DEVICE_FAMILY 3, local-network usage description included). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 13:10:40 +02:00
enricobuehler	154da2dc58	fix(apple/iOS): immersive streaming — edge-to-edge, no status bar, hidden cursor, native default mode ... Streaming on iPad left the status bar up and the video boxed inside the safe areas, on top of a 16:9 default mode letterboxing on the 4:3 screen, with the iPadOS cursor hovering over the video. The session view is now immersive on iOS: - .ignoresSafeArea + .statusBarHidden + .persistentSystemOverlays(.hidden) for the session only (home gets its chrome back on disconnect). - First run seeds the stream mode from the device's native screen (UIScreen.nativeBounds + maximumFramesPerSecond) instead of 1920×1080 — verified live: a fresh install negotiated the iPad's 2752×2064 with the host. macOS keeps the 1080p default (a desktop window is not the screen). - The iPadOS cursor hides while over the video (UIPointerInteraction .hidden(), re-resolved on capture toggles) — the host renders its own cursor from our deltas; true pointer lock through UIHostingController remains the documented gap. Found along the way (host-side, not fixed here): at very high modes a keyframe burst can fill the UDP send buffer and m3 treats the sendmmsg WouldBlock as fatal ("session ended with error: submit_frame: WouldBlock") instead of backpressuring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 12:44:37 +02:00
enricobuehler	e1af4d57c6	feat(apple): iOS/iPadOS client — touch, pointer lock, shared SwiftUI shell ... The whole client now runs on iPadOS/iOS from the same sources, first-lit live in the iPad simulator against the real host at 1280x720@60 (60 fps on the HUD, capture state machine active, mic permission flow shown). - PunktfunkCore.xcframework grows iOS device + universal-simulator slices (BUILD_IOS=1; rustup targets aarch64-apple-ios{,-sim} + x86_64-apple-ios). - The decode pump is extracted into a shared StreamPump (identical IDR re-gate logic on both platforms); the iOS StreamView (StreamViewIOS.swift) has the same name/signature as the macOS one, so ContentView & co. are byte-identical across platforms — hosted in a UIViewController for prefersPointerLocked (the iPadOS cursor capture; see README note 9 for the UIHostingController forwarding caveat). - Touch is always forwarded: per-finger wire ids, coordinates mapped through the aspect-fit letterbox into LIVE host-mode pixels (surface == host mode, identity rescale host-side; follows mid-stream requestMode switches). - InputCapture is cross-platform: GC works the same on iPadOS, ⌘⎋ is detected from the HID stream there; stale-⌘ tracking after focus loss fixed on both platforms (releaseAll now drops the modifier/latch state — a ⌘ released in another app otherwise hijacked Esc forever). - SessionAudio: AVAudioSession on iOS (.playAndRecord + .defaultToSpeaker — without it iPhones route host audio to the EARPIECE; deactivated with notifyOthersOnDeactivation on stop so interrupted background audio resumes); HAL device pinning + the Settings pickers stay macOS-only. - New Punktfunk-iOS app target (shared synchronized sources, generated Info.plist with mic + local-network usage descriptions — QUIC to a LAN host trips local network privacy on real devices — scene manifest + indirect input events for Stage Manager / external displays), shared scheme, macOS min-window frames gated off iOS. For the iPad-on-an-external-screen idea: with multiple scenes + indirect input enabled, Stage Manager iPads can drag the punktfunk window onto the external display and drive the PC with keyboard/mouse/touch. Known gaps (README note 9): the pointer-lock preference isn't consulted through UIHostingController (relative mouse works, the local cursor just stays visible) and AVAudioSession interruptions don't auto-restart audio. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 11:18:25 +02:00
enricobuehler	b26f138699	feat(apple): session audio — host playback + mic uplink, device pickers in Settings ... Both directions of the audio plane, on CoreAudio's built-in Opus codec (kAudioFormatOpus — no bundled libopus; OpusCodec.swift, round trip unit-tested): - Playback: a drain thread pulls nextAudio() packets, decodes, and writes a priming jitter ring feeding an AVAudioSourceNode (~20 ms prefill, adaptive to the device's render quantum so large-buffer devices don't oscillate prime/dropout; a high-water clamp sheds stall backlog so one network hiccup can't permanently lag audio behind video; underrun re-primes — one dip, not sustained crackle). - Mic: a second engine taps the input device, resamples to 48 kHz stereo, Opus-encodes 20 ms chunks and sendMic()s them into the host's virtual PipeWire source. Permission via AVCaptureDevice (NSMicrophoneUsageDescription added to the Xcode target). - Settings: Speaker + Microphone pickers (CoreAudio HAL enumeration, persisted by device UID — "System default" leaves the engine unpinned so it follows macOS device changes) and a "Send microphone" toggle (default on). Applies from the next session. - Audio starts with streaming, never during the trust prompt (no host sound — and no mic uplink — before the user trusted the host); teardown stops audio before close(). Adversarial-review fixes baked in: stop() and the dangling mic-permission callback share one lock+flag protocol (no hot mic with no owner), the connect-success handler bails when the attempt was abandoned mid-handshake (no session/mic for a dead window), SessionAudio gets a deinit backstop (a dropped instance can't pin the connection via its drain thread), and the render scratch buffer is block-owned (was leaked per session). Verified live against the box: remote test decodes 100 host Opus packets to PCM and the host opens its virtual mic on the first uplinked frame ("punktfunk/1 virtual mic ready"); on-glass session runs with both engines up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 09:39:15 +02:00
enricobuehler	3a51551f97	feat(apple): mic uplink + touch events in PunktfunkKit ... Adopts the new ABI surface (still v2, additive): - PunktfunkConnection.sendMic(_:seq:ptsNs:) — Opus mic frames (48 kHz) to the host's virtual PipeWire source; enqueue-only, empty data = DTX silence. Wiring the actual Mac microphone (AVAudioEngine input → Opus) into the app is the follow-up, alongside audio playback (README note 5). - PunktfunkInputEvent.touchDown/touchMove/touchUp — absolute pixels + surface size in flags, host injects via libei ei_touchscreen. Built for the iOS variant; nothing on macOS emits them yet. - Loopback round trip now also sends touch events and mic frames (incl. a DTX frame) through the wrapper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 09:08:04 +02:00
enricobuehler	a730ca8557	fix(apple): scroll from trackpads/Magic Mouse — forward NSEvent scrollWheel, drop GC scroll ... Scroll was wired to GCMouse's scroll dpad, which only fires for plain HID wheel deltas — trackpad and Magic Mouse scrolling are gesture events that never reach GameController, so scrolling was dead on the default Mac setups. The stream view now overrides scrollWheel (while captured the cursor is parked mid-view, so it receives every scroll event) and feeds InputCapture.sendScroll: precise gesture deltas are pixels (~0.1 notch/px, SDL's factor → ×12 for WHEEL_DELTA(120)), classic wheels are lines (×120), fractional remainders accumulate, and the GC scroll handler is gone so wheel mice can't double-deliver. Signs pass through as-is, preserving the local (natural-)scrolling preference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 23:17:23 +02:00
enricobuehler	a9d1c16067	feat(apple): client-selectable compositor in the macOS client ... Adopts punktfunk_connect_ex from the compositor-selection batch: a Compositor enum on PunktfunkConnection (auto/kwin/wlroots/mutter/gamescope, with the host's name aliases for env parsing), a "Host compositor" picker in Settings (default Automatic — a concrete choice is honored only if that backend is available host-side), and PUNKTFUNK_COMPOSITOR / PUNKTFUNK_REMOTE_COMPOSITOR pass-throughs for the autoconnect dev hook and the remote first-light test. The wire change is backward-compatible (optional trailing byte), so no behavior changes at the default. Validated live against the box: host with no compositor env (auto-detect = KWin) logged "honoring client compositor request compositor=gamescope" and streamed 60/60 decoded frames from the spawned gamescope. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 22:51:42 +02:00
enricobuehler	a4eacabecd	feat(apple): explicit input-capture state machine — no more cursor grabs on window chrome ... Capture used to engage whenever the app became active, so the click that activates the window — on the title bar (a drag) or a resize edge — got the cursor warped away mid-gesture, and raw deltas kept streaming to the host while the user fought the window. Reworked Moonlight-style, with capture as a deliberate, reversible state owned by StreamLayerView: - Engage: automatically once when the stream starts / trust is confirmed (one-shot, can never fire surprisingly later), or by clicking into the video (that click's press/release are suppressed toward the host; acceptsFirstMouse makes it one click from another app). NEVER on app re-activation. - Release: ⌘⎋ (toggles, key-window-scoped), focus loss — now including same-app window switches (⌘, / ⌘N / ⌘M resign key without resigning the app; previously the new window inherited a hidden frozen cursor and its typing was double-delivered to the host) — and disconnect. - While released: nothing is forwarded (InputCapture.forwarding gates the GC handlers; held keys/buttons are flushed host-side so nothing sticks), the cursor is free, and the HUD (now showing the capture state) is clickable. - The no-beep behavior moved from the NSEvent monitor to first-responder key consumption — swallowing at the monitor risked starving GC's own delivery (the "input broken altogether" report). The monitor now only intercepts ⌘⎋. - Adversarial-review fixes: a second session preempts the previous one cleanly instead of leaving it captured with dead GC handlers (onPreempted); the engage click's suppression latch can't outlive the click (mouseUp backstop); ⌘⎋'s physical Esc can't type into the host in either toggle direction (suppressedVK latch + Esc-while-⌘ guard); capture callbacks defer out of the SwiftUI update pass. Validated live against the box: 16185 input datagrams injected during a captured session (gamescope EIS), title-bar drag/resize free while released, and visible cursor + typing on a streamed KWin desktop, all user-confirmed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 22:42:44 +02:00
enricobuehler	acf44eed5f	fix(apple): stop the macOS beep on every keystroke while streaming ... GCKeyboard reads the HID state directly, so the key NSEvents kept traveling the responder chain unhandled — and an unhandled keyDown makes NSWindow play the "invalid input" sound on every keystroke. InputCapture now installs a local event monitor for its lifetime that swallows key events, except ⌘-combos, which still reach the local app (the HUD's ⌘D disconnect, ⌘Q) in addition to the host. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 22:03:56 +02:00
enricobuehler	ff4fe197be	fix(punktfunk/1): adversarial-review fixes — SPAKE2 pairing, renegotiation hardening, +more ... Triaged the multi-agent review of the renegotiation + pairing + Sway + AV1/surround batch (1 critical, 11 major/minor confirmed). Fixes: CRITICAL — PIN pairing was offline-brute-forceable. The HMAC-of-PIN proof let an active MITM who terminates the TOFU ceremony recover the 4-digit PIN by offline dictionary search (all other inputs observable) and forge a correctly-bound proof. Replaced with **SPAKE2** (balanced PAKE, `spake2` crate) + key-confirmation MACs, binding both cert fingerprints as the SPAKE2 identities: an attacker gets exactly ONE online guess, no offline search, and mismatched cert views (a real MITM) never reach a shared key. Also reworked the UX to an "arming PIN" — one PIN per arming window shown at host startup (the SPAKE2 client needs the PIN to build its first message, so it can't be minted per-connection). Validated live: wrong PIN rejected in 0.1s, right PIN pairs + persists + the paired identity streams. Pairing hardening: `--allow-pairing`/`--require-pairing` must arm pairing (default rejects unsolicited ceremonies); per-host cooldown bounds online guessing; the client flushes its CONNECTION_CLOSE so a refused ceremony can't wedge the sequential host for the full timeout; atomic (temp+rename) paired-store writes. Protocol: control/pairing messages use a distinct CTL_MAGIC (PKFc) — fully disjoint from the positional Hello namespace (a future abi_version can't be misparsed as a control message); all typed decodes are length-exact. ABI_VERSION → 2 (punktfunk_connect signature gained the identity params; header regenerated). Renegotiation: drain the reconfig channel to the NEWEST mode (one rebuild, not one per stale step); validate refresh_hz; build the new pipeline BEFORE dropping the old so a rebuild failure keeps the session on its current mode instead of killing it. GameStream: packetDuration snaps to {5,10} (an in-between value isn't a legal Opus frame size and would kill audio). Sway: chooser file moved to $XDG_RUNTIME_DIR (was a fixed world-writable /tmp path — DoS / capture-misdirection by another local user). Swift: fixed two compile breakers in the new pairing/identity APIs (Int32 status .rawValue, UInt cap cast). New SPAKE2 + namespace-disjointness + pairing-roundtrip unit tests; the in-process pairing test now also exercises the arming PIN + cooldown. 114 tests green, clippy -D warnings clean (both feature sets), fmt, C-ABI harness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 16:26:48 +00:00
enricobuehler	4d26ac5c85	feat: punktfunk/1 — mid-stream mode renegotiation + PIN pairing ceremony ... Renegotiation (no reconnect on resize): the handshake bi-stream stays open; the client sends Reconfigure{mode} (typed post-handshake message), the host validates + acks Reconfigured and rebuilds capture/encoder/virtual output at the new mode while the data plane (keys, ports, FEC) runs untouched — the first new-mode AU is an IDR with in-band parameter sets. NativeClient::request_mode / punktfunk_connection_request_mode; mode() reflects the active mode. Validated live on KWin: one continuous stream, 225 frames @1280x720 then 395 @1920x1080, ~90 ms pipeline rebuild (ffprobe shows both resolutions). PIN pairing (mutual trust, kills TOFU MITM): clients get persistent self-signed identities presented via QUIC client auth (generate_identity / client auth offered but optional server-side — legacy clients still connect). Ceremony on the control stream: PairRequest{name} → host shows a 4-digit PIN (log) + PairChallenge{salt} → client proves with HMAC-SHA256(PIN‖salt, client_fp‖host_fp) — binding both certs means a MITM can't forward a proof, single attempt per PIN, constant-time compare → PairResult; host persists the fingerprint (~/.config/punktfunk/punktfunk1-paired.json), client pins the host's. m3-host --require-pairing gates sessions on the paired set. NativeClient::pair + punktfunk_pair/punktfunk_generate_identity in the ABI; reference client: --pair PIN --name LABEL + auto-generated persistent identity, --remode for live renegotiation testing. Swift wrapper: ClientIdentity/generateIdentity()/pair(), requestMode()/currentMode(); README handoff updated. Tested: reconfigure/pairing wire roundtrips, C-ABI mode switch ack, full in-process ceremony (wrong PIN → Crypto, anonymous-vs-gate rejection, success → pinned session); live wrong-PIN ceremony against the serving host (PIN logged, proof rejected). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 15:42:29 +00:00
enricobuehler	5e77731da0	feat: hosts grid + trust-on-first-use UX + settings pane ... The app grows from a dev connect form into a real client shell: - Home is a grid of saved hosts (UserDefaults-persisted; context menu: Remove / Forget Identity), "+" in the toolbar opens the add-host sheet, the stream mode moved into Settings (⌘, / gear) — native resolution stays the only mode, no scaling. - Trust is now explicit: the protocol always supported certificate pinning, but the app passed no pin and discarded the observed fingerprint — silently trusting any host. First connect now shows the host's SHA-256 fingerprint (compare with the "clients pin this fingerprint" line in the host log) over the live-but-blurred stream; the stream must pump immediately (the opening IDR is the only guaranteed one), so StreamView gains a capturesCursor switch to keep the cursor free while the prompt needs clicking, and input capture starts only after confirmation. Trusting pins the fingerprint per host; a changed host identity then refuses to connect. - PUNKTFUNK_AUTOCONNECT keeps working (auto-trusts, doesn't touch the saved hosts). Host→client authorization (pairing PIN) remains a punktfunk-core roadmap item — the host still accepts any client that can reach its port. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 16:15:37 +02:00
enricobuehler	9aa2d71f49	fix: hide + freeze the local cursor while streaming ... The host renders its own cursor from our raw deltas, so the local macOS cursor both stays visible and drifts away from the remote one — and it can wander out of the window, where a click focuses another app. While the stream has focus, do what Moonlight does: warp the cursor mid-view, disconnect it from mouse movement (CGAssociateMouseAndMouseCursorPosition(false) — GCMouse still delivers raw HID deltas), and hide it. Released on app deactivation (Cmd+Tab is the escape hatch), view teardown, and disconnect; re-captured when the stream regains focus. The HUD's Disconnect gains ⌘D since a hidden, frozen cursor can't click it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 15:38:03 +02:00
enricobuehler	bfd64ce871	rename: lumen → punktfunk, everywhere ... Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 13:11:59 +00:00