14 Commits

Author	SHA1	Message	Date
enricobuehler	118752c136	fix(apple): drive DualSense rumble over raw HID (CoreHaptics is silent on macOS) ... GameController's CHHapticEngine never reaches the DualSense's motors on macOS — its adaptive triggers and lightbar work, but rumble stays silent (a documented platform gap). Drive the motors directly via the DualSense HID output report instead, the way SDL and the Linux hid-playstation driver do — the same report that already rumbles the pad on a Linux host. Confirmed live on macOS. - DualSenseHID (macOS): opens the Sony DualSense via IOHIDManager and writes the USB (0x02, 48 bytes) and Bluetooth (0x31, 78 bytes + CRC32) output reports through IOHIDDeviceSetReport. Allowed under the App Sandbox by the existing device.usb + device.bluetooth entitlements; coexists with GameController (non-seized open). Flags mirror the kernel driver (COMPATIBLE_VIBRATION | HAPTICS_SELECT + COMPATIBLE_VIBRATION2); valid_flag1 = 0 so a rumble report leaves the GameController-managed lightbar / triggers / player LEDs untouched. - RumbleRenderer routes a DualSense to the HID backend and keeps CoreHaptics for every other pad, fixing both live sessions and the test panel (shared renderer). - CoreHaptics path reworked too: bake the target intensity + an explicit sharpness into the continuous event (the dynamic-parameter scaling is silent on controller engines) and tear down outside the inout access to fix a latent exclusivity hazard. Adds a DEBUG-only Settings -> Controllers -> "Test Controller" panel (ControllerTestView + ControllerTester) that shows live input and fires rumble / adaptive triggers / lightbar / player LEDs straight at the pad, with a readout of the active rumble backend ("DualSense HID - USB/Bluetooth"). Used to validate the fix. Tests: DualSenseHIDTests pins the USB/BT report layout and the BT CRC32 (canonical 0xCBF43926 check vector). Debug + release build clean; gamepad suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-22 13:16:41 +02:00
enricobuehler	9c8fa9340c	refactor: drop milestone names + consolidate clients; loss-recovery & rumble fixes ... Two bodies of work in one commit (the rename moved files the fixes also touched). Naming/structure cleanup (pre-launch): - Host modules m3.rs->punktfunk1.rs, m0.rs->spike.rs; CLI m3-host->punktfunk1-host, m0->spike; bare `punktfunk-host` now prints help. Types M3Options/M3Source-> Punktfunk1Options/Punktfunk1Source. - Clients consolidated out of crates/ into clients/: punktfunk-client-rs-> clients/probe (crate punktfunk-probe), client-linux->clients/linux, client-windows->clients/windows, punktfunk-android->clients/android/native (crate punktfunk-client-android; kept [lib] name=punktfunk_android so the JNI contract is unchanged). crates/ now holds only core + host. - Milestone codes M0-M4 purged from code/CLI/CLAUDE.md/README/docs/docs-site, kept only in docs/implementation-plan.md. docs/m2-plan.md-> docs/gamestream-host-plan.md. CI/gradle/flatpak paths updated. Client loss-recovery (video froze and never recovered after a brief drop): - Export punktfunk_connection_frames_dropped through the C ABI (the core already tracked it for the client keyframe-recovery loop; it was never reachable from the ABI clients). Regenerated punktfunk_core.h. - Apple (StreamPump + Stage2Pipeline) and Android (decode.rs) now poll frames_dropped and request a keyframe when it climbs -- the same loss-driven recovery Linux/Windows already had. Under infinite GOP the decoder silently conceals reference-missing frames, so the decode-error trigger rarely fires. Apple rumble robustness (worked then went spotty -- DualSense + Xbox): - Add CHHapticEngine stopped/reset handlers (rebuild on app background / audio interruption / server reset) and drop the permanent `broken` latch on a transient drive failure; latch only when the controller truly has no haptics. - Surface swallowed SDL set_rumble errors on Linux/Windows + diagnostic logging. Verified: cargo build/clippy/fmt --workspace, C-ABI harness, header drift. Not runnable on this box (verify in CI): Gitea workflows, gradle/Android, flatpak, Swift/decky. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-18 21:05:58 +00:00
enricobuehler	1b610d6bf5	feat(apple/library): experimental game-library browser (flagged off) ... Plan step 3 — the Apple client surfaces the host's game library, behind a feature flag (`DefaultsKey.libraryEnabled`, default OFF). Browsing only; launching a chosen title is step 4. - PunktfunkKit `LibraryClient`: Codable GameEntry/Artwork/LaunchSpec mirroring crates/punktfunk-host/src/library.rs, and an async fetch of GET /api/v1/library with a bearer token. Typed LibraryError guides setup (the common case is "needs a --mgmt-token"). `Artwork.posterCandidates` = portrait → header → hero. - `LibraryView`: cross-platform poster grid (LazyVGrid, AsyncImage that walks the art candidates past load failures to a text placeholder), a store badge, and an inline Connection form (mgmt port + token) that surfaces when the API is unreachable / 401 / no token set. Read-only. - StoredHost gains `mgmtPort`/`mgmtToken` (the mgmt API is a distinct port from the data plane and needs a token off-loopback). Both OPTIONAL — synthesized Decodable ignores property defaults but treats a missing Optional as nil, so older saved hosts decode unchanged (a defaulted non-optional would wipe the list). HostStore.setMgmt. - Entry point: a flag-gated "Browse Library…" host-card context action → LibraryView (sheet on macOS/iOS, pushed on tvOS), mirroring the pair/speed-test plumbing. Plus a Settings "Experimental" toggle. Can't compile Swift on the Linux dev box; CI (apple.yml: swift build + swift test on the mac mini) verifies the macOS path. Added LibraryClientTests (decode + art order) for `swift test`. iOS/tvOS-only branches mirror existing patterns. Live-verify on the Mac pending. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 14:28:16 +00:00
enricobuehler	7b10714b62	feat(apple): stage-2 presenter — explicit decode + Metal present + glass-to-glass ... Opt-in (Settings -> Presenter; `punktfunk.presenter`, default stage-1). Stage-1's AVSampleBufferDisplayLayer decodes AND presents internally with no per-frame callback, so neither decode nor present can be stamped or hand-paced. Stage-2 takes explicit control: - VideoDecoder: VTDecompressionSession, async output callback stamps decode-completion, session rebuilt on every IDR / format change. Unit-tested (testVideoDecoderAsyncCallbackDeliversPixels). - MetalVideoPresenter: CAMetalLayer + CVMetalTextureCache + a runtime-compiled BT.709 limited-range NV12->RGB shader, present at the next vsync. The CVMetalTextures + pixel buffer are held until the GPU completes. - Stage2Pipeline: pump thread -> decoder -> newest-ready 1-slot ring; the hosting view's display link drains it once per vsync and stamps capture->present (the display-link target time projected into CLOCK_REALTIME). - LatencyMeter gains record(ptsNs:atNs:offsetNs:); the HUD shows a capture->present (glass-to-glass, modulo host render->capture) line, skew-corrected via clockOffsetNs. Measured live ~11 ms p50 vs ~2.2 ms capture->client. - StreamView / StreamViewIOS host the CAMetalLayer as a sublayer + a CADisplayLink (NSView.displayLink on macOS) when stage-2; input capture + HUD unchanged. The session-active gates switch from `pump != nil` to `connection != nil` so capture engages without a StreamPump. Validated: builds macOS/iOS/tvOS; the decode half is unit-tested; the Metal present is live-validated on glass (correct image + the capture->present number). Colorspace is BT.709 SDR for now; 10-bit/HDR + a pacing policy are later. Plan: docs-site/content/docs/apple-stage2-presenter.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 15:29:23 +02:00
enricobuehler	6d3ff37d9e	feat(client): cross-target input handling + LAN mDNS discovery ... Input handling, building on macOS/iOS/tvOS: - macOS recapture after navigating out: engageCapture no longer latches captured=true when the cursor grab is refused mid app-activation (which left a free cursor that no later click could re-grab); cursorCapture.capture() now reports success. + canBecomeKeyView. - iOS/iPadOS recapture: restore the prior capture on didBecomeActive (nothing re-grabbed mouse/keyboard on return before). - iPad indirect pointer (no lock) is forwarded as an absolute MOUSE (move + buttons + scroll via hover / UITouch.indirectPointer), not as touch, with the local cursor visible; GCMouse owns the locked regime, gated so the two never double-send. Adds the MouseMoveAbs wire helper. - Trackpad scroll on iOS (was entirely missing): GCMouse scroll dpad when locked + a scroll-only UIPanGestureRecognizer otherwise. - tvOS: no focusable control during play (a focusable Disconnect button ate the controller's A in the focus engine); Siri Remote Menu disconnects. - Don't leak touch to the host under the TOFU trust prompt (gate on captureEnabled). LAN discovery: HostDiscovery (NWBrowser over _punktfunk._udp, the host's crate::discovery advert) resolves each service to IP:port and parses the TXT (fp advisory, pair, id); an "On this network" section in the grid (tap to save + connect, or pair if required). iOS/tvOS get NSBonjourServices via a merged Config/Info.plist. Integration-tested end to end against a fake NWListener advert. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 14:08:19 +02:00
enricobuehler	6b4de5d738	feat(client/speedtest): request the host's full 3 Gbps probe ceiling ... The Apple speed test asked for only 400 Mbps, capping the measured throughput there and hiding the link's real headroom. Request the host's full MAX_PROBE_KBPS (3 Gbps) instead, and raise the recommended-bitrate clamp from 500 Mbps to the host's 2 Gbps session ceiling so a fast measurement yields a usable recommendation. Also fix the stale caps left when the host clamps were raised (b8a33e2): the resolved-bitrate range and the probe doc comments (abi.rs, client.rs, regenerated header), plus the section 9 roadmap copy, now read 3 Gbps probe / 2 Gbps session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 14:08:19 +02:00
enricobuehler	e04328f086	feat(apple): capture->client latency HUD (skew-corrected) via the connect offset ... The Apple client now consumes the connector's clock offset. PunktfunkConnection reads punktfunk_connection_clock_offset_ns into clockOffsetNs at connect; a new LatencyMeter (PunktfunkKit, NSLock + percentiles, mirrors FrameMeter) records each AU's capture->client-receipt latency = now(CLOCK_REALTIME) + offset - pts_ns, and SessionModel drains p50/p95 into the macOS HUD ("capture->client N/N ms p50/p95", "(same-host)" when the host didn't answer the skew handshake). Wired at the existing onFrame hook in ContentView — additive, no change to the decode/present path. Unit test for the meter (percentiles, skew flag, absurd-value guard). This is the first cross-machine latency the real Apple client reports. SCOPE: stage-1 AVSampleBufferDisplayLayer decodes+presents compressed samples internally with no per-frame callback, so this excludes decode+present; true decode->present needs the stage-2 presenter (VTDecompressionSession + CAMetalLayer). Rebuild PunktfunkCore.xcframework (for the new C getter) before swift build/test on a Mac. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 11:58:54 +00:00
enricobuehler	1d605fb781	feat(gamepad): controller discovery + client-negotiated pad type + rich DualSense end to end ... The Apple client grows full gamepad support and punktfunk/1 learns to negotiate the virtual pad type: - Protocol: Hello carries a GamepadPref byte (offset 21, the same trailing-byte back-compat pattern as the compositor; echoed resolved in Welcome at 54). Host precedence: explicit client choice > PUNKTFUNK_GAMEPAD env > Xbox 360, DualSense (UHID) only where available. ABI: punktfunk_connect_ex2 + punktfunk_connection_gamepad (connect_ex delegates; ABI_VERSION stays 2 — the trailing byte IS the compat mechanism). punktfunk-client-rs gets --gamepad. - Swift client: GamepadManager (app-lifetime discovery + selection — Settings lists every controller with capabilities/battery/"In use"; exactly ONE pad forwards as pad 0, auto = most recently connected, or pinned), GamepadCapture (snapshot-diff button/axis events, DualSense touchpad + ~250 Hz motion on the rich-input plane, held state released on switch/deactivate/stop), GamepadFeedback (rumble → CoreHaptics per-handle engines; lightbar → GCDeviceLight; player LEDs → playerIndex; adaptive-trigger blocks → the table-driven DualSenseTriggerEffect parser → GCDualSenseAdaptiveTrigger, exact for the 10-zone positional modes). The pad type auto-resolves from the physical controller at connect time, user-overridable in Settings. - Host DualSense fixes surfaced by adversarial review against hid-playstation / SDL / Nielk1 ground truth: input-report sensor/touch offsets were off by one (the kernel read garbage motion + phantom touches), the L2/R2 trigger blocks were swapped (the report is right-trigger-first), feedback now gates on the report's valid-flags (a plain rumble write no longer blanks lightbar/ triggers), and the touchpad rescale clamps to the advertised ABS_MT extents. - Tests: Hello/Welcome trailing-byte back-compat, pick_gamepad precedence, byte-exact input-report layout, valid-flag gating, per-mode trigger-parser table (incl. packed 3-bit zones), wire conversions, and a scripted loopback feedback burst (PUNKTFUNK_TEST_FEEDBACK=1) asserted through the xcframework on the rumble + HID-output planes. Validated: cargo test/clippy/fmt green on macOS + Linux (61 host tests), swift build/test green, test-loopback.sh green, tvOS/iOS targets compile. DualSense motion sign/scale is derived from the calibration blob, not yet live-verified (constants isolated in GamepadWire). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 16:28:33 +02:00
enricobuehler	b26f138699	feat(apple): session audio — host playback + mic uplink, device pickers in Settings ... Both directions of the audio plane, on CoreAudio's built-in Opus codec (kAudioFormatOpus — no bundled libopus; OpusCodec.swift, round trip unit-tested): - Playback: a drain thread pulls nextAudio() packets, decodes, and writes a priming jitter ring feeding an AVAudioSourceNode (~20 ms prefill, adaptive to the device's render quantum so large-buffer devices don't oscillate prime/dropout; a high-water clamp sheds stall backlog so one network hiccup can't permanently lag audio behind video; underrun re-primes — one dip, not sustained crackle). - Mic: a second engine taps the input device, resamples to 48 kHz stereo, Opus-encodes 20 ms chunks and sendMic()s them into the host's virtual PipeWire source. Permission via AVCaptureDevice (NSMicrophoneUsageDescription added to the Xcode target). - Settings: Speaker + Microphone pickers (CoreAudio HAL enumeration, persisted by device UID — "System default" leaves the engine unpinned so it follows macOS device changes) and a "Send microphone" toggle (default on). Applies from the next session. - Audio starts with streaming, never during the trust prompt (no host sound — and no mic uplink — before the user trusted the host); teardown stops audio before close(). Adversarial-review fixes baked in: stop() and the dangling mic-permission callback share one lock+flag protocol (no hot mic with no owner), the connect-success handler bails when the attempt was abandoned mid-handshake (no session/mic for a dead window), SessionAudio gets a deinit backstop (a dropped instance can't pin the connection via its drain thread), and the render scratch buffer is block-owned (was leaked per session). Verified live against the box: remote test decodes 100 host Opus packets to PCM and the host opens its virtual mic on the first uplinked frame ("punktfunk/1 virtual mic ready"); on-glass session runs with both engines up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 09:39:15 +02:00
enricobuehler	3a51551f97	feat(apple): mic uplink + touch events in PunktfunkKit ... Adopts the new ABI surface (still v2, additive): - PunktfunkConnection.sendMic(_:seq:ptsNs:) — Opus mic frames (48 kHz) to the host's virtual PipeWire source; enqueue-only, empty data = DTX silence. Wiring the actual Mac microphone (AVAudioEngine input → Opus) into the app is the follow-up, alongside audio playback (README note 5). - PunktfunkInputEvent.touchDown/touchMove/touchUp — absolute pixels + surface size in flags, host injects via libei ei_touchscreen. Built for the iOS variant; nothing on macOS emits them yet. - Loopback round trip now also sends touch events and mic frames (incl. a DTX frame) through the wrapper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 09:08:04 +02:00
enricobuehler	a9d1c16067	feat(apple): client-selectable compositor in the macOS client ... Adopts punktfunk_connect_ex from the compositor-selection batch: a Compositor enum on PunktfunkConnection (auto/kwin/wlroots/mutter/gamescope, with the host's name aliases for env parsing), a "Host compositor" picker in Settings (default Automatic — a concrete choice is honored only if that backend is available host-side), and PUNKTFUNK_COMPOSITOR / PUNKTFUNK_REMOTE_COMPOSITOR pass-throughs for the autoconnect dev hook and the remote first-light test. The wire change is backward-compatible (optional trailing byte), so no behavior changes at the default. Validated live against the box: host with no compositor env (auto-detect = KWin) logged "honoring client compositor request compositor=gamescope" and streamed 60/60 decoded frames from the spawned gamescope. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 22:51:42 +02:00
enricobuehler	0494e0200a	feat(apple): adapt the macOS client to ABI v2 — client identity + SPAKE2 PIN pairing ... The pairing/renegotiation batch bumped the punktfunk/1 ABI to v2 and the host now hard-rejects v1 Hellos (m3.rs), so streaming from the Mac was dead until the bundled PunktfunkCore.xcframework is rebuilt — it is gitignored, so that is a per-checkout step: bash scripts/build-xcframework.sh. The Swift wrapper itself was already adapted upstream; this lands the app on top of it. - ClientIdentityStore: persistent client identity in the login Keychain, presented on every connect so paired hosts recognize this Mac. Keychain access failure throws instead of regenerating (a fresh identity would silently un-pair this Mac from every --require-pairing host); a lost first-run race resolves toward the stored identity; pairing uses the strict loadForPairing() so a memory-only identity can't strand a ceremony. - PairSheet: the SPAKE2 PIN ceremony, reachable from a host card's context menu and from the trust prompt's "Pair with PIN instead…" (which drops the live session first — the host's accept loop is sequential). Success pins the verified fingerprint and connects; an in-flight ceremony self-discards when the sheet is dismissed, so a late success can't pin + auto-connect behind the user's back. Wrong PIN and Keychain failures get distinct, actionable error text. - Tests: identity unit tests; the full pairing ceremony + --require-pairing gate on loopback (test-loopback.sh arms a second host, parses its PIN from the log, and gives both hosts throwaway config homes — no more writes to the real ~/.config/punktfunk); remote pairing + pinned stream over the LAN (PUNKTFUNK_REMOTE_PIN, _PORT). Validated live against the box: SPAKE2 ceremony with the host's arming PIN → verified fingerprint → pinned + identified 720p60 session (host persisted the client identity); first light 60/60 AUs decoded to pixels; vkcube on glass through the app. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 21:49:43 +02:00
enricobuehler	bfd64ce871	rename: lumen → punktfunk, everywhere ... Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 13:11:59 +00:00
enricobuehler	bf8a974e8b	feat: M4 stage 1 — the SwiftUI client is real: compiles, tested, first light on glass ... The clients/apple scaffold is now a working macOS client, validated live against this repo's host across the LAN: gamescope virtual output → NVENC HEVC → lumen/1 (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox → AVSampleBufferDisplayLayer at 720p60, mouse/keyboard flowing back as QUIC datagrams into the host's gamescope EIS injector (~3.7k events injected in one session). LumenKit: - LumenConnection: the predicted cbindgen compile fixes (C17 header spells the typedefs as integers while the enum constants import as a distinct Swift type — bridge by rawValue); close() is now safe from any thread (a close flag + pumpLock held across the blocking poll enforce the C contract "never close with a next_au in flight"; flag prevents lock-starvation by back-to-back polls). - StreamView: per-pump cancellation token (reconnects can't double-pump), flush + re-gate on the next in-band parameter sets when the layer fails, no stale enqueue after restart. - InputCapture: fractional-delta accumulation (sub-pixel motion isn't truncated away), pressed-state tracking with release-all on focus loss and stop() (nothing sticks down host-side), global-singleton ownership guard (GC has one handler slot per process), X1/X2 buttons, horizontal scroll, full keypad/CapsLock/ISO-102nd/PrintScreen/Menu VKs. - LumenClient app shell (swift run LumenClient): connect form, fps/Mb-s HUD, LUMEN_AUTOCONNECT/LUMEN_MODE for scripted first-light runs. - Tests: Annex-B byte-level units; real-codec round trip (VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → AnnexB → VTDecompressionSession → pixels); test-loopback.sh (Swift client vs a real local m3-host over loopback — the Swift twin of c_abi_connection_roundtrip); RemoteFirstLightTests (full pipeline over the LAN). Host/build fixes that fell out: - The workspace builds on non-Linux again: gamestream audio (opus) and sendmmsg batching are now platform-gated with stubs/fallback, per the crate's "compiles everywhere" rule. - Horizontal scroll was inverted end-to-end: the injectors negated BOTH axes onto the ei/wl axes, but GameStream's horizontal convention is positive = right (moonlight-qt/Sunshine pass it through unnegated) — only vertical flips now. This also un-inverts real Moonlight clients. - AnnexB drops all zeros preceding a start code (trailing_zero_8bits padding), ffmpeg's policy, instead of leaking them into the preceding NAL. - build-xcframework.sh: deployment targets pinned to the package floor + an otool guard — cargo does not fingerprint MACOSX_DEPLOYMENT_TARGET, so warm caches can silently ship too-new minos objects. Adversarially reviewed (5-dimension multi-agent pass, every finding refutation-verified): 14 confirmed findings, all fixed above; the send-while-polling core-contract gap flagged here is closed by the lumen/1 session-planes work (&self pulls + per-plane borrow slots). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 14:46:45 +02:00