10 Commits

Author	SHA1	Message	Date
enricobuehler	7b10714b62	feat(apple): stage-2 presenter — explicit decode + Metal present + glass-to-glass ... Opt-in (Settings -> Presenter; `punktfunk.presenter`, default stage-1). Stage-1's AVSampleBufferDisplayLayer decodes AND presents internally with no per-frame callback, so neither decode nor present can be stamped or hand-paced. Stage-2 takes explicit control: - VideoDecoder: VTDecompressionSession, async output callback stamps decode-completion, session rebuilt on every IDR / format change. Unit-tested (testVideoDecoderAsyncCallbackDeliversPixels). - MetalVideoPresenter: CAMetalLayer + CVMetalTextureCache + a runtime-compiled BT.709 limited-range NV12->RGB shader, present at the next vsync. The CVMetalTextures + pixel buffer are held until the GPU completes. - Stage2Pipeline: pump thread -> decoder -> newest-ready 1-slot ring; the hosting view's display link drains it once per vsync and stamps capture->present (the display-link target time projected into CLOCK_REALTIME). - LatencyMeter gains record(ptsNs:atNs:offsetNs:); the HUD shows a capture->present (glass-to-glass, modulo host render->capture) line, skew-corrected via clockOffsetNs. Measured live ~11 ms p50 vs ~2.2 ms capture->client. - StreamView / StreamViewIOS host the CAMetalLayer as a sublayer + a CADisplayLink (NSView.displayLink on macOS) when stage-2; input capture + HUD unchanged. The session-active gates switch from `pump != nil` to `connection != nil` so capture engages without a StreamPump. Validated: builds macOS/iOS/tvOS; the decode half is unit-tested; the Metal present is live-validated on glass (correct image + the capture->present number). Colorspace is BT.709 SDR for now; 10-bit/HDR + a pacing policy are later. Plan: docs-site/content/docs/apple-stage2-presenter.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 15:29:23 +02:00
enricobuehler	6d3ff37d9e	feat(client): cross-target input handling + LAN mDNS discovery ... Input handling, building on macOS/iOS/tvOS: - macOS recapture after navigating out: engageCapture no longer latches captured=true when the cursor grab is refused mid app-activation (which left a free cursor that no later click could re-grab); cursorCapture.capture() now reports success. + canBecomeKeyView. - iOS/iPadOS recapture: restore the prior capture on didBecomeActive (nothing re-grabbed mouse/keyboard on return before). - iPad indirect pointer (no lock) is forwarded as an absolute MOUSE (move + buttons + scroll via hover / UITouch.indirectPointer), not as touch, with the local cursor visible; GCMouse owns the locked regime, gated so the two never double-send. Adds the MouseMoveAbs wire helper. - Trackpad scroll on iOS (was entirely missing): GCMouse scroll dpad when locked + a scroll-only UIPanGestureRecognizer otherwise. - tvOS: no focusable control during play (a focusable Disconnect button ate the controller's A in the focus engine); Siri Remote Menu disconnects. - Don't leak touch to the host under the TOFU trust prompt (gate on captureEnabled). LAN discovery: HostDiscovery (NWBrowser over _punktfunk._udp, the host's crate::discovery advert) resolves each service to IP:port and parses the TXT (fp advisory, pair, id); an "On this network" section in the grid (tap to save + connect, or pair if required). iOS/tvOS get NSBonjourServices via a merged Config/Info.plist. Integration-tested end to end against a fake NWListener advert. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 14:08:19 +02:00
enricobuehler	6b4de5d738	feat(client/speedtest): request the host's full 3 Gbps probe ceiling ... The Apple speed test asked for only 400 Mbps, capping the measured throughput there and hiding the link's real headroom. Request the host's full MAX_PROBE_KBPS (3 Gbps) instead, and raise the recommended-bitrate clamp from 500 Mbps to the host's 2 Gbps session ceiling so a fast measurement yields a usable recommendation. Also fix the stale caps left when the host clamps were raised (b8a33e2): the resolved-bitrate range and the probe doc comments (abi.rs, client.rs, regenerated header), plus the section 9 roadmap copy, now read 3 Gbps probe / 2 Gbps session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 14:08:19 +02:00
enricobuehler	e04328f086	feat(apple): capture->client latency HUD (skew-corrected) via the connect offset ... The Apple client now consumes the connector's clock offset. PunktfunkConnection reads punktfunk_connection_clock_offset_ns into clockOffsetNs at connect; a new LatencyMeter (PunktfunkKit, NSLock + percentiles, mirrors FrameMeter) records each AU's capture->client-receipt latency = now(CLOCK_REALTIME) + offset - pts_ns, and SessionModel drains p50/p95 into the macOS HUD ("capture->client N/N ms p50/p95", "(same-host)" when the host didn't answer the skew handshake). Wired at the existing onFrame hook in ContentView — additive, no change to the decode/present path. Unit test for the meter (percentiles, skew flag, absurd-value guard). This is the first cross-machine latency the real Apple client reports. SCOPE: stage-1 AVSampleBufferDisplayLayer decodes+presents compressed samples internally with no per-frame callback, so this excludes decode+present; true decode->present needs the stage-2 presenter (VTDecompressionSession + CAMetalLayer). Rebuild PunktfunkCore.xcframework (for the new C getter) before swift build/test on a Mac. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 11:58:54 +00:00
enricobuehler	1d605fb781	feat(gamepad): controller discovery + client-negotiated pad type + rich DualSense end to end ... The Apple client grows full gamepad support and punktfunk/1 learns to negotiate the virtual pad type: - Protocol: Hello carries a GamepadPref byte (offset 21, the same trailing-byte back-compat pattern as the compositor; echoed resolved in Welcome at 54). Host precedence: explicit client choice > PUNKTFUNK_GAMEPAD env > Xbox 360, DualSense (UHID) only where available. ABI: punktfunk_connect_ex2 + punktfunk_connection_gamepad (connect_ex delegates; ABI_VERSION stays 2 — the trailing byte IS the compat mechanism). punktfunk-client-rs gets --gamepad. - Swift client: GamepadManager (app-lifetime discovery + selection — Settings lists every controller with capabilities/battery/"In use"; exactly ONE pad forwards as pad 0, auto = most recently connected, or pinned), GamepadCapture (snapshot-diff button/axis events, DualSense touchpad + ~250 Hz motion on the rich-input plane, held state released on switch/deactivate/stop), GamepadFeedback (rumble → CoreHaptics per-handle engines; lightbar → GCDeviceLight; player LEDs → playerIndex; adaptive-trigger blocks → the table-driven DualSenseTriggerEffect parser → GCDualSenseAdaptiveTrigger, exact for the 10-zone positional modes). The pad type auto-resolves from the physical controller at connect time, user-overridable in Settings. - Host DualSense fixes surfaced by adversarial review against hid-playstation / SDL / Nielk1 ground truth: input-report sensor/touch offsets were off by one (the kernel read garbage motion + phantom touches), the L2/R2 trigger blocks were swapped (the report is right-trigger-first), feedback now gates on the report's valid-flags (a plain rumble write no longer blanks lightbar/ triggers), and the touchpad rescale clamps to the advertised ABS_MT extents. - Tests: Hello/Welcome trailing-byte back-compat, pick_gamepad precedence, byte-exact input-report layout, valid-flag gating, per-mode trigger-parser table (incl. packed 3-bit zones), wire conversions, and a scripted loopback feedback burst (PUNKTFUNK_TEST_FEEDBACK=1) asserted through the xcframework on the rumble + HID-output planes. Validated: cargo test/clippy/fmt green on macOS + Linux (61 host tests), swift build/test green, test-loopback.sh green, tvOS/iOS targets compile. DualSense motion sign/scale is derived from the calibration blob, not yet live-verified (constants isolated in GamepadWire). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 16:28:33 +02:00
enricobuehler	b26f138699	feat(apple): session audio — host playback + mic uplink, device pickers in Settings ... Both directions of the audio plane, on CoreAudio's built-in Opus codec (kAudioFormatOpus — no bundled libopus; OpusCodec.swift, round trip unit-tested): - Playback: a drain thread pulls nextAudio() packets, decodes, and writes a priming jitter ring feeding an AVAudioSourceNode (~20 ms prefill, adaptive to the device's render quantum so large-buffer devices don't oscillate prime/dropout; a high-water clamp sheds stall backlog so one network hiccup can't permanently lag audio behind video; underrun re-primes — one dip, not sustained crackle). - Mic: a second engine taps the input device, resamples to 48 kHz stereo, Opus-encodes 20 ms chunks and sendMic()s them into the host's virtual PipeWire source. Permission via AVCaptureDevice (NSMicrophoneUsageDescription added to the Xcode target). - Settings: Speaker + Microphone pickers (CoreAudio HAL enumeration, persisted by device UID — "System default" leaves the engine unpinned so it follows macOS device changes) and a "Send microphone" toggle (default on). Applies from the next session. - Audio starts with streaming, never during the trust prompt (no host sound — and no mic uplink — before the user trusted the host); teardown stops audio before close(). Adversarial-review fixes baked in: stop() and the dangling mic-permission callback share one lock+flag protocol (no hot mic with no owner), the connect-success handler bails when the attempt was abandoned mid-handshake (no session/mic for a dead window), SessionAudio gets a deinit backstop (a dropped instance can't pin the connection via its drain thread), and the render scratch buffer is block-owned (was leaked per session). Verified live against the box: remote test decodes 100 host Opus packets to PCM and the host opens its virtual mic on the first uplinked frame ("punktfunk/1 virtual mic ready"); on-glass session runs with both engines up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 09:39:15 +02:00
enricobuehler	3a51551f97	feat(apple): mic uplink + touch events in PunktfunkKit ... Adopts the new ABI surface (still v2, additive): - PunktfunkConnection.sendMic(_:seq:ptsNs:) — Opus mic frames (48 kHz) to the host's virtual PipeWire source; enqueue-only, empty data = DTX silence. Wiring the actual Mac microphone (AVAudioEngine input → Opus) into the app is the follow-up, alongside audio playback (README note 5). - PunktfunkInputEvent.touchDown/touchMove/touchUp — absolute pixels + surface size in flags, host injects via libei ei_touchscreen. Built for the iOS variant; nothing on macOS emits them yet. - Loopback round trip now also sends touch events and mic frames (incl. a DTX frame) through the wrapper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 09:08:04 +02:00
enricobuehler	a9d1c16067	feat(apple): client-selectable compositor in the macOS client ... Adopts punktfunk_connect_ex from the compositor-selection batch: a Compositor enum on PunktfunkConnection (auto/kwin/wlroots/mutter/gamescope, with the host's name aliases for env parsing), a "Host compositor" picker in Settings (default Automatic — a concrete choice is honored only if that backend is available host-side), and PUNKTFUNK_COMPOSITOR / PUNKTFUNK_REMOTE_COMPOSITOR pass-throughs for the autoconnect dev hook and the remote first-light test. The wire change is backward-compatible (optional trailing byte), so no behavior changes at the default. Validated live against the box: host with no compositor env (auto-detect = KWin) logged "honoring client compositor request compositor=gamescope" and streamed 60/60 decoded frames from the spawned gamescope. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 22:51:42 +02:00
enricobuehler	0494e0200a	feat(apple): adapt the macOS client to ABI v2 — client identity + SPAKE2 PIN pairing ... The pairing/renegotiation batch bumped the punktfunk/1 ABI to v2 and the host now hard-rejects v1 Hellos (m3.rs), so streaming from the Mac was dead until the bundled PunktfunkCore.xcframework is rebuilt — it is gitignored, so that is a per-checkout step: bash scripts/build-xcframework.sh. The Swift wrapper itself was already adapted upstream; this lands the app on top of it. - ClientIdentityStore: persistent client identity in the login Keychain, presented on every connect so paired hosts recognize this Mac. Keychain access failure throws instead of regenerating (a fresh identity would silently un-pair this Mac from every --require-pairing host); a lost first-run race resolves toward the stored identity; pairing uses the strict loadForPairing() so a memory-only identity can't strand a ceremony. - PairSheet: the SPAKE2 PIN ceremony, reachable from a host card's context menu and from the trust prompt's "Pair with PIN instead…" (which drops the live session first — the host's accept loop is sequential). Success pins the verified fingerprint and connects; an in-flight ceremony self-discards when the sheet is dismissed, so a late success can't pin + auto-connect behind the user's back. Wrong PIN and Keychain failures get distinct, actionable error text. - Tests: identity unit tests; the full pairing ceremony + --require-pairing gate on loopback (test-loopback.sh arms a second host, parses its PIN from the log, and gives both hosts throwaway config homes — no more writes to the real ~/.config/punktfunk); remote pairing + pinned stream over the LAN (PUNKTFUNK_REMOTE_PIN, _PORT). Validated live against the box: SPAKE2 ceremony with the host's arming PIN → verified fingerprint → pinned + identified 720p60 session (host persisted the client identity); first light 60/60 AUs decoded to pixels; vkcube on glass through the app. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-10 21:49:43 +02:00
enricobuehler	bfd64ce871	rename: lumen → punktfunk, everywhere ... Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 13:11:59 +00:00