97 Commits

Author	SHA1	Message	Date
enricobuehler	14c5e7c11c	feat(apple/M2): opt-in background keep-alive (audio + video-drop + timeout) ... Backgrounding a live session no longer freezes it when the user opts in: audio keeps playing (UIBackgroundModes audio), the QUIC connection + pump stay live, video decode is DROPPED, and a bounded timer auto-disconnects. Off by default. - PunktfunkConnection.setVideoDropped/isVideoDropped: a tiny lock-guarded flag both pumps read every iteration. StreamPump (stage-1), Stage2Pipeline (VT + PyroWave) drain nextAU() for flow control but DISCARD the AU before any VideoToolbox/Metal work — the crash/jetsam-safe seam (no GPU off-screen). - SessionModel.enterBackground(timeoutMinutes:) / exitBackground(): set the drop flag, mute the mic (privacy — SessionAudio.setMicMuted pauses the capture engine), arm a DispatchSourceTimer that disconnect(deliberate:false)s on fire (keeps host linger → fast late reconnect). exitBackground clears the flag and requestKeyframe()s; the pump's freeze gate auto-arms on the resumed frame-index gap so concealed frames are withheld until the IDR re-anchors. disconnect() cancels the timer + clears isBackgrounded. - ContentView scenePhase driver (iOS): .background+streaming+setting → enterBackground; .active → exitBackground. scenePhase (not willResignActive) so Control-Center/app-switcher peeks don't start the timer. - Settings → General (iOS-only keepAliveSection): toggle + 1/5/10/30 timeout; new keys backgroundKeepAlive (def off) / backgroundTimeoutMinutes (def 10). - Info.plist: UIBackgroundModes [audio] + NSSupportsLiveActivities (for M3). macOS swift build + swift test green (142 tests). The iOS-gated scenePhase handler + settings section are not exercised by the macOS CI target (known §9 gap) — need on-glass verification (audio never gaps, video re-anchors <1s LAN, timeout ends the session, phone-call audio-steal degrades gracefully). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-16 18:41:46 +02:00
enricobuehler	4cae1b8bb8	feat(apple/M0): App Group + PunktfunkShared + punktfunk:// deep links ... Foundation milestone for Live Activities & Widgets (design/apple-live- activities-and-widgets.md). No user-visible change beyond the URL scheme. - New dependency-free PunktfunkShared SwiftPM target (+ library product) so a future widget extension can link it WITHOUT PunktfunkKit (Rust staticlib + presentation layer). Moves StoredHost (model + JSON codec), DefaultsKeys, and punktfunkDefaultMgmtPort there; adds AppGroup.suiteName and the punktfunk:// DeepLink builder/parser. PunktfunkKit @_exported-imports it (no call-site churn for consumers; intra-Kit files import it explicitly since imports are file-scoped). - HostStore reads/writes the shared App-Group suite (group.io.unom.punktfunk) with a one-time migration from UserDefaults.standard (old value left in place for staged rollout); reloads the "PunktfunkHosts" widget timeline on change. - App Group entitlement on iOS/tvOS + macOS. - CFBundleURLTypes scheme `punktfunk`; ContentView.onOpenURL routes connect/<uuid>[?launch=<GameEntry.id>] into the existing connect() path (unknown host / already-streaming guards; never tears down a live session). - Round-trip tests: StoredHost JSON codec (+ legacy missing-optional decode), DeepLink grammar. `swift build` + `swift test` green (142 tests, 0 failures). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-16 18:16:39 +02:00
enricobuehler	ff38933312	feat(core,apple,session): report decode latency from the Apple + Windows/Linux clients too ... Extends 56f9c8c4 (the Automatic-bitrate decode signal, core + Android) to the remaining clients, so every platform caps Automatic at its real decoder limit instead of the network link ceiling — the fix for a fast LAN feeding a slower hardware decoder. - core/abi: punktfunk_connection_report_decode_us + _wants_decode_latency expose the NativeClient methods to the C-ABI embedders (regenerated punktfunk_core.h, additive only). - apple: PunktfunkConnection wrappers + Stage2Pipeline reports received→decoded from the VideoToolbox decode-completion callback — every decoded frame, before the newest-wins ring can drop the backlog. Stage-1 (AVSampleBufferDisplayLayer, no per-frame decode callback) stays network-only; stage-2 is the metered path. - windows/linux: the shared punktfunk-session client (pf-client-core) links core directly, so it calls the NativeClient methods — report received→decoded from the pump, gated on wants_decode_latency. Exact for the synchronous D3D11VA/software decode; received→submit (still the decoder-input backpressure signal) for the async Vulkan-Video path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-15 17:43:54 +02:00
enricobuehler	739a5f76bf	feat(apple): PyroWave Phase 5 — native Metal decode on Mac / Apple TV / iPad (§4.7) ... The Apple client now decodes PyroWave natively on the presenter's own MTLDevice — no MoltenVK, no upstream C++ in the app. Completes and wires up the decoder whose early working-tree snapshot rode along in 9127c346: - MetalWaveletShaders.swift: wavelet_dequant + idwt hand-ported from the vendored GLSL (STORAGE_MODE 0 only; subgroup scans → 32-wide simdgroups; DCShift spec constant → function constant; precision-1 split: fp16 levels 0-1 / fp32 2-4). - MetalWaveletDecoder.swift: Swift reimplementation of push_packet/decode_packet incl. the Phase-4 chunk-aligned window walk (FRAG chains, zeroed missing shards, the >half-blocks partial rule), init_block_meta's block-index space, and the 42-dequant + 13-idwt dispatch structure with encoder-boundary barriers. SOF-dims changes rebuild the size-dependent resources, which is also the mid-stream resize path. Ring of 4 output plane sets on the presenter's queue. - Presenter: pf_frag_planar (3xR8, the planar_csc.frag twin) + renderPlanar with a shared present tail; ReadyFrame carries an image enum (.video | .planar). - Stage2Pipeline: a dedicated PyroWave pump — no VideoToolbox machinery, no keyframe/re-anchor recovery (all-intra; partials render as localized blur by design), newest-frame-index staleness guard for late partials. - Opt-in: "PyroWave (wired LAN)" codec entry (probe-gated, ≈A13 floor via a real kernel-compile probe), selecting it advertises + prefers the codec and forces the session SDR (HDR/10-bit/4:4:4 caps dropped, plan contract). - Core ABI: punktfunk_connection_shard_payload() — the Welcome's negotiated shard payload, needed by native decoders to walk chunk-aligned AUs. - Validation: golden fixtures generated by the host encoder + upstream's own decoder (pyrowave_dump_golden, RTX 5070 Ti); the Metal decode PSNR-matches at 77-88 dB across all planes for dense AND chunk-aligned AUs, and a hole-punched partial still decodes. Parser unit tests cover the window walk, FRAG chains, broken chains, the half-blocks gate, and the block-index layout. Tests: apple 134 green (mac; iOS/tvOS build), host 312 w/ pyrowave on .21, core 148 w/ quic; clippy/fmt clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 12:16:43 +02:00
enricobuehler	9127c3465f	feat(client,host): PyroWave Apple Metal decoder + per-mode bitrate pin ... - clients/apple: native Metal wavelet decoder + compute shaders (Phase 5), decoding PyroWave without embedding MoltenVK. - pf-client-core: plumb user_flags/completeness through Decoder::decode_frame so the PyroWave backend parses chunk-aligned + partial AUs; gate the param's unused-warning to exactly the non-pyrowave builds (fixes -D warnings on the featureless Linux client build). - punktfunk-host: on a mid-stream mode switch, re-resolve the "Automatic" PyroWave bitrate for the new mode's ~1.6 bpp operating point (explicit rates and H.26x ABR stay put); reject sub-128px PyroWave modes before the encoder rebuild instead of after the ack. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 11:47:42 +02:00
enricobuehler	2621b6e6b1	feat(core,host,android): Steam Controller 2 as-is passthrough to Linux hosts ... The 2026 Steam Controller (Valve "Ibex" / SDL "Triton") captured on an Android client is passed through AS-IS: the host presents a virtual pad with the real wired identity (28DE:1302) and mirrors the physical pad's raw HID reports, so Steam on the host drives it over hidraw exactly like the real thing — trackpads, gyro, paddles, and its rumble/settings writes flow back onto the physical controller. Protocol ground truth: SDL's Valve-maintained SDL_hidapi_steam_triton.c + steam/controller_structs.h. Core: - GamepadPref::SteamController2 (wire byte 9; names steamcontroller2/ sc2/ibex) + PUNKTFUNK_GAMEPAD_STEAMCONTROLLER2 in the C ABI. - Raw HID planes: RichInput::HidReport (0xCC/0x04, client→host input reports verbatim, Copy fixed-64 body) and HidOutput::HidRaw (0xCD/0x05, host→client feature/output writes for replay). Best-effort is sound by the device protocol's own design (rumble re-sent every ~40 ms, settings every ~3 s — losses self-heal); HidRaw bypasses hidout dedup for exactly that reason. Host (Linux): - triton_proto.rs + steam_controller2.rs: Triton2Manager UHID backend — no kernel driver binds the PID (hidraw only; Steam Input is the consumer), raw mirroring with a typed-fallback 0x42 synthesizer until the first raw report, SET_REPORT ack + raw forward, canned GET_REPORT serial reply, rumble also parsed onto the universal 0xCA plane (phone mirror). Rides the uhid + 28DE-conflict degrades; UHID promotion by Steam is flagged in the creation log (usbip transport is the known follow-up if Steam ignores Interface:-1 devices for Triton too). Android: - Sc2UsbLink (wired/Puck: vendor-interface claim detaches the OS driver, interrupt read loop, lizard-off on the watchdog cadence, raw replay via interrupt-OUT / SET_REPORT with hidapi report-id framing) and Sc2BleLink (Valve vendor GATT service, notify subscribe machine, 0x45 re-framing, HIGH connection priority). - Sc2Capture orchestrator: raw plane + typed mirror (exit chord + host degrade paths keep working) on a GamepadRouter external slot; raw return path via GamepadFeedback.onHidRaw. - nativeSendPadHidReport JNI (direct ByteBuffer, no per-report copy), hidout raw decode, usb-host/BLUETOOTH_CONNECT manifest bits, opt-out settings toggle, StreamScreen engagement incl. the USB permission flow. Verified: core 149 + host 312 tests green on Linux (.21), on-box uhid smoke creates/mirrors/tears down the virtual 28DE:1302, C ABI harness round-trips, Android compileDebugKotlin green. On-glass with the real controller owed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 11:22:16 +02:00
enricobuehler	1fc9ef0050	feat(core,host,clients): typed pairing rejections — every client says WHY, not "not accepted" ... A host's pairing-gate rejections (not armed / bound to another device / rate-limited / identity required / denied / approval timeout / superseded / wire-version mismatch) used to drop the connection with a bare code-0 close, and every client collapsed that — plus plain unreachability — into one "wrong PIN / not accepted" message. A dead network path, a disarmed host, and an operator denial were indistinguishable, which is exactly the misdiagnosis behind the recent Android pairing support thread. - core: new ungated `reject` module — shared close-code block 0x60–0x67 (+ 0x42 busy promoted from the host), `RejectReason`, and `PunktfunkError::Rejected`; `pair()`/`connect()` decode the host's ApplicationClosed code into `Rejected` instead of a generic Io error. C ABI v7: status block −20…−28 and `punktfunk_connect_ex8` (`status_out` reports the failure cause; NULL-return alone can't). Wire unchanged — old peers see exactly the old bare close. - host: every gate rejection `conn.close()`s with its typed code (and the human reason as close bytes) before erroring out of the session task. - pf-client-core: shared `pair_error_message`/`connect_reject_message` wording consumed by the Windows + Linux + console-UI + CLI surfaces; a connect failure now renders the host's stated reason. - android: `nativeTakeLastError()` JNI token + `ConnectErrors.kt` — a network timeout is no longer reported as "wrong PIN, or the host isn't armed", and a typed rejection skips the wake-and-wait fallback (the host is demonstrably awake). - apple: `HostRejection` + `.rejected`; the pair sheet and session alerts show the stated reason; connect moves to `ex8`. Completes the cross-client half of the hunks that rode along in 12148243 (client.rs / trust.rs / punktfunk1.rs) — main did not build without this. Validated: workspace clippy -D warnings + full test suite green on .21 (EXIT=0, 309 host / 148 core suites); macOS core 147+c_abi green; swift build green; Android Kotlin + native crate green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 09:58:43 +02:00
enricobuehler	d58524c899	feat(client): opt-in "Rumble on this phone" mirrors pad-0 rumble onto the device ... iOS + Android: a new opt-in setting mirrors controller 1's rumble onto the device's own actuator (Apple RumbleRenderer Actuator.device / CoreHaptics, Android deviceBodyVibrator), so a motor-less clip-on pad still gives haptic feedback through the phone/tablet it's clamped to. Default off; wired through the gamepad settings on both platforms. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-14 16:32:19 +02:00
enricobuehler	6db91cbf40	feat(client): 3-finger swipe toggles the on-screen keyboard mid-stream ... iOS + Android: a three-finger vertical swipe up/down summons/dismisses the device soft keyboard while streaming (trackpad + pointer modes). Mobile scroll is now exactly two fingers so it never collides with the 3+-finger gesture (3+ only fell into the old `>= 2` scroll path by accident). Android: a TYPE_NULL KeyCaptureView plus IME meta-shift wrapping feeds key events through. iOS: UIKeyInput plus a SoftKeyMap char->VK table with a GCKeyboard dup gate so a hardware keyboard and the soft keyboard don't double-emit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-14 16:32:01 +02:00
enricobuehler	61c752e91e	feat(gamepad): Apple + Android pickers declare DualSense Edge / Switch Pro ... Plan 0.4 for the N1/N2 backends (SDL landed with them): - Apple: GamepadType grows dualSenseEdge=7 / switchPro=8 (wire-byte parity + name parsing). padKind splits the Edge out of the shared GCDualSenseGamepad subclass by product category, and resolves Switch Pro / a paired Joy-Con set by category (GameController has no Nintendo subclass; single Joy-Cons stay on the Xbox 360 fallback — half a pad). The DualSense-only gates (adaptive-trigger feedback, player LEDs, the touchpad+motion rich capture) now include the Edge — same surfaces. Paddle CAPTURE stays gated on G22 (needs a real pad to pin the paddleButton1..4 correspondence); the declared identity is right meanwhile. swift build + 124 tests green. - Android: PREF_DUALSENSEEDGE/PREF_SWITCHPRO wire bytes; the Sony PID table splits 0x0DF2 (Edge) out of DualSense; Nintendo 057E:2009 declares Switch Pro; ControllersScreen labels the new kinds. :kit/:app Kotlin compile green (-PskipRustBuild). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-14 11:21:42 +02:00
enricobuehler	764b5d938b	fix(gamepad): resolve the menu diagonal tie-break horizontally on all clients (G25) ... The gamepad-UI navigation resolvers disagreed on which way a perfect 45-degree stick push (|x| == |y|) resolves: the SDL core picked horizontal (`ax >= ay`) while Apple (`abs(x) > abs(y)`) and Android (`abs(Y) >= abs(X)`) picked vertical. Align Apple (`>` -> `>=`) and Android (`>=` -> `>`) to the SDL core so an exact diagonal moves focus the same way on every client (horizontal wins). This is client-local menu navigation only and never reaches the wire. Completes the last deferred G25 sub-part. Verified: Apple `swift build` + full suite (124 pass); Android `:app:compileDebugKotlin`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-13 22:24:03 +02:00
enricobuehler	26cac9ce20	fix(gamepad): truncate stick/trigger axes uniformly across clients (G25) ... Apple's GamepadCapture rounded axis values (`(v * scale).rounded()`) while SDL-core and Android truncate, so a half-pressed control emitted 128 on Apple vs 127 elsewhere. Drop `.rounded()` so `Int32(Float)` truncates toward zero on Apple too; rails are unchanged (full deflection stays 255 / ±32767). Also clamp SDL-core's LeftX/RightX to a symmetric -32767 like the Y axes and the other clients already do, instead of letting the raw i16 reach -32768. Verified: Apple `swift build` + full PunktfunkKit suite (124 pass); SDL half on Windows .173 `cargo clippy -p pf-client-core -- -D warnings` (green). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-13 22:05:13 +02:00
enricobuehler	68b9f108ab	feat(gamepad/apple): send Share/Create as BTN_MISC1 + pin wire bits to the C ABI ... G5: buttonMask mapped the dedicated share/create/capture element onto BTN_BACK, the same bit as View (buttonOptions). On an Xbox-Series pad those are two distinct physical buttons, so Share was indistinguishable from View on the host and never delivered the capture bit the host already decodes (DualSense mute / Steam quick-access). Route it to BTN_MISC1 instead, matching the Rust client's `Button::Misc1 => wire::BTN_MISC1`. Adds `misc1` to GamepadWire and allButtons so a held capture button is released on flush like the others. (On-glass verify owed on a real Xbox-Series pad; a clone pad that exposes one button as both buttonOptions and Share now emits back+misc1 for it — harmless on a plain xpad session and rare otherwise.) G22 (partial): define paddle1..4 for wire completeness, but leave them out of buttonMask/allButtons until the GameController paddleButton1..4 ↔ BTN_PADDLE physical correspondence is confirmed on a real Elite pad. G15: replace the 3-bit spot-check with an exhaustive assertion of every GamepadWire button/axis constant against the generated C ABI header (punktfunk_core.h), so any Swift-side drift from punktfunk_core::input::gamepad fails CI. swift build + full PunktfunkKit suite green (124 passed, 5 skipped). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 11:14:26 +02:00
enricobuehler	5cd66eca59	fix(gamepad/apple): stop releasing held guide on concurrent input ... `sync()` XOR-diffs the full `GamepadWire.allButtons` set (which includes guide) against `slot.buttons`, but `buttonMask` deliberately omits guide — it's driven separately by the Home handler via `sendGuide`. So while guide was physically held, the first stick/trigger/face-button move made `changed` carry the guide bit and the diff loop emitted a spurious guide-UP (then the real release was swallowed by `sendGuide`'s `guard now != slot.buttons`). Effect: you could not hold PS/guide while doing anything else — e.g. holding guide to keep the host's Steam overlay engaged released it the instant you touched a stick. The Rust reference client folds guide through the same diff as every other button and has no such split. Fix: preserve the current held guide bit through the diff (`buttonMask(g) | (slot.buttons & GamepadWire.guide)`) so guide is never seen as "changed"; `sendGuide` stays the sole toggler and `flush`/`allButtons` still release it on close/deactivation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 11:12:47 +02:00
enricobuehler	c0fc2d8ee8	feat(apple): iPad ⌃⌥⇧Q release chord + click-to-recapture, pixel-grid snap, match-window opt-in ... - InputCapture / StreamViewIOS: iPad ⌃⌥⇧Q un-capture chord, recognized from the GCKeyboard HID stream (no NSEvent monitor on iOS) for cross-client parity with the macOS/Windows/Linux combo; and a click into the video re-engages capture — the iPad analogue of macOS mouseDown → engageCapture(fromClick:), with the engaging click suppressed toward the host. - SessionPresenter: snap the aspect-fit sublayer frame to the backing pixel grid. AVMakeRect centers the fit rect at fractional points, so the compositor resampled the layer — a uniform "everything soft" blur even when the drawable was pixel-exact 1:1. Rounding origin + size to device pixels makes the composite a true 1:1 blit; idempotent when already aligned. - MetalVideoPresenter: PUNKTFUNK_BILINEAR_LUMA=1 A/B lever — compiles the shader with Catmull-Rom luma off (plain bilinear) to isolate bicubic overshoot from upstream fringing. - SettingsView / StreamView / StreamViewIOS: match-window reverted to opt-in (default OFF) — the explicit mode is used and never auto-resized unless enabled.	2026-07-13 01:22:09 +02:00
enricobuehler	8a18e130a2	feat(client): freeze-until-reanchor loss recovery on Android + Apple via shared core gate ... After unrecoverable loss the host keeps sending delta frames that reference a picture the client never received; hardware decoders conceal these as gray/ garbage with a success status. Linux already withheld them and held the last good frame until a proven clean re-anchor — this brings that behavior to the Android and Apple clients. Extract the Linux pump's freeze state machine into a shared `ReanchorGate` in punktfunk-core (reanchor.rs, 18 tests) exposed over the C ABI (ABI v6, additive — no wire change) for the Swift clients. Migrate the Linux/Deck pump (pf-client-core) onto it as the parity proof (no-op refactor). Then wire: - Android (decode.rs, both sync + async loops): arm on the frame-index gap, a pts-keyed flag map carries the wire flags to the output-buffer release, fold the gate per drained output, gate.poll replaces the dropped-climb block. - Apple Stage2Pipeline (default): arm on a gap (new noteFrameIndexGap), withhold at the ring-submit seam (CAMetalLayer holds its last drawable), poll framesDropped, fold VT decode errors through the no-output streak. - Apple StreamPump (stage-1): fold at enqueue, withhold via kCMSampleAttachmentKey_DoNotDisplay so the layer keeps decoding (reference chain intact) but holds the last displayed frame. - Apple VideoDecoder: thread the AU's wire flags to the async decode callback via a retained FrameContext refcon (replaces the receivedNs bit-pattern scalar). Lifts only on a proven re-anchor (IDR / RFI anchor / 2nd recovery mark) with a 500 ms backstop so a lost re-anchor can never freeze forever. Apple: swift build clean, 123/123 tests pass (incl. VideoToolboxRoundTripTests). On-glass loss-injection validation still owed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 01:22:09 +02:00
enricobuehler	97c67b2692	feat(apple): multi-controller support ... Roll the pf-client-core slot pattern to the Apple client (Swift): - GamepadManager tracks all connected GCControllers, assigning each a stable lowest-free wire pad index + concrete type, emitting GamepadArrival on connect and GamepadRemove on disconnect (index freed for reuse on re-plug). - GamepadCapture binds every controller with per-controller Slot state (buttons/axes/fingers/motion), threading the pad index into flags on every event; GamepadWire/InputEvents carry the pad + the two new events. - GamepadFeedback + RumbleRenderer go per-pad (rumbleByPad, slots[pad]), routing rumble/HID back to the correct controller by wire index. - ContentView/Settings surface every forwarded controller. pad 0 => flags 0, so single-controller wire is byte-identical. Cannot build on the Linux dev box (no Swift toolchain / Apple frameworks); wire bytes hand-checked against input.rs and GamepadWireTests extended for multi-pad. CI apple.yml (swift build/test on macOS) is the compile gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 21:53:07 +02:00
enricobuehler	e87fd42cee	feat(apple): default-UI connect/wake modal, auto-wake toggle, pixel-exact windowed streaming ... Three client-UX changes that share the connect/present path (and settings files): - Connect/wake overlay is now mode-aware: the console/gamepad UI keeps the full-screen aurora takeover, while the default (touch/desktop) UI shows a Liquid Glass modal over the host grid — the takeover looked out of place there. - Add an auto-wake toggle (DefaultsKey.autoWake, default on) across macOS/iOS/tvOS Settings + the gamepad settings view; gate startSession/prepareWake and the gamepad "Wake & Connect" label on it. MAC-address learning stays always-on. - Windowed sessions now stream at the window's native pixels (Match-window default-on) so the picture is 1:1 pixel-exact instead of the presenter resampling a fixed-mode frame; fullscreen reports full-display px, also 1:1. Also lands the mid-resize aspect-fit tracking (decoded contentSize) that keeps the picture undistorted after a resize. swift build + swift test (121 tests) green; screenshot scenes verified. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-12 01:29:12 +02:00
enricobuehler	dd02e1f402	feat(clients): unified full-screen connect/wake takeover + iOS/tvOS Wake-on-LAN ... Give instant feedback the moment a host is picked, and make the wake wait a full-screen takeover instead of a modal card — unified into one ConnectOverlay across every client: - android: new ConnectOverlay (aurora backdrop; Connecting / Waking / timed-out phases) replaces the tiny inline "Connecting…" row and the WakeOverlay card. The dial phase is now cancelable and hands off to the wake wait in one frame. - console (pf-console-ui): the connect/wake overlays become a full-screen aurora takeover (draw_takeover) instead of a centered card over a dim scrim; the Waking → Connecting handoff no longer blinks. - apple: new ConnectOverlay mirrors it (macOS / iOS / tvOS), replacing the per-tile connecting spinner + the WakeOverlay card; instant "Connecting…" from model.phase, and the carousel is gated inactive during the dial. Also enable Wake-on-LAN on iOS/tvOS now that the multicast entitlement is approved: enable com.apple.developer.networking.multicast and flip wakeOnLANAvailable to true on every platform (MACs were already learned from mDNS, so wake works immediately). Verified: Android compileDebugKotlin + screenshot renders; console clippy + 36 tests + rendered phases on Linux; Apple swift build + 121 tests + rendered phases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 23:29:35 +02:00
enricobuehler	7cea893db5	feat(recovery): wire LTR-RFI loss recovery into every client ... Centralize the client-side loss-range detector in punktfunk-core so every embedder shares one implementation instead of re-deriving the wrapping frame-index arithmetic: - NativeClient::note_frame_index(frame_index) folds each received AU (in receive order) through RfiRecovery::observe, firing a throttled RFI request for the exact lost span [first_missing, frame_index-1] on a forward gap. A host that can RFI (AMD LTR / NVENC) re-references a known-good frame instead of paying a 20-40x IDR spike; the frames_dropped-driven keyframe path stays the backstop for when the recovery frame itself is lost. - Export request_rfi + note_frame_index over the C ABI (Apple client). - Call it from the Android (hw+sw pumps), Apple (StreamPump + Stage2Pipeline via PunktfunkConnection.noteFrameIndex), and Windows in-process pumps. Linux/Deck inherit it through pf-client-core's session pump. - Split the decision into a pure RfiRecovery::observe(frame_index, now) and add 8 unit tests: arming, contiguous runs, exact lost-range, single-frame drop, the 100ms throttle (burst-suppress then re-open), reorder stragglers, and u32 wraparound (contiguous + gap-range). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 19:11:01 +02:00
enricobuehler	f01c5e210c	feat(resize/apple): resize overlay — blur + spinner during mid-stream resize ... Make a Match-window resize deliberate instead of a stutter: blur the live stream and show a spinner while the host rebuilds its virtual display + encoder and VideoToolbox re-inits on the new-mode IDR. No new protocol — driven entirely by existing client signals. - ResizeIndicator (pure core, unit-tested): START = follower steering, END = a decoded frame at the target size, TIMEOUT = 2.5s safety net for a rejected/capped switch that never yields a new-size frame; re-arms only on a CHANGED target, not a repeated same-size drag. - MatchWindowFollower.onResizeTarget fires the instant the window differs from the live mode (deduped via lastSteered); a new onDecodedSize callback threads each new-mode IDR's coded dims through StreamPump/Stage2Pipeline → SessionPresenter → both stream views. - SessionModel gains @Published resizing (+ resizeTargeted/resizeDecoded, a tick on the 1 Hz stats timer, reset on disconnect); ContentView blurs the stream 16px and overlays ResizeIndicatorView while resizing (the 32px trust-prompt blur is unchanged and takes precedence). tvOS declares the props but never fires the follower (it drives modes via AVDisplayManager), so the overlay stays dormant there. Pure core verified on the Linux toolchain; full AppKit/UIKit build pending on a Mac. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 15:58:14 +02:00
enricobuehler	89ff326ebf	feat(resize/apple): Match-window mid-stream resize trigger + settings (C3) ... design/midstream-resolution-resize.md Phase 2, Apple client. The stream mode follows the session window/scene: a windowed macOS window resize or an iPad Stage Manager / Split View scene change renegotiates the host's virtual display + encoder via the existing PunktfunkConnection.requestMode, so a windowed session streams native-resolution pixels instead of scaling. Decode/present need nothing — VideoToolbox recreates its session on the keyframe-derived format-description change (§1 table). - MatchWindowFollower (new): the shared D2 trigger discipline — physical pixels even-floored + clamped ≥320×200, debounce to resize-end, ≥1 s between requests, skip a size equal to the live mode, request each distinct size at most once (stops re-asking a rejected size / looping on a host rollback). Pure normalize/request core is unit-tested (MatchWindowTests). - macOS StreamLayerView: fed from layoutPresenter() (bounds → convertToBacking), guarded to once-in-a-window. - iOS StreamViewController: fed from viewDidLayoutSubviews (bounds × render scale); iOS-only (iPhone fullscreen no-ops, tvOS uses AVDisplayManager). - Settings: "Match window" toggle in the Stream mode section (iOS + macOS), DefaultsKey.matchWindow, read per session by the follower. Verified on a Linux Swift 6.1.2 toolchain (the app target needs AppKit/UIKit, unavailable here): the real MatchWindowFollower.swift type-checks in Swift 5 mode against a connection stub, and the pure discipline + the follower's decision path pass a standalone harness (drag-settle + grow → exactly two switches, refresh preserved, no re-request loop). A full build + on-device run (macOS window, iPad Stage Manager) remains for a Mac. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 13:36:03 +02:00
enricobuehler	73c911cae4	feat(rumble): host-authoritative self-terminating envelopes (0xCA v2) ... Rumble was level-triggered, unbounded state on a lossy channel: a non-zero level meant "buzz until further notice", healed only by the host re-sending state every 500 ms, and every client guessed when the host had died with its own magic timeout (SDL 1.5 s, Apple 1.6 s, Android up to 60 s). A lost stop, a reordered start, or a dead host could drone the motor for seconds. Make "stuck rumble" inexpressible on the wire. The 0xCA datagram grows a length-tolerant tail — [u8 seq][u16 ttl_ms] — so it self-terminates: the host authorizes a level for at most ttl_ms and renews it (~120 ms) while it holds, letting an abandoned one lapse client-side. seq is a per-pad wrapping reorder gate (reusing GamepadSnapshot::seq_newer) so a reordered stale start can't re-light a stopped motor. Decoders read the first 7 bytes as a plain level and ignore the tail, so no wire-version bump: an old client renders a new host's levels, and a new client falls back to its prior staleness heuristic against an old host (ttl = None). All four generation pairings render correctly. - core: encode_rumble_datagram_v2 / decode_rumble_envelope (datagram.rs); the client demux applies the seq gate then forwards (pad, low, high, Option<ttl>); next_rumble is unchanged (drops ttl), next_rumble_ttl keeps it; ABI adds punktfunk_connection_next_rumble2 + PUNKTFUNK_RUMBLE_NO_TTL, ABI_VERSION 4->5 (WIRE_VERSION unchanged — the tail is backward-compatible). - host (punktfunk1.rs): the flat 500 ms refresh becomes a renewal loop that bumps seq + stamps a fresh TTL on active pads and drains a short post-stop zero burst, then goes quiet. Hatches: PUNKTFUNK_RUMBLE_ENVELOPE=0 (legacy v1 + flat refresh, a bisect switch), PUNKTFUNK_RUMBLE_TTL_MS (clamped [150, 5000]). - renderers honor the TTL as their playback duration/deadline and keep their old heuristic only for a legacy (ttl=None) update: pf-client-core (the Deck haptic keep-alive is now deadline-bounded so it can't sustain a host-stopped rumble), clients/windows (SDL duration), android (JNI packs the lease out-of-band in bit 48 so any u16 ttl is unambiguous; Kotlin createOneShot(ttl)), apple (RumbleRenderer.envelopeDeadline + nextRumble2; sessionStaleSeconds demoted to the legacy fallback). - tests: codec round-trip + tail tolerance + seq-gate reorder (Rust); the probe asserts the v2 tail arrived under PUNKTFUNK_TEST_FEEDBACK; the Apple loopback asserts ttlMs round-trips end to end; RumbleTuning lease-decision cases. The host-side idle-timeout from the previous commit is defense in depth on the game side; this is the guarantee on the client side. Design: punktfunk-planning/design/rumble-envelope-plan.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-11 03:08:27 +02:00
enricobuehler	3ba19f28a2	feat(apple): the gamepad UI comes to tvOS - focus-driven, with real session controls ... The console UI now runs on tvOS through the NATIVE focus engine: carousel cards and settings rows are focusable Buttons (Siri Remote and pads both navigate; imperative scrollTo replaces the drop-prone scrollPosition binding), while iOS/macOS keep the 60 Hz poll untouched - on tvOS it carries only what focus has no concept of: X/Y screen actions and left/right value adjust with the poll's dominant-axis feel (onMoveCommand proved input-source-dependent: keyboard intercepted, pad dpad not -> double steps). Text entry uses the system fullscreen keyboard (TVTextEntry); pairing + library present as covers under the launcher; the game library defaults ON; settings values slide a quiet 14 pt in the step's direction. Session controls: controller/remote input routes EXCLUSIVELY through GameController during a stream (GCEventViewController, interaction disabled) - a pad's B no longer doubles as a UIKit menu press that ended sessions mid-game. Deliberate exits only: the cross-client escape chord (hold L1+R1+Start+Select 1.5 s - pf-client-core's contract, now implemented on all Apple platforms) and holding the remote's Back >= 1 s; the start-of-stream banner (now also on tvOS) teaches both. The Siri Remote's touch surface drives the host pointer - press = left click, Play/Pause = right click, release-tail jumps gated so motion stays truly relative. tvOS 26 regressions fixed at the root: the app-wide brand tint rendered every unfocused control as a blank pill (tint dropped on tvOS) and the 17 pt root font shrank the whole platform (29 pt there), plus 10-foot sizing across host cards, the gamepad screens, and the stats HUD (whose misleading "Press Menu" hint is gone). Acknowledgements scrolls by focus-sized chunks and Menu pops instead of suspending; full-width focusSections make the home actions reachable from any column. The presenter defaults to stage-3 glass pacing on tvOS (a 60 Hz panel fed a 60 fps stream is the sticky-FIFO worst case behind the 50 ms display stage) and is pickable from the gamepad settings; HDR capability advertises from AVPlayer.eligibleForHDRPlayback instead of the current mode's EDR headroom, so an SDR home screen no longer hides an HDR TV. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 16:59:01 +02:00
enricobuehler	1fcf9e11ec	fix(video): honor the signaled CSC matrix end-to-end + tvOS HDR presentation ... Clients derive Y'CbCr->RGB from the stream's SIGNALED matrix x range x depth via shared csc rows (Rust csc_rows + Swift CscRows) instead of hardcoded 709/2020 - a BT.601-signaled stream (a Linux host's RGB-input NVENC) no longer renders with a constant hue error. Host-side signaling made honest across NVENC/VAAPI/openh264/GameStream and the session plan's chroma/bit-depth. Decoded color-bar fixtures (601/709 x limited/full) pin the math in tests on both cores. Same presenter, tvOS HDR: tvOS has no Metal EDR API and a bare PQ colorspace tag composites UNTONE-MAPPED (the "overblown" Apple TV report), so HDR now splits on the display's live EDR headroom - PQ passthrough when the per-session AVDisplayManager mode switch landed (a real HDR10 output tone-maps itself), else an in-shader PQ->SDR tone-map (203-nit reference white, extended-Reinhard 1000-nit knee, 2020->709) into the proven SDR layer config. The 10-bit stream keeps its full decode depth either way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 16:59:01 +02:00
enricobuehler	08694b4026	feat(apple): stage-3 presenter — glass-gated present pacing as a live A/B ... Stage-2's present-on-arrival saturates CAMetalLayer's FIFO image queue whenever the stream rate runs at the panel's refresh (iOS always vsync-latches; the macOS 26 compositor latch-paces our out-of-band presents the same way): one early burst fills the queue to maximumDrawableCount and — arrivals then matching latches one-for-one — it never drains. That sticky depth is the measured 29-30 ms display stage on the 120 Hz ProMotion devices, and the full-queue regime is where host<->panel clock drift turns into the reported fixed-interval repeats/drops. The 240 Hz Studio never saturates, which is why it never showed either symptom. Stage-3 is the same pipeline with a PresentGate: at most ONE presented-but-undisplayed drawable in flight; the drawable's presented handler reopens the gate and re-signals the render thread, so the next present always takes the freshest newest-wins ring frame — the hidden queue latency becomes explicit, correct frame drops. A 100 ms stale fallback force-opens a gate whose handler never fires (the "presents aren't damage" hazard class) so a pathological system degrades visibly instead of freezing; the PUNKTFUNK_PRESENT_DEBUG `forced` counter exposes it (0 on healthy systems). Selection: the Settings > Display presenter picker now ships in release builds (stage 2 default / stage 3 experimental; the freeze-prone stage-1 diagnostic stays DEBUG-only), resolved per session with a PUNKTFUNK_PRESENTER=stage1|stage2|stage3 env override for CLI A/B. The pf-present debug line gains gated/forced/inflightMax — inflightMax is the direct image-queue-depth measurement for the A/B. Live-verified both ways against a real host at 1080p120: stage-3 holds 120/120 fps with inflightMax=1, forced=0, glass deltas p50 8.33 ms; stage-2 is behaviorally unchanged (120/120 fps, inflightMax=2 even on the wired 240 Hz setup — the saturation signal in miniature). Unit tests cover the gate (one-in-flight, stale force-open, idempotent release) and the presenter resolution (env override, release stage-1 gating); macOS tests green, iOS/tvOS xcodebuild clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 02:04:42 +02:00
enricobuehler	38b9f310e2	feat(clients): tiered stats overlay everywhere — Compact/Normal/Detailed on every platform ... Ship the Android client's 3-tier stats-overlay semantics in every other client (design/stats-unification.md vocabulary): Off → Compact (one line: fps · e2e ms · Mb/s + loss flag) → Normal (mode + e2e p50/p95 + loss counters) → Detailed (decoder path, HDR tag, per-stage latency equation). Apple: new StatsVerbosity in PunktfunkKit persisted under punktfunk.statsVerbosity (migrates the legacy hudEnabled bool: explicit off → Off, else Normal). The existing three-finger tap (TouchMouse, trackpad/pointer modes only — touch passthrough untouched) now cycles the tiers instead of toggling, matching Android; ⌃⌥⇧S (menu + captured-state monitor) cycles the same ladder. Tiered StreamHUDView (compact glass pill / headline HUD / full equation HUD); the iOS corner disconnect also shows in Compact (the pill carries no button). Tier pickers on iOS, macOS, tvOS and the gamepad settings UI. Session stack (Linux + Windows + Deck share punktfunk-session): shared pf_client_core::trust::StatsVerbosity; Settings grows stats_verbosity with a show_stats fallback, and writes keep the legacy bool in sync so pre-tier binaries reading the same JSON agree on off vs on. Ctrl+Alt+Shift+S cycles the tier and re-renders the OSD immediately from the last stats window; the stdout stats: line always carries the full Detailed text so the shell status card and scripts keep a stable shape; --stats bumps Off → Normal without demoting a richer tier. Tier pickers in the GTK dialog, the WinUI settings page and the console-UI settings row; shortcut copy updated (GTK shortcuts window, Windows help, session README). The Windows legacy builtin path keeps its bool HUD. Tests: tier migration/round-trip in trust.rs, tiered stats_text output in pf-presenter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 01:42:46 +02:00
enricobuehler	d0fa8bd3ee	fix(clients): pin mDNS discovery to IPv4 on every client ... Host machines' OS mDNS responders answer AAAA for hostname.local even though the punktfunk stack binds IPv4 sockets exclusively, so a v6-resolved address rendered a host card whose connect always failed. Apple's Network.framework prefers IPv6 (RFC 6724) — force the resolve connection to v4; pf-client-core (Linux/Windows/Deck shells) now picks get_addresses_v4() instead of an arbitrary first address. Android already did exactly this, for the same reason. Lift all three when the stack actually speaks IPv6 (per-family shard sizing is the gating item — the IPv6 header costs 20 bytes of the MTU budget shard_payload is maximal against). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 10:57:41 +02:00
enricobuehler	c3fa6c1514	feat(clients/apple): AV1 decode support — OBU plumbing, hardware-gated advertisement ... The Apple client was HEVC/H.264-only: the receive path spoke Annex-B NALs exclusively, so AV1 was never advertised and the codec picker hid it. Add the OBU flavor of the same plumbing (AV1.swift, sibling of AnnexB.swift): a zero-copy OBU walker, a full spec-5.5.1 sequence-header parser, an av1C CMVideoFormatDescription with colorimetry extensions (so isHDRFormat and the presenter stay codec-agnostic), and an ISOBMFF 'av01' sample repack (temporal delimiter stripped, everything size-fielded, one copy per AU). VideoCodec gains .av1 (wire 0x04); both pumps and VideoDecoder route through dispatching formatDescription(fromKeyframe:)/sampleBuffer(au:) — keyframe gating keys on the in-band sequence header exactly as the NAL codecs key on in-band parameter sets, so loss recovery and mid-session reconfigure work unchanged. AV1 sessions require a hardware decoder (VideoToolbox has no software AV1; same fail-fast policy as 4:4:4), and both the Hello advertisement and the Settings picker are gated on VTIsHardwareDecodeSupported — AV1 only appears on devices that can actually decode it (M3-class Macs, A17 Pro-class iPhones; no Apple TV). Tests: real SVT-AV1 blobs (generation recipe in the file) cover the walk, the parse against an independent reference, av1C bytes, delta-TU gating, repack byte-exactness, and — on AV1 hardware — a real VTDecompressionSession decode through VideoDecoder. Host precedence stays HEVC > AV1 > H.264, so AV1 engages only when explicitly picked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-09 01:22:56 +02:00
enricobuehler	069fcdfca6	feat(clients/apple): user-configurable Allow VRR (macOS + iOS) ... Add an "Allow VRR" toggle (Settings → Display; default on, non-tvOS) that hands the presenter's display link a wide frame-rate RANGE — preferred = the stream rate — so a ProMotion / adaptive-sync display can vary its physical refresh to match the stream. `SessionPresenter.syncFrameRate` now runs on macOS too: on it requests min 24 / max display / preferred stream Hz; off restores the prior behavior (macOS free-runs at the native rate, iOS keeps its 30 Hz floor). A no-op on fixed-refresh displays. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-09 00:47:15 +02:00
enricobuehler	01428ced58	feat(clients): reachability-probed presence + shareable Decky host management ... Add a bounded, trust-agnostic, mDNS-INDEPENDENT QUIC reachability probe and surface it everywhere saved-host presence is shown, so a host reached over a routed network (Tailscale/VPN/multicast-filtering LAN) no longer reads Offline just because it isn't advertising — the display-side companion to the 0.8.4 dial-first connect fix. Core: - punktfunk-core: NativeClient::probe (bounded handshake; a real host answers even on trust mismatch, a wrong/closed/TCP-only port fails) + punktfunk_probe C ABI (ABI_VERSION 3->4, header regenerated). - pf-client-core: trust::probe_reachable_many (parallel per-host sweep). Presence pips now read `advertising OR probed-reachable`, refreshed by a ~10-12s background sweep off the UI thread: - Linux (relm4): ui_hosts probed map + HostsMsg::Probed sweep. - Windows (windows-reactor): pf-probe worker -> HostsProps.probed. - Apple (SwiftUI): HostStore.refreshReachability, driven by HomeView + GamepadHomeView .task. - Android (Compose): nativeProbe JNI seam + periodic LaunchedEffect (LNP-gated), online dot added to the touch HostCard. - Decky already probes via --list-hosts --probe. Decky client: make the flatpak client's known-hosts store the single source of truth via new headless CLI modes (--list-hosts / --add-host / --set-host / --forget-host / --reset / --reachable). The plugin can now add a host by address, edit/forget hosts, reset all state (keeping the client identity), and shows probe-backed online pips — state is shared with the desktop client, not duplicated. Also lands in-progress Android 17 LNP groundwork (targetSdk 37 + ACCESS_LOCAL_NETWORK runtime flow, permission dialogs) that was already present in the working tree. Verified: cargo check + clippy clean (punktfunk-core, pf-client-core, linux, android native); android assembleDebug BUILD SUCCESSFUL; decky typecheck + rollup build clean; probe true/false-positive behaviour exercised against a live host. Windows and Apple were not compiled locally (no MSVC/Xcode on this Linux box). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-09 00:14:56 +02:00
enricobuehler	6198da3daf	feat(clients/apple): cross-client shortcuts + start banner, opt-in V-Sync, presenter rework ... Align the macOS/iPad Stream menu on the cross-client Ctrl+Alt+Shift set the Windows and Linux clients reserve — Release Mouse (⌃⌥⇧Q), Disconnect (⌃⌥⇧D), HUD toggle (⌃⌥⇧S) — with ⌘⎋ kept as the macOS/iPad capture toggle, and surface them on a 6-second banner at stream start. Add an opt-in V-Sync present mode (punktfunk.vsync, default OFF = lowest-latency immediate present; PUNKTFUNK_PRESENT_MODE overrides for A/B), with the presenter reworked to a frame-arrival-triggered render thread across Stage2Pipeline / MetalVideoPresenter / SessionPresenter, plus the windowed title-bar safe-area handling. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-09 00:12:02 +02:00
enricobuehler	838a1239cf	feat(codec): make client codec selection real — GPU-aware native advertisement ... The clients' codec pickers sent a preference the host threw away: host_wire_caps() hardcoded HEVC for every GPU backend, so the native path never emitted H.264/AV1 regardless of what the user chose. Host: advertise what the backend actually encodes, mirroring the GameStream serverinfo logic — software openh264 emits H.264; probed backends (Linux VAAPI, Windows AMF/QSV) advertise their per-GPU CodecSupport via the new wire_mask(); NVENC keeps the Moonlight-validated H.264|HEVC|AV1 static superset; an empty probe falls back to the superset so auto clients still resolve HEVC. Gate 10-bit to HEVC (like the 4:4:4 gate) now that a client can steer the codec — Main10 is the only 10-bit encode path. Fix the web-console stats label hardcoded to "hevc" (new Codec::label(), shared with the GameStream register_session mapping). Android: replace the hardcoded H264|HEVC Hello advertisement with a videoCodecs param fed by VideoDecoders.decodableCodecBits() (AV1 bit only when a real hardware video/av01 decoder exists — the decode loop was already mime-driven); offer AV1 in the codec picker on capable devices. Decky: add the missing "Video codec" dropdown (auto/hevc/h264/av1) to the plugin settings, round-tripping the same codec key the flatpak client reads. Apple: unchanged by design (AnnexB.swift is NAL-only, AV1 is never advertised); refresh the stale "hosts don't emit AV1 on the native path yet" comments here and host-side. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 16:45:37 +02:00
enricobuehler	7f29796e81	fix(apple): resolve macOS modifiers by keyCode — Control was silently dropped ... Modifier keys arrive only as flagsChanged, and the direction was recovered by diffing the device-dependent L/R bits (NX_DEVICE*KEYMASK) alone. Those bits are undocumented and some keyboards omit them (only the class bit, e.g. NX_CONTROLMASK, is set), so the diff saw no transition and the key never reached the host — no Ctrl shortcuts. SDL/Moonlight key off the event's keyCode for exactly this reason; do the same: keyCode names the changed key, the class bit says up, the device bits (when present) pick the side, and a tracked-held-state flip covers keyboards without them. PUNKTFUNK_INPUT_DEBUG=1 now also logs every flagsChanged (keyCode + raw flags) so a field report is diagnosable from client logs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-06 18:29:40 +02:00
enricobuehler	91fadce900	feat(clients): signal explicit exit (QUIT_CLOSE_CODE) on deliberate disconnect ... The core's deliberate-quit close (NativeClient::disconnect_quit → QUIT_CLOSE_CODE, host skips the keep-alive linger) was implemented but never called by any client. Wire it to each client's explicit user-disconnect action — NOT to a network drop / host-ended / app-background (those keep the linger for a reconnect): - core: new C-ABI punktfunk_connection_disconnect_quit(c) for the ABI clients - Linux (direct-core): Ctrl+Alt+Shift+D + the controller escape chord - Windows (direct-core): Ctrl+Alt+Shift+D - Apple (C-ABI): PunktfunkConnection.disconnectQuit() + a `deliberate` flag on SessionModel.disconnect() (sessionEnded passes false → keeps the linger) - Android (JNI): new nativeDisconnectQuit export, called from the back gesture + the Select+Start+L1+R1 chord (not the host-gone watchdog) - probe already did this via --quit (77871d6) Verified: core + Linux client + Android (cargo-ndk + gradle) build clean; Windows/Apple compile-checked by CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-06 13:37:45 +00:00
enricobuehler	fb709d68ce	feat(clients): Wake-on-LAN in apple/linux/windows/android/decky ... Each client learns a host's MAC from the mDNS `mac` TXT while it's awake, persists it on the saved-host record, and — when reconnecting to an offline host — sends a magic packet before connecting, plus an explicit "Wake host" action. Apple wraps the C-ABI; linux/windows call the core fn directly (linux also gains a --wake CLI mode); android via a new nativeWakeOnLan JNI export (the mDNS browse record gains a 7th mac field); decky shells out to the linux client's --wake before launching the stream. iOS/tvOS need the managed com.apple.developer.networking.multicast entitlement (pending Apple approval), so the wake path + UI are gated off via PunktfunkConnection.wakeOnLANAvailable and the entitlement is commented out — keeping iOS/tvOS releasable. MAC-learning stays active on every platform so it lights up the moment it's ungated. macOS works today. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-04 13:39:44 +02:00
enricobuehler	e56470cf56	fix(apple-client/audio): capture the right channel of a multi-channel mic + diagnostics ... The mic uplink handed the host pure digital silence on a multi-channel interface: AVAudioConverter's N→stereo downmix takes channels 0/1, but a pro interface puts the mic on ONE higher discrete channel. Fold the input to a mono bus ourselves instead — pick the mic's channel (or sum all) and resample that to the encoder's 48 kHz stereo, so the silent 0/1 downmix never happens. - New "Microphone channel" setting (macOS): Auto (sum every channel — a lone hot mic passes at full level) or pin 1-based channel N. Picker appears only for multi-channel devices, driven by the device's input channel count. - Diagnostics that make this class of failure self-naming next session: log the actual live capture device + format + fold mode, warn on a silent UID fallback, and a one-shot silence tripwire on the EXTRACTED signal (WARN on 10 s of zeros, else peak dBFS). - foldToMono extracted as a pure, unit-tested helper (pin / sum-clamp x interleaved / deinterleaved / mono / out-of-range). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-04 00:36:35 +02:00
enricobuehler	d6dbce1dcc	feat(clients): host/network split in every stats HUD (stats phase 2, client side) ... Consumes the 0xCF host-timing plane (99cb8dd) on all four GUI clients: each keeps a bounded pending ring of receipt samples keyed by pts, matches the host's per-AU capture→sent reports against it, and the HUD equation becomes = host 3.1 + network 6.7 + decode 2.1 + display 2.3 falling back to the combined `= host+network …` term whenever no timing matched the window (old host / datagram loss) — same total, one split fewer, never a misleading zero. Apple additionally gains the split as the only equation line under the stage-1 fallback presenter (receipt is presenter-independent), a `nextHostTiming` wrapper with its own plane lock, and a unit-tested `HostNetworkSplitter`; Android extends the JNI stats array 16→18 doubles (0–15 unchanged); Windows/Linux thread the split through `Stats` into the HUD and the headless/debug logs. Docs updated: design/stats-unification.md Phase 2 → implemented (wire format, fallback semantics), and the docs-site matrix's Sunshine "Host processing latency" row is now a direct match (ours includes the paced send; avg vs p50). Verified here: linux client clippy -D warnings green on the live tree, windows stub check + hand-verified diff, android cargo-ndk arm64 check green, apple loopback test extended (needs the rebuilt xcframework + swift test on the mac). On-glass: pending on all platforms. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 21:31:49 +00:00
enricobuehler	83460cd2c0	feat(clients): unified stats vocabulary across every client + Moonlight comparison docs ... One stat model everywhere (design/stats-unification.md): four measurement points (capture/received/decoded/displayed), three stages that tile the interval exactly, and a HUD that shows the addition explicitly — end-to-end 14.2 ms p50 · 19.8 p95 · capture→on-glass = host+network 9.8 + decode 2.1 + display 2.3 replacing each client's ad-hoc mix of overlapping absolutes (the Apple HUD's three arrow lines that looked sequential but weren't), mean-vs-median decode times (Windows/Linux), missing same-host-clock flags (Windows/Linux), and three different names for the same capture→received measurement (probe's "reassembled", Apple/Android's "client", Windows/Linux's post-decode "lat"). Per client: Apple threads receivedNs through the VT decode via the frame refcon bit pattern so the decode stage exists at all (stage-1 fallback honestly degrades to a capture→received headline); Windows carries FrameTimes through the existing frame channel to the render thread and adds e2e p50/p95 post-Present; Linux stamps received at AU pop and rides decoded_ns on DecodedFrame to the paintable-set site; Android pairs receipt stamps with MediaCodec output buffers via the codec's pts round-trip (JNI stats array 14→16 doubles, indexes 0-13 unchanged). fps now uniformly counts received AUs; lost/(received+lost) per window, hidden at zero. docs-site gains "Understanding the Stats Overlay": what each line means, why the equation only approximately sums (percentiles), and a line-by-line Moonlight/Sunshine matrix — including that Moonlight has no end-to-end number and its "network latency" is an ENet control RTT, so punktfunk's headline must not be compared against any single Moonlight line. Verified here: linux client + probe + core check/clippy/fmt green, android native cargo-ndk arm64 check green. Pending: Windows CI + on-glass, swift test on the mac, on-device Android. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 21:01:29 +00:00
enricobuehler	7e2d7c893c	feat(apple,android): three-way touch input — trackpad cursor (default), direct pointer, real multi-touch passthrough ... The two touch clients had exactly complementary gaps: iOS forwarded fingers ONLY as raw wire touches (no way to drive the host cursor from the touch screen), Android had the two mouse modes but no passthrough. Both now share one three-way "Touch input" setting: Trackpad (default) / Direct pointer / Touch passthrough. iOS/iPadOS: Input/TouchMouse.swift ports the Android gesture engine 1:1 (same px-based acceleration curve; tap=click, two-finger tap=right-click, two-finger drag=scroll, tap-then-drag=held drag, three-finger tap=stats HUD via the shared hudEnabled default); direct-pointer mode maps through the aspect-fit letterbox; the previous always-on behavior lives on as the passthrough option. The mode latches per gesture (a Settings change never splits one gesture across models), touchesCancelled releases held state without synthesizing a click, and session stop flushes a mid-drag button. Settings picker on iPhone + iPad next to the iPad-only pointer-capture toggle. Deliberate default change: trackpad, not passthrough. Android: new nativeSendTouch JNI shim → wire TouchDown/Move/Up (the host already injects real touch on every backend — libei touchscreen, wlroots, KWin fake-input, SendInput); streamTouchPassthrough forwards every finger with stable ids and lifts still-held contacts on teardown; the trackpadMode Boolean becomes the TouchMode enum (old pref migrated on load, never rewritten) with a Settings dropdown. Verified: macOS swift build + full suite (incl. new TouchMouseTests), iOS Simulator Swift compile, cargo check/fmt/clippy on the native crate, Kotlin app+kit compile + unit tests. On-glass feel of the iOS ballistics and Android passthrough against a touch-aware app still pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 00:02:12 +02:00
enricobuehler	a35a761c37	feat(apple/gamepad): claim controller system gestures during capture — PS button opens the Steam overlay, share/create stops screenshotting locally ... While a pad drives a stream, GamepadCapture now sets EVERY element's preferredSystemGestureState to .disabled (restored to .enabled on unbind). iOS/macOS attach system gestures to several controller buttons — share/create took a LOCAL screenshot instead of reaching the game, and only the Home element was opted out before. With the gestures claimed, the already-wired chains do their job: PS/Home → wire guide → BTN_MODE on the virtual xpad (the Steam-overlay button) / the PS bit on the virtual DualSense. Also fold the share/create/capture element (GCInputButtonShare) into the back/select wire bit — clone pads like the GameSir G8 expose their screenshot button only as the share element, not buttonOptions (OR onto the same bit, so double-exposed pads are harmless). The G8's other extra button (M) is a firmware-local modifier (turbo/hair-trigger/swap) invisible to the OS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 23:36:16 +02:00
enricobuehler	1b9d81928e	feat(apple/gamepad): rewrite rumble renderer — bounded divergence + iOS 27 plain-player fix ... Ground-up RumbleRenderer rewrite around one principle: rumble is idempotent state on a lossy channel, and the actuator's divergence from it must be bounded, not best-effort. The old renderer rebuilt an infinite-duration CHHapticAdvancedPatternPlayer per 0xCA datagram via an async stop; one stop lost inside CoreHaptics left an unstoppable player buzzing forever (the "entered the menu and rumble never stopped" bug). - Finite 4 s segments, never infinite events — a leaked player self-silences; steady levels re-arm seamlessly ON the engine timeline (no stop/start race) - GamepadFeedback drains the rumble plane DRY per cycle, newest-wins (was one datagram per 8 ms through a 16-deep drop-newest queue = lag + shed stops) - Host 500 ms state refreshes dedupe to a liveness stamp; zero applies immediately; nonzero ramps throttle to one rebake/25 ms per motor - Throwing player stop escalates to engine.stop() (kills leaked players); 1.6 s staleness watchdog (Policy.session) force-silences on a dead channel; the test panel holds levels via Policy.manual - Plain makePlayer, NEVER makeAdvancedPlayer: gamecontrollerd's controller haptics server advertises `adv players: 0`, and iOS 27 beta 2 hard-drops advanced loads with an XPC decode fault (-4811/4097, rumble silently dead). Live-verified on an iOS 27 beta 2 iPhone: DualSense rumble works - Split-handle engines fall back to one combined .default engine on repeated failure; renderer publishes health transitions and the test panel shows them (a refused system service no longer reads as silent app breakage) - Per-motor sharpness on split handles (0.3 heavy / 0.7 light); macOS DualSense raw-HID path gains a ~1 s keepalive re-write while nonzero - RumbleTuningTests pin the scheduling math, tuning relations, and a queue/ticker teardown smoke test Stuck-rumble streaming repro revalidation on glass still pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 23:06:45 +02:00
enricobuehler	34fe896ade	feat(apple): gamepad UI v2 — controller settings + add host, aurora, macOS ... Sources reorganized (client: Home/Session/Settings/Stores/Support/Trust; kit: Audio/Connection/Gamepad/Input/Support/Video/Views) with the big files split along the same seams. The gamepad mode is couch-complete, and now on macOS too (the living-room Mac case), not just iOS/iPadOS: - GamepadSettingsView: a console-style, fully controller-navigable settings screen (X from the launcher) — up/down moves focus, left/right steps values (clamped, boundary thud), A cycles/toggles, B closes; the focused row shows a one-line description. Backed by GamepadMenuList, the vertical sibling of GamepadCarousel, and SettingsOptions — the option lists hoisted out of SettingsView statics and shared by the touch, tvOS and gamepad settings. - GamepadAddHostView + GamepadKeyboard: register a host end to end with a pad — field rows open an on-screen controller keyboard (dpad grid, A types, X backspaces, B done); the launcher carousel ends in an Add Host tile, so the dead-end "add one with touch first" empty state is gone. - Launcher polish: contextual hint bar with the pad's real button glyphs, controller name + battery chip, one shared console chrome. - GamepadScreenBackground: an animated aurora (TimelineView-driven drifting blobs in the brand's violet family, breathing radii, slow hue shift, legibility scrim; freezes under Reduce Motion). Pure SwiftUI on purpose — a .metal library only bundles reliably in one of the two build systems (SPM vs the xcodeproj's synced folders) these sources compile under. - macOS port: settings/add-host/library present as sized sheets (a macOS sheet takes its content's IDEAL size, and the GeometryReader-driven screens collapsed to nothing), NSScreen-based mode lists, scroll indicators .never (the "always show scroll bars" setting overrides .hidden), tray scrims so scrolled rows dim under the pinned title/hints, extra title clearance, and a PUNKTFUNK_FORCE_GAMEPAD_UI=1 dev hook — launcher/settings/add-host/keyboard/ library render-verified live on a real Mac + LAN hosts. - GamepadMenuInput: X button support, and (re)start now snapshots held buttons so a controller handoff press never fires twice (the B that closed the keyboard no longer also cancels the screen underneath). - Cleanups: one "Connection failed" alert in ContentView instead of one per home screen; HostDiscovery.advertises/unsaved shared by both home screens. - host: can_encode_444 stub for the non-Linux/Windows host build (the macOS synthetic-source loopback used by the Swift tests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 11:24:44 +02:00
enricobuehler	b357934cb1	feat(clients): codec preference on Windows/Apple/Android clients (Phase 2b) ... Rounds out codec negotiation across the last three clients — each advertises what it can decode, builds its decoder from the resolved Welcome.codec, and exposes a "Video codec" preference picker. **Windows** (Rust, mirrors Linux): `decodable_codecs()` + `ffmpeg_codec_id()`; the D3D11VA and software FFmpeg decoders (and the mid-session D3D11VA→software demotion) open the negotiated codec instead of hardcoding HEVC; settings gain a `codec` field + reactor ComboBox; `--codec` CLI flag. **Apple** (Swift/C-ABI): AnnexB is now codec-aware — a `VideoCodec` enum drives H.264 vs HEVC NAL parsing / parameter-set extraction (`CMVideoFormatDescriptionCreateFromH264ParameterSets` for H.264, no VPS) and AVCC repacking; `PunktfunkConnection` advertises H264|HEVC via `punktfunk_connect_ex7`, reads `resolvedCodec` (`punktfunk_connection_codec`), and threads `videoCodec` into the stage-1/2 pipelines + `VideoDecoder`; SettingsView "Video codec" Picker (auto/HEVC/H.264). AV1 is left out (hosts don't emit it on the native path, and it's not an AnnexB codec). Test call sites updated. **Android** (Kotlin + Rust JNI): the JNI `nativeConnect` gains `preferredCodec`; the native decode loop picks the AMediaCodec MIME (`video/hevc`|`video/avc`) from `connector.codec` and advertises H264|HEVC; Settings `codec` field + Compose dropdown. Core/host/probe/Linux clippy + tests green (unchanged from 2a). Windows/Apple/Android compile on their platform CI (this Linux box can't build them — Windows toolchain / Xcode / the Android NDK's opus-cmake toolchain). All follow the Linux client's validated pattern. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-02 00:29:38 +00:00
enricobuehler	546b178ecc	feat(apple): gamepad ui	2026-07-01 15:14:19 +02:00
enricobuehler	ae51276a03	feat(web): consolidate paired devices, self-contained sections, docs + lint ... Web console - Pairing/Library/Stats refactored into self-contained subsections that each own their own queries + mutations; a shared slot-based layout (view.tsx) is filled by the live page (containers) and Storybook (pure cards + fixtures) so the layout can't drift. - All paired devices in one list on Pairing with a protocol column (punktfunk/1 + Moonlight), routing each unpair to the right endpoint; the redundant Clients page is removed. - Library: overview grid split from the add/edit form into separate files. - Login screen links out to the docs. Docs - "Console login password" section on every host page (apt/RPM/Bazzite/SteamOS/Windows) plus a new "Forgot your Password?" troubleshooting page, linked from the login screen. - Console served as HTTP/1.1 over TLS (drop the unusable HTTP/3 advertising) across the Bun entry, launchers, systemd units, and packaging. Tooling - Biome now respects .gitignore (stops linting generated code), config migrated to 2.5.1; all lint issues fixed cleanly. Also includes this branch's in-progress host, Apple client, packaging, and CI changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-30 19:05:22 +02:00
enricobuehler	ecec7cfa71	fix(apple/tvOS): guard EDR HDR APIs unavailable on tvOS ... The tvOS archive failed compiling PunktfunkKit: a recent presenter HDR change dropped the `#if os(macOS)` guard around the EDR calls and applied them "on all platforms", but `wantsExtendedDynamicRangeContent`, `CAEDRMetadata`, and `CAMetalLayer.edrMetadata` are all explicitly unavailable on tvOS. Wrap the EDR usage (and the makeEDR helper, whose return type is the unavailable CAEDRMetadata) in `#if !os(tvOS)`. macOS + iOS keep the reference-white-anchored EDR path unchanged; tvOS now sets only the rgba16Float pixel format + itur_2100_PQ colour space and lets its compositor tone-map from those. The 0xCE grade is still cached on tvOS (harmless), it just can't be pushed to the layer there. tvOS Simulator build: BUILD SUCCEEDED (PunktfunkKit Swift compile, the step that failed). macOS build + test green (49 tests); iOS compiles clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-30 11:12:33 +02:00
enricobuehler	06bee687c3	fix(apple/iOS): capture all attached mice; gate UIKit pointer path under lock ... The iPad pointer lock engaged but a Magic Keyboard trackpad went dead the moment a second pointer (a Universal Control "V-UC Automouse") was connected — on-device PUNKTFUNK_INPUT_DEBUG logs showed only ONE GCMouse attached (whichever was GCMouse.current), so the other device's motion handler was never installed. InputCapture.start() now attaches a handler to EVERY GCMouse.mice(), not just GCMouse.current, so a trackpad and a second mouse both drive (each GCMouse delivers its own deltas through its own handler). New arrivals still come via the GCMouseDidConnect observer. Also gate the WHOLE UIKit indirect-pointer path (motion, buttons AND scroll) on !gcMouseForwarding, not just motion+scroll: under pointer lock GCMouse owns buttons too, and the trackpad/mouse also emit UIKit indirect-pointer events pinned at the locked position — without the gate a click double-sent (GCMouse + UIKit). The two paths are now exact mirrors on `gcMouseForwarding` (== locked). Removes the investigation-only diagnostics (attachedMiceSummary/hasGCMouse, the per-event UIKit pointer/scroll logs, the GCMouse attach/became-current logs); the pre-existing `pointer lock isLocked=… captured=…` debug line is restored. iOS compiles against the SDK; macOS swift build + test green (49 tests). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-30 10:56:37 +02:00
enricobuehler	ae1b392011	feat(apple): Improve presenter ... feat(apple): add cursor capture on iPad	2026-06-30 01:31:48 +02:00
enricobuehler	df9470a0f2	feat(steam): M4 complete — C-ABI send path, Decky UX, Apple/Android parity ... Finish the client side of the Steam Controller / Steam Deck pipeline. - C-ABI (core abi.rs): PunktfunkRichInputEx — a size-prefixed superset of PunktfunkRichInput that can express the second trackpad (surface), a distinct click vs touch, signed coords + pressure — plus punktfunk_connection_send_rich_input2 (the struct_size ABI-skew-guard precedent). The only way a C client (Apple/embedders) can emit a TouchpadEx; the legacy struct + send_rich_input stay byte-for-byte. punktfunk_core.h regenerated. - Decky (clients/decky): a "Steam Deck" gamepad type in Settings + an unmissable Disable-Steam-Input instruction shown when it's selected (in Game Mode Steam Input holds 0x1205, so the SDL HIDAPI Steam driver can't open the Deck's controls until the user disables Steam Input for the shortcut). Plus a best-effort, feature-detected disableSteamInputForShortcut() in launchStream — never blocks/throws; the manual toggle is the documented source of truth. - Apple parity (PunktfunkConnection.swift): GamepadType.steamController/steamDeck (wire 5/6) + name parsing, so the resolved type round-trips. Capture is blocked (GameController never surfaces a 0x28DE HID device). - Android parity (Gamepad.kt): PREF_STEAMCONTROLLER/STEAMDECK + the Valve 0x28DE PIDs in prefFor(). Rich-input capture stays out of scope (no rich-input plane yet) — standard buttons/sticks resolve to the host's Steam Deck pad. Rust workspace clippy/fmt/test green; Decky src/ typechecks clean (only a pre-existing @decky/api dep resolution error remains); Swift/Kotlin compile on their CI. The full pipeline is now BUILT; what remains is validation that needs hardware we don't have (a running Steam on the host, a live Deck client, the Moonlight paddle regression). Not pushed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-29 19:17:37 +00:00