10 Commits

Author	SHA1	Message	Date
enricobuehler	75627c8afe	feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients ... Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client (previously stereo-only): - core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome `audio_channels` negotiation via the trailing-byte back-compat pattern (old peers fall back to stereo); C-ABI `punktfunk_connect_ex6`, `punktfunk_connection_audio_channels`, and in-core multistream decode `punktfunk_connection_next_audio_pcm` for embedders without a multistream Opus decoder. Real-libopus channel-identity round-trip test. - host: native audio thread captures + Opus-(multi)stream-encodes at the negotiated count (with a cross-session cached-capturer channel-mismatch fix); GameStream surround unified onto the safe `opus::MSEncoder`, dropping `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround; WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask. - clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM → AVAudioEngine with an explicit wire-order channel layout; each gains a Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless validator. Verified on Linux: core/host/linux/probe test suites + the Android Rust (cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple builds, all on-glass checks, and the live native loopback are pending (CI / a free box). Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so cannot be committed separately from the surround changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:11:05 +00:00
enricobuehler	4e79e6cdad	fix(android/audio): kill the AAudio crackle (RT-safe ring + deeper buffer + XRun sizing) ... The jitter ring was a port of the Linux client's, but Linux runs on PipeWire (adaptive resampling masks host↔DAC drift + a shallow buffer); AAudio hands us a raw realtime callback and we own the buffer, so the same code crackled only on Android. Three converging causes, all fixed: - Heap free on the realtime audio thread every quantum (Android's Scudo free() has unbounded tail latency → XRun → click). Decoded buffers are now recycled back to the producer via a free-list instead of freed on the audio thread; the ring is pre-reserved so extend() never reallocates there. - The ring collapsed to ~15 ms on the tiny LowLatency burst and re-primed (a fresh silence) on every single empty callback. Now ~40 ms prime / ~150 ms hard cap, decoupled from the burst size, with de-prime hysteresis (re-prime only after a sustained drain). - AAudio's anti-glitch knobs were unused: prime the HW buffer above its 2-burst default and grow it on getXRunCount(). The post-open log now reports perf/sharing/buffer so a fall to a resampled legacy path is visible. Steady-state audio latency ~15 → ~40 ms (within lip-sync tolerance; matches the Moonlight/Sunshine operating point). cargo-ndk build both ABIs + fmt + clippy green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-27 11:33:51 +00:00
enricobuehler	095540efc2	feat(android): native mDNS discovery, host naming, touch mouse, stock selects ... Discovery: replace the flaky per-OEM NsdManager with the same mdns-sd browse the Linux/Windows clients use, in the Rust core over JNI and polled by Kotlin (discovery.rs + nativeDiscovery{Start,Poll,Stop}); Kotlin keeps only the Wi-Fi MulticastLock + permission UX. IPv4-only (the core can't dial a bare/scoped v6 literal); daemon + fold-thread cleanup on every failure path; field sanitization so a rogue advert can't corrupt the picker snapshot. Discovery now starts regardless of NEARBY_WIFI_DEVICES (raw multicast only needs the MulticastLock) — a denial no longer kills it forever. ParseTxtTest replaced by ParseRecordTest. Hosts: hide already-saved hosts from the "Discovered" section (match by fingerprint, else address:port — mirrors the Apple client); add an optional Name field to the Add-host sheet and a Rename action on saved cards. Input: touch -> absolute mouse "direct pointing" like the Apple client — the host cursor follows the finger (new nativeSendPointerAbs -> MouseMoveAbs). Tap = left click, two-finger tap = right click, two-finger drag = scroll, tap-then-drag = left-drag, three-finger tap = HUD toggle. Settings: revert the dropdowns to the stock ExposedDropdownMenuBox look (a controller-focus UI will come separately); even out the Add-host field gaps. Docs updated (CLAUDE.md, client READMEs, docs-site status). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-22 23:48:45 +02:00
enricobuehler	3e6c9f6060	feat(gamepad): add virtual Xbox One/Series + DualShock 4 pad types ... Extends virtual-controller support beyond Xbox 360 + DualSense. Goal: a physical Xbox One or PS4 pad on the client gets a near-native matching virtual pad on the host, auto-resolved from the controller type. Protocol/core: - GamepadPref gains XboxOne (wire 3) + DualShock4 (wire 4); to_u8/from_u8/ from_name/as_str + C ABI PUNKTFUNK_GAMEPAD_XBOXONE/_DUALSHOCK4 constants (compile-time guard ties them to the enum). Single-byte wire form is unchanged, so it's forward-compatible (older peers degrade to Auto). Host (Linux): - New UHID DualShock 4 backend (inject/dualshock4.rs) bound by hid-playstation: lightbar, touchpad, motion, rumble — DualSense minus adaptive triggers / player LEDs / mute. Reuses the DualSense pure state + button mapping; only the report byte layout, the real-DS4 HID descriptor, the GET_REPORT handshake (0x12 MAC mandatory; 0x02 calibration; 0xa3 firmware) and the touchpad resolution (1920x942) differ. Touchpad/motion ride the existing 0xCC plane, lightbar the 0xCD Led plane (deduped); rumble the universal 0xCA plane. - Xbox One/Series is the uinput Xbox-360 backend parameterized with the One S USB identity (045e:02ea) for matching glyphs — XInput-identical otherwise. - PadBackend dispatch + resolver handle both; off Linux the UHID pads and One/Series fold into Xbox 360. Windows-host DS4 (ViGEm) deferred. Clients (auto-resolve physical pad -> virtual type, plus manual settings): - Linux/Windows (SDL3): SDL_GAMEPAD_TYPE_PS4 -> DualShock 4, _XBOXONE -> Xbox One; PadInfo carries the resolved pref; DS4 touchpad/motion capture + lightbar already type-agnostic. Linux settings combo + label updated. - Apple (GameController): GCDualShockGamepad/GCXboxGamepad detection, DS4 touchpad capture, settings picker entries. - Android (Kotlin): InputDevice VID/PID auto-detect (matching the other clients) + settings entries. - probe: --gamepad help/aliases. Also hardens the Android JNI boundary: wrap the teardown + poll-thread shims in catch_unwind so a panic degrades to a logged no-op instead of aborting the app. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-21 13:34:44 +00:00
enricobuehler	551012bb43	feat(clients): HDR Steps 2-3 — apply mastering metadata + display capability-gate ... Continues docs/hdr-pipeline-plan.md. Steps 0/1 + Step 2 (Windows/Android) already landed in 3526517; this is Step 2 (Apple) + Step 3 (all clients). Client-only — no core/host/ABI change (the 0xCE/next_hdr_meta/color_info surfaces shipped in Step 0). Step 2 — clients APPLY the host's HDR metadata (each remaps from the wire form: ST.2086 G,B,R order, mastering luminance in 0.0001 cd/m2): - Apple: connect via punktfunk_connect_ex5 (resurrects the previously-dead HDR pipeline); nextHdrMeta/colorInfo wrappers + HdrMeta SEI-blob builders; the pump drains nextHdrMeta -> VideoDecoder.setHdrMeta -> CVBufferSetAttachment of MasteringDisplayColorVolume (24B BE) + ContentLightLevelInfo (4B BE) on each HDR pixel buffer (correct for the itur_2100_PQ layer; CAEDRMetadata avoided as ambiguous there). Step 3 — capability-gate: advertise HDR caps ONLY when the display can present it, so an SDR display gets a proper BT.709 stream instead of PQ it would mis-tone-map; an HDR display self-tone-maps from the Step-1/2 mastering metadata. - Windows: present::display_supports_hdr() (DXGI any IDXGIOutput6 colour space == G2084), ANDed with the user HDR setting in session.rs; logs the SDR drop. - Apple: NSScreen.maximumExtendedDynamicRangeColorComponentValue>1 (macOS) / UIScreen.main.potentialEDRHeadroom>1 (iOS) in SessionModel. - Android: Settings.displaySupportsHdr (Display.getHdrCapabilities HDR10/HDR10+) passed through a new hdr_enabled jboolean on nativeConnect; session.rs gates the caps. Validation: Android native (incl. the jboolean gate) builds + clippy clean via cargo-ndk; fmt clean. Windows (MSVC), Apple (Swift) and the Kotlin side are CI/on-glass validated — not compilable on the Linux dev box. Deferred to the RTX box: mid-session Reconfigure SDR-downgrade on monitor move, and confirming the host emits SDR for an SDR client off an HDR desktop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-21 09:46:58 +00:00
enricobuehler	3526517eb1	feat: HDR Step-0 colour-metadata transport + security-audit hardening ... Two strands, entangled in punktfunk1.rs, committed together (one builds-green tree). HDR pipeline Step 0 — glass-to-glass colour-metadata transport (docs/hdr-pipeline-plan.md): - Protocol/ABI: ColorInfo on the Welcome + a 0xCE HdrMeta datagram carry the source colour space + HDR10 static mastering metadata (quic.rs, abi.rs connect_ex5 fixing caps=0). - New platform-independent, unit-tested HDR static-metadata helpers (hdr.rs): chromaticities (1/50000), mastering luminance (0.0001 cd/m2), MaxCLL/MaxFALL in HDR10/ST.2086 units. - Capture/encode hooks (capture.rs, encode.rs set_hdr_meta) + Linux client / probe plumbing. Security-audit hardening — top 3 from docs/security-review.md, each adversarially verified: - #1 [HIGH] Secret file permissions. The host key.pem/cert.pem and both trust stores are now written owner-only: 0600 + dir 0700 on Unix (mirrors mgmt_token), best-effort SYSTEM/Administrators/OWNER-only icacls DACL on Windows (%ProgramData% is Users-readable). Closes a local key-disclosure -> host-impersonation gap. New gamestream::{create_private_dir, write_secret_file} + a 0600 regression test. - #2 [HIGH] Native SPAKE2 PIN is single-use. The PIN is consumed the moment the host sends its key-confirmation (which lets the client test its one guess), before reading the proof, so any completed attempt -- right OR wrong -- disarms the window. A wrong PIN isn't observable host-side (the client aborts before sending its proof), so consuming on first attempt is what delivers the documented "one online guess" instead of an unbounded brute-force of the static 4-digit PIN. Test verifies single-use. - #3 [MEDIUM] RTSP packetSize is bounded ([64,2048] in stream_config) and VideoPacketizer::new uses saturating .max(1), killing a PRE-AUTH div-by-zero/underflow panic of the video thread. Tests for {0,15,16,17} + out-of-range rejection. fmt + clippy -D warnings clean; full workspace test suite green (93 host tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-21 09:07:59 +00:00
enricobuehler	55cd58e487	fix(android): DataSpace impls Display not Debug — use {ds} in HDR logs ... ndk's DataSpace derives Copy/PartialEq/Eq and impls Display (no Debug), so the {ds:?} in the HDR dataspace log statements wouldn't compile under cargo-ndk. Host clippy can't catch it — decode.rs is android-gated. Switch to {ds}. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-18 22:13:20 +00:00
enricobuehler	1cd5e0e375	feat(android): HDR (Main10 / BT.2020 PQ) + fix ndk feature gating ... Mirrors the Apple client's HDR path so the Android client can display HDR from a Windows HDR host: - nativeConnect now advertises VIDEO_CAP_10BIT | VIDEO_CAP_HDR (was 0), so the host upgrades to a Main10 / BT.2020 PQ encode. - decode.rs detects HDR reactively from the decoder's reported output colour (color-transfer ST2084=6 / HLG=7, color-range) -- the AMediaCodec analogue of VideoToolbox's format description on Apple -- and signals the Surface dataspace (Bt2020[Itu]Pq / Bt2020[Itu]Hlg) so the compositor/display switch to HDR. AMediaCodec decodes Main10 from the in-band SPS; no profile override needed. Also fixes the Android build: set_frame_rate (added in 5262e28) is gated on the ndk `nativewindow` + `api-level-30` features, which weren't enabled -- so that commit could not compile under cargo-ndk. Enable features = ["media","audio","nativewindow","api-level-31"] (minSdk 31): covers set_frame_rate (api-30), set_buffers_data_space + the DataSpace module (api-28), and ANativeWindow (nativewindow). Verified host-side: fmt --all + clippy --workspace (the caps advertise + JNI surface). The android-gated decode + NDK gating verified against the ndk 0.9 sources; android.yml (cargo-ndk) is the compile gate, and real HDR display needs an HDR device + Windows HDR host. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-18 22:09:54 +00:00
enricobuehler	5262e28b79	feat(android): live stats HUD + low-latency decode tuning ... Stats HUD (mirrors the Apple client): the decode thread accumulates FPS, receive throughput, and capture->client latency (p50/p95, skew-corrected) in Rust (clients/android/native/src/stats.rs); nativeVideoStats drains a snapshot ~1 Hz over JNI as a DoubleArray. StreamScreen renders a Compose overlay (W*H@Hz / fps / Mb/s / latency, + dropped-under-loss), toggled by a Settings switch (persisted, default on) or a 3-finger tap. Performance (decode.rs): - ANativeWindow_setFrameRate(refresh_hz): align display vsync to the stream rate (no 60-in-120 judder); safe since minSdk 31 >= API 30. - Raise the decode thread toward URGENT_DISPLAY (best-effort setpriority) so background work can't preempt it under load. - Codec low-latency hints KEY_PRIORITY=0 (realtime) + KEY_OPERATING_RATE. Verified host-side: cargo build/clippy/fmt --workspace (the ungated stats + JNI accessor). The android-gated decode.rs (NDK) and the Kotlin build only in CI (android.yml: gradle + cargo-ndk) -- APIs verified against crate sources. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-18 21:49:29 +00:00
enricobuehler	9c8fa9340c	refactor: drop milestone names + consolidate clients; loss-recovery & rumble fixes ... Two bodies of work in one commit (the rename moved files the fixes also touched). Naming/structure cleanup (pre-launch): - Host modules m3.rs->punktfunk1.rs, m0.rs->spike.rs; CLI m3-host->punktfunk1-host, m0->spike; bare `punktfunk-host` now prints help. Types M3Options/M3Source-> Punktfunk1Options/Punktfunk1Source. - Clients consolidated out of crates/ into clients/: punktfunk-client-rs-> clients/probe (crate punktfunk-probe), client-linux->clients/linux, client-windows->clients/windows, punktfunk-android->clients/android/native (crate punktfunk-client-android; kept [lib] name=punktfunk_android so the JNI contract is unchanged). crates/ now holds only core + host. - Milestone codes M0-M4 purged from code/CLI/CLAUDE.md/README/docs/docs-site, kept only in docs/implementation-plan.md. docs/m2-plan.md-> docs/gamestream-host-plan.md. CI/gradle/flatpak paths updated. Client loss-recovery (video froze and never recovered after a brief drop): - Export punktfunk_connection_frames_dropped through the C ABI (the core already tracked it for the client keyframe-recovery loop; it was never reachable from the ABI clients). Regenerated punktfunk_core.h. - Apple (StreamPump + Stage2Pipeline) and Android (decode.rs) now poll frames_dropped and request a keyframe when it climbs -- the same loss-driven recovery Linux/Windows already had. Under infinite GOP the decoder silently conceals reference-missing frames, so the decode-error trigger rarely fires. Apple rumble robustness (worked then went spotty -- DualSense + Xbox): - Add CHHapticEngine stopped/reset handlers (rebuild on app background / audio interruption / server reset) and drop the permanent `broken` latch on a transient drive failure; latch only when the controller truly has no haptics. - Surface swallowed SDL set_rumble errors on Linux/Windows + diagnostic logging. Verified: cargo build/clippy/fmt --workspace, C-ABI harness, header drift. Not runnable on this box (verify in CI): Gitea workflows, gradle/Android, flatpak, Swift/decky. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-18 21:05:58 +00:00