5 Commits

Author	SHA1	Message	Date
enricobuehler	75627c8afe	feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients ... Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client (previously stereo-only): - core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome `audio_channels` negotiation via the trailing-byte back-compat pattern (old peers fall back to stereo); C-ABI `punktfunk_connect_ex6`, `punktfunk_connection_audio_channels`, and in-core multistream decode `punktfunk_connection_next_audio_pcm` for embedders without a multistream Opus decoder. Real-libopus channel-identity round-trip test. - host: native audio thread captures + Opus-(multi)stream-encodes at the negotiated count (with a cross-session cached-capturer channel-mismatch fix); GameStream surround unified onto the safe `opus::MSEncoder`, dropping `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround; WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask. - clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM → AVAudioEngine with an explicit wire-order channel layout; each gains a Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless validator. Verified on Linux: core/host/linux/probe test suites + the Android Rust (cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple builds, all on-glass checks, and the live native loopback are pending (CI / a free box). Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so cannot be committed separately from the surround changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:11:05 +00:00
enricobuehler	9e8135ccec	refactor(apple): code-quality pass — audit fixes + centralized defaults keys ... A 6-agent adversarial audit of the client (11 confirmed of 39 findings, the rest filtered) drove these: - fix: SessionAudio ring buffer — guard a write larger than the ring (would push readIdx past writeIdx and corrupt the buffer; never happens, but guard not corrupt). - fix: CADisplayLink retain cycle (stage-2 presenter) — a weak-target DisplayLinkProxy so the view can deallocate (the link retains its target); stage-2 teardown added to both StreamView/StreamViewController deinits as a safety net. - fix: GamepadFeedback deinit { flag.stop() } — the drain thread holds the connection strongly and self weakly, so an abrupt teardown without stop() would leak it. - refactor: centralize the 12 UserDefaults/@AppStorage key literals (scattered across 8 files) into one DefaultsKey enum — a typo silently splits a setting's reader from its writer. - docs: RumbleRenderer @unchecked Sendable invariant; the HID digit-row table; the stage-2 layer compositing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-12 16:30:34 +02:00
enricobuehler	bfd8c7be93	feat(apple): tvOS client — third app target, first-lit in the Apple TV simulator ... The same app now runs on tvOS (target Punktfunk-tvOS, bundle io.unom.punktfunk.tvos), validated live against the box: vkcube at 1280x720@60, 60 fps in the Apple TV 4K simulator, glass HUD with a focusable Disconnect button. - PunktfunkCore.xcframework grows tvOS device + universal-simulator slices. These are TIER-3 Rust targets (no prebuilt std): BUILD_TVOS=1 builds them with nightly and -Zbuild-std from rust-src — the full quic stack (quinn/rustls-ring/tokio) compiles for tvOS unchanged. - The UIKit stream view covers iOS AND tvOS, with pointer interaction, pointer lock, touch forwarding and InputCapture gated to iOS — tvOS is view-only until gamepad capture lands (the natural tvOS input). - SessionAudio on tvOS: .playback session, no mic (no app-accessible microphone). - App chrome gates: keyboardShortcut/textSelection/controlSize/statusBarHidden are iOS/macOS-only; host cards use the focus-native .card button style on tvOS; the Audio settings section hides (system-routed); mode seeding works from the TV screen (1920x1080@60). - Package platforms += .tvOS(.v17); new Xcode target + shared scheme (TARGETED_DEVICE_FAMILY 3, local-network usage description included). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 13:10:40 +02:00
enricobuehler	e1af4d57c6	feat(apple): iOS/iPadOS client — touch, pointer lock, shared SwiftUI shell ... The whole client now runs on iPadOS/iOS from the same sources, first-lit live in the iPad simulator against the real host at 1280x720@60 (60 fps on the HUD, capture state machine active, mic permission flow shown). - PunktfunkCore.xcframework grows iOS device + universal-simulator slices (BUILD_IOS=1; rustup targets aarch64-apple-ios{,-sim} + x86_64-apple-ios). - The decode pump is extracted into a shared StreamPump (identical IDR re-gate logic on both platforms); the iOS StreamView (StreamViewIOS.swift) has the same name/signature as the macOS one, so ContentView & co. are byte-identical across platforms — hosted in a UIViewController for prefersPointerLocked (the iPadOS cursor capture; see README note 9 for the UIHostingController forwarding caveat). - Touch is always forwarded: per-finger wire ids, coordinates mapped through the aspect-fit letterbox into LIVE host-mode pixels (surface == host mode, identity rescale host-side; follows mid-stream requestMode switches). - InputCapture is cross-platform: GC works the same on iPadOS, ⌘⎋ is detected from the HID stream there; stale-⌘ tracking after focus loss fixed on both platforms (releaseAll now drops the modifier/latch state — a ⌘ released in another app otherwise hijacked Esc forever). - SessionAudio: AVAudioSession on iOS (.playAndRecord + .defaultToSpeaker — without it iPhones route host audio to the EARPIECE; deactivated with notifyOthersOnDeactivation on stop so interrupted background audio resumes); HAL device pinning + the Settings pickers stay macOS-only. - New Punktfunk-iOS app target (shared synchronized sources, generated Info.plist with mic + local-network usage descriptions — QUIC to a LAN host trips local network privacy on real devices — scene manifest + indirect input events for Stage Manager / external displays), shared scheme, macOS min-window frames gated off iOS. For the iPad-on-an-external-screen idea: with multiple scenes + indirect input enabled, Stage Manager iPads can drag the punktfunk window onto the external display and drive the PC with keyboard/mouse/touch. Known gaps (README note 9): the pointer-lock preference isn't consulted through UIHostingController (relative mouse works, the local cursor just stays visible) and AVAudioSession interruptions don't auto-restart audio. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 11:18:25 +02:00
enricobuehler	b26f138699	feat(apple): session audio — host playback + mic uplink, device pickers in Settings ... Both directions of the audio plane, on CoreAudio's built-in Opus codec (kAudioFormatOpus — no bundled libopus; OpusCodec.swift, round trip unit-tested): - Playback: a drain thread pulls nextAudio() packets, decodes, and writes a priming jitter ring feeding an AVAudioSourceNode (~20 ms prefill, adaptive to the device's render quantum so large-buffer devices don't oscillate prime/dropout; a high-water clamp sheds stall backlog so one network hiccup can't permanently lag audio behind video; underrun re-primes — one dip, not sustained crackle). - Mic: a second engine taps the input device, resamples to 48 kHz stereo, Opus-encodes 20 ms chunks and sendMic()s them into the host's virtual PipeWire source. Permission via AVCaptureDevice (NSMicrophoneUsageDescription added to the Xcode target). - Settings: Speaker + Microphone pickers (CoreAudio HAL enumeration, persisted by device UID — "System default" leaves the engine unpinned so it follows macOS device changes) and a "Send microphone" toggle (default on). Applies from the next session. - Audio starts with streaming, never during the trust prompt (no host sound — and no mic uplink — before the user trusted the host); teardown stops audio before close(). Adversarial-review fixes baked in: stop() and the dangling mic-permission callback share one lock+flag protocol (no hot mic with no owner), the connect-success handler bails when the attempt was abandoned mid-handshake (no session/mic for a dead window), SessionAudio gets a deinit backstop (a dropped instance can't pin the connection via its drain thread), and the render scratch buffer is block-owned (was leaked per session). Verified live against the box: remote test decodes 100 host Opus packets to PCM and the host opens its virtual mic on the first uplinked frame ("punktfunk/1 virtual mic ready"); on-glass session runs with both engines up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 09:39:15 +02:00