9 Commits

Include renames

Author	SHA1	Message	Date
enricobuehlerandClaude Opus 5	2cfc82e96c	fix(audio): budget the audio plane against the link, and close the review's gaps ... Findings from the post-implementation review of design/audio-quality-and-latency.md. **The bandwidth gap (highest).** Tier `High` (256 kbps) and the redundant `0xD2` plane were added separately, each costed as "~1 % of the video budget", and nobody added them together: 256 kbps sent twice is 512 kbps — ~2.5 % of a 20 Mbps session but ~10 % of a 5 Mbps one. Audio rides QUIC datagrams, OUTSIDE the ABR loop, so ABR could neither see that nor reclaim it; a constrained link quietly handed a tenth of its bandwidth to audio while ABR carefully managed the rest. `plan_audio_budget` now makes tier and redundancy ONE decision against the session's resolved video bitrate, ordered by preference rather than cost — transparent audio beats redundant audio, since the field report was about quality and redundancy only pays under loss, so `High` alone outranks `Standard`+redundancy even though they cost the same. It can lower what the operator asked for, never raise it, and never goes below `Low`: a stream with unintelligible audio is worse than one spending a few percent more. **The Linux host kept the exact defect fixed on Windows.** `let _ = tx.try_send(samples)` — silent, uncounted data loss, where the encoder concatenates across the hole, so every drop is a click AND a permanent shift of everything after it. WP0.2 turned out to be Windows-only and had not said so. Linux now shares `capture_policy::CaptureStats`: drops counted and warned, plus per-window peak/RMS/delivered%. A Linux audio report was until now exactly as un-triageable as the Windows one was on 2026-08-03. **Apple's WP0.3 was half-done** — `bufferedMS` was added and wired to nothing. The drain thread now logs buffer/target/underruns/sheds like the other three, from one locked snapshot so the numbers in a line describe the same instant. Also: the Linux "audio format negotiated" line now says WHICH mode produced it, because that changes what it is worth — in stream-sink mode the host owns the sink so the mix cannot have been narrowed upstream, but in legacy monitor mode a 16 kHz Bluetooth sink would still be reported as a clean 48 kHz through PipeWire's resampler, the same way WASAPI's autoconvert hid it on Windows. Reading the monitored node's own rate needs a registry lookup this stream does not do; recorded as an open gap rather than implied to be covered. Two stale docs: `audio_wasapi.rs` cited `clients/windows/src/audio.rs` (deleted) and still described the pre-shared-policy "prime to ~3 quanta" behaviour. And the Apple ring's `prefill:` parameter, dead since the depth moved into the ring, is gone. Verified: clippy --all-targets -D warnings on Linux (docker) AND Windows (runner .133, forced clean rebuild of punktfunk-host + pf-client-core); core 167 tests; host 57 audio tests on Windows; Android clippy count identical to pristine (6, all documented arm64 artifacts); Apple ring re-simulated. The host suite's `gamestream::stream::tests::sender_delivers_batches` fails under qemu — the recorded environmental flake, unrelated to audio, green on the earlier less-loaded run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-04 17:58:17 +02:00
enricobuehlerandClaude Opus 5	a12f1f092c	feat(clients/audio): one de-jitter policy for all four rings, and lossless single-packet recovery ... Phase 4 + WP3.2 of design/audio-quality-and-latency.md. **The defect.** Every client ring primed *up* to a target and clamped at a ceiling, and none walked the depth back *down*. Any transient — a Wi-Fi arrival burst, a host stall, or plain host-DAC-vs-client-DAC skew of a few dozen ppm — therefore added latency permanently, until an underrun happened to re-prime. Android, with no shed at all, converged on its 120 ms hard cap and stayed there for the rest of the session; that is the "audio latency is too high" report. Apple did shed, 40 ms in one go, which its own comment called "one audible blip". All four now share `punktfunk_core::audio::JitterPolicy`: depths in MILLISECONDS rather than device quanta (`3 x quantum` meant 15 ms at a 5 ms quantum and a silent 64 ms at a 20 ms one), a crossfaded 5 ms shed once the depth average has sat above target for 2 s of consumed audio, and de-prime hysteresis. Linux and Windows had never had that hysteresis — they still carried the `if ring.is_empty()` instant re-prime that Android identified as self-inflicted crackle, where one transient drain manufactured a whole target's worth of silence. Android's floor drops 40 -> 25 ms: the policy grows the target on the devices that actually underrun, instead of every device pre-paying for the worst one. The Windows ring moves from raw bytes to interleaved f32 so it can share the policy and the crossfade helper at all. Apple is the one client where the policy is hand-written in a second language, so it gets its own XCTest (`AudioRingDriftTests`). Verified here by compiling `AudioRing.swift` standalone against a simulation harness — +200 ppm for 5 minutes settles at 30 ms with zero silent callbacks, where the old ring would have ridden its 80 ms high-water mark. **WP3.2 — recovery lives in core, not in the clients.** The rebuilt frame is re-inserted into the demux queue in order, so every embedder (including any C-ABI consumer) gets a complete stream without knowing the `0xD2` plane exists, and their `AudioGapTracker` simply stops seeing the gap. `recovery_and_the_gap_tracker_agree` pins exactly that. For the same reason core advertises CLIENT_CAP_AUDIO_RED itself rather than making four embedders remember to. Verified: clippy --all-targets -D warnings and the full test suites for punktfunk-core, pf-client-core, punktfunk-host, pf-host-config under Linux/docker (163 + 61 tests); punktfunk-client-android `cargo ndk check` for aarch64 with the gate proven non-vacuous by a planted type error, and its 6 clippy findings confirmed IDENTICAL to the pristine file (all are the documented arm64-only artifacts); AudioRing.swift type-checked and simulated on macOS; fmt. The Windows client half (audio_wasapi.rs) is still not compile-verified anywhere. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-04 09:28:01 +02:00
enricobuehlerandClaude Fable 5	766991cf6a	feat(apple): the mic has an off switch you can reach mid-stream ... Until now the only way to stop sending the room was to end the session and turn "Send microphone to the host" off in Settings — a per-app setting for something that is really a per-conversation act. The mic now mutes from inside the stream: a button on the HUD card, the Stream menu's ⌃⌥⇧A (macOS menu bar and iPad hardware keyboards), the same chord while input is captured (both platforms detect it in InputCapture, where the reserved ⌃⌥⇧Q/D/S already live — ⌃⌥⇧M is the mouse-model flip, cross-client, so the mic gets A), and on iPhone/iPad a mic disc beside the touch exit for the stats tiers whose HUD carries no buttons. Muting is local and instant: it gates capture on this device, the host is never asked and never told. The muted state gets its own badge over the stream — independent of the stats overlay, because "am I muted?" is not a statistic and the overlay is exactly what a player turns off. The badge is also the way back: tapping it unmutes, which is the guaranteed path for a touch user who muted with the overlay off. Mute is session state and is deliberately not persisted. Every stream starts live if the mic is enabled at all, rather than carrying a mute nobody remembers making into a call three days later. The mechanism is the one wave 1 built. `SessionAudio.setMicMuted` — which mutes the voice processor's input on the combined engine and pauses the capture engine on the split one — stays the single muting path; what changes is that it now takes an EFFECTIVE mute the session composes from its two reasons: the user's mute and the background keep-alive's privacy mute. Neither can clear the other, so a user who muted before pocketing their phone comes back still muted, and backgrounding no longer un-mutes anyone on return. It also latches the state, so a mute made while the microphone permission prompt is still open lands on the engine that grant creates instead of being lost. The control is offered only where there is something to mute: the session's resolved `micEnabled` (a profile can turn the mic on or off), a platform with an app-accessible input (never tvOS), and a TCC grant the OS hasn't refused. Absent rather than greyed on the HUD and the touch discs, greyed on the menu, and the macOS start-of-stream banner only teaches ⌃⌥⇧A when the session actually sends a microphone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-31 22:35:48 +02:00
enricobuehlerandClaude Fable 5	f3e122b0d0	feat(apple): the mic hears you, not the game — voice processing joins the engines ... The Apple client on a loudspeaker was the primary reported echo source: two AVAudioEngines by design (arbitrary mic/speaker pairs, two clocks) meant no unit ever saw render and capture together, which is exactly what setVoiceProcessingEnabled needs. With the mic enabled and the new "Echo cancellation" toggle on (both defaults), playback and capture now share ONE engine with the system voice processor engaged — AEC, noise suppression and AGC — and the input node's other-audio ducking pinned to .min with advanced ducking off, because this is a game stream, not a call: the host's audio must never dip under the outgoing voice. The two-engine path survives where it is the right answer: echo cancel off, macOS sessions with a hand-picked speaker/mic UID or input channel (the voice processor only follows the system default devices, and its capture side is its own mono mix — a per-channel pick can't survive it), and any machine whose voice processor refuses to engage. Every failure falls back to a working configuration rather than silence. The capture chain reads the input format after the processor is enabled, so its mono/lower-rate output flows through the existing converter unchanged; a first-run permission grant swaps the playback-only engine for the combined one in place, reusing the same jitter ring and drain thread; background mic muting mutes the processor's input instead of pausing the shared engine (playback keeps running). echo_cancel rides the full settings plumbing micEnabled has — defaults key, effective settings, profile overlay (serialized as `echo_cancel`; the Rust catalog carries it via unknown-key passthrough until it grows the field), a captioned toggle beside the Microphone row, and a gamepad settings row. tvOS behavior is untouched (playback only, as before). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-31 21:58:31 +02:00
enricobuehlerandClaude Fable 5	e54258b8ac	feat(apple): the mic uplink drops to mono 10 ms packets and stops shoveling its FIFO ... The capture path carried three latencies that had nothing to do with the network: a 2048-frame tap request (42.7 ms bursts before the first sample could even be sliced), 20 ms Opus framing, and a mono signal duplicated into both stereo channels because the encoder was configured like the downlink. CoreAudio's Opus encoder takes mFramesPerPacket=480 and mChannelsPerFrame=1 just fine — probed empirically: the converter truly emits 10 ms mono CELT packets (TOC config 30), one per 480-frame chunk, and the stereo-shaped decoder upmixes them with the tone intact — so the uplink now asks for 10 ms tap buffers and encodes 48 kbps mono 10 ms packets, with 960 kept only as an init-time fallback. The host decodes any Opus frame ≤120 ms, so nothing changes on the wire's far end. The tap thread also stops burning cycles per callback: the mono fold and resampler scratch buffers are allocated once (regrown only if a larger device quantum ever arrives), and the chunk slicer walks a head index instead of removeFirst — which memmoved the entire backlog for every packet on a render-adjacent thread. iOS additionally asks the session for 5 ms IO quanta at 48 kHz when the mic is on (best-effort; the hardware decides). Verified: swift build (macOS arm64), swift test OpusCodecTests + AudioChannelFoldTests, and an iOS arm64 cross-build; the 480/mono behavior confirmed by TOC inspection on macOS 15. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-31 21:57:48 +02:00
enricobuehlerandClaude Opus 4.8	14c5e7c11c	feat(apple/M2): opt-in background keep-alive (audio + video-drop + timeout) ... Backgrounding a live session no longer freezes it when the user opts in: audio keeps playing (UIBackgroundModes audio), the QUIC connection + pump stay live, video decode is DROPPED, and a bounded timer auto-disconnects. Off by default. - PunktfunkConnection.setVideoDropped/isVideoDropped: a tiny lock-guarded flag both pumps read every iteration. StreamPump (stage-1), Stage2Pipeline (VT + PyroWave) drain nextAU() for flow control but DISCARD the AU before any VideoToolbox/Metal work — the crash/jetsam-safe seam (no GPU off-screen). - SessionModel.enterBackground(timeoutMinutes:) / exitBackground(): set the drop flag, mute the mic (privacy — SessionAudio.setMicMuted pauses the capture engine), arm a DispatchSourceTimer that disconnect(deliberate:false)s on fire (keeps host linger → fast late reconnect). exitBackground clears the flag and requestKeyframe()s; the pump's freeze gate auto-arms on the resumed frame-index gap so concealed frames are withheld until the IDR re-anchors. disconnect() cancels the timer + clears isBackgrounded. - ContentView scenePhase driver (iOS): .background+streaming+setting → enterBackground; .active → exitBackground. scenePhase (not willResignActive) so Control-Center/app-switcher peeks don't start the timer. - Settings → General (iOS-only keepAliveSection): toggle + 1/5/10/30 timeout; new keys backgroundKeepAlive (def off) / backgroundTimeoutMinutes (def 10). - Info.plist: UIBackgroundModes [audio] + NSSupportsLiveActivities (for M3). macOS swift build + swift test green (142 tests). The iOS-gated scenePhase handler + settings section are not exercised by the macOS CI target (known §9 gap) — need on-glass verification (audio never gaps, video re-anchors <1s LAN, timeout ends the session, phone-call audio-steal degrades gracefully). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-16 18:41:46 +02:00
enricobuehlerandClaude Fable 5	3ba19f28a2	feat(apple): the gamepad UI comes to tvOS - focus-driven, with real session controls ... The console UI now runs on tvOS through the NATIVE focus engine: carousel cards and settings rows are focusable Buttons (Siri Remote and pads both navigate; imperative scrollTo replaces the drop-prone scrollPosition binding), while iOS/macOS keep the 60 Hz poll untouched - on tvOS it carries only what focus has no concept of: X/Y screen actions and left/right value adjust with the poll's dominant-axis feel (onMoveCommand proved input-source-dependent: keyboard intercepted, pad dpad not -> double steps). Text entry uses the system fullscreen keyboard (TVTextEntry); pairing + library present as covers under the launcher; the game library defaults ON; settings values slide a quiet 14 pt in the step's direction. Session controls: controller/remote input routes EXCLUSIVELY through GameController during a stream (GCEventViewController, interaction disabled) - a pad's B no longer doubles as a UIKit menu press that ended sessions mid-game. Deliberate exits only: the cross-client escape chord (hold L1+R1+Start+Select 1.5 s - pf-client-core's contract, now implemented on all Apple platforms) and holding the remote's Back >= 1 s; the start-of-stream banner (now also on tvOS) teaches both. The Siri Remote's touch surface drives the host pointer - press = left click, Play/Pause = right click, release-tail jumps gated so motion stays truly relative. tvOS 26 regressions fixed at the root: the app-wide brand tint rendered every unfocused control as a blank pill (tint dropped on tvOS) and the 17 pt root font shrank the whole platform (29 pt there), plus 10-foot sizing across host cards, the gamepad screens, and the stats HUD (whose misleading "Press Menu" hint is gone). Acknowledgements scrolls by focus-sized chunks and Menu pops instead of suspending; full-width focusSections make the home actions reachable from any column. The presenter defaults to stage-3 glass pacing on tvOS (a 60 Hz panel fed a 60 fps stream is the sticky-FIFO worst case behind the 50 ms display stage) and is pickable from the gamepad settings; HDR capability advertises from AVPlayer.eligibleForHDRPlayback instead of the current mode's EDR headroom, so an SDR home screen no longer hides an HDR TV. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 16:59:01 +02:00
enricobuehlerandClaude Opus 4.8	e56470cf56	fix(apple-client/audio): capture the right channel of a multi-channel mic + diagnostics ... The mic uplink handed the host pure digital silence on a multi-channel interface: AVAudioConverter's N→stereo downmix takes channels 0/1, but a pro interface puts the mic on ONE higher discrete channel. Fold the input to a mono bus ourselves instead — pick the mic's channel (or sum all) and resample that to the encoder's 48 kHz stereo, so the silent 0/1 downmix never happens. - New "Microphone channel" setting (macOS): Auto (sum every channel — a lone hot mic passes at full level) or pin 1-based channel N. Picker appears only for multi-channel devices, driven by the device's input channel count. - Diagnostics that make this class of failure self-naming next session: log the actual live capture device + format + fold mode, warn on a silent UID fallback, and a one-shot silence tripwire on the EXTRACTED signal (WARN on 10 s of zeros, else peak dBFS). - foldToMono extracted as a pure, unit-tested helper (pin / sum-clamp x interleaved / deinterleaved / mono / out-of-range). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-04 00:36:35 +02:00
enricobuehlerandClaude Fable 5	34fe896ade	feat(apple): gamepad UI v2 — controller settings + add host, aurora, macOS ... Sources reorganized (client: Home/Session/Settings/Stores/Support/Trust; kit: Audio/Connection/Gamepad/Input/Support/Video/Views) with the big files split along the same seams. The gamepad mode is couch-complete, and now on macOS too (the living-room Mac case), not just iOS/iPadOS: - GamepadSettingsView: a console-style, fully controller-navigable settings screen (X from the launcher) — up/down moves focus, left/right steps values (clamped, boundary thud), A cycles/toggles, B closes; the focused row shows a one-line description. Backed by GamepadMenuList, the vertical sibling of GamepadCarousel, and SettingsOptions — the option lists hoisted out of SettingsView statics and shared by the touch, tvOS and gamepad settings. - GamepadAddHostView + GamepadKeyboard: register a host end to end with a pad — field rows open an on-screen controller keyboard (dpad grid, A types, X backspaces, B done); the launcher carousel ends in an Add Host tile, so the dead-end "add one with touch first" empty state is gone. - Launcher polish: contextual hint bar with the pad's real button glyphs, controller name + battery chip, one shared console chrome. - GamepadScreenBackground: an animated aurora (TimelineView-driven drifting blobs in the brand's violet family, breathing radii, slow hue shift, legibility scrim; freezes under Reduce Motion). Pure SwiftUI on purpose — a .metal library only bundles reliably in one of the two build systems (SPM vs the xcodeproj's synced folders) these sources compile under. - macOS port: settings/add-host/library present as sized sheets (a macOS sheet takes its content's IDEAL size, and the GeometryReader-driven screens collapsed to nothing), NSScreen-based mode lists, scroll indicators .never (the "always show scroll bars" setting overrides .hidden), tray scrims so scrolled rows dim under the pinned title/hints, extra title clearance, and a PUNKTFUNK_FORCE_GAMEPAD_UI=1 dev hook — launcher/settings/add-host/keyboard/ library render-verified live on a real Mac + LAN hosts. - GamepadMenuInput: X button support, and (re)start now snapshots held buttons so a controller handoff press never fires twice (the B that closed the keyboard no longer also cancels the screen underneath). - Cleanups: one "Connection failed" alert in ContentView instead of one per home screen; HostDiscovery.advertises/unsaved shared by both home screens. - host: can_encode_444 stub for the non-Linux/Windows host build (the macOS synthetic-source loopback used by the Swift tests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 11:24:44 +02:00