7 Commits

Author	SHA1	Message	Date
enricobuehler	00acf5e44e	fix(host/audio): WASAPI virtual mic — port the priming jitter buffer (crackling fix) ... Mac → Windows mic passthrough crackled heavily while the identical stream was clean on the Linux host. Cause: clients push mic audio in BURSTS on their own clock (the Mac input tap yields ~two 20 ms Opus packets every ~42 ms) while the WASAPI render loop pulled a block every ~10 ms device period and greedily drained whatever was queued, padding the rest with zeros — the queue sat near-empty and most periods inserted mid-stream silence. The Linux backend has absorbed this since day one with its priming jitter buffer; the WASAPI loop had none. Port the same semantics: emit silence until ~48 ms is buffered (covers the worst inter-burst gap), then play from the cushion (zero-filling only a momentary shortfall), re-prime only after a genuine full drain (client went quiet). Queue cap raised 80 → 120 ms for burst headroom; steady-state added latency ≈ the 48 ms cushion. Diagnosed live on .173: probe tone recording from CABLE Output proved the endpoint wiring, then the burst-vs-period math explained the crackle. Build-verified on Windows; on-glass listen pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 22:31:18 +00:00
enricobuehler	2c7ded0f3c	fix(host/audio): rebuild mic passthrough — eager, self-healing virtual mic on both hosts ... Mic passthrough silently died on real hosts. Root causes, all fixed: - No liveness anywhere: a PipeWire restart (Linux) or any WASAPI device error (Windows) killed the backend worker; push() fed the dead queue for the rest of the host's life. VirtualMic now has a liveness contract (push -> bool, alive(), discard()) and the new shared audio::MicPump reopens with backoff, probing on an idle heartbeat so the mic heals BETWEEN sessions too. Validated live: systemctl restart pipewire -> node back in ~0.5 s, tone flows through the reopened backend. - Lazy creation: the mic device didn't exist until the first 0xCB frame, but games bind their capture device at launch and never re-follow. The pump opens eagerly at host start (node exists with zero clients, elected default source). - Windows headless dead-end: with VB-CABLE as the ONLY render endpoint (exactly what the installer ships), the anti-echo guard rejected the cable as the default render endpoint -> mic permanently dead. The new wiring_plan (pure, unit-tested on every platform) assigns the mic its endpoint FIRST (cable reserved for the mic), points the loopback at a DIFFERENT endpoint, and the capture side now yields (explicit endpoint or honest error) instead of the mic dying. Plan recomputed per (re)open — endpoints churn at boot/logon/driver installs. - Stale bursts: buffered audio from a previous session played into a newly-attached recorder (observed live). Timestamped chunks + a consumer-gap check in the process callback age everything past 1 s. The Linux node mechanism stays the stream-based Audio/Source with RT_PROCESS + priority.session: the canonical null-audio-sink adapter recipe was tested on this box (PipeWire 1.6.2) and never gets a clock (QUANT 0 -> pure silence), and WirePlumber reroutes a feeder targeting it to the default sink (echo). Decision documented in the module docs. Live-validated on this box (synthetic host + probe --mic-test, pw-record): eager node, both attach orderings, PipeWire-restart self-heal, post-session silence. Windows side compile/CI + on-glass validation pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-03 20:41:19 +00:00
enricobuehler	83ee53290e	feat(windows-host): mic passthrough — auto-wire audio devices + bundle VB-CABLE ... The Windows virtual mic worked only with manual Sound-settings fiddling: on a headless host (no real audio output) BOTH the desktop-audio loopback and the virtual mic must run on virtual cables, and on DIFFERENT ones or the loopback re-captures the injected mic (echo). The Steam pair gives only one usable cable (Steam Streaming Speakers loopback is silent — validated), so the mic + loopback collided and echoed, and when the default playback happened to be the mic device the anti-echo guard reported the mic "unavailable". Host now auto-wires the devices at startup (audio/windows/audio_control.rs, ensure_wired_once, hooked from open_audio_capture/open_virtual_mic): default playback = a loopback-capable render that is NOT a cable and NOT the dead Steam Speakers (real output > Steam Streaming Microphone); default recording = the mic capture (VB-Cable "CABLE Output" preferred). Uses a hand-rolled IPolicyConfig vtable (the only way to set a default endpoint; not in windows/wasapi crates). Opt out with PUNKTFUNK_KEEP_DEFAULT. wasapi_mic candidates now prefer "cable input". Validated live: from a deliberately-wrong start (playback=CABLE Input) the host corrected both default endpoints at the OS level. A Windows audio endpoint can only be created by a kernel-mode driver (no UMDF path — ACX is KMDF-only), so we cannot self-sign our own like the UMDF gamepad/ display drivers. Instead the installer bundles + silently installs the official base VB-CABLE (VB-Audio donationware, vendor-signed → loads with no test-signing, redistributed under VB-Audio's bundling grant): install-vbcable.ps1 (seed the VB-Audio cert into TrustedPublisher, run -i -h) + an installaudiocable task, gated on -VbCableDir/$env:VBCABLE_DIR (the package binary is not in the repo). Attribution in packaging/windows/licenses/VB-CABLE-NOTICE.txt. .iss compiles with the path enabled. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-30 09:09:26 +02:00
enricobuehler	75627c8afe	feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients ... Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client (previously stereo-only): - core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome `audio_channels` negotiation via the trailing-byte back-compat pattern (old peers fall back to stereo); C-ABI `punktfunk_connect_ex6`, `punktfunk_connection_audio_channels`, and in-core multistream decode `punktfunk_connection_next_audio_pcm` for embedders without a multistream Opus decoder. Real-libopus channel-identity round-trip test. - host: native audio thread captures + Opus-(multi)stream-encodes at the negotiated count (with a cross-session cached-capturer channel-mismatch fix); GameStream surround unified onto the safe `opus::MSEncoder`, dropping `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround; WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask. - clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM → AVAudioEngine with an explicit wire-order channel layout; each gains a Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless validator. Verified on Linux: core/host/linux/probe test suites + the Android Rust (cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple builds, all on-glass checks, and the live native loopback are pending (CI / a free box). Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so cannot be committed separately from the surround changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:11:05 +00:00
enricobuehler	3477cbe7ce	fix(audio/windows): stop the client mic echoing back through the loopback ... The Windows virtual mic fakes a capture endpoint by writing the client's uplinked PCM into a virtual device's *render* endpoint, while the desktop-audio plane loopback-captures the *default render* endpoint — with no mutual exclusion between the two. WASAPI loopback captures the mixed output of an endpoint (everything any app renders to it, including our mic writes), so when both resolve to the same device — VB-CABLE used for both, or the auto-installed Steam Streaming Microphone being the default render on a headless box — the injected mic is captured straight back into the host->client audio stream: an infinite echo. find_device() now resolves the loopback's endpoint id (default render) and skips any candidate matching it, scanning on to the next non-loopback match, so the mic can never land on the device the loopback reads. The auto-install path now provisions the full Steam pair (Streaming Microphone + Streaming Speakers) so a bare host gets two distinct devices instead of one shared one. Errors distinguish "no device" from "only candidate is the loopback device". Linux was already immune (its mic is a dedicated Audio/Source node, structurally separate from the monitored sink). Windows-only (#[cfg(windows)]); rustfmt-clean, compile-checked in windows-host CI, needs on-glass validation on the RTX box. Does not force the system default playback onto Steam Streaming Speakers (IPolicyConfig) — not required to break the echo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 23:51:46 +00:00
enricobuehler	327a5fa828	docs(host): prove unsafe blocks in the Windows + cross-platform files + gate them (unsafe-proof program 3/N) ... Continues the unsafe-proof program across the Windows/cross-platform host files (~75 blocks, 21 files), each with a SAFETY proof of the real invariant and a per-file #![deny(clippy::undocumented_unsafe_blocks)] gate: capture/windows: dxgi.rs, wgc_relay.rs, wgc.rs, desktop_watch.rs, composed_flip.rs (windows-rs COM: interface validity, same-D3D11-device textures, immediate-context single-thread, borrowed args outlive the call) windows: service.rs (SCM/token/CreateProcessAsUserW/event handles — OwnedHandle liveness, no double-close/signal race), win_display, wgc_helper, interactive vdisplay/windows: manager.rs, pf_vdisplay.rs (SwDeviceCreate/IddCx/ioctl handle liveness via the OnceLock VDM singleton + OwnedHandle) encode/windows: ffmpeg_win.rs (full AVBufferRef refcount audit — balanced, NO leaks, unlike the vaapi sibling), sw.rs cross-platform: gamestream/audio.rs (libopus), gamestream/stream.rs (sendmmsg), inject/windows/sendinput.rs, audio/windows/wasapi_mic.rs, session_tuning.rs, vdisplay.rs Two findings (handled separately): - wgc_relay.rs `unsafe impl Sync for HelperRelay` is UNSOUND (its mpsc Receiver is !Sync) though not live-exploited — marked SUSPECT inline; fix pending box check (it touches the in-flight punktfunk1.rs). - capture.rs / encode.rs (PARENT modules of the WIP idd_push.rs / nvenc.rs) do NOT get the file deny yet — it would propagate the lint into the undocumented WIP children. The deny lands there once those are documented (after the WIP commits). Linux-visible parts verified green (cargo clippy -p punktfunk-host --all-targets -- -D warnings). The cfg(windows) deny gates are box-verified next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 09:23:25 +00:00
enricobuehler	38c68c33e5	refactor(windows-host): confine platform code under windows/ + linux/ folders (Goal-1 stage 6) ... Move 36 platform-specific files into per-module `windows/` and `linux/` subfolders (and the shared HID codecs into `inject/proto/`): capture/{windows,linux}/ encode/{windows,linux}/ inject/{windows,linux,proto}/ audio/{windows,linux}/ vdisplay/{windows,linux}/ src/windows/ (service, wgc_helper, win_adapter, win_display) src/linux/ (dmabuf_fence, drm_sync, zerocopy/) Done with `#[path]`, NOT a module rename: every file moves into its folder while the `crate::*::*` module names stay FLAT, so all caller paths and every internal `super::`/`crate::` reference are unchanged — only the parent `mod` decls gained `#[path = "..."]`. This is the codebase's existing pattern (inject's gamepad_windows) and makes the move byte-identical in behaviour with ZERO reference churn, far lower risk than collapsing to a single `crate::capture::windows::` namespace (that deeper rename is an optional follow-on; this delivers the cfg-sprawl folder confinement the stage is about). Done LAST, after the semantic stages, so the path churn didn't fight them. Verified: Linux cargo check + clippy (-D warnings) clean; my mod-decl changes fmt-clean (the 3 remaining fmt diffs are pre-existing local-rustfmt-version skew that moved with their files); all 36 `#[path]` targets exist; no internal `#[path]`/`include!`/file-child-mod in any moved file (the inline `mod X {` blocks are self-contained). Box build to follow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 18:53:45 +00:00