Root cause of the field report (Android client, Windows host: no audio until
the user manually cycled Sound-output devices, then audio on BOTH PC and
phone): the WASAPI loopback captured whatever the default render endpoint
was at open time — not the wiring plan's chosen endpoint — the plan's
IPolicyConfig default-set is warn-only and racy, and nothing reacted to
mid-stream default-device changes.
- Explicit-endpoint capture: the capture thread opens the plan's
loopback_render by id, never "the default" (KEEP_DEFAULT preserves the old
default-capturing behavior with the echo guard).
- Client-only playback default: wiring_plan::plan(..., host_audio) prefers a
silent sink (Steam Streaming Microphone render side — loopback-validated,
silent on host) over real hardware, so stream audio plays on the client
only; PUNKTFUNK_HOST_AUDIO=1 restores real-hw-first (audible on the host).
The capture side auto-installs the Steam pair once per process when no
silent sink exists; open() handshake timeout 3s -> 30s to cover it, and a
handshake timeout now stops the detached thread (it used to run for the
process lifetime with the default still parked).
- Self-healing capture thread (wasapi_cap): outer capture_once loop
(Assert|Follow) with a ~1s watchdog on the default render id. A user
switch to a capturable endpoint is followed (their choice wins, audio on
both); a switch to a dud (cable/SSS/mic target) re-asserts the plan;
IPolicyConfig-denied converges to Follow instead of churning. Device
errors reopen with 2s backoff; only the FIRST open failure is fatal.
Zero-packets breadcrumb after 30s distinguishes broken-loopback from
quiet-desktop.
- Park/restore of the default playback device (audio_control):
wire_now(set_playback) parks the default on the loopback sink only for the
capture's lifetime (the mic pump passes false — it runs while the host is
idle); crash marker audio-default.prev + recover_orphaned_default() at
first wire; restore is skipped if the operator changed the default
themselves. A mic-default hygiene pass keeps VB-Cable installs audible and
never records the mic target as the restore target.
- Session-end park_audio_capture(): Windows DROPS the capturer (thread join
restores the default) instead of caching it; Linux keeps the parked
PipeWire thread. Composes with the stream-sink idle() hook at all three
park sites (idle is a no-op on Windows).
Verified: Linux (.21) clippy -D warnings + 176 punktfunk-host tests green
(incl. the new wiring-plan preference tests); Windows (.173) clippy with
nvenc,amf-qsv --all-targets -D warnings green at this exact tree.
On-glass winbox/Android validation still owed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Merges 44b71e74 (see its message for the full design: the "Punktfunk Stream
Speaker" Audio/Sink claimed as the session's default output — the crackle
root fix for hardware-sink churn — plus the core-error liveness fix and true
5.1/7.1 capture) across the W1–W8 refactor that landed since the branch
forked:
- punktfunk1.rs was split on main: the audio_thread idle()/drain() hunks are
hand-ported into native/audio.rs (same three sites: reuse-drain comment,
encoder-fail park, end-of-thread park).
- pwinit moved to pf-capture (W6.2): the branch's crate::pwinit calls in
pw_thread and stream_sink.rs become pf_capture::pwinit.
- Kept main's log-tier demotions (stream-state + setup lines at debug)
inside the restructured pw_thread.
Resolution verified on Linux (home-worker-5): clippy -D warnings clean +
173 punktfunk-host tests green at the 691c064a-based equivalent ba5973a2;
re-verified at this base before landing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per plan §2.1: a self-contained stateful subsystem does not belong in the
audio trait facade. Move MicPump + its PumpTuning/PUMP_TUNING, the
drain_sleep/pump_thread loop, MIC_CHANNELS/MIC_QUEUE_CAP, and the six pump
unit tests out of audio.rs into audio/mic_pump.rs. audio.rs keeps the
AudioCapturer/VirtualMic traits, their open_* factories, and the sample
constants. Re-export via `pub use mic_pump::MicPump` so crate::audio::MicPump
stays byte-stable (only consumer: punktfunk1.rs). Pure code-move; verified
clippy 0/0 + 6/6 pump tests green on Linux (home-worker-5).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Linux desktop-audio capture stream now registers itself as an Audio/Sink
node ("Punktfunk Stream Speaker", default-on, PUNKTFUNK_STREAM_SINK=0 =
legacy escape hatch) and claims the configured default sink for the duration
of a session (saved and restored around it, refcounted across concurrent
sessions, crash-stale claims degrade to automatic election). Host apps play
directly into the capture stream, so the capture link no longer depends on
any hardware sink.
Root cause this fixes (live-diagnosed on a bazzite/LG-TV host): gamescope
modesets drop the NVIDIA HDMI audio endpoint, WirePlumber ping-pongs the
default sink HDMI<->auto_null ~8x/s, and the old monitor-follower relinked
its capture on every flip - Paused/renegotiate/Streaming storms (~1300 log
lines/min) heard as crackle on the client.
Also fixes a latent liveness bug in both modes: the capture thread had no
core-error listener, so a PipeWire daemon restart mid-session left a zombie
thread returning quiet-sink empty chunks forever (same class as the historic
virtual-mic death bug). Now the thread exits and sessions reopen with backoff.
Bonus: the sink advertises the session's true channel count, so games can
produce real 5.1/7.1 even when local hardware is stereo. New AudioCapturer::
idle() hook releases the routing claim when a capturer is parked between
sessions; drain() re-claims on reuse.
Verified: cargo clippy -D warnings + 295 tests green on Linux (.21).
On-glass validation on the bazzite host pending.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Found by a live boot-order test (host started before the user session's
PipeWire): PwMicSource::open returned Ok before the daemon connection was
attempted, so a PipeWire that wasn't running surfaced as an instantly-dead
instance instead of an open failure — and the pump churned
open→die→reopen at heartbeat rate (1 Hz "virtual mic ready" log spam)
instead of backing off.
- PwMicSource::open now has a bring-up handshake (mirrors the Windows
backend): ready only after connect + stream connect succeed, so a
down daemon is an open ERROR and the pump's backoff engages.
- The pump triages deaths: an instance that lived >= 5 s (a one-off
daemon restart) reopens immediately with the backoff reset; one that
died right after opening counts as a failed open and backs off
(2 s → 60 s cap). New pump test rapid_death_backs_off.
Re-validated live: host started with PipeWire stopped → throttled
"unavailable" warns, zero churn; daemon started → mic node up on the
next retry; exactly one pump + one loop thread (no leak).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mic passthrough silently died on real hosts. Root causes, all fixed:
- No liveness anywhere: a PipeWire restart (Linux) or any WASAPI device
error (Windows) killed the backend worker; push() fed the dead queue
for the rest of the host's life. VirtualMic now has a liveness
contract (push -> bool, alive(), discard()) and the new shared
audio::MicPump reopens with backoff, probing on an idle heartbeat so
the mic heals BETWEEN sessions too. Validated live: systemctl restart
pipewire -> node back in ~0.5 s, tone flows through the reopened
backend.
- Lazy creation: the mic device didn't exist until the first 0xCB
frame, but games bind their capture device at launch and never
re-follow. The pump opens eagerly at host start (node exists with
zero clients, elected default source).
- Windows headless dead-end: with VB-CABLE as the ONLY render endpoint
(exactly what the installer ships), the anti-echo guard rejected the
cable as the default render endpoint -> mic permanently dead. The new
wiring_plan (pure, unit-tested on every platform) assigns the mic its
endpoint FIRST (cable reserved for the mic), points the loopback at a
DIFFERENT endpoint, and the capture side now yields (explicit
endpoint or honest error) instead of the mic dying. Plan recomputed
per (re)open — endpoints churn at boot/logon/driver installs.
- Stale bursts: buffered audio from a previous session played into a
newly-attached recorder (observed live). Timestamped chunks + a
consumer-gap check in the process callback age everything past 1 s.
The Linux node mechanism stays the stream-based Audio/Source with
RT_PROCESS + priority.session: the canonical null-audio-sink adapter
recipe was tested on this box (PipeWire 1.6.2) and never gets a clock
(QUANT 0 -> pure silence), and WirePlumber reroutes a feeder targeting
it to the default sink (echo). Decision documented in the module docs.
Live-validated on this box (synthetic host + probe --mic-test,
pw-record): eager node, both attach orderings, PipeWire-restart
self-heal, post-session silence. Windows side compile/CI + on-glass
validation pending.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Windows virtual mic worked only with manual Sound-settings fiddling: on a
headless host (no real audio output) BOTH the desktop-audio loopback and the
virtual mic must run on virtual cables, and on DIFFERENT ones or the loopback
re-captures the injected mic (echo). The Steam pair gives only one usable cable
(Steam Streaming Speakers loopback is silent — validated), so the mic + loopback
collided and echoed, and when the default playback happened to be the mic device
the anti-echo guard reported the mic "unavailable".
Host now auto-wires the devices at startup (audio/windows/audio_control.rs,
ensure_wired_once, hooked from open_audio_capture/open_virtual_mic): default
playback = a loopback-capable render that is NOT a cable and NOT the dead Steam
Speakers (real output > Steam Streaming Microphone); default recording = the mic
capture (VB-Cable "CABLE Output" preferred). Uses a hand-rolled IPolicyConfig
vtable (the only way to set a default endpoint; not in windows/wasapi crates).
Opt out with PUNKTFUNK_KEEP_DEFAULT. wasapi_mic candidates now prefer "cable
input". Validated live: from a deliberately-wrong start (playback=CABLE Input)
the host corrected both default endpoints at the OS level.
A Windows audio endpoint can only be created by a kernel-mode driver (no UMDF
path — ACX is KMDF-only), so we cannot self-sign our own like the UMDF gamepad/
display drivers. Instead the installer bundles + silently installs the official
base VB-CABLE (VB-Audio donationware, vendor-signed → loads with no test-signing,
redistributed under VB-Audio's bundling grant): install-vbcable.ps1 (seed the
VB-Audio cert into TrustedPublisher, run -i -h) + an installaudiocable task,
gated on -VbCableDir/$env:VBCABLE_DIR (the package binary is not in the repo).
Attribution in packaging/windows/licenses/VB-CABLE-NOTICE.txt. .iss compiles
with the path enabled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move 36 platform-specific files into per-module `windows/` and `linux/` subfolders (and the
shared HID codecs into `inject/proto/`):
capture/{windows,linux}/ encode/{windows,linux}/ inject/{windows,linux,proto}/
audio/{windows,linux}/ vdisplay/{windows,linux}/
src/windows/ (service, wgc_helper, win_adapter, win_display)
src/linux/ (dmabuf_fence, drm_sync, zerocopy/)
Done with `#[path]`, NOT a module rename: every file moves into its folder while the
`crate::*::*` module names stay FLAT, so all caller paths and every internal `super::`/`crate::`
reference are unchanged — only the parent `mod` decls gained `#[path = "..."]`. This is the
codebase's existing pattern (inject's gamepad_windows) and makes the move byte-identical in
behaviour with ZERO reference churn, far lower risk than collapsing to a single
`crate::capture::windows::` namespace (that deeper rename is an optional follow-on; this delivers
the cfg-sprawl folder confinement the stage is about). Done LAST, after the semantic stages, so
the path churn didn't fight them.
Verified: Linux cargo check + clippy (-D warnings) clean; my mod-decl changes fmt-clean (the 3
remaining fmt diffs are pre-existing local-rustfmt-version skew that moved with their files); all
36 `#[path]` targets exist; no internal `#[path]`/`include!`/file-child-mod in any moved file
(the inline `mod X {` blocks are self-contained). Box build to follow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two bodies of work in one commit (the rename moved files the fixes also touched).
Naming/structure cleanup (pre-launch):
- Host modules m3.rs->punktfunk1.rs, m0.rs->spike.rs; CLI m3-host->punktfunk1-host,
m0->spike; bare `punktfunk-host` now prints help. Types M3Options/M3Source->
Punktfunk1Options/Punktfunk1Source.
- Clients consolidated out of crates/ into clients/: punktfunk-client-rs->
clients/probe (crate punktfunk-probe), client-linux->clients/linux,
client-windows->clients/windows, punktfunk-android->clients/android/native
(crate punktfunk-client-android; kept [lib] name=punktfunk_android so the JNI
contract is unchanged). crates/ now holds only core + host.
- Milestone codes M0-M4 purged from code/CLI/CLAUDE.md/README/docs/docs-site,
kept only in docs/implementation-plan.md. docs/m2-plan.md->
docs/gamestream-host-plan.md. CI/gradle/flatpak paths updated.
Client loss-recovery (video froze and never recovered after a brief drop):
- Export punktfunk_connection_frames_dropped through the C ABI (the core already
tracked it for the client keyframe-recovery loop; it was never reachable from
the ABI clients). Regenerated punktfunk_core.h.
- Apple (StreamPump + Stage2Pipeline) and Android (decode.rs) now poll
frames_dropped and request a keyframe when it climbs -- the same loss-driven
recovery Linux/Windows already had. Under infinite GOP the decoder silently
conceals reference-missing frames, so the decode-error trigger rarely fires.
Apple rumble robustness (worked then went spotty -- DualSense + Xbox):
- Add CHHapticEngine stopped/reset handlers (rebuild on app background / audio
interruption / server reset) and drop the permanent `broken` latch on a
transient drive failure; latch only when the controller truly has no haptics.
- Surface swallowed SDL set_rumble errors on Linux/Windows + diagnostic logging.
Verified: cargo build/clippy/fmt --workspace, C-ABI harness, header drift.
Not runnable on this box (verify in CI): Gitea workflows, gradle/Android,
flatpak, Swift/decky.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The host received the client's mic uplink (0xCB Opus) but dropped it on Windows ("requires
Linux"). Windows has no user-mode way to CREATE a capture endpoint, so target an existing
virtual audio device and write the decoded mic PCM into its RENDER endpoint — the device's
CAPTURE endpoint then surfaces as a microphone host apps record from (the inverse of a
virtual cable). New audio::wasapi_mic::WasapiVirtualMic: finds the device by friendly-name
(Steam Streaming Microphone / VB-Audio CABLE Input / VoiceMeeter / "virtual", override with
PUNKTFUNK_MIC_DEVICE), opens a WASAPI shared event-driven RENDER client (48 kHz stereo f32,
autoconvert), and a dedicated COM thread writes a bounded (~80 ms drop-oldest) inject queue
with silence-fill. open_virtual_mic() gets a Windows arm; mic_service_thread (Opus decode →
push) now compiles for windows too (opus is already a windows dep). Clear error + install
guidance when no virtual device is present.
Linux/cross-platform side cargo-checks; the Windows path is built/validated when the box is
back (the wasapi render API was cross-checked against the docs + the existing capture path).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two steady-state faults previously bubbled a bare `?` to conn.close / silently
muted the rest of a session. Recover in place instead.
#4 — capture loss (virtual_stream): a mid-session capture stall/disconnect
(`try_latest` Err: PipeWire/compositor thread ended, virtual output gone) ended
the whole session — and the native client has no reconnect path, so it had to
cold-restart the handshake. Now rebuild the pipeline IN PLACE at the current
mode via build_pipeline_with_retry (same primitive the mode/session switch uses),
force a keyframe, and only propagate when the bounded retry is exhausted. A
consecutive-rebuild cap stops a flapping source from looping the client through
endless cold IDRs. Track the live mode so a rebuild after a mode switch targets
the right mode (also fixes the session-switch rebuild using the stale mode).
#3 — native audio thread (audio_thread): broke the loop on ANY next_chunk Err,
spawned once per session and never restarted, so a transient 5 s quiet-sink
timeout permanently muted a multi-hour session. Make a quiet sink return an empty
chunk (not an Err) in both backends so only a genuinely dead capture thread is an
Err, and reopen-with-backoff (INJECTOR_REOPEN_BACKOFF) on death, keeping the Opus
encoder + monotonic seq. Documents the next_chunk contract; also makes the
GameStream audio sender survive quiet sinks for free.
Resolves reliability backlog #3 and #4.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Windows AudioCapturer via the wasapi crate (0.23): loopback the default render endpoint (Render device + Direction::Capture + shared mode => STREAMFLAGS_LOOPBACK) at 48 kHz stereo f32 with autoconvert, feeding the existing Opus path with no resampling. Dedicated COM-MTA thread owns the !Send WASAPI objects; interleaved f32 chunks leave over a bounded lossy channel; RAII Drop stops + joins. Bring-up handshake reports a missing endpoint as Err so a session continues without audio. open_audio_capture Windows factory arm + module. Init chain validated live on the VM (open succeeds; next_chunk waits on a silent system). Virtual mic deferred (no Windows virtual-audio endpoint). m3 audio_thread wiring + opus hoist land with the integration task.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a
new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire
Audio/Source its apps can record from (voice chat, etc.).
Protocol (punktfunk-core):
- MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram).
- NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task
(mirror of send_input); C ABI punktfunk_connection_send_mic for native clients.
Host:
- audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class=
Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM.
- MicService: host-lifetime owner of the source + Opus decoder (mirror of
InjectorService / the audio capturer slot); lazily opened, persists across sessions,
self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a
single read_datagram loop (two loops would race).
- Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting,
so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output
was silence; with it, glitch-free across consumer quanta.
Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic
uplink (opus dep added) for end-to-end validation without a real microphone.
Validated live on headless KWin: client tone → host source → pw-record shows the
punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001
elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms
consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codec negotiation (M2 polish):
- ServerCodecModeSupport now advertises what we encode: H264|HEVC|AV1_MAIN8
= 65793 (flags verified against moonlight-common-c Limelight.h). The old
placeholder 3843 wrongly claimed HEVC Main10 + 4:4:4 and no AV1. Main10
bits stay off on purpose: Moonlight ties 10-bit to HDR, and capture is
8-bit SDR BGRx with no HDR metadata path (av1_nvenc -highbitdepth was
validated working for later).
- RTSP ANNOUNCE: bitStreamFormat 0/1/2 -> H264/HEVC/AV1 (already plumbed to
av1_nvenc; validated e2e via `m0 --codec av1` + ffprobe av01), and a
dynamicRangeMode!=0 request now logs + falls back to 8-bit SDR.
Surround audio (M2 polish):
- ANNOUNCE x-nv-audio.surround.{numChannels,AudioQuality} +
x-nv-aqos.packetDuration -> per-session AudioParams; DESCRIBE advertises
all six Opus configs (normal before HQ per channel count). Normal-quality
mappings are pre-rotated for the client's GFE-order LFE swap
(RtspConnection.c, verified verbatim) so its derived decoder mapping
equals our encoder mapping — including 7.1, where Sunshine's rotate only
covers [3,6) and scrambles LFE/SL/SR.
- 5.1/7.1 encode via libopus multistream (audiopus_sys, the sys layer the
opus crate already links) with Sunshine's layouts/bitrates, RAII wrapper;
the live-validated stereo wire is byte-identical (plain Opus, no FEC).
- Surround sessions add Sunshine-style RS(4,2) audio FEC (packetType 127 +
AUDIO_FEC_HEADER, the OpenFEC parity matrix both ends hardcode, nanors
gemm semantics verified from nanors/rs.c).
- PipeWire capture generalized to the negotiated channel count with explicit
FL FR FC LFE RL RR [SL SR] positions; missing sink channels are zero-
filled by the channel-mixer. PwAudioCapturer now tears down cleanly on
Drop (pipewire channel -> loop quit), so a channel-count change can
reopen without leaking a capture stream.
Tests: serverinfo mask, RTSP codec/audio param parsing, DESCRIBE contents,
surround-params strings + client-swap round trip, FEC parity self-recovery
and packet layout, real-codec 5.1 channel-identity round trip, and an
ignored live test (ran green against a 6ch null sink monitor).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Full project rename, decided 2026-06-10:
- Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs.
- C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h,
PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl.
PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants).
- Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1.
WIRE BREAK: clients must be rebuilt from this revision.
- Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / ….
- Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the
persistent identity is unchanged, pinned fingerprints stay valid).
- Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection
(Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated.
- scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated.
Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of
"desktop but no apps/settings" over the stream: plasmashell launched without
XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and
rendered an empty menu. The script sets the complete KDE session env (menu prefix,
KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell.
Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS,
zero lumen references left outside .git.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler