Field reports: Windows AMD/Intel streams freeze after ~3-5 min regardless of
desktop activity. Root cause: the libavcodec AMF/QSV poll is non-blocking
(EAGAIN -> Ok(None)), and the encode loop's drain treated None as benign
without popping `inflight` — a wedged driver (QueryOutput stops producing)
meant frames kept being submitted, inflight grew unboundedly, no AU ever
reached the send thread, and nothing logged: a silent permanent freeze. The
input-side twin: once libavcodec's one-frame buffer fills, avcodec_send_frame
EAGAINs and the submit `?` killed the whole session.
Add `Encoder::reset()` (in-place encoder rebuild; implemented for AMF/QSV by
dropping the wedged libavcodec encoder so the next submit re-opens it on the
current device, forced IDR) and an encode-stall watchdog in the stream loop:
trip on a poll error, on no AU within max(2 s, 8 frame intervals) while frames
are owed, or on an owed backlog worth more than the window's frames (the
slow-leak latency-runaway form). Recovery is a bounded (5 consecutive, cleared
by any delivered AU) in-place rebuild + forced IDR — a logged ~one-second
hiccup instead of a dead stream; exhaustion or a reset-less backend still
fails the session with a clear error. Submit failures route through the same
bounded recovery. The three existing pipeline-rebuild paths (session switch,
mode switch, capture loss) now also clear the stale in-flight records that
pointed at the dropped encoder.
Backends whose poll blocks (direct NVENC sync, software) can't false-trip:
they never return Ok(None) mid-stream and drain inflight below depth each
tick. Validated: clippy -D warnings (nvenc,amf-qsv), 191 host tests, synthetic
E2E 300/300 frames, and an on-glass AMD iGPU session (1080p120 HDR hevc_amf).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On-box A/B on the .173 Ryzen 7000 iGPU (720p60, real composition via input
injection — an idle virtual desktop composes ~1 fps and gives meaningless
encode timings): the encode-time-first `quality=speed` preset + explicit `bf=0`
cut host-side encode_us from ~36 ms to ~19.5 ms.
The blocking-poll idea from the prior commit was WRONG and is reverted to a
single non-blocking receive (default PUNKTFUNK_FFWIN_POLL_MS=0): libavcodec's
hevc_amf holds ~2 frames before releasing the oldest (needs frame N+2 to flush
N), so a spin between submits provably never yields the owed AU — verified with
a 150 ms cap pegging at exactly 150 ms across every usage preset and pipeline
depth. That ~2-frame buffer is inherent to the libavcodec wrapper, not host
scheduling; the real latency lever is a direct AMF SDK encoder (the AMF
analogue of the direct-NVENC path), tracked as the next AMD work item. The
env knob is retained for a future VCN/driver where a bounded spin can help.
Also measured and rejected: PUNKTFUNK_ZEROCOPY=1 on AMF is ~2x WORSE (68 ms vs
36 ms) — the D3D11 import path adds sync overhead beyond the readback it saves,
so the system-memory default stays. GPU-priority elevation is already
process-wide (dxgi.rs), so it covers the iGPU encode session with no change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The blocking poll landed but wait_us pegs at exactly the 2-frame-period cap:
AMF holds the AU ~2 frame periods regardless of retrieval. Field knobs to
bisect on-box (usage preset × poll cap) without rebuild cycles.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The session loop's pipeline deferral was designed around direct NVENC, whose
poll() BLOCKS in lock_bitstream; libavcodec's AMF wrapper is truly async
(EAGAIN until the ASIC finishes), so a single non-blocking receive quantized AU
retrieval to the submit cadence: +1–2 frame periods flat (~43 ms p50 at 720p60
on the Ryzen iGPU vs ~3.5 ms of actual encode). FfmpegWinEncoder now tracks
in-flight frames and, while an AU is owed, spin-polls with short sleeps bounded
to ~2 frame periods (an overloaded encoder degrades to next-tick pickup instead
of stalling capture). Also: quality=speed (latency-first, iGPU-class VCN),
explicit bf=0 (h264_amf defaults >0 on RDNA3+), AMF low-latency submission
mode (FFmpeg ≥6.1, ignored on older).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- new crate::gpu (compiled on all platforms so the OpenAPI doc stays
platform-independent): DXGI / sysfs GPU inventory with reboot-stable ids
(PCI vendor:device + occurrence — LUIDs are per-boot), persisted auto/manual
preference (<config>/gpu-settings.json, atomic temp+rename with in-memory
rollback), one selection with precedence console preference >
PUNKTFUNK_RENDER_ADAPTER > max VRAM and graceful fallback when the preferred
GPU is absent, plus a live "in use" record (RAII session guard wrapped around
every encoder open_video returns)
- fix: windows_gpu_vendor derived the encoder backend from DXGI adapter 0
instead of the selected render adapter — on a hybrid box (e.g. Intel iGPU at
index 0 + NVIDIA dGPU) the backend could disagree with the GPU the capture
ring / IddCx render pin sit on. The NVENC 4:4:4 probe now also runs on the
selected adapter (was: OS default), the codec/4:4:4 probe caches are keyed
per selected GPU (were process-lifetime OnceLocks), and an explicit
PUNKTFUNK_ENCODER conflicting with the selected GPU's vendor warns up front
- mgmt API: GET /api/v1/gpus (inventory + mode + preferred + next-session
selection with reason + in-use GPU/backend/session-count) and
PUT /api/v1/gpus/preference (validates mode/gpu_id before writing);
openapi.json regenerated; the vdisplay render pin now also engages for a
console preference (not just the env pin)
- web console: GPU card on the Host page — list with vendor + VRAM,
Automatic / Prefer controls, Preferred / Next session / "In use · backend"
badges, missing-preferred-GPU warning and env-pin note; en + de messages
- Linux: a matched manual preference picks the VAAPI render node and the
NVENC-vs-VAAPI auto choice; auto mode is exactly the previous behavior
Validated live on the hybrid laptop (RTX 3500 Ada + Intel Arc Pro, which
enumerates twice — the occurrence ids disambiguate): enumerate, prefer,
bad-id 400, restart persistence, auto-restore keeping the stored pick.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client
(previously stereo-only):
- core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream
mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome
`audio_channels` negotiation via the trailing-byte back-compat pattern (old
peers fall back to stereo); C-ABI `punktfunk_connect_ex6`,
`punktfunk_connection_audio_channels`, and in-core multistream decode
`punktfunk_connection_next_audio_pcm` for embedders without a multistream
Opus decoder. Real-libopus channel-identity round-trip test.
- host: native audio thread captures + Opus-(multi)stream-encodes at the
negotiated count (with a cross-session cached-capturer channel-mismatch fix);
GameStream surround unified onto the safe `opus::MSEncoder`, dropping
`audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround;
WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask.
- clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via
`opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM →
AVAudioEngine with an explicit wire-order channel layout; each gains a
Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless
validator.
Verified on Linux: core/host/linux/probe test suites + the Android Rust
(cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple
builds, all on-glass checks, and the live native loopback are pending (CI / a
free box).
Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it
shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so
cannot be committed separately from the surround changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Continues the unsafe-proof program across the Windows/cross-platform host files
(~75 blocks, 21 files), each with a SAFETY proof of the real invariant and a
per-file #![deny(clippy::undocumented_unsafe_blocks)] gate:
capture/windows: dxgi.rs, wgc_relay.rs, wgc.rs, desktop_watch.rs, composed_flip.rs
(windows-rs COM: interface validity, same-D3D11-device textures,
immediate-context single-thread, borrowed args outlive the call)
windows: service.rs (SCM/token/CreateProcessAsUserW/event handles — OwnedHandle
liveness, no double-close/signal race), win_display, wgc_helper, interactive
vdisplay/windows: manager.rs, pf_vdisplay.rs (SwDeviceCreate/IddCx/ioctl handle
liveness via the OnceLock VDM singleton + OwnedHandle)
encode/windows: ffmpeg_win.rs (full AVBufferRef refcount audit — balanced, NO leaks,
unlike the vaapi sibling), sw.rs
cross-platform: gamestream/audio.rs (libopus), gamestream/stream.rs (sendmmsg),
inject/windows/sendinput.rs, audio/windows/wasapi_mic.rs,
session_tuning.rs, vdisplay.rs
Two findings (handled separately):
- wgc_relay.rs `unsafe impl Sync for HelperRelay` is UNSOUND (its mpsc Receiver is
!Sync) though not live-exploited — marked SUSPECT inline; fix pending box check
(it touches the in-flight punktfunk1.rs).
- capture.rs / encode.rs (PARENT modules of the WIP idd_push.rs / nvenc.rs) do NOT
get the file deny yet — it would propagate the lint into the undocumented WIP
children. The deny lands there once those are documented (after the WIP commits).
Linux-visible parts verified green (cargo clippy -p punktfunk-host --all-targets
-- -D warnings). The cfg(windows) deny gates are box-verified next.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move 36 platform-specific files into per-module `windows/` and `linux/` subfolders (and the
shared HID codecs into `inject/proto/`):
capture/{windows,linux}/ encode/{windows,linux}/ inject/{windows,linux,proto}/
audio/{windows,linux}/ vdisplay/{windows,linux}/
src/windows/ (service, wgc_helper, win_adapter, win_display)
src/linux/ (dmabuf_fence, drm_sync, zerocopy/)
Done with `#[path]`, NOT a module rename: every file moves into its folder while the
`crate::*::*` module names stay FLAT, so all caller paths and every internal `super::`/`crate::`
reference are unchanged — only the parent `mod` decls gained `#[path = "..."]`. This is the
codebase's existing pattern (inject's gamepad_windows) and makes the move byte-identical in
behaviour with ZERO reference churn, far lower risk than collapsing to a single
`crate::capture::windows::` namespace (that deeper rename is an optional follow-on; this delivers
the cfg-sprawl folder confinement the stage is about). Done LAST, after the semantic stages, so
the path churn didn't fight them.
Verified: Linux cargo check + clippy (-D warnings) clean; my mod-decl changes fmt-clean (the 3
remaining fmt diffs are pre-existing local-rustfmt-version skew that moved with their files); all
36 `#[path]` targets exist; no internal `#[path]`/`include!`/file-child-mod in any moved file
(the inline `mod X {` blocks are self-contained). Box build to follow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler