The gpu-contention plan's §5.B lever: today submit and the blocking
lock_bitstream share one thread, so under a GPU-saturating game the
pipeline serializes on the WDDM scheduling wait (1000/17ms ≈ 59fps —
the depth-1 collapse; the old 'deeper pipeline just stacks latency'
result was a same-thread implementation, not a disproof). Async mode
opens the session enableEncodeAsync=1, registers an auto-reset
completion event per pool bitstream, and moves the wait+lock+copy+
unlock onto an internal retrieve thread feeding poll() through a
channel — the exact split the NVENC guide mandates. Register/map/unmap
stay on the encode thread; teardown drops the job channel, joins the
thread, THEN destroys the session. In-flight depth is bounded by
PUNKTFUNK_NVENC_ASYNC_DEPTH (default 4, hard cap POOL-1) — both for
output-buffer reuse and because NVENC encodes the capture ring's
textures in place. Idle latency cost ≈ 0 (same-tick pickup); under
contention completed frames queue instead of stalling capture.
CI-compile validated only — on-glass A/B under game load on the RTX
box still pending (box offline).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- new crate::gpu (compiled on all platforms so the OpenAPI doc stays
platform-independent): DXGI / sysfs GPU inventory with reboot-stable ids
(PCI vendor:device + occurrence — LUIDs are per-boot), persisted auto/manual
preference (<config>/gpu-settings.json, atomic temp+rename with in-memory
rollback), one selection with precedence console preference >
PUNKTFUNK_RENDER_ADAPTER > max VRAM and graceful fallback when the preferred
GPU is absent, plus a live "in use" record (RAII session guard wrapped around
every encoder open_video returns)
- fix: windows_gpu_vendor derived the encoder backend from DXGI adapter 0
instead of the selected render adapter — on a hybrid box (e.g. Intel iGPU at
index 0 + NVIDIA dGPU) the backend could disagree with the GPU the capture
ring / IddCx render pin sit on. The NVENC 4:4:4 probe now also runs on the
selected adapter (was: OS default), the codec/4:4:4 probe caches are keyed
per selected GPU (were process-lifetime OnceLocks), and an explicit
PUNKTFUNK_ENCODER conflicting with the selected GPU's vendor warns up front
- mgmt API: GET /api/v1/gpus (inventory + mode + preferred + next-session
selection with reason + in-use GPU/backend/session-count) and
PUT /api/v1/gpus/preference (validates mode/gpu_id before writing);
openapi.json regenerated; the vdisplay render pin now also engages for a
console preference (not just the env pin)
- web console: GPU card on the Host page — list with vendor + VRAM,
Automatic / Prefer controls, Preferred / Next session / "In use · backend"
badges, missing-preferred-GPU warning and env-pin note; en + de messages
- Linux: a matched manual preference picks the VAAPI render node and the
NVENC-vs-VAAPI auto choice; auto mode is exactly the previous behavior
Validated live on the hybrid laptop (RTX 3500 Ada + Intel Arc Pro, which
enumerates twice — the occurrence ids disambiguate): enumerate, prefer,
bad-id 400, restart persistence, auto-restore keeping the stored pick.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A line-wrap put `+`/`*`-style markers at the start of two doc lines, which
clippy (Windows host job, rust 1.96) reads as markdown list items whose
unindented follow-on lines trip `doc_lazy_continuation` under `-D warnings`:
- encode/windows/nvenc.rs `chroma_444` field doc (the failing Windows-host
clippy job): "+ chromaFormatIDC = 3" → "and chromaFormatIDC = 3".
- encode/linux/vaapi.rs `probe_can_encode_444` doc: "+ validate" → "and
validate" (last line, didn't fire yet, but fragile — fixed pre-emptively).
Pure doc rewording, no behaviour change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client
(previously stereo-only):
- core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream
mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome
`audio_channels` negotiation via the trailing-byte back-compat pattern (old
peers fall back to stereo); C-ABI `punktfunk_connect_ex6`,
`punktfunk_connection_audio_channels`, and in-core multistream decode
`punktfunk_connection_next_audio_pcm` for embedders without a multistream
Opus decoder. Real-libopus channel-identity round-trip test.
- host: native audio thread captures + Opus-(multi)stream-encodes at the
negotiated count (with a cross-session cached-capturer channel-mismatch fix);
GameStream surround unified onto the safe `opus::MSEncoder`, dropping
`audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround;
WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask.
- clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via
`opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM →
AVAudioEngine with an explicit wire-order channel layout; each gains a
Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless
validator.
Verified on Linux: core/host/linux/probe test suites + the Android Rust
(cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple
builds, all on-glass checks, and the live native loopback are pending (CI / a
free box).
Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it
shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so
cannot be committed separately from the surround changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes the unsafe-proof program now that the parallel WIP has landed:
- idd_push.rs (25 sites), nvenc.rs (7), punktfunk1.rs (21): a SAFETY proof on
every unsafe block — D3D11/DXGI COM (same-device textures, immediate-context
single-thread, keyed-mutex-held convert), the NVENC SDK table (versioned POD,
register/map/lock-bitstream pairing), cross-process shm reads (atomic
magic/generation handshake), and the C-ABI harness (each call cross-checked
against its abi.rs `# Safety` doc). No SUSPECT (UB) blocks.
- capture.rs / encode.rs: the parent-module deny is restored (their WIP children
are now proven), and main.rs gains a crate-root
#![deny(clippy::undocumented_unsafe_blocks)] — the permanent catch-all gate so
no future unsafe block anywhere in the crate can land without a proof.
- Fixed 4 blocks the agents missed: unsafe blocks nested inside `assert_eq!(...)`
macro args (the comment-above-statement didn't associate) — hoisted to a `let`.
- rustfmt-canonicalized the Windows files (the agents' SAFETY comments + some
pre-existing 1.9.0 drift) so `cargo fmt --all --check` is clean.
Verified: cargo clippy -p punktfunk-host --all-targets -- -D warnings AND
cargo fmt -p punktfunk-host --check both green with the crate-root deny active.
Windows cfg(windows) re-verified on the box next.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GPU-contention work (host-latency plan §5.A): the IDD-push output ring now hands
NVENC native YUV instead of RGB, so NVENC skips its internal RGB→YUV colour
conversion on the SM/3D engine the running game saturates.
- idd_push.rs: out_ring is now NV12 (SDR, BT.709 limited) via a D3D11 VIDEO-engine
BGRA→NV12 VideoConverter (keeps the CSC off the contended 3D/compute engine), or
P010 (HDR, BT.2020 PQ limited) via the FP16→P010 shader (NVIDIA's VideoProcessor
can't do RGB→P010). The ring drops its per-slot RTV (textures only), matching the
WGC YUV ring; converters rebuild on a size/HDR flip.
- nvenc.rs: NV12 input forces bit_depth=8 so an HDR→SDR toggle (or a 10-bit-
negotiated client on an SDR display) re-inits the session at the matching depth —
NV12 can't feed a 10-bit session (register_resource rejects it).
- punktfunk1.rs: per-stage latency instrumentation under PUNKTFUNK_PERF
(cap=try_latest, submit=encode_picture, wait=lock_bitstream µs p50/p99/max) to
pinpoint where capture→encoded latency goes under GPU saturation.
The last §2.3 seam-trait tightening: give `Encoder` a `caps() -> EncoderCaps`
so the session glue routes by *query* instead of relying on the no-op/`false`
defaults of `invalidate_ref_frames`/`set_hdr_meta`.
`EncoderCaps { supports_rfi, supports_hdr_metadata }` is a cheap `Copy` struct.
The trait gains a default `caps()` returning `EncoderCaps::default()` (all
false) — correct for every SDR/libavcodec backend (Linux NVENC, VAAPI, AMF/QSV,
software openh264), so they need no change. Only the Windows direct-NVENC path
(`NvencD3d11Encoder`) overrides it, reporting the real `rfi_supported` (probed
once at open via `nvEncGetEncodeCaps`) and `hdr` (HDR-SEI on keyframes).
Consumer: the GameStream encode loop (`gamestream/stream.rs`) hoists
`supports_rfi` once before the loop and gates the loss-recovery path on it —
`!(supports_rfi && enc.invalidate_ref_frames(..))` forces a keyframe directly
on non-RFI encoders instead of making an always-`false` call every loss event.
Behaviour-preserving (same keyframe/RFI outcome), one fewer no-op call, intent
explicit. The native host (punktfunk1) uses FEC+keyframes, no RFI consumer.
Linux `cargo clippy -p punktfunk-host --all-targets -D warnings` clean; the
three edited files are rustfmt-clean. The NVENC override is Windows-only
(1:1 with the existing impl style) → CI/on-glass gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move 36 platform-specific files into per-module `windows/` and `linux/` subfolders (and the
shared HID codecs into `inject/proto/`):
capture/{windows,linux}/ encode/{windows,linux}/ inject/{windows,linux,proto}/
audio/{windows,linux}/ vdisplay/{windows,linux}/
src/windows/ (service, wgc_helper, win_adapter, win_display)
src/linux/ (dmabuf_fence, drm_sync, zerocopy/)
Done with `#[path]`, NOT a module rename: every file moves into its folder while the
`crate::*::*` module names stay FLAT, so all caller paths and every internal `super::`/`crate::`
reference are unchanged — only the parent `mod` decls gained `#[path = "..."]`. This is the
codebase's existing pattern (inject's gamepad_windows) and makes the move byte-identical in
behaviour with ZERO reference churn, far lower risk than collapsing to a single
`crate::capture::windows::` namespace (that deeper rename is an optional follow-on; this delivers
the cfg-sprawl folder confinement the stage is about). Done LAST, after the semantic stages, so
the path churn didn't fight them.
Verified: Linux cargo check + clippy (-D warnings) clean; my mod-decl changes fmt-clean (the 3
remaining fmt diffs are pre-existing local-rustfmt-version skew that moved with their files); all
36 `#[path]` targets exist; no internal `#[path]`/`include!`/file-child-mod in any moved file
(the inline `mod X {` blocks are self-contained). Box build to follow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler