The Windows host was NVIDIA-only (NVENC) with an openh264 software fallback. Add
AMD AMF and Intel QSV via libavcodec — the Windows analogue of the Linux VAAPI
backend — so one installer serves all three GPU vendors.
- encode/ffmpeg_win.rs: new WinVendor{Amf,Qsv} encoder. System-memory NV12/P010
readback (default, robust) + opt-in zero-copy D3D11 (PUNKTFUNK_ZEROCOPY: shares
the capturer's ID3D11Device; AMF takes AV_PIX_FMT_D3D11, QSV derives a QSV frames
ctx and maps) with a system fallback for the format-group mismatch the capturer's
video-processor fallback can produce. HDR Main10 (P010 + BT.2020/PQ VUI; an
Rgb10a2->P010 swscale covers the shader fallback).
- encode.rs: Codec::amf_name/qsv_name; open_video + windows_resolved_backend()
resolve PUNKTFUNK_ENCODER=auto|nvenc|amf|qsv|sw via a DXGI adapter VendorId probe.
- capture/dxgi.rs: gpu_mode mirrors the resolved backend (D3D11 NV12/P010 for AMF/QSV).
- gamestream/serverinfo.rs: GPU-aware codec advertisement (windows_codec_support;
AV1 gated to RDNA3+/Arc, like the VAAPI path).
- Cargo.toml: amf-qsv feature (optional ffmpeg-next in the windows target block).
- CI/installer: windows-host.yml sets FFMPEG_DIR + builds --features nvenc,amf-qsv;
the Inno installer bundles the FFmpeg DLLs; host.env default nvenc -> auto.
CI-green target; AMF/QSV not yet on-glass validated (no AMD/Intel Windows box in the
lab) — NVENC stays live-validated. An adversarial-review pass caught + fixed real
FFI bugs (AV_PIX_FMT_P010 is a macro -> P010LE; windows-rs 0.62 GetImmediateContext/
GetDesc1 return Result; AV_HWFRAME_MAP_* is a bindgen enum with no BitOr).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two strands, entangled in punktfunk1.rs, committed together (one builds-green tree).
HDR pipeline Step 0 — glass-to-glass colour-metadata transport (docs/hdr-pipeline-plan.md):
- Protocol/ABI: ColorInfo on the Welcome + a 0xCE HdrMeta datagram carry the source colour
space + HDR10 static mastering metadata (quic.rs, abi.rs connect_ex5 fixing caps=0).
- New platform-independent, unit-tested HDR static-metadata helpers (hdr.rs): chromaticities
(1/50000), mastering luminance (0.0001 cd/m2), MaxCLL/MaxFALL in HDR10/ST.2086 units.
- Capture/encode hooks (capture.rs, encode.rs set_hdr_meta) + Linux client / probe plumbing.
Security-audit hardening — top 3 from docs/security-review.md, each adversarially verified:
- #1 [HIGH] Secret file permissions. The host key.pem/cert.pem and both trust stores are now
written owner-only: 0600 + dir 0700 on Unix (mirrors mgmt_token), best-effort
SYSTEM/Administrators/OWNER-only icacls DACL on Windows (%ProgramData% is Users-readable).
Closes a local key-disclosure -> host-impersonation gap. New gamestream::{create_private_dir,
write_secret_file} + a 0600 regression test.
- #2 [HIGH] Native SPAKE2 PIN is single-use. The PIN is consumed the moment the host sends its
key-confirmation (which lets the client test its one guess), before reading the proof, so any
completed attempt -- right OR wrong -- disarms the window. A wrong PIN isn't observable
host-side (the client aborts before sending its proof), so consuming on first attempt is what
delivers the documented "one online guess" instead of an unbounded brute-force of the static
4-digit PIN. Test verifies single-use.
- #3 [MEDIUM] RTSP packetSize is bounded ([64,2048] in stream_config) and VideoPacketizer::new
uses saturating .max(1), killing a PRE-AUTH div-by-zero/underflow panic of the video thread.
Tests for {0,15,16,17} + out-of-range rejection.
fmt + clippy -D warnings clean; full workspace test suite green (93 host tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A pass over the apollo-comparison backlog (re-verified against current code).
Lands four items end-to-end plus a Windows-DualSense scoping doc.
- #5/#92/#26 — GameStream paired-cert allow-list. tls.rs surfaces the verified
peer cert to handlers (serve_https + PeerCertFingerprint, now shared with the
mgmt API instead of duplicated); nvhttp gates /launch /resume /applist /cancel
on AppState.paired and reports a real PairStatus; save_paired writes atomically
(temp+rename). Closes the "mTLS accepts any client cert" hole. + regression test.
- #6/#51/#19/#22 — NVENC caps query -> reference-frame invalidation. nvenc.rs
query_caps probes nvEncGetEncodeCaps (max dims / 10-bit / custom-VBV / RFI),
rejecting over-range modes and degrading 10-bit->8-bit instead of an opaque
InvalidParam. New Encoder::invalidate_ref_frames (default false -> caller
keyframes); the Windows NVENC path implements real RFI (multi-ref DPB +
nvEncInvalidateRefFrames, dedup + IDR-on-overflow). control.rs decodes the
0x0301 lost-frame range (Apollo's IDX_INVALIDATE_REF_FRAMES) -> AppState.rfi_range
-> encode loop, falling back to a keyframe. NOTE: the Windows NVENC impl is
RTX-box/CI-pending (can't compile on Linux); adversarially reviewed vs the SDK.
- #43/#72 — media socket QoS + buffer growth. New punktfunk_core::transport::qos:
grow_socket_buffers (factored out the native plane's 32MB SO_SNDBUF growth so the
GameStream sockets reuse it) + set_media_qos (opt-in PUNKTFUNK_DSCP=1: DSCP CS5
video / CS6 audio + Linux SO_PRIORITY, Apollo's scheme). Wired into UdpTransport
and the GameStream video/audio sockets. Windows IP_TOS needs qWAVE (follow-up).
- #8/#45 — GameStream input injection off the ENet service thread. on_receive no
longer injects inline (a slow inject head-blocked ENet keepalive/retransmit); it
forwards to a dedicated injector thread. The hardened InjectorService moved from
punktfunk1 into crate::inject (shared by both planes) + a coalesce step that sums
adjacent relative-mouse/scroll deltas while preserving button/key/abs ordering.
Docs: re-verified apollo-comparison.md status (22 items already done/obsolete since
the snapshot) + windows-dualsense-scoping.md (ViGEm can't emulate a DualSense; real
DS5 on Windows needs a VHF virtual-HID driver — web-research pass pending).
fmt + clippy -D warnings clean; full workspace test suite green; no C-ABI/OpenAPI drift.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Polish for AMD/Intel support:
- GameStream serverinfo advertises only codecs the GPU can ACTUALLY encode on
the VAAPI backend (probed once by opening a tiny encoder per codec). AV1
encode is narrow (Intel Arc/Xe2+, AMD RDNA3+/RDNA4) and an old iGPU may lack
HEVC, so a Moonlight client never negotiates a codec the encoder can't open.
NVENC/Windows keep the Moonlight-validated static mask. Validated on a Radeon
780M: h264/h265/av1 all probe true -> mask unchanged (65793).
- Packaging: Recommends mesa-va-drivers + intel-media-va-driver (deb) /
mesa-va-drivers + intel-media-driver (rpm) so the auto-selected VAAPI backend
works out of the box on AMD/Intel; NVIDIA boxes can --no-install-recommends.
(Fedora note: stock mesa-va-drivers disables HEVC/AV1 -- needs the freeworld
variant from RPM Fusion.)
- De-NVIDIA-fy the user-facing encoder log/context strings ("open NVENC" ->
"open video encoder") now that VAAPI is a first-class backend.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 2 of AMD/Intel support: the VAAPI encoder now takes the capture dmabuf
directly and does the RGB->NV12 colour conversion on the GPU's video engine,
eliminating the host-side de-pad + swscale CSC + upload the CPU path pays.
- capture: a vendor-neutral FramePayload::Dmabuf (dup'd fd + fourcc/modifier/
layout). When zero-copy is on, the EGL->CUDA importer is unavailable (any
non-NVIDIA host), and the backend is VAAPI, the capturer advertises LINEAR
dmabuf and hands the raw buffer to the encoder instead of CPU-copying it.
- encode/vaapi: the encoder self-configures from the first frame's payload (no
open_video signature change). The dmabuf arm wraps the buffer as an
AV_PIX_FMT_DRM_PRIME frame and pushes it through a filter graph
buffer(drm_prime) -> hwmap(vaapi) -> scale_vaapi=nv12 -> buffersink; the
encoder takes NV12 surfaces straight from the sink. The Phase 1 CPU-upload
path is kept as the other arm (used when capture produces CPU frames).
Live-validated on a Radeon 780M (real Sway/xdpw desktop capture): correct,
pixel-perfect HEVC, and ~10x less host CPU at 1440p (4.2s -> 0.4s of CPU for
300 frames) -- the de-pad/CSC/upload moves to the GPU. NVIDIA unchanged
(zero-copy still imports to CUDA; the passthrough path only engages on
non-NVIDIA hosts).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Linux host was NVENC/CUDA-only. Add a VAAPI encoder — one libavcodec
backend (h264/hevc/av1_vaapi) covering both AMD (Mesa radeonsi) and Intel
(iHD) — behind the existing `Encoder` trait, and turn `open_video`'s Linux
arm into a vendor dispatcher: `PUNKTFUNK_ENCODER=auto|nvenc|vaapi` (default
auto: NVENC when a CUDA frame or /dev/nvidia* is present, else VAAPI). The
NVIDIA path is unchanged — auto resolves to NVENC on an NVIDIA box and the
bitrate-probe loop moved verbatim into `open_nvenc_probed`.
`VaapiEncoder` mirrors the NVENC hwframes pattern with AV_HWDEVICE_TYPE_VAAPI.
The CPU-input path swscales packed RGB -> NV12 (BT.709 limited, VUI signalled)
and uploads into a pooled VA surface (av_hwframe_transfer_data), preserving the
low-latency model (infinite GOP, on-demand forced IDR, async_depth=1, CBR when
the driver supports it). It works on a non-NVIDIA box with no capture changes:
the capturer already falls back to CPU frames when its EGL->CUDA importer can't
initialise (no libcuda).
Live-validated on a Radeon 780M (RDNA3): hevc/h264/av1_vaapi all encode,
HEVC/H264 decode cleanly with correct BT.709-limited colours, infinite GOP
preserved. Zero-copy dmabuf import (the high-res perf lever) is next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two bodies of work in one commit (the rename moved files the fixes also touched).
Naming/structure cleanup (pre-launch):
- Host modules m3.rs->punktfunk1.rs, m0.rs->spike.rs; CLI m3-host->punktfunk1-host,
m0->spike; bare `punktfunk-host` now prints help. Types M3Options/M3Source->
Punktfunk1Options/Punktfunk1Source.
- Clients consolidated out of crates/ into clients/: punktfunk-client-rs->
clients/probe (crate punktfunk-probe), client-linux->clients/linux,
client-windows->clients/windows, punktfunk-android->clients/android/native
(crate punktfunk-client-android; kept [lib] name=punktfunk_android so the JNI
contract is unchanged). crates/ now holds only core + host.
- Milestone codes M0-M4 purged from code/CLI/CLAUDE.md/README/docs/docs-site,
kept only in docs/implementation-plan.md. docs/m2-plan.md->
docs/gamestream-host-plan.md. CI/gradle/flatpak paths updated.
Client loss-recovery (video froze and never recovered after a brief drop):
- Export punktfunk_connection_frames_dropped through the C ABI (the core already
tracked it for the client keyframe-recovery loop; it was never reachable from
the ABI clients). Regenerated punktfunk_core.h.
- Apple (StreamPump + Stage2Pipeline) and Android (decode.rs) now poll
frames_dropped and request a keyframe when it climbs -- the same loss-driven
recovery Linux/Windows already had. Under infinite GOP the decoder silently
conceals reference-missing frames, so the decode-error trigger rarely fires.
Apple rumble robustness (worked then went spotty -- DualSense + Xbox):
- Add CHHapticEngine stopped/reset handlers (rebuild on app background / audio
interruption / server reset) and drop the permanent `broken` latch on a
transient drive failure; latch only when the controller truly has no haptics.
- Surface swallowed SDL set_rumble errors on Linux/Windows + diagnostic logging.
Verified: cargo build/clippy/fmt --workspace, C-ABI harness, header drift.
Not runnable on this box (verify in CI): Gitea workflows, gradle/Android,
flatpak, Swift/decky.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an
8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the
capture losing the stream across a secure-desktop transition.
Protocol (punktfunk-core/quic.rs):
- Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`,
both as optional trailing bytes (back-compat). client-rs advertises 10-bit via
PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives
the native clients). Regenerated punktfunk_core.h.
Windows host:
- 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND
PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10,
pixelBitDepthMinus8).
- HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to
an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit
(R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI
(ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the
BGRA path. Reacts dynamically to the HDR toggle.
- Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s,
repeating the last good frame between attempts (format-tagged last_present). During a
secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the
send loop for seconds so the client timed out and disconnected — now the session stays
fed (frozen) until the desktop returns. Also seeds a black frame on recovery.
Apple client (PunktfunkKit):
- Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010,
and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged.
Switches automatically on a mid-session HDR toggle.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Zero-copy capture->encode on the GPU via the raw NVENC API (nvidia_video_codec_sdk sys + ENCODE_API; the safe wrapper is CUDA-only). Opens an NV_ENC_DEVICE_TYPE_DIRECTX session on the SAME ID3D11Device as the DXGI capturer (carried on the new FramePayload::D3d11), registers a pool of BGRA textures once, CopyResources each captured texture in and encode_picture; CBR/ULL, infinite GOP, P-only, forced-IDR for RFI. The DXGI capturer gains a D3D11 zero-copy output (selected, like the encoder, by PUNKTFUNK_ENCODER=nvenc) so capture+encode share textures.
OFF by default (the nvenc feature pulls the NVENC SDK + cudarc): the default Windows host links without it (openh264 path). cudarc builds toolkit-less via the SDK ci-check feature (dynamic-loading). At link time --features nvenc needs nvencodeapi.lib (NVENC SDK, or an import lib generated from the driver's nvEncodeAPI64.dll) on PUNKTFUNK_NVENC_LIB_DIR. Both default and --features nvenc builds validated to compile+link GPU-less on the VM (import lib generated from the driver DLL). Runtime needs a real NVIDIA GPU.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Windows Encoder impl via the openh264 crate (statically-bundled, BSD-2): low-latency screen-content config (Baseline/no-B-frames, bitrate RC, BT.709 limited, near-infinite GOP + forced-IDR recovery via request_keyframe), packed CPU pixels (BGRx/BGRA/RGB/RGBA/RGBx/BGR) -> I420 -> AnnexB with in-band SPS/PPS each IDR. Synchronous: submit encodes immediately, poll hands back the one AU, flush is a no-op. Windows open_video factory selects it (PUNKTFUNK_ENCODER=software|nvenc|auto; NVENC arm lands later), H.264-only with a clear error otherwise, SW bitrate ceiling. Unit-tested live on the VM: synthetic BGRx -> AnnexB IDR + SPS NAL. Unblocks the GPU-less capture->encode->FEC->send pipeline. Compiles clean on Windows + Linux.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the Mac "session ended" at 880 Mbps / 1.3 Gbps: the host requests a
bitrate NVENC can't express at any codec level and `avcodec_open2` returns EINVAL
("Invalid argument"), so the pipeline build fails after 4 identical retries and the
session dies at encoder init — before a single video packet (which is why the
client's UDP counters never moved). The ceiling is GPU/driver-specific: an RTX 4090
caps HEVC at ~800 Mbps (Level 6.2 High tier) and rejects above it, while an RTX
5070 Ti accepts 1.3 Gbps.
Rather than hard-cap every build to a conservative guess (which would needlessly
throttle capable cards), open_video now PROBES: open at the requested bitrate, and
step down (codec spec ceiling, then 0.75x to a 50 Mbps floor) ONLY when this GPU
returns EINVAL. Each GPU runs at its own real maximum — the 5070 Ti keeps 1.3 Gbps,
the 4090 lands at 800 Mbps and streams instead of dying. Non-EINVAL failures (no
GPU, bad mode, OOM) still surface immediately rather than being masked by retries.
Codec::max_bitrate_bps is now just the first step-down candidate, not a clamp.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Full project rename, decided 2026-06-10:
- Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs.
- C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h,
PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl.
PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants).
- Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1.
WIRE BREAK: clients must be rebuilt from this revision.
- Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / ….
- Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the
persistent identity is unchanged, pinned fingerprints stay valid).
- Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection
(Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated.
- scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated.
Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of
"desktop but no apps/settings" over the stream: plasmashell launched without
XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and
rendered an empty menu. The script sets the complete KDE session env (menu prefix,
KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell.
Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS,
zero lumen references left outside .git.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler