The SYSTEM host now sources the normal-desktop video from a user-session WGC
helper instead of capturing in-process (WGC won't activate as SYSTEM). New
`capture/wgc_relay.rs`: `HelperRelay::spawn` launches `m3-host wgc-helper` in the
interactive user session via CreateProcessAsUserW (WTSQueryUserToken →
DuplicateTokenEx(TokenPrimary) → lpDesktop="winsta0\\default", CREATE_NO_WINDOW)
with three anonymous pipes — stdout (framed Annex-B AUs → parsed back to
RelayAu), stdin (control: force-keyframe), stderr (helper logs → host tracing).
The host holds the SudoVDA keepalive (sole isolation/topology owner); the helper
captures by GDI name only.
m3.rs: `virtual_stream` dispatches to the new `virtual_stream_relay` when
`should_use_helper()` (running as SYSTEM, or PUNKTFUNK_FORCE_HELPER; disable with
PUNKTFUNK_NO_HELPER). The relay loop feeds the existing send thread — same
FEC/seal/paced-send path. Reconfigure rebuilds the output + re-spawns the helper;
keyframe requests forward over the control pipe; helper pts_ns (same-machine
monotonic clock) is used directly as capture_ns. Disconnect ends the stream
(step 6 adds the relaunch watchdog).
wgc_helper.rs: reads the stdin control byte to request an IDR; --bit-depth flag
threaded through so SDR 10-bit (Main10) negotiation reaches the helper's encoder.
cfg-gated windows-only; Linux/macOS build unaffected. Step 5 (DesktopWatcher mux
to host DDA on the Winlogon secure desktop) is next.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Polls the input-desktop name (OpenInputDesktop + GetUserObjectInformationW(UOI_NAME)) on its own
thread → Default/Winlogon atomic; the authoritative normal-vs-secure signal for the capture mux +
input path (WTS notifications miss UAC). Not yet wired into the mux (needs the SYSTEM host + WGC
helper, steps 3-5 in docs/windows-secure-desktop.md). NOTE: detecting the secure desktop requires the
host to run as SYSTEM (a user-token process can't OpenInputDesktop the Winlogon desktop).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The capture-architecture reset from the research: add a Windows.Graphics.Capture (WGC) backend that
captures the COMPOSED desktop — including the overlay/independent-flip/MPO planes DXGI Desktop
Duplication misses — which structurally fixes the frozen HDR animations + video (proven live: a WGC
frame decodes to the real 5120x1440 HDR content DDA freezes on). It reuses the whole pipeline
unchanged: the WGC frame's GPU texture → same scRGB→BT.2020-PQ shader → NVENC zero-copy; the OS
composites the cursor (IsCursorCaptureEnabled) so no manual cursor pass. crates/punktfunk-host/src/
capture/wgc.rs; find_output/make_device/HdrConverter/nudge_cursor_onto made pub(crate) for reuse.
Reliability findings + mitigations (live on the RTX 4090):
- WGC can't activate under the SYSTEM account (0x80070424) — it needs the interactive user token. The
host must run as the user for WGC (run.cmd: drop PsExec -s). DDA still needs SYSTEM for the secure
desktop — that token reconciliation (impersonation) is the remaining task.
- WGC's Direct3D11CaptureFramePool::CreateFreeThreaded intermittently HANGS on the headless SudoVDA
(IddCx) display, correlated with accumulated SudoVDA churn (failed REMOVEs leaving lingering
displays); clean-state opens reliably. Since it's a blocking hang, capture_virtual_output runs WGC
open on a watchdog thread with a 5s timeout and falls back to DDA on hang/error — the session is
NEVER left black: WGC when it opens (fixed animations), DDA otherwise. First-frame nudge added (WGC
fires FrameArrived on change; a static desktop otherwise never delivers the first frame).
- Default WGC; PUNKTFUNK_CAPTURE=dda forces DDA. DDA path unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an
8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the
capture losing the stream across a secure-desktop transition.
Protocol (punktfunk-core/quic.rs):
- Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`,
both as optional trailing bytes (back-compat). client-rs advertises 10-bit via
PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives
the native clients). Regenerated punktfunk_core.h.
Windows host:
- 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND
PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10,
pixelBitDepthMinus8).
- HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to
an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit
(R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI
(ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the
BGRA path. Reacts dynamically to the HDR toggle.
- Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s,
repeating the last good frame between attempts (format-tagged last_present). During a
secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the
send loop for seconds so the client timed out and disconnected — now the session stays
fed (frozen) until the desktop returns. Also seeds a black frame on recovery.
Apple client (PunktfunkKit):
- Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010,
and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged.
Switches automatically on a mid-session HDR toggle.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Windows Capturer via DXGI Desktop Duplication: create a D3D11 device on the SudoVDA adapter (by LUID), find the matching output (by GDI name), DuplicateOutput, and per AcquireNextFrame copy the desktop into a CPU-readable staging texture -> tightly-packed BGRA (FramePayload::Cpu, feeds the openh264 software encoder GPU-lessly). Handles WAIT_TIMEOUT (reuse last frame) and ACCESS_LOST (re-duplicate). Adds FramePayload::D3d11(D3d11Frame) for the future NVENC zero-copy path, and a VirtualOutput.win_capture identity (adapter LUID + GDI name) carried out of the SudoVDA backend. Pure helpers (pack_luid/gdi_name_matches/depad_bgra) unit-tested on the VM; the live duplication path needs a real GPU + an activated SudoVDA monitor. Compiles clean on Windows + Linux.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Mac/iOS client's wall around ~380 Mbps on a 2.5 G path is the receive
drain, not the transport: a loopback speed-test pushes 380/600/1000 Mbps at
0.0% loss, but Darwin has no recvmmsg(2), so the macOS client was doing one
recv() syscall per packet — ~40-90k syscalls/s on one core. When the recv loop
can't drain fast enough the kernel socket buffer backs up and drops, which the
client sees as a sustained stream stalling/freezing in the 300-400 Mbps range
(and an immediate "session ended" when a 500 Mbps+ first keyframe bursts in).
- core/transport: flip recvmsg_x (the batched Darwin recv, ~30x fewer syscalls)
from opt-in to default ON, opt-out via PUNKTFUNK_RECVMSG_X=0. Keeps the
auto-fallback to the scalar loop on any unexpected syscall error. The Apple CI
swift-test loopback now exercises this path by default.
- packaging/kde host.env: enable PUNKTFUNK_GSO=1 — UDP segmentation offload on
the host send path (one sendmsg per ~64 packets), the dominant lever above
~1 Gbps. Already wired (send_sealed -> send_gso) with sendmmsg auto-fallback.
- apple SpeedTestSheet: lengthen the bandwidth probe 2 s -> 5 s so the measured
number stops swinging wildly (50 vs 900 Mbps on the same link) — long enough
for steady-state send + recv drain to settle. Matches host MAX_PROBE_MS.
- host capture: PUNKTFUNK_SYNTH_NOISE synthetic high-entropy source for
reproducible throughput testing of the encode->FEC->send->recv path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Full project rename, decided 2026-06-10:
- Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs.
- C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h,
PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl.
PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants).
- Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1.
WIRE BREAK: clients must be rebuilt from this revision.
- Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / ….
- Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the
persistent identity is unchanged, pinned fingerprints stay valid).
- Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection
(Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated.
- scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated.
Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of
"desktop but no apps/settings" over the stream: plasmashell launched without
XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and
rendered an empty menu. The script sets the complete KDE session env (menu prefix,
KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell.
Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS,
zero lumen references left outside .git.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler