48 Commits

Include renames

Author	SHA1	Message	Date
enricobuehlerandClaude Fable 5	087e7d0405	fix(inject/windows): map absolute input over the streamed output's rect, not the whole desktop ... Pen, touch, and absolute mouse arrive normalized to the STREAMED display's frame, but pointer_windows::to_screen and sendinput's MouseMoveAbs mapped them over the whole virtual desktop — correct only when the virtual display is the sole active display (Exclusive topology). In Extend — a physical monitor kept on beside the virtual output, or an Exclusive isolate degraded to the 0x57 keep-physicals fallback — the streamed output sits at a non-zero origin, so every sample landed shifted and mis-scaled. The pen exposed it first (field report 2026-07-24): a stylus is strictly absolute, with no closed-loop correction onto the target like a cursor. New pf-inject::stream_target (Windows): the host publishes the streamed output's CCD target id at capture bring-up (one central site — capture_virtual_output covers the native and GameStream planes); mapping sites resolve its current desktop rect through pf-win-display's source_desktop_rect — the same resolver the cursor-readback poller uses, so inject and readback always agree — TTL-cached (250 ms) because a group-layout re-arrange moves a live output's origin mid-session. No target set / never resolved falls back to the whole virtual desktop (the historical mapping, still right for Exclusive topology and devtest). One change in to_screen covers pen + touch; MouseMoveAbs converts the same desktop pixel into the 0..65535 MOUSEEVENTF_VIRTUALDESK coordinate, closing the identical latent absolute-mouse bug (apollo-comparison open item #14/#30). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-24 17:38:36 +02:00
enricobuehlerandClaude Fable 5	326d6e17c8	fix(windows/capture): stand the IddCx hardware cursor down while the secure desktop is up ... 0.18.0 regression: UAC consent and Winlogon (lock/logon) stopped appearing in streams. The cursor channel's IddCx hardware-cursor declare — re-issued by the driver on every swap-chain assign — keeps the path out of the OS's software-cursor mode, which is the only mode the secure desktop renders through; DWM then never presents UAC/Winlogon into our swap-chain and the stream repeats the last normal-desktop frame for the whole interaction. The GDI cursor poller now classifies the input desktop on its reattach cadence (UOI_NAME != Default = secure, 250 ms cadence instead of 2 s), and the capturer edge-triggers the existing-but-unused proto-v6 IOCTL_SET_CURSOR_FORWARD flip: OFF at secure entry (the driver stops its per-assign re-declare; the host facade forces the same-mode re-commit that actualises the software-cursor default, under the vdisplay manager lock via the new force_recommit()), back ON at dismissal for channel sessions. Channel-session open resets the driver's persisted desired state to ON so a session that died mid-secure-desktop can't leave the next one adopting UNdeclared (the §8.6 cross-session composite trap); orderly teardown does the same. The flip closure is built for every Windows session — a channel-less session can reuse a driver monitor whose earlier-session cursor worker still re-declares, and NOT_FOUND from never-declared targets is ignored. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-23 14:13:15 +02:00
enricobuehlerandClaude Fable 5	af87549052	fix(linux/vdisplay): KWin virtual outputs stream above 60 Hz via sacrificial-birth renegotiation ... KWin's ScreenCastStream builds its PipeWire format offer — including the maxFramerate cap it actively throttles delivery to — ONCE at stream creation, when the virtual output sits at its hardcoded birth 60 Hz. A kscreen custom-mode change afterwards updates the OUTPUT (readback says 240) but never the offer, so every consumer negotiates max=60 and the stream delivers 60 (pw-dump-verified on KWin 6.6.4). The only path that rebuilds the offer is the stream's own resize handling: a source SIZE change while recording re-runs buildFormats — picking up the output's current refresh — and renegotiates the live stream. So above 60 Hz, birth the output at a sacrificial height (+16), then install + select the real WxH@hz custom mode: the first frame recorded after the consumer connects triggers KWin's resize path, which renegotiates the live stream to WxH@hz. The capturer holds (requeues) buffers until the negotiated size matches — new expect_exact_dims plumbing VirtualOutput → registry (fresh creates only) → open_virtual_output → a self-disarming gate in the process callback — bounded by a 3 s deadline that accepts the producer's dims rather than wedging the session into the first-frame retry loop. set_custom_refresh reads back the full active mode; a size reject (pre-6.6 KWin) recreates plain at the real size @60. Every kscreen operation now addresses the output by its NUMERIC id, resolved by matching the managed name-prefix AND the just-created birth size (newest id wins): a mode-switch supersede reuses the per-slot output NAME (deliberately, for KWin's per-name config persistence) while the superseded sibling is still alive, and name addressing hit the FIRST match = the OLD output — on-glass that resized the live session's display out from under it (wrong-res/black), read the old output back as 'mode applied', and set the old output primary while the replacement starved at its birth mode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Opus 4.8	0af280a793	feat(windows/capture): host-side cursor compositing for the capture model — the deterministic fix ... Root cause, proven on-glass across five driver builds: a declared IddCx hardware cursor is IRREVOCABLE. There is no un-declare DDI, the empty-caps re-setup is rejected INVALID_PARAMETER, and after a successful same-mode re-commit with the driver's re-declare provably suppressed (sticky per-target flag, zero re-setups logged) DWM still never composites the software cursor back into the monitor's frames. Every DWM-based composite flip is therefore unfixable. The composite (capture) model is now implemented in OUR pipeline: - the driver keeps its hardware cursor declared for the session's whole life (the state that works) — frames stay pointer-free always; - in composite mode try_consume routes the conversion through a blend scratch: slot copy + one alpha-blended quad of the GDI poller's shape at its polled position (CursorBlendPass — fullscreen-triangle VS with viewport placement, straight-alpha PS, sRGB→linear for FP16/HDR rings), covering all four convert paths (NV12, 4:4:4 copy, P010, PyroWave) GPU-side under the slot's keyed mutex; - pointer-only motion produces no driver publish (declared hw cursor), so try_consume REGENERATES from the last slot whenever the polled cursor state changes — the cursor moves on a static desktop at tick rate, without faking freshness for the stall/death watchdogs; - set_cursor_forward becomes purely host-internal state; the SET_CURSOR_FORWARD IOCTL machinery is no longer used by the host (driver keeps it, dormant) and its sender plumbing is removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8	92761813a9	feat(core+host+driver+client): mid-stream cursor-render flip — host composites in the capture model ... The cursor channel excluded the host pointer from the video for the whole session, but the client only draws it under the DESKTOP mouse model — flipping ⌃⌥⇧M to capture mid-stream left the session pointer-blind (user-found). The fix makes render ownership LIVE state: - wire: CursorRenderMode 0x51 (client→host, control stream) — client_draws: true = desktop model (host excludes + forwards), false = capture model / released (host composites, full fidelity — DWM on Windows incl. real XOR inversion, encoder blend on Linux). Sessions start client_draws=true (the pre-message behavior). - host: control task stores the flag; the encode loop edge-detects it — forwarding + frame.cursor strip while the client draws, quiet forwarder + overlay-into-blend while composited. SessionPlan now grants blend CAPABILITY wherever the capture has a pointer (dropping the !cursor_forward gate) so the composite side needs no encoder rebuild; per-tick frame.cursor decides what is drawn. - Windows: Capturer::set_cursor_forward → retained IOCTL_SET_CURSOR_FORWARD sender (proto v6) → driver (un)declares the IddCx hardware cursor on the LIVE monitor. Un-declare re-issues the setup with EMPTY caps (candidate mechanism — the DDI has no documented un-setup); resetup-on-commit is skipped while off, so a mode commit's software-cursor default is the fallback path. Failure against a pre-v6 driver logs and keeps declared-at-ADD behavior. - SDL client: one edge-detected reconciler at the cursor pump site — desktop-active = client draws; capture model or released = host composites. Covers the chord, the M3 auto-flip, and engage/release. - C ABI v12: punktfunk_connection_set_cursor_render (additive; wire version unchanged — pre-§8 hosts ignore the unknown message type). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5	3a34440a6b	feat(windows): IddCx hardware-cursor channel — remote-desktop sweep M2c ... Brings the cursor channel to Windows hosts. The pf-vdisplay driver declares an IddCx hardware cursor for sessions that negotiated cursor-forward — DWM then EXCLUDES the pointer from the IDD frame and delivers shape/position out-of-band, into the same CursorOverlay → forwarder → wire → client pipeline the Linux portal path uses. - pf-driver-proto v5 (additive, host floor stays 3): AddRequest's spare tail becomes hw_cursor (same size/offsets); IOCTL_SET_CURSOR_CHANNEL delivers a host-created CursorShm section (64-byte seqlock header + 256² shape buffer, layout pinned + tested). No event crosses the boundary — the host polls at encode-tick pace. - driver: wdk-iddcx grows the two cursor DDI wrappers; a per-monitor cursor worker (event wait → QueryHardwareCursor → seqlock publish) starts only when BOTH the ADD asked and the channel arrived, so a failed delivery leaves DWM compositing as today. Shape bytes ship raw (BGRA/masked + pitch); the host converts. - host: the section rides the existing sealed-channel broker (least- privilege dup, remote reap on failure); IddPushCapturer::cursor() seqlock-reads → CursorOverlay (BGRA→RGBA, masked-color approximation, desktop→frame origin shift, per-shape conversion cache). New Capturer::cursor() trait hook — the encode loop prefers it over the frame-attached overlay because hardware-cursor moves produce NO new frame on a static desktop. hw_cursor survives the re-arrival resize (carried on the manager's Monitor). - negotiation: cursor_forward grows the Windows arm (client cap ∧ driver proto ≥ 5, probed once via the control device); SessionPlan carries cursor_forward → OutputFormat.hw_cursor. - drive-by: wdk-probe's two pre-existing same-type casts (clippy) and pf-vdisplay's stale spike-test refs (crate::win_display moved to pf-win-display; tracing-subscriber was never a dep) repaired. Verified: proto tests on Mac AND MSVC (15/15 incl. the CursorShm layout pin); clippy -D warnings for proto/frame/capture/vdisplay/host on Linux (.21) and native Windows (.173); the DRIVER workspace clippy -D warnings green against the real WDK 10.0.26100 bindgen (DDI names, enum variants and IDARG layouts all bind). On-box driver deploy + on-glass validation follow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5	42198eb1b6	feat(linux/vulkan-encode): default-on native NV12 capture + RGB true-extent ... Flip both zero-copy levers from opt-in to default: - The capture negotiation now PREFERS gamescope's producer-side NV12 pod by default (PUNKTFUNK_PIPEWIRE_NV12=0 restores the packed-RGB negotiation). Codec-aware gating rides a new OutputFormat::nv12_native -> ZeroCopyPolicy::native_nv12_session edge resolved by the host facade from the session plan (pf_encode::linux_native_nv12_ok): only H265/AV1 sessions whose backend can open the raw Vulkan Video encoder ever see the NV12 pod -- an H264/Moonlight session (libav VAAPI, which would misread the two-plane buffer) keeps today's BGRx negotiation, as do the GameStream-resolve and portal-mirror paths, PyroWave, and NVENC prefs. - RGB-direct's unaligned modes default to the true-extent direct import (PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0 restores the padded-copy staging). Guarded-tested on Van Gogh with the kernel journal watched: clean, and at 5.38 ms p50 the fastest 1080p encode path measured on that hardware. The EFC only exists on Mesa >= 26, where the codedExtent-driven session_init padding is guaranteed (verified back to Mesa 24.2). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-21 14:01:28 +02:00
3 people	d2daeacc60	fix(pyrowave): per-session raw-dmabuf zero-copy capture on the Linux NVIDIA host ... A PyroWave session on an NVIDIA-auto host was forced onto CPU-RGB capture (session_plan flipped gpu=false): Mutter blits tiled->LINEAR, we mmap + de-pad ~30 MB, the encoder re-uploads it - three full-frame CPU touches per frame at 5120x1440 while an HEVC session on the same box rides the tiled EGL/CUDA zero-copy. The dmabuf passthrough + Vulkan tiled import were already validated (8dc5d672) but only reachable via the global PUNKTFUNK_ENCODER=pyrowave lab policy. ZeroCopyPolicy gains pyrowave_session (from OutputFormat.pyrowave, i.e. the negotiated codec): the capturer skips the NVENC-only EGL->CUDA importer, takes the raw-dmabuf passthrough, and advertises the wavelet encoder's Vulkan-importable modifiers so Mutter+NVIDIA negotiates tiled zero-copy. The forced-CPU flip in session_plan is gone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-19 16:25:00 +02:00
enricobuehlerandClaude Fable 5	0e977817f9	feat(hdr): GNOME 50 HDR screencast capture + Linux Main10 encode ... GNOME 50 (Mutter MR 4928, PipeWire >= 1.6) added HDR screen sharing for monitor streams: 10-bit PQ formats (xRGB_210LE/xBGR_210LE) with MANDATORY BT.2020 + SMPTE-2084 colorimetry props, advertised while the mirrored monitor is in BT.2100 colour mode. Wire the Linux host into it end-to-end on the GameStream desktop-mirror path (PUNKTFUNK_VIDEO_SOURCE=portal): * pf-frame: PixelFormat::X2Rgb10/X2Bgr10 (DRM XR30/XB30; X2Bgr10 is the Windows Rgb10a2 layout) + fourccs. * pf-capture: want_hdr portal offer — HDR-only LINEAR-dmabuf pods with MANDATORY PQ/BT.2020 props (SHM excluded: Mutter's SHM record path paints 8-bit ARGB32 regardless of format; tiled excluded: the EGL de-tile blit is 8-bit RGBA8), negotiated-colorimetry parse, generic HDR10 hdr_meta(), packed-10-bit CPU cursor blend, a process-wide SDR downgrade latch on negotiation timeout, and a DisplayConfig BT.2100 colour-mode probe (gnome_hdr_monitor_active). * pf-encode: libav NVENC X2RGB10->P010 swscale (BT.2020 limited) -> HEVC Main10 / 10-bit AV1 with PQ VUI; VAAPI 10-bit on both paths (CPU P010 upload + dmabuf XR30 scale_vaapi p010/bt2020); can_encode_10bit now probes for real on Linux; 10-bit sessions route around the 8-bit-only Vulkan-video/direct-NVENC backends. * GameStream: host_hdr_capable() Linux arm, live monitor-HDR check at RTSP honor time, capturer-pool reuse keyed on HDR-ness, gs_bit_depth covers the new formats. New `punktfunk-host hdr-probe` diagnostic and a PUNKTFUNK_SPIKE_HDR spike lever. * Native plane stays honestly 8-bit via capturer_supports_hdr(): Mutter RecordVirtual streams are SDR-only upstream (GNOME 50 and 51-dev), so virtual-display sources cannot deliver HDR yet. Validated on the RTX 5070 Ti (GNOME 50.3 / PipeWire 1.6.8): the Main10 probes pass and the ignored nvenc_hdr10_smoke GPU test emits an IDR that ffprobe reads as Main 10 / yuv420p10le / bt2020nc / smpte2084 / limited. Live HDR capture negotiation still needs an HDR monitor on glass; VAAPI 10-bit needs the AMD box. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-19 09:30:55 +02:00
enricobuehlerandClaude Opus 4.8	ebd9967547	feat(pyrowave): Windows host encoder — separate-plane zero-copy D3D11→Vulkan ... Wire PyroWave into the Windows host (design/pyrowave-windows-host-zerocopy.md). Before this a macOS client + Windows host that both selected PyroWave silently ran HEVC: the host never advertised CODEC_PYROWAVE and open_video_backend bailed. Approach (zero-copy, no GPU→CPU→GPU): pyrowave owns its own Vulkan device (create_device_by_compat, by render-GPU vendor/device-id — NOT LUID, invalid in Session 0). The capturer runs a BGRA→YUV BT.709-limited CSC (matching rgb2yuv.comp) into TWO SEPARATE shareable plane textures — full-res R8 Y + half-res R8G8 CbCr — which the encoder imports into pyrowave's device. Separate single/two-component textures import reliably on NVIDIA at any size; a single planar NV12 import does NOT (the vendored interop test: "only very specific resource sizes" — confirmed on-glass: 1024² fine, 720p/1080p/1440p garbage). A shared D3D11 fence, signalled after the CSC, is imported as a Vulkan timeline semaphore so the wavelet read is ordered after it. - pf-encode: enc/windows/pyrowave.rs (Encoder impl, two-plane import + Linux-style plane views); host_wire_caps advertises CODEC_PYROWAVE on Windows when the backend isn't Software; open_video_backend routes a negotiated PyroWave session first; pyrowave-sys on the Windows target; interop confirmed at open → clean HEVC fallback. - pf-encode: shared, unit-tested enc/pyrowave_wire.rs (single source of truth for the client-facing AU framing); Linux encoder uses it too. - pf-capture: dxgi.rs BgraToYuvPlanes CSC; idd_push.rs pyrowave mode — forces the virtual display SDR (the VideoProcessor can't ingest the FP16 HDR ring), a two-plane shareable out-ring, a shared fence passed every frame (so a rebuilt encoder re-imports it). Threaded via OutputFormat::pyrowave. - pf-frame: D3d11Frame::pyro carries the CbCr plane + fence; OutputFormat::pyrowave. Verified on .173 (RTX 4090): full-host build + clippy -D warnings (nvenc,amf-qsv) + fmt --all --check; pyrowave_wire unit tests; pyrowave_win_smoke GPU test round-trips distinct Y/Cb/Cr (100/180/60) exactly at 1024²/720p/1080p/1440p; Stage-0 interop validated in the real Session-0 service context on-glass. Deployed to the box. Owed: final on-glass picture/latency confirmation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-18 11:39:44 +02:00
enricobuehlerandClaude Opus 4.8	27a5d8daac	refactor(host/W6.2): extract virtual-display orchestration into the pf-vdisplay crate ... vdisplay.rs + vdisplay/* (the per-compositor Linux backends — KWin zkde-screencast, wlroots swaymsg, Mutter RemoteDesktop, Hyprland — and the Windows IddCx/pf-vdisplay driver backend, behind one VirtualDisplay trait; the mode-conflict admission registry, the display policy/identity/custom-preset state, and the session-env / gamescope routing) move into crates/pf-vdisplay (plan §W6). The DDC/CI panel-power control (used only here) and the KWin zkde protocol XML move with it. This completes the host-crate decomposition: capture, encode, inject, and vdisplay are now four subsystem crates over the shared leaves, and punktfunk-host is the orchestrator (serve/supervisor + native + gamestream + mgmt). Coupling breaks (all down-only, cargo-tree acyclic): - capture::dxgi identity -> pf_frame::dxgi; win_display/monitor_devnode/ console_session_mismatch -> pf-win-display leaf; can_open_another_session -> pf-encode (the NVENC session-budget admission gate — acyclic peer edge). - The registry's DisplayCreated/DisplayReleased emits into the host SSE event bus invert to a leaf hook: pf-vdisplay emits a neutral DisplayEvent to a host-registered DISPLAY_EVENT_SINK, so it never reaches the orchestrator's events module. - The IddCx driver module is renamed pf_vdisplay -> driver (its old name collided with the crate name through the host's `mod vdisplay` shim glob). The host keeps `mod vdisplay { pub use pf_vdisplay::* }` so every crate::vdisplay::* path (serve/mgmt/native/the capture FrameChannelSender seam) is unchanged; the heavy deps (wayland/ashpd/tokio + the zkde protocol) moved with the crate. Co-authored: a fail-closed IOCTL-reply-length security fix (reject short/zeroed pf-vdisplay driver replies before trusting protocol_version/target_id/wudf_pid/luid, security-review 2026-07-17) rides this commit in the moved driver module. Verified: Linux clippy -D warnings (pf-vdisplay + host nvenc,vulkan-encode,pyrowave --all-targets) + pf-vdisplay 63/63 + host 167/167 tests; Windows clippy -D warnings (pf-vdisplay --all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-17 12:14:08 +02:00
enricobuehlerandClaude Opus 4.8	94ca4041ca	refactor(host/W6.2): extract the frame-capture backends into the pf-capture crate ... capture/linux (PipeWire portal) + capture/windows (IDD direct-push: dxgi mechanics, idd_push + submodules, synthetic_nv12) + pwinit move into crates/pf-capture behind the Capturer trait + synthetic sources (plan §W6). The crate speaks pf-frame (CapturedFrame/PixelFormat + the DXGI identity), pf-zerocopy (CUDA import), and the pf-win-display leaves, and NEVER pf-encode — the capture->encode edge is one-way. This completes the deliberate capture/encode crate split (the invasive path the plan had merged into one pf-media): capture and encode are now separate subsystem crates sharing only pf-frame. Four seams keep the capturer off the orchestrator: - VirtualOutput is EXPLODED into primitives (remote_fd/node_id/preferred_mode/ keepalive) by the host facade, so pf-capture never depends on the vdisplay type; - FrameChannelSender: the sealed-channel delivery is a Send+Sync closure the host facade builds from the pf-vdisplay control device + send_frame_channel IOCTL and hands in; ChannelBroker holds the closure instead of the control HANDLE (the whole-desktop handle-duplication security boundary is byte-for-byte unchanged); - console_session_mismatch + desktop_bounds live in pf-win-display (leaf peers); - pwinit moves here (audio caller -> pf_capture::pwinit). The host keeps capture.rs as a thin BRIDGE: it re-exports the vocabulary + capturer types (every crate::capture::* path is unchanged) and keeps open_portal_monitor / capture_virtual_output, which resolve the ZeroCopyPolicy + FrameChannelSender and call into pf-capture. verify_is_wudfhost + install_gpu_pref_hook are re-exported (the gamepad-channel bootstrap + the main.rs subcommand consume them). Co-developed: the resident-HID-mouse compose-kick hook (HID_COMPOSE_KICK + the HID-first cursor kick + _display_wake) rides this commit into pf-capture; the host mouse_windows registration side lands separately on top. Verified: Linux clippy -D warnings (pf-capture + host nvenc,vulkan-encode,pyrowave --all-targets) + host tests 299/299; Windows clippy -D warnings (pf-capture --all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-17 11:28:56 +02:00
enricobuehlerandClaude Opus 4.8	1de83ba51d	refactor(host/W6.2): make the Linux capture→encode edge one-way via ZeroCopyPolicy ... The PipeWire capture thread re-derived three encode-backend facts by calling encode::{linux_zero_copy_is_vaapi, resolved_backend_is_gpu, pyrowave_capture_modifiers} — a capture→encode back-reference that would force pf-capture to depend on pf-encode (and vice versa, since encode already speaks the frame vocabulary). Resolve them ONCE in the host capture facade (which may reach crate::encode) as a ZeroCopyPolicy { backend_is_vaapi, backend_is_gpu, pyrowave_modifiers } and thread it into PortalCapturer::open / from_virtual_output → spawn_pipewire → pipewire_thread. capture/linux/mod.rs now makes ZERO crate::encode calls — the edge is one-way (plan §2.4 / §W6), so pf-capture can be extracted depending only on pf-frame (not pf-encode). pyrowave_modifiers is computed by the facade whenever the encoder pref is pyrowave (which implies the VAAPI backend); the thread still consumes them only inside its existing vaapi_passthrough guard, so behavior is unchanged. Verified: Linux clippy -D warnings (host nvenc,vulkan-encode,pyrowave --all-targets); Windows clippy nvenc,amf-qsv --all-targets Finished exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-17 10:20:17 +02:00
enricobuehlerandClaude Opus 4.8	b168790e0a	refactor(host/W6.2): extract the shared frame/format vocabulary into the pf-frame leaf crate ... The captured-frame types both capture (producer) and encode (consumer) speak — PixelFormat, OutputFormat, CursorOverlay, CapturedFrame, FramePayload, DmabufFrame, drm_fourcc — move into crates/pf-frame, alongside the small pure helpers that ride the same seam: hdr (HDR static metadata / in-band SEI), metronome (the metronomic-stall detector), thread_qos (per-thread scheduling QoS), session_tuning (Windows process tuning), and the Windows DXGI capture IDENTITY (WinCaptureTarget, D3d11Frame, pack_luid, make_device + the GPU scheduling-priority hardening it applies) (plan §W6). This is the crate that breaks the capture<->encode cycle: FramePayload's GPU variants own their backends from BELOW (Cuda -> pf_zerocopy::DeviceBuffer, D3d11 -> dxgi::D3d11Frame), so encode can speak the vocabulary without a path to capture, and vice versa. The Windows DXGI identity moving here lets capture, encode, and pf-vdisplay share ONE WinCaptureTarget/device factory instead of the old capture<->encode<->vdisplay reach-in. The host keeps thin facades: capture.rs re-exports the vocabulary (crate::capture::{PixelFormat,…} unchanged); capture/windows/dxgi.rs keeps the win32u GPU-preference hook + HDR/video-engine converters + self-test and re-exports the identity; native.rs re-exports boost_thread_priority from pf_frame. crate::hdr/metronome/session_tuning callers rewired to pf_frame::*. metronome's Metronome::new gained a Default impl (new_without_default fires once the type is public across the crate boundary). Verified: Linux clippy -D warnings (pf-frame --all-targets + host nvenc,vulkan-encode,pyrowave --all-targets) + 9/9 pf-frame tests; Windows clippy nvenc,amf-qsv --all-targets Finished exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-17 10:03:56 +02:00
enricobuehlerandClaude Opus 4.8	85bc5b9a3f	refactor(host/W6.2): extract the Linux zero-copy GPU plumbing into the pf-zerocopy leaf crate ... linux/zerocopy/* (CUDA context/buffers + EGL/Vulkan dmabuf import + the isolated import worker) and linux/dmabuf_fence.rs move wholesale into crates/pf-zerocopy, so the coming pf-frame vocabulary crate (FramePayload::Cuda owns a DeviceBuffer) and the pf-encode/pf-capture subsystem crates can reach the GPU plumbing without the host orchestrator in between (plan §W6). Content stays Linux-only; the crate compiles to an empty lib elsewhere, so dependents carry a plain dependency. drm_fourcc deliberately does NOT move: it consumes the frame vocabulary (PixelFormat), which sits ABOVE pf-zerocopy — it lives with capture for now and moves into pf-frame next. cuda's ffi re-export bumps pub(crate)->pub (the raw CUdeviceptr vocabulary is consumed across the crate boundary by the encode backends). A crate::zerocopy shim module keeps every existing path valid until capture/encode themselves move out. Verified: Linux clippy -D warnings (pf-zerocopy --all-targets + host nvenc,vulkan-encode,pyrowave --all-targets) + 17/17 pf-zerocopy tests + 321/321 host tests; Windows clippy nvenc,amf-qsv --all-targets Finished exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-17 09:41:15 +02:00
enricobuehlerandClaude Opus 4.8	aaa3dcec32	refactor(host/W4): make the capture→encode edge one-way (OutputFormat back-ref) ... The capture.rs facade no longer re-derives the encode backend. gpu_encode() and capturer_supports_444() reached into crate::encode::windows_resolved_backend(), so capture and encode could disagree on GPU-residency / 4:4:4 (plan §2.4). Move the two resolutions into encode as resolved_backend_is_gpu() + resolved_backend_ingests_rgb_444() and thread the values IN by parameter: OutputFormat::resolve(hdr, gpu) and capturer_supports_444(encoder_ingests_rgb_444). Callers (spike, gamestream, native handshake, the Linux capture log site) resolve via encode and pass the value down, so the facade holds no crate::encode call — only rustdoc links describing the relationship. Completes task #8 of W4. Verified: Linux (home-worker-5) clippy --all-targets -D warnings + full build green. Windows (.173) verify owed — box was offline this session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-16 23:59:41 +02:00
enricobuehlerandClaude Opus 4.8	5249d31dfa	feat(host/linux): cursor-as-metadata — pointer in gamescope streams, no perf hit ... gamescope draws its pointer on a hardware DRM cursor plane that never enters the framebuffer feeding its PipeWire capture node, so captured frames arrive cursorless. Rather than force the producer's Embedded full-frame composite, request the pointer as PipeWire SPA_META_Cursor and composite it ourselves — a ≤256×256 blit into the encoder-OWNED surface, never the compositor's read-only dmabuf. Capture (capture/linux/mod.rs, capture.rs): - choose_cursor_mode() gates on available_cursor_modes(): Metadata > Embedded > Hidden (defaults Embedded on query error — never silently lose the cursor). Applied on both the plain and remote-desktop portal paths. - build_cursor_meta_param() adds a SPA_PARAM_Meta pod requesting SPA_META_Cursor (bitmap up to 256x256) to the connect params on every path. - CursorState parses spa_meta_cursor (id 0 = hidden; position - hotspot; bitmap re-read only when bitmap_offset != 0), normalizing RGBA/BGRA/ARGB/ABGR. Updated in .process before the corrupted/size-0 skip so cursor-only Mutter buffers still track movement. - CapturedFrame gains cursor: Option<CursorOverlay> (Arc rgba + serial) riding the GPU (Dmabuf/Cuda) payloads; the CPU de-pad path composites inline. GPU composite into each zero-copy backend's owned surface: - Vulkan Video + PyroWave: folded into the shared rgb2yuv.comp CSC shader — cursor sampled and alpha-mixed over RGB before the YUV convert (correct chroma, no extra pass). binding 3 (combined image sampler) + 16B push constant, per-slot cursor image uploaded only on serial change. spv regenerated. - CUDA/NVENC: real on-GPU kernel (cursor_blend.cu -> cursor_blend.ptx, compute_75 Turing baseline, JIT-forward) with blend_argb/blend_yuv444/ blend_nv12 (BT.709 limited, matching the shader). Loaded via the hand-rolled libcuda fn-table; blended into the ring InputSurface after copy, degrading to no-cursor on any failure — never drops a frame. VAAPI (AMD/Intel fallback) deferred: Vulkan Video already covers those GPUs; blind libva struct-layout FFI shouldn't ship unverified. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-15 17:19:55 +02:00
enricobuehlerandClaude Fable 5	e1d7fa2a30	fix(host): activate lid-closed pf-vdisplay targets — explicit CCD path-commit fallback ... A lid-closed laptop defeats both activation stages for a fresh IddCx target: the clamshell lid policy suppresses the new-monitor auto-activate, and the SDC_TOPOLOGY_EXTEND preset returns success without committing a path for the IDD — so every session retry burned ~10s in resolve_target_gdi and the stream died with "not yet an active display path" after 8 attempts (RDP/Parsec still work there: neither needs a NEW console display path). Field report: Windows laptop host, Intel iGPU, lid closed, v0.10.1. New activate_target_path() (win_display.rs) is the supplied-config apply Windows' own display Settings uses to turn a monitor on, which doesn't consult the lid policy: QueryDisplayConfig(QDC_ALL_PATHS), keep every active path verbatim, append the target's inactive path with a source no active display is using (never a clone), both mode idxs DISPLAYCONFIG_PATH_MODE_IDX_INVALID, then SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES | SDC_SAVE_TO_DATABASE — SAVE_TO_DATABASE so the next same-identity ADD auto-activates from the persistence DB and skips the ladder. Wired as the THIRD stage of resolve_target_gdi; the on-glass-validated auto-activate → force-EXTEND order is unchanged. Also sweep stale "SudoVDA" out of logs/errors and current-behavior doc comments (the backend was removed; pf-vdisplay is the sole one): the capture error now names pf-vdisplay, the HDR toggle logs virtual-display, and the not-active warns list the exhausted fallbacks. Genuinely historical SudoVDA notes stay. cargo check + clippy green on the Windows box; on-glass lid-closed repro still owed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-15 16:44:41 +02:00
enricobuehlerandClaude Opus 4.8	2271f67202	style: rustfmt the recovery + resize changes (Windows CI fmt check) ... The `cargo fmt --check` step on the x86_64-pc-windows-msvc job was failing: the mid-stream loss-recovery and resize-overlay commits landed with unformatted wraps across pf-presenter, pf-client-core, punktfunk-core, pf-console-ui, and a few host files. Applied `cargo fmt`, and hand-relocated two trailing comments in session.rs (a decoded-frame note and the wrap-counter note) to their own lines so rustfmt no longer column-aligns the following comment block to a deep indent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 23:25:06 +02:00
enricobuehlerandClaude Opus 4.8	e55ff1bb28	feat(recovery): clean mid-stream loss recovery — freeze-until-reanchor + AMD LTR-RFI ... Removes the "gray frames with motion" artifact on Vulkan-Video clients and lets AMD/NVENC hosts re-anchor after loss WITHOUT a 20-40x IDR spike. Client (pf-client-core): after a reference loss the hardware decoder conceals the missing-reference deltas (on RADV, a gray plate with new motion painted over) and returns Ok. The pump now freezes on the last good picture until a clean re-anchor instead of showing the concealment — lifting on a real IDR, an intra-refresh recovery mark (2nd wave boundary), or an LTR-RFI recovery anchor (1st). The frame_index gap is the early, precise loss signal and drives an RFI request. Host recovery signals (inert unless the backend supports them): - USER_FLAG_RECOVERY_POINT — intra-refresh wave boundary (NVENC constrained GDR). - USER_FLAG_RECOVERY_ANCHOR — AMD LTR reference-frame-invalidation recovery frame. AMD LTR-RFI (encode/windows/amf.rs) — the AMD twin of NVENC RFI. AMF's AVC/HEVC API has no constrained-intra property (intra-refresh cannot heal; PSNR-proven), so the only clean-recovery lever is user LTR: mark frames as long-term references, and on loss force the next frame to re-reference the newest known-good one — a clean P-frame, not an IDR. Two rotating LTR slots, ~0.5s mark cadence, on by default for AVC/HEVC (PUNKTFUNK_NO_AMF_LTR disables). invalidate_ref_frames picks the newest LTR before the loss; a range older than the live slots falls back to a keyframe. Protocol (punktfunk-core): RfiRequest control message + NativeClient::request_rfi(). Host: RfiRequest dispatch -> invalidate_ref_frames (IDR fallback); an RFI success anchors the keyframe cooldown so the client's frames_dropped echo of the same loss is coalesced away rather than emitting a redundant IDR. Spike: synthetic NV12 GPU source for headless AMF encoder testing. Validated: core rfi_request_roundtrip; pf-client-core 31 unit tests (incl. an_rfi_anchor_lifts_immediately); punktfunk-host builds + 271 tests on Linux; punktfunk-host builds clean on Windows; real AMD iGPU spike (invalidate at frame 90 forced re-reference to LTR frame 60 — 180 frames, keyframes=1, no recovery IDR). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-11 17:31:17 +02:00
enricobuehlerandClaude Fable 5	0dacb37088	feat(linux): zero-copy 4:4:4 — the EGL worker converts to planar YUV444 on the GPU ... A 4:4:4 session no longer falls to the CPU path (SHM capture + swscale RGB→YUV444P + re-upload — the fps-ceiling triple tax). The zero-copy worker grows a Yuv444Blit: three full-res R8 GL passes (the proven BT.709 coefficients; studio or full range per PUNKTFUNK_444_FULLRANGE, read by both processes so pixels and VUI flip together) into ONE stacked 3-plane pitched CUDA allocation — which keeps the worker↔host wire and IPC single-plane. The encoder copies the planes into ffmpeg's yuv444p CUDA surface and hevc_nvenc emits Range-Extensions 4:4:4 natively. ImportKind::Tiled444 is APPENDED to the worker protocol (a worker outliving a replaced host binary must keep the old tags stable; an old worker just errors the import, which the fail machinery already handles). A 4:4:4 session on a LINEAR/gamescope capture — no convert wired there — fails with a clear message instead of letting hevc_nvenc silently subsample. caps().chroma_444 now keys off the session (it missed the GPU path when keyed off the swscale's existence). Live-verified on the CachyOS VM (RTX 5070 Ti): per-frame "imported to CUDA yuv444=true", stream Rext/yuv444p/bt709 in both tv and pc range, no CPU-path warning. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 18:09:28 +02:00
enricobuehlerandClaude Fable 5	1fcf9e11ec	fix(video): honor the signaled CSC matrix end-to-end + tvOS HDR presentation ... Clients derive Y'CbCr->RGB from the stream's SIGNALED matrix x range x depth via shared csc rows (Rust csc_rows + Swift CscRows) instead of hardcoded 709/2020 - a BT.601-signaled stream (a Linux host's RGB-input NVENC) no longer renders with a constant hue error. Host-side signaling made honest across NVENC/VAAPI/openh264/GameStream and the session plan's chroma/bit-depth. Decoded color-bar fixtures (601/709 x limited/full) pin the math in tests on both cores. Same presenter, tvOS HDR: tvOS has no Metal EDR API and a bare PQ colorspace tag composites UNTONE-MAPPED (the "overblown" Apple TV report), so HDR now splits on the display's live EDR headroom - PQ passthrough when the per-session AVDisplayManager mode switch landed (a real HDR10 output tone-maps itself), else an in-shader PQ->SDR tone-map (203-nit reference white, extended-Reinhard 1000-nit knee, 2020->709) into the proven SDR layer config. The 10-bit stream keeps its full decode depth either way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 16:59:01 +02:00
enricobuehlerandClaude Opus 4.8	28e04e5e62	feat(protocol,host): negotiate video codec + add a GPU-less software (openh264) encode path ... Phase 1 of codec negotiation, and the Linux software H.264 encode path it unblocks. **Codec negotiation (core `quic`):** - `Hello.video_codecs` (bitfield: CODEC_H264/HEVC/AV1) — the client advertises what it can decode; appended as a trailing byte (older client → 0 = HEVC-only, back-compat). - `Welcome.codec` — the single codec the host resolved and will emit; trailing byte (older host → HEVC). - `resolve_codec(client, host_capable)` picks the shared codec (precedence HEVC > AV1 > H.264) or `None` → the host refuses honestly rather than sending an undecodable stream. - Roundtrip + back-compat tests; cbindgen exports the CODEC_* constants. **Software encoder (host):** - The openh264 `OpenH264Encoder` (was Windows-only) is now built on Linux too — it's platform-agnostic (consumes CPU RGB `CapturedFrame`s, statically-bundled openh264). `openh264` moved to the shared linux+windows Cargo target. - `PUNKTFUNK_ENCODER=software` selects it: `open_video` gains a `software` branch (H.264 only), and `session_plan::resolve_encoder` / `capture::gpu_encode` resolve `EncoderBackend::Software` → `output_format().gpu = false`, so the portal capturer delivers CPU RGB. Explicit-only (auto never picks it — a box with a dead driver still has /dev/nvidiactl and would mis-resolve NVENC). **Host codec resolution (`punktfunk1`):** - The native path no longer hardcodes HEVC: it resolves the codec from the client's advertised set ∩ the host's capability (`Codec::host_wire_caps`: software→H.264, else HEVC), threads it through `SessionPlan.codec`, and opens the encoder + validates reconfigures at that codec. A software host + HEVC-only client is refused with a clear error. - 4:4:4 is gated on HEVC (it's HEVC-only). **Probe:** advertises H264|HEVC|AV1 and logs the resolved codec. Validated on the GPU-less dev box: negotiation is live end-to-end (probe advertises 0x07 → host resolves H.264 → Welcome reports it → plan = Software/H264), and the openh264 unit test (CPU RGB → AnnexB IDR) now runs on Linux. Full capture→encode still needs a GPU on this box — every compositor screencast path (KWin GL, gamescope VK_EXT_physical_device_drm, wlroots EGL) requires one; software render (llvmpipe/pixman) can't be captured — so this box exercises negotiation + encoder, not live capture. The software path unblocks GPU-less-*encode* boxes that still have a display GPU. Phase 2 (clients advertising real codecs + decoding per Welcome.codec) is a follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-01 23:13:39 +00:00
enricobuehlerandClaude Sonnet 4.6	b9c8d5ab43	refactor(host/windows): collapse Windows capture to IDD-push only ... Remove DXGI Desktop Duplication (DuplCapturer), Windows.Graphics.Capture (WgcCapturer), the two-process SYSTEM+helper relay (virtual_stream_relay / HelperRelay / DesktopWatcher / composed_flip), and the five source files that implemented them. IDD direct-push is now the sole Windows capture path; the session topology is always SingleProcess. Deleted files: wgc.rs, wgc_relay.rs, desktop_watch.rs, composed_flip.rs, windows/wgc_helper.rs (+ wgc-helper subcommand in main.rs). dxgi.rs is kept but carved to shared GPU primitives only (make_device, HdrP010Converter, VideoConverter, install_gpu_pref_hook, WinCaptureTarget, pack_luid) — ~2237 lines of DDA-only code removed; imports cleaned. capture.rs: IDD-push open failure fails the session cleanly (no fallback). Adds capturer_supports_444() — returns false on Windows (IDD-push 4:4:4 is a follow-up), replacing the stale single_process gate in 4:4:4 negotiation. session_plan.rs: CaptureBackend{Dda,Wgc} and SessionTopology::TwoProcessRelay removed. config.rs: no_helper/force_helper/no_wgc/capture_backend/secure_dda removed. merged_env_block relocated from wgc_relay to windows/interactive.rs. Linux cargo check clean. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-06-30 06:46:52 +00:00
enricobuehlerandClaude Opus 4.8	7612238a59	feat(audio): end-to-end 5.1/7.1 surround across the native path + all clients ... Adds negotiated 5.1/7.1 surround to the punktfunk/1 protocol and every client (previously stereo-only): - core: new shared `audio` layout table (LAYOUT_51/71 + identity multistream mapping, canonical wire order FL FR FC LFE RL RR SL SR); Hello/Welcome `audio_channels` negotiation via the trailing-byte back-compat pattern (old peers fall back to stereo); C-ABI `punktfunk_connect_ex6`, `punktfunk_connection_audio_channels`, and in-core multistream decode `punktfunk_connection_next_audio_pcm` for embedders without a multistream Opus decoder. Real-libopus channel-identity round-trip test. - host: native audio thread captures + Opus-(multi)stream-encodes at the negotiated count (with a cross-session cached-capturer channel-mismatch fix); GameStream surround unified onto the safe `opus::MSEncoder`, dropping `audiopus_sys` (~4 unsafe blocks) and un-gating Windows GameStream surround; WASAPI loopback capture relaxed to 2/6/8 with the correct dwChannelMask. - clients: Linux (PipeWire), Windows (WASAPI), Android (AAudio) decode via `opus::MSDecoder` + render multichannel; Apple decodes in-core to PCM → AVAudioEngine with an explicit wire-order channel layout; each gains a Stereo/5.1/7.1 setting. `punktfunk-probe --audio-channels N` is the headless validator. Verified on Linux: core/host/linux/probe test suites + the Android Rust (cargo-ndk) build, clippy -D warnings, and rustfmt all green. Windows/Apple builds, all on-glass checks, and the live native loopback are pending (CI / a free box). Also lands the concurrent in-tree HEVC 4:4:4 host work (PUNKTFUNK_444): it shares the same touched files (quic.rs, punktfunk1.rs, encode/*, ...) and so cannot be committed separately from the surround changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-28 21:11:05 +00:00
enricobuehlerandClaude Opus 4.8	33328ee417	docs(host): prove the last 3 files + crate-root deny (unsafe-proof program 4/N, final) ... Completes the unsafe-proof program now that the parallel WIP has landed: - idd_push.rs (25 sites), nvenc.rs (7), punktfunk1.rs (21): a SAFETY proof on every unsafe block — D3D11/DXGI COM (same-device textures, immediate-context single-thread, keyed-mutex-held convert), the NVENC SDK table (versioned POD, register/map/lock-bitstream pairing), cross-process shm reads (atomic magic/generation handshake), and the C-ABI harness (each call cross-checked against its abi.rs `# Safety` doc). No SUSPECT (UB) blocks. - capture.rs / encode.rs: the parent-module deny is restored (their WIP children are now proven), and main.rs gains a crate-root #![deny(clippy::undocumented_unsafe_blocks)] — the permanent catch-all gate so no future unsafe block anywhere in the crate can land without a proof. - Fixed 4 blocks the agents missed: unsafe blocks nested inside `assert_eq!(...)` macro args (the comment-above-statement didn't associate) — hoisted to a `let`. - rustfmt-canonicalized the Windows files (the agents' SAFETY comments + some pre-existing 1.9.0 drift) so `cargo fmt --all --check` is clean. Verified: cargo clippy -p punktfunk-host --all-targets -- -D warnings AND cargo fmt -p punktfunk-host --check both green with the crate-root deny active. Windows cfg(windows) re-verified on the box next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 09:57:00 +00:00
enricobuehlerandClaude Opus 4.8	79c8641c3f	docs(host): prove unsafe blocks in the Windows + cross-platform files + gate them (unsafe-proof program 3/N) ... Continues the unsafe-proof program across the Windows/cross-platform host files (~75 blocks, 21 files), each with a SAFETY proof of the real invariant and a per-file #![deny(clippy::undocumented_unsafe_blocks)] gate: capture/windows: dxgi.rs, wgc_relay.rs, wgc.rs, desktop_watch.rs, composed_flip.rs (windows-rs COM: interface validity, same-D3D11-device textures, immediate-context single-thread, borrowed args outlive the call) windows: service.rs (SCM/token/CreateProcessAsUserW/event handles — OwnedHandle liveness, no double-close/signal race), win_display, wgc_helper, interactive vdisplay/windows: manager.rs, pf_vdisplay.rs (SwDeviceCreate/IddCx/ioctl handle liveness via the OnceLock VDM singleton + OwnedHandle) encode/windows: ffmpeg_win.rs (full AVBufferRef refcount audit — balanced, NO leaks, unlike the vaapi sibling), sw.rs cross-platform: gamestream/audio.rs (libopus), gamestream/stream.rs (sendmmsg), inject/windows/sendinput.rs, audio/windows/wasapi_mic.rs, session_tuning.rs, vdisplay.rs Two findings (handled separately): - wgc_relay.rs `unsafe impl Sync for HelperRelay` is UNSOUND (its mpsc Receiver is !Sync) though not live-exploited — marked SUSPECT inline; fix pending box check (it touches the in-flight punktfunk1.rs). - capture.rs / encode.rs (PARENT modules of the WIP idd_push.rs / nvenc.rs) do NOT get the file deny yet — it would propagate the lint into the undocumented WIP children. The deny lands there once those are documented (after the WIP commits). Linux-visible parts verified green (cargo clippy -p punktfunk-host --all-targets -- -D warnings). The cfg(windows) deny gates are box-verified next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-26 09:23:25 +00:00
enricobuehlerandClaude Opus 4.8	fced221684	refactor(windows-host): confine platform code under windows/ + linux/ folders (Goal-1 stage 6) ... Move 36 platform-specific files into per-module `windows/` and `linux/` subfolders (and the shared HID codecs into `inject/proto/`): capture/{windows,linux}/ encode/{windows,linux}/ inject/{windows,linux,proto}/ audio/{windows,linux}/ vdisplay/{windows,linux}/ src/windows/ (service, wgc_helper, win_adapter, win_display) src/linux/ (dmabuf_fence, drm_sync, zerocopy/) Done with `#[path]`, NOT a module rename: every file moves into its folder while the `crate::*::*` module names stay FLAT, so all caller paths and every internal `super::`/`crate::` reference are unchanged — only the parent `mod` decls gained `#[path = "..."]`. This is the codebase's existing pattern (inject's gamepad_windows) and makes the move byte-identical in behaviour with ZERO reference churn, far lower risk than collapsing to a single `crate::capture::windows::` namespace (that deeper rename is an optional follow-on; this delivers the cfg-sprawl folder confinement the stage is about). Done LAST, after the semantic stages, so the path churn didn't fight them. Verified: Linux cargo check + clippy (-D warnings) clean; my mod-decl changes fmt-clean (the 3 remaining fmt diffs are pre-existing local-rustfmt-version skew that moved with their files); all 36 `#[path]` targets exist; no internal `#[path]`/`include!`/file-child-mod in any moved file (the inline `mod X {` blocks are self-contained). Box build to follow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 18:53:45 +00:00
enricobuehlerandClaude Opus 4.8	d32e161070	feat(windows-host): OutputFormat into the capturer — kill the dxgi back-reference (Goal-1 stage 5, tightening 1) ... The headline §2.3 seam tightening (the explicit Stage-3 deferral; §5's "highest-severity coupling"): the capturer is now TOLD its output format instead of re-deriving the encode backend. New `capture::OutputFormat { gpu, hdr }`, resolved once per session and passed INTO capture_virtual_output: * native punktfunk/1 path: `SessionPlan::output_format()` (gpu = encoder.is_gpu(), from the already-resolved plan.encoder — no second probe; hdr = plan.hdr). * GameStream + spike paths: `OutputFormat::resolve(hdr)` (gpu from the single `gpu_encode()` source, which maps windows_resolved_backend()). `capture/dxgi.rs DuplCapturer::open` takes `gpu` in and its internal `!matches!(windows_resolved_backend(), Software)` recompute is DELETED — the capture layer no longer re-calls the encode layer (the back-reference that could let capture and encode disagree on whether frames are GPU-resident, plan §2.3/§5). The relay's secure-desktop DDA passes `gpu_encode()` likewise. Behavior-preserving: the `gpu` passed in equals the value the capturer used to compute (same encode-backend resolution). The DDA opens keep `want_hdr=false` (the SDR fallback, unchanged). Tightenings 2 (HDR/release -> VirtualLease) and 3 (EncoderCaps) split off: (2) needs the monitor-generation carried on the lease + the keepalive becoming Box<dyn VirtualLease> — that's the §2.5 ownership-model change (CURRENT_MON_GEN / sudovda::wait_for_monitor_released), so it moves there; (3) is a small additive follow-on. Documented in the plan. Verified: Linux cargo check + clippy (-D warnings) + fmt clean. Box build to follow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 18:37:48 +00:00
enricobuehlerandClaude Opus 4.8	d40f96c3da	feat(windows-host): SessionPlan — resolve capture/topology/encoder once per session (Goal-1 stage 3) ... New src/session_plan.rs: a Copy `SessionPlan { capture, topology, encoder, bit_depth, hdr }` resolved ONCE from HostConfig (+ the negotiated bit_depth) at the top of `virtual_stream`, logged, and threaded through build_pipeline_with_retry/build_pipeline. The three scattered Windows dispatch points now read this one typed artifact instead of re-deriving from config (plan §2.4, the "capture and encode disagree on the backend" hazard): * capture: capture::capture_virtual_output takes a CaptureBackend IN (was re-reading config().idd_push / capture_backend / no_wgc internally). CaptureBackend::resolve() is the single resolver, shared with the GameStream + spike call sites. * topology: virtual_stream reads plan.topology; should_use_helper is deleted (its body is session_plan::resolve_topology, verbatim). The IDD-push reconnect-preempt guard reads plan.capture too. * encoder: recorded as EncoderBackend from encode::windows_resolved_backend (config-backed + GPU-vendor cached since stage 2 -> already a single source). Threading encoder/input_format into the encoder+capturer opens (which removes the capture->windows_resolved_backend() back-reference recomputed in dxgi.rs) is stage 5. Behavior-preserving by construction: each resolved decision is provably equivalent to the pre-stage-3 reads (same config() + the same cached running_as_system()/GPU-vendor probes), so old==new. SessionPlan is platform-neutral so it threads the shared virtual_stream/build_pipeline signatures; on Linux it resolves to the single portal/single-process path. Also fixes a pre-existing mod-ordering fmt drift in main.rs (mod config; / mod capture;). Verified: Linux cargo check + clippy (-D warnings) + fmt clean on the touched files. Box build (Windows compile) + on-glass (NVENC + IDD-push + mode switch) pending on the RTX box. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 17:47:48 +00:00
enricobuehlerandClaude Opus 4.8	f58c1438d7	feat(windows-host): finish HostConfig migration — resolve operator/dispatch knobs once (Goal-1 stage 2) ... Migrate 31 genuinely-constant operator/dispatch env::var sites onto HostConfig, so the capture/topology/encoder decision reads ONE owner instead of being recomputed at each call site (the latent bug where capture and encode could disagree on the resolved backend, plan §2.4): idd_push x7, no_wgc, capture_backend, render_adapter, encoder_pref (Linux open_video + linux_zero_copy_is_vaapi), the Windows vdisplay-backend select, plus the plan-named secure_dda/idd_depth/zerocopy/ten_bit and the multi-site perf x4 / compositor x5 / video_source x3 / gamepad. Each HostConfig field's parser is byte-identical to the read it replaced, so old==new by construction (the plan's "a flipped bool is a silent regression" guard). Scope correction — the plan's "~64 sites / Linux XDG+compositor included / grep env::var -> 0" was unsafe as written. Two classes are deliberately KEPT as live reads and documented in config.rs: * Runtime-mutated session vars. vdisplay::apply_session_env REWRITES the process env on every connect (the Bazzite Gaming<->Desktop follow): WAYLAND_DISPLAY, XDG_CURRENT_DESKTOP, XDG_RUNTIME_DIR, DBUS_SESSION_BUS_ADDRESS, and the derived PUNKTFUNK_INPUT_BACKEND, GAMESCOPE_SESSION/NODE, KWIN/MUTTER_VIRTUAL_PRIMARY, FORCE_SHM. Parsing these once would freeze them at startup and silently break session-following — they are NOT constant. * Single-use local tuning with no resolve-once benefit (and FEC_PCT even has two different semantics): FEC_PCT, VIDEO_DROP, VBV_FRAMES, SPLIT_ENCODE, PACE_BURST_KB, the dxgi timing knobs, the *_LIVE/test gates, plus path/dynamic reads (config-dir, PATH search, env-forward-to-child). PUNKTFUNK_ZEROCOPY is split on purpose: Windows presence-semantics moved to the field; Linux keeps its own truthy (1|true|yes|on) parser. Verified: Linux cargo check + clippy (-D warnings) + fmt clean on the touched files. The Windows-only edits are 1:1 substitutions; they get a real Windows compile on the box with Stage 3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 17:24:00 +00:00
enricobuehlerandClaude Opus 4.8	a95842aeb7	feat(windows-host): IDD-push attach fallback to DDA, not the 20s black bail (audit §5.1) ... open() now hands the keepalive BACK on failure (the WGC attach_keepalive pattern) so the caller can fall back instead of tearing the virtual display down. Added a bounded wait_for_attach() that polls the driver's DRV_STATUS_OPENED — it checks ATTACH status, not frame arrival, so it never false-fails on an idle desktop that has composed no frame yet. An attach failure (e.g. a hybrid-GPU render mismatch -> DRV_STATUS_TEX_FAIL, or the driver never opening the ring within 4s) now fails open() -> capture.rs falls back to DDA, instead of next_frame's 20s deadline leaving the session black. Pairs with the driver SET_RENDER_ADAPTER fix (cdccec6). Verified: host clippy (nvenc) clean on the RTX box. Behavioral validation (fallback trigger + happy-path attach timing) needs an on-glass session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 13:09:28 +00:00
enricobuehlerandClaude Opus 4.8	c7ef0e411a	feat(windows): pf-vdisplay IDD-push — HDR + pipelined zero-copy capture ... HDR (display-driven, matching the WGC path): - CTA-861.3 HDR EDID (BT.2020 primaries + HDR Static Metadata block) so Windows offers "Use HDR" on the virtual display. The host FOLLOWS the display's live advanced-color state, recreating the shared ring at the matching format (FP16 in HDR / BGRA in SDR) on a toggle — no freeze. - Always emit Main10/BT.2020-PQ Rgb10a2 while the display is HDR; the client auto-detects PQ from the HEVC VUI (clients under-report VIDEO_CAP_10BIT). Generic HDR10 mastering SEI on every IDR. - Generation-tagged `latest` (gen<<40|seq<<8|slot) + driver `is_stale` re-attach kill the toggle-time garbage frame and any stale-ring read. Perf: - Pipeline the encode loop (Capturer::pipeline_depth; IDD-push = 2): submit N+1 before polling N so the convert/copy on the 3D engine overlaps the NVENC encode of N on the ASIC. PUNKTFUNK_IDD_DEPTH overrides (1 = synchronous). - Rotating host output ring (OUT_RING) so the in-flight encode and the next convert never touch the same texture. - HDR converts directly from the keyed-mutex slot's SRV into the output ring (drops the redundant slot->fp16 scratch copy); SDR copies the BGRA slot in. The slot mutex is held only across the convert/copy, not the encode. RING_LEN 3->6 for publish headroom. - Capture-health diagnostic: new_fps vs repeat_fps under PUNKTFUNK_PERF (a low new_fps at a high send rate means the source isn't compositing, not an encode stall). Validated live on the RTX box: 5120x1440@240 HDR streams; driver composes ~180 new fps, encode 240 fps @ ~4.3 ms p50. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-24 00:39:28 +02:00
enricobuehlerandClaude Opus 4.8	ffae2a31e4	feat: HDR Step-0 colour-metadata transport + security-audit hardening ... Two strands, entangled in punktfunk1.rs, committed together (one builds-green tree). HDR pipeline Step 0 — glass-to-glass colour-metadata transport (docs/hdr-pipeline-plan.md): - Protocol/ABI: ColorInfo on the Welcome + a 0xCE HdrMeta datagram carry the source colour space + HDR10 static mastering metadata (quic.rs, abi.rs connect_ex5 fixing caps=0). - New platform-independent, unit-tested HDR static-metadata helpers (hdr.rs): chromaticities (1/50000), mastering luminance (0.0001 cd/m2), MaxCLL/MaxFALL in HDR10/ST.2086 units. - Capture/encode hooks (capture.rs, encode.rs set_hdr_meta) + Linux client / probe plumbing. Security-audit hardening — top 3 from docs/security-review.md, each adversarially verified: - #1 [HIGH] Secret file permissions. The host key.pem/cert.pem and both trust stores are now written owner-only: 0600 + dir 0700 on Unix (mirrors mgmt_token), best-effort SYSTEM/Administrators/OWNER-only icacls DACL on Windows (%ProgramData% is Users-readable). Closes a local key-disclosure -> host-impersonation gap. New gamestream::{create_private_dir, write_secret_file} + a 0600 regression test. - #2 [HIGH] Native SPAKE2 PIN is single-use. The PIN is consumed the moment the host sends its key-confirmation (which lets the client test its one guess), before reading the proof, so any completed attempt -- right OR wrong -- disarms the window. A wrong PIN isn't observable host-side (the client aborts before sending its proof), so consuming on first attempt is what delivers the documented "one online guess" instead of an unbounded brute-force of the static 4-digit PIN. Test verifies single-use. - #3 [MEDIUM] RTSP packetSize is bounded ([64,2048] in stream_config) and VideoPacketizer::new uses saturating .max(1), killing a PRE-AUTH div-by-zero/underflow panic of the video thread. Tests for {0,15,16,17} + out-of-range rejection. fmt + clippy -D warnings clean; full workspace test suite green (93 host tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-21 09:07:59 +00:00
enricobuehlerandClaude Opus 4.8	07c9244eaf	feat(host/encode): VAAPI zero-copy dmabuf import (AMD/Intel GPU CSC) ... Phase 2 of AMD/Intel support: the VAAPI encoder now takes the capture dmabuf directly and does the RGB->NV12 colour conversion on the GPU's video engine, eliminating the host-side de-pad + swscale CSC + upload the CPU path pays. - capture: a vendor-neutral FramePayload::Dmabuf (dup'd fd + fourcc/modifier/ layout). When zero-copy is on, the EGL->CUDA importer is unavailable (any non-NVIDIA host), and the backend is VAAPI, the capturer advertises LINEAR dmabuf and hands the raw buffer to the encoder instead of CPU-copying it. - encode/vaapi: the encoder self-configures from the first frame's payload (no open_video signature change). The dmabuf arm wraps the buffer as an AV_PIX_FMT_DRM_PRIME frame and pushes it through a filter graph buffer(drm_prime) -> hwmap(vaapi) -> scale_vaapi=nv12 -> buffersink; the encoder takes NV12 surfaces straight from the sink. The Phase 1 CPU-upload path is kept as the other arm (used when capture produces CPU frames). Live-validated on a Radeon 780M (real Sway/xdpw desktop capture): correct, pixel-perfect HEVC, and ~10x less host CPU at 1440p (4.2s -> 0.4s of CPU for 300 frames) -- the de-pad/CSC/upload moves to the GPU. NVIDIA unchanged (zero-copy still imports to CUDA; the passthrough path only engages on non-NVIDIA hosts). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-20 09:57:00 +00:00
enricobuehlerandClaude Opus 4.8	2f2147b093	refactor: drop milestone names + consolidate clients; loss-recovery & rumble fixes ... Two bodies of work in one commit (the rename moved files the fixes also touched). Naming/structure cleanup (pre-launch): - Host modules m3.rs->punktfunk1.rs, m0.rs->spike.rs; CLI m3-host->punktfunk1-host, m0->spike; bare `punktfunk-host` now prints help. Types M3Options/M3Source-> Punktfunk1Options/Punktfunk1Source. - Clients consolidated out of crates/ into clients/: punktfunk-client-rs-> clients/probe (crate punktfunk-probe), client-linux->clients/linux, client-windows->clients/windows, punktfunk-android->clients/android/native (crate punktfunk-client-android; kept [lib] name=punktfunk_android so the JNI contract is unchanged). crates/ now holds only core + host. - Milestone codes M0-M4 purged from code/CLI/CLAUDE.md/README/docs/docs-site, kept only in docs/implementation-plan.md. docs/m2-plan.md-> docs/gamestream-host-plan.md. CI/gradle/flatpak paths updated. Client loss-recovery (video froze and never recovered after a brief drop): - Export punktfunk_connection_frames_dropped through the C ABI (the core already tracked it for the client keyframe-recovery loop; it was never reachable from the ABI clients). Regenerated punktfunk_core.h. - Apple (StreamPump + Stage2Pipeline) and Android (decode.rs) now poll frames_dropped and request a keyframe when it climbs -- the same loss-driven recovery Linux/Windows already had. Under infinite GOP the decoder silently conceals reference-missing frames, so the decode-error trigger rarely fires. Apple rumble robustness (worked then went spotty -- DualSense + Xbox): - Add CHHapticEngine stopped/reset handlers (rebuild on app background / audio interruption / server reset) and drop the permanent `broken` latch on a transient drive failure; latch only when the controller truly has no haptics. - Surface swallowed SDL set_rumble errors on Linux/Windows + diagnostic logging. Verified: cargo build/clippy/fmt --workspace, C-ABI harness, header drift. Not runnable on this box (verify in CI): Gitea workflows, gradle/Android, flatpak, Swift/decky. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-18 21:05:58 +00:00
enricobuehlerandClaude Opus 4.8	11c238e56f	perf(host/windows): move capture→encode off the 3D engine (NV12/P010 video-processor path, zero-copy, GPU priority) ... The Windows host capped at ~60 fps with 35-40 ms latency on a GPU-heavy game: the per-frame capture→encode path shared the 3D engine with the game and got scheduled behind it. Rework to minimize 3D-engine work per frame: - VideoConverter (D3D11 video processor): capture → NVENC-native NV12/P010 so NVENC skips its internal RGB→YUV (a 3D/compute step). Wired into both DDA (dxgi.rs) and WGC (wgc.rs). New PixelFormat::Nv12/P010 + NVENC YUV input. - GPU scheduling hardening (Apollo-style): D3DKMTSetProcessSchedulingPriorityClass HIGH, absolute SetGPUThreadPriority, SetMaximumFrameLatency(1). - WGC SDR zero-copy (hold pool frames; no CopyResource). DDA keeps a fast CopyResource to decouple its single-frame acquire/release from the async convert. - Pipelined helper encode loop (PUNKTFUNK_ENCODE_DEPTH, default 1) + perf split (cap_wait / encode / write). Live on the RTX 4090: hard 60 fps ceiling removed (now scene-scaling 40-200+), latency much reduced. Residual cap in GPU-pinned scenes is the irreducible RGB→YUV convert (no fixed-function unit on NVIDIA — VideoProcessing engine reads 0%) waiting behind an uncapped game under WDDM context time-slicing; Linux avoids it via gamescope capping the game to the display refresh. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-17 13:08:03 +00:00
enricobuehlerandClaude Opus 4.8	b0c518c21b	feat(host/windows): capture the secure desktop in HDR via DDA (no SDR drop) ... The secure-desktop DDA leg went black with HDR on: legacy DuplicateOutput (the SDR-era API) can't capture an FP16/HDR desktop, and dropping the SudoVDA out of HDR is denied on the Winlogon desktop (so the SDR-drop attempt just churned and stayed black). Instead capture HDR natively on the DDA path — the capturer already has the full FP16→BT.2020 PQ→R10G10B10A2 conversion (hdr_fp16 path), it just never requested FP16. Thread a want_hdr flag into duplicate_output: for an HDR session request DuplicateOutput1 with FP16 first and retry it (5×) instead of bailing to the HDR-incapable legacy fallback. The secure-desktop mux now reads the monitor's real HDR state and opens DDA in HDR when set — no advanced-color toggling at all. The normal-desktop DDA overlay/flip issues that pushed us to WGC don't apply to the composed Winlogon UI. want_hdr is threaded through every (re)duplication incl. ACCESS_LOST recovery. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 22:11:07 +00:00
enricobuehlerandClaude Opus 4.8	e51873f785	feat(host/windows): PUNKTFUNK_NO_WGC — force single-process DDA everywhere ... A single test flag to bring up / validate DDA on its own and as the base for the secure-desktop work. When set it (1) skips WGC in capture_virtual_output (forces dxgi::DuplCapturer, same as PUNKTFUNK_CAPTURE=dda) and (2) makes should_use_helper return false, so even a SYSTEM host bypasses the two-process WGC relay and captures in-process with one DDA capturer for both the normal AND the secure desktop — Apollo's model. All the WGC / relay code stays compiled; unset the flag to restore. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 11:15:47 +00:00
enricobuehlerandClaude Opus 4.8	26c43c39a3	feat(host/windows): force-composed-flip overlay to capture the secure desktop ... The secure (Winlogon: UAC/lock/login) desktop presents via fullscreen independent-flip/MPO — it scans out bypassing DWM composition, so DXGI Desktop Duplication returns born-lost DXGI_ERROR_ACCESS_LOST (the client sees black; the UAC only "flashes" during the brief composed transition). Confirmed live: stable 4090 LUID across the storm (NOT reparenting) on an FP16 HDR output, recovering only when the screen changes. Fix (non-input, no system-wide registry change): capture/composed_flip.rs keeps a tiny click-through near-invisible TOPMOST LAYERED window alive on the current input desktop. Any visible window on the output disqualifies independent-flip → DWM composites → DDA can capture. A dedicated thread follows the input desktop (Default↔Winlogon) and recreates the window there on each switch (a window is bound to its desktop), re-asserting topmost + pumping messages every 200ms. Started for the two-process stream's lifetime; gated by PUNKTFUNK_FORCE_COMPOSED (default on, =0 to disable). Needs GENERIC_ALL on OpenInputDesktop for DESKTOP_CREATEWINDOW (0x80070005 otherwise). Validated: overlay creates on the Default desktop; live lock test pending. Also includes SET_RENDER_ADAPTER (sudovda.rs, Apollo item #16): pins the IDD render GPU to the NVENC GPU before ADD — issued + accepted live, though the secure-desktop storm was proven to be independent-flip (stable LUID), not reparenting, so it's correctness/hygiene here rather than this bug's fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 10:25:55 +00:00
enricobuehlerandClaude Opus 4.8	89d363159f	feat(host/windows): two-process secure-desktop step 4 — spawn helper + relay AUs ... The SYSTEM host now sources the normal-desktop video from a user-session WGC helper instead of capturing in-process (WGC won't activate as SYSTEM). New `capture/wgc_relay.rs`: `HelperRelay::spawn` launches `m3-host wgc-helper` in the interactive user session via CreateProcessAsUserW (WTSQueryUserToken → DuplicateTokenEx(TokenPrimary) → lpDesktop="winsta0\\default", CREATE_NO_WINDOW) with three anonymous pipes — stdout (framed Annex-B AUs → parsed back to RelayAu), stdin (control: force-keyframe), stderr (helper logs → host tracing). The host holds the SudoVDA keepalive (sole isolation/topology owner); the helper captures by GDI name only. m3.rs: `virtual_stream` dispatches to the new `virtual_stream_relay` when `should_use_helper()` (running as SYSTEM, or PUNKTFUNK_FORCE_HELPER; disable with PUNKTFUNK_NO_HELPER). The relay loop feeds the existing send thread — same FEC/seal/paced-send path. Reconfigure rebuilds the output + re-spawns the helper; keyframe requests forward over the control pipe; helper pts_ns (same-machine monotonic clock) is used directly as capture_ns. Disconnect ends the stream (step 6 adds the relaunch watchdog). wgc_helper.rs: reads the stdin control byte to request an IDR; --bit-depth flag threaded through so SDR 10-bit (Main10) negotiation reaches the helper's encoder. cfg-gated windows-only; Linux/macOS build unaffected. Step 5 (DesktopWatcher mux to host DDA on the Winlogon secure desktop) is next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 07:43:20 +00:00
enricobuehlerandClaude Opus 4.8	d4a7b43f21	feat(host/windows): DesktopWatcher (secure-desktop detection) — step 1 of the two-process build ... Polls the input-desktop name (OpenInputDesktop + GetUserObjectInformationW(UOI_NAME)) on its own thread → Default/Winlogon atomic; the authoritative normal-vs-secure signal for the capture mux + input path (WTS notifications miss UAC). Not yet wired into the mux (needs the SYSTEM host + WGC helper, steps 3-5 in docs/windows-secure-desktop.md). NOTE: detecting the secure desktop requires the host to run as SYSTEM (a user-token process can't OpenInputDesktop the Winlogon desktop). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 07:13:33 +00:00
enricobuehlerandClaude Opus 4.8	4af59fe462	feat(host/windows): WGC capture backend (overlay/HDR-correct) with watchdog'd DDA fallback ... The capture-architecture reset from the research: add a Windows.Graphics.Capture (WGC) backend that captures the COMPOSED desktop — including the overlay/independent-flip/MPO planes DXGI Desktop Duplication misses — which structurally fixes the frozen HDR animations + video (proven live: a WGC frame decodes to the real 5120x1440 HDR content DDA freezes on). It reuses the whole pipeline unchanged: the WGC frame's GPU texture → same scRGB→BT.2020-PQ shader → NVENC zero-copy; the OS composites the cursor (IsCursorCaptureEnabled) so no manual cursor pass. crates/punktfunk-host/src/ capture/wgc.rs; find_output/make_device/HdrConverter/nudge_cursor_onto made pub(crate) for reuse. Reliability findings + mitigations (live on the RTX 4090): - WGC can't activate under the SYSTEM account (0x80070424) — it needs the interactive user token. The host must run as the user for WGC (run.cmd: drop PsExec -s). DDA still needs SYSTEM for the secure desktop — that token reconciliation (impersonation) is the remaining task. - WGC's Direct3D11CaptureFramePool::CreateFreeThreaded intermittently HANGS on the headless SudoVDA (IddCx) display, correlated with accumulated SudoVDA churn (failed REMOVEs leaving lingering displays); clean-state opens reliably. Since it's a blocking hang, capture_virtual_output runs WGC open on a watchdog thread with a 5s timeout and falls back to DDA on hang/error — the session is NEVER left black: WGC when it opens (fixed animations), DDA otherwise. First-frame nudge added (WGC fires FrameArrived on change; a static desktop otherwise never delivers the first frame). - Default WGC; PUNKTFUNK_CAPTURE=dda forces DDA. DDA path unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-16 06:32:54 +00:00
enricobuehlerandClaude Opus 4.8	16666aa938	feat(hdr): Windows HDR10 + 10-bit end-to-end, negotiated; non-blocking capture recovery ... Adds true HDR (BT.2020 PQ) and 10-bit (HEVC Main10) streaming, negotiated so an 8-bit/SDR client is never sent a stream it can't decode, plus a robust fix for the capture losing the stream across a secure-desktop transition. Protocol (punktfunk-core/quic.rs): - Hello gains `video_caps` (VIDEO_CAP_10BIT / VIDEO_CAP_HDR), Welcome gains `bit_depth`, both as optional trailing bytes (back-compat). client-rs advertises 10-bit via PUNKTFUNK_CLIENT_10BIT; the connector advertises 0 for now (in-band detection drives the native clients). Regenerated punktfunk_core.h. Windows host: - 10-bit Main10: host enables it only when the client advertised VIDEO_CAP_10BIT AND PUNKTFUNK_10BIT is set; threaded through open_video → NVENC (profile Main10, pixelBitDepthMinus8). - HDR: when the captured desktop is scRGB FP16 (R16G16B16A16_FLOAT, HDR on), copy it to an FP16 surface, composite the cursor there, convert scRGB → BT.2020 PQ 10-bit (R10G10B10A2) via a shader, and encode HEVC Main10 with the BT.2020/PQ colour VUI (ABGR10 input). Fixes the freeze + cursor-trail that came from feeding FP16 into the BGRA path. Reacts dynamically to the HDR toggle. - Capture recovery: rebuild is now a single NON-BLOCKING attempt, throttled to ~4×/s, repeating the last good frame between attempts (format-tagged last_present). During a secure-desktop dwell SudoVDA's output is gone; the old blocking 12 s retry starved the send loop for seconds so the client timed out and disconnected — now the session stays fed (frozen) until the desktop returns. Also seeds a black frame on recovery. Apple client (PunktfunkKit): - Detects HDR in-band from the stream VUI (PQ transfer function), decodes to 10-bit P010, and presents via an rgba16Float + BT.2020 PQ CAMetalLayer with EDR; SDR path unchanged. Switches automatically on a mid-session HDR toggle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 20:28:52 +00:00
enricobuehlerandClaude Opus 4.8	50dec9e7cb	style(host/windows): rustfmt the Windows backends ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:50:16 +00:00
enricobuehlerandClaude Opus 4.8	249da9beb3	feat(host/windows): DXGI Desktop Duplication capture backend ... Windows Capturer via DXGI Desktop Duplication: create a D3D11 device on the SudoVDA adapter (by LUID), find the matching output (by GDI name), DuplicateOutput, and per AcquireNextFrame copy the desktop into a CPU-readable staging texture -> tightly-packed BGRA (FramePayload::Cpu, feeds the openh264 software encoder GPU-lessly). Handles WAIT_TIMEOUT (reuse last frame) and ACCESS_LOST (re-duplicate). Adds FramePayload::D3d11(D3d11Frame) for the future NVENC zero-copy path, and a VirtualOutput.win_capture identity (adapter LUID + GDI name) carried out of the SudoVDA backend. Pure helpers (pack_luid/gdi_name_matches/depad_bgra) unit-tested on the VM; the live duplication path needs a real GPU + an activated SudoVDA monitor. Compiles clean on Windows + Linux. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 01:06:21 +00:00
enricobuehlerandClaude Opus 4.8	3eaf218a47	feat(net/mac): default-on recvmsg_x batched Mac recv + GSO host + longer probe ... The Mac/iOS client's wall around ~380 Mbps on a 2.5 G path is the receive drain, not the transport: a loopback speed-test pushes 380/600/1000 Mbps at 0.0% loss, but Darwin has no recvmmsg(2), so the macOS client was doing one recv() syscall per packet — ~40-90k syscalls/s on one core. When the recv loop can't drain fast enough the kernel socket buffer backs up and drops, which the client sees as a sustained stream stalling/freezing in the 300-400 Mbps range (and an immediate "session ended" when a 500 Mbps+ first keyframe bursts in). - core/transport: flip recvmsg_x (the batched Darwin recv, ~30x fewer syscalls) from opt-in to default ON, opt-out via PUNKTFUNK_RECVMSG_X=0. Keeps the auto-fallback to the scalar loop on any unexpected syscall error. The Apple CI swift-test loopback now exercises this path by default. - packaging/kde host.env: enable PUNKTFUNK_GSO=1 — UDP segmentation offload on the host send path (one sendmsg per ~64 packets), the dominant lever above ~1 Gbps. Already wired (send_sealed -> send_gso) with sendmmsg auto-fallback. - apple SpeedTestSheet: lengthen the bandwidth probe 2 s -> 5 s so the measured number stops swinging wildly (50 vs 900 Mbps on the same link) — long enough for steady-state send + recv drain to settle. Matches host MAX_PROBE_MS. - host capture: PUNKTFUNK_SYNTH_NOISE synthetic high-entropy source for reproducible throughput testing of the encode->FEC->send->recv path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 00:35:26 +00:00
enricobuehlerandClaude Opus 4.8	8cceb93541	rename: lumen → punktfunk, everywhere ... Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 13:11:59 +00:00