5 Commits

Author	SHA1	Message	Date
enricobuehler	a4c84ac620	feat(clients/windows): all-vendor video pipeline rewrite + app icon + hosts-page tiles ... Decode+present rewrite (first real pixels on glass for this client): - Decode: FFmpeg D3D11VA on NVIDIA/AMD/Intel. get_format now only returns AV_PIX_FMT_D3D11 and lets libavcodec build the decode pool from hw_device_ctx (hand-built frames contexts failed three different ways: NVIDIA rejects DECODER|SHADER_RESOURCE arrays, BindFlags=0 fails texture creation, Intel rejects non-128-aligned HEVC surfaces at the first SubmitDecoderBuffers). A DXVA profile probe before the hwdevice commits hardware-vs-software up front instead of burning the opening IDR; extra_hw_frames covers the frames the client holds. - Present: the decoded slice is copied with ONE display-size-boxed CopySubresourceRegion (a planar slice is a single subresource in D3D11; the old two-copy D3D12-style code silently no-opped - the black screen) into a sampleable NV12/P010 texture, per-plane SRVs + YUV->RGB shaders. - New dedicated render thread (render.rs): presenting is decoupled from the XAML thread; frame-latency-waitable swapchain + SetMaximumFrameLatency(1), newest-wins drain after the wait, crossbeam frame channel with pts for a capture->presented p50 log. - HiDPI: pixel-sized buffers + SetMatrixTransform(96/dpi) - was blurry at 125/150 % scaling. - Software fallback now feeds the same shaders (swscale -> NV12/P010 planes -> two dynamic plane textures); ps_rgba/X2BGR10 path deleted, hw/sw colour math identical. - Adapter selection for hybrid boxes: PUNKTFUNK_ADAPTER > the window's monitor's adapter > default; PUNKTFUNK_D3D_DEBUG=1 debug layer. - Session pump: request_keyframe at start and on hw->sw demotion (infinite GOP would otherwise sit on a black screen). Validated live on the Arc Pro + RTX 3500 Ada laptop against the local Windows host: 60 fps D3D11VA on both vendors, software path, GUI on glass. Also: embedded app icon (build.rs winresource + WM_SETICON, MSIX Square44x44 targetsize assets, pack-msix stages them) and the hosts-page tile rework (tap-to-connect tiles with sibling overflow menu - fixes forget-also-connects - in-tile rename editor, add-host modal via root state). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-02 16:24:23 +02:00
enricobuehler	3678c182d5	feat(clients): codec preference on Windows/Apple/Android clients (Phase 2b) ... Rounds out codec negotiation across the last three clients — each advertises what it can decode, builds its decoder from the resolved Welcome.codec, and exposes a "Video codec" preference picker. **Windows** (Rust, mirrors Linux): `decodable_codecs()` + `ffmpeg_codec_id()`; the D3D11VA and software FFmpeg decoders (and the mid-session D3D11VA→software demotion) open the negotiated codec instead of hardcoding HEVC; settings gain a `codec` field + reactor ComboBox; `--codec` CLI flag. **Apple** (Swift/C-ABI): AnnexB is now codec-aware — a `VideoCodec` enum drives H.264 vs HEVC NAL parsing / parameter-set extraction (`CMVideoFormatDescriptionCreateFromH264ParameterSets` for H.264, no VPS) and AVCC repacking; `PunktfunkConnection` advertises H264|HEVC via `punktfunk_connect_ex7`, reads `resolvedCodec` (`punktfunk_connection_codec`), and threads `videoCodec` into the stage-1/2 pipelines + `VideoDecoder`; SettingsView "Video codec" Picker (auto/HEVC/H.264). AV1 is left out (hosts don't emit it on the native path, and it's not an AnnexB codec). Test call sites updated. **Android** (Kotlin + Rust JNI): the JNI `nativeConnect` gains `preferredCodec`; the native decode loop picks the AMediaCodec MIME (`video/hevc`|`video/avc`) from `connector.codec` and advertises H264|HEVC; Settings `codec` field + Compose dropdown. Core/host/probe/Linux clippy + tests green (unchanged from 2a). Windows/Apple/Android compile on their platform CI (this Linux box can't build them — Windows toolchain / Xcode / the Android NDK's opus-cmake toolchain). All follow the Linux client's validated pattern. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-02 00:29:38 +00:00
enricobuehler	a58b6b8e76	fix(windows-client): clear clippy -D warnings on MSVC ... The cfg(windows) code can't be lint-checked on the Linux dev box, so three -D warnings slipped through (caught by windows.yml; the FFI + shaders compiled fine): - gpu.rs: SetMultithreadProtected returns a must-use BOOL -> `let _ =`. - video.rs: drop the unused GpuFrame::ten_bit field (present keys off `hdr`; the value is still computed locally for the first-frame log). - present.rs: GpuView::frame is an RAII keep-alive (its Drop returns the decoder surface to the pool), never read -> #[allow(dead_code)]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-18 23:21:18 +00:00
enricobuehler	0cc36fa130	feat(windows-client): D3D11VA zero-copy hw decode + HDR10 present + GUI polish ... The client was pure software HEVC decode + CPU swscale->RGBA + a full-frame dynamic-texture upload every frame -- the reason performance was poor on a GPU box (the GPU sat idle while the CPU churned). This adds a hardware path, HDR, and a GUI pass. Performance -- D3D11VA zero-copy: - gpu.rs (new): one D3D11 device (hardware + VIDEO_SUPPORT, WARP fallback, multithread-protected) shared by decoder and presenter via a Send/Sync OnceLock. Sharing is mandatory -- a decoded texture is only bindable on the device that created it. windows-rs COM interfaces are !Send/!Sync, so the unsafe impl is sound only under the multithread protection + disjoint decode(video ctx)/present(immediate ctx) split. - video.rs: D3d11vaDecoder (raw FFI mirroring the Linux VAAPI module). The COM-typed AVD3D11VA{Device,Frames}Context are declared here (stable FFmpeg ABI) to avoid ffmpeg-sys binding the d3d11 headers; get_format builds a frames ctx with BindFlags=SHADER_RESOURCE so the NV12/P010 array slices are sampleable. av_frame_clone guard keeps each surface out of the reuse pool until the presenter drops it. Software decode stays as the fallback (DecoderPref Auto/Hardware/Software; auto falls back on init/decode error). - present.rs: shared device; per-plane SRVs over the array slice (NV12->R8/R8G8, P010->R16/R16G16) + three pixel shaders (RGBA passthrough, NV12/BT.709, P010/BT.2020-PQ). present() now takes the frame by value so the GPU surface survives re-presents. HDR: - Detected in-band (transfer == SMPTE2084), same signal as the other clients. Swapchain flips to R10G10B10A2 + ST.2084 + HDR10 metadata. New Settings toggle gates advertising VIDEO_CAP_10BIT|HDR; host still gates 10-bit behind its own PUNKTFUNK_10BIT + actual-HDR-content checks. GUI (windows-reactor): - Host cards with accent-monogram avatars + colored status pills, InfoBar for errors/pairing hints, ToggleSwitch settings (+ HDR, decoder, bitrate), button icons, a richer connecting screen, and a stream HUD with GPU/CPU-decode + HDR status chips. Not yet on-glass validated: the Linux dev box can't compile the cfg(windows) code (ffmpeg/windows crates unfetched; WARP has no hw decode) -- only cargo fmt checks it here. API shapes verified against the windows-rs/reactor source and the YUV->RGB coefficients checked by hand, but D3D11VA + shaders + the GUI need a real build (Windows CI / build VM) and on-glass test on the RTX box. The host-side HDR encode path is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-18 23:16:07 +00:00
enricobuehler	9c8fa9340c	refactor: drop milestone names + consolidate clients; loss-recovery & rumble fixes ... Two bodies of work in one commit (the rename moved files the fixes also touched). Naming/structure cleanup (pre-launch): - Host modules m3.rs->punktfunk1.rs, m0.rs->spike.rs; CLI m3-host->punktfunk1-host, m0->spike; bare `punktfunk-host` now prints help. Types M3Options/M3Source-> Punktfunk1Options/Punktfunk1Source. - Clients consolidated out of crates/ into clients/: punktfunk-client-rs-> clients/probe (crate punktfunk-probe), client-linux->clients/linux, client-windows->clients/windows, punktfunk-android->clients/android/native (crate punktfunk-client-android; kept [lib] name=punktfunk_android so the JNI contract is unchanged). crates/ now holds only core + host. - Milestone codes M0-M4 purged from code/CLI/CLAUDE.md/README/docs/docs-site, kept only in docs/implementation-plan.md. docs/m2-plan.md-> docs/gamestream-host-plan.md. CI/gradle/flatpak paths updated. Client loss-recovery (video froze and never recovered after a brief drop): - Export punktfunk_connection_frames_dropped through the C ABI (the core already tracked it for the client keyframe-recovery loop; it was never reachable from the ABI clients). Regenerated punktfunk_core.h. - Apple (StreamPump + Stage2Pipeline) and Android (decode.rs) now poll frames_dropped and request a keyframe when it climbs -- the same loss-driven recovery Linux/Windows already had. Under infinite GOP the decoder silently conceals reference-missing frames, so the decode-error trigger rarely fires. Apple rumble robustness (worked then went spotty -- DualSense + Xbox): - Add CHHapticEngine stopped/reset handlers (rebuild on app background / audio interruption / server reset) and drop the permanent `broken` latch on a transient drive failure; latch only when the controller truly has no haptics. - Surface swallowed SDL set_rumble errors on Linux/Windows + diagnostic logging. Verified: cargo build/clippy/fmt --workspace, C-ABI harness, header drift. Not runnable on this box (verify in CI): Gitea workflows, gradle/Android, flatpak, Swift/decky. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-18 21:05:58 +00:00