feat(windows-client): WinUI 3 (windows-reactor) UI — host list, settings, pairing, SwapChainPanel present
audit / cargo-audit (push) Failing after 1m5s
apple / swift (push) Successful in 3m37s
ci / rust (push) Failing after 3m46s
android / android (push) Successful in 5m20s
ci / web (push) Successful in 33s
ci / docs-site (push) Successful in 27s
ci / bench (push) Successful in 4m39s
decky / build-publish (push) Successful in 22s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 3m12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
deb / build-publish (push) Successful in 9m20s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m38s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 21s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m10s
flatpak / build-publish (push) Failing after 4m55s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 4m36s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m18s
docker / deploy-docs (push) Successful in 20s

Replaces the winit + raw-HWND-D3D11 shell with a native WinUI 3 UI via windows-reactor (a
declarative React-like framework backed by WinUI). The earlier "Reactor can't host a
swapchain" read was wrong — PR #4499 (merged 2026-06-01) added a SwapChainPanel widget with
`set_swap_chain` over `CreateSwapChainForComposition`. Builds + clippy + fmt green on
x86_64-pc-windows-msvc.

- Cargo: drop winit/raw-window-handle; add windows-reactor + the `windows` crate, both pinned
  to the SAME windows-rs commit (b4129fcc) so the `IDXGISwapChain1` handed to `set_swap_chain`
  satisfies reactor's `windows_core::Interface`. Reactor's build.rs downloads the Windows App
  SDK NuGets + stages the bootstrap DLL/resources.pri — it requires `CARGO_WORKSPACE_DIR` set
  (now in the VM build env); /temp + /winmd gitignored.
- present.rs: composition swapchain (B8G8R8A8 FLIP_SEQUENTIAL premultiplied) bound to the
  SwapChainPanel; WARP fallback, runtime D3DCompile shaders, dynamic RGBA texture, Contain-fit
  letterbox; driven by reactor's per-frame `on_rendering`.
- app.rs: the WinUI 3 shell — host list (live mDNS + saved + manual), settings (resolution/
  refresh/mic combos+toggle), in-app SPAKE2 PIN pairing screen, and the stream page. Trust gate
  mirrors the GTK client (pinned → silent, pair=optional → TOFU, else PIN); a pinned-fp
  mismatch routes to re-pair. The session pump + decoded-frame handoff cross to the UI thread
  via a Mutex side-channel + thread-locals (the SwapChainPanel sample's pattern).
- gamepad: `ctl` sender now `Arc<Mutex<…>>` so GamepadService is `Sync` (shared across the UI
  and session-pump threads). main.rs: windowed = in-app UI; `--headless`/`--discover` keep the
  CLI paths.

Not yet wired: raw stream keyboard/mouse input (next commit — reactor exposes no raw key/
pointer events, so it needs Win32 low-level hooks or Microsoft.UI.Xaml bindings). On-glass
validation pending a display (the dev VM is headless/GPU-less).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-15 22:58:48 +00:00
parent 296b976b8f
commit 4994f7f4ba
9 changed files with 1033 additions and 1783 deletions
+167 -172
View File
@@ -1,17 +1,19 @@
//! Direct3D11 presenter: upload a decoded `CpuFrame` (RGBA) into a dynamic texture and draw
//! it Contain-fit into a flip-model swapchain bound to the window's HWND, then present.
//! Direct3D11 presenter for a WinUI 3 `SwapChainPanel`: upload a decoded `CpuFrame` (RGBA)
//! into a dynamic texture and draw it Contain-fit into a **composition** flip-model swapchain,
//! which the reactor stream page binds to the panel via `SwapChainPanelHandle::set_swap_chain`.
//!
//! The device prefers a hardware adapter and falls back to **WARP** (the GPU-less dev box
//! runs the whole present path in software). The draw is a single full-screen triangle
//! sampling the video texture; a letterbox is produced by clearing the back buffer black and
//! setting the viewport to the Contain-fit rect (no per-frame vertex buffer). This is the
//! SDR 8-bit path; the 10-bit/HDR present (`R10G10B10A2` + `SetColorSpace1(...G2084_P2020)`)
//! is a follow-up alongside the P010 D3D11VA decode.
//! The device prefers a hardware adapter and falls back to **WARP** (the GPU-less dev box runs
//! the whole present path in software). The draw is a single full-screen triangle sampling the
//! video texture; a letterbox is produced by clearing the back buffer black and setting the
//! viewport to the Contain-fit rect (no per-frame vertex buffer). SDR 8-bit path; the
//! 10-bit/HDR present (`R10G10B10A2` + `SetColorSpace1`) is a follow-up alongside P010 decode.
//!
//! All `windows` types here come from the same windows-rs commit as `windows-reactor`, so the
//! `IDXGISwapChain1` handed to `set_swap_chain` satisfies reactor's `windows_core::Interface`.
use crate::video::CpuFrame;
use anyhow::{anyhow, Context, Result};
use windows::core::{Interface, PCSTR};
use windows::Win32::Foundation::{HMODULE, HWND};
use windows::Win32::Graphics::Direct3D::Fxc::{D3DCompile, D3DCOMPILE_OPTIMIZATION_LEVEL3};
use windows::Win32::Graphics::Direct3D::{
ID3DBlob, D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_WARP, D3D_FEATURE_LEVEL_11_0,
@@ -35,63 +37,114 @@ SamplerState smp : register(s0);
float4 ps_main(VSOut i) : SV_Target { return tex.Sample(smp, i.uv); }
"#;
pub struct Renderer {
pub struct Presenter {
device: ID3D11Device,
context: ID3D11DeviceContext,
vs: ID3D11VertexShader,
ps: ID3D11PixelShader,
sampler: ID3D11SamplerState,
/// Video texture + its SRV + dimensions; recreated when the decoded size changes.
swap: IDXGISwapChain1,
rtv: Option<ID3D11RenderTargetView>,
/// Video texture + SRV + dimensions; recreated when the decoded size changes.
tex: Option<(ID3D11Texture2D, ID3D11ShaderResourceView, u32, u32)>,
/// Panel (swapchain) size in pixels, updated on resize.
panel_w: u32,
panel_h: u32,
}
impl Renderer {
pub fn new() -> Result<Renderer> {
impl Presenter {
/// Create the D3D11 device + composition swapchain + shaders, sized to the panel.
pub fn new(width: u32, height: u32) -> Result<Presenter> {
let (device, context) = create_device()?;
let vs_blob = compile(SHADER_HLSL, "vs_main", "vs_5_0")?;
let ps_blob = compile(SHADER_HLSL, "ps_main", "ps_5_0")?;
let (vs, ps) = unsafe {
let mut vs = None;
device
.CreateVertexShader(blob_bytes(&vs_blob), None, Some(&mut vs))
.context("CreateVertexShader")?;
let mut ps = None;
device
.CreatePixelShader(blob_bytes(&ps_blob), None, Some(&mut ps))
.context("CreatePixelShader")?;
(vs.unwrap(), ps.unwrap())
};
let sampler = unsafe {
let desc = D3D11_SAMPLER_DESC {
Filter: D3D11_FILTER_MIN_MAG_MIP_LINEAR,
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
MaxLOD: D3D11_FLOAT32_MAX,
..Default::default()
};
let mut s = None;
device
.CreateSamplerState(&desc, Some(&mut s))
.context("CreateSamplerState")?;
s.unwrap()
};
Ok(Renderer {
let (vs, ps, sampler) = build_pipeline(&device)?;
let swap = create_composition_swapchain(&device, width.max(1), height.max(1))?;
Ok(Presenter {
device,
context,
vs,
ps,
sampler,
swap,
rtv: None,
tex: None,
panel_w: width.max(1),
panel_h: height.max(1),
})
}
pub fn device(&self) -> &ID3D11Device {
&self.device
/// The DXGI swapchain to hand to `SwapChainPanelHandle::set_swap_chain`.
pub fn swap_chain(&self) -> &IDXGISwapChain1 {
&self.swap
}
/// Upload one decoded RGBA frame, recreating the GPU texture if the size changed.
pub fn upload(&mut self, frame: &CpuFrame) -> Result<()> {
/// Resize the back buffers to the panel's new size (drops the stale RTV).
pub fn resize(&mut self, width: u32, height: u32) {
if width == 0 || height == 0 || (width == self.panel_w && height == self.panel_h) {
return;
}
self.rtv = None; // release all back-buffer refs before ResizeBuffers
unsafe {
let _ = self.swap.ResizeBuffers(
0,
width,
height,
DXGI_FORMAT_UNKNOWN,
DXGI_SWAP_CHAIN_FLAG(0),
);
}
self.panel_w = width;
self.panel_h = height;
}
/// Present one decoded frame (Contain-fit) — or, when `frame` is `None`, just re-present the
/// last texture (or black). Called from the reactor `on_rendering` per-frame callback.
pub fn present(&mut self, frame: Option<&CpuFrame>) {
if let Some(f) = frame {
if let Err(e) = self.upload(f) {
tracing::warn!(error = %e, "frame upload failed");
}
}
let Ok(rtv) = self.rtv() else {
return;
};
let (pw, ph) = (self.panel_w, self.panel_h);
unsafe {
let c = &self.context;
c.ClearRenderTargetView(&rtv, &[0.0, 0.0, 0.0, 1.0]);
if let Some((_, srv, vw, vh)) = &self.tex {
// Contain-fit viewport: scale to the smaller axis, centre, letterbox the rest.
let (ww, wh, vfw, vfh) = (
pw as f32,
ph as f32,
(*vw).max(1) as f32,
(*vh).max(1) as f32,
);
let scale = (ww / vfw).min(wh / vfh);
let (dw, dh) = (vfw * scale, vfh * scale);
let (ox, oy) = ((ww - dw) / 2.0, (wh - dh) / 2.0);
c.OMSetRenderTargets(Some(&[Some(rtv.clone())]), None);
let vp = D3D11_VIEWPORT {
TopLeftX: ox,
TopLeftY: oy,
Width: dw,
Height: dh,
MinDepth: 0.0,
MaxDepth: 1.0,
};
c.RSSetViewports(Some(&[vp]));
c.IASetInputLayout(None);
c.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
c.VSSetShader(&self.vs, None);
c.PSSetShader(&self.ps, None);
c.PSSetShaderResources(0, Some(&[Some(srv.clone())]));
c.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
c.Draw(3, 0);
}
let _ = self.swap.Present(1, DXGI_PRESENT(0));
}
}
fn upload(&mut self, frame: &CpuFrame) -> Result<()> {
let (w, h) = (frame.width, frame.height);
let need_new = !matches!(&self.tex, Some((_, _, tw, th)) if *tw == w && *th == h);
if need_new {
@@ -148,122 +201,7 @@ impl Renderer {
Ok(())
}
/// Clear the target black and draw the current video texture Contain-fit into the window.
pub fn draw(
&self,
rtv: &ID3D11RenderTargetView,
win_w: u32,
win_h: u32,
vid_w: u32,
vid_h: u32,
) {
let Some((_, srv, _, _)) = &self.tex else {
return;
};
// Contain-fit: scale to the smaller axis, centre, letterbox the rest.
let (ww, wh, vw, vh) = (
win_w as f32,
win_h as f32,
vid_w.max(1) as f32,
vid_h.max(1) as f32,
);
let scale = (ww / vw).min(wh / vh);
let (dw, dh) = (vw * scale, vh * scale);
let (ox, oy) = ((ww - dw) / 2.0, (wh - dh) / 2.0);
unsafe {
let c = &self.context;
c.ClearRenderTargetView(rtv, &[0.0, 0.0, 0.0, 1.0]);
c.OMSetRenderTargets(Some(&[Some(rtv.clone())]), None);
let vp = D3D11_VIEWPORT {
TopLeftX: ox,
TopLeftY: oy,
Width: dw,
Height: dh,
MinDepth: 0.0,
MaxDepth: 1.0,
};
c.RSSetViewports(Some(&[vp]));
c.IASetInputLayout(None);
c.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
c.VSSetShader(&self.vs, None);
c.PSSetShader(&self.ps, None);
c.PSSetShaderResources(0, Some(&[Some(srv.clone())]));
c.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
c.Draw(3, 0);
}
}
}
/// A flip-model swapchain bound to a window HWND, with a lazily-(re)built render-target view.
pub struct SwapChain {
swap: IDXGISwapChain1,
device: ID3D11Device,
rtv: Option<ID3D11RenderTargetView>,
pub width: u32,
pub height: u32,
}
impl SwapChain {
pub fn new(device: &ID3D11Device, hwnd: HWND, width: u32, height: u32) -> Result<SwapChain> {
let dxdev: IDXGIDevice = device.cast().context("IDXGIDevice cast")?;
let factory: IDXGIFactory2 = unsafe {
let adapter = dxdev.GetAdapter().context("GetAdapter")?;
adapter.GetParent().context("GetParent (IDXGIFactory2)")?
};
let desc = DXGI_SWAP_CHAIN_DESC1 {
Width: width.max(1),
Height: height.max(1),
Format: DXGI_FORMAT_R8G8B8A8_UNORM,
Stereo: false.into(),
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
BufferUsage: DXGI_USAGE_RENDER_TARGET_OUTPUT,
BufferCount: 2,
Scaling: DXGI_SCALING_STRETCH,
SwapEffect: DXGI_SWAP_EFFECT_FLIP_DISCARD,
AlphaMode: DXGI_ALPHA_MODE_IGNORE,
Flags: 0,
};
let swap = unsafe {
factory
.CreateSwapChainForHwnd(device, hwnd, &desc, None, None)
.context("CreateSwapChainForHwnd")?
};
Ok(SwapChain {
swap,
device: device.clone(),
rtv: None,
width: width.max(1),
height: height.max(1),
})
}
/// Resize the back buffers (window resize); drops the stale RTV so it rebuilds lazily.
pub fn resize(&mut self, width: u32, height: u32) -> Result<()> {
if width == 0 || height == 0 || (width == self.width && height == self.height) {
return Ok(());
}
self.rtv = None; // must release all back-buffer references before ResizeBuffers
unsafe {
self.swap
.ResizeBuffers(
0,
width,
height,
DXGI_FORMAT_UNKNOWN,
DXGI_SWAP_CHAIN_FLAG(0),
)
.context("ResizeBuffers")?;
}
self.width = width;
self.height = height;
Ok(())
}
/// The current back-buffer render-target view (built on first use after create/resize).
pub fn rtv(&mut self) -> Result<ID3D11RenderTargetView> {
fn rtv(&mut self) -> Result<ID3D11RenderTargetView> {
if self.rtv.is_none() {
let back: ID3D11Texture2D = unsafe { self.swap.GetBuffer(0).context("GetBuffer")? };
let rtv = unsafe {
@@ -277,13 +215,6 @@ impl SwapChain {
}
Ok(self.rtv.clone().unwrap())
}
/// Present the back buffer (vsync on — a stream is host-paced, tearing-free wins here).
pub fn present(&self) {
unsafe {
let _ = self.swap.Present(1, DXGI_PRESENT(0));
}
}
}
fn create_device() -> Result<(ID3D11Device, ID3D11DeviceContext)> {
@@ -294,7 +225,7 @@ fn create_device() -> Result<(ID3D11Device, ID3D11DeviceContext)> {
D3D11CreateDevice(
None,
driver,
HMODULE::default(),
None,
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
@@ -304,12 +235,12 @@ fn create_device() -> Result<(ID3D11Device, ID3D11DeviceContext)> {
)
};
if r.is_ok() {
let driver_name = if driver == D3D_DRIVER_TYPE_HARDWARE {
let name = if driver == D3D_DRIVER_TYPE_HARDWARE {
"hardware"
} else {
"WARP (software)"
};
tracing::info!(driver = driver_name, "D3D11 device created");
tracing::info!(driver = name, "D3D11 device created");
return Ok((device.unwrap(), context.unwrap()));
}
}
@@ -318,6 +249,70 @@ fn create_device() -> Result<(ID3D11Device, ID3D11DeviceContext)> {
))
}
/// A composition flip-model swapchain (no HWND) for binding to a XAML `SwapChainPanel`.
fn create_composition_swapchain(
device: &ID3D11Device,
width: u32,
height: u32,
) -> Result<IDXGISwapChain1> {
let dxdev: IDXGIDevice = device.cast().context("IDXGIDevice cast")?;
let factory: IDXGIFactory2 = unsafe {
let adapter = dxdev.GetAdapter().context("GetAdapter")?;
adapter.GetParent().context("GetParent (IDXGIFactory2)")?
};
let desc = DXGI_SWAP_CHAIN_DESC1 {
Width: width,
Height: height,
Format: DXGI_FORMAT_B8G8R8A8_UNORM,
Stereo: false.into(),
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
BufferUsage: DXGI_USAGE_RENDER_TARGET_OUTPUT,
BufferCount: 2,
Scaling: DXGI_SCALING_STRETCH,
SwapEffect: DXGI_SWAP_EFFECT_FLIP_SEQUENTIAL,
AlphaMode: DXGI_ALPHA_MODE_PREMULTIPLIED,
Flags: 0,
};
unsafe {
factory
.CreateSwapChainForComposition(device, &desc, None)
.context("CreateSwapChainForComposition")
}
}
fn build_pipeline(
device: &ID3D11Device,
) -> Result<(ID3D11VertexShader, ID3D11PixelShader, ID3D11SamplerState)> {
let vs_blob = compile(SHADER_HLSL, "vs_main", "vs_5_0")?;
let ps_blob = compile(SHADER_HLSL, "ps_main", "ps_5_0")?;
unsafe {
let mut vs = None;
device
.CreateVertexShader(blob_bytes(&vs_blob), None, Some(&mut vs))
.context("CreateVertexShader")?;
let mut ps = None;
device
.CreatePixelShader(blob_bytes(&ps_blob), None, Some(&mut ps))
.context("CreatePixelShader")?;
let sdesc = D3D11_SAMPLER_DESC {
Filter: D3D11_FILTER_MIN_MAG_MIP_LINEAR,
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
MaxLOD: D3D11_FLOAT32_MAX,
..Default::default()
};
let mut sampler = None;
device
.CreateSamplerState(&sdesc, Some(&mut sampler))
.context("CreateSamplerState")?;
Ok((vs.unwrap(), ps.unwrap(), sampler.unwrap()))
}
}
fn compile(src: &str, entry: &str, target: &str) -> Result<ID3DBlob> {
let entry_c = std::ffi::CString::new(entry).unwrap();
let target_c = std::ffi::CString::new(target).unwrap();