feat(windows-client): WinUI 3 (windows-reactor) UI — host list, settings, pairing, SwapChainPanel present
audit / cargo-audit (push) Failing after 1m5s
apple / swift (push) Successful in 3m37s
ci / rust (push) Failing after 3m46s
android / android (push) Successful in 5m20s
ci / web (push) Successful in 33s
ci / docs-site (push) Successful in 27s
ci / bench (push) Successful in 4m39s
decky / build-publish (push) Successful in 22s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 3m12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
deb / build-publish (push) Successful in 9m20s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m38s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 21s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m10s
flatpak / build-publish (push) Failing after 4m55s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 4m36s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m18s
docker / deploy-docs (push) Successful in 20s

Replaces the winit + raw-HWND-D3D11 shell with a native WinUI 3 UI via windows-reactor (a
declarative React-like framework backed by WinUI). The earlier "Reactor can't host a
swapchain" read was wrong — PR #4499 (merged 2026-06-01) added a SwapChainPanel widget with
`set_swap_chain` over `CreateSwapChainForComposition`. Builds + clippy + fmt green on
x86_64-pc-windows-msvc.

- Cargo: drop winit/raw-window-handle; add windows-reactor + the `windows` crate, both pinned
  to the SAME windows-rs commit (b4129fcc) so the `IDXGISwapChain1` handed to `set_swap_chain`
  satisfies reactor's `windows_core::Interface`. Reactor's build.rs downloads the Windows App
  SDK NuGets + stages the bootstrap DLL/resources.pri — it requires `CARGO_WORKSPACE_DIR` set
  (now in the VM build env); /temp + /winmd gitignored.
- present.rs: composition swapchain (B8G8R8A8 FLIP_SEQUENTIAL premultiplied) bound to the
  SwapChainPanel; WARP fallback, runtime D3DCompile shaders, dynamic RGBA texture, Contain-fit
  letterbox; driven by reactor's per-frame `on_rendering`.
- app.rs: the WinUI 3 shell — host list (live mDNS + saved + manual), settings (resolution/
  refresh/mic combos+toggle), in-app SPAKE2 PIN pairing screen, and the stream page. Trust gate
  mirrors the GTK client (pinned → silent, pair=optional → TOFU, else PIN); a pinned-fp
  mismatch routes to re-pair. The session pump + decoded-frame handoff cross to the UI thread
  via a Mutex side-channel + thread-locals (the SwapChainPanel sample's pattern).
- gamepad: `ctl` sender now `Arc<Mutex<…>>` so GamepadService is `Sync` (shared across the UI
  and session-pump threads). main.rs: windowed = in-app UI; `--headless`/`--discover` keep the
  CLI paths.

Not yet wired: raw stream keyboard/mouse input (next commit — reactor exposes no raw key/
pointer events, so it needs Win32 low-level hooks or Microsoft.UI.Xaml bindings). On-glass
validation pending a display (the dev VM is headless/GPU-less).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-15 22:58:48 +00:00
parent 296b976b8f
commit 4994f7f4ba
9 changed files with 1033 additions and 1783 deletions
+10 -15
View File
@@ -1,6 +1,6 @@
[package]
name = "punktfunk-client-windows"
description = "Native Windows punktfunk/1 client — winit/D3D11 shell, FFmpeg decode, WASAPI audio, SDL3 gamepads"
description = "Native Windows punktfunk/1 client — WinUI 3 (windows-reactor) shell, D3D11/SwapChainPanel present, FFmpeg decode, WASAPI audio, SDL3 gamepads"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
@@ -21,28 +21,23 @@ path = "src/main.rs"
# is Sync (mutexed plane receivers), so it drops into a UI app cleanly.
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
# Win32 / Direct3D11 / DXGI surface for the present path + raw input. Software (WARP) device on
# the GPU-less dev box; the same code drives a hardware adapter on a real GPU.
windows = { version = "0.62", features = [
# WinUI 3 UI via windows-reactor (a declarative React-like framework backed by WinUI). Its
# `build.rs` downloads the Windows App SDK NuGets and stages the bootstrap DLL + resources.pri
# next to the exe; it requires `CARGO_WORKSPACE_DIR` to be set in the build env. Unpublished
# (version 0.0.0) and fast-moving, so pinned to a verified commit.
windows-reactor = { git = "https://github.com/microsoft/windows-rs", rev = "b4129fcc1ae81eec8bf1217539883db821bca3a1" }
# Win32 / Direct3D11 / DXGI for the SwapChainPanel composition swapchain. Pulled from the SAME
# windows-rs commit as windows-reactor so their `windows-core` unifies — the `IDXGISwapChain1`
# we hand to `SwapChainPanelHandle::set_swap_chain` must satisfy reactor's `windows_core::Interface`.
windows = { git = "https://github.com/microsoft/windows-rs", rev = "b4129fcc1ae81eec8bf1217539883db821bca3a1", features = [
"Win32_Foundation",
"Win32_System_Com",
"Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common",
"Win32_Graphics_Direct3D",
"Win32_Graphics_Direct3D11",
"Win32_Graphics_Direct3D_Fxc",
"Win32_UI_Input",
"Win32_UI_Input_KeyboardAndMouse",
"Win32_UI_WindowsAndMessaging",
"Win32_UI_HiDpi",
] }
# UI shell: a winit window + a raw DXGI flip-model swapchain on its HWND (the proven present
# path; the WinUI3/Reactor option is a documented follow-up). raw-window-handle extracts the
# HWND for swapchain creation.
winit = "0.30"
raw-window-handle = "0.6"
# Video decode (same FFmpeg pin as the host/Linux client) — software HEVC on the GPU-less dev
# box; D3D11VA hardware decode is a follow-up for the real-GPU box.
ffmpeg-next = "8"
File diff suppressed because it is too large Load Diff
@@ -4,7 +4,7 @@
use mdns_sd::{ServiceDaemon, ServiceEvent};
#[derive(Clone, Debug)]
#[derive(Clone, Debug, PartialEq)]
pub struct DiscoveredHost {
/// Stable row key: the advertised host id, falling back to the mDNS fullname.
pub key: String,
@@ -51,7 +51,9 @@ pub struct GamepadService {
pads: Arc<Mutex<Vec<PadInfo>>>,
active: Arc<Mutex<Option<PadInfo>>>,
pinned: Arc<Mutex<Option<u32>>>,
ctl: Sender<Ctl>,
// `Arc<Mutex<…>>` (not a bare `Sender`, which is `!Sync`) so the service is `Sync` — the
// WinUI app shares it across the UI thread and the session-pump thread (attach/detach).
ctl: Arc<Mutex<Sender<Ctl>>>,
}
impl GamepadService {
@@ -75,7 +77,7 @@ impl GamepadService {
pads,
active,
pinned,
ctl,
ctl: Arc::new(Mutex::new(ctl)),
}
}
@@ -95,15 +97,15 @@ impl GamepadService {
#[allow(dead_code)] // consumed by the settings GUI (follow-up)
pub fn set_pinned(&self, id: Option<u32>) {
let _ = self.ctl.send(Ctl::Pin(id));
let _ = self.ctl.lock().unwrap().send(Ctl::Pin(id));
}
pub fn attach(&self, connector: Arc<NativeClient>) {
let _ = self.ctl.send(Ctl::Attach(connector));
let _ = self.ctl.lock().unwrap().send(Ctl::Attach(connector));
}
pub fn detach(&self) {
let _ = self.ctl.send(Ctl::Detach);
let _ = self.ctl.lock().unwrap().send(Ctl::Detach);
}
/// What "Automatic" resolves to right now — the virtual pad matching the physical one
@@ -1,162 +0,0 @@
//! Local key/button codes → the punktfunk input wire contract.
//!
//! The wire carries Windows Virtual-Key codes (the GameStream convention; the host maps them
//! back with `inject::vk_to_evdev`). On Windows the VK is the *native* source — but winit
//! hands us a layout-independent **physical** `KeyCode` (`KeyA` is always the QWERTY-A
//! position), which is exactly what a game wants: positional keys map positionally regardless
//! of the user's keyboard layout. This table is that physical-position → VK mapping, the
//! direct analogue of the Linux client's evdev table.
use winit::keyboard::KeyCode;
/// Map a winit physical `KeyCode` to the Windows VK code the host expects. `None` = a key the
/// wire contract doesn't cover (media keys etc.) — drop it rather than guess.
pub fn keycode_to_vk(code: KeyCode) -> Option<u8> {
use KeyCode::*;
Some(match code {
// --- Navigation / editing / whitespace ---
Backspace => 0x08,
Tab => 0x09,
Enter => 0x0D,
Pause => 0x13,
CapsLock => 0x14,
Escape => 0x1B,
Space => 0x20,
PageUp => 0x21,
PageDown => 0x22,
End => 0x23,
Home => 0x24,
ArrowLeft => 0x25,
ArrowUp => 0x26,
ArrowRight => 0x27,
ArrowDown => 0x28,
PrintScreen => 0x2C,
Insert => 0x2D,
Delete => 0x2E,
// --- Digit row ---
Digit0 => 0x30,
Digit1 => 0x31,
Digit2 => 0x32,
Digit3 => 0x33,
Digit4 => 0x34,
Digit5 => 0x35,
Digit6 => 0x36,
Digit7 => 0x37,
Digit8 => 0x38,
Digit9 => 0x39,
// --- Letters ---
KeyA => 0x41,
KeyB => 0x42,
KeyC => 0x43,
KeyD => 0x44,
KeyE => 0x45,
KeyF => 0x46,
KeyG => 0x47,
KeyH => 0x48,
KeyI => 0x49,
KeyJ => 0x4A,
KeyK => 0x4B,
KeyL => 0x4C,
KeyM => 0x4D,
KeyN => 0x4E,
KeyO => 0x4F,
KeyP => 0x50,
KeyQ => 0x51,
KeyR => 0x52,
KeyS => 0x53,
KeyT => 0x54,
KeyU => 0x55,
KeyV => 0x56,
KeyW => 0x57,
KeyX => 0x58,
KeyY => 0x59,
KeyZ => 0x5A,
// --- Meta / context-menu ---
SuperLeft => 0x5B, // VK_LWIN
SuperRight => 0x5C, // VK_RWIN
ContextMenu => 0x5D,
// --- Numpad ---
Numpad0 => 0x60,
Numpad1 => 0x61,
Numpad2 => 0x62,
Numpad3 => 0x63,
Numpad4 => 0x64,
Numpad5 => 0x65,
Numpad6 => 0x66,
Numpad7 => 0x67,
Numpad8 => 0x68,
Numpad9 => 0x69,
NumpadMultiply => 0x6A,
NumpadAdd => 0x6B,
NumpadEnter => 0x6C, // VK_SEPARATOR (matches the Linux client's KP_ENTER mapping)
NumpadSubtract => 0x6D,
NumpadDecimal => 0x6E,
NumpadDivide => 0x6F,
// --- Function keys ---
F1 => 0x70,
F2 => 0x71,
F3 => 0x72,
F4 => 0x73,
F5 => 0x74,
F6 => 0x75,
F7 => 0x76,
F8 => 0x77,
F9 => 0x78,
F10 => 0x79,
F11 => 0x7A,
F12 => 0x7B,
// --- Locks ---
NumLock => 0x90,
ScrollLock => 0x91,
// --- Left/right modifiers ---
ShiftLeft => 0xA0,
ShiftRight => 0xA1,
ControlLeft => 0xA2,
ControlRight => 0xA3,
AltLeft => 0xA4,
AltRight => 0xA5,
// --- OEM punctuation (US-layout positions) ---
Semicolon => 0xBA, // VK_OEM_1
Equal => 0xBB, // VK_OEM_PLUS
Comma => 0xBC, // VK_OEM_COMMA
Minus => 0xBD, // VK_OEM_MINUS
Period => 0xBE, // VK_OEM_PERIOD
Slash => 0xBF, // VK_OEM_2
Backquote => 0xC0, // VK_OEM_3
BracketLeft => 0xDB, // VK_OEM_4
Backslash => 0xDC, // VK_OEM_5
BracketRight => 0xDD, // VK_OEM_6
Quote => 0xDE, // VK_OEM_7
IntlBackslash => 0xE2, // VK_OEM_102 (the 102nd key)
_ => return None,
})
}
#[cfg(test)]
mod tests {
use super::*;
/// Spot-check positions against the Windows VK constants the host's `vk_to_evdev` knows.
#[test]
fn maps_known_positions() {
assert_eq!(keycode_to_vk(KeyCode::KeyA), Some(0x41));
assert_eq!(keycode_to_vk(KeyCode::KeyZ), Some(0x5A));
assert_eq!(keycode_to_vk(KeyCode::Digit0), Some(0x30));
assert_eq!(keycode_to_vk(KeyCode::Escape), Some(0x1B));
assert_eq!(keycode_to_vk(KeyCode::F11), Some(0x7A));
assert_eq!(keycode_to_vk(KeyCode::ShiftLeft), Some(0xA0));
assert_eq!(keycode_to_vk(KeyCode::IntlBackslash), Some(0xE2));
assert_eq!(keycode_to_vk(KeyCode::Numpad9), Some(0x69));
// A key outside the wire contract is dropped, not guessed.
assert_eq!(keycode_to_vk(KeyCode::AudioVolumeMute), None);
}
}
+76 -128
View File
@@ -1,18 +1,16 @@
//! `punktfunk-client` — the native Windows punktfunk/1 client.
//!
//! Pure Rust: `NativeClient` linked as a crate (no C ABI, like the GTK Linux client) ·
//! FFmpeg decode · WASAPI audio · SDL3 gamepads · a winit window + Direct3D11 flip-model
//! swapchain present surface. The trust surface mirrors the other native clients: persistent
//! identity, trust-on-first-use, SPAKE2 PIN pairing.
//! Pure Rust: `NativeClient` linked as a crate (no C ABI, like the GTK Linux client) · FFmpeg
//! decode · WASAPI audio · SDL3 gamepads · a **WinUI 3** shell (windows-reactor) with the video
//! on a `SwapChainPanel` bound to a D3D11 composition swapchain. The trust surface mirrors the
//! other native clients: persistent identity, trust-on-first-use, SPAKE2 PIN pairing — all in-app
//! (host list, settings, pairing). `--headless` keeps a CLI connect path for tests/measurement.
//!
//! Usage:
//! punktfunk-client --connect host[:port] [--pin HEX] [--pair PIN] [--mode WxHxHz]
//! [--bitrate MBPS] [--mic]
//! punktfunk-client --headless --connect … (no window; count frames + print stats)
//!
//! Trust: an explicit `--pin HEX` (or a host already pinned in the known-hosts store) connects
//! silently; `--pair PIN` runs the SPAKE2 ceremony first; otherwise the connect is
//! trust-on-first-use (the observed fingerprint is pinned on success).
//! punktfunk-client (open the WinUI 3 window: host list, settings, pairing)
//! punktfunk-client --discover (list punktfunk hosts on the LAN)
//! punktfunk-client --headless --connect host[:port] [--pin HEX] [--pair PIN] [--mode WxHxHz]
//! [--bitrate MBPS] [--mic] (no window; count frames + print stats)
#[cfg(windows)]
mod app;
@@ -23,8 +21,6 @@ mod discovery;
#[cfg(windows)]
mod gamepad;
#[cfg(windows)]
mod keymap;
#[cfg(windows)]
mod present;
#[cfg(windows)]
mod session;
@@ -35,8 +31,6 @@ mod video;
#[cfg(windows)]
fn main() {
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env().unwrap_or_else(|_| "info".into()),
@@ -44,12 +38,6 @@ fn main() {
.init();
let args: Vec<String> = std::env::args().collect();
let arg = |name: &str| -> Option<String> {
args.iter()
.position(|a| a == name)
.and_then(|i| args.get(i + 1))
.cloned()
};
let flag = |name: &str| args.iter().any(|a| a == name);
if flag("--discover") {
@@ -57,57 +45,6 @@ fn main() {
return;
}
let Some(target) = arg("--connect") else {
eprintln!(
"punktfunk-client: --connect host[:port] [--pin HEX] [--pair PIN] [--mode WxHxHz] \
[--bitrate MBPS] [--mic] [--headless]\n\
punktfunk-client --discover (list punktfunk hosts on the LAN)"
);
std::process::exit(2);
};
// Saved settings supply defaults when a CLI flag is absent (the GUI host-list/settings
// chrome is a follow-up; until then these are the persisted preferences). A CLI flag both
// overrides and is written back, so the next bare run reuses it.
let mut settings = trust::Settings::load();
let (host, port) = match target.rsplit_once(':') {
Some((a, p)) => (a.to_string(), p.parse().unwrap_or(9777)),
None => (target.clone(), 9777u16),
};
// CLI overrides fold into the persisted settings, then we derive the effective values.
if let Some(m) = arg("--mode").and_then(|m| {
let mut it = m.split(['x', 'X']);
Some((
it.next()?.parse::<u32>().ok()?,
it.next()?.parse::<u32>().ok()?,
it.next()?.parse::<u32>().ok()?,
))
}) {
(settings.width, settings.height, settings.refresh_hz) = m;
}
if let Some(b) = arg("--bitrate").and_then(|b| b.parse::<u32>().ok()) {
settings.bitrate_kbps = b * 1000;
}
if flag("--mic") {
settings.mic_enabled = true;
}
settings.save();
let mode = if settings.width != 0 && settings.refresh_hz != 0 {
Mode {
width: settings.width,
height: settings.height,
refresh_hz: settings.refresh_hz,
}
} else {
Mode {
width: 1920,
height: 1080,
refresh_hz: 60,
}
};
let bitrate_kbps = settings.bitrate_kbps;
let mic_enabled = settings.mic_enabled;
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
@@ -116,7 +53,61 @@ fn main() {
}
};
// Resolve trust: explicit pin > already-pinned host > pairing ceremony > TOFU.
if flag("--headless") {
run_headless_cli(&args, identity);
return;
}
// Windowed (default): the WinUI 3 app owns host selection, settings, and pairing.
let gamepad = gamepad::GamepadService::start();
if let Err(e) = app::run(identity, gamepad) {
tracing::error!(error = %e, "WinUI app failed");
std::process::exit(1);
}
}
/// `--headless --connect host[:port] …`: connect from the CLI, count frames, print stats — the
/// Windows analogue of `punktfunk-client-rs`.
#[cfg(windows)]
fn run_headless_cli(args: &[String], identity: (String, String)) {
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use std::time::{Duration, Instant};
let arg = |name: &str| -> Option<String> {
args.iter()
.position(|a| a == name)
.and_then(|i| args.get(i + 1))
.cloned()
};
let flag = |name: &str| args.iter().any(|a| a == name);
let Some(target) = arg("--connect") else {
eprintln!("--headless requires --connect host[:port]");
std::process::exit(2);
};
let (host, port) = match target.rsplit_once(':') {
Some((a, p)) => (a.to_string(), p.parse().unwrap_or(9777)),
None => (target.clone(), 9777u16),
};
let mode = arg("--mode")
.and_then(|m| {
let mut it = m.split(['x', 'X']);
Some(Mode {
width: it.next()?.parse().ok()?,
height: it.next()?.parse().ok()?,
refresh_hz: it.next()?.parse().ok()?,
})
})
.unwrap_or(Mode {
width: 1280,
height: 720,
refresh_hz: 60,
});
let bitrate_kbps = arg("--bitrate")
.and_then(|b| b.parse::<u32>().ok())
.map(|m| m * 1000)
.unwrap_or(0);
let known = trust::KnownHosts::load();
let mut pin = arg("--pin")
.and_then(|h| trust::parse_hex32(&h))
@@ -133,7 +124,7 @@ fn main() {
(&identity.0, &identity.1),
code.trim(),
&name,
std::time::Duration::from_secs(90),
Duration::from_secs(90),
) {
Ok(fp) => {
let mut k = trust::KnownHosts::load();
@@ -149,59 +140,25 @@ fn main() {
pin = Some(fp);
}
Err(e) => {
eprintln!("Pairing failed: {e:?} (wrong PIN, or pairing not armed on the host?)");
eprintln!("Pairing failed: {e:?}");
std::process::exit(1);
}
}
}
let headless = flag("--headless");
// The app-lifetime gamepad service runs only for the windowed client; it also resolves the
// "Automatic" pad type to whatever physical controller is attached (other-client parity).
let gamepad_service = (!headless).then(gamepad::GamepadService::start);
let gamepad_pref = match GamepadPref::from_name(&settings.gamepad) {
Some(GamepadPref::Auto) | None => gamepad_service
.as_ref()
.map_or(GamepadPref::Auto, |s| s.auto_pref()),
Some(explicit) => explicit,
};
tracing::info!(%host, port, ?mode, tofu = pin.is_none(), "connecting");
tracing::info!(%host, port, ?mode, tofu = pin.is_none(), "connecting (headless)");
let handle = session::start(session::SessionParams {
host: host.clone(),
host,
port,
mode,
compositor: CompositorPref::Auto,
gamepad: gamepad_pref,
gamepad: GamepadPref::Auto,
bitrate_kbps,
mic_enabled,
mic_enabled: flag("--mic"),
pin,
identity,
});
if headless {
run_headless(handle);
return;
}
let info = app::ConnectInfo {
name: host.clone(),
addr: host,
port,
tofu: pin.is_none(),
};
let gamepad_service = gamepad_service.expect("started for the windowed path");
if let Err(e) = app::WinApp::new(handle, info, gamepad_service).run() {
tracing::error!(error = %e, "windowed app failed");
std::process::exit(1);
}
}
/// Headless runner (`--headless`): drain events + frames, print stats, exit when the host
/// ends or the harness deadline elapses — the Windows analogue of `punktfunk-client-rs`.
#[cfg(windows)]
fn run_headless(handle: session::SessionHandle) {
use std::time::{Duration, Instant};
let deadline = Instant::now() + Duration::from_secs(60);
let mut frames_seen = 0u64;
loop {
@@ -218,16 +175,8 @@ fn run_headless(handle: session::SessionHandle) {
frames_seen,
"stats"
),
session::SessionEvent::Failed {
msg,
trust_rejected,
} => {
tracing::error!(%msg, trust_rejected, "connect failed");
if trust_rejected {
tracing::error!(
"host fingerprint changed or pairing required — re-pair with --pair PIN"
);
}
session::SessionEvent::Failed { msg, .. } => {
tracing::error!(%msg, "connect failed");
return;
}
session::SessionEvent::Ended(err) => {
@@ -248,8 +197,7 @@ fn run_headless(handle: session::SessionHandle) {
}
}
/// `--discover`: browse the LAN for punktfunk hosts (mDNS) and print them, then exit — the
/// CLI analogue of the GTK client's discovered-hosts list.
/// `--discover`: browse the LAN for punktfunk hosts (mDNS) and print them, then exit.
#[cfg(windows)]
fn discover_and_print() {
use std::time::{Duration, Instant};
@@ -281,9 +229,9 @@ fn discover_and_print() {
}
}
/// Win32/Direct3D11/WASAPI/SDL3 are Windows turf; this stub keeps `cargo build --workspace`
/// green on Linux/macOS (the other native clients live in crates/punktfunk-client-linux and
/// clients/apple).
/// WinUI 3 / Direct3D11 / WASAPI / SDL3 are Windows turf; this stub keeps `cargo build
/// --workspace` green on Linux/macOS (the other native clients live in
/// crates/punktfunk-client-linux and clients/apple).
#[cfg(not(windows))]
fn main() {
eprintln!(
+167 -172
View File
@@ -1,17 +1,19 @@
//! Direct3D11 presenter: upload a decoded `CpuFrame` (RGBA) into a dynamic texture and draw
//! it Contain-fit into a flip-model swapchain bound to the window's HWND, then present.
//! Direct3D11 presenter for a WinUI 3 `SwapChainPanel`: upload a decoded `CpuFrame` (RGBA)
//! into a dynamic texture and draw it Contain-fit into a **composition** flip-model swapchain,
//! which the reactor stream page binds to the panel via `SwapChainPanelHandle::set_swap_chain`.
//!
//! The device prefers a hardware adapter and falls back to **WARP** (the GPU-less dev box
//! runs the whole present path in software). The draw is a single full-screen triangle
//! sampling the video texture; a letterbox is produced by clearing the back buffer black and
//! setting the viewport to the Contain-fit rect (no per-frame vertex buffer). This is the
//! SDR 8-bit path; the 10-bit/HDR present (`R10G10B10A2` + `SetColorSpace1(...G2084_P2020)`)
//! is a follow-up alongside the P010 D3D11VA decode.
//! The device prefers a hardware adapter and falls back to **WARP** (the GPU-less dev box runs
//! the whole present path in software). The draw is a single full-screen triangle sampling the
//! video texture; a letterbox is produced by clearing the back buffer black and setting the
//! viewport to the Contain-fit rect (no per-frame vertex buffer). SDR 8-bit path; the
//! 10-bit/HDR present (`R10G10B10A2` + `SetColorSpace1`) is a follow-up alongside P010 decode.
//!
//! All `windows` types here come from the same windows-rs commit as `windows-reactor`, so the
//! `IDXGISwapChain1` handed to `set_swap_chain` satisfies reactor's `windows_core::Interface`.
use crate::video::CpuFrame;
use anyhow::{anyhow, Context, Result};
use windows::core::{Interface, PCSTR};
use windows::Win32::Foundation::{HMODULE, HWND};
use windows::Win32::Graphics::Direct3D::Fxc::{D3DCompile, D3DCOMPILE_OPTIMIZATION_LEVEL3};
use windows::Win32::Graphics::Direct3D::{
ID3DBlob, D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_WARP, D3D_FEATURE_LEVEL_11_0,
@@ -35,63 +37,114 @@ SamplerState smp : register(s0);
float4 ps_main(VSOut i) : SV_Target { return tex.Sample(smp, i.uv); }
"#;
pub struct Renderer {
pub struct Presenter {
device: ID3D11Device,
context: ID3D11DeviceContext,
vs: ID3D11VertexShader,
ps: ID3D11PixelShader,
sampler: ID3D11SamplerState,
/// Video texture + its SRV + dimensions; recreated when the decoded size changes.
swap: IDXGISwapChain1,
rtv: Option<ID3D11RenderTargetView>,
/// Video texture + SRV + dimensions; recreated when the decoded size changes.
tex: Option<(ID3D11Texture2D, ID3D11ShaderResourceView, u32, u32)>,
/// Panel (swapchain) size in pixels, updated on resize.
panel_w: u32,
panel_h: u32,
}
impl Renderer {
pub fn new() -> Result<Renderer> {
impl Presenter {
/// Create the D3D11 device + composition swapchain + shaders, sized to the panel.
pub fn new(width: u32, height: u32) -> Result<Presenter> {
let (device, context) = create_device()?;
let vs_blob = compile(SHADER_HLSL, "vs_main", "vs_5_0")?;
let ps_blob = compile(SHADER_HLSL, "ps_main", "ps_5_0")?;
let (vs, ps) = unsafe {
let mut vs = None;
device
.CreateVertexShader(blob_bytes(&vs_blob), None, Some(&mut vs))
.context("CreateVertexShader")?;
let mut ps = None;
device
.CreatePixelShader(blob_bytes(&ps_blob), None, Some(&mut ps))
.context("CreatePixelShader")?;
(vs.unwrap(), ps.unwrap())
};
let sampler = unsafe {
let desc = D3D11_SAMPLER_DESC {
Filter: D3D11_FILTER_MIN_MAG_MIP_LINEAR,
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
MaxLOD: D3D11_FLOAT32_MAX,
..Default::default()
};
let mut s = None;
device
.CreateSamplerState(&desc, Some(&mut s))
.context("CreateSamplerState")?;
s.unwrap()
};
Ok(Renderer {
let (vs, ps, sampler) = build_pipeline(&device)?;
let swap = create_composition_swapchain(&device, width.max(1), height.max(1))?;
Ok(Presenter {
device,
context,
vs,
ps,
sampler,
swap,
rtv: None,
tex: None,
panel_w: width.max(1),
panel_h: height.max(1),
})
}
pub fn device(&self) -> &ID3D11Device {
&self.device
/// The DXGI swapchain to hand to `SwapChainPanelHandle::set_swap_chain`.
pub fn swap_chain(&self) -> &IDXGISwapChain1 {
&self.swap
}
/// Upload one decoded RGBA frame, recreating the GPU texture if the size changed.
pub fn upload(&mut self, frame: &CpuFrame) -> Result<()> {
/// Resize the back buffers to the panel's new size (drops the stale RTV).
pub fn resize(&mut self, width: u32, height: u32) {
if width == 0 || height == 0 || (width == self.panel_w && height == self.panel_h) {
return;
}
self.rtv = None; // release all back-buffer refs before ResizeBuffers
unsafe {
let _ = self.swap.ResizeBuffers(
0,
width,
height,
DXGI_FORMAT_UNKNOWN,
DXGI_SWAP_CHAIN_FLAG(0),
);
}
self.panel_w = width;
self.panel_h = height;
}
/// Present one decoded frame (Contain-fit) — or, when `frame` is `None`, just re-present the
/// last texture (or black). Called from the reactor `on_rendering` per-frame callback.
pub fn present(&mut self, frame: Option<&CpuFrame>) {
if let Some(f) = frame {
if let Err(e) = self.upload(f) {
tracing::warn!(error = %e, "frame upload failed");
}
}
let Ok(rtv) = self.rtv() else {
return;
};
let (pw, ph) = (self.panel_w, self.panel_h);
unsafe {
let c = &self.context;
c.ClearRenderTargetView(&rtv, &[0.0, 0.0, 0.0, 1.0]);
if let Some((_, srv, vw, vh)) = &self.tex {
// Contain-fit viewport: scale to the smaller axis, centre, letterbox the rest.
let (ww, wh, vfw, vfh) = (
pw as f32,
ph as f32,
(*vw).max(1) as f32,
(*vh).max(1) as f32,
);
let scale = (ww / vfw).min(wh / vfh);
let (dw, dh) = (vfw * scale, vfh * scale);
let (ox, oy) = ((ww - dw) / 2.0, (wh - dh) / 2.0);
c.OMSetRenderTargets(Some(&[Some(rtv.clone())]), None);
let vp = D3D11_VIEWPORT {
TopLeftX: ox,
TopLeftY: oy,
Width: dw,
Height: dh,
MinDepth: 0.0,
MaxDepth: 1.0,
};
c.RSSetViewports(Some(&[vp]));
c.IASetInputLayout(None);
c.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
c.VSSetShader(&self.vs, None);
c.PSSetShader(&self.ps, None);
c.PSSetShaderResources(0, Some(&[Some(srv.clone())]));
c.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
c.Draw(3, 0);
}
let _ = self.swap.Present(1, DXGI_PRESENT(0));
}
}
fn upload(&mut self, frame: &CpuFrame) -> Result<()> {
let (w, h) = (frame.width, frame.height);
let need_new = !matches!(&self.tex, Some((_, _, tw, th)) if *tw == w && *th == h);
if need_new {
@@ -148,122 +201,7 @@ impl Renderer {
Ok(())
}
/// Clear the target black and draw the current video texture Contain-fit into the window.
pub fn draw(
&self,
rtv: &ID3D11RenderTargetView,
win_w: u32,
win_h: u32,
vid_w: u32,
vid_h: u32,
) {
let Some((_, srv, _, _)) = &self.tex else {
return;
};
// Contain-fit: scale to the smaller axis, centre, letterbox the rest.
let (ww, wh, vw, vh) = (
win_w as f32,
win_h as f32,
vid_w.max(1) as f32,
vid_h.max(1) as f32,
);
let scale = (ww / vw).min(wh / vh);
let (dw, dh) = (vw * scale, vh * scale);
let (ox, oy) = ((ww - dw) / 2.0, (wh - dh) / 2.0);
unsafe {
let c = &self.context;
c.ClearRenderTargetView(rtv, &[0.0, 0.0, 0.0, 1.0]);
c.OMSetRenderTargets(Some(&[Some(rtv.clone())]), None);
let vp = D3D11_VIEWPORT {
TopLeftX: ox,
TopLeftY: oy,
Width: dw,
Height: dh,
MinDepth: 0.0,
MaxDepth: 1.0,
};
c.RSSetViewports(Some(&[vp]));
c.IASetInputLayout(None);
c.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
c.VSSetShader(&self.vs, None);
c.PSSetShader(&self.ps, None);
c.PSSetShaderResources(0, Some(&[Some(srv.clone())]));
c.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
c.Draw(3, 0);
}
}
}
/// A flip-model swapchain bound to a window HWND, with a lazily-(re)built render-target view.
pub struct SwapChain {
swap: IDXGISwapChain1,
device: ID3D11Device,
rtv: Option<ID3D11RenderTargetView>,
pub width: u32,
pub height: u32,
}
impl SwapChain {
pub fn new(device: &ID3D11Device, hwnd: HWND, width: u32, height: u32) -> Result<SwapChain> {
let dxdev: IDXGIDevice = device.cast().context("IDXGIDevice cast")?;
let factory: IDXGIFactory2 = unsafe {
let adapter = dxdev.GetAdapter().context("GetAdapter")?;
adapter.GetParent().context("GetParent (IDXGIFactory2)")?
};
let desc = DXGI_SWAP_CHAIN_DESC1 {
Width: width.max(1),
Height: height.max(1),
Format: DXGI_FORMAT_R8G8B8A8_UNORM,
Stereo: false.into(),
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
BufferUsage: DXGI_USAGE_RENDER_TARGET_OUTPUT,
BufferCount: 2,
Scaling: DXGI_SCALING_STRETCH,
SwapEffect: DXGI_SWAP_EFFECT_FLIP_DISCARD,
AlphaMode: DXGI_ALPHA_MODE_IGNORE,
Flags: 0,
};
let swap = unsafe {
factory
.CreateSwapChainForHwnd(device, hwnd, &desc, None, None)
.context("CreateSwapChainForHwnd")?
};
Ok(SwapChain {
swap,
device: device.clone(),
rtv: None,
width: width.max(1),
height: height.max(1),
})
}
/// Resize the back buffers (window resize); drops the stale RTV so it rebuilds lazily.
pub fn resize(&mut self, width: u32, height: u32) -> Result<()> {
if width == 0 || height == 0 || (width == self.width && height == self.height) {
return Ok(());
}
self.rtv = None; // must release all back-buffer references before ResizeBuffers
unsafe {
self.swap
.ResizeBuffers(
0,
width,
height,
DXGI_FORMAT_UNKNOWN,
DXGI_SWAP_CHAIN_FLAG(0),
)
.context("ResizeBuffers")?;
}
self.width = width;
self.height = height;
Ok(())
}
/// The current back-buffer render-target view (built on first use after create/resize).
pub fn rtv(&mut self) -> Result<ID3D11RenderTargetView> {
fn rtv(&mut self) -> Result<ID3D11RenderTargetView> {
if self.rtv.is_none() {
let back: ID3D11Texture2D = unsafe { self.swap.GetBuffer(0).context("GetBuffer")? };
let rtv = unsafe {
@@ -277,13 +215,6 @@ impl SwapChain {
}
Ok(self.rtv.clone().unwrap())
}
/// Present the back buffer (vsync on — a stream is host-paced, tearing-free wins here).
pub fn present(&self) {
unsafe {
let _ = self.swap.Present(1, DXGI_PRESENT(0));
}
}
}
fn create_device() -> Result<(ID3D11Device, ID3D11DeviceContext)> {
@@ -294,7 +225,7 @@ fn create_device() -> Result<(ID3D11Device, ID3D11DeviceContext)> {
D3D11CreateDevice(
None,
driver,
HMODULE::default(),
None,
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
@@ -304,12 +235,12 @@ fn create_device() -> Result<(ID3D11Device, ID3D11DeviceContext)> {
)
};
if r.is_ok() {
let driver_name = if driver == D3D_DRIVER_TYPE_HARDWARE {
let name = if driver == D3D_DRIVER_TYPE_HARDWARE {
"hardware"
} else {
"WARP (software)"
};
tracing::info!(driver = driver_name, "D3D11 device created");
tracing::info!(driver = name, "D3D11 device created");
return Ok((device.unwrap(), context.unwrap()));
}
}
@@ -318,6 +249,70 @@ fn create_device() -> Result<(ID3D11Device, ID3D11DeviceContext)> {
))
}
/// A composition flip-model swapchain (no HWND) for binding to a XAML `SwapChainPanel`.
fn create_composition_swapchain(
device: &ID3D11Device,
width: u32,
height: u32,
) -> Result<IDXGISwapChain1> {
let dxdev: IDXGIDevice = device.cast().context("IDXGIDevice cast")?;
let factory: IDXGIFactory2 = unsafe {
let adapter = dxdev.GetAdapter().context("GetAdapter")?;
adapter.GetParent().context("GetParent (IDXGIFactory2)")?
};
let desc = DXGI_SWAP_CHAIN_DESC1 {
Width: width,
Height: height,
Format: DXGI_FORMAT_B8G8R8A8_UNORM,
Stereo: false.into(),
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
BufferUsage: DXGI_USAGE_RENDER_TARGET_OUTPUT,
BufferCount: 2,
Scaling: DXGI_SCALING_STRETCH,
SwapEffect: DXGI_SWAP_EFFECT_FLIP_SEQUENTIAL,
AlphaMode: DXGI_ALPHA_MODE_PREMULTIPLIED,
Flags: 0,
};
unsafe {
factory
.CreateSwapChainForComposition(device, &desc, None)
.context("CreateSwapChainForComposition")
}
}
fn build_pipeline(
device: &ID3D11Device,
) -> Result<(ID3D11VertexShader, ID3D11PixelShader, ID3D11SamplerState)> {
let vs_blob = compile(SHADER_HLSL, "vs_main", "vs_5_0")?;
let ps_blob = compile(SHADER_HLSL, "ps_main", "ps_5_0")?;
unsafe {
let mut vs = None;
device
.CreateVertexShader(blob_bytes(&vs_blob), None, Some(&mut vs))
.context("CreateVertexShader")?;
let mut ps = None;
device
.CreatePixelShader(blob_bytes(&ps_blob), None, Some(&mut ps))
.context("CreatePixelShader")?;
let sdesc = D3D11_SAMPLER_DESC {
Filter: D3D11_FILTER_MIN_MAG_MIP_LINEAR,
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
MaxLOD: D3D11_FLOAT32_MAX,
..Default::default()
};
let mut sampler = None;
device
.CreateSamplerState(&sdesc, Some(&mut sampler))
.context("CreateSamplerState")?;
Ok((vs.unwrap(), ps.unwrap(), sampler.unwrap()))
}
}
fn compile(src: &str, entry: &str, target: &str) -> Result<ID3DBlob> {
let entry_c = std::ffi::CString::new(entry).unwrap();
let target_c = std::ffi::CString::new(target).unwrap();