unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests Actions 9 Packages Projects Releases 15 Wiki Activity

feat(clients/windows): all-vendor video pipeline rewrite + app icon + hosts-page tiles

Browse Source 
Decode+present rewrite (first real pixels on glass for this client):

- Decode: FFmpeg D3D11VA on NVIDIA/AMD/Intel. get_format now only returns
  AV_PIX_FMT_D3D11 and lets libavcodec build the decode pool from
  hw_device_ctx (hand-built frames contexts failed three different ways:
  NVIDIA rejects DECODER|SHADER_RESOURCE arrays, BindFlags=0 fails texture
  creation, Intel rejects non-128-aligned HEVC surfaces at the first
  SubmitDecoderBuffers). A DXVA profile probe before the hwdevice commits
  hardware-vs-software up front instead of burning the opening IDR;
  extra_hw_frames covers the frames the client holds.
- Present: the decoded slice is copied with ONE display-size-boxed
  CopySubresourceRegion (a planar slice is a single subresource in D3D11;
  the old two-copy D3D12-style code silently no-opped - the black screen)
  into a sampleable NV12/P010 texture, per-plane SRVs + YUV->RGB shaders.
- New dedicated render thread (render.rs): presenting is decoupled from the
  XAML thread; frame-latency-waitable swapchain + SetMaximumFrameLatency(1),
  newest-wins drain after the wait, crossbeam frame channel with pts for a
  capture->presented p50 log.
- HiDPI: pixel-sized buffers + SetMatrixTransform(96/dpi) - was blurry at
  125/150 % scaling.
- Software fallback now feeds the same shaders (swscale -> NV12/P010 planes
  -> two dynamic plane textures); ps_rgba/X2BGR10 path deleted, hw/sw colour
  math identical.
- Adapter selection for hybrid boxes: PUNKTFUNK_ADAPTER > the window's
  monitor's adapter > default; PUNKTFUNK_D3D_DEBUG=1 debug layer.
- Session pump: request_keyframe at start and on hw->sw demotion (infinite
  GOP would otherwise sit on a black screen).

Validated live on the Arc Pro + RTX 3500 Ada laptop against the local
Windows host: 60 fps D3D11VA on both vendors, software path, GUI on glass.

Also: embedded app icon (build.rs winresource + WM_SETICON, MSIX
Square44x44 targetsize assets, pack-msix stages them) and the hosts-page
tile rework (tap-to-connect tiles with sibling overflow menu - fixes
forget-also-connects - in-tile rename editor, add-host modal via root state).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Enrico Bühler
2026-07-02 16:24:23 +02:00
parent 2c416a4bff
commit a4c84ac620
36 changed files with 1797 additions and 581 deletions
Download Patch File Download Diff File Expand all files Collapse all files
clients/windows/src/gpu.rs
+165 -53
View File
@@ -7,6 +7,16 @@
//! pull it from a process-global `OnceLock` (initialised on whichever thread asks first: the
//! session pump when it builds the decoder, or the UI thread when it builds the presenter).
//!
//! **Adapter selection** (matters on hybrid boxes — e.g. an Intel iGPU driving the panel next to
//! an NVIDIA dGPU): `PUNKTFUNK_ADAPTER` (index or case-insensitive name substring) wins; else the
//! adapter whose output owns the monitor our window is on — that's the adapter DWM composes that
//! monitor with, so presents are copy-free and decode runs on the near GPU; else the default
//! adapter. Deliberately NOT "the adapter with the best decoder": if the monitor's adapter can't
//! decode the codec we demote to software, which beats a per-frame cross-adapter present copy.
//!
//! `PUNKTFUNK_D3D_DEBUG=1` adds the D3D11 debug layer (validation messages in the debugger /
//! DebugView) — invaluable for present-path bugs, which D3D11 otherwise drops silently.
//!
//! **Thread-safety.** windows-rs COM interfaces are deliberately `!Send`/`!Sync` — thread-safety
//! is per-object, not universal. An `ID3D11Device` and its immediate context become free-threaded
//! once `ID3D11Multithread::SetMultithreadProtected(TRUE)` is set, which FFmpeg's D3D11VA backend
@@ -20,12 +30,15 @@ use anyhow::{anyhow, Result};
use std::sync::OnceLock;
use windows::core::Interface;
use windows::Win32::Graphics::Direct3D::{
D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_WARP, D3D_FEATURE_LEVEL_11_0, D3D_FEATURE_LEVEL_11_1,
D3D_DRIVER_TYPE, D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_UNKNOWN, D3D_DRIVER_TYPE_WARP,
D3D_FEATURE_LEVEL_11_0, D3D_FEATURE_LEVEL_11_1,
};
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, ID3D11Device, ID3D11DeviceContext, ID3D11Multithread,
D3D11_CREATE_DEVICE_BGRA_SUPPORT, D3D11_CREATE_DEVICE_VIDEO_SUPPORT, D3D11_SDK_VERSION,
D3D11_CREATE_DEVICE_BGRA_SUPPORT, D3D11_CREATE_DEVICE_DEBUG, D3D11_CREATE_DEVICE_FLAG,
D3D11_CREATE_DEVICE_VIDEO_SUPPORT, D3D11_SDK_VERSION,
};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter, IDXGIFactory1};
pub struct SharedDevice {
pub device: ID3D11Device,
@@ -60,61 +73,160 @@ fn create() -> Option<SharedDevice> {
}
}
fn create_device() -> Result<SharedDevice> {
// Preference order: a hardware adapter with video support (enables D3D11VA); the same without
// the VIDEO flag (a driver that rejects it still presents + software-decodes); finally WARP for
// the GPU-less box. BGRA_SUPPORT is required for the composition swapchain in every case.
let attempts = [
(D3D_DRIVER_TYPE_HARDWARE, true, true),
(D3D_DRIVER_TYPE_HARDWARE, false, true),
(D3D_DRIVER_TYPE_WARP, false, false),
];
for (driver, video, hardware) in attempts {
let flags = if video {
D3D11_CREATE_DEVICE_BGRA_SUPPORT | D3D11_CREATE_DEVICE_VIDEO_SUPPORT
/// The adapter's human-readable description, for the logs.
fn adapter_name(adapter: &IDXGIAdapter) -> String {
unsafe {
adapter
.GetDesc()
.map(|d| {
String::from_utf16_lossy(&d.Description)
.trim_end_matches('\0')
.to_string()
})
.unwrap_or_else(|_| "<unknown adapter>".into())
}
}
/// Resolve an explicit adapter: `PUNKTFUNK_ADAPTER` (index or case-insensitive name substring)
/// wins; else the adapter whose output owns the monitor the app window is on (see module docs);
/// else `None` → the default adapter (also the headless-CLI path, where no window exists).
fn resolve_adapter() -> Option<IDXGIAdapter> {
let factory: IDXGIFactory1 = unsafe { CreateDXGIFactory1() }.ok()?;
let adapters: Vec<IDXGIAdapter> = {
let mut v = Vec::new();
let mut i = 0u32;
while let Ok(a) = unsafe { factory.EnumAdapters1(i) } {
i += 1;
if let Ok(a) = a.cast::<IDXGIAdapter>() {
v.push(a);
}
}
v
};
if let Ok(pref) = std::env::var("PUNKTFUNK_ADAPTER") {
let pref = pref.trim();
let found = if let Ok(idx) = pref.parse::<usize>() {
adapters.get(idx).cloned()
} else {
D3D11_CREATE_DEVICE_BGRA_SUPPORT
let needle = pref.to_lowercase();
adapters
.iter()
.find(|a| adapter_name(a).to_lowercase().contains(&needle))
.cloned()
};
let mut device = None;
let mut context = None;
let r = unsafe {
D3D11CreateDevice(
None,
driver,
None,
flags,
Some(&[D3D_FEATURE_LEVEL_11_1, D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
};
if r.is_ok() {
let (device, context) = (device.unwrap(), context.unwrap());
// Make the device + immediate context free-threaded: the decoder (D3D11VA video context,
// pump thread) and the presenter (immediate context, UI thread) both touch this device.
// FFmpeg also sets this during hwdevice init, but doing it up front keeps the
// cross-thread `Send`/`Sync` sound from the moment the device exists.
if let Ok(mt) = context.cast::<ID3D11Multithread>() {
unsafe {
let _ = mt.SetMultithreadProtected(true); // returns the prior state; ignore
match &found {
Some(a) => {
tracing::info!(pref, adapter = %adapter_name(a), "PUNKTFUNK_ADAPTER matched")
}
None => tracing::warn!(pref, "PUNKTFUNK_ADAPTER matched no adapter — using default"),
}
if found.is_some() {
return found;
}
}
// The adapter driving the monitor our window sits on: DWM composes that monitor with it, so
// presenting from it is copy-free (a hybrid box's other adapter would pay a cross-adapter
// copy per frame).
let monitor = unsafe {
use windows::Win32::Graphics::Gdi::{MonitorFromWindow, MONITOR_DEFAULTTONULL};
use windows::Win32::UI::WindowsAndMessaging::FindWindowW;
let hwnd = FindWindowW(None, windows::core::w!("Punktfunk")).ok()?;
MonitorFromWindow(hwnd, MONITOR_DEFAULTTONULL)
};
if monitor.is_invalid() {
return None;
}
for adapter in &adapters {
let mut oi = 0u32;
while let Ok(output) = unsafe { adapter.EnumOutputs(oi) } {
oi += 1;
if let Ok(desc) = unsafe { output.GetDesc() } {
if desc.Monitor == monitor {
tracing::info!(adapter = %adapter_name(adapter), "using the window's monitor adapter");
return Some(adapter.clone());
}
}
tracing::info!(
driver = if hardware {
"hardware"
} else {
"WARP (software)"
},
video,
"shared D3D11 device created"
);
return Ok(SharedDevice {
device,
context,
hardware,
});
}
}
None
}
fn create_device() -> Result<SharedDevice> {
// Preference order: the resolved adapter (or the default hardware adapter) with video support
// (enables D3D11VA); the same without the VIDEO flag (a driver that rejects it still presents +
// software-decodes); finally WARP for the GPU-less box. BGRA_SUPPORT is required for the
// composition swapchain in every case. An explicit adapter requires D3D_DRIVER_TYPE_UNKNOWN.
let adapter = resolve_adapter();
let attempts: [(Option<&IDXGIAdapter>, D3D_DRIVER_TYPE, bool, bool); 3] = match &adapter {
Some(a) => [
(Some(a), D3D_DRIVER_TYPE_UNKNOWN, true, true),
(Some(a), D3D_DRIVER_TYPE_UNKNOWN, false, true),
(None, D3D_DRIVER_TYPE_WARP, false, false),
],
None => [
(None, D3D_DRIVER_TYPE_HARDWARE, true, true),
(None, D3D_DRIVER_TYPE_HARDWARE, false, true),
(None, D3D_DRIVER_TYPE_WARP, false, false),
],
};
// The debug layer needs the SDK layers installed (Graphics Tools); when they're missing the
// creation fails, so each attempt retries without the flag rather than failing the ladder.
let debug = std::env::var("PUNKTFUNK_D3D_DEBUG").is_ok_and(|v| v == "1");
for (adapter, driver, video, hardware) in attempts {
let mut flags = D3D11_CREATE_DEVICE_BGRA_SUPPORT;
if video {
flags |= D3D11_CREATE_DEVICE_VIDEO_SUPPORT;
}
let flag_sets: &[D3D11_CREATE_DEVICE_FLAG] = if debug {
&[flags | D3D11_CREATE_DEVICE_DEBUG, flags]
} else {
&[flags]
};
for &flags in flag_sets {
let mut device = None;
let mut context = None;
let r = unsafe {
D3D11CreateDevice(
adapter,
driver,
None,
flags,
Some(&[D3D_FEATURE_LEVEL_11_1, D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
};
if r.is_ok() {
let (device, context) = (device.unwrap(), context.unwrap());
// Make the device + immediate context free-threaded: the decoder (D3D11VA video
// context, pump thread) and the presenter (immediate context, render thread) both
// touch this device. FFmpeg also sets this during hwdevice init, but doing it up
// front keeps the cross-thread `Send`/`Sync` sound from the moment the device exists.
if let Ok(mt) = context.cast::<ID3D11Multithread>() {
unsafe {
let _ = mt.SetMultithreadProtected(true); // returns the prior state; ignore
}
}
tracing::info!(
adapter = %adapter.map(adapter_name).unwrap_or_else(|| if hardware {
"default".into()
} else {
"WARP (software)".into()
}),
video,
debug = (flags & D3D11_CREATE_DEVICE_DEBUG).0 != 0,
"shared D3D11 device created"
);
return Ok(SharedDevice {
device,
context,
hardware,
});
}
}
}
Err(anyhow!(