unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 18 Wiki Activity

feat(clients/windows): port the Vulkan session client to Windows — session-always

Browse Source 
The punktfunk-session Vulkan client (clients/linux-session, now clients/session)
builds and runs on Windows; the WinUI shell spawns it for every stream. Verified
live: 10-bit HEVC via Vulkan Video on both AMD (iGPU) and NVIDIA, 5120x1440 at
130 fps / 8 ms end-to-end on the RTX 4090.

- pf-ffvk: Windows bindgen branch (FFMPEG_DIR + PF_FFVK_VULKAN_INCLUDE, no
  pkg-config); provisioning fetches Vulkan-Headers (pinned v1.4.309).
- pf-client-core: builds on Windows — WASAPI audio (audio_wasapi.rs, cfg-swapped
  via #[path], same surface as the PipeWire twin), VAAPI/dmabuf gated inline
  (chain = vulkan -> software), trust reads the WinUI shell's %APPDATA% stores
  (parity tests pin both serialized shapes), Settings gains adapter/hdr_enabled
  (serde-defaulted; Linux stores unaffected).
- pf-presenter: builds on Windows — dmabuf module Linux-gated; SDL keyboard grab
  while captured (Alt+Tab/Win reach the host); pick_device ranks discrete over
  integrated (device 0 was the iGPU on hybrid boxes — the silent footgun) and
  honors PUNKTFUNK_VK_ADAPTER (the Settings GPU pick, exported by the session).
- run loop: block in one SDL wait woken by input AND decoded frames (a per-
  session forwarder pushes a FrameWake user event) instead of a 1 ms poll —
  measured 111%% -> 5%% of a core (NVIDIA), 86%% -> 3.5%% (AMD), stats unchanged.
  The pump's decode-fence wait became once-per-window sampling (no per-frame
  pipeline stall; the stat now shows true backlog).
- pf-console-ui: builds on Windows (skia-safe msvc prebuilts); font lookup falls
  through fontconfig aliases to concrete DirectWrite families (Consolas/Segoe UI)
  — browse/coverflow works, verified against a live host.
- WinUI shell: session-always via new src/spawn.rs (GTK spawn.rs port —
  CREATE_NO_WINDOW, stdout contract, kill handle); the Stream screen is a status
  card (chips + stage lines from the child's stats). The legacy in-process
  D3D11VA path stays behind Settings "Streaming engine" / PUNKTFUNK_BUILTIN_
  STREAM=1 as the A/B baseline until Phase 8 deletes it. SessionParams.video_caps
  makes the HDR toggle real.
- clients/linux-session renamed to clients/session (builds for both OSes).
- CI/MSIX: both workflows build/test both bins with widened path filters; the
  MSIX ships punktfunk-session.exe. ARM64 session builds --no-default-features
  (rust-skia has no aarch64-pc-windows-msvc prebuilts; flip when it does).

A/B on this box (5120x1440 HEVC vs home-worker-5): NVIDIA Vulkan 130 fps / 8 ms
e2e / 1.6 ms decode — clearly better than the built-in path. The AMD iGPU VCN
saturates at ~52 fps where its own D3D11VA does ~70 — Adrenalin Vulkan decode is
slower on APU silicon; discrete RDNA validation gates Phase 8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Enrico Bühler
2026-07-08 23:21:36 +02:00
parent 838a1239cf
commit d6647b9183
37 changed files with 1447 additions and 195 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/pf-client-core/Cargo.toml
+19 -11
View File
@@ -1,6 +1,6 @@
[package]
name = "pf-client-core"
description = "Shared Linux-client plumbing — session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads, trust store, discovery — extracted from the GTK client so the shell and the Vulkan session binary build on one implementation"
description = "Shared client plumbing (Linux + Windows) — session pump, FFmpeg decode, PipeWire/WASAPI audio, SDL3 gamepads, trust store, discovery — extracted from the GTK client so the shells and the Vulkan session binary build on one implementation"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
@@ -8,23 +8,20 @@ license.workspace = true
authors.workspace = true
repository.workspace = true
# Same Linux gating as clients/linux: `cargo build --workspace` stays green on macOS
# (the Mac client lives in clients/apple); elsewhere this crate is `wol` plus stubs-free
# emptiness. `wol` is pure std and stays cross-platform, matching the old main.rs.
[target.'cfg(target_os = "linux")'.dependencies]
# Linux + Windows: the Vulkan session client builds on both; `cargo build --workspace`
# stays green on macOS (the Mac client lives in clients/apple) — there this crate is
# `wol` plus stubs-free emptiness. `wol` is pure std and stays cross-platform, matching
# the old main.rs. Audio is the one per-OS swap: PipeWire on Linux, WASAPI on Windows
# (same public surface — see lib.rs).
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
# FFmpeg's Vulkan hwcontext surface (Vulkan Video decode on the presenter's device).
pf-ffvk = { path = "../pf-ffvk" }
async-channel = "2"
# Video decode (same FFmpeg pin as the host) and audio.
# Video decode (same FFmpeg pin as the host) and Opus for the audio planes.
ffmpeg-next = "8"
opus = "0.3"
pipewire = "0.9"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver).
sdl3 = { version = "0.18", features = ["hidapi"] }
mdns-sd = "0.20"
# Game-library fetch from the host's management API over mTLS + fingerprint pinning.
@@ -36,3 +33,14 @@ serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
tracing = "0.1"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver). Linux links the system SDL3; Windows builds it from source
# (no system SDL3 there — same choice as clients/windows).
[target.'cfg(target_os = "linux")'.dependencies]
pipewire = "0.9"
sdl3 = { version = "0.18", features = ["hidapi"] }
[target.'cfg(windows)'.dependencies]
wasapi = "0.23"
sdl3 = { version = "0.18", features = ["hidapi", "build-from-source"] }
crates/pf-client-core/src/audio_wasapi.rs
+333
View File
@@ -0,0 +1,333 @@
//! Audio: playback (decoded PCM → a WASAPI shared-mode render stream) and the microphone
//! uplink (WASAPI capture → Opus → 0xCB datagrams, the inverse of the host's virtual mic).
//!
//! The WASAPI twin of `audio.rs` (PipeWire) — same public surface (`AudioPlayer::spawn`/
//! `take_buffer`/`push`, `MicStreamer::spawn`), swapped in by lib.rs's `#[path]` so the
//! session pump compiles against one `crate::audio` on both OSes. Adapted from
//! `clients/windows/src/audio.rs` (which remains the WinUI shell's own copy until its
//! built-in streaming path is deleted).
//!
//! Playback mirrors the host's virtual-mic producer's adaptive jitter buffer: the session
//! pump pushes 5 ms Opus-decoded chunks on the network clock; the WASAPI render thread
//! pulls whole event-driven quanta on the device clock. Prime to ~3 quanta before
//! producing, cap the ring so latency stays bounded, re-prime after a real drain.
//!
//! WASAPI objects are COM-apartment-bound and not `Send`, so they live on a dedicated
//! thread (the same discipline as the host's `wasapi_cap`); only the channels + stop flag
//! + join handle cross the boundary.
use anyhow::{anyhow, Context, Result};
use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{Receiver, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::Duration;
use wasapi::{DeviceEnumerator, Direction, SampleType, StreamMode, WaveFormat};
const SAMPLE_RATE: usize = 48_000;
/// The microphone uplink stays stereo (the host's virtual mic is stereo). The render path is
/// multichannel — its channel count + block align are runtime, driven by the host-resolved layout.
const CHANNELS: usize = 2;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
pub struct AudioPlayer {
pcm_tx: SyncSender<Vec<f32>>,
/// Drained chunk Vecs coming back from the render thread for reuse (the pool half of
/// the pcm channel — see [`AudioPlayer::take_buffer`]).
recycle_rx: Receiver<Vec<f32>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl AudioPlayer {
/// Spawn the WASAPI render thread for `channels` (2/6/8, canonical wire order
/// FL FR FC LFE RL RR SL SR). Failure (no render endpoint on this box) is survivable — the
/// caller streams video-only.
pub fn spawn(channels: u32) -> Result<AudioPlayer> {
// 64 × 5 ms = 320 ms of slack between the pump and the WASAPI loop.
let (pcm_tx, pcm_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
// Return path: the render thread sends each drained Vec back for reuse, so
// steady-state playback stops allocating (~200 chunks/s otherwise). Same capacity
// as the data channel; a full pool just drops the Vec (plain deallocation).
let (recycle_tx, recycle_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
let stop = Arc::new(AtomicBool::new(false));
let (ready_tx, ready_rx) = std::sync::mpsc::sync_channel::<Result<()>>(1);
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-audio".into())
.spawn(move || {
if let Err(e) = render_thread(pcm_rx, recycle_tx, stop_t, ready_tx, channels as u8)
{
tracing::warn!(error = format!("{e:#}"), "audio playback thread ended");
}
})
.context("spawn audio thread")?;
match ready_rx.recv_timeout(Duration::from_secs(3)) {
Ok(Ok(())) => {
tracing::info!(channels, "WASAPI render: 48 kHz f32 (default endpoint)");
Ok(AudioPlayer {
pcm_tx,
recycle_rx,
stop,
thread: Some(thread),
})
}
Ok(Err(e)) => Err(e),
Err(_) => Err(anyhow!(
"wasapi render init timed out (no render endpoint?)"
)),
}
}
/// A recycled chunk Vec from the pool, empty but with its capacity intact — fill it
/// and hand it back through [`push`](Self::push). Allocates only when the pool is dry
/// (startup, or after the WASAPI side dropped chunks).
pub fn take_buffer(&self) -> Vec<f32> {
self.recycle_rx.try_recv().unwrap_or_default()
}
/// Queue one interleaved f32 chunk (in the session's channel layout). Drops the chunk if the
/// WASAPI side is wedged (the renderer conceals the gap; never block the session pump).
pub fn push(&self, pcm: Vec<f32>) {
if let Err(TrySendError::Disconnected(_)) = self.pcm_tx.try_send(pcm) {
// Thread already dead — Drop will reap it; nothing to do per-chunk.
}
}
}
impl Drop for AudioPlayer {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
fn render_thread(
pcm_rx: Receiver<Vec<f32>>,
recycle_tx: SyncSender<Vec<f32>>,
stop: Arc<AtomicBool>,
ready: SyncSender<Result<()>>,
channels: u8,
) -> Result<()> {
if let Err(e) = wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")
{
let _ = ready.send(Err(e));
return Ok(());
}
let res = (|| -> Result<()> {
// F32LE interleaved: channels × 4 bytes/sample. Stereo (channels == 2) is byte-identical
// to the old fixed path (mask 0x3, block align 8).
let block_align = channels as usize * 4;
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Render)
.context("default render endpoint")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
// The explicit dwChannelMask is the wire order (FL FR FC LFE RL RR SL SR); 5.1 = 0x3F,
// 7.1 = 0x63F. WASAPI delivers channels in ascending mask-bit order, which equals the wire
// order, so the render mapping is the identity — no permute. `autoconvert` (below) lets the
// audio engine downmix when the endpoint has fewer speakers.
let desired = WaveFormat::new(
32,
32,
&SampleType::Float,
SAMPLE_RATE,
channels as usize,
Some(punktfunk_core::audio::wasapi_channel_mask(channels)),
);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
autoconvert: true,
buffer_duration_hns: default_period,
};
audio_client
.initialize_client(&desired, &Direction::Render, &mode)
.context("initialize render client")?;
let h_event = audio_client.set_get_eventhandle().context("event handle")?;
let render_client = audio_client
.get_audiorenderclient()
.context("IAudioRenderClient")?;
audio_client.start_stream().context("start render stream")?;
let _ = ready.send(Ok(()));
// Adaptive jitter buffer, in f32-byte units (same shape as the host's virtual mic).
let mut ring: VecDeque<u8> = VecDeque::new();
let mut primed = false;
let mut out = Vec::new(); // per-quantum scratch, reused across iterations
while !stop.load(Ordering::Relaxed) {
if h_event.wait_for_event(100).is_err() {
continue;
}
// Drain everything the pump has queued into the ring, returning each drained
// Vec to the pool (a full/closed pool drops it).
while let Ok(mut chunk) = pcm_rx.try_recv() {
for s in chunk.iter() {
ring.extend(s.to_le_bytes());
}
chunk.clear();
let _ = recycle_tx.try_send(chunk);
}
let avail_frames = audio_client
.get_available_space_in_frames()
.context("available space")? as usize;
if avail_frames == 0 {
continue;
}
let want_bytes = avail_frames * block_align;
// Prime to ~3 quanta; cap at ~1 quantum of slack beyond that; re-prime on drain.
let target = (3 * want_bytes).clamp(720 * block_align, 9600 * block_align);
let cap = target.max(want_bytes) + want_bytes;
if ring.len() > cap {
ring.drain(..ring.len() - cap);
}
if !primed && ring.len() >= target {
primed = true;
}
out.clear();
out.resize(want_bytes, 0);
if primed {
let n = ring.len().min(want_bytes);
for (dst, b) in out.iter_mut().zip(ring.drain(..n)) {
*dst = b;
}
}
if ring.is_empty() {
primed = false;
}
render_client
.write_to_device(avail_frames, &out, None)
.context("write_to_device")?;
}
audio_client.stop_stream().ok();
Ok(())
})();
if let Err(ref e) = res {
let _ = ready.send(Err(anyhow!("{e:#}")));
}
res
}
/// The microphone uplink: capture the default input device, Opus-encode 20 ms chunks, ship
/// them as 0xCB datagrams into the host's virtual mic source.
pub struct MicStreamer {
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl MicStreamer {
pub fn spawn(connector: Arc<NativeClient>) -> Result<MicStreamer> {
let stop = Arc::new(AtomicBool::new(false));
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, stop_t) {
tracing::warn!(error = format!("{e:#}"), "mic uplink thread ended");
}
})
.context("spawn mic thread")?;
Ok(MicStreamer {
stop,
thread: Some(thread),
})
}
}
impl Drop for MicStreamer {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
fn mic_thread(connector: &Arc<NativeClient>, stop: Arc<AtomicBool>) -> Result<()> {
wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")?;
let mut encoder = opus::Encoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
opus::Application::Voip,
)
.map_err(|e| anyhow!("opus encoder: {e}"))?;
let _ = encoder.set_bitrate(opus::Bitrate::Bits(64_000));
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Capture)
.context("default capture endpoint (no microphone?)")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CHANNELS, None);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
autoconvert: true,
buffer_duration_hns: default_period,
};
audio_client
.initialize_client(&desired, &Direction::Capture, &mode)
.context("initialize capture client")?;
let h_event = audio_client.set_get_eventhandle().context("event handle")?;
let capture_client = audio_client
.get_audiocaptureclient()
.context("IAudioCaptureClient")?;
audio_client
.start_stream()
.context("start capture stream")?;
let mut bytes: VecDeque<u8> = VecDeque::new();
let mut ring: VecDeque<f32> = VecDeque::new();
let mut out = vec![0u8; 4000];
let mut seq = 0u32;
while !stop.load(Ordering::Relaxed) {
if h_event.wait_for_event(100).is_err() {
continue;
}
loop {
match capture_client.get_next_packet_size() {
Ok(Some(0)) | Ok(None) => break,
Ok(Some(_n)) => {
capture_client
.read_from_device_to_deque(&mut bytes)
.context("read capture")?;
}
Err(e) => return Err(anyhow!("get_next_packet_size: {e}")),
}
}
let whole = (bytes.len() / 4) * 4;
for c in bytes.drain(..whole).collect::<Vec<u8>>().chunks_exact(4) {
ring.push_back(f32::from_le_bytes([c[0], c[1], c[2], c[3]]));
}
// Ship every complete 20 ms stereo frame.
while ring.len() >= MIC_FRAME * CHANNELS {
let pcm: Vec<f32> = ring.drain(..MIC_FRAME * CHANNELS).collect();
match encoder.encode_float(&pcm, &mut out) {
Ok(len) => {
let pts = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as u64)
.unwrap_or(0);
let _ = connector.send_mic(seq, pts, out[..len].to_vec());
seq = seq.wrapping_add(1);
}
Err(e) => tracing::debug!(error = %e, "opus mic encode"),
}
}
}
audio_client.stop_stream().ok();
Ok(())
}
crates/pf-client-core/src/lib.rs
+21 -10
View File
@@ -1,26 +1,37 @@
//! Shared, UI-agnostic Linux-client plumbing, extracted verbatim from the GTK client
//! Shared, UI-agnostic client plumbing, extracted verbatim from the GTK client
//! (design: punktfunk-planning `linux-client-rearchitecture.md`, Phase 0) so the desktop
//! shell and the Vulkan session binary build on one implementation.
//! shells and the Vulkan session binary build on one implementation — on Linux AND
//! Windows (the session binary runs on both; macOS stays `wol`-only, clients/apple is
//! the client there).
//!
//! Nothing here may depend on a UI toolkit: the presenter contract is `session`'s
//! channels (`SessionHandle`) and `video`'s `DecodedImage` (RGBA bytes or dmabuf fds +
//! plane layout) — how frames reach the screen is the consumer's business.
//! channels (`SessionHandle`) and `video`'s `DecodedImage` (RGBA bytes, dmabuf fds +
//! plane layout, or a decoded VkImage) — how frames reach the screen is the consumer's
//! business.
//!
//! Audio is the one per-OS module swap: `audio.rs` (PipeWire) on Linux,
//! `audio_wasapi.rs` (WASAPI) on Windows — same public surface, picked here by `#[path]`
//! so `crate::audio` is the only name the session pump ever sees. `keymap` (evdev-keyed)
//! stays Linux: the session path uses pf-presenter's SDL-scancode table instead.
#[cfg(target_os = "linux")]
pub mod audio;
#[cfg(target_os = "linux")]
#[cfg(windows)]
#[path = "audio_wasapi.rs"]
pub mod audio;
#[cfg(any(target_os = "linux", windows))]
pub mod discovery;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod gamepad;
#[cfg(target_os = "linux")]
pub mod keymap;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod library;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod session;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod trust;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod video;
pub mod wol;
crates/pf-client-core/src/session.rs
+30 -17
View File
@@ -27,6 +27,12 @@ pub struct SessionParams {
/// The user's preferred video codec (a `quic::CODEC_*` bit, `0` = auto). Soft — the host honors
/// it when it can emit it, else falls back; the resolved codec drives the decoder.
pub preferred_codec: u8,
/// The advertised `quic::VIDEO_CAP_*` bits. Normally 10-bit + HDR (Main10/PQ: the
/// Vulkan presenter decodes P010 everywhere and presents PQ on an HDR10 swapchain
/// where the desktop offers one, tonemapping in the CSC shader where it doesn't;
/// the host still gates the upgrade behind its own PUNKTFUNK_10BIT policy) — `0`
/// when the user turned HDR off in Settings ("never send me 10-bit").
pub video_caps: u8,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see
@@ -211,11 +217,7 @@ fn pump(
params.compositor,
params.gamepad,
params.bitrate_kbps,
// 10-bit Main10 + PQ HDR10: the Vulkan presenter decodes P010 (Vulkan
// Video/VAAPI/software) and presents PQ on an HDR10 swapchain where the desktop
// offers one, tonemapping in the CSC shader where it doesn't. The host still
// gates the upgrade behind its own PUNKTFUNK_10BIT policy.
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR,
params.video_caps,
params.audio_channels,
crate::video::decodable_codecs(), // codecs FFmpeg can decode (HEVC/H.264/AV1)
params.preferred_codec, // the user's soft codec preference (0 = auto)
@@ -328,12 +330,14 @@ fn pump(
total_frames += 1;
dec_path = match &image {
DecodedImage::Cpu(_) => "software",
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(_) => "vaapi",
DecodedImage::VkFrame(_) => "vulkan",
};
if total_frames == 1 {
let (w, h, path) = match &image {
DecodedImage::Cpu(c) => (c.width, c.height, "software"),
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(d) => (d.width, d.height, "vaapi-dmabuf"),
DecodedImage::VkFrame(v) => (v.width, v.height, "vulkan-video"),
};
@@ -357,10 +361,16 @@ fn pump(
}
// Ship the frame FIRST, then settle the decode stat: on the
// Vulkan path receive_frame returns at SUBMISSION (~0.1 ms) and
// the hardware decodes asynchronously — waiting the frame's
// timeline fence here (after the presenter already has the
// frame) measures true received→decode-complete at zero
// pipeline cost. Software/VAAPI keep the synchronous stamp.
// the hardware decodes asynchronously — the frame's timeline
// fence measures true received→decode-complete. But the fence
// wait BLOCKS this thread, and per-frame that serializes the
// pipeline to 1/decode_latency (observed: an APU's 19 ms decode
// capping a 5120×1440 stream at ~51 fps while the engine could
// pipeline several frames — and drivers may spin-wait, burning
// CPU). So sample ONE frame per stats window: the p50 the OSD
// shows becomes that sample — honest, at zero pipeline cost on
// every other frame. Software keeps the synchronous stamp on
// every frame (its decode really is done by now).
let hw_fence = match &image {
DecodedImage::VkFrame(v) => Some((v.timeline_sem, v.decode_done_value)),
_ => None,
@@ -371,15 +381,18 @@ fn pump(
image,
});
// `decode` stage: received→decode COMPLETE, single clock.
let decode_done_ns = match hw_fence {
Some((sem, value))
if decoder.wait_hw_decoded(sem, value, 50_000_000) =>
{
now_ns()
match hw_fence {
Some((sem, value)) => {
if decode_us.is_empty()
&& decoder.wait_hw_decoded(sem, value, 50_000_000)
{
decode_us.push(now_ns().saturating_sub(received_ns) / 1000);
}
}
_ => decoded_ns,
};
decode_us.push(decode_done_ns.saturating_sub(received_ns) / 1000);
None => {
decode_us.push(decoded_ns.saturating_sub(received_ns) / 1000);
}
}
}
Ok(None) => no_output_streak += 1,
// Survivable (loss until the next IDR/RFI recovery) — keep feeding.
crates/pf-client-core/src/trust.rs
+90 -4
View File
@@ -1,7 +1,12 @@
//! Client identity, the known-hosts (pinned fingerprint) store, and app settings.
//!
//! The identity shares `~/.config/punktfunk/client-{cert,key}.pem` with `punktfunk-probe`
//! so a box pairs once whichever client it uses.
//! The identity shares `~/.config/punktfunk/client-{cert,key}.pem` (Linux; on Windows
//! `%APPDATA%\punktfunk`, the WinUI shell's directory) with `punktfunk-probe` so a box
//! pairs once whichever client it uses. On Windows the session binary reads the SAME
//! stores the WinUI shell (`clients/windows/src/trust.rs`) writes — pairing there makes
//! the session connect silently, mirroring the GTK-shell arrangement on Linux. The two
//! `Settings` structs differ in shape; `#[serde(default)]` on both sides reconciles them
//! (see the parity tests below), and the shell stays the settings file's only writer.
use anyhow::{anyhow, Context, Result};
use punktfunk_core::client::NativeClient;
@@ -10,8 +15,16 @@ use serde::{Deserialize, Serialize};
use std::path::PathBuf;
pub fn config_dir() -> Result<PathBuf> {
let home = std::env::var("HOME").context("HOME unset")?;
Ok(PathBuf::from(home).join(".config/punktfunk"))
#[cfg(windows)]
{
let appdata = std::env::var("APPDATA").context("APPDATA unset")?;
Ok(PathBuf::from(appdata).join("punktfunk"))
}
#[cfg(not(windows))]
{
let home = std::env::var("HOME").context("HOME unset")?;
Ok(PathBuf::from(home).join(".config/punktfunk"))
}
}
/// This client's persistent identity, generated on first use — presented on every connect
@@ -256,6 +269,18 @@ pub struct Settings {
/// `"vulkan"`, `"vaapi"`, `"software"`.
/// The `PUNKTFUNK_DECODER` env var overrides this (see `video::Decoder::new`).
pub decoder: String,
/// Decode/present GPU (multi-GPU boxes): the adapter's marketing name, as the WinUI
/// shell's GPU picker stores it; empty = automatic. The session maps it onto the
/// presenter's device pick (`PUNKTFUNK_VK_ADAPTER`). `default` so pre-existing
/// stores (and the Linux shells, which have no picker yet) load.
#[serde(default)]
pub adapter: String,
/// Advertise 10-bit + HDR10 so the host upgrades HDR content to a Main10/PQ stream.
/// The presenter handles the display side dynamically either way (HDR10 swapchain
/// where offered, tonemap where not) — off means "never send me 10-bit".
/// `default = true`: the Linux stores never carried this and always advertised.
#[serde(default = "default_true")]
pub hdr_enabled: bool,
/// Show the on-stream statistics overlay (toggle live with Ctrl+Alt+Shift+S).
pub show_stats: bool,
/// Enter fullscreen when a stream starts (F11 / the controller chord / the top-edge
@@ -270,6 +295,10 @@ fn default_codec() -> String {
"auto".into()
}
fn default_true() -> bool {
true
}
impl Settings {
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 {
@@ -297,6 +326,8 @@ impl Default for Settings {
audio_channels: 2,
codec: "auto".into(),
decoder: "auto".into(),
adapter: String::new(),
hdr_enabled: true,
show_stats: true,
fullscreen_on_stream: true,
library_enabled: false,
@@ -306,6 +337,13 @@ impl Default for Settings {
impl Settings {
fn path() -> Result<PathBuf> {
// The shell's settings file on each OS: the GTK shell's on Linux, the WinUI
// shell's on Windows. The shells own (and write) these files; the session binary
// only reads them, so `save` must never be called on Windows — it would rewrite
// the file in THIS struct's shape and drop the WinUI-only fields.
#[cfg(windows)]
return Ok(config_dir()?.join("client-windows-settings.json"));
#[cfg(not(windows))]
Ok(config_dir()?.join("client-gtk-settings.json"))
}
@@ -340,4 +378,52 @@ mod tests {
let round: Settings = serde_json::from_str(&serde_json::to_string(&s).unwrap()).unwrap();
assert_eq!(round.forward_pad, "");
}
/// On Windows the session reads the WinUI shell's settings file. This fixture is the
/// shell's `Settings` shape (clients/windows/src/trust.rs) verbatim — if that struct
/// changes, update this fixture with it. WinUI-only fields (hdr_enabled, adapter,
/// show_hud) must be ignored; fields this struct has and the shell's lacks
/// (forward_pad, show_stats, …) must default; the shell's D3D11VA-era
/// `decoder: "hardware"` must survive as-is (video::Decoder::new reads it as auto).
#[test]
fn settings_reads_winui_shell_shape() {
let shell = r#"{
"width": 2560, "height": 1440, "refresh_hz": 120, "bitrate_kbps": 20000,
"gamepad": "dualsense", "compositor": "auto",
"inhibit_shortcuts": true, "mic_enabled": true, "audio_channels": 6,
"hdr_enabled": true, "decoder": "hardware", "codec": "av1",
"adapter": "NVIDIA GeForce RTX 4080", "show_hud": false
}"#;
let s: Settings = serde_json::from_str(shell).unwrap();
assert_eq!((s.width, s.height, s.refresh_hz), (2560, 1440, 120));
assert_eq!(s.bitrate_kbps, 20000);
assert_eq!(s.audio_channels, 6);
assert!(s.mic_enabled);
assert_eq!(s.decoder, "hardware");
assert_eq!(s.preferred_codec(), punktfunk_core::quic::CODEC_AV1);
assert_eq!(s.adapter, "NVIDIA GeForce RTX 4080");
assert!(s.hdr_enabled);
// Fields the shell's file doesn't carry take this struct's defaults.
assert_eq!(s.forward_pad, "");
assert!(s.show_stats);
assert!(s.fullscreen_on_stream);
assert!(!s.library_enabled);
}
/// The WinUI shell's known-hosts shape (no `last_used` field) loads losslessly — same
/// filename, same directory, so on Windows the two clients genuinely share the store.
#[test]
fn known_hosts_reads_winui_shell_shape() {
let shell = r#"{"hosts":[{
"name": "Gaming PC", "addr": "192.168.1.50", "port": 9777,
"fp_hex": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"paired": true, "mac": ["aa:bb:cc:dd:ee:ff"]
}]}"#;
let k: KnownHosts = serde_json::from_str(shell).unwrap();
let h = k.find_by_addr("192.168.1.50", 9777).unwrap();
assert!(h.paired);
assert_eq!(h.last_used, None);
assert_eq!(h.mac, vec!["aa:bb:cc:dd:ee:ff".to_string()]);
assert!(parse_hex32(&h.fp_hex).is_some());
}
}
crates/pf-client-core/src/video.rs
+44 -12
View File
@@ -18,12 +18,22 @@
//!
//! Both run `AV_CODEC_FLAG_LOW_DELAY`; the host encodes zero-reorder streams (no
//! B-frames, in-band parameter sets on every IDR), so decode is strictly one-in/one-out.
//!
//! On Windows the VAAPI/dmabuf backend does not exist (DRM-PRIME is a Linux concept), so
//! the chain is Vulkan → software — Intel (no Vulkan Video in its Windows driver) lands
//! on software. Everything dmabuf-shaped is `cfg(target_os = "linux")`-gated inline.
// bindgen's C-enum repr is target-dependent (u32 on Linux/clang, i32 on MSVC), so the
// pf-ffvk Vulkan flag/enum casts below are required on one platform and no-ops on the
// other — the lint would fire on whichever platform the cast is a no-op for.
#![allow(clippy::unnecessary_cast)]
use anyhow::{anyhow, bail, Context as _, Result};
use ffmpeg::format::Pixel;
use ffmpeg::software::scaling;
use ffmpeg::util::frame::Video as AvFrame;
use ffmpeg_next as ffmpeg;
#[cfg(target_os = "linux")]
use std::os::fd::RawFd;
use std::ptr;
@@ -42,6 +52,7 @@ pub struct DecodedFrame {
pub enum DecodedImage {
Cpu(CpuFrame),
#[cfg(target_os = "linux")]
Dmabuf(DmabufFrame),
/// FFmpeg Vulkan Video output: a VkImage already on the PRESENTER's device.
VkFrame(VkVideoFrame),
@@ -136,6 +147,7 @@ pub struct CpuFrame {
/// A decoded frame still on the GPU: dmabuf fds + plane layout for
/// `GdkDmabufTextureBuilder`. The fds belong to `guard`'s mapped DRM frame — they stay
/// valid until the guard drops (the texture's release func).
#[cfg(target_os = "linux")]
pub struct DmabufFrame {
pub width: u32,
pub height: u32,
@@ -150,6 +162,7 @@ pub struct DmabufFrame {
pub guard: DrmFrameGuard,
}
#[cfg(target_os = "linux")]
pub struct DmabufPlane {
pub fd: RawFd,
pub offset: u32,
@@ -170,6 +183,7 @@ impl Drop for DrmFrameGuard {
enum Backend {
Vulkan(VulkanDecoder),
#[cfg(target_os = "linux")]
Vaapi(VaapiDecoder),
Software(SoftwareDecoder),
}
@@ -240,12 +254,13 @@ fn quiet_ffmpeg_log() {
impl Decoder {
/// `codec_id` is the codec the host resolved in the Welcome (never assume HEVC).
/// `pref` is the Settings "Video decoder" value (`auto`/`vulkan`/`vaapi`/`software`).
/// `pref` is the Settings "Video decoder" value (`auto`/`vulkan`/`vaapi`/`software`;
/// `hardware` — the WinUI shell's stored value from its D3D11VA era — reads as auto).
/// `vk` is the presenter's shared Vulkan device when its stack can run FFmpeg's
/// Vulkan Video decoder — decode lands as VkImages the presenter samples directly.
/// Precedence: the `PUNKTFUNK_DECODER` env override wins (support/debug escape
/// hatch, and the documented knob), then the setting; both default to auto
/// (Vulkan → VAAPI → software).
/// (Vulkan → VAAPI → software; no VAAPI on Windows).
pub fn new(
codec_id: ffmpeg::codec::Id,
pref: &str,
@@ -265,7 +280,7 @@ impl Decoder {
want_keyframe: false,
})
};
if matches!(choice.as_str(), "auto" | "" | "vulkan") {
if matches!(choice.as_str(), "auto" | "" | "vulkan" | "hardware") {
match vk {
Some(vk) => match VulkanDecoder::new(codec_id, vk) {
Ok(v) => {
@@ -294,7 +309,9 @@ impl Decoder {
}
// Deck note: `auto` reaches VAAPI when Vulkan Video isn't available. A presenter
// that can't display the dmabufs demotes this decoder to software mid-session
// via [`Decoder::force_software`].
// via [`Decoder::force_software`]. Windows has no VAAPI — auto falls straight
// through to software there.
#[cfg(target_os = "linux")]
if choice != "software" && choice != "vulkan" {
match VaapiDecoder::new(codec_id) {
Ok(v) => {
@@ -361,6 +378,7 @@ impl Decoder {
pub fn decode(&mut self, au: &[u8]) -> Result<Option<DecodedImage>> {
let result = match &mut self.backend {
Backend::Vulkan(v) => v.decode(au).map(|f| f.map(DecodedImage::VkFrame)),
#[cfg(target_os = "linux")]
Backend::Vaapi(v) => v.decode(au).map(|f| f.map(DecodedImage::Dmabuf)),
Backend::Software(s) => return Ok(s.decode(au)?.map(DecodedImage::Cpu)),
};
@@ -532,7 +550,8 @@ impl SoftwareDecoder {
// Raw FFI: ffmpeg-next has no hwaccel wrappers. All pointers are owned here and freed in
// Drop; decoded surfaces transfer out through DrmFrameGuard.
const AVERROR_EAGAIN: i32 = -11; // -EAGAIN; Linux-only crate
// -EAGAIN. FFmpeg uses POSIX errno values on both our targets (MinGW's EAGAIN is 11 too).
const AVERROR_EAGAIN: i32 = -11;
fn averr(what: &str, code: i32) -> anyhow::Error {
anyhow!("{what}: {}", ffmpeg::Error::from(code))
@@ -542,6 +561,7 @@ fn averr(what: &str, code: i32) -> anyhow::Error {
/// back to the first (software) entry would silently decode on the CPU *and* break our
/// dmabuf mapping — return NONE instead so the error surfaces and the session demotes
/// to the software backend explicitly.
#[cfg(target_os = "linux")]
unsafe extern "C" fn pick_vaapi(
_ctx: *mut ffmpeg::ffi::AVCodecContext,
mut list: *const ffmpeg::ffi::AVPixelFormat,
@@ -557,6 +577,7 @@ unsafe extern "C" fn pick_vaapi(
ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE
}
#[cfg(target_os = "linux")]
struct VaapiDecoder {
ctx: *mut ffmpeg::ffi::AVCodecContext,
hw_device: *mut ffmpeg::ffi::AVBufferRef,
@@ -565,8 +586,10 @@ struct VaapiDecoder {
}
// Single-owner pointers, only touched from the session pump thread.
#[cfg(target_os = "linux")]
unsafe impl Send for VaapiDecoder {}
#[cfg(target_os = "linux")]
impl VaapiDecoder {
fn new(codec_id: ffmpeg::codec::Id) -> Result<VaapiDecoder> {
use ffmpeg::ffi;
@@ -764,6 +787,9 @@ const fn fourcc(a: u8, b: u8, c: u8, d: u8) -> u32 {
/// The combined DRM FourCC for a decoder software pixel format. The host streams 8-bit
/// 4:2:0 (NV12); P010 is here for the eventual 10-bit/HDR path.
// Only the (Linux-gated) VAAPI path calls this outside tests; the constants are worth
// locking on every platform, so it stays compiled rather than cfg-gated with its caller.
#[cfg_attr(windows, allow(dead_code))]
fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32> {
use ffmpeg_next::ffi::AVPixelFormat::*;
Some(match sw {
@@ -775,6 +801,7 @@ fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32> {
/// One-time dump of the DRM descriptor layout (objects, layers, planes, modifier) — so a
/// new client/driver combination's real layout is visible in the logs without a debugger.
#[cfg(target_os = "linux")]
fn log_descriptor_once(
d: &ffmpeg_next::ffi::AVDRMFrameDescriptor,
sw: ffmpeg_next::ffi::AVPixelFormat,
@@ -801,6 +828,7 @@ fn log_descriptor_once(
);
}
#[cfg(target_os = "linux")]
impl Drop for VaapiDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
@@ -915,26 +943,28 @@ impl VulkanDecoder {
(*hwctx).queue_family_decode_index = d;
(*hwctx).nb_decode_queues = 1;
const VIDEO_DECODE_BIT: u32 = 0x20; // VK_QUEUE_VIDEO_DECODE_BIT_KHR
// `flags`/`video_caps` are bindgen enum types: i32 under MSVC, u32 under
// Linux clang — the `as _` casts absorb the difference.
if g == d {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags | VIDEO_DECODE_BIT,
video_caps: vk.decode_video_caps,
flags: (vk.graphics_queue_flags | VIDEO_DECODE_BIT) as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 1;
} else {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags,
flags: vk.graphics_queue_flags as _,
video_caps: 0,
};
(*hwctx).qf[1] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: d,
num: 1,
flags: VIDEO_DECODE_BIT,
video_caps: vk.decode_video_caps,
flags: VIDEO_DECODE_BIT as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 2;
}
@@ -1155,8 +1185,10 @@ unsafe extern "C" fn pick_vulkan(
let fc = (*fr).data as *mut ffi::AVHWFramesContext;
let vkfc = (*fc).hwctx as *mut pf_ffvk::AVVulkanFramesContext;
// MUTABLE_FORMAT: per-plane views (spec requirement); ALIAS is FFmpeg's default.
(*vkfc).img_flags = pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT
| pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_ALIAS_BIT;
// (`as _`: the FlagBits constants are i32 under MSVC, the img_flags field u32.)
(*vkfc).img_flags = (pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT
| pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_ALIAS_BIT)
as _;
let r = ffi::av_hwframe_ctx_init(fr);
if r < 0 {
tracing::warn!("av_hwframe_ctx_init(VULKAN) failed ({r})");
crates/pf-console-ui/Cargo.toml
+12 -3
View File
@@ -8,17 +8,26 @@ license.workspace = true
authors.workspace = true
repository.workspace = true
[target.'cfg(target_os = "linux")'.dependencies]
# Same Linux+Windows gating as the rest of the client stack.
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
pf-presenter = { path = "../pf-presenter" }
# MenuEvent/MenuPulse (the gamepad service's menu mode drives the library).
pf-client-core = { path = "../pf-client-core" }
# Skia on the presenter's VkDevice (`vulkan`); `textlayout` = skparagraph/harfbuzz for
# the typography the console library needs (~15 MB stripped, prebuilt binaries exist for
# this feature set on x86_64-unknown-linux-gnu — a source build is never triggered).
# this feature set on x86_64-unknown-linux-gnu AND x86_64-pc-windows-msvc — a source
# build is never triggered on either).
skia-safe = { version = "0.87", features = ["vulkan", "textlayout"] }
ash = { version = "0.38", features = ["loaded"] }
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
anyhow = "1"
tracing = "0.1"
# Linux links the system SDL3; Windows builds it from source (same choice as the rest
# of the workspace's Windows SDL consumers).
[target.'cfg(target_os = "linux")'.dependencies]
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
[target.'cfg(windows)'.dependencies]
sdl3 = { version = "0.18", features = ["hidapi", "ash", "build-from-source"] }
crates/pf-console-ui/src/lib.rs
+5 -5
View File
@@ -8,14 +8,14 @@
//! purpose, it proves the whole shared-device pipeline. The gamepad library moves in
//! next.
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod library;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
mod library_ui;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
mod skia_overlay;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub use library::{LibraryGame, LibraryPhase, LibraryShared};
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub use skia_overlay::SkiaOverlay;
crates/pf-console-ui/src/library_ui.rs
+19 -3
View File
@@ -655,11 +655,27 @@ impl Fonts {
}
}
/// Resolve the first available family. Generic aliases ("sans-serif", "monospace")
/// resolve through fontconfig on Linux; Windows' DirectWrite-backed FontMgr has no
/// generic aliases, so the list falls through to concrete family names there.
pub(crate) fn match_first_family(
mgr: &skia_safe::FontMgr,
families: &[&str],
style: FontStyle,
) -> Option<Typeface> {
families
.iter()
.find_map(|f| mgr.match_family_style(f, style))
}
pub(crate) fn build_fonts() -> Result<Fonts> {
let mgr = skia_safe::FontMgr::new();
let sans = mgr
.match_family_style("sans-serif", FontStyle::normal())
.ok_or_else(|| anyhow!("no sans-serif typeface via fontconfig"))?;
let sans = match_first_family(
&mgr,
&["sans-serif", "Segoe UI", "Arial"],
FontStyle::normal(),
)
.ok_or_else(|| anyhow!("no sans-serif typeface (fontconfig alias or system family)"))?;
let mut collection = FontCollection::new();
collection.set_default_font_manager(mgr, None);
Ok(Fonts {
crates/pf-console-ui/src/skia_overlay.rs
+6 -3
View File
@@ -146,9 +146,12 @@ impl Overlay for SkiaOverlay {
.ok_or_else(|| anyhow!("Skia DirectContext over the shared device"))?;
context.set_resource_cache_limit(RESOURCE_CACHE_BYTES);
let typeface = FontMgr::new()
.match_family_style("monospace", skia_safe::FontStyle::normal())
.context("no monospace typeface via fontconfig")?;
let typeface = crate::library_ui::match_first_family(
&FontMgr::new(),
&["monospace", "Consolas", "Cascadia Mono", "Courier New"],
skia_safe::FontStyle::normal(),
)
.context("no monospace typeface (fontconfig alias or system family)")?;
self.font = Some(Font::new(typeface, 14.0));
self.fonts = Some(build_fonts()?);
crates/pf-ffvk/Cargo.toml
+1 -1
View File
@@ -13,7 +13,7 @@ repository.workspace = true
# FF_API_VULKAN_* deprecation gates that change AVVulkanDeviceContext's layout between
# FFmpeg builds. This is deliberately not hand-transcribed.
[target.'cfg(target_os = "linux")'.dependencies]
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
ash = { version = "0.38", features = ["loaded"] }
[build-dependencies]
crates/pf-ffvk/build.rs
+81 -28
View File
@@ -7,41 +7,33 @@
//! libavutil/version.h, so bindgen over the installed header is the only ABI-safe
//! source of truth — hand transcription would silently skew on the next FFmpeg bump.
//!
//! Non-Linux targets get an empty file: the workspace builds on macOS (clients/apple is
//! the client there), and this shim is Linux-client plumbing only.
//! Header discovery is per-OS: Linux asks pkg-config; Windows reuses the FFMPEG_DIR
//! tree ffmpeg-sys-next links against (BtbN trees ship no .pc files) plus an explicit
//! Vulkan-Headers include dir, since Windows has no system <vulkan/vulkan.h>. Other
//! targets get an empty file: the workspace builds on macOS (clients/apple is the
//! client there).
use std::env;
use std::path::PathBuf;
fn main() {
println!("cargo:rerun-if-changed=wrapper.h");
println!("cargo:rerun-if-env-changed=PF_FFVK_VULKAN_INCLUDE");
let out = PathBuf::from(env::var("OUT_DIR").unwrap()).join("bindings.rs");
if env::var("CARGO_CFG_TARGET_OS").as_deref() != Ok("linux") {
std::fs::write(&out, "// pf-ffvk: Linux-only, empty on this target\n").unwrap();
return;
}
// Include paths from pkg-config (libavutil for the hwcontext header; the Vulkan
// headers usually live in /usr/include, but honor a registered vulkan.pc too).
// PF_FFVK_VULKAN_INCLUDE prepends an explicit Vulkan-Headers include dir — for
// cross builds and boxes without the system package.
println!("cargo:rerun-if-env-changed=PF_FFVK_VULKAN_INCLUDE");
let mut includes: Vec<PathBuf> = Vec::new();
if let Ok(dir) = env::var("PF_FFVK_VULKAN_INCLUDE") {
includes.push(PathBuf::from(dir));
}
let avutil = pkg_config::Config::new()
.cargo_metadata(false)
.probe("libavutil")
.expect("pkg-config: libavutil not found — install the FFmpeg dev package");
includes.extend(avutil.include_paths);
if let Ok(vk) = pkg_config::Config::new()
.cargo_metadata(false)
.probe("vulkan")
{
includes.extend(vk.include_paths);
}
let target_os = env::var("CARGO_CFG_TARGET_OS").unwrap_or_default();
let includes = match target_os.as_str() {
"linux" => linux_includes(),
"windows" => windows_includes(),
_ => {
std::fs::write(
&out,
"// pf-ffvk: Linux/Windows-only, empty on this target\n",
)
.unwrap();
return;
}
};
let mut builder = bindgen::Builder::default()
.header("wrapper.h")
@@ -81,6 +73,67 @@ fn main() {
// The av_vk_* symbols live in libavutil, which ffmpeg-sys-next already links into
// every consumer of this crate; no extra link flags needed. Emitting the lib anyway
// keeps `cargo test -p pf-ffvk` linking standalone.
// keeps `cargo test -p pf-ffvk` linking standalone — which on Windows also needs the
// import-lib search path (there is no system linker path for FFmpeg there).
if target_os == "windows" {
// windows_includes() already required FFMPEG_DIR.
let ff = PathBuf::from(env::var("FFMPEG_DIR").unwrap());
println!(
"cargo:rustc-link-search=native={}",
ff.join("lib").display()
);
}
println!("cargo:rustc-link-lib=avutil");
}
/// Include paths from pkg-config (libavutil for the hwcontext header; the Vulkan
/// headers usually live in /usr/include, but honor a registered vulkan.pc too).
/// PF_FFVK_VULKAN_INCLUDE prepends an explicit Vulkan-Headers include dir — for
/// cross builds and boxes without the system package.
fn linux_includes() -> Vec<PathBuf> {
let mut includes: Vec<PathBuf> = Vec::new();
if let Ok(dir) = env::var("PF_FFVK_VULKAN_INCLUDE") {
includes.push(PathBuf::from(dir));
}
let avutil = pkg_config::Config::new()
.cargo_metadata(false)
.probe("libavutil")
.expect("pkg-config: libavutil not found — install the FFmpeg dev package");
includes.extend(avutil.include_paths);
if let Ok(vk) = pkg_config::Config::new()
.cargo_metadata(false)
.probe("vulkan")
{
includes.extend(vk.include_paths);
}
includes
}
/// No pkg-config on Windows: headers come from the FFMPEG_DIR tree (the same BtbN
/// lgpl-shared tree ffmpeg-sys-next links against) plus an explicit Vulkan-Headers
/// dir — PF_FFVK_VULKAN_INCLUDE (provision-windows-punktfunk-extras.ps1 stages
/// C:\Users\Public\vulkan-headers) or an installed Vulkan SDK. Only headers are
/// needed at build time; the loader (vulkan-1.dll) is a GPU-driver component and is
/// never linked here.
fn windows_includes() -> Vec<PathBuf> {
println!("cargo:rerun-if-env-changed=FFMPEG_DIR");
println!("cargo:rerun-if-env-changed=VULKAN_SDK");
let mut includes: Vec<PathBuf> = Vec::new();
if let Ok(dir) = env::var("PF_FFVK_VULKAN_INCLUDE") {
includes.push(PathBuf::from(dir));
} else if let Ok(sdk) = env::var("VULKAN_SDK") {
includes.push(PathBuf::from(sdk).join("Include"));
} else {
panic!(
"pf-ffvk: no Vulkan headers — set PF_FFVK_VULKAN_INCLUDE to a Vulkan-Headers \
include dir (scripts/ci/provision-windows-punktfunk-extras.ps1 stages \
C:\\Users\\Public\\vulkan-headers\\include) or install the Vulkan SDK (VULKAN_SDK)"
);
}
let ff = env::var("FFMPEG_DIR").expect(
"pf-ffvk: FFMPEG_DIR not set — point it at the FFmpeg tree \
(scripts/ci/provision-windows-punktfunk-extras.ps1 stages C:\\Users\\Public\\ffmpeg)",
);
includes.push(PathBuf::from(ff).join("include"));
includes
}
crates/pf-ffvk/src/lib.rs
+6 -3
View File
@@ -14,11 +14,11 @@
#![allow(deref_nullptr)]
#![allow(unnecessary_transmutes)]
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
include!(concat!(env!("OUT_DIR"), "/bindings.rs"));
/// Conversions between the generated vulkan.h handle types and ash's.
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod ashx {
use super::*;
use ash::vk::Handle as _;
@@ -33,6 +33,9 @@ pub mod ashx {
ash::vk::Semaphore::from_raw(h as u64)
}
// bindgen's enum repr is target-dependent: u32 on Linux (clang default), i32 on
// MSVC — so the cast is required on one target and a same-type no-op on the other.
#[allow(clippy::unnecessary_cast)]
pub fn image_layout(l: VkImageLayout) -> ash::vk::ImageLayout {
ash::vk::ImageLayout::from_raw(l as i32)
}
@@ -64,7 +67,7 @@ pub mod ashx {
}
}
#[cfg(all(test, target_os = "linux"))]
#[cfg(all(test, any(target_os = "linux", windows)))]
mod tests {
use super::*;
crates/pf-presenter/Cargo.toml
+14 -5
View File
@@ -8,19 +8,28 @@ license.workspace = true
authors.workspace = true
repository.workspace = true
# Same Linux gating as the rest of the client stack.
[target.'cfg(target_os = "linux")'.dependencies]
# Same Linux+Windows gating as the rest of the client stack (dmabuf import is the one
# Linux-only module — see lib.rs).
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
pf-client-core = { path = "../pf-client-core" }
# AVVkFrame access (Vulkan Video frames: live sync state under the frames lock).
pf-ffvk = { path = "../pf-ffvk" }
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
# `loaded` dlopens libvulkan at runtime (no link-time dependency — GPU-less boxes still
# start and fail into a clean error). `ash` on sdl3 types SDL_Vulkan_CreateSurface with
# ash 0.38 handles so the surface hands over without transmutes.
# start and fail into a clean error; on Windows vulkan-1.dll is a GPU-driver component).
# `ash` on sdl3 types SDL_Vulkan_CreateSurface with ash 0.38 handles so the surface
# hands over without transmutes.
ash = { version = "0.38", features = ["loaded"] }
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
async-channel = "2"
anyhow = "1"
tracing = "0.1"
# Linux links the system SDL3; Windows builds it from source (same choice as the rest
# of the workspace's Windows SDL consumers).
[target.'cfg(target_os = "linux")'.dependencies]
sdl3 = { version = "0.18", features = ["hidapi", "ash"] }
[target.'cfg(windows)'.dependencies]
sdl3 = { version = "0.18", features = ["hidapi", "ash", "build-from-source"] }
crates/pf-presenter/src/keymap_sdl.rs
+3 -1
View File
@@ -155,7 +155,9 @@ pub fn mouse_button_to_gs(b: sdl3::mouse::MouseButton) -> Option<u32> {
})
}
#[cfg(test)]
// Linux-only: the reference table it cross-checks (pf_client_core::keymap, evdev-keyed)
// only exists there. The SDL table under test is itself cross-platform.
#[cfg(all(test, target_os = "linux"))]
mod tests {
use super::*;
use pf_client_core::keymap::evdev_to_vk;
crates/pf-presenter/src/lib.rs
+16 -12
View File
@@ -3,26 +3,30 @@
//! decoded frames, captures input on the `ui_stream` state-machine contract, and reports
//! the unified stats window on stdout. No UI toolkit anywhere in the dependency tree.
//!
//! Two frame paths: software (`CpuFrame` RGBA staging upload) and hardware (the
//! decoder's NV12 dmabuf imported per-plane into Vulkan + the CICP-driven CSC pass —
//! `dmabuf.rs`/`csc.rs`), both composited by a letterboxed blit. Devices without the
//! import extensions, and any import/present failure streak, demote the decoder to
//! software via the session pump's `force_software` contract, same as the GTK presenter.
//! Three frame paths: software (`CpuFrame` RGBA staging upload), Vulkan Video (the
//! decoder's VkImage on THIS device — plane views + the CICP-driven CSC pass), and on
//! Linux additionally VAAPI hardware (NV12 dmabuf imported per-plane — `dmabuf.rs`),
//! all composited by a letterboxed blit. Devices without the import extensions, and any
//! import/present failure streak, demote the decoder to software via the session pump's
//! `force_software` contract, same as the GTK presenter.
//!
//! Builds on Linux AND Windows; `dmabuf` is the one Linux-only module (DRM-PRIME does
//! not exist on Windows — the decode chain there is Vulkan → software).
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod csc;
#[cfg(target_os = "linux")]
pub mod dmabuf;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod input;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod keymap_sdl;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod overlay;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
mod run;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod vk;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub use run::{run_browse, run_session, ActionOutcome, Outcome, SessionOpts};
crates/pf-presenter/src/run.rs
+82 -32
View File
@@ -124,10 +124,18 @@ enum ModeCtl<'a> {
Browse(OnAction<'a>),
}
/// The custom SDL event a decoded frame's arrival pushes (see [`StreamState::new`]):
/// pure wake-up — the loop drains the frame channel regardless of why it woke.
struct FrameWake;
/// Everything one stream session accumulates — created at session start, dropped at
/// session end (browse mode cycles through several per process lifetime).
struct StreamState {
handle: SessionHandle,
/// Decoded frames, re-queued by the wake forwarder (newest-wins, like the pump's
/// own queue). The loop drains THIS, never `handle.frames` — the forwarder is that
/// channel's one consumer.
frames: async_channel::Receiver<DecodedFrame>,
connector: Option<Arc<NativeClient>>,
capture: Option<Capture>,
force_software: Arc<AtomicBool>,
@@ -150,9 +158,31 @@ struct StreamState {
}
impl StreamState {
fn new(params: SessionParams, force_software: Arc<AtomicBool>) -> StreamState {
/// `wake`: pushes a [`FrameWake`] SDL event as each decoded frame lands, via a tiny
/// forwarder thread that owns the pump's frame channel. This is what lets the run
/// loop BLOCK in `wait_event_timeout` (instead of a 1 ms poll — measured as a full
/// core burned at any frame rate) yet still present a frame the instant it arrives:
/// input events and frames both wake the same wait. The forwarder exits when the
/// pump drops its sender (session end/shutdown).
fn new(
params: SessionParams,
force_software: Arc<AtomicBool>,
wake: sdl3::event::EventSender,
) -> StreamState {
let handle = session::start(params);
let (wake_tx, wake_rx) = async_channel::bounded(2);
let pump_rx = handle.frames.clone();
let _ = std::thread::Builder::new()
.name("pf-frame-wake".into())
.spawn(move || {
while let Ok(f) = pump_rx.recv_blocking() {
let _ = wake_tx.force_send(f); // newest wins, like the pump's queue
let _ = wake.push_custom_event(FrameWake);
}
});
StreamState {
handle: session::start(params),
handle,
frames: wake_rx,
connector: None,
capture: None,
force_software,
@@ -199,6 +229,10 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
sdl3::hint::set("SDL_JOYSTICK_THREAD", "1");
let sdl = sdl3::init().context("SDL init")?;
let video = sdl.video().context("SDL video")?;
let events = sdl.event().context("SDL events")?;
events
.register_custom_event::<FrameWake>()
.map_err(|e| anyhow::anyhow!("register FrameWake event: {e}"))?;
let mut window = {
let mut b = video.window(&opts.window_title, 1280, 720);
b.position_centered().resizable().vulkan();
@@ -262,7 +296,11 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
force_software.clone(),
presenter.vulkan_decode(),
);
Some(StreamState::new(params, force_software))
Some(StreamState::new(
params,
force_software,
events.event_sender(),
))
}
ModeCtl::Browse(_) => None,
};
@@ -278,11 +316,12 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
let outcome = 'main: loop {
// --- SDL events (input, window, gamepads) ---------------------------------------
// Block briefly in SDL's own wait so idle costs nothing; while streaming, frames
// arrive on the channel below and 1 ms bounds the added present latency. In
// browse-idle the per-iteration FIFO present vsync-throttles the loop anyway.
let streaming = stream.as_ref().is_some_and(|s| s.connector.is_some());
let timeout = Duration::from_millis(if streaming { 1 } else { 5 });
// Block in SDL's own wait: input/window events AND decoded frames (the wake
// forwarder's FrameWake) all land in this one queue, so the loop wakes exactly
// when there is work — a short-timeout poll here burned a full core (measured;
// the timeout only bounds stop-flag/pump-tick latency now). In browse-idle the
// per-iteration FIFO present vsync-throttles the loop anyway.
let timeout = Duration::from_millis(15);
let first = event_pump.wait_event_timeout(timeout);
let mut queued: Vec<Event> = Vec::new();
if let Some(e) = first {
@@ -311,8 +350,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
WindowEvent::FocusLost => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(false) {
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
tracing::info!("focus lost — input released");
}
}
@@ -323,8 +361,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.should_reengage() {
cap.engage();
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
apply_capture(&mut window, &mouse, true);
tracing::info!("focus gained — input recaptured");
}
}
@@ -352,12 +389,10 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.captured() {
cap.release(true);
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
} else {
cap.engage();
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
apply_capture(&mut window, &mouse, true);
}
tracing::info!(captured = cap.captured(), "chord: release/engage");
}
@@ -367,8 +402,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = &mut stream {
tracing::info!("chord: disconnect");
st.request_quit();
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
// The pump emits Ended(None); the end path routes per mode.
}
continue;
@@ -410,8 +444,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if !cap.captured() {
// The engaging click is suppressed toward the host.
cap.engage();
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
apply_capture(&mut window, &mouse, true);
} else {
cap.on_button_down(mouse_btn);
}
@@ -427,6 +460,9 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
cap.on_wheel(x, y);
}
}
// The wake forwarder's FrameWake (and any other user event): pure
// wake-up — the frame drain below runs this iteration either way.
Event::User { .. } => {}
// Everything else (gamepad add/remove/button/axis/touchpad/sensor…) is
// the pumped gamepad worker's — it ignores what it doesn't know.
other => pump.handle_event(other),
@@ -445,8 +481,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
while escape_rx.try_recv().is_ok() {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(true) {
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
}
}
if fullscreen && !opts.fullscreen {
@@ -459,8 +494,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = &mut stream {
tracing::info!("controller chord: disconnect");
st.request_quit();
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
}
}
@@ -486,7 +520,11 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
) {
ActionOutcome::Handled => {}
ActionOutcome::Start(params) => {
stream = Some(StreamState::new(*params, force_software));
stream = Some(StreamState::new(
*params,
force_software,
events.event_sender(),
));
if let Some(o) = overlay.as_mut() {
o.session_phase(SessionPhase::Connecting);
}
@@ -518,8 +556,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
st.clock_offset_ns = c.clock_offset_ns;
let mut cap = Capture::new(c.clone());
cap.engage(); // capture engages when the stream starts (ui_stream parity)
mouse.set_relative_mouse_mode(&window, true);
mouse.show_cursor(false);
apply_capture(&mut window, &mouse, true);
st.capture = Some(cap);
st.connector = Some(c);
if let Some(f) = opts.on_connected.as_mut() {
@@ -556,8 +593,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = stream.take() {
st.shutdown();
}
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
if let Some(o) = overlay.as_mut() {
o.session_phase(SessionPhase::Failed(&msg));
}
@@ -569,8 +605,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = &mut st.capture {
cap.release(true);
}
mouse.set_relative_mouse_mode(&window, false);
mouse.show_cursor(true);
apply_capture(&mut window, &mouse, false);
match &mode {
ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)),
ModeCtl::Browse(_) => {
@@ -641,7 +676,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
}
}
let mut newest: Option<DecodedFrame> = None;
while let Ok(f) = st.handle.frames.try_recv() {
while let Ok(f) = st.frames.try_recv() {
newest = Some(f);
}
if let Some(f) = newest {
@@ -655,6 +690,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
st.hdr = c.color.is_pq();
presenter.present(&window, FrameInput::Cpu(&c), overlay_frame.as_ref())?
}
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(d)
if presenter.supports_dmabuf() && !st.dmabuf_demoted =>
{
@@ -685,6 +721,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
}
}
}
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(_) => {
// No import extensions on this device (or already demoted) — the
// pump rebuilds the decoder as software; frames flow again soon.
@@ -782,6 +819,19 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
Ok(outcome)
}
/// Apply the capture state to the window: pointer lock (relative mouse + hidden cursor)
/// and — on Windows — a keyboard grab, so system chords (Alt+Tab, the Windows key) reach
/// the host while captured instead of the local shell. SDL implements the grab there
/// with a low-level keyboard hook, the same mechanism the WinUI shell's in-process
/// client used its own WH_KEYBOARD_LL hooks for. Not engaged on Linux: the compositor
/// shortcut-inhibit story stays the shells' concern (Settings.inhibit_shortcuts).
fn apply_capture(window: &mut sdl3::video::Window, mouse: &sdl3::mouse::MouseUtil, on: bool) {
mouse.set_relative_mouse_mode(window, on);
mouse.show_cursor(!on);
#[cfg(windows)]
window.set_keyboard_grab(on);
}
/// The presenter's share of the unified stats window — folded into each printed line.
#[derive(Default)]
struct PresentedWindow {
crates/pf-presenter/src/vk.rs
+71 -9
View File
@@ -17,12 +17,15 @@
//! (expose/resize redraws).
use crate::csc::{build_fullscreen_pipeline, csc_rows, CscPass};
#[cfg(target_os = "linux")]
use crate::dmabuf::{self, HwFrame};
use crate::overlay::{OverlayFrame, SharedDevice};
use anyhow::{anyhow, bail, Context as _, Result};
use ash::vk;
use ash::vk::Handle as _;
use pf_client_core::video::{CpuFrame, DmabufFrame, VkVideoFrame};
#[cfg(target_os = "linux")]
use pf_client_core::video::DmabufFrame;
use pf_client_core::video::{CpuFrame, VkVideoFrame};
use std::ffi::CString;
/// One presenter iteration's video input.
@@ -30,12 +33,14 @@ pub enum FrameInput<'a> {
/// No new frame — re-composite the retained video image (expose/resize).
Redraw,
Cpu(&'a CpuFrame),
#[cfg(target_os = "linux")]
Dmabuf(DmabufFrame),
/// FFmpeg Vulkan Video output — a VkImage already on THIS device (zero copy).
VkFrame(VkVideoFrame),
}
/// The dmabuf/CSC machinery, present only when the device carries the import extensions.
#[cfg(target_os = "linux")]
struct HwCtx {
ext_mem_fd: ash::khr::external_memory_fd::Device,
}
@@ -44,6 +49,7 @@ struct HwCtx {
/// done: imported dmabuf planes, or a Vulkan-Video frame (FFmpeg's image — we own only
/// the plane views; dropping the frame's guard releases the AVFrame back to the pool).
enum Retired {
#[cfg(target_os = "linux")]
Dmabuf(HwFrame),
Vk {
frame: VkVideoFrame,
@@ -54,6 +60,7 @@ enum Retired {
impl Retired {
fn destroy(self, device: &ash::Device) {
match self {
#[cfg(target_os = "linux")]
Retired::Dmabuf(f) => f.destroy(device),
Retired::Vk { frame, views } => {
unsafe {
@@ -324,6 +331,7 @@ pub struct Presenter {
qfi: u32,
/// Dmabuf import — `None` when the device lacks the import extensions (the CSC
/// pass itself is unconditional: Vulkan-Video frames need it everywhere).
#[cfg(target_os = "linux")]
hw: Option<HwCtx>,
csc: CscPass,
/// FFmpeg Vulkan Video decode handles — `None` when the stack can't do it.
@@ -423,15 +431,18 @@ impl Presenter {
}
// The dmabuf import set is optional: enabled when the device offers all four,
// else that path is off (`supports_dmabuf() == false`).
// else that path is off (`supports_dmabuf() == false`). Windows has no
// dmabuf/DRM-PRIME — the whole import path is compiled out there.
let available = unsafe { instance.enumerate_device_extension_properties(pdev) }?;
let has = |name: &std::ffi::CStr| {
available
.iter()
.any(|e| e.extension_name_as_c_str() == Ok(name))
};
#[cfg(target_os = "linux")]
let hw_capable = dmabuf::DEVICE_EXTENSIONS.iter().all(|n| has(n));
let mut dev_exts = vec![ash::khr::swapchain::NAME.as_ptr()];
#[cfg(target_os = "linux")]
if hw_capable {
dev_exts.extend(dmabuf::DEVICE_EXTENSIONS.iter().map(|n| n.as_ptr()));
} else {
@@ -577,6 +588,7 @@ impl Presenter {
let hdr_metadata_d =
has_hdr_metadata.then(|| ash::ext::hdr_metadata::Device::new(&instance, &device));
let queue = unsafe { device.get_device_queue(qfi, 0) };
#[cfg(target_os = "linux")]
let hw = if hw_capable {
Some(HwCtx {
ext_mem_fd: ash::khr::external_memory_fd::Device::new(&instance, &device),
@@ -593,6 +605,7 @@ impl Presenter {
let qf_props = unsafe { instance.get_physical_device_queue_family_properties(pdev) };
let mut device_extensions: Vec<CString> =
vec![CString::from(ash::khr::swapchain::NAME)];
#[cfg(target_os = "linux")]
if hw_capable {
device_extensions
.extend(dmabuf::DEVICE_EXTENSIONS.iter().map(|n| CString::from(*n)));
@@ -670,6 +683,7 @@ impl Presenter {
swap_d,
queue,
qfi,
#[cfg(target_os = "linux")]
hw,
csc,
video_export,
@@ -862,6 +876,7 @@ impl Presenter {
/// Whether the hardware (dmabuf) path exists on this device — callers keep the
/// decoder on software when it doesn't.
#[cfg(target_os = "linux")]
pub fn supports_dmabuf(&self) -> bool {
self.hw.is_some()
}
@@ -962,6 +977,7 @@ impl Presenter {
let frame_pq = match &input {
FrameInput::Redraw => None,
FrameInput::Cpu(f) => Some(f.color.is_pq()),
#[cfg(target_os = "linux")]
FrameInput::Dmabuf(d) => Some(d.color.is_pq()),
FrameInput::VkFrame(v) => Some(v.color.is_pq()),
};
@@ -974,11 +990,13 @@ impl Presenter {
// Hardware frames prepare before anything touches the queue: an import/view the
// driver rejects must fail out here, before this present consumed the acquire
// semaphore.
#[cfg(target_os = "linux")]
let mut hw_frame: Option<HwFrame> = None;
let mut vk_frame: Option<(VkVideoFrame, [vk::ImageView; 2])> = None;
let cpu_frame = match input {
FrameInput::Redraw => None,
FrameInput::Cpu(f) => Some(f),
#[cfg(target_os = "linux")]
FrameInput::Dmabuf(d) => {
let hw = self
.hw
@@ -1015,6 +1033,7 @@ impl Presenter {
if let Some(f) = cpu_frame {
self.stage_frame(f)?;
}
#[cfg(target_os = "linux")]
if let Some(f) = &hw_frame {
if self
.video
@@ -1063,6 +1082,7 @@ impl Presenter {
Ok(r) => r,
Err(vk::Result::ERROR_OUT_OF_DATE_KHR) => {
// Never submitted — the import (if any) dies here, GPU never saw it.
#[cfg(target_os = "linux")]
if let Some(f) = hw_frame {
f.destroy(&self.device);
}
@@ -1083,6 +1103,7 @@ impl Presenter {
// Dmabuf frame: acquire the foreign planes, then the CSC pass renders
// NV12→RGBA into the video image (render pass ends it in TRANSFER_SRC for
// the blit below).
#[cfg(target_os = "linux")]
if let (Some(f), Some(v)) = (&hw_frame, &self.video) {
for view_image in [f.luma_image(), f.chroma_image()] {
foreign_acquire_barrier(&self.device, self.cmd_buf, view_image, self.qfi);
@@ -1328,12 +1349,15 @@ impl Presenter {
submitted?;
self.submitted = true;
// The hw frame is on the GPU now — park it until the fence proves the reads
// done (destroyed at the next present's fence wait, or in Drop).
self.retired_hw = hw_frame.take().map(Retired::Dmabuf).or_else(|| {
vk_frame
.take()
.map(|(frame, views)| Retired::Vk { frame, views })
});
// done (destroyed at the next present's fence wait, or in Drop). At most one
// of hw_frame/vk_frame is set (they route from the same `input`).
self.retired_hw = vk_frame
.take()
.map(|(frame, views)| Retired::Vk { frame, views });
#[cfg(target_os = "linux")]
if let Some(f) = hw_frame.take() {
self.retired_hw = Some(Retired::Dmabuf(f));
}
let swapchains = [self.swapchain];
let indices = [index];
@@ -1688,6 +1712,7 @@ impl Drop for Presenter {
let device = self.device.clone();
g.destroy(&device, &self.swap_d);
}
#[cfg(target_os = "linux")]
self.hw.take();
self.csc.destroy(&self.device);
self.overlay_pipe.destroy(&self.device);
@@ -1720,10 +1745,43 @@ fn pick_device(
let forced: Option<usize> = std::env::var("PUNKTFUNK_VK_DEVICE")
.ok()
.and_then(|v| v.parse().ok());
let candidates: Vec<vk::PhysicalDevice> = match forced {
let mut candidates: Vec<vk::PhysicalDevice> = match forced {
Some(i) => devices.get(i).copied().into_iter().collect(),
None => devices,
};
// Rank the candidates (stable sort; the index override wins outright):
// 1. The Settings GPU pick — `PUNKTFUNK_VK_ADAPTER` carries the adapter's marketing
// name (the WinUI shell's picker stores DXGI's, which matches Vulkan's for the
// same GPU): exact match, then substring, plain order when nothing matches
// (eGPU unplugged, stale setting).
// 2. Discrete over integrated: enumeration order puts the iGPU FIRST on some
// hybrids (observed: Ryzen iGPU ahead of an RTX dGPU), and the iGPU's video
// engine is the far weaker decoder — first-enumerated was a silent footgun.
if forced.is_none() {
let want = std::env::var("PUNKTFUNK_VK_ADAPTER")
.ok()
.map(|w| w.trim().to_lowercase())
.filter(|w| !w.is_empty());
candidates.sort_by_key(|d| {
let props = unsafe { instance.get_physical_device_properties(*d) };
let name = props
.device_name_as_c_str()
.map(|c| c.to_string_lossy().to_lowercase())
.unwrap_or_default();
let name_rank = match &want {
Some(w) if name == *w => 0,
Some(w) if name.contains(w.as_str()) || w.contains(&name) => 1,
Some(_) => 2,
None => 0,
};
let type_rank = match props.device_type {
vk::PhysicalDeviceType::DISCRETE_GPU => 0,
vk::PhysicalDeviceType::INTEGRATED_GPU => 1,
_ => 2,
};
(name_rank, type_rank)
});
}
for pdev in candidates {
let families = unsafe { instance.get_physical_device_queue_family_properties(pdev) };
for (i, f) in families.iter().enumerate() {
@@ -1855,6 +1913,9 @@ struct VkFrameSync {
/// Lock the frame and read its live sync state (the presenter's submit must wait
/// `sem_value` and signal `sem_value + 1`). The lock is held until [`unlock_vkframe`].
// bindgen's enum repr is target-dependent (u32 Linux/clang, i32 MSVC) — the layout cast
// is required on one platform and a no-op on the other.
#[allow(clippy::unnecessary_cast)]
fn lock_vkframe(f: &VkVideoFrame) -> VkFrameSync {
unsafe {
let lock: unsafe extern "C" fn(*mut pf_ffvk::AVHWFramesContext, *mut pf_ffvk::AVVkFrame) =
@@ -1944,6 +2005,7 @@ fn vkframe_acquire_barrier(
/// Acquire a dmabuf plane image from its foreign owner (the VAAPI decoder): queue-family
/// transfer FOREIGN → ours, UNDEFINED → SHADER_READ_ONLY (content is preserved across
/// the transfer regardless of the UNDEFINED old-layout, per the external-memory rules).
#[cfg(target_os = "linux")]
fn foreign_acquire_barrier(
device: &ash::Device,
cmd: vk::CommandBuffer,