feat(clients/windows): port the Vulkan session client to Windows — session-always

The punktfunk-session Vulkan client (clients/linux-session, now clients/session)
builds and runs on Windows; the WinUI shell spawns it for every stream. Verified
live: 10-bit HEVC via Vulkan Video on both AMD (iGPU) and NVIDIA, 5120x1440 at
130 fps / 8 ms end-to-end on the RTX 4090.

- pf-ffvk: Windows bindgen branch (FFMPEG_DIR + PF_FFVK_VULKAN_INCLUDE, no
  pkg-config); provisioning fetches Vulkan-Headers (pinned v1.4.309).
- pf-client-core: builds on Windows — WASAPI audio (audio_wasapi.rs, cfg-swapped
  via #[path], same surface as the PipeWire twin), VAAPI/dmabuf gated inline
  (chain = vulkan -> software), trust reads the WinUI shell's %APPDATA% stores
  (parity tests pin both serialized shapes), Settings gains adapter/hdr_enabled
  (serde-defaulted; Linux stores unaffected).
- pf-presenter: builds on Windows — dmabuf module Linux-gated; SDL keyboard grab
  while captured (Alt+Tab/Win reach the host); pick_device ranks discrete over
  integrated (device 0 was the iGPU on hybrid boxes — the silent footgun) and
  honors PUNKTFUNK_VK_ADAPTER (the Settings GPU pick, exported by the session).
- run loop: block in one SDL wait woken by input AND decoded frames (a per-
  session forwarder pushes a FrameWake user event) instead of a 1 ms poll —
  measured 111%% -> 5%% of a core (NVIDIA), 86%% -> 3.5%% (AMD), stats unchanged.
  The pump's decode-fence wait became once-per-window sampling (no per-frame
  pipeline stall; the stat now shows true backlog).
- pf-console-ui: builds on Windows (skia-safe msvc prebuilts); font lookup falls
  through fontconfig aliases to concrete DirectWrite families (Consolas/Segoe UI)
  — browse/coverflow works, verified against a live host.
- WinUI shell: session-always via new src/spawn.rs (GTK spawn.rs port —
  CREATE_NO_WINDOW, stdout contract, kill handle); the Stream screen is a status
  card (chips + stage lines from the child's stats). The legacy in-process
  D3D11VA path stays behind Settings "Streaming engine" / PUNKTFUNK_BUILTIN_
  STREAM=1 as the A/B baseline until Phase 8 deletes it. SessionParams.video_caps
  makes the HDR toggle real.
- clients/linux-session renamed to clients/session (builds for both OSes).
- CI/MSIX: both workflows build/test both bins with widened path filters; the
  MSIX ships punktfunk-session.exe. ARM64 session builds --no-default-features
  (rust-skia has no aarch64-pc-windows-msvc prebuilts; flip when it does).

A/B on this box (5120x1440 HEVC vs home-worker-5): NVIDIA Vulkan 130 fps / 8 ms
e2e / 1.6 ms decode — clearly better than the built-in path. The AMD iGPU VCN
saturates at ~52 fps where its own D3D11VA does ~70 — Adrenalin Vulkan decode is
slower on APU silicon; discrete RDNA validation gates Phase 8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-08 23:21:36 +02:00
parent 838a1239cf
commit d6647b9183
37 changed files with 1447 additions and 195 deletions
+19 -11
View File
@@ -1,6 +1,6 @@
[package]
name = "pf-client-core"
description = "Shared Linux-client plumbing — session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads, trust store, discovery — extracted from the GTK client so the shell and the Vulkan session binary build on one implementation"
description = "Shared client plumbing (Linux + Windows) — session pump, FFmpeg decode, PipeWire/WASAPI audio, SDL3 gamepads, trust store, discovery — extracted from the GTK client so the shells and the Vulkan session binary build on one implementation"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
@@ -8,23 +8,20 @@ license.workspace = true
authors.workspace = true
repository.workspace = true
# Same Linux gating as clients/linux: `cargo build --workspace` stays green on macOS
# (the Mac client lives in clients/apple); elsewhere this crate is `wol` plus stubs-free
# emptiness. `wol` is pure std and stays cross-platform, matching the old main.rs.
[target.'cfg(target_os = "linux")'.dependencies]
# Linux + Windows: the Vulkan session client builds on both; `cargo build --workspace`
# stays green on macOS (the Mac client lives in clients/apple) — there this crate is
# `wol` plus stubs-free emptiness. `wol` is pure std and stays cross-platform, matching
# the old main.rs. Audio is the one per-OS swap: PipeWire on Linux, WASAPI on Windows
# (same public surface — see lib.rs).
[target.'cfg(any(target_os = "linux", windows))'.dependencies]
punktfunk-core = { path = "../punktfunk-core", features = ["quic"] }
# FFmpeg's Vulkan hwcontext surface (Vulkan Video decode on the presenter's device).
pf-ffvk = { path = "../pf-ffvk" }
async-channel = "2"
# Video decode (same FFmpeg pin as the host) and audio.
# Video decode (same FFmpeg pin as the host) and Opus for the audio planes.
ffmpeg-next = "8"
opus = "0.3"
pipewire = "0.9"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver).
sdl3 = { version = "0.18", features = ["hidapi"] }
mdns-sd = "0.20"
# Game-library fetch from the host's management API over mTLS + fingerprint pinning.
@@ -36,3 +33,14 @@ serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
tracing = "0.1"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver). Linux links the system SDL3; Windows builds it from source
# (no system SDL3 there — same choice as clients/windows).
[target.'cfg(target_os = "linux")'.dependencies]
pipewire = "0.9"
sdl3 = { version = "0.18", features = ["hidapi"] }
[target.'cfg(windows)'.dependencies]
wasapi = "0.23"
sdl3 = { version = "0.18", features = ["hidapi", "build-from-source"] }
+333
View File
@@ -0,0 +1,333 @@
//! Audio: playback (decoded PCM → a WASAPI shared-mode render stream) and the microphone
//! uplink (WASAPI capture → Opus → 0xCB datagrams, the inverse of the host's virtual mic).
//!
//! The WASAPI twin of `audio.rs` (PipeWire) — same public surface (`AudioPlayer::spawn`/
//! `take_buffer`/`push`, `MicStreamer::spawn`), swapped in by lib.rs's `#[path]` so the
//! session pump compiles against one `crate::audio` on both OSes. Adapted from
//! `clients/windows/src/audio.rs` (which remains the WinUI shell's own copy until its
//! built-in streaming path is deleted).
//!
//! Playback mirrors the host's virtual-mic producer's adaptive jitter buffer: the session
//! pump pushes 5 ms Opus-decoded chunks on the network clock; the WASAPI render thread
//! pulls whole event-driven quanta on the device clock. Prime to ~3 quanta before
//! producing, cap the ring so latency stays bounded, re-prime after a real drain.
//!
//! WASAPI objects are COM-apartment-bound and not `Send`, so they live on a dedicated
//! thread (the same discipline as the host's `wasapi_cap`); only the channels + stop flag
//! + join handle cross the boundary.
use anyhow::{anyhow, Context, Result};
use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{Receiver, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::Duration;
use wasapi::{DeviceEnumerator, Direction, SampleType, StreamMode, WaveFormat};
const SAMPLE_RATE: usize = 48_000;
/// The microphone uplink stays stereo (the host's virtual mic is stereo). The render path is
/// multichannel — its channel count + block align are runtime, driven by the host-resolved layout.
const CHANNELS: usize = 2;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
pub struct AudioPlayer {
pcm_tx: SyncSender<Vec<f32>>,
/// Drained chunk Vecs coming back from the render thread for reuse (the pool half of
/// the pcm channel — see [`AudioPlayer::take_buffer`]).
recycle_rx: Receiver<Vec<f32>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl AudioPlayer {
/// Spawn the WASAPI render thread for `channels` (2/6/8, canonical wire order
/// FL FR FC LFE RL RR SL SR). Failure (no render endpoint on this box) is survivable — the
/// caller streams video-only.
pub fn spawn(channels: u32) -> Result<AudioPlayer> {
// 64 × 5 ms = 320 ms of slack between the pump and the WASAPI loop.
let (pcm_tx, pcm_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
// Return path: the render thread sends each drained Vec back for reuse, so
// steady-state playback stops allocating (~200 chunks/s otherwise). Same capacity
// as the data channel; a full pool just drops the Vec (plain deallocation).
let (recycle_tx, recycle_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
let stop = Arc::new(AtomicBool::new(false));
let (ready_tx, ready_rx) = std::sync::mpsc::sync_channel::<Result<()>>(1);
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-audio".into())
.spawn(move || {
if let Err(e) = render_thread(pcm_rx, recycle_tx, stop_t, ready_tx, channels as u8)
{
tracing::warn!(error = format!("{e:#}"), "audio playback thread ended");
}
})
.context("spawn audio thread")?;
match ready_rx.recv_timeout(Duration::from_secs(3)) {
Ok(Ok(())) => {
tracing::info!(channels, "WASAPI render: 48 kHz f32 (default endpoint)");
Ok(AudioPlayer {
pcm_tx,
recycle_rx,
stop,
thread: Some(thread),
})
}
Ok(Err(e)) => Err(e),
Err(_) => Err(anyhow!(
"wasapi render init timed out (no render endpoint?)"
)),
}
}
/// A recycled chunk Vec from the pool, empty but with its capacity intact — fill it
/// and hand it back through [`push`](Self::push). Allocates only when the pool is dry
/// (startup, or after the WASAPI side dropped chunks).
pub fn take_buffer(&self) -> Vec<f32> {
self.recycle_rx.try_recv().unwrap_or_default()
}
/// Queue one interleaved f32 chunk (in the session's channel layout). Drops the chunk if the
/// WASAPI side is wedged (the renderer conceals the gap; never block the session pump).
pub fn push(&self, pcm: Vec<f32>) {
if let Err(TrySendError::Disconnected(_)) = self.pcm_tx.try_send(pcm) {
// Thread already dead — Drop will reap it; nothing to do per-chunk.
}
}
}
impl Drop for AudioPlayer {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
fn render_thread(
pcm_rx: Receiver<Vec<f32>>,
recycle_tx: SyncSender<Vec<f32>>,
stop: Arc<AtomicBool>,
ready: SyncSender<Result<()>>,
channels: u8,
) -> Result<()> {
if let Err(e) = wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")
{
let _ = ready.send(Err(e));
return Ok(());
}
let res = (|| -> Result<()> {
// F32LE interleaved: channels × 4 bytes/sample. Stereo (channels == 2) is byte-identical
// to the old fixed path (mask 0x3, block align 8).
let block_align = channels as usize * 4;
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Render)
.context("default render endpoint")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
// The explicit dwChannelMask is the wire order (FL FR FC LFE RL RR SL SR); 5.1 = 0x3F,
// 7.1 = 0x63F. WASAPI delivers channels in ascending mask-bit order, which equals the wire
// order, so the render mapping is the identity — no permute. `autoconvert` (below) lets the
// audio engine downmix when the endpoint has fewer speakers.
let desired = WaveFormat::new(
32,
32,
&SampleType::Float,
SAMPLE_RATE,
channels as usize,
Some(punktfunk_core::audio::wasapi_channel_mask(channels)),
);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
autoconvert: true,
buffer_duration_hns: default_period,
};
audio_client
.initialize_client(&desired, &Direction::Render, &mode)
.context("initialize render client")?;
let h_event = audio_client.set_get_eventhandle().context("event handle")?;
let render_client = audio_client
.get_audiorenderclient()
.context("IAudioRenderClient")?;
audio_client.start_stream().context("start render stream")?;
let _ = ready.send(Ok(()));
// Adaptive jitter buffer, in f32-byte units (same shape as the host's virtual mic).
let mut ring: VecDeque<u8> = VecDeque::new();
let mut primed = false;
let mut out = Vec::new(); // per-quantum scratch, reused across iterations
while !stop.load(Ordering::Relaxed) {
if h_event.wait_for_event(100).is_err() {
continue;
}
// Drain everything the pump has queued into the ring, returning each drained
// Vec to the pool (a full/closed pool drops it).
while let Ok(mut chunk) = pcm_rx.try_recv() {
for s in chunk.iter() {
ring.extend(s.to_le_bytes());
}
chunk.clear();
let _ = recycle_tx.try_send(chunk);
}
let avail_frames = audio_client
.get_available_space_in_frames()
.context("available space")? as usize;
if avail_frames == 0 {
continue;
}
let want_bytes = avail_frames * block_align;
// Prime to ~3 quanta; cap at ~1 quantum of slack beyond that; re-prime on drain.
let target = (3 * want_bytes).clamp(720 * block_align, 9600 * block_align);
let cap = target.max(want_bytes) + want_bytes;
if ring.len() > cap {
ring.drain(..ring.len() - cap);
}
if !primed && ring.len() >= target {
primed = true;
}
out.clear();
out.resize(want_bytes, 0);
if primed {
let n = ring.len().min(want_bytes);
for (dst, b) in out.iter_mut().zip(ring.drain(..n)) {
*dst = b;
}
}
if ring.is_empty() {
primed = false;
}
render_client
.write_to_device(avail_frames, &out, None)
.context("write_to_device")?;
}
audio_client.stop_stream().ok();
Ok(())
})();
if let Err(ref e) = res {
let _ = ready.send(Err(anyhow!("{e:#}")));
}
res
}
/// The microphone uplink: capture the default input device, Opus-encode 20 ms chunks, ship
/// them as 0xCB datagrams into the host's virtual mic source.
pub struct MicStreamer {
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl MicStreamer {
pub fn spawn(connector: Arc<NativeClient>) -> Result<MicStreamer> {
let stop = Arc::new(AtomicBool::new(false));
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, stop_t) {
tracing::warn!(error = format!("{e:#}"), "mic uplink thread ended");
}
})
.context("spawn mic thread")?;
Ok(MicStreamer {
stop,
thread: Some(thread),
})
}
}
impl Drop for MicStreamer {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
fn mic_thread(connector: &Arc<NativeClient>, stop: Arc<AtomicBool>) -> Result<()> {
wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")?;
let mut encoder = opus::Encoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
opus::Application::Voip,
)
.map_err(|e| anyhow!("opus encoder: {e}"))?;
let _ = encoder.set_bitrate(opus::Bitrate::Bits(64_000));
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Capture)
.context("default capture endpoint (no microphone?)")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CHANNELS, None);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
autoconvert: true,
buffer_duration_hns: default_period,
};
audio_client
.initialize_client(&desired, &Direction::Capture, &mode)
.context("initialize capture client")?;
let h_event = audio_client.set_get_eventhandle().context("event handle")?;
let capture_client = audio_client
.get_audiocaptureclient()
.context("IAudioCaptureClient")?;
audio_client
.start_stream()
.context("start capture stream")?;
let mut bytes: VecDeque<u8> = VecDeque::new();
let mut ring: VecDeque<f32> = VecDeque::new();
let mut out = vec![0u8; 4000];
let mut seq = 0u32;
while !stop.load(Ordering::Relaxed) {
if h_event.wait_for_event(100).is_err() {
continue;
}
loop {
match capture_client.get_next_packet_size() {
Ok(Some(0)) | Ok(None) => break,
Ok(Some(_n)) => {
capture_client
.read_from_device_to_deque(&mut bytes)
.context("read capture")?;
}
Err(e) => return Err(anyhow!("get_next_packet_size: {e}")),
}
}
let whole = (bytes.len() / 4) * 4;
for c in bytes.drain(..whole).collect::<Vec<u8>>().chunks_exact(4) {
ring.push_back(f32::from_le_bytes([c[0], c[1], c[2], c[3]]));
}
// Ship every complete 20 ms stereo frame.
while ring.len() >= MIC_FRAME * CHANNELS {
let pcm: Vec<f32> = ring.drain(..MIC_FRAME * CHANNELS).collect();
match encoder.encode_float(&pcm, &mut out) {
Ok(len) => {
let pts = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as u64)
.unwrap_or(0);
let _ = connector.send_mic(seq, pts, out[..len].to_vec());
seq = seq.wrapping_add(1);
}
Err(e) => tracing::debug!(error = %e, "opus mic encode"),
}
}
}
audio_client.stop_stream().ok();
Ok(())
}
+21 -10
View File
@@ -1,26 +1,37 @@
//! Shared, UI-agnostic Linux-client plumbing, extracted verbatim from the GTK client
//! Shared, UI-agnostic client plumbing, extracted verbatim from the GTK client
//! (design: punktfunk-planning `linux-client-rearchitecture.md`, Phase 0) so the desktop
//! shell and the Vulkan session binary build on one implementation.
//! shells and the Vulkan session binary build on one implementation — on Linux AND
//! Windows (the session binary runs on both; macOS stays `wol`-only, clients/apple is
//! the client there).
//!
//! Nothing here may depend on a UI toolkit: the presenter contract is `session`'s
//! channels (`SessionHandle`) and `video`'s `DecodedImage` (RGBA bytes or dmabuf fds +
//! plane layout) — how frames reach the screen is the consumer's business.
//! channels (`SessionHandle`) and `video`'s `DecodedImage` (RGBA bytes, dmabuf fds +
//! plane layout, or a decoded VkImage) — how frames reach the screen is the consumer's
//! business.
//!
//! Audio is the one per-OS module swap: `audio.rs` (PipeWire) on Linux,
//! `audio_wasapi.rs` (WASAPI) on Windows — same public surface, picked here by `#[path]`
//! so `crate::audio` is the only name the session pump ever sees. `keymap` (evdev-keyed)
//! stays Linux: the session path uses pf-presenter's SDL-scancode table instead.
#[cfg(target_os = "linux")]
pub mod audio;
#[cfg(target_os = "linux")]
#[cfg(windows)]
#[path = "audio_wasapi.rs"]
pub mod audio;
#[cfg(any(target_os = "linux", windows))]
pub mod discovery;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod gamepad;
#[cfg(target_os = "linux")]
pub mod keymap;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod library;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod session;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod trust;
#[cfg(target_os = "linux")]
#[cfg(any(target_os = "linux", windows))]
pub mod video;
pub mod wol;
+30 -17
View File
@@ -27,6 +27,12 @@ pub struct SessionParams {
/// The user's preferred video codec (a `quic::CODEC_*` bit, `0` = auto). Soft — the host honors
/// it when it can emit it, else falls back; the resolved codec drives the decoder.
pub preferred_codec: u8,
/// The advertised `quic::VIDEO_CAP_*` bits. Normally 10-bit + HDR (Main10/PQ: the
/// Vulkan presenter decodes P010 everywhere and presents PQ on an HDR10 swapchain
/// where the desktop offers one, tonemapping in the CSC shader where it doesn't;
/// the host still gates the upgrade behind its own PUNKTFUNK_10BIT policy) — `0`
/// when the user turned HDR off in Settings ("never send me 10-bit").
pub video_caps: u8,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see
@@ -211,11 +217,7 @@ fn pump(
params.compositor,
params.gamepad,
params.bitrate_kbps,
// 10-bit Main10 + PQ HDR10: the Vulkan presenter decodes P010 (Vulkan
// Video/VAAPI/software) and presents PQ on an HDR10 swapchain where the desktop
// offers one, tonemapping in the CSC shader where it doesn't. The host still
// gates the upgrade behind its own PUNKTFUNK_10BIT policy.
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR,
params.video_caps,
params.audio_channels,
crate::video::decodable_codecs(), // codecs FFmpeg can decode (HEVC/H.264/AV1)
params.preferred_codec, // the user's soft codec preference (0 = auto)
@@ -328,12 +330,14 @@ fn pump(
total_frames += 1;
dec_path = match &image {
DecodedImage::Cpu(_) => "software",
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(_) => "vaapi",
DecodedImage::VkFrame(_) => "vulkan",
};
if total_frames == 1 {
let (w, h, path) = match &image {
DecodedImage::Cpu(c) => (c.width, c.height, "software"),
#[cfg(target_os = "linux")]
DecodedImage::Dmabuf(d) => (d.width, d.height, "vaapi-dmabuf"),
DecodedImage::VkFrame(v) => (v.width, v.height, "vulkan-video"),
};
@@ -357,10 +361,16 @@ fn pump(
}
// Ship the frame FIRST, then settle the decode stat: on the
// Vulkan path receive_frame returns at SUBMISSION (~0.1 ms) and
// the hardware decodes asynchronously — waiting the frame's
// timeline fence here (after the presenter already has the
// frame) measures true received→decode-complete at zero
// pipeline cost. Software/VAAPI keep the synchronous stamp.
// the hardware decodes asynchronously — the frame's timeline
// fence measures true received→decode-complete. But the fence
// wait BLOCKS this thread, and per-frame that serializes the
// pipeline to 1/decode_latency (observed: an APU's 19 ms decode
// capping a 5120×1440 stream at ~51 fps while the engine could
// pipeline several frames — and drivers may spin-wait, burning
// CPU). So sample ONE frame per stats window: the p50 the OSD
// shows becomes that sample — honest, at zero pipeline cost on
// every other frame. Software keeps the synchronous stamp on
// every frame (its decode really is done by now).
let hw_fence = match &image {
DecodedImage::VkFrame(v) => Some((v.timeline_sem, v.decode_done_value)),
_ => None,
@@ -371,15 +381,18 @@ fn pump(
image,
});
// `decode` stage: received→decode COMPLETE, single clock.
let decode_done_ns = match hw_fence {
Some((sem, value))
if decoder.wait_hw_decoded(sem, value, 50_000_000) =>
{
now_ns()
match hw_fence {
Some((sem, value)) => {
if decode_us.is_empty()
&& decoder.wait_hw_decoded(sem, value, 50_000_000)
{
decode_us.push(now_ns().saturating_sub(received_ns) / 1000);
}
}
_ => decoded_ns,
};
decode_us.push(decode_done_ns.saturating_sub(received_ns) / 1000);
None => {
decode_us.push(decoded_ns.saturating_sub(received_ns) / 1000);
}
}
}
Ok(None) => no_output_streak += 1,
// Survivable (loss until the next IDR/RFI recovery) — keep feeding.
+90 -4
View File
@@ -1,7 +1,12 @@
//! Client identity, the known-hosts (pinned fingerprint) store, and app settings.
//!
//! The identity shares `~/.config/punktfunk/client-{cert,key}.pem` with `punktfunk-probe`
//! so a box pairs once whichever client it uses.
//! The identity shares `~/.config/punktfunk/client-{cert,key}.pem` (Linux; on Windows
//! `%APPDATA%\punktfunk`, the WinUI shell's directory) with `punktfunk-probe` so a box
//! pairs once whichever client it uses. On Windows the session binary reads the SAME
//! stores the WinUI shell (`clients/windows/src/trust.rs`) writes — pairing there makes
//! the session connect silently, mirroring the GTK-shell arrangement on Linux. The two
//! `Settings` structs differ in shape; `#[serde(default)]` on both sides reconciles them
//! (see the parity tests below), and the shell stays the settings file's only writer.
use anyhow::{anyhow, Context, Result};
use punktfunk_core::client::NativeClient;
@@ -10,8 +15,16 @@ use serde::{Deserialize, Serialize};
use std::path::PathBuf;
pub fn config_dir() -> Result<PathBuf> {
let home = std::env::var("HOME").context("HOME unset")?;
Ok(PathBuf::from(home).join(".config/punktfunk"))
#[cfg(windows)]
{
let appdata = std::env::var("APPDATA").context("APPDATA unset")?;
Ok(PathBuf::from(appdata).join("punktfunk"))
}
#[cfg(not(windows))]
{
let home = std::env::var("HOME").context("HOME unset")?;
Ok(PathBuf::from(home).join(".config/punktfunk"))
}
}
/// This client's persistent identity, generated on first use — presented on every connect
@@ -256,6 +269,18 @@ pub struct Settings {
/// `"vulkan"`, `"vaapi"`, `"software"`.
/// The `PUNKTFUNK_DECODER` env var overrides this (see `video::Decoder::new`).
pub decoder: String,
/// Decode/present GPU (multi-GPU boxes): the adapter's marketing name, as the WinUI
/// shell's GPU picker stores it; empty = automatic. The session maps it onto the
/// presenter's device pick (`PUNKTFUNK_VK_ADAPTER`). `default` so pre-existing
/// stores (and the Linux shells, which have no picker yet) load.
#[serde(default)]
pub adapter: String,
/// Advertise 10-bit + HDR10 so the host upgrades HDR content to a Main10/PQ stream.
/// The presenter handles the display side dynamically either way (HDR10 swapchain
/// where offered, tonemap where not) — off means "never send me 10-bit".
/// `default = true`: the Linux stores never carried this and always advertised.
#[serde(default = "default_true")]
pub hdr_enabled: bool,
/// Show the on-stream statistics overlay (toggle live with Ctrl+Alt+Shift+S).
pub show_stats: bool,
/// Enter fullscreen when a stream starts (F11 / the controller chord / the top-edge
@@ -270,6 +295,10 @@ fn default_codec() -> String {
"auto".into()
}
fn default_true() -> bool {
true
}
impl Settings {
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 {
@@ -297,6 +326,8 @@ impl Default for Settings {
audio_channels: 2,
codec: "auto".into(),
decoder: "auto".into(),
adapter: String::new(),
hdr_enabled: true,
show_stats: true,
fullscreen_on_stream: true,
library_enabled: false,
@@ -306,6 +337,13 @@ impl Default for Settings {
impl Settings {
fn path() -> Result<PathBuf> {
// The shell's settings file on each OS: the GTK shell's on Linux, the WinUI
// shell's on Windows. The shells own (and write) these files; the session binary
// only reads them, so `save` must never be called on Windows — it would rewrite
// the file in THIS struct's shape and drop the WinUI-only fields.
#[cfg(windows)]
return Ok(config_dir()?.join("client-windows-settings.json"));
#[cfg(not(windows))]
Ok(config_dir()?.join("client-gtk-settings.json"))
}
@@ -340,4 +378,52 @@ mod tests {
let round: Settings = serde_json::from_str(&serde_json::to_string(&s).unwrap()).unwrap();
assert_eq!(round.forward_pad, "");
}
/// On Windows the session reads the WinUI shell's settings file. This fixture is the
/// shell's `Settings` shape (clients/windows/src/trust.rs) verbatim — if that struct
/// changes, update this fixture with it. WinUI-only fields (hdr_enabled, adapter,
/// show_hud) must be ignored; fields this struct has and the shell's lacks
/// (forward_pad, show_stats, …) must default; the shell's D3D11VA-era
/// `decoder: "hardware"` must survive as-is (video::Decoder::new reads it as auto).
#[test]
fn settings_reads_winui_shell_shape() {
let shell = r#"{
"width": 2560, "height": 1440, "refresh_hz": 120, "bitrate_kbps": 20000,
"gamepad": "dualsense", "compositor": "auto",
"inhibit_shortcuts": true, "mic_enabled": true, "audio_channels": 6,
"hdr_enabled": true, "decoder": "hardware", "codec": "av1",
"adapter": "NVIDIA GeForce RTX 4080", "show_hud": false
}"#;
let s: Settings = serde_json::from_str(shell).unwrap();
assert_eq!((s.width, s.height, s.refresh_hz), (2560, 1440, 120));
assert_eq!(s.bitrate_kbps, 20000);
assert_eq!(s.audio_channels, 6);
assert!(s.mic_enabled);
assert_eq!(s.decoder, "hardware");
assert_eq!(s.preferred_codec(), punktfunk_core::quic::CODEC_AV1);
assert_eq!(s.adapter, "NVIDIA GeForce RTX 4080");
assert!(s.hdr_enabled);
// Fields the shell's file doesn't carry take this struct's defaults.
assert_eq!(s.forward_pad, "");
assert!(s.show_stats);
assert!(s.fullscreen_on_stream);
assert!(!s.library_enabled);
}
/// The WinUI shell's known-hosts shape (no `last_used` field) loads losslessly — same
/// filename, same directory, so on Windows the two clients genuinely share the store.
#[test]
fn known_hosts_reads_winui_shell_shape() {
let shell = r#"{"hosts":[{
"name": "Gaming PC", "addr": "192.168.1.50", "port": 9777,
"fp_hex": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"paired": true, "mac": ["aa:bb:cc:dd:ee:ff"]
}]}"#;
let k: KnownHosts = serde_json::from_str(shell).unwrap();
let h = k.find_by_addr("192.168.1.50", 9777).unwrap();
assert!(h.paired);
assert_eq!(h.last_used, None);
assert_eq!(h.mac, vec!["aa:bb:cc:dd:ee:ff".to_string()]);
assert!(parse_hex32(&h.fp_hex).is_some());
}
}
+44 -12
View File
@@ -18,12 +18,22 @@
//!
//! Both run `AV_CODEC_FLAG_LOW_DELAY`; the host encodes zero-reorder streams (no
//! B-frames, in-band parameter sets on every IDR), so decode is strictly one-in/one-out.
//!
//! On Windows the VAAPI/dmabuf backend does not exist (DRM-PRIME is a Linux concept), so
//! the chain is Vulkan → software — Intel (no Vulkan Video in its Windows driver) lands
//! on software. Everything dmabuf-shaped is `cfg(target_os = "linux")`-gated inline.
// bindgen's C-enum repr is target-dependent (u32 on Linux/clang, i32 on MSVC), so the
// pf-ffvk Vulkan flag/enum casts below are required on one platform and no-ops on the
// other — the lint would fire on whichever platform the cast is a no-op for.
#![allow(clippy::unnecessary_cast)]
use anyhow::{anyhow, bail, Context as _, Result};
use ffmpeg::format::Pixel;
use ffmpeg::software::scaling;
use ffmpeg::util::frame::Video as AvFrame;
use ffmpeg_next as ffmpeg;
#[cfg(target_os = "linux")]
use std::os::fd::RawFd;
use std::ptr;
@@ -42,6 +52,7 @@ pub struct DecodedFrame {
pub enum DecodedImage {
Cpu(CpuFrame),
#[cfg(target_os = "linux")]
Dmabuf(DmabufFrame),
/// FFmpeg Vulkan Video output: a VkImage already on the PRESENTER's device.
VkFrame(VkVideoFrame),
@@ -136,6 +147,7 @@ pub struct CpuFrame {
/// A decoded frame still on the GPU: dmabuf fds + plane layout for
/// `GdkDmabufTextureBuilder`. The fds belong to `guard`'s mapped DRM frame — they stay
/// valid until the guard drops (the texture's release func).
#[cfg(target_os = "linux")]
pub struct DmabufFrame {
pub width: u32,
pub height: u32,
@@ -150,6 +162,7 @@ pub struct DmabufFrame {
pub guard: DrmFrameGuard,
}
#[cfg(target_os = "linux")]
pub struct DmabufPlane {
pub fd: RawFd,
pub offset: u32,
@@ -170,6 +183,7 @@ impl Drop for DrmFrameGuard {
enum Backend {
Vulkan(VulkanDecoder),
#[cfg(target_os = "linux")]
Vaapi(VaapiDecoder),
Software(SoftwareDecoder),
}
@@ -240,12 +254,13 @@ fn quiet_ffmpeg_log() {
impl Decoder {
/// `codec_id` is the codec the host resolved in the Welcome (never assume HEVC).
/// `pref` is the Settings "Video decoder" value (`auto`/`vulkan`/`vaapi`/`software`).
/// `pref` is the Settings "Video decoder" value (`auto`/`vulkan`/`vaapi`/`software`;
/// `hardware` — the WinUI shell's stored value from its D3D11VA era — reads as auto).
/// `vk` is the presenter's shared Vulkan device when its stack can run FFmpeg's
/// Vulkan Video decoder — decode lands as VkImages the presenter samples directly.
/// Precedence: the `PUNKTFUNK_DECODER` env override wins (support/debug escape
/// hatch, and the documented knob), then the setting; both default to auto
/// (Vulkan → VAAPI → software).
/// (Vulkan → VAAPI → software; no VAAPI on Windows).
pub fn new(
codec_id: ffmpeg::codec::Id,
pref: &str,
@@ -265,7 +280,7 @@ impl Decoder {
want_keyframe: false,
})
};
if matches!(choice.as_str(), "auto" | "" | "vulkan") {
if matches!(choice.as_str(), "auto" | "" | "vulkan" | "hardware") {
match vk {
Some(vk) => match VulkanDecoder::new(codec_id, vk) {
Ok(v) => {
@@ -294,7 +309,9 @@ impl Decoder {
}
// Deck note: `auto` reaches VAAPI when Vulkan Video isn't available. A presenter
// that can't display the dmabufs demotes this decoder to software mid-session
// via [`Decoder::force_software`].
// via [`Decoder::force_software`]. Windows has no VAAPI — auto falls straight
// through to software there.
#[cfg(target_os = "linux")]
if choice != "software" && choice != "vulkan" {
match VaapiDecoder::new(codec_id) {
Ok(v) => {
@@ -361,6 +378,7 @@ impl Decoder {
pub fn decode(&mut self, au: &[u8]) -> Result<Option<DecodedImage>> {
let result = match &mut self.backend {
Backend::Vulkan(v) => v.decode(au).map(|f| f.map(DecodedImage::VkFrame)),
#[cfg(target_os = "linux")]
Backend::Vaapi(v) => v.decode(au).map(|f| f.map(DecodedImage::Dmabuf)),
Backend::Software(s) => return Ok(s.decode(au)?.map(DecodedImage::Cpu)),
};
@@ -532,7 +550,8 @@ impl SoftwareDecoder {
// Raw FFI: ffmpeg-next has no hwaccel wrappers. All pointers are owned here and freed in
// Drop; decoded surfaces transfer out through DrmFrameGuard.
const AVERROR_EAGAIN: i32 = -11; // -EAGAIN; Linux-only crate
// -EAGAIN. FFmpeg uses POSIX errno values on both our targets (MinGW's EAGAIN is 11 too).
const AVERROR_EAGAIN: i32 = -11;
fn averr(what: &str, code: i32) -> anyhow::Error {
anyhow!("{what}: {}", ffmpeg::Error::from(code))
@@ -542,6 +561,7 @@ fn averr(what: &str, code: i32) -> anyhow::Error {
/// back to the first (software) entry would silently decode on the CPU *and* break our
/// dmabuf mapping — return NONE instead so the error surfaces and the session demotes
/// to the software backend explicitly.
#[cfg(target_os = "linux")]
unsafe extern "C" fn pick_vaapi(
_ctx: *mut ffmpeg::ffi::AVCodecContext,
mut list: *const ffmpeg::ffi::AVPixelFormat,
@@ -557,6 +577,7 @@ unsafe extern "C" fn pick_vaapi(
ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE
}
#[cfg(target_os = "linux")]
struct VaapiDecoder {
ctx: *mut ffmpeg::ffi::AVCodecContext,
hw_device: *mut ffmpeg::ffi::AVBufferRef,
@@ -565,8 +586,10 @@ struct VaapiDecoder {
}
// Single-owner pointers, only touched from the session pump thread.
#[cfg(target_os = "linux")]
unsafe impl Send for VaapiDecoder {}
#[cfg(target_os = "linux")]
impl VaapiDecoder {
fn new(codec_id: ffmpeg::codec::Id) -> Result<VaapiDecoder> {
use ffmpeg::ffi;
@@ -764,6 +787,9 @@ const fn fourcc(a: u8, b: u8, c: u8, d: u8) -> u32 {
/// The combined DRM FourCC for a decoder software pixel format. The host streams 8-bit
/// 4:2:0 (NV12); P010 is here for the eventual 10-bit/HDR path.
// Only the (Linux-gated) VAAPI path calls this outside tests; the constants are worth
// locking on every platform, so it stays compiled rather than cfg-gated with its caller.
#[cfg_attr(windows, allow(dead_code))]
fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32> {
use ffmpeg_next::ffi::AVPixelFormat::*;
Some(match sw {
@@ -775,6 +801,7 @@ fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32> {
/// One-time dump of the DRM descriptor layout (objects, layers, planes, modifier) — so a
/// new client/driver combination's real layout is visible in the logs without a debugger.
#[cfg(target_os = "linux")]
fn log_descriptor_once(
d: &ffmpeg_next::ffi::AVDRMFrameDescriptor,
sw: ffmpeg_next::ffi::AVPixelFormat,
@@ -801,6 +828,7 @@ fn log_descriptor_once(
);
}
#[cfg(target_os = "linux")]
impl Drop for VaapiDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
@@ -915,26 +943,28 @@ impl VulkanDecoder {
(*hwctx).queue_family_decode_index = d;
(*hwctx).nb_decode_queues = 1;
const VIDEO_DECODE_BIT: u32 = 0x20; // VK_QUEUE_VIDEO_DECODE_BIT_KHR
// `flags`/`video_caps` are bindgen enum types: i32 under MSVC, u32 under
// Linux clang — the `as _` casts absorb the difference.
if g == d {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags | VIDEO_DECODE_BIT,
video_caps: vk.decode_video_caps,
flags: (vk.graphics_queue_flags | VIDEO_DECODE_BIT) as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 1;
} else {
(*hwctx).qf[0] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: g,
num: 1,
flags: vk.graphics_queue_flags,
flags: vk.graphics_queue_flags as _,
video_caps: 0,
};
(*hwctx).qf[1] = pf_ffvk::AVVulkanDeviceQueueFamily {
idx: d,
num: 1,
flags: VIDEO_DECODE_BIT,
video_caps: vk.decode_video_caps,
flags: VIDEO_DECODE_BIT as _,
video_caps: vk.decode_video_caps as _,
};
(*hwctx).nb_qf = 2;
}
@@ -1155,8 +1185,10 @@ unsafe extern "C" fn pick_vulkan(
let fc = (*fr).data as *mut ffi::AVHWFramesContext;
let vkfc = (*fc).hwctx as *mut pf_ffvk::AVVulkanFramesContext;
// MUTABLE_FORMAT: per-plane views (spec requirement); ALIAS is FFmpeg's default.
(*vkfc).img_flags = pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT
| pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_ALIAS_BIT;
// (`as _`: the FlagBits constants are i32 under MSVC, the img_flags field u32.)
(*vkfc).img_flags = (pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT
| pf_ffvk::VkImageCreateFlagBits_VK_IMAGE_CREATE_ALIAS_BIT)
as _;
let r = ffi::av_hwframe_ctx_init(fr);
if r < 0 {
tracing::warn!("av_hwframe_ctx_init(VULKAN) failed ({r})");