unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 18 Wiki Activity

refactor(client): extract UI-agnostic plumbing into pf-client-core (phase 0)

Browse Source 
Session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads, keymap, trust
store, mDNS discovery, library client and Wake-on-LAN move verbatim from
clients/linux into crates/pf-client-core, shared with the upcoming Vulkan
session binary (punktfunk-planning: linux-client-rearchitecture.md).
The GTK client re-exports them at the crate root so every existing
crate::-path keeps resolving; its manifest drops the moved-only deps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Enrico Bühler
2026-07-07 16:36:03 +02:00
parent bf07700c74
commit 0ab97b597c
20 changed files with 102 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
clients/android/native/Cargo.toml
+1 -1
View File
@@ -20,7 +20,7 @@ punktfunk-core = { path = "../../../crates/punktfunk-core", features = ["quic"]
jni = "0.21"
log = "0.4"
# LAN host discovery: browse the host's `_punktfunk._udp` mDNS advert — the SAME crate + service the
# Linux/Windows clients use (`clients/linux/src/discovery.rs`), replacing Android's per-OEM
# Linux/Windows clients use (`crates/pf-client-core/src/discovery.rs`), replacing Android's per-OEM
# `NsdManager` system daemon with one tested browse path. Pure Rust (socket2/if-addrs/mio), so it
# cross-compiles to the Android targets AND builds on the host (the JNI seam links into
# `cargo build --workspace`). Kotlin keeps only the Wi-Fi `MulticastLock` + permission UX.
clients/android/native/src/discovery.rs
+1 -1
View File
@@ -1,5 +1,5 @@
//! LAN host discovery over mDNS, in Rust via `mdns-sd` — the same crate + service type the
//! Linux/Windows clients use (`clients/linux/src/discovery.rs`), exposed to Kotlin over JNI.
//! Linux/Windows clients use (`crates/pf-client-core/src/discovery.rs`), exposed to Kotlin over JNI.
//!
//! Why not `NsdManager`: that API delegates to a per-OEM system mDNS daemon whose reliability
//! varies wildly (the Android client's discovery was "mostly broken"). Browsing in our own Rust
clients/linux/Cargo.toml
+4 -15
View File
@@ -15,6 +15,10 @@ path = "src/main.rs"
# Everything is Linux-gated so `cargo build --workspace` stays green on macOS (the Mac
# client lives in clients/apple); on other platforms this builds as a stub binary.
[target.'cfg(target_os = "linux")'.dependencies]
# Session pump, decode, audio, gamepads, trust, discovery, keymap — the UI-agnostic
# plumbing, shared with the upcoming Vulkan session binary (design: punktfunk-planning
# linux-client-rearchitecture.md). This crate keeps only the GTK shell + GL presenter.
pf-client-core = { path = "../../crates/pf-client-core" }
punktfunk-core = { path = "../../crates/punktfunk-core", features = ["quic"] }
# UI shell. GraphicsOffload needs GTK ≥ 4.14; black-background ≥ 4.16. AlertDialog/
@@ -23,25 +27,10 @@ gtk = { package = "gtk4", version = "0.11", features = ["v4_16"] }
adw = { package = "libadwaita", version = "0.9", features = ["v1_5"] }
async-channel = "2"
# Video decode (same FFmpeg pin as the host) and audio.
ffmpeg-next = "8"
opus = "0.3"
pipewire = "0.9"
# Gamepads: capture + feedback (full DualSense fidelity — touchpad/motion/triggers/LEDs
# need the hidapi driver).
sdl3 = { version = "0.18", features = ["hidapi"] }
# The VAAPI GL presenter (video_gl.rs): EGL dmabuf import into a GDK-shared context, dlopened
# at runtime (`dynamic`) so GPU-less boxes and the software path never touch libEGL.
khronos-egl = { version = "6", features = ["dynamic"] }
mdns-sd = "0.20"
# Game-library fetch from the host's management API over mTLS + fingerprint pinning.
# `ureq` is small + sync (the host uses it too) and its rustls unifies with the
# workspace's (quinn's) 0.23; the pinning verifier mirrors core's private `PinVerify`.
ureq = "2"
rustls = { version = "0.23", features = ["ring"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
anyhow = "1"
tracing = "0.1"
clients/linux/README.md
+6 -6
View File
@@ -78,15 +78,15 @@ src/
ui_trust.rs TOFU / PIN-pairing / request-access dialogs
ui_settings.rs resolution · refresh · decoder · bitrate · compositor · mic
ui_stream.rs the stream window (GtkGraphicsOffload present) + input capture
launch.rs · session.rs session launch/UI glue; lifecycle over the NativeClient connector
video.rs FFmpeg VAAPI / software decode → dmabuf / texture
audio.rs PipeWire playback + mic uplink
gamepad.rs · keymap.rs SDL3 controllers + feedback; keyboard VK mapping
trust.rs · discovery.rs persistent identity, known hosts + settings, mDNS browse
library.rs mgmt-API library client (mTLS + pinned fingerprint, art proxy)
launch.rs session launch/UI glue over the shared session pump
video_gl.rs VAAPI dmabuf → RGBA GL presenter (EGL import, CICP-driven CSC)
tools/screenshots.sh store screenshot capture (app self-capture; Xvfb fallback)
```
The UI-agnostic plumbing — session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads +
keymap, trust store, mDNS discovery, library client, Wake-on-LAN — lives in
`crates/pf-client-core`, shared with the Vulkan session binary.
## Related
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
clients/linux/src/audio.rs
-415
View File
@@ -1,415 +0,0 @@
//! Audio: playback (decoded PCM → a PipeWire playback stream) and the microphone uplink
//! (PipeWire capture → Opus → 0xCB datagrams, the inverse of the host's virtual mic).
//!
//! Playback mirrors the host's virtual-mic producer (`punktfunk-host::audio::linux`) with
//! the same adaptive jitter buffer: the session pump pushes 5 ms Opus-decoded chunks on
//! the network clock; PipeWire pulls whole quanta on the device clock. Prime to ~3
//! quanta before producing, cap the ring so latency stays bounded, re-prime after a real
//! drain.
use anyhow::{Context, Result};
use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
use std::sync::mpsc::{Receiver, SyncSender, TrySendError};
use std::sync::Arc;
const SAMPLE_RATE: u32 = 48_000;
const CHANNELS: usize = 2;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
struct Terminate;
pub struct AudioPlayer {
pcm_tx: SyncSender<Vec<f32>>,
/// Drained chunk Vecs coming back from the PipeWire consumer for reuse (the pool half
/// of the pcm channel — see [`AudioPlayer::take_buffer`]).
recycle_rx: Receiver<Vec<f32>>,
quit_tx: pipewire::channel::Sender<Terminate>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl AudioPlayer {
/// Spawn the PipeWire playback thread for `channels` (2/6/8, canonical wire order
/// FL FR FC LFE RL RR SL SR). Failure (no PipeWire in the session) is survivable — the
/// caller streams video-only.
pub fn spawn(channels: u32) -> Result<AudioPlayer> {
// 64 × 5 ms = 320 ms of slack between the pump and the PipeWire loop.
let (pcm_tx, pcm_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
// Return path: the process callback sends each drained Vec back for reuse, so
// steady-state playback stops allocating (~200 chunks/s otherwise). Same capacity
// as the data channel; a full pool just drops the Vec (plain deallocation).
let (recycle_tx, recycle_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
let (quit_tx, quit_rx) = pipewire::channel::channel::<Terminate>();
let thread = std::thread::Builder::new()
.name("punktfunk-audio".into())
.spawn(move || {
if let Err(e) = pw_thread(pcm_rx, recycle_tx, quit_rx, channels as usize) {
tracing::warn!(error = %e, "audio playback thread ended");
}
})
.context("spawn audio thread")?;
Ok(AudioPlayer {
pcm_tx,
recycle_rx,
quit_tx,
thread: Some(thread),
})
}
/// A recycled chunk Vec from the pool, empty but with its capacity intact — fill it
/// and hand it back through [`push`](Self::push). Allocates only when the pool is dry
/// (startup, or after the PipeWire side dropped chunks).
pub fn take_buffer(&self) -> Vec<f32> {
self.recycle_rx.try_recv().unwrap_or_default()
}
/// Queue one interleaved f32 chunk (in the session's channel layout). Drops the chunk if the
/// PipeWire side is wedged (the renderer conceals the gap; never block the session pump).
pub fn push(&self, pcm: Vec<f32>) {
if let Err(TrySendError::Disconnected(_)) = self.pcm_tx.try_send(pcm) {
// Thread already dead — Drop will reap it; nothing to do per-chunk.
}
}
}
impl Drop for AudioPlayer {
fn drop(&mut self) {
let _ = self.quit_tx.send(Terminate);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
/// Producer-side state: incoming decoded PCM and the ring the process callback drains.
struct PlayerData {
rx: Receiver<Vec<f32>>,
/// Drained chunk Vecs go back here for the decode side to refill (allocation pool).
recycle: SyncSender<Vec<f32>>,
ring: VecDeque<f32>,
primed: bool,
/// Interleaved channel count this stream was opened with (2/6/8).
channels: usize,
}
fn pw_thread(
pcm_rx: Receiver<Vec<f32>>,
recycle_tx: SyncSender<Vec<f32>>,
quit_rx: pipewire::channel::Receiver<Terminate>,
channels: usize,
) -> Result<()> {
use pipewire as pw;
use pw::{properties::properties, spa};
use spa::param::audio::{AudioFormat, AudioInfoRaw};
use spa::pod::Pod;
static PW_INIT: std::sync::Once = std::sync::Once::new();
PW_INIT.call_once(pw::init);
let mainloop = pw::main_loop::MainLoopRc::new(None).context("pw MainLoop")?;
let context = pw::context::ContextRc::new(&mainloop, None).context("pw Context")?;
let core = context
.connect_rc(None)
.context("pw connect (is PipeWire running in this session?)")?;
let _quit_guard = quit_rx.attach(mainloop.loop_(), {
let mainloop = mainloop.clone();
move |_| mainloop.quit()
});
let stream = pw::stream::StreamBox::new(
&core,
"punktfunk-client",
properties! {
*pw::keys::MEDIA_TYPE => "Audio",
*pw::keys::MEDIA_CATEGORY => "Playback",
*pw::keys::MEDIA_ROLE => "Game",
*pw::keys::NODE_NAME => "punktfunk-client",
*pw::keys::NODE_DESCRIPTION => "Punktfunk Stream",
// ~5 ms quantum (one Opus frame) keeps the ring — and so the latency — small.
*pw::keys::NODE_LATENCY => "240/48000",
},
)
.context("pw Stream")?;
let ud = PlayerData {
rx: pcm_rx,
recycle: recycle_tx,
ring: VecDeque::new(),
primed: false,
channels,
};
let _listener = stream
.add_local_listener_with_user_data(ud)
.state_changed(|_s, _ud, old, new| {
tracing::debug!(?old, ?new, "pipewire playback stream state");
})
.process(|stream, ud| {
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
let Some(mut buffer) = stream.dequeue_buffer() else {
return;
};
while let Ok(mut chunk) = ud.rx.try_recv() {
ud.ring.extend(chunk.iter().copied());
// Return the drained Vec to the pool; a full/closed pool drops it.
chunk.clear();
let _ = ud.recycle.try_send(chunk);
}
let stride = 4 * ud.channels; // F32LE interleaved
let datas = buffer.datas_mut();
if datas.is_empty() {
return;
}
let data = &mut datas[0];
let want_frames = data.data().map(|s| s.len() / stride).unwrap_or(0);
let want = want_frames * ud.channels;
// Adaptive jitter buffer (same shape as the host's virtual mic): prime to
// ~3 quanta, cap at ~1 quantum of slack beyond that, re-prime after a
// genuine drain.
let target = (3 * want).clamp(720 * ud.channels, 9600 * ud.channels);
while ud.ring.len() > target.max(want) + want {
ud.ring.pop_front();
}
if !ud.primed && ud.ring.len() >= target {
ud.primed = true;
}
let n_frames = if let Some(slice) = data.data() {
for k in 0..want {
let s = if ud.primed {
ud.ring.pop_front().unwrap_or(0.0)
} else {
0.0
};
let off = k * 4;
slice[off..off + 4].copy_from_slice(&s.to_le_bytes());
}
want_frames
} else {
0
};
if ud.ring.is_empty() {
ud.primed = false;
}
let chunk = data.chunk_mut();
*chunk.offset_mut() = 0;
*chunk.stride_mut() = stride as _;
*chunk.size_mut() = (stride * n_frames) as _;
}));
if outcome.is_err() {
tracing::error!("panic in pipewire playback callback");
}
})
.register()
.context("register playback listener")?;
let mut info = AudioInfoRaw::new();
info.set_format(AudioFormat::F32LE);
info.set_rate(SAMPLE_RATE);
info.set_channels(channels as u32);
// Channel positions in canonical wire order (FL FR FC LFE RL RR SL SR) so PipeWire routes each
// slot to the matching speaker (and downmixes when the sink has fewer). Identity, no permute.
let order = punktfunk_core::audio::spa_positions(channels as u8);
let mut positions = [0u32; 64];
positions[..order.len()].copy_from_slice(order);
info.set_position(positions);
let obj = pw::spa::pod::Object {
type_: pw::spa::utils::SpaTypes::ObjectParamFormat.as_raw(),
id: pw::spa::param::ParamType::EnumFormat.as_raw(),
properties: info.into(),
};
let values: Vec<u8> = pw::spa::pod::serialize::PodSerializer::serialize(
std::io::Cursor::new(Vec::new()),
&pw::spa::pod::Value::Object(obj),
)
.context("serialize format pod")?
.0
.into_inner();
let mut params = [Pod::from_bytes(&values).context("pod from bytes")?];
stream
.connect(
spa::utils::Direction::Output,
None,
pw::stream::StreamFlags::AUTOCONNECT | pw::stream::StreamFlags::MAP_BUFFERS,
&mut params,
)
.context("pw stream connect")?;
mainloop.run();
tracing::debug!("pipewire playback loop exited");
Ok(())
}
/// The microphone uplink: capture the default input device, Opus-encode 20 ms chunks,
/// ship them as 0xCB datagrams into the host's virtual PipeWire source.
pub struct MicStreamer {
quit_tx: pipewire::channel::Sender<Terminate>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl MicStreamer {
pub fn spawn(connector: Arc<NativeClient>) -> Result<MicStreamer> {
let (quit_tx, quit_rx) = pipewire::channel::channel::<Terminate>();
let thread = std::thread::Builder::new()
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, quit_rx) {
tracing::warn!(error = %e, "mic uplink thread ended");
}
})
.context("spawn mic thread")?;
Ok(MicStreamer {
quit_tx,
thread: Some(thread),
})
}
}
impl Drop for MicStreamer {
fn drop(&mut self) {
let _ = self.quit_tx.send(Terminate);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
/// Capture-side state: accumulated PCM and the Opus encoder (encoding a 20 ms frame is
/// ~100 µs — fine inside the process callback).
struct MicData {
connector: Arc<NativeClient>,
ring: VecDeque<f32>,
encoder: opus::Encoder,
seq: u32,
out: Vec<u8>,
}
fn mic_thread(
connector: &Arc<NativeClient>,
quit_rx: pipewire::channel::Receiver<Terminate>,
) -> Result<()> {
use pipewire as pw;
use pw::{properties::properties, spa};
use spa::param::audio::{AudioFormat, AudioInfoRaw};
use spa::pod::Pod;
static PW_INIT: std::sync::Once = std::sync::Once::new();
PW_INIT.call_once(pw::init);
let mut encoder =
opus::Encoder::new(SAMPLE_RATE, opus::Channels::Stereo, opus::Application::Voip)
.map_err(|e| anyhow::anyhow!("opus encoder: {e}"))?;
let _ = encoder.set_bitrate(opus::Bitrate::Bits(64_000));
let mainloop = pw::main_loop::MainLoopRc::new(None).context("pw mic MainLoop")?;
let context = pw::context::ContextRc::new(&mainloop, None).context("pw mic Context")?;
let core = context
.connect_rc(None)
.context("pw mic connect (is PipeWire running in this session?)")?;
let _quit_guard = quit_rx.attach(mainloop.loop_(), {
let mainloop = mainloop.clone();
move |_| mainloop.quit()
});
let stream = pw::stream::StreamBox::new(
&core,
"punktfunk-mic-capture",
properties! {
*pw::keys::MEDIA_TYPE => "Audio",
*pw::keys::MEDIA_CATEGORY => "Capture",
*pw::keys::MEDIA_ROLE => "Communication",
*pw::keys::NODE_NAME => "punktfunk-mic-capture",
*pw::keys::NODE_DESCRIPTION => "Punktfunk Microphone",
},
)
.context("pw mic Stream")?;
let ud = MicData {
connector: connector.clone(),
ring: VecDeque::new(),
encoder,
seq: 0,
out: vec![0u8; 4000],
};
let _listener = stream
.add_local_listener_with_user_data(ud)
.state_changed(|_s, _ud, old, new| {
tracing::debug!(?old, ?new, "pipewire mic capture stream state");
})
.process(|stream, ud| {
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
let Some(mut buffer) = stream.dequeue_buffer() else {
return;
};
let datas = buffer.datas_mut();
if datas.is_empty() {
return;
}
let data = &mut datas[0];
let n = data.chunk().size() as usize;
if let Some(slice) = data.data() {
for s in slice[..n.min(slice.len())].chunks_exact(4) {
ud.ring
.push_back(f32::from_le_bytes([s[0], s[1], s[2], s[3]]));
}
}
// Ship every complete 20 ms stereo frame.
while ud.ring.len() >= MIC_FRAME * CHANNELS {
let pcm: Vec<f32> = ud.ring.drain(..MIC_FRAME * CHANNELS).collect();
match ud.encoder.encode_float(&pcm, &mut ud.out) {
Ok(len) => {
let pts = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as u64)
.unwrap_or(0);
let _ = ud.connector.send_mic(ud.seq, pts, ud.out[..len].to_vec());
ud.seq = ud.seq.wrapping_add(1);
}
Err(e) => tracing::debug!(error = %e, "opus mic encode"),
}
}
}));
if outcome.is_err() {
tracing::error!("panic in pipewire mic callback");
}
})
.register()
.context("register mic listener")?;
let mut info = AudioInfoRaw::new();
info.set_format(AudioFormat::F32LE);
info.set_rate(SAMPLE_RATE);
info.set_channels(CHANNELS as u32);
let obj = pw::spa::pod::Object {
type_: pw::spa::utils::SpaTypes::ObjectParamFormat.as_raw(),
id: pw::spa::param::ParamType::EnumFormat.as_raw(),
properties: info.into(),
};
let values: Vec<u8> = pw::spa::pod::serialize::PodSerializer::serialize(
std::io::Cursor::new(Vec::new()),
&pw::spa::pod::Value::Object(obj),
)
.context("serialize mic format pod")?
.0
.into_inner();
let mut params = [Pod::from_bytes(&values).context("mic pod from bytes")?];
stream
.connect(
spa::utils::Direction::Input,
None,
pw::stream::StreamFlags::AUTOCONNECT | pw::stream::StreamFlags::MAP_BUFFERS,
&mut params,
)
.context("pw mic stream connect")?;
mainloop.run();
tracing::debug!("pipewire mic capture loop exited");
Ok(())
}
clients/linux/src/discovery.rs
-107
View File
@@ -1,107 +0,0 @@
//! LAN host discovery: browse the host's mDNS advert (`_punktfunk._udp`, TXT keys
//! `fp`/`pair`/`id` — see the host crate's `discovery.rs`) on a worker thread and stream
//! results to the UI. Removal events are forwarded too, so the hosts page can drop stale
//! cards and flip a saved host's online pip when its advert disappears.
use mdns_sd::{ServiceDaemon, ServiceEvent};
#[derive(Clone, Debug)]
pub struct DiscoveredHost {
/// Stable row key: the advertised host id, falling back to the mDNS fullname.
pub key: String,
/// The mDNS service fullname — what a later `Removed` event identifies the advert by.
pub fullname: String,
pub name: String,
pub addr: String,
pub port: u16,
/// Host certificate fingerprint to pin (lowercase hex), empty if not advertised.
pub fp_hex: String,
/// Pairing requirement: `"required"` or `"optional"`.
pub pair: String,
/// The management API's port (mDNS `mgmt` TXT) — where the game library is served.
/// `None` when not advertised (older host / standalone `punktfunk1-host`); the
/// library client then falls back to the well-known default.
pub mgmt_port: Option<u16>,
/// Wake-on-LAN MAC(s) from the mDNS `mac` TXT (comma-separated `aa:bb:cc:dd:ee:ff`), which the
/// hosts page persists onto the matching saved host so it can wake it later. Empty if absent.
pub mac: Vec<String>,
}
/// One discovery update for the UI's advert map.
pub enum DiscoveryEvent {
/// A host advert appeared or refreshed (new address, pairing flipped, …).
Resolved(DiscoveredHost),
/// The advert went away (host stopped / left the network).
Removed { fullname: String },
}
/// Browse continuously for the app's lifetime. The thread exits when the receiver is
/// dropped (the send fails) or the daemon dies.
pub fn browse() -> async_channel::Receiver<DiscoveryEvent> {
let (tx, rx) = async_channel::unbounded();
std::thread::Builder::new()
.name("punktfunk-mdns".into())
.spawn(move || {
let daemon = match ServiceDaemon::new() {
Ok(d) => d,
Err(e) => {
tracing::warn!(error = %e, "mDNS daemon failed — discovery disabled");
return;
}
};
let receiver = match daemon.browse("_punktfunk._udp.local.") {
Ok(r) => r,
Err(e) => {
tracing::warn!(error = %e, "mDNS browse failed — discovery disabled");
return;
}
};
while let Ok(event) = receiver.recv() {
let update = match event {
ServiceEvent::ServiceResolved(info) => {
let props = info.get_properties();
let val = |k: &str| props.get_property_val_str(k).unwrap_or("").to_string();
let Some(addr) = info.get_addresses().iter().next().map(|a| a.to_string())
else {
continue;
};
let id = val("id");
DiscoveryEvent::Resolved(DiscoveredHost {
key: if id.is_empty() {
info.get_fullname().to_string()
} else {
id
},
fullname: info.get_fullname().to_string(),
name: info
.get_fullname()
.split('.')
.next()
.unwrap_or("?")
.to_string(),
addr,
port: info.get_port(),
fp_hex: val("fp"),
pair: val("pair"),
mgmt_port: val("mgmt").parse().ok(),
mac: val("mac")
.split(',')
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
.collect(),
})
}
ServiceEvent::ServiceRemoved(_ty, fullname) => {
DiscoveryEvent::Removed { fullname }
}
_ => continue,
};
if tx.send_blocking(update).is_err() {
break; // UI gone — stop browsing
}
}
let _ = daemon.shutdown();
})
.expect("spawn mdns thread");
rx
}
clients/linux/src/gamepad.rs
-1460
View File
File diff suppressed because it is too large Load Diff
clients/linux/src/keymap.rs
-203
View File
@@ -1,203 +0,0 @@
//! Local key/button codes → the punktfunk input wire contract.
//!
//! The wire carries Windows Virtual-Key codes (the GameStream convention; the host maps
//! them back with `inject::vk_to_evdev`). GTK hands us the hardware keycode, which on
//! Wayland (and X11) is the evdev code + 8 — so this table is the exact inverse of the
//! host's, keyed on evdev codes. Layout-independent by construction: positional keys map
//! positionally, exactly what a game expects.
/// Map a Linux evdev key code to the Windows VK code the host expects. `None` = a key the
/// wire contract doesn't cover (media keys etc.) — drop it rather than guess.
pub fn evdev_to_vk(evdev: u16) -> Option<u8> {
Some(match evdev {
// --- Navigation / editing / whitespace ---
14 => 0x08, // KEY_BACKSPACE -> VK_BACK
15 => 0x09, // KEY_TAB -> VK_TAB
28 => 0x0D, // KEY_ENTER -> VK_RETURN
119 => 0x13, // KEY_PAUSE -> VK_PAUSE
58 => 0x14, // KEY_CAPSLOCK -> VK_CAPITAL
1 => 0x1B, // KEY_ESC -> VK_ESCAPE
57 => 0x20, // KEY_SPACE -> VK_SPACE
104 => 0x21, // KEY_PAGEUP -> VK_PRIOR
109 => 0x22, // KEY_PAGEDOWN -> VK_NEXT
107 => 0x23, // KEY_END -> VK_END
102 => 0x24, // KEY_HOME -> VK_HOME
105 => 0x25, // KEY_LEFT -> VK_LEFT
103 => 0x26, // KEY_UP -> VK_UP
106 => 0x27, // KEY_RIGHT -> VK_RIGHT
108 => 0x28, // KEY_DOWN -> VK_DOWN
99 => 0x2C, // KEY_SYSRQ -> VK_SNAPSHOT
110 => 0x2D, // KEY_INSERT -> VK_INSERT
111 => 0x2E, // KEY_DELETE -> VK_DELETE
// --- Digit row (KEY_1..KEY_9 are 2..10, KEY_0 is 11) ---
11 => 0x30,
2 => 0x31,
3 => 0x32,
4 => 0x33,
5 => 0x34,
6 => 0x35,
7 => 0x36,
8 => 0x37,
9 => 0x38,
10 => 0x39,
// --- Letters (evdev order is QWERTY rows, not alphabetical) ---
30 => 0x41, // A
48 => 0x42, // B
46 => 0x43, // C
32 => 0x44, // D
18 => 0x45, // E
33 => 0x46, // F
34 => 0x47, // G
35 => 0x48, // H
23 => 0x49, // I
36 => 0x4A, // J
37 => 0x4B, // K
38 => 0x4C, // L
50 => 0x4D, // M
49 => 0x4E, // N
24 => 0x4F, // O
25 => 0x50, // P
16 => 0x51, // Q
19 => 0x52, // R
31 => 0x53, // S
20 => 0x54, // T
22 => 0x55, // U
47 => 0x56, // V
17 => 0x57, // W
45 => 0x58, // X
21 => 0x59, // Y
44 => 0x5A, // Z
// --- Meta / context-menu ---
125 => 0x5B, // KEY_LEFTMETA -> VK_LWIN
126 => 0x5C, // KEY_RIGHTMETA -> VK_RWIN
127 => 0x5D, // KEY_COMPOSE -> VK_APPS
// --- Numpad ---
82 => 0x60, // KP0
79 => 0x61,
80 => 0x62,
81 => 0x63,
75 => 0x64,
76 => 0x65,
77 => 0x66,
71 => 0x67,
72 => 0x68,
73 => 0x69, // KP9
55 => 0x6A, // KEY_KPASTERISK -> VK_MULTIPLY
78 => 0x6B, // KEY_KPPLUS -> VK_ADD
96 => 0x6C, // KEY_KPENTER -> VK_SEPARATOR
74 => 0x6D, // KEY_KPMINUS -> VK_SUBTRACT
83 => 0x6E, // KEY_KPDOT -> VK_DECIMAL
98 => 0x6F, // KEY_KPSLASH -> VK_DIVIDE
// --- Function keys ---
59 => 0x70, // F1
60 => 0x71,
61 => 0x72,
62 => 0x73,
63 => 0x74,
64 => 0x75,
65 => 0x76,
66 => 0x77,
67 => 0x78,
68 => 0x79, // F10
87 => 0x7A, // F11
88 => 0x7B, // F12
// --- Locks ---
69 => 0x90, // KEY_NUMLOCK -> VK_NUMLOCK
70 => 0x91, // KEY_SCROLLLOCK -> VK_SCROLL
// --- Left/right modifiers (specific VKs; the host maps both generics here too) ---
42 => 0xA0, // KEY_LEFTSHIFT -> VK_LSHIFT
54 => 0xA1, // KEY_RIGHTSHIFT -> VK_RSHIFT
29 => 0xA2, // KEY_LEFTCTRL -> VK_LCONTROL
97 => 0xA3, // KEY_RIGHTCTRL -> VK_RCONTROL
56 => 0xA4, // KEY_LEFTALT -> VK_LMENU
100 => 0xA5, // KEY_RIGHTALT -> VK_RMENU
// --- OEM punctuation (US-layout positions) ---
39 => 0xBA, // KEY_SEMICOLON -> VK_OEM_1
13 => 0xBB, // KEY_EQUAL -> VK_OEM_PLUS
51 => 0xBC, // KEY_COMMA -> VK_OEM_COMMA
12 => 0xBD, // KEY_MINUS -> VK_OEM_MINUS
52 => 0xBE, // KEY_DOT -> VK_OEM_PERIOD
53 => 0xBF, // KEY_SLASH -> VK_OEM_2
41 => 0xC0, // KEY_GRAVE -> VK_OEM_3
26 => 0xDB, // KEY_LEFTBRACE -> VK_OEM_4
43 => 0xDC, // KEY_BACKSLASH -> VK_OEM_5
27 => 0xDD, // KEY_RIGHTBRACE -> VK_OEM_6
40 => 0xDE, // KEY_APOSTROPHE -> VK_OEM_7
86 => 0xE2, // KEY_102ND -> VK_OEM_102
_ => return None,
})
}
/// Map a GTK/GDK mouse button number to the GameStream button id the wire expects
/// (1=left, 2=middle, 3=right, 4=X1, 5=X2). GDK reports back/forward as 8/9.
pub fn gdk_button_to_gs(button: u32) -> Option<u32> {
Some(match button {
1 => 1,
2 => 2,
3 => 3,
8 => 4,
9 => 5,
_ => return None,
})
}
#[cfg(test)]
mod tests {
use super::*;
/// The table must be the exact inverse of the host's `vk_to_evdev` for every key the
/// host knows (modulo the generic-modifier VKs, which collapse onto the same evdev
/// codes as the specific left-hand ones).
#[test]
fn roundtrips_through_the_host_table() {
// Mirror of the host's table (inject::vk_to_evdev), generic modifiers excluded.
let host_pairs: &[(u8, u16)] = &[
(0x08, 14),
(0x09, 15),
(0x0D, 28),
(0x13, 119),
(0x14, 58),
(0x1B, 1),
(0x20, 57),
(0x21, 104),
(0x22, 109),
(0x23, 107),
(0x24, 102),
(0x25, 105),
(0x26, 103),
(0x27, 106),
(0x28, 108),
(0x2C, 99),
(0x2D, 110),
(0x2E, 111),
(0x30, 11),
(0x31, 2),
(0x39, 10),
(0x41, 30),
(0x5A, 44),
(0x5B, 125),
(0x60, 82),
(0x69, 73),
(0x70, 59),
(0x7B, 88),
(0x90, 69),
(0xA0, 42),
(0xA5, 100),
(0xBA, 39),
(0xE2, 86),
];
for &(vk, evdev) in host_pairs {
assert_eq!(evdev_to_vk(evdev), Some(vk), "evdev {evdev}");
}
assert_eq!(evdev_to_vk(113), None); // KEY_MUTE — not in the wire contract
}
}
clients/linux/src/library.rs
-364
View File
@@ -1,364 +0,0 @@
//! Game-library client for the host's management REST API (the Apple `LibraryClient`
//! ported): `GET https://<host>:<mgmt>/api/v1/library` plus the per-title art proxy.
//! Authentication is **mTLS** — this client presents its persistent identity (the same
//! cert the host paired over QUIC) and the host authorizes paired certificates for the
//! read-only library routes, no bearer token. The host's self-signed certificate is
//! verified by its pinned SHA-256 fingerprint (`KnownHost::fp_hex`), not a CA chain.
use serde::Deserialize;
use std::collections::VecDeque;
use std::io::Read;
use std::sync::{Arc, Mutex};
use std::time::Duration;
/// The management API's default port — matches `mgmt::DEFAULT_PORT` on the host. A
/// discovered host may override it via its mDNS `mgmt` TXT (`DiscoveredHost::mgmt_port`);
/// saved-but-not-advertising hosts fall back here (Apple parity).
pub const DEFAULT_MGMT_PORT: u16 = 47990;
/// Cover-art URLs, mirroring the host's `library::Artwork`: absolute CDN URLs for custom
/// entries, host-relative proxy paths (`/api/v1/library/art/...`) for Steam titles. The
/// wire shape also carries a `logo` (a transparent title logo) — not a poster kind, so
/// serde just skips it here.
#[derive(Clone, Debug, Default, Deserialize)]
pub struct Artwork {
#[serde(default)]
pub portrait: Option<String>,
#[serde(default)]
pub hero: Option<String>,
#[serde(default)]
pub header: Option<String>,
}
impl Artwork {
/// Poster candidates in the Apple client's fallback order — portrait (the 600×900
/// capsule) → header (near-universal) → hero — with host-relative paths resolved
/// against `base` so the loader only ever sees absolute URLs.
pub fn poster_candidates(&self, base: &str) -> Vec<String> {
[&self.portrait, &self.header, &self.hero]
.into_iter()
.flatten()
.map(|u| {
if u.starts_with('/') {
format!("{base}{u}")
} else {
u.clone()
}
})
.collect()
}
}
/// One title in the host's unified library. `id` is store-qualified (`steam:<appid>`,
/// `custom:<id>`) and is also the launch handle the Hello carries when a session is
/// started from the library. The host's `launch` spec field is deliberately not
/// deserialized — launching goes by id, the host resolves the spec itself.
#[derive(Clone, Debug, Deserialize)]
pub struct GameEntry {
pub id: String,
/// Which store surfaced it (`"steam"`, `"custom"`, future `"heroic"`/`"gog"`/…) —
/// drives the poster's store badge.
pub store: String,
pub title: String,
#[serde(default)]
pub art: Artwork,
}
/// Errors surfaced to the UI so it can guide setup (the common case is "not paired yet").
#[derive(Debug)]
pub enum LibraryError {
/// The host rejected our certificate — this device isn't on its paired list.
NotPaired,
/// The host's certificate didn't hash to the pinned fingerprint (impostor/rotated cert).
PinMismatch,
Http(u16),
Unreachable(String),
}
impl std::fmt::Display for LibraryError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
LibraryError::NotPaired => f.write_str(
"The host didn't recognize this device. Pair with the host first — the \
library is authorized by this device's certificate (no token needed).",
),
LibraryError::PinMismatch => f.write_str(
"The host's certificate doesn't match the pinned fingerprint. \
Re-pair with a PIN to re-establish trust.",
),
LibraryError::Http(code) => {
write!(f, "The management API returned HTTP {code}.")
}
LibraryError::Unreachable(why) => write!(
f,
"Couldn't reach the host's management API: {why}. Check the host is \
updated and reachable (a host pinned to --mgmt-bind 127.0.0.1 is \
loopback-only and can't be browsed remotely)."
),
}
}
}
/// `https://addr:port`, IPv6 literals bracketed.
pub fn base_url(addr: &str, mgmt_port: u16) -> String {
if addr.contains(':') {
format!("https://[{addr}]:{mgmt_port}")
} else {
format!("https://{addr}:{mgmt_port}")
}
}
/// An HTTPS agent presenting `identity` via TLS client auth and verifying the server by
/// `pin` (`None` = accept any cert, the TOFU special case — same semantics as the QUIC
/// connect). Reused across a whole grid's worth of poster loads.
pub fn agent(
identity: &(String, String),
pin: Option<[u8; 32]>,
) -> Result<ureq::Agent, LibraryError> {
use rustls::pki_types::pem::PemObject;
let bad =
|what: &str, e: &dyn std::fmt::Display| LibraryError::Unreachable(format!("{what}: {e}"));
// The ring provider, explicitly — the same one core's QUIC endpoints install, so the
// process never mixes rustls crypto providers.
let provider = Arc::new(rustls::crypto::ring::default_provider());
let builder = rustls::ClientConfig::builder_with_provider(provider)
.with_safe_default_protocol_versions()
.map_err(|e| bad("tls config", &e))?
.dangerous()
.with_custom_certificate_verifier(Arc::new(PinVerify { pin }));
let cert = rustls::pki_types::CertificateDer::from_pem_slice(identity.0.as_bytes())
.map_err(|e| bad("client cert pem", &e))?;
let key = rustls::pki_types::PrivateKeyDer::from_pem_slice(identity.1.as_bytes())
.map_err(|e| bad("client key pem", &e))?;
let cfg = builder
.with_client_auth_cert(vec![cert], key)
.map_err(|e| bad("client auth", &e))?;
Ok(ureq::AgentBuilder::new()
.tls_config(Arc::new(cfg))
.timeout_connect(Duration::from_secs(5))
.timeout(Duration::from_secs(10))
.build())
}
/// Fetch the host's unified library. Errors are pre-classified for the UI (401/403 →
/// [`LibraryError::NotPaired`], a pin-verifier rejection → [`LibraryError::PinMismatch`]).
pub fn fetch_games(
addr: &str,
mgmt_port: u16,
identity: &(String, String),
pin: Option<[u8; 32]>,
) -> Result<Vec<GameEntry>, LibraryError> {
let agent = agent(identity, pin)?;
let url = format!("{}/api/v1/library", base_url(addr, mgmt_port));
let body = match agent.get(&url).call() {
Ok(resp) => resp
.into_string()
.map_err(|e| LibraryError::Unreachable(format!("read body: {e}")))?,
Err(e) => return Err(classify(e)),
};
serde_json::from_str(&body).map_err(|e| LibraryError::Unreachable(format!("bad JSON: {e}")))
}
/// Poster-art byte fetch cap — largest Steam hero assets run a few MB; anything bigger is
/// not an image we want to hand to the texture decoder.
const ART_MAX_BYTES: u64 = 16 * 1024 * 1024;
/// Fetch one cover-art image. URLs on the host itself (under `base`) go through the
/// pinned mTLS agent (the host's art proxy requires the paired cert); any other origin —
/// a public CDN URL on a custom entry — uses ureq's default agent with normal webpki
/// trust and no client cert (Apple's `LibraryTLSDelegate` does the same split).
pub fn fetch_art(pinned: &ureq::Agent, base: &str, url: &str) -> Result<Vec<u8>, LibraryError> {
let resp = if url.starts_with(base) {
pinned.get(url).call()
} else {
ureq::get(url).timeout(Duration::from_secs(10)).call()
}
.map_err(classify)?;
let mut bytes = Vec::new();
resp.into_reader()
.take(ART_MAX_BYTES)
.read_to_end(&mut bytes)
.map_err(|e| LibraryError::Unreachable(format!("read image: {e}")))?;
Ok(bytes)
}
/// Concurrent poster fetches — a handful is plenty for a LAN art proxy without turning a
/// big library into a connection burst.
const ART_WORKERS: usize = 3;
/// Fetch poster bytes for `jobs` (entry id → candidate URLs, walked in order until one
/// loads) on a small worker pool; results stream on the returned channel as they land.
/// Dropping the receiver (the consuming page popped) winds the workers down. Shared by
/// the touch grid and the gamepad launcher — the consumer does its own texture decode on
/// the main loop.
pub fn spawn_art_fetch(
base: String,
identity: (String, String),
pin: Option<[u8; 32]>,
jobs: VecDeque<(String, Vec<String>)>,
) -> async_channel::Receiver<(String, Vec<u8>)> {
let queue = Arc::new(Mutex::new(jobs));
let (tx, rx) = async_channel::unbounded::<(String, Vec<u8>)>();
for _ in 0..ART_WORKERS {
let queue = queue.clone();
let tx = tx.clone();
let base = base.clone();
let identity = identity.clone();
std::thread::Builder::new()
.name("punktfunk-lib-art".into())
.spawn(move || {
let Ok(agent) = agent(&identity, pin) else {
return;
};
loop {
let job = queue.lock().unwrap().pop_front();
let Some((id, candidates)) = job else { break };
for url in &candidates {
match fetch_art(&agent, &base, url) {
Ok(bytes) => {
// Receiver gone (page popped) — stop fetching.
if tx.send_blocking((id, bytes)).is_err() {
return;
}
break;
}
// 404 on a guessed CDN path is routine — try the next kind.
Err(e) => tracing::debug!(%id, url, error = %e, "poster miss"),
}
}
}
})
.expect("spawn art thread");
}
rx
}
fn classify(e: ureq::Error) -> LibraryError {
match e {
ureq::Error::Status(401 | 403, _) => LibraryError::NotPaired,
ureq::Error::Status(code, _) => LibraryError::Http(code),
ureq::Error::Transport(t) => {
// A pin rejection surfaces as a TLS alert wrapped in a transport error; the
// verifier's error kind survives in the message.
let msg = t.to_string();
if msg.contains("ApplicationVerificationFailure") || msg.contains("InvalidCertificate")
{
LibraryError::PinMismatch
} else {
LibraryError::Unreachable(msg)
}
}
}
}
/// Fingerprint-pinning verifier — the client-HTTP twin of core's (private) QUIC
/// `PinVerify`: trust is the SHA-256 of the host's self-signed leaf cert. The handshake
/// signatures MUST still be verified for real: CertificateVerify is what proves the peer
/// *holds the pinned cert's private key* — skip it and an active MITM can replay the
/// host's (public) certificate, match the pin, and complete the handshake with its own key.
#[derive(Debug)]
struct PinVerify {
pin: Option<[u8; 32]>,
}
impl rustls::client::danger::ServerCertVerifier for PinVerify {
fn verify_server_cert(
&self,
end_entity: &rustls::pki_types::CertificateDer<'_>,
_intermediates: &[rustls::pki_types::CertificateDer<'_>],
_server_name: &rustls::pki_types::ServerName<'_>,
_ocsp: &[u8],
_now: rustls::pki_types::UnixTime,
) -> Result<rustls::client::danger::ServerCertVerified, rustls::Error> {
if let Some(expected) = self.pin {
let fp = punktfunk_core::quic::endpoint::cert_fingerprint(end_entity.as_ref());
if fp != expected {
return Err(rustls::Error::InvalidCertificate(
rustls::CertificateError::ApplicationVerificationFailure,
));
}
}
Ok(rustls::client::danger::ServerCertVerified::assertion())
}
fn verify_tls12_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::crypto::verify_tls12_signature(
message,
cert,
dss,
&rustls::crypto::ring::default_provider().signature_verification_algorithms,
)
}
fn verify_tls13_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::crypto::verify_tls13_signature(
message,
cert,
dss,
&rustls::crypto::ring::default_provider().signature_verification_algorithms,
)
}
fn supported_verify_schemes(&self) -> Vec<rustls::SignatureScheme> {
rustls::crypto::ring::default_provider()
.signature_verification_algorithms
.supported_schemes()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn poster_candidates_order_and_resolution() {
// Fallback order is portrait → header → hero, host-relative paths resolved.
let art = Artwork {
portrait: Some("/api/v1/library/art/steam:570/portrait".into()),
hero: Some("https://cdn.example/hero.jpg".into()),
header: Some("/api/v1/library/art/steam:570/header".into()),
};
assert_eq!(
art.poster_candidates("https://192.168.1.42:47990"),
vec![
"https://192.168.1.42:47990/api/v1/library/art/steam:570/portrait",
"https://192.168.1.42:47990/api/v1/library/art/steam:570/header",
"https://cdn.example/hero.jpg",
]
);
assert!(Artwork::default()
.poster_candidates("https://h:47990")
.is_empty());
}
#[test]
fn game_entry_decodes_the_wire_shape() {
// The exact shape mgmt.rs serializes (optional art fields omitted, launch ignored).
let json = r#"[
{"id":"steam:570","store":"steam","title":"Dota 2",
"art":{"portrait":"/api/v1/library/art/steam:570/portrait"},
"launch":{"kind":"steam_appid","value":"570"}},
{"id":"custom:abc","store":"custom","title":"My Emu","art":{}}
]"#;
let games: Vec<GameEntry> = serde_json::from_str(json).unwrap();
assert_eq!(games.len(), 2);
assert_eq!(games[0].id, "steam:570");
assert!(games[1].art.portrait.is_none());
}
#[test]
fn ipv6_base_url_is_bracketed() {
assert_eq!(base_url("fe80::1", 47990), "https://[fe80::1]:47990");
assert_eq!(base_url("192.168.1.42", 1234), "https://192.168.1.42:1234");
}
}
clients/linux/src/main.rs
+6 -18
View File
@@ -5,27 +5,19 @@
//! mirrors the Apple client: persistent identity, TOFU prompt with the host fingerprint,
//! SPAKE2 PIN pairing.
// The UI-agnostic plumbing lives in `pf-client-core`, shared with the upcoming Vulkan
// session binary (design: punktfunk-planning linux-client-rearchitecture.md, Phase 0).
// Root re-exports keep every existing `crate::video`-style path resolving unchanged.
#[cfg(target_os = "linux")]
pub use pf_client_core::{audio, discovery, gamepad, keymap, library, session, trust, video, wol};
#[cfg(target_os = "linux")]
mod app;
#[cfg(target_os = "linux")]
mod audio;
#[cfg(target_os = "linux")]
mod cli;
#[cfg(target_os = "linux")]
mod discovery;
#[cfg(target_os = "linux")]
mod gamepad;
#[cfg(target_os = "linux")]
mod keymap;
#[cfg(target_os = "linux")]
mod launch;
#[cfg(target_os = "linux")]
mod library;
#[cfg(target_os = "linux")]
mod session;
#[cfg(target_os = "linux")]
mod trust;
#[cfg(target_os = "linux")]
mod ui_gamepad_library;
#[cfg(target_os = "linux")]
mod ui_hosts;
@@ -38,12 +30,8 @@ mod ui_stream;
#[cfg(target_os = "linux")]
mod ui_trust;
#[cfg(target_os = "linux")]
mod video;
#[cfg(target_os = "linux")]
mod video_gl;
mod wol;
#[cfg(target_os = "linux")]
fn main() -> gtk::glib::ExitCode {
app::run()
clients/linux/src/session.rs
-498
View File
@@ -1,498 +0,0 @@
//! Session controller: the worker thread runs connect → pump (video pull + decode +
//! stats), a dedicated audio thread pulls + Opus-decodes the audio plane (Apple
//! `SessionAudio` parity — audio never waits behind a video decode), both feeding the GTK
//! main loop / PipeWire over channels. The UI keeps the `Arc<NativeClient>` from the
//! `Connected` event for direct input sends (no extra hop on the input path) —
//! `NativeClient` is `Sync`, planes stay one-consumer-per-thread: video here, audio on
//! its own thread, rumble+hidout on the gamepad thread.
use crate::audio;
use crate::video::{DecodedFrame, DecodedImage, Decoder};
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use punktfunk_core::PunktfunkError;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
pub struct SessionParams {
pub host: String,
pub port: u16,
pub mode: Mode,
pub compositor: CompositorPref,
pub gamepad: GamepadPref,
pub bitrate_kbps: u32,
/// Requested audio channel count (2/6/8); the host echoes the resolved value.
pub audio_channels: u8,
/// The user's preferred video codec (a `quic::CODEC_*` bit, `0` = auto). Soft — the host honors
/// it when it can emit it, else falls back; the resolved codec drives the decoder.
pub preferred_codec: u8,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see
/// `video::Decoder::new`).
pub decoder: String,
/// Library id for the host to launch this session (`"steam:570"`, from the library
/// page); `None` = plain desktop session.
pub launch: Option<String>,
/// Pinned host fingerprint; `None` = trust on first use (caller persists the observed one).
pub pin: Option<[u8; 32]>,
pub identity: (String, String),
/// How long to wait for the handshake. The normal path uses a short budget; the
/// "request access" (delegated-approval) path uses a long one, because the host PARKS the
/// connection until the operator clicks Approve in its console (so this must exceed the
/// host's approval window — see `PENDING_APPROVAL_WAIT`).
pub connect_timeout: Duration,
/// Raised by the PRESENTER when hardware frames can't be displayed (GL converter init
/// failed / dmabuf import rejected): the pump demotes the decoder to software and
/// re-requests a keyframe. Decode itself succeeds in that state, so nothing else
/// would recover — without this the stream stays black.
pub force_software: Arc<AtomicBool>,
}
/// The session pump's share of the unified stats window (design/stats-unification.md):
/// stream facts plus the two stages measured before the presenter. The frame consumer in
/// `ui_stream` contributes the `display` stage and the end-to-end percentiles.
#[derive(Clone, Copy, Default)]
pub struct Stats {
/// AUs received (reassembled) per second, actual-elapsed-time denominator.
pub fps: f32,
/// Received payload bytes × 8 / elapsed (goodput, excludes FEC overhead).
pub mbps: f32,
/// p50 `host+network` stage: capture → received, host-clock corrected (ms).
pub host_net_ms: f32,
/// p50 `host` stage: the host's own capture→fully-sent, from the per-AU 0xCF host
/// timings (design/stats-unification.md Phase 2). Valid only when `split`.
pub host_ms: f32,
/// p50 `network` stage: capture→received minus the host-reported share
/// (`hostnet − host`, per-frame, saturating). Valid only when `split`.
pub net_ms: f32,
/// The window had matched host timings — the OSD splits `host+network` into
/// `host + network`. An old host never emits 0xCF, so this stays false and the
/// combined stage renders unchanged.
pub split: bool,
/// p50 `decode` stage: received → decoded, single-clock client-local (ms).
pub decode_ms: f32,
/// Unrecoverable network frame drops this window, and their share of
/// received+lost (%). The OSD renders the counter line only when nonzero.
pub lost: u32,
pub lost_pct: f32,
/// The decode path frames actually took this window (`"vaapi"`/`"software"`, empty
/// until the first frame) — the OSD's trailing tag; tracks a mid-session fallback.
pub decoder: &'static str,
}
/// Frames the pump keeps waiting for their 0xCF host timing (pts → capture→received µs).
/// ~2 s at 120 Hz — a timing arrives within a frame or two of its AU, and against an old
/// host (no 0xCF at all) this just caps the dead-weight ring.
const PENDING_SPLIT_CAP: usize = 256;
/// Sort a window of µs samples in place and return `(p50, p95)` per the spec's index
/// rules (`sorted[len/2]`, `sorted[min(len*95/100, len-1)]`); an empty window reads 0.
pub fn window_percentiles(samples: &mut [u64]) -> (u64, u64) {
if samples.is_empty() {
return (0, 0);
}
samples.sort_unstable();
let p50 = samples[samples.len() / 2];
let p95 = samples[(samples.len() * 95 / 100).min(samples.len() - 1)];
(p50, p95)
}
pub enum SessionEvent {
Connected {
connector: Arc<NativeClient>,
mode: Mode,
fingerprint: [u8; 32],
},
/// `trust_rejected` is set when the connect failed the TLS trust check (a `Crypto`
/// error): for a pinned connect this is the fingerprint-changed signal, so the UI can
/// offer a re-pair (PIN) path rather than a dead-end error.
Failed {
msg: String,
trust_rejected: bool,
},
Ended(Option<String>),
Stats(Stats),
}
pub struct SessionHandle {
pub events: async_channel::Receiver<SessionEvent>,
pub frames: async_channel::Receiver<DecodedFrame>,
pub stop: Arc<AtomicBool>,
}
pub fn start(params: SessionParams) -> SessionHandle {
let (ev_tx, ev_rx) = async_channel::unbounded();
// Tiny frame queue, newest wins: force_send displaces the oldest when the UI lags.
let (frame_tx, frame_rx) = async_channel::bounded(2);
let stop = Arc::new(AtomicBool::new(false));
let stop_w = stop.clone();
std::thread::Builder::new()
.name("punktfunk-session".into())
.spawn(move || pump(params, ev_tx, frame_tx, stop_w))
.expect("spawn session thread");
SessionHandle {
events: ev_rx,
frames: frame_rx,
stop,
}
}
pub fn now_ns() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as u64)
.unwrap_or(0)
}
/// Opus decoder for the audio plane: a plain stereo decoder (the validated path) or a multistream
/// decoder for 5.1/7.1, both behind one `decode_float`. Built from the host-RESOLVED channel count
/// via the shared layout table.
enum AudioDec {
Stereo(opus::Decoder),
Surround(opus::MSDecoder),
}
impl AudioDec {
fn new(channels: u8) -> Result<AudioDec, opus::Error> {
if channels == 2 {
Ok(AudioDec::Stereo(opus::Decoder::new(
48_000,
opus::Channels::Stereo,
)?))
} else {
let l = punktfunk_core::audio::layout_for(channels, false);
Ok(AudioDec::Surround(opus::MSDecoder::new(
48_000, l.streams, l.coupled, l.mapping,
)?))
}
}
fn decode_float(
&mut self,
input: &[u8],
out: &mut [f32],
fec: bool,
) -> Result<usize, opus::Error> {
match self {
AudioDec::Stereo(d) => d.decode_float(input, out, fec),
AudioDec::Surround(d) => d.decode_float(input, out, fec),
}
}
}
fn pump(
params: SessionParams,
ev_tx: async_channel::Sender<SessionEvent>,
frame_tx: async_channel::Sender<DecodedFrame>,
stop: Arc<AtomicBool>,
) {
let connector = match NativeClient::connect(
&params.host,
params.port,
params.mode,
params.compositor,
params.gamepad,
params.bitrate_kbps,
0, // video_caps: the Linux client has no 10-bit/HDR present path yet
params.audio_channels,
crate::video::decodable_codecs(), // codecs FFmpeg can decode (HEVC/H.264/AV1)
params.preferred_codec, // the user's soft codec preference (0 = auto)
params.launch.clone(),
params.pin,
Some(params.identity),
params.connect_timeout,
) {
Ok(c) => Arc::new(c),
Err(e) => {
let trust_rejected = matches!(e, PunktfunkError::Crypto);
let msg = match e {
PunktfunkError::Crypto => {
"Host identity rejected — wrong fingerprint, or the host requires pairing"
.to_string()
}
PunktfunkError::Timeout => "Connection timed out".to_string(),
other => format!("Connect failed: {other:?}"),
};
let _ = ev_tx.send_blocking(SessionEvent::Failed {
msg,
trust_rejected,
});
return;
}
};
let _ = ev_tx.send_blocking(SessionEvent::Connected {
connector: connector.clone(),
mode: connector.mode(),
fingerprint: connector.host_fingerprint,
});
// Build the decoder for the codec the host resolved (never assume HEVC), honoring the
// Settings backend preference (auto/vaapi/software).
let codec_id = crate::video::ffmpeg_codec_id(connector.codec);
tracing::info!(
?codec_id,
welcome_codec = connector.codec,
"negotiated video codec"
);
let mut decoder = match Decoder::new(codec_id, &params.decoder) {
Ok(d) => d,
Err(e) => {
let _ = ev_tx.send_blocking(SessionEvent::Ended(Some(format!("video decoder: {e}"))));
return;
}
};
let force_software = params.force_software.clone();
// Audio is best-effort: a session without it still streams. Gamepads are the
// app-lifetime service's job (the UI attaches it on Connected). Audio runs on its own
// thread (one puller per plane), blocking on the audio queue like the Apple client.
let audio_thread = spawn_audio(connector.clone(), stop.clone());
let _mic = params
.mic_enabled
.then(|| {
audio::MicStreamer::spawn(connector.clone())
.map_err(|e| tracing::warn!(error = %e, "mic uplink disabled"))
.ok()
})
.flatten();
let clock_offset = connector.clock_offset_ns;
let mut total_frames = 0u64;
let mut window_start = Instant::now();
let mut frames_n = 0u32;
let mut bytes_n = 0u64;
// Stage windows (µs samples): `host+network` = capture→received (host-clock
// corrected), `decode` = received→decoded (client-local). p50 per 1 s window.
let mut hostnet_us: Vec<u64> = Vec::with_capacity(256);
let mut decode_us: Vec<u64> = Vec::with_capacity(256);
// Host/network split (Phase 2): frames awaiting their per-AU 0xCF host timing,
// correlated by pts_ns. Bounded — an old host never sends any, so entries just age out.
let mut pending_split: std::collections::VecDeque<(u64, u64)> =
std::collections::VecDeque::with_capacity(PENDING_SPLIT_CAP);
let mut host_us_win: Vec<u64> = Vec::with_capacity(256);
let mut net_us_win: Vec<u64> = Vec::with_capacity(256);
// What actually decoded the last frame — a VAAPI failure demotes mid-session, so
// this is read off each frame's image variant rather than fixed at startup.
let mut dec_path: &'static str = "";
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut last_dropped = connector.frames_dropped();
// The stats window keeps its own drop cursor — the OSD shows the per-window delta.
let mut window_dropped = last_dropped;
let mut last_kf_req: Option<Instant> = None;
let end: Option<String> = loop {
if stop.load(Ordering::SeqCst) {
break None;
}
// 20 ms wait: audio has its own thread now, so this only bounds stop-flag
// responsiveness and the per-iteration keyframe-recovery check (a frame arrives
// every ~8–16 ms at 60–120 Hz anyway, so this rarely times out mid-stream).
match connector.next_frame(Duration::from_millis(20)) {
Ok(frame) => {
// The `received` point: AU fully reassembled, in hand, before decode.
let received_ns = now_ns();
// fps / goodput count every received AU (spec), decoded or not.
frames_n += 1;
bytes_n += frame.data.len() as u64;
match decoder.decode(&frame.data) {
Ok(Some(image)) => {
total_frames += 1;
dec_path = match &image {
DecodedImage::Cpu(_) => "software",
DecodedImage::Dmabuf(_) => "vaapi",
};
if total_frames == 1 {
let (w, h, path) = match &image {
DecodedImage::Cpu(c) => (c.width, c.height, "software"),
DecodedImage::Dmabuf(d) => (d.width, d.height, "vaapi-dmabuf"),
};
tracing::info!(width = w, height = h, path, "first frame decoded");
}
// The `decoded` point — travels with the frame so the presenter
// can measure its `display` stage against it.
let decoded_ns = now_ns();
// `host+network` stage: received expressed in the host's capture
// clock, minus the host-stamped capture pts (clamped (0, 10 s)).
let hn = (received_ns as i128 + clock_offset as i128 - frame.pts_ns as i128)
.max(0) as u64;
if hn > 0 && hn < 10_000_000_000 {
hostnet_us.push(hn / 1000);
// Remember the sample for the host/network split — matched
// against the AU's 0xCF host timing when it arrives.
if pending_split.len() >= PENDING_SPLIT_CAP {
pending_split.pop_front();
}
pending_split.push_back((frame.pts_ns, hn / 1000));
}
// `decode` stage: received→decoded, single clock, no skew.
decode_us.push(decoded_ns.saturating_sub(received_ns) / 1000);
let _ = frame_tx.force_send(DecodedFrame {
pts_ns: frame.pts_ns,
decoded_ns,
image,
});
}
Ok(None) => {}
// Survivable (loss until the next IDR/RFI recovery) — keep feeding.
Err(e) => tracing::debug!(error = %e, "decode error (recovering)"),
}
// The presenter's verdict: hardware frames can't be displayed (GL converter
// init failed / dmabuf import rejected) — demote to software here, on the
// decoder's own thread. Decode succeeds in that state, so the error-streak
// demotion above never fires.
if force_software.swap(false, Ordering::Relaxed) {
if let Err(e) = decoder.force_software() {
break Some(format!("software decoder rebuild: {e}"));
}
}
// A decode error / VAAPI→software demotion asks for a fresh IDR: the infinite
// GOP has no periodic keyframe, so a rebuilt/erroring decoder would stay
// gray/frozen until an unrelated packet drop happened to request one. Route it
// through the same throttle as loss recovery below.
if decoder.take_keyframe_request() {
let now = Instant::now();
if last_kf_req
.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100))
{
last_kf_req = Some(now);
let _ = connector.request_keyframe();
tracing::debug!("requested keyframe (decoder recovery)");
}
}
}
Err(PunktfunkError::NoFrame) => {}
Err(PunktfunkError::Closed) => break Some("Host ended the session".to_string()),
Err(e) => break Some(format!("session: {e:?}")),
}
// Drain the per-AU host timings (0xCF) non-blockingly and match them to received
// frames by pts: host = the host's own capture→sent, network = our
// capture→received minus it (the two tile per frame by construction). An old
// host never emits any — the deque fills to its cap and the OSD keeps the
// combined `host+network` stage.
while let Ok(t) = connector.next_host_timing(Duration::ZERO) {
if let Some(i) = pending_split.iter().position(|(p, _)| *p == t.pts_ns) {
let (_, hn_us) = pending_split.remove(i).unwrap();
host_us_win.push(t.host_us as u64);
net_us_win.push(hn_us.saturating_sub(t.host_us as u64));
}
}
// Loss recovery: under infinite GOP the only recovery keyframe is one we request. The
// reassembler drops unrecoverable AUs (frames_dropped); the decoder then conceals the
// reference-missing delta frames that follow and returns Ok, so keying off a decode error
// rarely fires. Request an IDR when the drop count climbs, throttled — the decode stays
// wedged for several frames until the IDR lands, so requesting every frame would flood.
let dropped = connector.frames_dropped();
if dropped > last_dropped {
last_dropped = dropped;
let now = Instant::now();
if last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100)) {
last_kf_req = Some(now);
let _ = connector.request_keyframe();
tracing::debug!(dropped, "requested keyframe (loss recovery)");
}
}
if window_start.elapsed() >= Duration::from_secs(1) {
let secs = window_start.elapsed().as_secs_f32();
let (hn_p50, _) = window_percentiles(&mut hostnet_us);
let (dec_p50, _) = window_percentiles(&mut decode_us);
// Host/network split — present only when this window matched 0xCF timings.
let split = !host_us_win.is_empty();
let (host_p50, _) = window_percentiles(&mut host_us_win);
let (net_p50, _) = window_percentiles(&mut net_us_win);
let lost = dropped.saturating_sub(window_dropped) as u32;
window_dropped = dropped;
tracing::debug!(
fps = frames_n,
hostnet_p50_us = hn_p50,
host_p50_us = host_p50,
net_p50_us = net_p50,
decode_p50_us = dec_p50,
lost,
total_frames,
"stream window"
);
let _ = ev_tx.try_send(SessionEvent::Stats(Stats {
fps: frames_n as f32 / secs,
mbps: bytes_n as f32 * 8.0 / 1e6 / secs,
host_net_ms: hn_p50 as f32 / 1000.0,
host_ms: host_p50 as f32 / 1000.0,
net_ms: net_p50 as f32 / 1000.0,
split,
decode_ms: dec_p50 as f32 / 1000.0,
lost,
lost_pct: if lost > 0 {
lost as f32 * 100.0 / (frames_n + lost) as f32
} else {
0.0
},
decoder: dec_path,
}));
window_start = Instant::now();
frames_n = 0;
bytes_n = 0;
hostnet_us.clear();
decode_us.clear();
host_us_win.clear();
net_us_win.clear();
}
};
tracing::info!(
total_frames,
reason = end.as_deref().unwrap_or("user"),
"session ended"
);
stop.store(true, Ordering::SeqCst);
if let Some(t) = audio_thread {
let _ = t.join(); // exits within its 100 ms pull timeout once `stop` is set
}
let _ = ev_tx.send_blocking(SessionEvent::Ended(end));
}
/// The dedicated audio thread: owns the Opus decoder, the PCM scratch, and the PipeWire
/// player, and blocks on `next_audio` (the plane's single consumer — packets land every
/// 5 ms). Decoded chunks are pushed in Vecs recycled from the player's pool, so the
/// steady state allocates nothing. Best-effort like before: any setup failure logs and
/// the session streams video-only. Exits on the stop flag or a closed plane.
fn spawn_audio(
connector: Arc<NativeClient>,
stop: Arc<AtomicBool>,
) -> Option<std::thread::JoinHandle<()>> {
// Decoder + playback are built from the host-RESOLVED channel count (never the
// request), so an older/clamping host that resolves stereo is decoded as stereo.
let channels = connector.audio_channels;
let player = audio::AudioPlayer::spawn(channels as u32)
.map_err(|e| tracing::warn!(error = %e, "audio disabled"))
.ok()?;
let mut dec = AudioDec::new(channels)
.map_err(|e| tracing::warn!(error = %e, "opus decoder failed — audio disabled"))
.ok()?;
std::thread::Builder::new()
.name("punktfunk-audio-rx".into())
.spawn(move || {
let mut pcm = vec![0f32; 5760 * channels as usize]; // scratch: max Opus frame (120 ms) × channels
while !stop.load(Ordering::SeqCst) {
match connector.next_audio(Duration::from_millis(100)) {
Ok(pkt) => match dec.decode_float(&pkt.data, &mut pcm, false) {
// `samples` is per-channel; the interleaved frame is `samples * channels`.
Ok(samples) => {
let n = samples * channels as usize;
let mut buf = player.take_buffer();
buf.extend_from_slice(&pcm[..n]);
player.push(buf);
}
Err(e) => tracing::debug!(error = %e, "opus decode"),
},
Err(PunktfunkError::NoFrame) => {}
Err(_) => break, // plane closed — the session is ending
}
}
tracing::debug!("audio pull thread exited");
})
.map_err(|e| tracing::warn!(error = %e, "audio thread failed to start — audio disabled"))
.ok()
}
clients/linux/src/trust.rs
-342
View File
@@ -1,342 +0,0 @@
//! Client identity, the known-hosts (pinned fingerprint) store, and app settings.
//!
//! The identity shares `~/.config/punktfunk/client-{cert,key}.pem` with `punktfunk-probe`
//! so a box pairs once whichever client it uses.
use anyhow::{anyhow, Context, Result};
use punktfunk_core::client::NativeClient;
use punktfunk_core::quic::endpoint;
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
pub fn config_dir() -> Result<PathBuf> {
let home = std::env::var("HOME").context("HOME unset")?;
Ok(PathBuf::from(home).join(".config/punktfunk"))
}
/// This client's persistent identity, generated on first use — presented on every connect
/// so hosts can recognize it once paired.
pub fn load_or_create_identity() -> Result<(String, String)> {
let dir = config_dir()?;
let (cp, kp) = (dir.join("client-cert.pem"), dir.join("client-key.pem"));
if let (Ok(c), Ok(k)) = (std::fs::read_to_string(&cp), std::fs::read_to_string(&kp)) {
return Ok((c, k));
}
let (c, k) = endpoint::generate_identity().map_err(|e| anyhow!("generate identity: {e}"))?;
std::fs::create_dir_all(&dir)?;
std::fs::write(&cp, &c)?;
std::fs::write(&kp, &k)?;
tracing::info!(cert = %cp.display(), "generated client identity");
Ok((c, k))
}
pub fn hex(fp: &[u8; 32]) -> String {
fp.iter().map(|b| format!("{b:02x}")).collect()
}
pub fn parse_hex32(s: &str) -> Option<[u8; 32]> {
if s.len() != 64 {
return None;
}
let mut out = [0u8; 32];
for (i, b) in out.iter_mut().enumerate() {
*b = u8::from_str_radix(&s[2 * i..2 * i + 2], 16).ok()?;
}
Some(out)
}
/// One trusted host: its pinned certificate fingerprint plus how we got there (TOFU or a
/// PIN ceremony) and where we last reached it.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct KnownHost {
pub name: String,
pub addr: String,
pub port: u16,
/// SHA-256 of the host certificate, lowercase hex — the pin for every later connect.
pub fp_hex: String,
/// True if trust came from the SPAKE2 PIN ceremony (vs. trust-on-first-use).
pub paired: bool,
/// Unix seconds of the last successful connect — the hosts page marks the
/// most-recent card with the accent bar. `default` so pre-existing stores load.
#[serde(default)]
pub last_used: Option<u64>,
/// Wake-on-LAN MAC(s) (`aa:bb:cc:dd:ee:ff`) learned from the host's mDNS `mac` TXT while it
/// was online, so we can wake it once it sleeps and stops advertising. `default` so
/// pre-existing stores load; empty until first learned.
#[serde(default)]
pub mac: Vec<String>,
}
#[derive(Default, Serialize, Deserialize)]
pub struct KnownHosts {
pub hosts: Vec<KnownHost>,
}
impl KnownHosts {
fn path() -> Result<PathBuf> {
Ok(config_dir()?.join("client-known-hosts.json"))
}
pub fn load() -> KnownHosts {
Self::path()
.and_then(|p| Ok(std::fs::read_to_string(p)?))
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default()
}
pub fn save(&self) -> Result<()> {
let p = Self::path()?;
std::fs::create_dir_all(p.parent().unwrap())?;
std::fs::write(&p, serde_json::to_string_pretty(self)?)?;
Ok(())
}
pub fn find_by_fp(&self, fp_hex: &str) -> Option<&KnownHost> {
self.hosts.iter().find(|h| h.fp_hex == fp_hex)
}
pub fn find_by_addr(&self, addr: &str, port: u16) -> Option<&KnownHost> {
self.hosts.iter().find(|h| h.addr == addr && h.port == port)
}
/// Forget the entry with this fingerprint. Returns true if one was removed (the user
/// will have to pair/trust again to reconnect).
pub fn remove_by_fp(&mut self, fp_hex: &str) -> bool {
let before = self.hosts.len();
self.hosts.retain(|h| h.fp_hex != fp_hex);
self.hosts.len() != before
}
/// Insert or refresh an entry, keyed by fingerprint. `paired` only ever upgrades
/// (a later TOFU connect must not demote a PIN-paired host).
pub fn upsert(&mut self, entry: KnownHost) {
if let Some(h) = self.hosts.iter_mut().find(|h| h.fp_hex == entry.fp_hex) {
h.name = entry.name;
h.addr = entry.addr;
h.port = entry.port;
h.paired |= entry.paired;
// A refresh without a timestamp must not erase the stored one.
if entry.last_used.is_some() {
h.last_used = entry.last_used;
}
// Likewise a trust-decision upsert (which carries no MAC) must not wipe learned MACs.
if !entry.mac.is_empty() {
h.mac = entry.mac;
}
} else {
self.hosts.push(entry);
}
}
}
/// Load-upsert-save in one step — the pin every trust decision (TOFU accept, PIN
/// ceremony, delegated approval, headless pairing) ends in.
pub fn persist_host(name: &str, addr: &str, port: u16, fp_hex: &str, paired: bool) {
let mut known = KnownHosts::load();
known.upsert(KnownHost {
name: name.to_string(),
addr: addr.to_string(),
port,
fp_hex: fp_hex.to_string(),
paired,
last_used: None,
mac: Vec::new(),
});
let _ = known.save();
}
/// Learn/refresh a saved host's Wake-on-LAN MAC(s) from its live advert (called while the host
/// is online, matched by fingerprint or address). No-op — and no disk write — when unchanged, so
/// the hosts page can call it on every discovery tick without churning the store.
pub fn learn_mac(fp_hex: &str, addr: &str, port: u16, mac: &[String]) {
if mac.is_empty() {
return;
}
let mut known = KnownHosts::load();
let Some(h) = known
.hosts
.iter_mut()
.find(|h| (!fp_hex.is_empty() && h.fp_hex == fp_hex) || (h.addr == addr && h.port == port))
else {
return;
};
if h.mac == mac {
return;
}
h.mac = mac.to_vec();
let _ = known.save();
}
/// Re-key a saved host's address/port after it rediscovered on a new DHCP lease (matched by
/// fingerprint). No-op — and no disk write — when unchanged. Called from the wake-and-wait flow when
/// a woken host reappears on a different IP than the stored one, so this and future connects dial the
/// live address instead of the stale one.
pub fn rekey_addr(fp_hex: &str, addr: &str, port: u16) {
if fp_hex.is_empty() {
return;
}
let mut known = KnownHosts::load();
let Some(h) = known.hosts.iter_mut().find(|h| h.fp_hex == fp_hex) else {
return;
};
if h.addr == addr && h.port == port {
return;
}
h.addr = addr.to_string();
h.port = port;
let _ = known.save();
}
/// Stamp "now" as this host's last successful connect (drives the hosts page's
/// most-recent accent). No-op when the fingerprint isn't stored.
pub fn touch_last_used(fp_hex: &str) {
let mut known = KnownHosts::load();
if let Some(h) = known.hosts.iter_mut().find(|h| h.fp_hex == fp_hex) {
h.last_used = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs())
.ok();
let _ = known.save();
}
}
/// Run the SPAKE2 PIN ceremony against a host. `device_name` is the label the HOST
/// stores this client under (its paired-devices list); the 90 s budget covers a
/// human-typed PIN. Returns the host's now-verified certificate fingerprint to pin.
pub fn pair_with_host(
addr: &str,
port: u16,
identity: &(String, String),
pin: &str,
device_name: &str,
) -> std::result::Result<[u8; 32], punktfunk_core::PunktfunkError> {
NativeClient::pair(
addr,
port,
(&identity.0, &identity.1),
pin.trim(),
device_name,
std::time::Duration::from_secs(90),
)
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)]
#[serde(default)]
pub struct Settings {
/// Stream mode; `0` = the native size/refresh of the monitor the window is on,
/// resolved at connect time.
pub width: u32,
pub height: u32,
pub refresh_hz: u32,
/// Requested encoder bitrate (kbps); 0 = host default.
pub bitrate_kbps: u32,
pub gamepad: String,
/// Stable identity (`vid:pid:name`, see `PadInfo::key`) of the physical controller
/// forwarded as pad 0; empty = automatic (most recently connected). Applied to the
/// gamepad service at startup so the choice survives restarts.
pub forward_pad: String,
/// Which host compositor backend to request (advisory; the host falls back to
/// auto-detect when unavailable).
pub compositor: String,
/// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured.
pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the decoder + playback layout.
pub audio_channels: u8,
/// Preferred video codec: `"auto"` (host decides), `"hevc"`, `"h264"`, or `"av1"`. A soft
/// preference — the host honors it when it can emit it, else falls back to the best shared codec.
#[serde(default = "default_codec")]
pub codec: String,
/// Video decoder preference: `"auto"` (VAAPI → software), `"vaapi"`, `"software"`.
/// The `PUNKTFUNK_DECODER` env var overrides this (see `video::Decoder::new`).
pub decoder: String,
/// Show the on-stream statistics overlay (toggle live with Ctrl+Alt+Shift+S).
pub show_stats: bool,
/// Enter fullscreen when a stream starts (F11 / the controller chord / the top-edge
/// header reveal exit it). Gaming-Mode launches (`--fullscreen`) fullscreen regardless.
pub fullscreen_on_stream: bool,
/// Experimental: the game-library browser ("Browse library…" on saved cards) —
/// mirrors the Apple client's "Show game library" toggle, default off.
pub library_enabled: bool,
}
fn default_codec() -> String {
"auto".into()
}
impl Settings {
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 {
match self.codec.as_str() {
"h264" | "avc" => punktfunk_core::quic::CODEC_H264,
"hevc" | "h265" => punktfunk_core::quic::CODEC_HEVC,
"av1" => punktfunk_core::quic::CODEC_AV1,
_ => 0,
}
}
}
impl Default for Settings {
fn default() -> Self {
Settings {
width: 0,
height: 0,
refresh_hz: 0,
bitrate_kbps: 0,
gamepad: "auto".into(),
forward_pad: String::new(),
compositor: "auto".into(),
inhibit_shortcuts: true,
mic_enabled: false,
audio_channels: 2,
codec: "auto".into(),
decoder: "auto".into(),
show_stats: true,
fullscreen_on_stream: true,
library_enabled: false,
}
}
}
impl Settings {
fn path() -> Result<PathBuf> {
Ok(config_dir()?.join("client-gtk-settings.json"))
}
pub fn load() -> Settings {
Self::path()
.and_then(|p| Ok(std::fs::read_to_string(p)?))
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default()
}
pub fn save(&self) {
let Ok(p) = Self::path() else { return };
let _ = std::fs::create_dir_all(p.parent().unwrap());
if let Ok(s) = serde_json::to_string_pretty(self) {
let _ = std::fs::write(&p, s);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// A pre-`forward_pad` settings file (≤ 0.5.0) loads with the pin on automatic.
#[test]
fn settings_forward_pad_defaults_empty() {
let old = r#"{"width":1280,"height":720,"refresh_hz":60,"bitrate_kbps":0,
"gamepad":"auto","compositor":"auto","inhibit_shortcuts":true,"mic_enabled":true}"#;
let s: Settings = serde_json::from_str(old).unwrap();
assert_eq!(s.forward_pad, "");
let round: Settings = serde_json::from_str(&serde_json::to_string(&s).unwrap()).unwrap();
assert_eq!(round.forward_pad, "");
}
}
clients/linux/src/video.rs
-718
View File
@@ -1,718 +0,0 @@
//! Video decode: reassembled HEVC access units → frames for the GTK presenter.
//!
//! Two backends, picked at session start (override: `PUNKTFUNK_DECODER=software|vaapi`):
//!
//! * **VAAPI** (Intel/AMD): libavcodec hwaccel decodes on the GPU; each frame is mapped
//! to a DRM-PRIME dmabuf (`av_hwframe_map`, zero copy) and handed to the UI as fds +
//! plane layout for `GdkDmabufTextureBuilder` — inside `GtkGraphicsOffload` that is the
//! decoder-to-subsurface path, direct-scanout eligible when fullscreen. NVIDIA boxes
//! have no usable VAAPI (nvidia-vaapi-driver is broken for this — Moonlight blacklists
//! it); device creation fails there and the software path takes over. A mid-session
//! VAAPI error also falls back — the host's IDR/RFI recovery resynchronizes.
//! * **Software**: libavcodec on the CPU + swscale to RGBA (`GdkMemoryTexture` upload).
//! Slice threading only — frame threading would add a frame of latency per thread.
//!
//! Both run `AV_CODEC_FLAG_LOW_DELAY`; the host encodes zero-reorder streams (no
//! B-frames, in-band parameter sets on every IDR), so decode is strictly one-in/one-out.
use anyhow::{anyhow, bail, Context as _, Result};
use ffmpeg::format::Pixel;
use ffmpeg::software::scaling;
use ffmpeg::util::frame::Video as AvFrame;
use ffmpeg_next as ffmpeg;
use std::os::fd::RawFd;
use std::ptr;
/// One decoded frame headed for the presenter, carrying the host capture timestamp so the
/// UI can measure capture→displayed latency at the moment it presents.
pub struct DecodedFrame {
/// Host-clock capture pts (ns) of the AU this image decoded from — compare against
/// the local wall clock + `clock_offset_ns` at paintable-set time.
pub pts_ns: u64,
/// Local wall clock (ns) when the decoder emitted this image — the `decoded`
/// measurement point (design/stats-unification.md); the presenter subtracts it from
/// its paintable-set stamp for the client-local `display` stage.
pub decoded_ns: u64,
pub image: DecodedImage,
}
pub enum DecodedImage {
Cpu(CpuFrame),
Dmabuf(DmabufFrame),
}
/// The stream's colour signaling, read PER-FRAME from the decoder (HEVC VUI → the
/// `AVFrame` CICP fields). The Windows host switches an HDR desktop to Main10 BT.2020 PQ
/// **in-band** (the Welcome still says SDR — clients are expected to follow the VUI, as
/// the Windows/Apple/Android clients do), so rendering must follow the frames, not the
/// handshake — else PQ content drawn as BT.709 comes out washed out and desaturated.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub struct ColorDesc {
/// H.273 code points as signaled (2 = unspecified → the renderer picks the SDR default).
pub primaries: u8,
pub transfer: u8,
pub matrix: u8,
pub full_range: bool,
}
impl ColorDesc {
/// Read the CICP fields off a raw decoded frame.
///
/// # Safety
/// `frame` must point to a valid `AVFrame` (alive for the duration of the call).
unsafe fn from_raw(frame: *const ffmpeg::ffi::AVFrame) -> ColorDesc {
// SAFETY: caller guarantees a live AVFrame; these are plain enum field reads.
unsafe {
ColorDesc {
primaries: (*frame).color_primaries as u32 as u8,
transfer: (*frame).color_trc as u32 as u8,
matrix: (*frame).colorspace as u32 as u8,
full_range: (*frame).color_range == ffmpeg::ffi::AVColorRange::AVCOL_RANGE_JPEG,
}
}
}
/// PQ (SMPTE ST.2084) transfer — the HDR10 signal.
pub fn is_pq(&self) -> bool {
self.transfer == 16
}
}
/// RGBA pixels for `GdkMemoryTexture` (which takes a stride).
pub struct CpuFrame {
pub width: u32,
pub height: u32,
/// RGBA row stride in bytes (≥ width*4 — swscale pads rows for SIMD).
pub stride: usize,
pub rgba: Vec<u8>,
/// Signaling of the source frame. swscale already undid the YUV matrix + range (the
/// pixels are full-range RGB), but a PQ/BT.2020 stream keeps its transfer + primaries
/// baked in — the presenter tags the texture so GTK tone-maps it.
pub color: ColorDesc,
}
/// A decoded frame still on the GPU: dmabuf fds + plane layout for
/// `GdkDmabufTextureBuilder`. The fds belong to `guard`'s mapped DRM frame — they stay
/// valid until the guard drops (the texture's release func).
pub struct DmabufFrame {
pub width: u32,
pub height: u32,
/// Combined DRM fourcc of the whole surface (NV12 for 8-bit VAAPI output), derived
/// from the decoder's software format — NOT the per-plane component formats.
pub fourcc: u32,
pub modifier: u64,
pub planes: Vec<DmabufPlane>,
/// Signaling of the source frame — drives the `GdkDmabufTexture` color state (BT.709
/// narrow for SDR, BT.2020 PQ for an HDR stream).
pub color: ColorDesc,
pub guard: DrmFrameGuard,
}
pub struct DmabufPlane {
pub fd: RawFd,
pub offset: u32,
pub stride: u32,
}
/// Owns the mapped DRM-PRIME `AVFrame` (which in turn references the VAAPI surface).
/// Dropping it releases the surface back to the decoder pool and closes the fds.
pub struct DrmFrameGuard(*mut ffmpeg::ffi::AVFrame);
// An AVFrame is plain refcounted data; freeing it from the GTK main thread is fine.
unsafe impl Send for DrmFrameGuard {}
impl Drop for DrmFrameGuard {
fn drop(&mut self) {
unsafe { ffmpeg::ffi::av_frame_free(&mut self.0) };
}
}
enum Backend {
Vaapi(VaapiDecoder),
Software(SoftwareDecoder),
}
pub struct Decoder {
backend: Backend,
/// The negotiated codec (from the host's Welcome), so a mid-session VAAPI→software demotion
/// rebuilds the software decoder for the SAME codec.
codec_id: ffmpeg::codec::Id,
/// Consecutive VAAPI decode errors — a single transient failure (e.g. a reference-missing
/// frame after packet loss) shouldn't cost the whole session its hardware decoder.
vaapi_fails: u32,
/// Set when the decoder needs a fresh IDR to resynchronize (after an error or a demotion).
/// The pump drains it and asks the host — under the infinite GOP there is no periodic
/// keyframe, so a rebuilt/erroring decoder would otherwise stay gray/frozen forever.
want_keyframe: bool,
}
/// Demote VAAPI→software only after this many consecutive hardware decode errors; a lone
/// transient error just re-requests an IDR and keeps the hardware decoder.
const VAAPI_DEMOTE_AFTER: u32 = 3;
/// Map a negotiated `quic` codec bit to the FFmpeg decoder id the client opens.
pub fn ffmpeg_codec_id(wire: u8) -> ffmpeg::codec::Id {
match wire {
punktfunk_core::quic::CODEC_H264 => ffmpeg::codec::Id::H264,
punktfunk_core::quic::CODEC_AV1 => ffmpeg::codec::Id::AV1,
_ => ffmpeg::codec::Id::HEVC,
}
}
/// The `quic` codec bitfield this client can decode — whatever FFmpeg has a decoder for (HEVC/H.264
/// always; AV1 when built in). Advertised to the host so it never emits a codec we can't decode.
pub fn decodable_codecs() -> u8 {
let _ = ffmpeg::init();
let mut bits = 0u8;
for (id, bit) in [
(ffmpeg::codec::Id::HEVC, punktfunk_core::quic::CODEC_HEVC),
(ffmpeg::codec::Id::H264, punktfunk_core::quic::CODEC_H264),
(ffmpeg::codec::Id::AV1, punktfunk_core::quic::CODEC_AV1),
] {
if ffmpeg::decoder::find(id).is_some() {
bits |= bit;
}
}
bits
}
impl Decoder {
/// `codec_id` is the codec the host resolved in the Welcome (never assume HEVC).
/// `pref` is the Settings "Video decoder" value (`auto`/`vaapi`/`software`).
/// Precedence: the `PUNKTFUNK_DECODER` env override wins (support/debug escape
/// hatch, and the documented knob), then the setting; both default to auto
/// (VAAPI → software).
pub fn new(codec_id: ffmpeg::codec::Id, pref: &str) -> Result<Decoder> {
ffmpeg::init().context("ffmpeg init")?;
let choice = std::env::var("PUNKTFUNK_DECODER")
.ok()
.filter(|v| !v.is_empty())
.unwrap_or_else(|| pref.to_string());
// Deck note: `auto` means VAAPI here too. GTK's tiled-NV12 dmabuf import is broken on
// the Deck (Mesa ≥ 25.1 exports VCN surfaces TILED; artifacts/gray/washed-out), but the
// presenter routes Deck frames through the in-process GL converter (`video_gl`) instead
// of GdkDmabufTexture — and if THAT can't initialize, it demotes this decoder to
// software mid-session via [`Decoder::force_software`]. The broken direct path is never
// the fallback.
if choice != "software" {
match VaapiDecoder::new(codec_id) {
Ok(v) => {
tracing::info!(?codec_id, "VAAPI hardware decode active (zero-copy dmabuf)");
return Ok(Decoder {
backend: Backend::Vaapi(v),
codec_id,
vaapi_fails: 0,
want_keyframe: false,
});
}
Err(e) => {
if choice == "vaapi" {
return Err(e.context("PUNKTFUNK_DECODER=vaapi but VAAPI failed"));
}
tracing::info!(reason = %e, "VAAPI unavailable — software decode");
}
}
}
Ok(Decoder {
backend: Backend::Software(SoftwareDecoder::new(codec_id)?),
codec_id,
vaapi_fails: 0,
want_keyframe: false,
})
}
/// Drain the "please ask the host for an IDR" flag — the pump calls this each iteration
/// (throttled) so a demoted/erroring decoder can resynchronize under the infinite GOP.
pub fn take_keyframe_request(&mut self) -> bool {
std::mem::take(&mut self.want_keyframe)
}
/// Demote to software decode on the PRESENTER's verdict (dmabuf presentation impossible:
/// GL converter init failed, texture import rejected). Decode itself succeeds in that
/// state, so the error-streak demotion never fires — without this the stream would stay
/// black forever. No-op when already software.
pub fn force_software(&mut self) -> Result<()> {
if matches!(self.backend, Backend::Software(_)) {
return Ok(());
}
tracing::warn!("presenter can't display hardware frames — demoting to software decode");
self.backend = Backend::Software(SoftwareDecoder::new(self.codec_id)?);
self.vaapi_fails = 0;
self.want_keyframe = true;
Ok(())
}
/// Feed one access unit; returns the decoded frame (the host's streams are
/// one-in/one-out). A software decode error after packet loss is survivable — log
/// upstream and keep feeding. A VAAPI error re-requests an IDR and retries the hardware
/// decoder; only a persistent streak of failures (a genuinely broken driver, e.g.
/// nvidia-vaapi-driver) demotes to software. Either way `want_keyframe` is set so the
/// pump asks the host for a fresh IDR — under the infinite GOP nothing else resyncs a
/// rebuilt/erroring decoder, so skipping this leaves the picture gray/frozen for good.
pub fn decode(&mut self, au: &[u8]) -> Result<Option<DecodedImage>> {
match &mut self.backend {
Backend::Vaapi(v) => match v.decode(au) {
Ok(f) => {
self.vaapi_fails = 0;
Ok(f.map(DecodedImage::Dmabuf))
}
Err(e) => {
self.vaapi_fails += 1;
self.want_keyframe = true;
if self.vaapi_fails >= VAAPI_DEMOTE_AFTER {
tracing::warn!(error = %e, fails = self.vaapi_fails,
"VAAPI decode failing repeatedly — demoting to software");
self.backend = Backend::Software(SoftwareDecoder::new(self.codec_id)?);
self.vaapi_fails = 0;
} else {
tracing::warn!(error = %e,
"VAAPI decode error — requesting keyframe, keeping hardware decode");
}
Ok(None)
}
},
Backend::Software(s) => Ok(s.decode(au)?.map(DecodedImage::Cpu)),
}
}
}
// --- software backend ---------------------------------------------------------------
struct SoftwareDecoder {
decoder: ffmpeg::decoder::Video,
/// Rebuilt whenever the decoded format/size — or the colour signaling (a mid-stream
/// SDR↔HDR flip) — changes.
sws: Option<(scaling::Context, Pixel, u32, u32, ColorDesc)>,
}
impl SoftwareDecoder {
fn new(codec_id: ffmpeg::codec::Id) -> Result<SoftwareDecoder> {
let codec = ffmpeg::decoder::find(codec_id)
.ok_or_else(|| anyhow!("no {codec_id:?} decoder in libavcodec"))?;
let mut ctx = ffmpeg::codec::Context::new_with_codec(codec);
unsafe {
let raw = ctx.as_mut_ptr();
(*raw).flags |= ffmpeg::ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
// Slice threading adds no frame delay (frame threading adds thread_count-1).
(*raw).thread_type = ffmpeg::ffi::FF_THREAD_SLICE;
(*raw).thread_count = 0; // auto
}
let decoder = ctx.decoder().video().context("open video decoder")?;
Ok(SoftwareDecoder { decoder, sws: None })
}
fn decode(&mut self, au: &[u8]) -> Result<Option<CpuFrame>> {
let packet = ffmpeg::Packet::copy(au);
self.decoder
.send_packet(&packet)
.map_err(|e| anyhow!("send_packet: {e}"))?;
let mut frame = AvFrame::empty();
let mut out = None;
while self.decoder.receive_frame(&mut frame).is_ok() {
out = Some(self.convert_rgba(&frame)?);
}
Ok(out)
}
fn convert_rgba(&mut self, frame: &AvFrame) -> Result<CpuFrame> {
let (fmt, w, h) = (frame.format(), frame.width(), frame.height());
// SAFETY: `frame.as_ptr()` is the decoder-owned live AVFrame for this call.
let color = unsafe { ColorDesc::from_raw(frame.as_ptr()) };
let rebuild = !matches!(&self.sws,
Some((_, f, sw, sh, c)) if *f == fmt && *sw == w && *sh == h && *c == color);
if rebuild {
let mut ctx =
scaling::Context::get(fmt, w, h, Pixel::RGBA, w, h, scaling::Flags::POINT)
.context("swscale context")?;
// swscale defaults to BT.601 coefficients — set them from the FRAME's signaling
// (unspecified → BT.709 limited, the host's SDR default; a Windows HDR desktop
// streams BT.2020 in-band). Without this, YUV→RGB decodes with the wrong matrix
// and colours shift. Destination = full-range RGB; the transfer function stays
// baked in (the presenter tags PQ textures so GTK applies the EOTF).
const SWS_CS_ITU709: i32 = 1;
const SWS_CS_ITU601: i32 = 5;
const SWS_CS_BT2020: i32 = 9;
let cs = match color.matrix {
9 | 10 => SWS_CS_BT2020,
5 | 6 => SWS_CS_ITU601,
_ => SWS_CS_ITU709,
};
unsafe {
let coeffs = ffmpeg::ffi::sws_getCoefficients(cs);
ffmpeg::ffi::sws_setColorspaceDetails(
ctx.as_mut_ptr(),
coeffs, // inv_table: source (YUV) coefficients per the VUI
color.full_range as i32, // srcRange: 0 = limited/studio (MPEG)
coeffs, // table: destination coefficients (ignored for RGB output)
1, // dstRange: 1 = full-range RGB
0,
1 << 16,
1 << 16, // brightness, contrast, saturation (defaults)
);
}
self.sws = Some((ctx, fmt, w, h, color));
}
let (sws, ..) = self.sws.as_mut().unwrap();
// Single-pass conversion: swscale writes straight into the Vec the texture will
// wrap. (The old path scaled into a scratch AVFrame and then copied `data(0)` out
// — a second full-frame pass per frame.) 64-byte row alignment keeps swscale on
// aligned SIMD stores; `GdkMemoryTexture` takes the resulting stride explicitly.
const ALIGN: i32 = 64;
use ffmpeg::ffi;
let dst_fmt = ffi::AVPixelFormat::AV_PIX_FMT_RGBA;
// SAFETY: pure size computation from format/dimensions; no pointers involved.
let size = unsafe { ffi::av_image_get_buffer_size(dst_fmt, w as i32, h as i32, ALIGN) };
if size < 0 {
return Err(averr("av_image_get_buffer_size", size));
}
let rgba = vec![0u8; size as usize];
let mut dst_data: [*mut u8; 4] = [ptr::null_mut(); 4];
let mut dst_linesize: [i32; 4] = [0; 4];
// SAFETY: fill_arrays only derives plane pointers/strides into `rgba` (sized by
// av_image_get_buffer_size above, same format/align) — no allocation, no
// ownership transfer; `rgba` outlives the scale below.
let r = unsafe {
ffi::av_image_fill_arrays(
dst_data.as_mut_ptr(),
dst_linesize.as_mut_ptr(),
rgba.as_ptr(),
dst_fmt,
w as i32,
h as i32,
ALIGN,
)
};
if r < 0 {
return Err(averr("av_image_fill_arrays", r));
}
// SAFETY: src pointers/strides belong to the decoder-owned `frame` (alive for the
// call); dst pointers were just filled over `rgba`, and sws_scale writes rows
// [0, h) only — exactly the buffer fill_arrays sized.
let r = unsafe {
ffi::sws_scale(
sws.as_mut_ptr(),
(*frame.as_ptr()).data.as_ptr() as *const *const u8,
(*frame.as_ptr()).linesize.as_ptr(),
0,
h as i32,
dst_data.as_ptr(),
dst_linesize.as_ptr(),
)
};
if r < 0 {
return Err(averr("sws_scale", r));
}
Ok(CpuFrame {
width: w,
height: h,
stride: dst_linesize[0] as usize,
rgba,
color,
})
}
}
// --- VAAPI backend --------------------------------------------------------------------
//
// Raw FFI: ffmpeg-next has no hwaccel wrappers. All pointers are owned here and freed in
// Drop; decoded surfaces transfer out through DrmFrameGuard.
const AVERROR_EAGAIN: i32 = -11; // -EAGAIN; Linux-only crate
fn averr(what: &str, code: i32) -> anyhow::Error {
anyhow!("{what}: {}", ffmpeg::Error::from(code))
}
/// libavcodec offers the formats it can decode into; pick the VAAPI hw surface. Falling
/// back to the first (software) entry would silently decode on the CPU *and* break our
/// dmabuf mapping — return NONE instead so the error surfaces and the session demotes
/// to the software backend explicitly.
unsafe extern "C" fn pick_vaapi(
_ctx: *mut ffmpeg::ffi::AVCodecContext,
mut list: *const ffmpeg::ffi::AVPixelFormat,
) -> ffmpeg::ffi::AVPixelFormat {
unsafe {
while *list != ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE {
if *list == ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_VAAPI {
return ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_VAAPI;
}
list = list.add(1);
}
}
ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NONE
}
struct VaapiDecoder {
ctx: *mut ffmpeg::ffi::AVCodecContext,
hw_device: *mut ffmpeg::ffi::AVBufferRef,
packet: *mut ffmpeg::ffi::AVPacket,
frame: *mut ffmpeg::ffi::AVFrame,
}
// Single-owner pointers, only touched from the session pump thread.
unsafe impl Send for VaapiDecoder {}
impl VaapiDecoder {
fn new(codec_id: ffmpeg::codec::Id) -> Result<VaapiDecoder> {
use ffmpeg::ffi;
unsafe {
let mut hw_device: *mut ffi::AVBufferRef = ptr::null_mut();
let r = ffi::av_hwdevice_ctx_create(
&mut hw_device,
ffi::AVHWDeviceType::AV_HWDEVICE_TYPE_VAAPI,
ptr::null(),
ptr::null_mut(),
0,
);
if r < 0 {
bail!("no VAAPI device ({})", ffmpeg::Error::from(r));
}
// The negotiated codec's decoder id (av_codec_id maps 1:1 from ffmpeg::codec::Id).
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
ffi::av_buffer_unref(&mut hw_device);
bail!("no {codec_id:?} decoder");
}
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device);
(*ctx).get_format = Some(pick_vaapi);
(*ctx).flags |= ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
(*ctx).thread_count = 1; // hwaccel: threads only add latency
// The presenter holds mapped surfaces PAST receive_frame (the paintable's
// current texture + the newest frame in flight each pin one until GDK's
// release func) — surfaces libavcodec doesn't know are missing from its
// fixed-size VAAPI pool. Without headroom the decoder can recycle a surface
// the renderer is still sampling (intermittent block corruption) or fail
// allocation under scheduling jitter.
(*ctx).extra_hw_frames = 4;
let r = ffi::avcodec_open2(ctx, codec, ptr::null_mut());
if r < 0 {
let mut ctx = ctx;
ffi::avcodec_free_context(&mut ctx);
let mut hw_device = hw_device;
ffi::av_buffer_unref(&mut hw_device);
bail!("avcodec_open2: {}", ffmpeg::Error::from(r));
}
Ok(VaapiDecoder {
ctx,
hw_device,
packet: ffi::av_packet_alloc(),
frame: ffi::av_frame_alloc(),
})
}
}
fn decode(&mut self, au: &[u8]) -> Result<Option<DmabufFrame>> {
use ffmpeg::ffi;
unsafe {
let r = ffi::av_new_packet(self.packet, au.len() as i32);
if r < 0 {
return Err(averr("av_new_packet", r));
}
ptr::copy_nonoverlapping(au.as_ptr(), (*self.packet).data, au.len());
let r = ffi::avcodec_send_packet(self.ctx, self.packet);
ffi::av_packet_unref(self.packet);
if r < 0 {
return Err(averr("send_packet", r));
}
let mut out = None;
loop {
let r = ffi::avcodec_receive_frame(self.ctx, self.frame);
if r == AVERROR_EAGAIN {
break;
}
if r < 0 {
return Err(averr("receive_frame", r));
}
out = Some(self.map_dmabuf()?); // newest wins; older guards drop here
ffi::av_frame_unref(self.frame);
}
Ok(out)
}
}
/// Map the VAAPI surface to DRM PRIME (zero copy) and lift the descriptor into a
/// `DmabufFrame`. The mapped frame keeps the surface alive via its buffer refs.
///
/// FFmpeg's VAAPI export uses `VA_EXPORT_SURFACE_SEPARATE_LAYERS`, so an NV12 surface
/// comes back as TWO layers (`R8` luma + `GR88` chroma), each one plane — NOT a single
/// `NV12` layer. The previous code took `layers[0]` only: GTK then saw an `R8`
/// single-plane texture with the chroma dropped, painting the screen green. The fix:
/// derive the COMBINED fourcc from the decoder's software pixel format (NV12 →
/// `DRM_FORMAT_NV12`) and flatten every plane across every layer in order (Y then UV).
unsafe fn map_dmabuf(&mut self) -> Result<DmabufFrame> {
use ffmpeg::ffi;
unsafe {
if (*self.frame).format != ffi::AVPixelFormat::AV_PIX_FMT_VAAPI as i32 {
bail!("decoder returned a software frame (no VAAPI surface)");
}
// The real pixel layout lives on the hardware frames context, not the
// DRM-PRIME layer formats (those are the per-plane R8/GR88 component formats).
let sw_format = {
let hwfc = (*self.frame).hw_frames_ctx;
if hwfc.is_null() {
bail!("VAAPI frame without a hardware frames context");
}
(*((*hwfc).data as *const ffi::AVHWFramesContext)).sw_format
};
let fourcc = drm_fourcc_for(sw_format)
.ok_or_else(|| anyhow!("unsupported VAAPI output format {sw_format:?}"))?;
let drm = ffi::av_frame_alloc();
(*drm).format = ffi::AVPixelFormat::AV_PIX_FMT_DRM_PRIME as i32;
let r = ffi::av_hwframe_map(drm, self.frame, ffi::AV_HWFRAME_MAP_READ as i32);
if r < 0 {
let mut drm = drm;
ffi::av_frame_free(&mut drm);
return Err(averr("av_hwframe_map", r));
}
let desc = (*drm).data[0] as *const ffi::AVDRMFrameDescriptor;
let guard = DrmFrameGuard(drm);
let d = &*desc;
if d.nb_layers < 1 || d.nb_objects < 1 {
bail!("DRM descriptor without layers/objects");
}
// Flatten planes across ALL layers, in declared order — the combined fourcc's
// plane order (Y, then UV for NV12) matches the layer order FFmpeg emits.
let mut planes = Vec::new();
for layer in &d.layers[..d.nb_layers as usize] {
for p in &layer.planes[..layer.nb_planes as usize] {
let obj = &d.objects[p.object_index as usize];
planes.push(DmabufPlane {
fd: obj.fd,
offset: p.offset as u32,
stride: p.pitch as u32,
});
}
}
// The whole surface shares one tiling modifier (one BO on radeonsi); GTK takes
// a single modifier for the texture.
let modifier = d.objects[0].format_modifier;
log_descriptor_once(d, sw_format, fourcc, modifier);
Ok(DmabufFrame {
width: (*self.frame).width as u32,
height: (*self.frame).height as u32,
fourcc,
modifier,
planes,
// SAFETY: `self.frame` is the live decoded AVFrame (unref'd only after
// this returns); plain CICP field reads.
color: ColorDesc::from_raw(self.frame),
guard,
})
}
}
}
/// `fourcc(a,b,c,d)` — the DRM FourCC packing (little-endian, `a | b<<8 | c<<16 | d<<24`).
const fn fourcc(a: u8, b: u8, c: u8, d: u8) -> u32 {
(a as u32) | ((b as u32) << 8) | ((c as u32) << 16) | ((d as u32) << 24)
}
/// The combined DRM FourCC for a decoder software pixel format. The host streams 8-bit
/// 4:2:0 (NV12); P010 is here for the eventual 10-bit/HDR path.
fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32> {
use ffmpeg_next::ffi::AVPixelFormat::*;
Some(match sw {
AV_PIX_FMT_NV12 => fourcc(b'N', b'V', b'1', b'2'),
AV_PIX_FMT_P010LE => fourcc(b'P', b'0', b'1', b'0'),
_ => return None,
})
}
/// One-time dump of the DRM descriptor layout (objects, layers, planes, modifier) — so a
/// new client/driver combination's real layout is visible in the logs without a debugger.
fn log_descriptor_once(
d: &ffmpeg_next::ffi::AVDRMFrameDescriptor,
sw: ffmpeg_next::ffi::AVPixelFormat,
fourcc: u32,
modifier: u64,
) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
if !ONCE.swap(false, Ordering::Relaxed) {
return;
}
let layers: Vec<(u32, i32)> = d.layers[..d.nb_layers.max(0) as usize]
.iter()
.map(|l| (l.format, l.nb_planes))
.collect();
tracing::info!(
sw_format = ?sw,
chosen_fourcc = format_args!("{:#010x}", fourcc),
nb_objects = d.nb_objects,
nb_layers = d.nb_layers,
?layers,
modifier = format_args!("{:#018x}", modifier),
"VAAPI dmabuf descriptor layout (first frame)"
);
}
impl Drop for VaapiDecoder {
fn drop(&mut self) {
use ffmpeg::ffi;
unsafe {
ffi::av_packet_free(&mut self.packet);
ffi::av_frame_free(&mut self.frame);
ffi::avcodec_free_context(&mut self.ctx);
ffi::av_buffer_unref(&mut self.hw_device);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Lock the DRM FourCC magic numbers against typos — these are the exact values
/// `<drm_fourcc.h>` defines, and a wrong one is what painted the Steam Deck green.
#[test]
fn drm_fourcc_constants() {
assert_eq!(fourcc(b'N', b'V', b'1', b'2'), 0x3231_564e);
assert_eq!(fourcc(b'P', b'0', b'1', b'0'), 0x3031_3050);
assert_eq!(
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NV12),
Some(0x3231_564e)
);
assert_eq!(
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_RGBA),
None
);
}
/// The wire → `ColorDesc` plumbing: an HDR10 stream's VUI (BT.2020 primaries, PQ
/// transfer, BT.2020-NCL matrix, limited range) must arrive on the decoded frame —
/// this is what the Windows host emits in-band for an HDR desktop, and mis-rendering
/// it as BT.709 is the washed-out-colors bug. Fixture: one 64×64 Main10 IDR
/// (`tests/pq-frame.h265`, x265 with explicit VUI).
#[test]
fn software_decode_carries_pq_signaling() {
let au = include_bytes!("../tests/pq-frame.h265");
let mut dec = SoftwareDecoder::new(ffmpeg::codec::Id::HEVC).expect("hevc decoder");
let mut got = dec.decode(au).expect("decode");
if got.is_none() {
// Low-delay decoders may still hold the frame until a flush — send EOF.
dec.decoder.send_eof().ok();
let mut frame = AvFrame::empty();
if dec.decoder.receive_frame(&mut frame).is_ok() {
got = Some(dec.convert_rgba(&frame).expect("convert"));
}
}
let f = got.expect("no frame decoded from the PQ fixture");
assert_eq!(
f.color,
ColorDesc {
primaries: 9,
transfer: 16,
matrix: 9,
full_range: false
}
);
assert!(f.color.is_pq());
assert_eq!((f.width, f.height), (64, 64));
}
}
clients/linux/src/wol.rs
-24
View File
@@ -1,24 +0,0 @@
//! Client-side Wake-on-LAN: parse stored MAC strings and hand them to the shared core sender
//! (`punktfunk_core::wol`). A sleeping host has no ARP entry, so the broadcast the core sends is
//! what actually wakes it; this is called just before connecting to an offline saved host, and
//! from the explicit "Wake host" menu item / `--wake` CLI mode.
use std::net::Ipv4Addr;
/// Fire a Wake-on-LAN magic packet at `macs` (each `aa:bb:cc:dd:ee:ff`), also unicasting
/// `last_ip` when given. Best-effort — logs the outcome and never blocks the caller meaningfully
/// (the core sends a short burst of datagrams and returns).
pub fn wake(macs: &[String], last_ip: Option<Ipv4Addr>) {
let parsed: Vec<[u8; 6]> = macs
.iter()
.filter_map(|s| punktfunk_core::wol::parse_mac(s))
.collect();
if parsed.is_empty() {
tracing::warn!("wake requested but no valid MAC is known for this host");
return;
}
match punktfunk_core::wol::send_magic_packet(&parsed, last_ip) {
Ok(()) => tracing::info!(count = parsed.len(), "sent Wake-on-LAN magic packet"),
Err(e) => tracing::warn!(error = %e, "Wake-on-LAN send failed"),
}
}
clients/linux/tests/pq-frame.h265
BIN
View File
Binary file not shown.