diff --git a/CLAUDE.md b/CLAUDE.md index 7ef94f0..1dccbbe 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -86,8 +86,20 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc `RemoteFirstLightTests` (full pipeline over the LAN). See [`clients/apple/README.md`](clients/apple/README.md). Next: stage 2 presenter (`VTDecompressionSession` + `CAMetalLayer` frame pacing), glass-to-glass numbers via - `tools/latency-probe` (scaffold), iOS variant. The Linux reference client - (`punktfunk-client-rs`) gets VAAPI + wgpu on the same connector later. + `tools/latency-probe` (scaffold), iOS variant. + **Linux stage 1 done, first light 2026-06-12** (`crates/punktfunk-client-linux`, binary + `punktfunk-client`): GTK4/libadwaita shell linking `punktfunk-core` directly (no C ABI; + `NativeClient` is now `Sync` — mutexed plane receivers), mDNS host list, TOFU + SPAKE2 + PIN dialogs (identity shared with client-rs), FFmpeg software HEVC decode (LOW_DELAY, + slice threads) → `GtkGraphicsOffload`-wrapped picture, PipeWire playback (mic-player + jitter ring inverted), SDL3 gamepad capture + rumble/lightbar feedback, keyboard via + exact inverse of the host VK table, absolute mouse + 120-unit scroll. Validated live + against `serve --native` on this box: 1080p60, steady 60 fps, capture→decoded p50 + ≈6.4 ms (debug build). `--connect host[:port]` for scripting. Next (per the 2026-06-12 + research, memory `linux-client-option-a`): VAAPI dmabuf → `GdkDmabufTexture` (Tier-1 + zero-copy on Intel/AMD), then the stage-2 raw-Wayland presenter (wp_presentation + feedback, tearing-control, Vulkan Video on NVIDIA) — **wgpu/winit rejected** (no dmabuf + import / presentation feedback / shortcuts-inhibit). 2. **Sub-frame pipelining**: overlap encode and transmit within a frame. Requires a direct NVENC SDK wrapper (libavcodec only emits whole AUs) — the next big latency lever (~2–4 ms at high res). @@ -141,7 +153,8 @@ crates/punktfunk-host/ zerocopy/{egl,cuda,vulkan}.rs dmabuf → CUDA → NVENC (tiled via EGL/GL, LINEAR via Vulkan) inject/{libei,wlr,gamepad,dualsense}.rs input backends (uinput xpad + UHID DualSense) capture.rs · encode.rs · audio.rs · m0.rs · m3.rs · mgmt.rs · native_pairing.rs -crates/punktfunk-client-rs/ punktfunk/1 reference client (M3 headless; M4 adds decode+present) +crates/punktfunk-client-rs/ punktfunk/1 reference client (M3 headless test/measurement tool) +crates/punktfunk-client-linux/ native Linux client (GTK4/libadwaita · FFmpeg · PipeWire · SDL3) web/ TanStack web console over the mgmt API (status · devices · pairing) packaging/ Fedora/Bazzite RPM · bootc · COPR (packaging/bazzite/README.md) tools/{loss-harness,latency-probe}/ measurement (plan §10) diff --git a/Cargo.lock b/Cargo.lock index d2eb78f..cc28907 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -196,6 +196,18 @@ dependencies = [ "pin-project-lite", ] +[[package]] +name = "async-channel" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "924ed96dd52d1b75e9c1a3e6275715fd320f5f9439fb5a4a11fa51f4221158d2" +dependencies = [ + "concurrent-queue", + "event-listener-strategy", + "futures-core", + "pin-project-lite", +] + [[package]] name = "async-recursion" version = "1.1.1" @@ -419,6 +431,29 @@ version = "1.11.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33" +[[package]] +name = "cairo-rs" +version = "0.22.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5cc8d9aa793480744cd9a0524fef1a2e197d9eaa0f739cde19d16aba530dcb95" +dependencies = [ + "bitflags", + "cairo-sys-rs", + "glib", + "libc", +] + +[[package]] +name = "cairo-sys-rs" +version = "0.22.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8b4985713047f5faee02b8db6a6ef32bbb50269ff53c1aee716d1d195b76d54" +dependencies = [ + "glib-sys", + "libc", + "system-deps", +] + [[package]] name = "cbc" version = "0.1.2" @@ -896,6 +931,16 @@ version = "0.2.9" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d" +[[package]] +name = "field-offset" +version = "0.3.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "38e2275cc4e4fc009b0669731a1e5ab7ebf11f469eaede2bab9309a5b4d6057f" +dependencies = [ + "memoffset", + "rustc_version", +] + [[package]] name = "find-msvc-tools" version = "0.1.9" @@ -1057,6 +1102,63 @@ dependencies = [ "slab", ] +[[package]] +name = "gdk-pixbuf" +version = "0.22.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "25f420376dbee041b2db374ce4573892a36222bb3f6c0c43e24f0d67eae9b646" +dependencies = [ + "gdk-pixbuf-sys", + "gio", + "glib", + "libc", +] + +[[package]] +name = "gdk-pixbuf-sys" +version = "0.22.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "48f31b37b1fc4b48b54f6b91b7ef04c18e00b4585d98359dd7b998774bbd91fb" +dependencies = [ + "gio-sys", + "glib-sys", + "gobject-sys", + "libc", + "system-deps", +] + +[[package]] +name = "gdk4" +version = "0.11.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fd42fdbbf48612c6e8f47c65fb92d2e8f39c25aecd6af047e83897c1a22d2a4e" +dependencies = [ + "cairo-rs", + "gdk-pixbuf", + "gdk4-sys", + "gio", + "glib", + "libc", + "pango", +] + +[[package]] +name = "gdk4-sys" +version = "0.11.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9d974ac4f15e67472c3a9728daf612590b4a5762a4b33f0edd298df0b80d043c" +dependencies = [ + "cairo-sys-rs", + "gdk-pixbuf-sys", + "gio-sys", + "glib-sys", + "gobject-sys", + "libc", + "pango-sys", + "pkg-config", + "system-deps", +] + [[package]] name = "generic-array" version = "0.14.7" @@ -1117,12 +1219,202 @@ dependencies = [ "polyval", ] +[[package]] +name = "gio" +version = "0.22.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e3848bcba3a35cc0a71df8ba8ecfd799d6bfb862342a53a4a915fb62213aa4e6" +dependencies = [ + "futures-channel", + "futures-core", + "futures-io", + "futures-util", + "gio-sys", + "glib", + "libc", + "pin-project-lite", + "smallvec", +] + +[[package]] +name = "gio-sys" +version = "0.22.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "64729ba2772c080448f9f966dba8f4456beeb100d8c28a865ef8a0f2ef4987e1" +dependencies = [ + "glib-sys", + "gobject-sys", + "libc", + "system-deps", + "windows-sys 0.61.2", +] + +[[package]] +name = "glib" +version = "0.22.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c207e04e51605dcf7b2924c41591b3a10e1438eaac5bcf448fb91f325381104a" +dependencies = [ + "bitflags", + "futures-channel", + "futures-core", + "futures-executor", + "futures-task", + "futures-util", + "gio-sys", + "glib-macros", + "glib-sys", + "gobject-sys", + "libc", + "memchr", + "smallvec", +] + +[[package]] +name = "glib-macros" +version = "0.22.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "506d23499707c7142898429757e8d9a3871d965239a2cb66dfa05052be6d6f19" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "glib-sys" +version = "0.22.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5f7fbac234ed5bc2a28359b7bde8e1b9cdf1441cc2d7f068e4824672d7db9445" +dependencies = [ + "libc", + "system-deps", +] + [[package]] name = "glob" version = "0.3.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0cc23270f6e1808e30a928bdc84dea0b9b4136a8bc82338574f23baf47bbd280" +[[package]] +name = "gobject-sys" +version = "0.22.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "22a861859b887a79cf461359c192c97a57d8fb0229dd291232e57aa11f6fa72c" +dependencies = [ + "glib-sys", + "libc", + "system-deps", +] + +[[package]] +name = "graphene-rs" +version = "0.22.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c7d1b7881f96869f49808b6adfe906a93a57a34204952253444d68c3208d71f1" +dependencies = [ + "glib", + "graphene-sys", + "libc", +] + +[[package]] +name = "graphene-sys" +version = "0.22.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "517f062f3fd6b7fd3e57a3f038a74b3c23ca32f51199ff028aa704609943f79c" +dependencies = [ + "glib-sys", + "libc", + "pkg-config", + "system-deps", +] + +[[package]] +name = "gsk4" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53c912dfcbd28acace5fc99c40bb9f25e1dcb73efb1f2608327f66a99acdcb62" +dependencies = [ + "cairo-rs", + "gdk4", + "glib", + "graphene-rs", + "gsk4-sys", + "libc", + "pango", +] + +[[package]] +name = "gsk4-sys" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7d54bbc7a9d8b6ffe4f0c95eede15ccfb365c8bf521275abe6bcfb57b18fb8a" +dependencies = [ + "cairo-sys-rs", + "gdk4-sys", + "glib-sys", + "gobject-sys", + "graphene-sys", + "libc", + "pango-sys", + "system-deps", +] + +[[package]] +name = "gtk4" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7181b837f04cbe93f79441475f7a00560a92cba7a72e38cc1a68b6f8b78eaae2" +dependencies = [ + "cairo-rs", + "field-offset", + "futures-channel", + "gdk-pixbuf", + "gdk4", + "gio", + "glib", + "graphene-rs", + "gsk4", + "gtk4-macros", + "gtk4-sys", + "libc", + "pango", +] + +[[package]] +name = "gtk4-macros" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3581b242ba62fdff122ebb626ea641582ec326031622bd19d60f85029c804a87" +dependencies = [ + "proc-macro-crate", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "gtk4-sys" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "20ba8e695e2640455561274e65e45f0a151619e450746007667f4b23ceae4e1b" +dependencies = [ + "cairo-sys-rs", + "gdk-pixbuf-sys", + "gdk4-sys", + "gio-sys", + "glib-sys", + "gobject-sys", + "graphene-sys", + "gsk4-sys", + "libc", + "pango-sys", + "system-deps", +] + [[package]] name = "h2" version = "0.4.14" @@ -1427,6 +1719,37 @@ version = "0.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2" +[[package]] +name = "libadwaita" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc0da4e27b20d3e71f830e5b0f0188d22c257986bf421c02cfde777fe07932a4" +dependencies = [ + "gdk4", + "gio", + "glib", + "gtk4", + "libadwaita-sys", + "libc", + "pango", +] + +[[package]] +name = "libadwaita-sys" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aaee067051c5d3c058d050d167688b80b67de1950cfca77730549aa761fc5d7d" +dependencies = [ + "gdk4-sys", + "gio-sys", + "glib-sys", + "gobject-sys", + "gtk4-sys", + "libc", + "pango-sys", + "system-deps", +] + [[package]] name = "libc" version = "0.2.186" @@ -1753,6 +2076,30 @@ dependencies = [ "pin-project-lite", ] +[[package]] +name = "pango" +version = "0.22.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "251bdc6e6487b811be0e406a21e301e07e45c0aa8fa39e00c0c8e12a91752438" +dependencies = [ + "gio", + "glib", + "libc", + "pango-sys", +] + +[[package]] +name = "pango-sys" +version = "0.22.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bbd111a20ca90fedf03e09c59783c679c00900f1d8491cca5399f5e33609d5d6" +dependencies = [ + "glib-sys", + "gobject-sys", + "libc", + "system-deps", +] + [[package]] name = "parking" version = "2.2.1" @@ -1948,6 +2295,26 @@ dependencies = [ "unarray", ] +[[package]] +name = "punktfunk-client-linux" +version = "0.0.1" +dependencies = [ + "anyhow", + "async-channel", + "ffmpeg-next", + "gtk4", + "libadwaita", + "mdns-sd", + "opus", + "pipewire", + "punktfunk-core", + "sdl3", + "serde", + "serde_json", + "tracing", + "tracing-subscriber", +] + [[package]] name = "punktfunk-client-rs" version = "0.0.1" @@ -2516,6 +2883,27 @@ version = "1.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" +[[package]] +name = "sdl3" +version = "0.18.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "25bd22eb1bbc9137e914022b4994ed35591eea0884e9e3e98e6d9895cad6e1d2" +dependencies = [ + "bitflags", + "libc", + "sdl3-sys", +] + +[[package]] +name = "sdl3-sys" +version = "0.6.6+SDL-3.4.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "04e7f134def04ed72e6f55187c6c29c72f7dab5d359c4be0dd49c9b97fef59c7" +dependencies = [ + "pkg-config", + "vcpkg", +] + [[package]] name = "security-framework" version = "3.7.0" diff --git a/Cargo.toml b/Cargo.toml index fbb8d1d..6d563dd 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -4,6 +4,7 @@ members = [ "crates/punktfunk-core", "crates/punktfunk-host", "crates/punktfunk-client-rs", + "crates/punktfunk-client-linux", "tools/latency-probe", "tools/loss-harness", ] diff --git a/crates/punktfunk-client-linux/Cargo.toml b/crates/punktfunk-client-linux/Cargo.toml new file mode 100644 index 0000000..57a3936 --- /dev/null +++ b/crates/punktfunk-client-linux/Cargo.toml @@ -0,0 +1,39 @@ +[package] +name = "punktfunk-client-linux" +description = "Native Linux punktfunk/1 client — GTK4/libadwaita shell, FFmpeg decode, PipeWire audio, SDL3 gamepads" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +authors.workspace = true +repository.workspace = true + +[[bin]] +name = "punktfunk-client" +path = "src/main.rs" + +# Everything is Linux-gated so `cargo build --workspace` stays green on macOS (the Mac +# client lives in clients/apple); on other platforms this builds as a stub binary. +[target.'cfg(target_os = "linux")'.dependencies] +punktfunk-core = { path = "../punktfunk-core", features = ["quic"] } + +# UI shell. GraphicsOffload needs GTK ≥ 4.14; black-background ≥ 4.16. AlertDialog/ +# PreferencesDialog need libadwaita ≥ 1.5. +gtk = { package = "gtk4", version = "0.11", features = ["v4_16"] } +adw = { package = "libadwaita", version = "0.9", features = ["v1_5"] } +async-channel = "2" + +# Video decode (same FFmpeg pin as the host) and audio. +ffmpeg-next = "8" +opus = "0.3" +pipewire = "0.9" + +# Gamepads: capture + rumble/lightbar feedback (full DualSense fidelity lives here). +sdl3 = "0.18" + +mdns-sd = "0.20" +serde = { version = "1", features = ["derive"] } +serde_json = "1" +anyhow = "1" +tracing = "0.1" +tracing-subscriber = { version = "0.3", features = ["env-filter"] } diff --git a/crates/punktfunk-client-linux/src/app.rs b/crates/punktfunk-client-linux/src/app.rs new file mode 100644 index 0000000..03f7a9a --- /dev/null +++ b/crates/punktfunk-client-linux/src/app.rs @@ -0,0 +1,335 @@ +//! The application shell: window, navigation, trust dialogs, session lifecycle. + +use crate::session::{SessionEvent, SessionParams}; +use crate::trust::{KnownHost, KnownHosts, Settings}; +use crate::ui_hosts::ConnectRequest; +use adw::prelude::*; +use gtk::glib; +use punktfunk_core::client::NativeClient; +use punktfunk_core::config::GamepadPref; +use std::cell::RefCell; +use std::rc::Rc; + +const APP_ID: &str = "io.unom.Punktfunk"; + +struct App { + window: adw::ApplicationWindow, + nav: adw::NavigationView, + toasts: adw::ToastOverlay, + settings: Rc>, + identity: (String, String), + /// One session at a time — ignore connects while one is starting/running. + busy: std::cell::Cell, +} + +impl App { + fn toast(&self, msg: &str) { + self.toasts.add_toast(adw::Toast::new(msg)); + } +} + +pub fn run() -> glib::ExitCode { + tracing_subscriber::fmt() + .with_env_filter( + tracing_subscriber::EnvFilter::try_from_default_env().unwrap_or_else(|_| "info".into()), + ) + .init(); + let app = adw::Application::builder().application_id(APP_ID).build(); + app.connect_activate(build_ui); + // GTK doesn't see our argv (`--connect` is handled in `build_ui`); an empty argv also + // keeps GApplication from rejecting unknown options. + app.run_with_args(&[] as &[&str]) +} + +/// `--connect host[:port]` — skip the hosts page and start a session immediately +/// (scripting + headless testing; trust follows the same known-hosts/TOFU rules). +fn cli_connect_request() -> Option { + let args: Vec = std::env::args().collect(); + let target = args + .iter() + .skip_while(|a| *a != "--connect") + .nth(1)? + .clone(); + let (addr, port) = match target.rsplit_once(':') { + Some((a, p)) => (a.to_string(), p.parse().ok()?), + None => (target.clone(), 9777), + }; + Some(ConnectRequest { + name: addr.clone(), + addr, + port, + fp_hex: None, + pair_required: false, + }) +} + +fn build_ui(gtk_app: &adw::Application) { + let identity = match crate::trust::load_or_create_identity() { + Ok(i) => i, + Err(e) => { + tracing::error!("client identity: {e:#}"); + std::process::exit(1); + } + }; + + let nav = adw::NavigationView::new(); + let toasts = adw::ToastOverlay::new(); + toasts.set_child(Some(&nav)); + let window = adw::ApplicationWindow::builder() + .application(gtk_app) + .title("Punktfunk") + .default_width(1100) + .default_height(720) + .content(&toasts) + .build(); + + let app = Rc::new(App { + window: window.clone(), + nav: nav.clone(), + toasts, + settings: Rc::new(RefCell::new(Settings::load())), + identity, + busy: std::cell::Cell::new(false), + }); + + let hosts_page = crate::ui_hosts::new( + { + let app = app.clone(); + Rc::new(move |req| initiate_connect(app.clone(), req)) + }, + { + let app = app.clone(); + Rc::new(move || crate::ui_settings::show(&app.window, app.settings.clone())) + }, + ); + nav.add(&hosts_page); + window.present(); + + if let Some(req) = cli_connect_request() { + initiate_connect(app, req); + } +} + +/// The trust gate in front of every connect. Discovered hosts carry their fingerprint in +/// the mDNS advert, so trust is decided *before* any traffic: known → pinned connect; +/// unknown → TOFU prompt (or straight to pairing when the host requires it). Manual +/// entries have no advance fingerprint: trust on first use, pin from then on. +fn initiate_connect(app: Rc, req: ConnectRequest) { + if app.busy.get() { + return; + } + let known = KnownHosts::load(); + match &req.fp_hex { + Some(fp_hex) => { + if known.find_by_fp(fp_hex).is_some() { + start_session(app, req.clone(), crate::trust::parse_hex32(fp_hex)); + } else if req.pair_required { + // TOFU alone won't pass the host's gate — go straight to the ceremony. + pin_dialog(app, req); + } else { + tofu_dialog(app, req); + } + } + None => { + let pin = known + .find_by_addr(&req.addr, req.port) + .and_then(|k| crate::trust::parse_hex32(&k.fp_hex)); + start_session(app, req, pin); + } + } +} + +/// First contact with a discovered host: show the advertised fingerprint and let the user +/// trust it (TOFU), run the PIN ceremony instead, or walk away. +fn tofu_dialog(app: Rc, req: ConnectRequest) { + let fp = req.fp_hex.clone().unwrap_or_default(); + let dialog = adw::AlertDialog::new( + Some("New Host"), + Some(&format!( + "{} at {}:{}\n\nCertificate fingerprint:\n{}\n\nPairing with a PIN verifies it; \ + trusting accepts it as-is.", + req.name, req.addr, req.port, fp + )), + ); + dialog.add_responses(&[ + ("cancel", "Cancel"), + ("pair", "Pair with PIN…"), + ("trust", "Trust & Connect"), + ]); + dialog.set_response_appearance("trust", adw::ResponseAppearance::Suggested); + dialog.set_default_response(Some("trust")); + dialog.set_close_response("cancel"); + let parent = app.window.clone(); + dialog.connect_response(None, move |_, response| match response { + "trust" => { + let mut known = KnownHosts::load(); + known.upsert(KnownHost { + name: req.name.clone(), + addr: req.addr.clone(), + port: req.port, + fp_hex: fp.clone(), + paired: false, + }); + let _ = known.save(); + start_session(app.clone(), req.clone(), crate::trust::parse_hex32(&fp)); + } + "pair" => pin_dialog(app.clone(), req.clone()), + _ => {} + }); + dialog.present(Some(&parent)); +} + +/// The SPAKE2 ceremony: the host is armed and displays a 4-digit PIN; proving knowledge +/// of it pins the host's certificate (and registers ours) with no offline-guessable +/// transcript. +fn pin_dialog(app: Rc, req: ConnectRequest) { + let entry = gtk::Entry::builder() + .input_purpose(gtk::InputPurpose::Digits) + .placeholder_text("4-digit PIN shown by the host") + .activates_default(true) + .build(); + let dialog = adw::AlertDialog::new( + Some("Pair with PIN"), + Some(&format!( + "Arm pairing on {} (console or web UI), then enter the PIN it displays.", + req.name + )), + ); + dialog.set_extra_child(Some(&entry)); + dialog.add_responses(&[("cancel", "Cancel"), ("pair", "Pair")]); + dialog.set_response_appearance("pair", adw::ResponseAppearance::Suggested); + dialog.set_default_response(Some("pair")); + dialog.set_close_response("cancel"); + let parent = app.window.clone(); + dialog.connect_response(Some("pair"), move |_, _| { + let pin = entry.text().to_string(); + let app = app.clone(); + let req = req.clone(); + let identity = app.identity.clone(); + let (tx, rx) = async_channel::bounded::>(1); + let (host, port, name) = (req.addr.clone(), req.port, glib::host_name().to_string()); + std::thread::spawn(move || { + let result = NativeClient::pair( + &host, + port, + (&identity.0, &identity.1), + pin.trim(), + &name, + std::time::Duration::from_secs(90), + ) + .map_err(|e| format!("Pairing failed: {e:?} (wrong PIN, or pairing not armed?)")); + let _ = tx.send_blocking(result); + }); + glib::spawn_future_local(async move { + match rx.recv().await { + Ok(Ok(fp)) => { + let fp_hex = crate::trust::hex(&fp); + let mut known = KnownHosts::load(); + known.upsert(KnownHost { + name: req.name.clone(), + addr: req.addr.clone(), + port: req.port, + fp_hex, + paired: true, + }); + let _ = known.save(); + app.toast("Paired — connecting…"); + start_session(app.clone(), req, Some(fp)); + } + Ok(Err(msg)) => app.toast(&msg), + Err(_) => {} + } + }); + }); + dialog.present(Some(&parent)); +} + +fn start_session(app: Rc, req: ConnectRequest, pin: Option<[u8; 32]>) { + if app.busy.replace(true) { + return; + } + let s = app.settings.borrow(); + let params = SessionParams { + host: req.addr.clone(), + port: req.port, + mode: punktfunk_core::config::Mode { + width: s.width, + height: s.height, + refresh_hz: s.refresh_hz, + }, + gamepad: GamepadPref::from_name(&s.gamepad).unwrap_or(GamepadPref::Auto), + bitrate_kbps: s.bitrate_kbps, + pin, + identity: app.identity.clone(), + }; + let inhibit = s.inhibit_shortcuts; + drop(s); + let tofu = pin.is_none(); + + let mut handle = crate::session::start(params); + let frames = std::mem::replace(&mut handle.frames, async_channel::bounded(1).1); + glib::spawn_future_local(async move { + let mut frames = Some(frames); + let mut page: Option = None; + while let Ok(event) = handle.events.recv().await { + match event { + SessionEvent::Connected { + connector, + mode, + fingerprint, + } => { + // A TOFU connect just observed the real fingerprint — pin it from now on. + if tofu { + let fp_hex = crate::trust::hex(&fingerprint); + let mut known = KnownHosts::load(); + known.upsert(KnownHost { + name: req.name.clone(), + addr: req.addr.clone(), + port: req.port, + fp_hex: fp_hex.clone(), + paired: false, + }); + let _ = known.save(); + app.toast(&format!( + "Trusted on first use — fingerprint {}…", + &fp_hex[..16] + )); + } + tracing::debug!(?mode, "connected — pushing stream page"); + let title = format!( + "{} · {}×{}@{}", + req.name, mode.width, mode.height, mode.refresh_hz + ); + let p = crate::ui_stream::new( + &app.window, + connector, + frames.take().expect("Connected delivered once"), + handle.stop.clone(), + inhibit, + &title, + ); + app.nav.push(&p.page); + page = Some(p); + } + SessionEvent::Stats(s) => { + if let Some(p) = &page { + p.update_stats(s); + } + } + SessionEvent::Failed(msg) => { + app.toast(&msg); + app.busy.set(false); + break; + } + SessionEvent::Ended(err) => { + app.nav.pop_to_tag("hosts"); + if let Some(e) = err { + app.toast(&e); + } + app.busy.set(false); + break; + } + } + } + }); +} diff --git a/crates/punktfunk-client-linux/src/audio.rs b/crates/punktfunk-client-linux/src/audio.rs new file mode 100644 index 0000000..895f8d3 --- /dev/null +++ b/crates/punktfunk-client-linux/src/audio.rs @@ -0,0 +1,206 @@ +//! Audio playback: decoded PCM → a PipeWire playback stream. +//! +//! Mirrors the host's virtual-mic producer (`punktfunk-host::audio::linux`) with the same +//! adaptive jitter buffer: the session pump pushes 5 ms Opus-decoded chunks on the +//! network clock; PipeWire pulls whole quanta on the device clock. Prime to ~3 quanta +//! before producing, cap the ring so latency stays bounded, re-prime after a real drain. + +use anyhow::{Context, Result}; +use std::collections::VecDeque; +use std::sync::mpsc::{Receiver, SyncSender, TrySendError}; + +const SAMPLE_RATE: u32 = 48_000; +const CHANNELS: usize = 2; + +struct Terminate; + +pub struct AudioPlayer { + pcm_tx: SyncSender>, + quit_tx: pipewire::channel::Sender, + thread: Option>, +} + +impl AudioPlayer { + /// Spawn the PipeWire playback thread. Failure (no PipeWire in the session) is + /// survivable — the caller streams video-only. + pub fn spawn() -> Result { + // 64 × 5 ms = 320 ms of slack between the pump and the PipeWire loop. + let (pcm_tx, pcm_rx) = std::sync::mpsc::sync_channel::>(64); + let (quit_tx, quit_rx) = pipewire::channel::channel::(); + let thread = std::thread::Builder::new() + .name("punktfunk-audio".into()) + .spawn(move || { + if let Err(e) = pw_thread(pcm_rx, quit_rx) { + tracing::warn!(error = %e, "audio playback thread ended"); + } + }) + .context("spawn audio thread")?; + Ok(AudioPlayer { + pcm_tx, + quit_tx, + thread: Some(thread), + }) + } + + /// Queue one interleaved-stereo f32 chunk. Drops the chunk if the PipeWire side is + /// wedged (the renderer conceals the gap; never block the session pump). + pub fn push(&self, pcm: Vec) { + if let Err(TrySendError::Disconnected(_)) = self.pcm_tx.try_send(pcm) { + // Thread already dead — Drop will reap it; nothing to do per-chunk. + } + } +} + +impl Drop for AudioPlayer { + fn drop(&mut self) { + let _ = self.quit_tx.send(Terminate); + if let Some(t) = self.thread.take() { + let _ = t.join(); + } + } +} + +/// Producer-side state: incoming decoded PCM and the ring the process callback drains. +struct PlayerData { + rx: Receiver>, + ring: VecDeque, + primed: bool, +} + +fn pw_thread( + pcm_rx: Receiver>, + quit_rx: pipewire::channel::Receiver, +) -> Result<()> { + use pipewire as pw; + use pw::{properties::properties, spa}; + use spa::param::audio::{AudioFormat, AudioInfoRaw}; + use spa::pod::Pod; + + static PW_INIT: std::sync::Once = std::sync::Once::new(); + PW_INIT.call_once(pw::init); + + let mainloop = pw::main_loop::MainLoopRc::new(None).context("pw MainLoop")?; + let context = pw::context::ContextRc::new(&mainloop, None).context("pw Context")?; + let core = context + .connect_rc(None) + .context("pw connect (is PipeWire running in this session?)")?; + + let _quit_guard = quit_rx.attach(mainloop.loop_(), { + let mainloop = mainloop.clone(); + move |_| mainloop.quit() + }); + + let stream = pw::stream::StreamBox::new( + &core, + "punktfunk-client", + properties! { + *pw::keys::MEDIA_TYPE => "Audio", + *pw::keys::MEDIA_CATEGORY => "Playback", + *pw::keys::MEDIA_ROLE => "Game", + *pw::keys::NODE_NAME => "punktfunk-client", + *pw::keys::NODE_DESCRIPTION => "Punktfunk Stream", + // ~5 ms quantum (one Opus frame) keeps the ring — and so the latency — small. + *pw::keys::NODE_LATENCY => "240/48000", + }, + ) + .context("pw Stream")?; + + let ud = PlayerData { + rx: pcm_rx, + ring: VecDeque::new(), + primed: false, + }; + + let _listener = stream + .add_local_listener_with_user_data(ud) + .state_changed(|_s, _ud, old, new| { + tracing::debug!(?old, ?new, "pipewire playback stream state"); + }) + .process(|stream, ud| { + let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { + let Some(mut buffer) = stream.dequeue_buffer() else { + return; + }; + while let Ok(chunk) = ud.rx.try_recv() { + ud.ring.extend(chunk); + } + let stride = 4 * CHANNELS; // F32LE interleaved + let datas = buffer.datas_mut(); + if datas.is_empty() { + return; + } + let data = &mut datas[0]; + let want_frames = data.data().map(|s| s.len() / stride).unwrap_or(0); + let want = want_frames * CHANNELS; + + // Adaptive jitter buffer (same shape as the host's virtual mic): prime to + // ~3 quanta, cap at ~1 quantum of slack beyond that, re-prime after a + // genuine drain. + let target = (3 * want).clamp(720 * CHANNELS, 9600 * CHANNELS); + while ud.ring.len() > target.max(want) + want { + ud.ring.pop_front(); + } + if !ud.primed && ud.ring.len() >= target { + ud.primed = true; + } + + let n_frames = if let Some(slice) = data.data() { + for k in 0..want { + let s = if ud.primed { + ud.ring.pop_front().unwrap_or(0.0) + } else { + 0.0 + }; + let off = k * 4; + slice[off..off + 4].copy_from_slice(&s.to_le_bytes()); + } + want_frames + } else { + 0 + }; + if ud.ring.is_empty() { + ud.primed = false; + } + let chunk = data.chunk_mut(); + *chunk.offset_mut() = 0; + *chunk.stride_mut() = stride as _; + *chunk.size_mut() = (stride * n_frames) as _; + })); + if outcome.is_err() { + tracing::error!("panic in pipewire playback callback"); + } + }) + .register() + .context("register playback listener")?; + + let mut info = AudioInfoRaw::new(); + info.set_format(AudioFormat::F32LE); + info.set_rate(SAMPLE_RATE); + info.set_channels(CHANNELS as u32); + let obj = pw::spa::pod::Object { + type_: pw::spa::utils::SpaTypes::ObjectParamFormat.as_raw(), + id: pw::spa::param::ParamType::EnumFormat.as_raw(), + properties: info.into(), + }; + let values: Vec = pw::spa::pod::serialize::PodSerializer::serialize( + std::io::Cursor::new(Vec::new()), + &pw::spa::pod::Value::Object(obj), + ) + .context("serialize format pod")? + .0 + .into_inner(); + let mut params = [Pod::from_bytes(&values).context("pod from bytes")?]; + + stream + .connect( + spa::utils::Direction::Output, + None, + pw::stream::StreamFlags::AUTOCONNECT | pw::stream::StreamFlags::MAP_BUFFERS, + &mut params, + ) + .context("pw stream connect")?; + + mainloop.run(); + tracing::debug!("pipewire playback loop exited"); + Ok(()) +} diff --git a/crates/punktfunk-client-linux/src/discovery.rs b/crates/punktfunk-client-linux/src/discovery.rs new file mode 100644 index 0000000..1fcc8cf --- /dev/null +++ b/crates/punktfunk-client-linux/src/discovery.rs @@ -0,0 +1,76 @@ +//! LAN host discovery: browse the host's mDNS advert (`_punktfunk._udp`, TXT keys +//! `fp`/`pair`/`id` — see the host crate's `discovery.rs`) on a worker thread and stream +//! results to the UI. + +use mdns_sd::{ServiceDaemon, ServiceEvent}; + +#[derive(Clone, Debug)] +pub struct DiscoveredHost { + /// Stable row key: the advertised host id, falling back to the mDNS fullname. + pub key: String, + pub name: String, + pub addr: String, + pub port: u16, + /// Host certificate fingerprint to pin (lowercase hex), empty if not advertised. + pub fp_hex: String, + /// Pairing requirement: `"required"` or `"optional"`. + pub pair: String, +} + +/// Browse continuously for the app's lifetime. The thread exits when the receiver is +/// dropped (the send fails) or the daemon dies. +pub fn browse() -> async_channel::Receiver { + let (tx, rx) = async_channel::unbounded(); + std::thread::Builder::new() + .name("punktfunk-mdns".into()) + .spawn(move || { + let daemon = match ServiceDaemon::new() { + Ok(d) => d, + Err(e) => { + tracing::warn!(error = %e, "mDNS daemon failed — discovery disabled"); + return; + } + }; + let receiver = match daemon.browse("_punktfunk._udp.local.") { + Ok(r) => r, + Err(e) => { + tracing::warn!(error = %e, "mDNS browse failed — discovery disabled"); + return; + } + }; + while let Ok(event) = receiver.recv() { + if let ServiceEvent::ServiceResolved(info) = event { + let props = info.get_properties(); + let val = |k: &str| props.get_property_val_str(k).unwrap_or("").to_string(); + let Some(addr) = info.get_addresses().iter().next().map(|a| a.to_string()) + else { + continue; + }; + let id = val("id"); + let host = DiscoveredHost { + key: if id.is_empty() { + info.get_fullname().to_string() + } else { + id + }, + name: info + .get_fullname() + .split('.') + .next() + .unwrap_or("?") + .to_string(), + addr, + port: info.get_port(), + fp_hex: val("fp"), + pair: val("pair"), + }; + if tx.send_blocking(host).is_err() { + break; // UI gone — stop browsing + } + } + } + let _ = daemon.shutdown(); + }) + .expect("spawn mdns thread"); + rx +} diff --git a/crates/punktfunk-client-linux/src/gamepad.rs b/crates/punktfunk-client-linux/src/gamepad.rs new file mode 100644 index 0000000..7d234f2 --- /dev/null +++ b/crates/punktfunk-client-linux/src/gamepad.rs @@ -0,0 +1,176 @@ +//! Gamepad capture + feedback over SDL3, on a dedicated thread. +//! +//! Mirrors the Apple client's selection model: exactly one pad is forwarded as pad 0 — +//! the first connected (a pin/auto picker lands with the settings work). SDL3 is the one +//! library with full DualSense fidelity (touchpad/gyro/lightbar/player LEDs/rumble + +//! adaptive triggers via raw effect packets), matching the wire planes; this stage wires +//! buttons/axes out and rumble/lightbar back. Touchpad/motion capture (0xCC) and +//! adaptive-trigger replay (0xCD `Trigger`) are follow-ups on the same loop. +//! +//! This thread also owns the rumble and HID-output pull planes (one consumer per plane). + +use punktfunk_core::client::NativeClient; +use punktfunk_core::input::{gamepad as wire, InputEvent, InputKind}; +use punktfunk_core::quic::HidOutput; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Arc; +use std::time::Duration; + +pub fn spawn( + connector: Arc, + stop: Arc, +) -> Option> { + std::thread::Builder::new() + .name("punktfunk-gamepad".into()) + .spawn(move || { + if let Err(e) = run(&connector, &stop) { + tracing::warn!(error = %e, "gamepad thread ended — pads disabled"); + } + }) + .ok() +} + +fn send(connector: &NativeClient, kind: InputKind, code: u32, x: i32) { + let _ = connector.send_input(&InputEvent { + kind, + _pad: [0; 3], + code, + x, + y: 0, + flags: 0, // pad index 0 — single-pad model + }); +} + +fn button_bit(b: sdl3::gamepad::Button) -> Option { + use sdl3::gamepad::Button; + Some(match b { + Button::South => wire::BTN_A, + Button::East => wire::BTN_B, + Button::West => wire::BTN_X, + Button::North => wire::BTN_Y, + Button::Back => wire::BTN_BACK, + Button::Start => wire::BTN_START, + Button::Guide => wire::BTN_GUIDE, + Button::LeftStick => wire::BTN_LS_CLICK, + Button::RightStick => wire::BTN_RS_CLICK, + Button::LeftShoulder => wire::BTN_LB, + Button::RightShoulder => wire::BTN_RB, + Button::DPadUp => wire::BTN_DPAD_UP, + Button::DPadDown => wire::BTN_DPAD_DOWN, + Button::DPadLeft => wire::BTN_DPAD_LEFT, + Button::DPadRight => wire::BTN_DPAD_RIGHT, + Button::Touchpad => wire::BTN_TOUCHPAD, + _ => return None, + }) +} + +/// SDL axis → (wire axis id, wire value). SDL sticks are +y = down; the wire (XInput +/// convention) is +y = up. SDL triggers span 0..32767; the wire wants 0..255. +fn axis_value(axis: sdl3::gamepad::Axis, v: i16) -> (u32, i32) { + use sdl3::gamepad::Axis; + match axis { + Axis::LeftX => (wire::AXIS_LS_X, v as i32), + Axis::LeftY => (wire::AXIS_LS_Y, -(v as i32).max(-32767)), + Axis::RightX => (wire::AXIS_RS_X, v as i32), + Axis::RightY => (wire::AXIS_RS_Y, -(v as i32).max(-32767)), + Axis::TriggerLeft => (wire::AXIS_LT, (v as i32).clamp(0, 32767) >> 7), + Axis::TriggerRight => (wire::AXIS_RT, (v as i32).clamp(0, 32767) >> 7), + } +} + +fn run(connector: &NativeClient, stop: &AtomicBool) -> Result<(), String> { + // Off-main-thread + no video subsystem: keep SDL away from signals, poll pads on its + // own thread. + sdl3::hint::set("SDL_NO_SIGNAL_HANDLERS", "1"); + sdl3::hint::set("SDL_JOYSTICK_THREAD", "1"); + let sdl = sdl3::init().map_err(|e| e.to_string())?; + let subsystem = sdl.gamepad().map_err(|e| e.to_string())?; + let mut pump = sdl.event_pump().map_err(|e| e.to_string())?; + + let mut active: Option = None; + let pad_id = |p: &Option| -> Option { + p.as_ref().and_then(|p| p.id().ok()).map(|id| id.0) + }; + // Last sent wire value per axis id — suppress no-op repeats (SDL re-reports). + let mut last_axis = [i32::MIN; 6]; + + while !stop.load(Ordering::SeqCst) { + while let Some(event) = pump.poll_event() { + use sdl3::event::Event; + match event { + Event::ControllerDeviceAdded { which, .. } => { + if active.is_none() { + match subsystem.open(sdl3::sys::joystick::SDL_JoystickID(which)) { + Ok(pad) => { + tracing::info!( + name = pad.name().unwrap_or_default(), + "gamepad attached as pad 0" + ); + active = Some(pad); + last_axis = [i32::MIN; 6]; + } + Err(e) => tracing::warn!(error = %e, "gamepad open failed"), + } + } + } + Event::ControllerDeviceRemoved { which, .. } => { + if pad_id(&active) == Some(which) { + tracing::info!("gamepad detached"); + active = None; + } + } + Event::ControllerButtonDown { which, button, .. } => { + if pad_id(&active) == Some(which) { + if let Some(bit) = button_bit(button) { + send(connector, InputKind::GamepadButton, bit, 1); + } + } + } + Event::ControllerButtonUp { which, button, .. } => { + if pad_id(&active) == Some(which) { + if let Some(bit) = button_bit(button) { + send(connector, InputKind::GamepadButton, bit, 0); + } + } + } + Event::ControllerAxisMotion { + which, axis, value, .. + } if pad_id(&active) == Some(which) => { + let (id, v) = axis_value(axis, value); + if last_axis[id as usize] != v { + last_axis[id as usize] = v; + send(connector, InputKind::GamepadAxis, id, v); + } + } + _ => {} + } + } + + // Feedback planes (this thread is their single consumer). The host re-sends + // rumble state periodically, so a generous duration with refresh-on-update is + // safe — a dropped stop heals within ~500 ms. + while let Ok((pad, low, high)) = connector.next_rumble(Duration::ZERO) { + if pad == 0 { + if let Some(p) = active.as_mut() { + let _ = p.set_rumble(low, high, 5_000); + } + } + } + loop { + match connector.next_hidout(Duration::ZERO) { + Ok(HidOutput::Led { pad: 0, r, g, b }) => { + if let Some(p) = active.as_mut() { + let _ = p.set_led(r, g, b); + } + } + Ok(HidOutput::PlayerLeds { .. }) => {} // TODO: SDL player-index mapping + Ok(HidOutput::Trigger { .. }) => {} // TODO: DS5 effect packet replay + Ok(_) => {} + Err(_) => break, + } + } + + std::thread::sleep(Duration::from_millis(2)); + } + Ok(()) +} diff --git a/crates/punktfunk-client-linux/src/keymap.rs b/crates/punktfunk-client-linux/src/keymap.rs new file mode 100644 index 0000000..a75d726 --- /dev/null +++ b/crates/punktfunk-client-linux/src/keymap.rs @@ -0,0 +1,203 @@ +//! Local key/button codes → the punktfunk input wire contract. +//! +//! The wire carries Windows Virtual-Key codes (the GameStream convention; the host maps +//! them back with `inject::vk_to_evdev`). GTK hands us the hardware keycode, which on +//! Wayland (and X11) is the evdev code + 8 — so this table is the exact inverse of the +//! host's, keyed on evdev codes. Layout-independent by construction: positional keys map +//! positionally, exactly what a game expects. + +/// Map a Linux evdev key code to the Windows VK code the host expects. `None` = a key the +/// wire contract doesn't cover (media keys etc.) — drop it rather than guess. +pub fn evdev_to_vk(evdev: u16) -> Option { + Some(match evdev { + // --- Navigation / editing / whitespace --- + 14 => 0x08, // KEY_BACKSPACE -> VK_BACK + 15 => 0x09, // KEY_TAB -> VK_TAB + 28 => 0x0D, // KEY_ENTER -> VK_RETURN + 119 => 0x13, // KEY_PAUSE -> VK_PAUSE + 58 => 0x14, // KEY_CAPSLOCK -> VK_CAPITAL + 1 => 0x1B, // KEY_ESC -> VK_ESCAPE + 57 => 0x20, // KEY_SPACE -> VK_SPACE + 104 => 0x21, // KEY_PAGEUP -> VK_PRIOR + 109 => 0x22, // KEY_PAGEDOWN -> VK_NEXT + 107 => 0x23, // KEY_END -> VK_END + 102 => 0x24, // KEY_HOME -> VK_HOME + 105 => 0x25, // KEY_LEFT -> VK_LEFT + 103 => 0x26, // KEY_UP -> VK_UP + 106 => 0x27, // KEY_RIGHT -> VK_RIGHT + 108 => 0x28, // KEY_DOWN -> VK_DOWN + 99 => 0x2C, // KEY_SYSRQ -> VK_SNAPSHOT + 110 => 0x2D, // KEY_INSERT -> VK_INSERT + 111 => 0x2E, // KEY_DELETE -> VK_DELETE + + // --- Digit row (KEY_1..KEY_9 are 2..10, KEY_0 is 11) --- + 11 => 0x30, + 2 => 0x31, + 3 => 0x32, + 4 => 0x33, + 5 => 0x34, + 6 => 0x35, + 7 => 0x36, + 8 => 0x37, + 9 => 0x38, + 10 => 0x39, + + // --- Letters (evdev order is QWERTY rows, not alphabetical) --- + 30 => 0x41, // A + 48 => 0x42, // B + 46 => 0x43, // C + 32 => 0x44, // D + 18 => 0x45, // E + 33 => 0x46, // F + 34 => 0x47, // G + 35 => 0x48, // H + 23 => 0x49, // I + 36 => 0x4A, // J + 37 => 0x4B, // K + 38 => 0x4C, // L + 50 => 0x4D, // M + 49 => 0x4E, // N + 24 => 0x4F, // O + 25 => 0x50, // P + 16 => 0x51, // Q + 19 => 0x52, // R + 31 => 0x53, // S + 20 => 0x54, // T + 22 => 0x55, // U + 47 => 0x56, // V + 17 => 0x57, // W + 45 => 0x58, // X + 21 => 0x59, // Y + 44 => 0x5A, // Z + + // --- Meta / context-menu --- + 125 => 0x5B, // KEY_LEFTMETA -> VK_LWIN + 126 => 0x5C, // KEY_RIGHTMETA -> VK_RWIN + 127 => 0x5D, // KEY_COMPOSE -> VK_APPS + + // --- Numpad --- + 82 => 0x60, // KP0 + 79 => 0x61, + 80 => 0x62, + 81 => 0x63, + 75 => 0x64, + 76 => 0x65, + 77 => 0x66, + 71 => 0x67, + 72 => 0x68, + 73 => 0x69, // KP9 + 55 => 0x6A, // KEY_KPASTERISK -> VK_MULTIPLY + 78 => 0x6B, // KEY_KPPLUS -> VK_ADD + 96 => 0x6C, // KEY_KPENTER -> VK_SEPARATOR + 74 => 0x6D, // KEY_KPMINUS -> VK_SUBTRACT + 83 => 0x6E, // KEY_KPDOT -> VK_DECIMAL + 98 => 0x6F, // KEY_KPSLASH -> VK_DIVIDE + + // --- Function keys --- + 59 => 0x70, // F1 + 60 => 0x71, + 61 => 0x72, + 62 => 0x73, + 63 => 0x74, + 64 => 0x75, + 65 => 0x76, + 66 => 0x77, + 67 => 0x78, + 68 => 0x79, // F10 + 87 => 0x7A, // F11 + 88 => 0x7B, // F12 + + // --- Locks --- + 69 => 0x90, // KEY_NUMLOCK -> VK_NUMLOCK + 70 => 0x91, // KEY_SCROLLLOCK -> VK_SCROLL + + // --- Left/right modifiers (specific VKs; the host maps both generics here too) --- + 42 => 0xA0, // KEY_LEFTSHIFT -> VK_LSHIFT + 54 => 0xA1, // KEY_RIGHTSHIFT -> VK_RSHIFT + 29 => 0xA2, // KEY_LEFTCTRL -> VK_LCONTROL + 97 => 0xA3, // KEY_RIGHTCTRL -> VK_RCONTROL + 56 => 0xA4, // KEY_LEFTALT -> VK_LMENU + 100 => 0xA5, // KEY_RIGHTALT -> VK_RMENU + + // --- OEM punctuation (US-layout positions) --- + 39 => 0xBA, // KEY_SEMICOLON -> VK_OEM_1 + 13 => 0xBB, // KEY_EQUAL -> VK_OEM_PLUS + 51 => 0xBC, // KEY_COMMA -> VK_OEM_COMMA + 12 => 0xBD, // KEY_MINUS -> VK_OEM_MINUS + 52 => 0xBE, // KEY_DOT -> VK_OEM_PERIOD + 53 => 0xBF, // KEY_SLASH -> VK_OEM_2 + 41 => 0xC0, // KEY_GRAVE -> VK_OEM_3 + 26 => 0xDB, // KEY_LEFTBRACE -> VK_OEM_4 + 43 => 0xDC, // KEY_BACKSLASH -> VK_OEM_5 + 27 => 0xDD, // KEY_RIGHTBRACE -> VK_OEM_6 + 40 => 0xDE, // KEY_APOSTROPHE -> VK_OEM_7 + 86 => 0xE2, // KEY_102ND -> VK_OEM_102 + + _ => return None, + }) +} + +/// Map a GTK/GDK mouse button number to the GameStream button id the wire expects +/// (1=left, 2=middle, 3=right, 4=X1, 5=X2). GDK reports back/forward as 8/9. +pub fn gdk_button_to_gs(button: u32) -> Option { + Some(match button { + 1 => 1, + 2 => 2, + 3 => 3, + 8 => 4, + 9 => 5, + _ => return None, + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The table must be the exact inverse of the host's `vk_to_evdev` for every key the + /// host knows (modulo the generic-modifier VKs, which collapse onto the same evdev + /// codes as the specific left-hand ones). + #[test] + fn roundtrips_through_the_host_table() { + // Mirror of the host's table (inject::vk_to_evdev), generic modifiers excluded. + let host_pairs: &[(u8, u16)] = &[ + (0x08, 14), + (0x09, 15), + (0x0D, 28), + (0x13, 119), + (0x14, 58), + (0x1B, 1), + (0x20, 57), + (0x21, 104), + (0x22, 109), + (0x23, 107), + (0x24, 102), + (0x25, 105), + (0x26, 103), + (0x27, 106), + (0x28, 108), + (0x2C, 99), + (0x2D, 110), + (0x2E, 111), + (0x30, 11), + (0x31, 2), + (0x39, 10), + (0x41, 30), + (0x5A, 44), + (0x5B, 125), + (0x60, 82), + (0x69, 73), + (0x70, 59), + (0x7B, 88), + (0x90, 69), + (0xA0, 42), + (0xA5, 100), + (0xBA, 39), + (0xE2, 86), + ]; + for &(vk, evdev) in host_pairs { + assert_eq!(evdev_to_vk(evdev), Some(vk), "evdev {evdev}"); + } + assert_eq!(evdev_to_vk(113), None); // KEY_MUTE — not in the wire contract + } +} diff --git a/crates/punktfunk-client-linux/src/main.rs b/crates/punktfunk-client-linux/src/main.rs new file mode 100644 index 0000000..8688125 --- /dev/null +++ b/crates/punktfunk-client-linux/src/main.rs @@ -0,0 +1,42 @@ +//! `punktfunk-client` — the native Linux punktfunk/1 client (design: Option A, 2026-06-12). +//! +//! GTK4/libadwaita shell · `NativeClient` linked as a crate (no C ABI) · FFmpeg decode → +//! `GtkGraphicsOffload` present · PipeWire audio · SDL3 gamepads. The trust surface +//! mirrors the Apple client: persistent identity, TOFU prompt with the host fingerprint, +//! SPAKE2 PIN pairing. + +#[cfg(target_os = "linux")] +mod app; +#[cfg(target_os = "linux")] +mod audio; +#[cfg(target_os = "linux")] +mod discovery; +#[cfg(target_os = "linux")] +mod gamepad; +#[cfg(target_os = "linux")] +mod keymap; +#[cfg(target_os = "linux")] +mod session; +#[cfg(target_os = "linux")] +mod trust; +#[cfg(target_os = "linux")] +mod ui_hosts; +#[cfg(target_os = "linux")] +mod ui_settings; +#[cfg(target_os = "linux")] +mod ui_stream; +#[cfg(target_os = "linux")] +mod video; + +#[cfg(target_os = "linux")] +fn main() -> gtk::glib::ExitCode { + app::run() +} + +/// GTK4/PipeWire/SDL3 are Linux turf; this stub keeps `cargo build --workspace` green on +/// macOS (the Mac client lives in clients/apple). +#[cfg(not(target_os = "linux"))] +fn main() { + eprintln!("punktfunk-client is Linux-only — the macOS client lives in clients/apple"); + std::process::exit(2); +} diff --git a/crates/punktfunk-client-linux/src/session.rs b/crates/punktfunk-client-linux/src/session.rs new file mode 100644 index 0000000..aceb8b0 --- /dev/null +++ b/crates/punktfunk-client-linux/src/session.rs @@ -0,0 +1,226 @@ +//! Session controller: one worker thread runs connect → pump (video pull + decode, audio +//! pull + Opus decode, stats), feeding the GTK main loop over channels. The UI keeps the +//! `Arc` from the `Connected` event for direct input sends (no extra hop on +//! the input path) — `NativeClient` is `Sync`, planes stay one-consumer-per-thread: +//! video+audio here, rumble+hidout on the gamepad thread. + +use crate::video::{DecodedFrame, Decoder}; +use crate::{audio, gamepad}; +use punktfunk_core::client::NativeClient; +use punktfunk_core::config::{CompositorPref, GamepadPref, Mode}; +use punktfunk_core::PunktfunkError; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Arc; +use std::time::{Duration, Instant}; + +pub struct SessionParams { + pub host: String, + pub port: u16, + pub mode: Mode, + pub gamepad: GamepadPref, + pub bitrate_kbps: u32, + /// Pinned host fingerprint; `None` = trust on first use (caller persists the observed one). + pub pin: Option<[u8; 32]>, + pub identity: (String, String), +} + +#[derive(Clone, Copy, Default)] +pub struct Stats { + pub fps: f32, + pub mbps: f32, + pub decode_ms: f32, + /// Median capture→decoded latency over the last window (host-clock corrected). + pub latency_ms: f32, +} + +pub enum SessionEvent { + Connected { + connector: Arc, + mode: Mode, + fingerprint: [u8; 32], + }, + Failed(String), + Ended(Option), + Stats(Stats), +} + +pub struct SessionHandle { + pub events: async_channel::Receiver, + pub frames: async_channel::Receiver, + pub stop: Arc, +} + +pub fn start(params: SessionParams) -> SessionHandle { + let (ev_tx, ev_rx) = async_channel::unbounded(); + // Tiny frame queue, newest wins: force_send displaces the oldest when the UI lags. + let (frame_tx, frame_rx) = async_channel::bounded(2); + let stop = Arc::new(AtomicBool::new(false)); + let stop_w = stop.clone(); + std::thread::Builder::new() + .name("punktfunk-session".into()) + .spawn(move || pump(params, ev_tx, frame_tx, stop_w)) + .expect("spawn session thread"); + SessionHandle { + events: ev_rx, + frames: frame_rx, + stop, + } +} + +fn now_ns() -> u64 { + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.as_nanos() as u64) + .unwrap_or(0) +} + +fn pump( + params: SessionParams, + ev_tx: async_channel::Sender, + frame_tx: async_channel::Sender, + stop: Arc, +) { + let connector = match NativeClient::connect( + ¶ms.host, + params.port, + params.mode, + CompositorPref::Auto, + params.gamepad, + params.bitrate_kbps, + params.pin, + Some(params.identity), + Duration::from_secs(15), + ) { + Ok(c) => Arc::new(c), + Err(e) => { + let msg = match e { + PunktfunkError::Crypto => { + "Host identity rejected — wrong fingerprint, or the host requires pairing" + .to_string() + } + PunktfunkError::Timeout => "Connection timed out".to_string(), + other => format!("Connect failed: {other:?}"), + }; + let _ = ev_tx.send_blocking(SessionEvent::Failed(msg)); + return; + } + }; + let _ = ev_tx.send_blocking(SessionEvent::Connected { + connector: connector.clone(), + mode: connector.mode(), + fingerprint: connector.host_fingerprint, + }); + + let mut decoder = match Decoder::new() { + Ok(d) => d, + Err(e) => { + let _ = ev_tx.send_blocking(SessionEvent::Ended(Some(format!("video decoder: {e}")))); + return; + } + }; + // Audio and gamepads are best-effort: a session without them still streams. + let player = audio::AudioPlayer::spawn() + .map_err(|e| tracing::warn!(error = %e, "audio disabled")) + .ok(); + let mut opus_dec = opus::Decoder::new(48_000, opus::Channels::Stereo) + .map_err(|e| tracing::warn!(error = %e, "opus decoder failed — audio disabled")) + .ok(); + let gamepad_thread = gamepad::spawn(connector.clone(), stop.clone()); + + let clock_offset = connector.clock_offset_ns; + let mut total_frames = 0u64; + let mut window_start = Instant::now(); + let mut frames_n = 0u32; + let mut bytes_n = 0u64; + let mut decode_us_sum = 0u64; + let mut lat_us: Vec = Vec::with_capacity(256); + let mut pcm = vec![0f32; 5760 * 2]; // decode scratch: max Opus frame (120 ms stereo) + + let end: Option = loop { + if stop.load(Ordering::SeqCst) { + break None; + } + match connector.next_frame(Duration::from_millis(4)) { + Ok(frame) => { + let t0 = Instant::now(); + match decoder.decode(&frame.data) { + Ok(Some(decoded)) => { + total_frames += 1; + if total_frames == 1 { + tracing::info!( + width = decoded.width, + height = decoded.height, + "first frame decoded" + ); + } + // Latency: our wall clock expressed in the host's capture clock, + // minus the host-stamped capture pts (same math as client-rs). + let lat = (now_ns() as i128 + clock_offset as i128 - frame.pts_ns as i128) + .max(0) as u64; + if lat > 0 && lat < 10_000_000_000 { + lat_us.push(lat / 1000); + } + decode_us_sum += t0.elapsed().as_micros() as u64; + frames_n += 1; + bytes_n += frame.data.len() as u64; + let _ = frame_tx.force_send(decoded); + } + Ok(None) => {} + // Survivable (loss until the next IDR/RFI recovery) — keep feeding. + Err(e) => tracing::debug!(error = %e, "decode error (recovering)"), + } + } + Err(PunktfunkError::NoFrame) => {} + Err(PunktfunkError::Closed) => break Some("Host ended the session".to_string()), + Err(e) => break Some(format!("session: {e:?}")), + } + + // Drain audio between frames (packets land every 5 ms; the queue holds 320 ms). + while let Ok(pkt) = connector.next_audio(Duration::ZERO) { + if let (Some(player), Some(dec)) = (&player, opus_dec.as_mut()) { + match dec.decode_float(&pkt.data, &mut pcm, false) { + Ok(samples) => player.push(pcm[..samples * 2].to_vec()), + Err(e) => tracing::debug!(error = %e, "opus decode"), + } + } + } + + if window_start.elapsed() >= Duration::from_secs(1) { + let secs = window_start.elapsed().as_secs_f32(); + lat_us.sort_unstable(); + let p50 = lat_us.get(lat_us.len() / 2).copied().unwrap_or(0); + tracing::debug!( + fps = frames_n, + lat_p50_us = p50, + total_frames, + "stream window" + ); + let _ = ev_tx.try_send(SessionEvent::Stats(Stats { + fps: frames_n as f32 / secs, + mbps: bytes_n as f32 * 8.0 / 1e6 / secs, + decode_ms: if frames_n > 0 { + decode_us_sum as f32 / frames_n as f32 / 1000.0 + } else { + 0.0 + }, + latency_ms: p50 as f32 / 1000.0, + })); + window_start = Instant::now(); + frames_n = 0; + bytes_n = 0; + decode_us_sum = 0; + lat_us.clear(); + } + }; + + tracing::info!( + total_frames, + reason = end.as_deref().unwrap_or("user"), + "session ended" + ); + stop.store(true, Ordering::SeqCst); // take the gamepad thread down with us + if let Some(t) = gamepad_thread { + let _ = t.join(); + } + let _ = ev_tx.send_blocking(SessionEvent::Ended(end)); +} diff --git a/crates/punktfunk-client-linux/src/trust.rs b/crates/punktfunk-client-linux/src/trust.rs new file mode 100644 index 0000000..7052bcb --- /dev/null +++ b/crates/punktfunk-client-linux/src/trust.rs @@ -0,0 +1,154 @@ +//! Client identity, the known-hosts (pinned fingerprint) store, and app settings. +//! +//! The identity shares `~/.config/punktfunk/client-{cert,key}.pem` with `punktfunk-client-rs` +//! so a box pairs once whichever client it uses. + +use anyhow::{anyhow, Context, Result}; +use punktfunk_core::quic::endpoint; +use serde::{Deserialize, Serialize}; +use std::path::PathBuf; + +pub fn config_dir() -> Result { + let home = std::env::var("HOME").context("HOME unset")?; + Ok(PathBuf::from(home).join(".config/punktfunk")) +} + +/// This client's persistent identity, generated on first use — presented on every connect +/// so hosts can recognize it once paired. +pub fn load_or_create_identity() -> Result<(String, String)> { + let dir = config_dir()?; + let (cp, kp) = (dir.join("client-cert.pem"), dir.join("client-key.pem")); + if let (Ok(c), Ok(k)) = (std::fs::read_to_string(&cp), std::fs::read_to_string(&kp)) { + return Ok((c, k)); + } + let (c, k) = endpoint::generate_identity().map_err(|e| anyhow!("generate identity: {e}"))?; + std::fs::create_dir_all(&dir)?; + std::fs::write(&cp, &c)?; + std::fs::write(&kp, &k)?; + tracing::info!(cert = %cp.display(), "generated client identity"); + Ok((c, k)) +} + +pub fn hex(fp: &[u8; 32]) -> String { + fp.iter().map(|b| format!("{b:02x}")).collect() +} + +pub fn parse_hex32(s: &str) -> Option<[u8; 32]> { + if s.len() != 64 { + return None; + } + let mut out = [0u8; 32]; + for (i, b) in out.iter_mut().enumerate() { + *b = u8::from_str_radix(&s[2 * i..2 * i + 2], 16).ok()?; + } + Some(out) +} + +/// One trusted host: its pinned certificate fingerprint plus how we got there (TOFU or a +/// PIN ceremony) and where we last reached it. +#[derive(Clone, Debug, Serialize, Deserialize)] +pub struct KnownHost { + pub name: String, + pub addr: String, + pub port: u16, + /// SHA-256 of the host certificate, lowercase hex — the pin for every later connect. + pub fp_hex: String, + /// True if trust came from the SPAKE2 PIN ceremony (vs. trust-on-first-use). + pub paired: bool, +} + +#[derive(Default, Serialize, Deserialize)] +pub struct KnownHosts { + pub hosts: Vec, +} + +impl KnownHosts { + fn path() -> Result { + Ok(config_dir()?.join("client-known-hosts.json")) + } + + pub fn load() -> KnownHosts { + Self::path() + .and_then(|p| Ok(std::fs::read_to_string(p)?)) + .ok() + .and_then(|s| serde_json::from_str(&s).ok()) + .unwrap_or_default() + } + + pub fn save(&self) -> Result<()> { + let p = Self::path()?; + std::fs::create_dir_all(p.parent().unwrap())?; + std::fs::write(&p, serde_json::to_string_pretty(self)?)?; + Ok(()) + } + + pub fn find_by_fp(&self, fp_hex: &str) -> Option<&KnownHost> { + self.hosts.iter().find(|h| h.fp_hex == fp_hex) + } + + pub fn find_by_addr(&self, addr: &str, port: u16) -> Option<&KnownHost> { + self.hosts.iter().find(|h| h.addr == addr && h.port == port) + } + + /// Insert or refresh an entry, keyed by fingerprint. `paired` only ever upgrades + /// (a later TOFU connect must not demote a PIN-paired host). + pub fn upsert(&mut self, entry: KnownHost) { + if let Some(h) = self.hosts.iter_mut().find(|h| h.fp_hex == entry.fp_hex) { + h.name = entry.name; + h.addr = entry.addr; + h.port = entry.port; + h.paired |= entry.paired; + } else { + self.hosts.push(entry); + } + } +} + +/// App settings, persisted as JSON. Stringly-typed gamepad pref so the file stays +/// readable; parsed with `GamepadPref::from_name` at connect time. +#[derive(Clone, Serialize, Deserialize)] +pub struct Settings { + pub width: u32, + pub height: u32, + pub refresh_hz: u32, + /// Requested encoder bitrate (kbps); 0 = host default. + pub bitrate_kbps: u32, + pub gamepad: String, + /// Grab compositor shortcuts (Alt+Tab, Super…) while streaming. + pub inhibit_shortcuts: bool, +} + +impl Default for Settings { + fn default() -> Self { + Settings { + width: 1920, + height: 1080, + refresh_hz: 60, + bitrate_kbps: 0, + gamepad: "auto".into(), + inhibit_shortcuts: true, + } + } +} + +impl Settings { + fn path() -> Result { + Ok(config_dir()?.join("client-gtk-settings.json")) + } + + pub fn load() -> Settings { + Self::path() + .and_then(|p| Ok(std::fs::read_to_string(p)?)) + .ok() + .and_then(|s| serde_json::from_str(&s).ok()) + .unwrap_or_default() + } + + pub fn save(&self) { + let Ok(p) = Self::path() else { return }; + let _ = std::fs::create_dir_all(p.parent().unwrap()); + if let Ok(s) = serde_json::to_string_pretty(self) { + let _ = std::fs::write(&p, s); + } + } +} diff --git a/crates/punktfunk-client-linux/src/ui_hosts.rs b/crates/punktfunk-client-linux/src/ui_hosts.rs new file mode 100644 index 0000000..3dffcca --- /dev/null +++ b/crates/punktfunk-client-linux/src/ui_hosts.rs @@ -0,0 +1,175 @@ +//! The hosts page: live mDNS discovery list + manual connect entry. + +use crate::discovery::{self, DiscoveredHost}; +use adw::prelude::*; +use gtk::glib; +use std::cell::RefCell; +use std::collections::HashMap; +use std::rc::Rc; + +/// What the user asked to connect to. `fp_hex` comes from the mDNS TXT record when the +/// host was discovered (drives the TOFU prompt *before* connecting); manual entries have +/// none and trust on first use. +#[derive(Clone, Debug)] +pub struct ConnectRequest { + pub name: String, + pub addr: String, + pub port: u16, + pub fp_hex: Option, + pub pair_required: bool, +} + +pub fn new( + on_connect: Rc, + on_settings: Rc, +) -> adw::NavigationPage { + let list = gtk::ListBox::new(); + list.add_css_class("boxed-list"); + list.set_selection_mode(gtk::SelectionMode::None); + let placeholder = gtk::Label::new(Some("Searching the LAN for hosts…")); + placeholder.add_css_class("dim-label"); + placeholder.set_margin_top(24); + placeholder.set_margin_bottom(24); + list.set_placeholder(Some(&placeholder)); + + // key → (row, latest advert); the activation closure looks the advert up by key so + // re-adverts (new address, pairing flipped) take effect without rebuilding rows. + type Rows = Rc>>; + let rows: Rows = Rc::new(RefCell::new(HashMap::new())); + + { + let rx = discovery::browse(); + let rows = rows.clone(); + let list = list.downgrade(); + let on_connect = on_connect.clone(); + glib::spawn_future_local(async move { + while let Ok(host) = rx.recv().await { + let Some(list) = list.upgrade() else { break }; + let mut map = rows.borrow_mut(); + let subtitle = format!( + "{}:{} · pairing {}", + host.addr, + host.port, + if host.pair.is_empty() { + "optional" + } else { + &host.pair + } + ); + if let Some((row, stored)) = map.get_mut(&host.key) { + row.set_title(&host.name); + row.set_subtitle(&subtitle); + *stored = host; + } else { + let row = adw::ActionRow::builder() + .title(&host.name) + .subtitle(&subtitle) + .activatable(true) + .build(); + row.add_suffix(>k::Image::from_icon_name("go-next-symbolic")); + { + let rows = rows.clone(); + let key = host.key.clone(); + let on_connect = on_connect.clone(); + row.connect_activated(move |_| { + if let Some((_, h)) = rows.borrow().get(&key) { + on_connect(ConnectRequest { + name: h.name.clone(), + addr: h.addr.clone(), + port: h.port, + fp_hex: (!h.fp_hex.is_empty()).then(|| h.fp_hex.clone()), + pair_required: h.pair == "required", + }); + } + }); + } + list.append(&row); + map.insert(host.key.clone(), (row, host)); + } + } + }); + } + + // Manual connect: host:port (punktfunk/1 default port 9777). + let manual = adw::EntryRow::builder().title("host:port").build(); + let connect_btn = gtk::Button::with_label("Connect"); + connect_btn.set_valign(gtk::Align::Center); + connect_btn.add_css_class("suggested-action"); + manual.add_suffix(&connect_btn); + let submit = { + let manual = manual.clone(); + let on_connect = on_connect.clone(); + move || { + let text = manual.text().to_string(); + let text = text.trim(); + if text.is_empty() { + return; + } + let (addr, port) = match text.rsplit_once(':') { + Some((a, p)) => match p.parse::() { + Ok(port) => (a.to_string(), port), + Err(_) => return, + }, + None => (text.to_string(), 9777), + }; + on_connect(ConnectRequest { + name: addr.clone(), + addr, + port, + fp_hex: None, + pair_required: false, + }); + } + }; + { + let submit = submit.clone(); + connect_btn.connect_clicked(move |_| submit()); + } + manual.connect_entry_activated(move |_| submit()); + + let manual_list = gtk::ListBox::new(); + manual_list.add_css_class("boxed-list"); + manual_list.set_selection_mode(gtk::SelectionMode::None); + manual_list.append(&manual); + + let content = gtk::Box::new(gtk::Orientation::Vertical, 18); + content.set_margin_top(24); + content.set_margin_bottom(24); + content.set_margin_start(12); + content.set_margin_end(12); + let discovered_label = gtk::Label::new(Some("Hosts on this network")); + discovered_label.add_css_class("heading"); + discovered_label.set_halign(gtk::Align::Start); + content.append(&discovered_label); + content.append(&list); + let manual_label = gtk::Label::new(Some("Manual connection")); + manual_label.add_css_class("heading"); + manual_label.set_halign(gtk::Align::Start); + content.append(&manual_label); + content.append(&manual_list); + + let clamp = adw::Clamp::builder() + .maximum_size(560) + .child(&content) + .build(); + let scrolled = gtk::ScrolledWindow::builder() + .hscrollbar_policy(gtk::PolicyType::Never) + .child(&clamp) + .build(); + + let header = adw::HeaderBar::new(); + let settings_btn = gtk::Button::from_icon_name("preferences-system-symbolic"); + settings_btn.set_tooltip_text(Some("Preferences")); + settings_btn.connect_clicked(move |_| on_settings()); + header.pack_end(&settings_btn); + + let toolbar = adw::ToolbarView::new(); + toolbar.add_top_bar(&header); + toolbar.set_content(Some(&scrolled)); + + adw::NavigationPage::builder() + .title("Punktfunk") + .tag("hosts") + .child(&toolbar) + .build() +} diff --git a/crates/punktfunk-client-linux/src/ui_settings.rs b/crates/punktfunk-client-linux/src/ui_settings.rs new file mode 100644 index 0000000..03ce54b --- /dev/null +++ b/crates/punktfunk-client-linux/src/ui_settings.rs @@ -0,0 +1,93 @@ +//! Preferences dialog: stream mode, bitrate, gamepad type, capture behavior. Written +//! back to disk when the dialog closes. + +use crate::trust::Settings; +use adw::prelude::*; +use std::cell::RefCell; +use std::rc::Rc; + +const RESOLUTIONS: &[(u32, u32)] = &[(1280, 720), (1920, 1080), (2560, 1440), (3840, 2160)]; +const REFRESH: &[u32] = &[30, 60, 90, 120, 144, 165, 240]; +const GAMEPADS: &[&str] = &["auto", "xbox360", "dualsense"]; + +pub fn show(parent: &impl IsA, settings: Rc>) { + let page = adw::PreferencesPage::new(); + + let stream = adw::PreferencesGroup::builder().title("Stream").build(); + let res_names: Vec = RESOLUTIONS + .iter() + .map(|(w, h)| format!("{w} × {h}")) + .collect(); + let res_row = adw::ComboRow::builder() + .title("Resolution") + .subtitle("The host creates a virtual output at exactly this size") + .model(>k::StringList::new( + &res_names.iter().map(String::as_str).collect::>(), + )) + .build(); + let hz_row = adw::ComboRow::builder() + .title("Refresh rate") + .model(>k::StringList::new( + &REFRESH + .iter() + .map(|r| format!("{r} Hz")) + .collect::>() + .iter() + .map(String::as_str) + .collect::>(), + )) + .build(); + let bitrate_row = adw::SpinRow::with_range(0.0, 500.0, 5.0); + bitrate_row.set_title("Bitrate"); + bitrate_row.set_subtitle("Mbit/s · 0 = host default"); + stream.add(&res_row); + stream.add(&hz_row); + stream.add(&bitrate_row); + + let input = adw::PreferencesGroup::builder().title("Input").build(); + let pad_row = adw::ComboRow::builder() + .title("Gamepad type") + .subtitle("The virtual pad the host creates (DualSense needs a Linux host)") + .model(>k::StringList::new(&["Auto", "Xbox 360", "DualSense"])) + .build(); + let inhibit_row = adw::SwitchRow::builder() + .title("Capture system shortcuts") + .subtitle("Forward Alt+Tab, Super, … to the host while streaming") + .build(); + input.add(&pad_row); + input.add(&inhibit_row); + + page.add(&stream); + page.add(&input); + + // Seed from the current settings. + { + let s = settings.borrow(); + let res_i = RESOLUTIONS + .iter() + .position(|&(w, h)| w == s.width && h == s.height) + .unwrap_or(1); + res_row.set_selected(res_i as u32); + let hz_i = REFRESH.iter().position(|&r| r == s.refresh_hz).unwrap_or(1); + hz_row.set_selected(hz_i as u32); + bitrate_row.set_value(f64::from(s.bitrate_kbps) / 1000.0); + let pad_i = GAMEPADS.iter().position(|&g| g == s.gamepad).unwrap_or(0); + pad_row.set_selected(pad_i as u32); + inhibit_row.set_active(s.inhibit_shortcuts); + } + + let dialog = adw::PreferencesDialog::new(); + dialog.set_title("Preferences"); + dialog.add(&page); + dialog.connect_closed(move |_| { + let mut s = settings.borrow_mut(); + let (w, h) = RESOLUTIONS[(res_row.selected() as usize).min(RESOLUTIONS.len() - 1)]; + (s.width, s.height) = (w, h); + s.refresh_hz = REFRESH[(hz_row.selected() as usize).min(REFRESH.len() - 1)]; + s.bitrate_kbps = (bitrate_row.value() * 1000.0) as u32; + s.gamepad = GAMEPADS[(pad_row.selected() as usize).min(GAMEPADS.len() - 1)].to_string(); + s.inhibit_shortcuts = inhibit_row.is_active(); + s.save(); + }); + dialog.present(Some(parent)); +} diff --git a/crates/punktfunk-client-linux/src/ui_stream.rs b/crates/punktfunk-client-linux/src/ui_stream.rs new file mode 100644 index 0000000..bed7108 --- /dev/null +++ b/crates/punktfunk-client-linux/src/ui_stream.rs @@ -0,0 +1,286 @@ +//! The stream page: decoded frames into a `GtkGraphicsOffload`-wrapped picture, local +//! input captured and forwarded on the wire contract. +//! +//! Input mapping: keys are hardware keycodes (evdev + 8 on Wayland) → VK via `keymap`, +//! layout-independent. Mouse is absolute (`MouseMoveAbs` scaled into the negotiated mode +//! through the letterbox transform) — relative/pointer-lock capture is the stage-2 +//! presenter's job. While streaming, compositor shortcuts are inhibited (configurable); +//! Ctrl+Alt+Shift+Q ends the session, F11 toggles fullscreen — everything else goes to +//! the host. + +use crate::keymap; +use crate::session::Stats; +use crate::video::DecodedFrame; +use adw::prelude::*; +use gtk::{gdk, glib}; +use punktfunk_core::client::NativeClient; +use punktfunk_core::input::{InputEvent, InputKind}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Arc; + +pub struct StreamPage { + pub page: adw::NavigationPage, + stats_label: gtk::Label, +} + +impl StreamPage { + pub fn update_stats(&self, s: Stats) { + self.stats_label.set_text(&format!( + "{:.0} fps · {:.1} Mbit/s · dec {:.1} ms · lat {:.1} ms", + s.fps, s.mbps, s.decode_ms, s.latency_ms + )); + } +} + +fn send(connector: &NativeClient, kind: InputKind, code: u32, x: i32, y: i32, flags: u32) { + let _ = connector.send_input(&InputEvent { + kind, + _pad: [0; 3], + code, + x, + y, + flags, + }); +} + +/// Widget coordinates → video pixel coordinates through the Contain-fit letterbox. +fn map_xy(widget: &impl IsA, connector: &NativeClient, x: f64, y: f64) -> (i32, i32) { + let w = widget.as_ref(); + let mode = connector.mode(); + let (ww, wh) = (w.width().max(1) as f64, w.height().max(1) as f64); + let (vw, vh) = (mode.width.max(1) as f64, mode.height.max(1) as f64); + let scale = (ww / vw).min(wh / vh); + let (ox, oy) = ((ww - vw * scale) / 2.0, (wh - vh * scale) / 2.0); + ( + (((x - ox) / scale).round()).clamp(0.0, vw - 1.0) as i32, + (((y - oy) / scale).round()).clamp(0.0, vh - 1.0) as i32, + ) +} + +#[allow(clippy::too_many_lines)] +pub fn new( + window: &adw::ApplicationWindow, + connector: Arc, + frames: async_channel::Receiver, + stop: Arc, + inhibit_shortcuts: bool, + title: &str, +) -> StreamPage { + let picture = gtk::Picture::new(); + picture.set_content_fit(gtk::ContentFit::Contain); + + // The offload path: with a dmabuf-backed texture (stage 1.5) this becomes a + // subsurface the compositor can scan out directly; with memory textures it is a + // no-op wrapper. Black letterboxing keeps fullscreen scanout-eligible. + let offload = gtk::GraphicsOffload::new(Some(&picture)); + offload.set_black_background(true); + + let stats_label = gtk::Label::new(None); + stats_label.add_css_class("osd"); + stats_label.add_css_class("numeric"); + stats_label.set_halign(gtk::Align::Start); + stats_label.set_valign(gtk::Align::Start); + stats_label.set_margin_start(12); + stats_label.set_margin_top(12); + + let overlay = gtk::Overlay::new(); + overlay.set_child(Some(&offload)); + overlay.add_overlay(&stats_label); + overlay.set_focusable(true); + // The remote cursor is in the video — hide the local one over the stream. + overlay.set_cursor(gdk::Cursor::from_name("none", None).as_ref()); + + let header = adw::HeaderBar::new(); + let fullscreen_btn = gtk::Button::from_icon_name("view-fullscreen-symbolic"); + fullscreen_btn.set_tooltip_text(Some("Fullscreen (F11)")); + { + let window = window.clone(); + fullscreen_btn.connect_clicked(move |_| { + if window.is_fullscreen() { + window.unfullscreen(); + } else { + window.fullscreen(); + } + }); + } + header.pack_end(&fullscreen_btn); + + let toolbar = adw::ToolbarView::new(); + toolbar.add_top_bar(&header); + toolbar.set_content(Some(&overlay)); + // Fullscreen = the stream and nothing else. + { + let toolbar = toolbar.clone(); + window.connect_fullscreened_notify(move |w| { + toolbar.set_reveal_top_bars(!w.is_fullscreen()); + }); + } + + let page = adw::NavigationPage::builder() + .title(title) + .tag("stream") + .child(&toolbar) + .build(); + + // --- Frame consumer: newest texture wins, set on the GTK frame clock's cadence. --- + { + let picture = picture.downgrade(); + glib::spawn_future_local(async move { + while let Ok(f) = frames.recv().await { + let Some(picture) = picture.upgrade() else { + break; + }; + let bytes = glib::Bytes::from_owned(f.rgba); + let tex = gdk::MemoryTexture::new( + f.width as i32, + f.height as i32, + gdk::MemoryFormat::R8g8b8a8, + &bytes, + f.stride, + ); + picture.set_paintable(Some(&tex)); + } + }); + } + + // --- Keyboard --- + { + let key = gtk::EventControllerKey::new(); + key.set_propagation_phase(gtk::PropagationPhase::Capture); + let conn = connector.clone(); + let stop_k = stop.clone(); + let window_k = window.clone(); + key.connect_key_pressed(move |_, keyval, keycode, state| { + let chord = gdk::ModifierType::CONTROL_MASK + | gdk::ModifierType::ALT_MASK + | gdk::ModifierType::SHIFT_MASK; + if state.contains(chord) && keyval.to_lower() == gdk::Key::q { + stop_k.store(true, Ordering::SeqCst); // ends the session → page pops + return glib::Propagation::Stop; + } + if keyval == gdk::Key::F11 { + if window_k.is_fullscreen() { + window_k.unfullscreen(); + } else { + window_k.fullscreen(); + } + return glib::Propagation::Stop; + } + if let Some(vk) = keycode + .checked_sub(8) + .and_then(|c| keymap::evdev_to_vk(c as u16)) + { + send(&conn, InputKind::KeyDown, vk as u32, 0, 0, 0); + } + glib::Propagation::Stop + }); + let conn = connector.clone(); + key.connect_key_released(move |_, _keyval, keycode, _state| { + if let Some(vk) = keycode + .checked_sub(8) + .and_then(|c| keymap::evdev_to_vk(c as u16)) + { + send(&conn, InputKind::KeyUp, vk as u32, 0, 0, 0); + } + }); + overlay.add_controller(key); + } + + // --- Mouse: absolute motion, buttons, wheel --- + { + let motion = gtk::EventControllerMotion::new(); + let conn = connector.clone(); + let target = overlay.downgrade(); + motion.connect_motion(move |_, x, y| { + if let Some(w) = target.upgrade() { + let (px, py) = map_xy(&w, &conn, x, y); + send(&conn, InputKind::MouseMoveAbs, 0, px, py, 0); + } + }); + overlay.add_controller(motion); + } + { + let click = gtk::GestureClick::builder().button(0).build(); + let conn = connector.clone(); + let target = overlay.downgrade(); + click.connect_pressed(move |g, _n, x, y| { + if let Some(w) = target.upgrade() { + w.grab_focus(); + let (px, py) = map_xy(&w, &conn, x, y); + send(&conn, InputKind::MouseMoveAbs, 0, px, py, 0); + } + if let Some(gs) = keymap::gdk_button_to_gs(g.current_button()) { + send(&conn, InputKind::MouseButtonDown, gs, 0, 0, 0); + } + }); + let conn = connector.clone(); + click.connect_released(move |g, _n, _x, _y| { + if let Some(gs) = keymap::gdk_button_to_gs(g.current_button()) { + send(&conn, InputKind::MouseButtonUp, gs, 0, 0, 0); + } + }); + overlay.add_controller(click); + } + { + let scroll = gtk::EventControllerScroll::new(gtk::EventControllerScrollFlags::BOTH_AXES); + let conn = connector.clone(); + scroll.connect_scroll(move |_, dx, dy| { + // The wire carries WHEEL_DELTA(120) units, positive = up / right; GTK's dy is + // positive = down. Smooth fractions survive — libei's discrete scroll is + // 120-based too. + let vy = (-dy * 120.0) as i32; + if vy != 0 { + send(&conn, InputKind::MouseScroll, 0, vy, 0, 0); + } + let vx = (dx * 120.0) as i32; + if vx != 0 { + send(&conn, InputKind::MouseScroll, 1, vx, 0, 0); + } + glib::Propagation::Stop + }); + overlay.add_controller(scroll); + } + + // --- Capture lifecycle: grab focus + compositor shortcuts while mapped. --- + { + let window = window.clone(); + overlay.connect_map(move |w| { + tracing::debug!("stream overlay mapped"); + w.grab_focus(); + if inhibit_shortcuts { + if let Some(tl) = window + .surface() + .and_then(|s| s.downcast::().ok()) + { + tl.inhibit_system_shortcuts(None::<&gdk::Event>); + } + } + }); + } + { + let window = window.clone(); + overlay.connect_unmap(move |_| { + if let Some(tl) = window + .surface() + .and_then(|s| s.downcast::().ok()) + { + tl.restore_system_shortcuts(); + } + }); + } + // The page's `hidden` fires once navigation away completes (back button, pop on + // session end) — NOT on the transient unmap/map cycle a NavigationView push performs. + { + let window = window.clone(); + let stop_h = stop.clone(); + page.connect_hidden(move |_| { + tracing::debug!("stream page hidden — ending session"); + if window.is_fullscreen() { + window.unfullscreen(); + } + stop_h.store(true, Ordering::SeqCst); + }); + } + + StreamPage { page, stats_label } +} diff --git a/crates/punktfunk-client-linux/src/video.rs b/crates/punktfunk-client-linux/src/video.rs new file mode 100644 index 0000000..5145a0d --- /dev/null +++ b/crates/punktfunk-client-linux/src/video.rs @@ -0,0 +1,86 @@ +//! Video decode: reassembled HEVC access units → RGBA frames for the GTK presenter. +//! +//! Stage 1 is libavcodec software decode + swscale to RGBA (`GdkMemoryTexture` upload on +//! the UI side). The host encodes zero-reorder streams (no B-frames, in-band parameter +//! sets on every IDR), so with `AV_CODEC_FLAG_LOW_DELAY` the decoder is strictly +//! one-in/one-out with no hidden queue. Slice threading only — frame threading would add +//! a frame of latency per extra thread. +//! +//! Stage 1.5 (Intel/AMD boxes): VAAPI hwaccel → DRM-PRIME dmabuf → `GdkDmabufTexture`, +//! slotting in behind the same `decode()` signature. Stage 2 (NVIDIA): Vulkan Video in +//! the bespoke presenter (see the design notes in docs-site). + +use anyhow::{anyhow, Context as _, Result}; +use ffmpeg::format::Pixel; +use ffmpeg::software::scaling; +use ffmpeg::util::frame::Video as AvFrame; +use ffmpeg_next as ffmpeg; + +/// One decoded frame, tightly enough packed for `GdkMemoryTexture` (which takes a stride). +pub struct DecodedFrame { + pub width: u32, + pub height: u32, + /// RGBA row stride in bytes (≥ width*4 — swscale pads rows for SIMD). + pub stride: usize, + pub rgba: Vec, +} + +pub struct Decoder { + decoder: ffmpeg::decoder::Video, + /// Rebuilt whenever the decoded format/size changes (mid-stream `Reconfigure`). + sws: Option<(scaling::Context, Pixel, u32, u32)>, +} + +impl Decoder { + pub fn new() -> Result { + ffmpeg::init().context("ffmpeg init")?; + let codec = + ffmpeg::decoder::find(ffmpeg::codec::Id::HEVC).ok_or(anyhow!("no HEVC decoder"))?; + let mut ctx = ffmpeg::codec::Context::new_with_codec(codec); + unsafe { + let raw = ctx.as_mut_ptr(); + (*raw).flags |= ffmpeg::ffi::AV_CODEC_FLAG_LOW_DELAY as i32; + // Slice threading adds no frame delay (frame threading adds thread_count-1). + (*raw).thread_type = ffmpeg::ffi::FF_THREAD_SLICE; + (*raw).thread_count = 0; // auto + } + let decoder = ctx.decoder().video().context("open HEVC decoder")?; + Ok(Decoder { decoder, sws: None }) + } + + /// Feed one access unit; returns the decoded frame (the host's streams are + /// one-in/one-out). A decode error after packet loss is survivable — log upstream and + /// keep feeding; the host's RFI/IDR recovery resynchronizes the reference chain. + pub fn decode(&mut self, au: &[u8]) -> Result> { + let packet = ffmpeg::Packet::copy(au); + self.decoder + .send_packet(&packet) + .map_err(|e| anyhow!("send_packet: {e}"))?; + let mut frame = AvFrame::empty(); + let mut out = None; + while self.decoder.receive_frame(&mut frame).is_ok() { + out = Some(self.convert_rgba(&frame)?); + } + Ok(out) + } + + fn convert_rgba(&mut self, frame: &AvFrame) -> Result { + let (fmt, w, h) = (frame.format(), frame.width(), frame.height()); + let rebuild = + !matches!(&self.sws, Some((_, f, sw, sh)) if *f == fmt && *sw == w && *sh == h); + if rebuild { + let ctx = scaling::Context::get(fmt, w, h, Pixel::RGBA, w, h, scaling::Flags::POINT) + .context("swscale context")?; + self.sws = Some((ctx, fmt, w, h)); + } + let (sws, ..) = self.sws.as_mut().unwrap(); + let mut rgba = AvFrame::empty(); + sws.run(frame, &mut rgba).map_err(|e| anyhow!("sws: {e}"))?; + Ok(DecodedFrame { + width: w, + height: h, + stride: rgba.stride(0), + rgba: rgba.data(0).to_vec(), + }) + } +} diff --git a/docs-site/content/docs/status.md b/docs-site/content/docs/status.md index 45a02c7..73ff23d 100644 --- a/docs-site/content/docs/status.md +++ b/docs-site/content/docs/status.md @@ -14,7 +14,7 @@ and the design in the [Implementation Plan](/docs/implementation-plan); this pag | **M1** — `punktfunk-core` + C ABI (protocol · FEC · crypto) | ✅ complete & hardened | | **M2** — GameStream host (Moonlight-compatible) | ✅ working end-to-end; HDR/surround-audio polish open | | **M3** — `punktfunk/1` native protocol (QUIC control + UDP data) | ✅ full session planes, validated live | -| **M4** — native client decode + present (Apple first) | 🟡 stage 1 live; stage-2 presenter built + decode-tested (opt-in, present needs live validation) | +| **M4** — native client decode + present (Apple first) | 🟡 macOS stage 1 live; stage-2 presenter built + decode-tested (opt-in, present needs live validation). **Linux GTK client stage 1 live** (2026-06-12) | ## Live on the boxes @@ -29,6 +29,21 @@ All three appliances advertise over mDNS (`_punktfunk._udp`) and require PIN pai ## Progress log ### 2026-06-12 +- **Native Linux client — stage 1, first light** (`crates/punktfunk-client-linux`, binary + `punktfunk-client`). GTK4/libadwaita app on the **Option A** architecture picked after a + six-angle research pass (toolkits / hw decode / Wayland presentation / input capture / + prior art / codebase): links `punktfunk-core` directly as a crate (no C ABI; + `NativeClient` is `Sync` now), mDNS host list, TOFU + SPAKE2 PIN pairing dialogs + (identity shared with `client-rs`), FFmpeg software HEVC decode (`LOW_DELAY` + slice + threads) into a `GtkGraphicsOffload`-wrapped picture, PipeWire playback with the host + mic-player's jitter ring inverted, SDL3 gamepad capture + rumble/lightbar feedback, + layout-independent keyboard (exact inverse of the host's VK table), absolute mouse + + WHEEL_DELTA scroll, compositor-shortcut inhibition, fullscreen, stats overlay. + **Validated live** against this box's `serve --native`: 1080p60 at a locked 60 fps, + capture→decoded **p50 ≈ 6.4 ms** (software decode, debug build). Next: VAAPI dmabuf → + `GdkDmabufTexture` (Tier-1 zero-copy on Intel/AMD clients), DualSense + touchpad/motion/trigger replay over SDL3, then the stage-2 raw-Wayland presenter + (wp_presentation feedback, tearing-control, Vulkan Video for NVIDIA clients). - **Delegated pairing approval (§8b-1)** — an unpaired device that tries to connect to a pairing-required host now shows up as a **pending request** in the web console's Pairing page; one click approves it (optionally relabeling) and pairs its certificate fingerprint — no PIN