refactor: drop milestone names + consolidate clients; loss-recovery & rumble fixes
apple / swift (push) Failing after 40s
audit / cargo-audit (push) Failing after 1m12s
windows-msix / package (push) Successful in 1m37s
windows / build (push) Successful in 1m14s
android / android (push) Successful in 4m48s
ci / web (push) Successful in 27s
ci / rust (push) Successful in 4m21s
ci / docs-site (push) Successful in 31s
ci / bench (push) Successful in 4m39s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
deb / build-publish (push) Successful in 6m3s
flatpak / build-publish (push) Successful in 4m13s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m15s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m16s
docker / deploy-docs (push) Successful in 18s

Two bodies of work in one commit (the rename moved files the fixes also touched).

Naming/structure cleanup (pre-launch):
- Host modules m3.rs->punktfunk1.rs, m0.rs->spike.rs; CLI m3-host->punktfunk1-host,
  m0->spike; bare `punktfunk-host` now prints help. Types M3Options/M3Source->
  Punktfunk1Options/Punktfunk1Source.
- Clients consolidated out of crates/ into clients/: punktfunk-client-rs->
  clients/probe (crate punktfunk-probe), client-linux->clients/linux,
  client-windows->clients/windows, punktfunk-android->clients/android/native
  (crate punktfunk-client-android; kept [lib] name=punktfunk_android so the JNI
  contract is unchanged). crates/ now holds only core + host.
- Milestone codes M0-M4 purged from code/CLI/CLAUDE.md/README/docs/docs-site,
  kept only in docs/implementation-plan.md. docs/m2-plan.md->
  docs/gamestream-host-plan.md. CI/gradle/flatpak paths updated.

Client loss-recovery (video froze and never recovered after a brief drop):
- Export punktfunk_connection_frames_dropped through the C ABI (the core already
  tracked it for the client keyframe-recovery loop; it was never reachable from
  the ABI clients). Regenerated punktfunk_core.h.
- Apple (StreamPump + Stage2Pipeline) and Android (decode.rs) now poll
  frames_dropped and request a keyframe when it climbs -- the same loss-driven
  recovery Linux/Windows already had. Under infinite GOP the decoder silently
  conceals reference-missing frames, so the decode-error trigger rarely fires.

Apple rumble robustness (worked then went spotty -- DualSense + Xbox):
- Add CHHapticEngine stopped/reset handlers (rebuild on app background / audio
  interruption / server reset) and drop the permanent `broken` latch on a
  transient drive failure; latch only when the controller truly has no haptics.
- Surface swallowed SDL set_rumble errors on Linux/Windows + diagnostic logging.

Verified: cargo build/clippy/fmt --workspace, C-ABI harness, header drift.
Not runnable on this box (verify in CI): Gitea workflows, gradle/Android,
flatpak, Swift/decky.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-18 21:03:55 +00:00
parent 1faa6c6ad4
commit 9c8fa9340c
110 changed files with 534 additions and 341 deletions
+876
View File
@@ -0,0 +1,876 @@
//! The WinUI 3 (windows-reactor) application shell — host list, settings, PIN/TOFU pairing, and
//! the stream page (a `SwapChainPanel` bound to the D3D11 composition swapchain in
//! [`crate::present`], driven by reactor's per-frame `on_rendering`).
//!
//! Declarative React-like model: a single root component routes on a `Screen` value held in
//! `use_async_state` so background threads (discovery, the session pump) can drive navigation.
//! The present + decoded-frame handoff crosses to the UI thread through a `Mutex` side-channel
//! and thread-locals (the windows-reactor SwapChainPanel sample's pattern), since the per-frame
//! present must not go through state/rerender.
//!
//! The chrome follows the windows-reactor gallery's look: Mica backdrop, a centred max-width
//! column, theme brushes (`ThemeRef`), and rounded `border` cards.
use crate::discovery::{self, DiscoveredHost};
use crate::gamepad::GamepadService;
use crate::present::Presenter;
use crate::session::{self, SessionEvent, SessionParams, Stats};
use crate::trust::{self, KnownHost, KnownHosts, Settings};
use crate::video::DecodedFrame;
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use std::cell::RefCell;
use std::sync::{Arc, Mutex};
use windows_reactor::*;
const RESOLUTIONS: &[(u32, u32)] = &[
(0, 0),
(1280, 720),
(1920, 1080),
(2560, 1440),
(3840, 2160),
];
const REFRESH: &[u32] = &[0, 30, 60, 90, 120, 144, 165, 240];
#[derive(Clone, PartialEq)]
enum Screen {
Hosts,
Connecting,
Stream,
Settings,
Pair,
}
/// The host we're about to connect to / pair with (carried into the Pair screen).
#[derive(Clone, Default)]
struct Target {
name: String,
addr: String,
port: u16,
fp_hex: Option<String>,
pair_optional: bool,
}
/// Stable app services handed to the page components as props. Each routed screen that uses
/// hooks (`hosts_page`/`pair_page`/`stream_page`) is mounted as its own `component(...)`, so
/// its hooks live in an isolated slot list — calling them on the shared parent `cx` would
/// change the hook order whenever the screen changes (reactor's Rules-of-Hooks guard aborts).
///
/// `Svc` compares equal by `ctx` identity (it never meaningfully changes across renders), so a
/// page whose props are just `Svc` re-renders only via its own state hooks, never spuriously
/// from the parent.
#[derive(Clone)]
struct Svc {
ctx: Arc<AppCtx>,
set_screen: AsyncSetState<Screen>,
set_status: AsyncSetState<String>,
}
impl PartialEq for Svc {
fn eq(&self, other: &Self) -> bool {
Arc::ptr_eq(&self.ctx, &other.ctx)
}
}
/// Props for the hosts page: the services plus the changing discovery/status data that must
/// drive its re-render (compared by value, so a new host list or error refreshes the page).
#[derive(Clone)]
struct HostsProps {
svc: Svc,
hosts: Vec<DiscoveredHost>,
status: String,
}
impl PartialEq for HostsProps {
fn eq(&self, other: &Self) -> bool {
self.svc == other.svc && self.hosts == other.hosts && self.status == other.status
}
}
/// Props for the stream page: the services plus the live stats that drive the HUD overlay
/// (compared by value, so each new sample re-renders the overlay).
#[derive(Clone)]
struct StreamProps {
svc: Svc,
stats: Stats,
}
impl PartialEq for StreamProps {
fn eq(&self, other: &Self) -> bool {
self.svc == other.svc && self.stats == other.stats
}
}
/// UI-thread-only present context: the D3D11 presenter plus the decoded-frame receiver.
struct PresentCtx {
presenter: Presenter,
frames: async_channel::Receiver<DecodedFrame>,
}
thread_local! {
static PRESENT: RefCell<Option<PresentCtx>> = const { RefCell::new(None) };
static PENDING_FRAMES: RefCell<Option<async_channel::Receiver<DecodedFrame>>> =
const { RefCell::new(None) };
}
/// Cross-thread handoff from the session pump (off-thread) to the stream page (UI thread).
#[derive(Default)]
struct Shared {
handoff: Mutex<Option<(Arc<NativeClient>, async_channel::Receiver<DecodedFrame>)>>,
target: Mutex<Target>,
/// Latest stream stats, written by the session's event loop and mirrored into reactor state
/// by the stream page's HUD poll thread to drive the overlay.
stats: Mutex<Stats>,
}
pub struct AppCtx {
identity: (String, String),
settings: Mutex<Settings>,
gamepad: GamepadService,
shared: Arc<Shared>,
}
pub fn run(identity: (String, String), gamepad: GamepadService) -> windows_reactor::Result<()> {
let ctx = Arc::new(AppCtx {
identity,
settings: Mutex::new(Settings::load()),
gamepad,
shared: Arc::new(Shared::default()),
});
App::new()
.title("Punktfunk")
.inner_size(1000.0, 720.0)
.backdrop(Backdrop::Mica)
.render(move |cx| root(cx, &ctx))
}
// --- shared styling -----------------------------------------------------------------------
fn uniform(v: f64) -> Thickness {
Thickness::uniform(v)
}
fn edges(left: f64, top: f64, right: f64, bottom: f64) -> Thickness {
Thickness {
left,
top,
right,
bottom,
}
}
/// A rounded, bordered surface in the theme's card colours.
fn card(child: impl Into<Element>) -> Border {
border(child.into())
.background(ThemeRef::CardBackground)
.border_brush(ThemeRef::CardStroke)
.border_thickness(uniform(1.0))
.corner_radius(8.0)
.padding(uniform(16.0))
}
/// A small all-caps section label above a group of cards.
fn section(label: &str) -> Element {
text_block(label)
.font_size(12.0)
.semibold()
.foreground(ThemeRef::SecondaryText)
.margin(edges(2.0, 10.0, 0.0, 0.0))
.into()
}
/// Wrap a screen's children in a scrollable, centred, max-width column.
fn page(children: Vec<Element>) -> Element {
let col = vstack(children)
.spacing(10.0)
.max_width(640.0)
.horizontal_alignment(HorizontalAlignment::Center)
.margin(edges(24.0, 24.0, 24.0, 40.0));
scroll_view(col).into()
}
/// A clickable host row: name + address/badge + chevron.
fn host_card(name: &str, sub: &str, badge: &str, on_tap: impl Fn() + 'static) -> Element {
card(
grid((
vstack((
text_block(name).font_size(15.0).semibold(),
text_block(sub)
.font_size(12.0)
.foreground(ThemeRef::SecondaryText),
))
.spacing(2.0)
.grid_column(0)
.vertical_alignment(VerticalAlignment::Center),
text_block(badge)
.font_size(12.0)
.foreground(ThemeRef::SecondaryText)
.grid_column(1)
.vertical_alignment(VerticalAlignment::Center)
.margin(edges(0.0, 0.0, 12.0, 0.0)),
text_block("\u{203A}")
.font_size(18.0)
.foreground(ThemeRef::SecondaryText)
.grid_column(2)
.vertical_alignment(VerticalAlignment::Center),
))
.columns([GridLength::Star(1.0), GridLength::Auto, GridLength::Auto]),
)
.on_tapped(on_tap)
.into()
}
// --- screens ------------------------------------------------------------------------------
fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
let (screen, set_screen) = cx.use_async_state(Screen::Hosts);
let (hosts, set_hosts) = cx.use_async_state(Vec::<DiscoveredHost>::new());
let (status, set_status) = cx.use_async_state(String::new());
let (stats, set_stats) = cx.use_async_state(Stats::default());
// Continuous LAN discovery (spawned once).
cx.use_effect((), {
let set_hosts = set_hosts.clone();
move || {
let rx = discovery::browse();
std::thread::spawn(move || {
let mut acc: Vec<DiscoveredHost> = Vec::new();
while let Ok(h) = rx.recv_blocking() {
if let Some(e) = acc.iter_mut().find(|e| e.key == h.key) {
*e = h;
} else {
acc.push(h);
}
set_hosts.call(acc.clone());
}
});
}
});
// HUD stats: the session event loop writes `shared.stats`; this poll thread mirrors it into
// root state so the stream page gets it as a *prop*. (A child component's own async-state
// update is pruned when its props are unchanged — only a prop change re-renders it, exactly
// like discovery → hosts above.)
cx.use_effect((), {
let shared = ctx.shared.clone();
let set_stats = set_stats.clone();
move || {
std::thread::Builder::new()
.name("pf-hud".into())
.spawn(move || {
let mut last = Stats::default();
loop {
std::thread::sleep(std::time::Duration::from_millis(400));
let s = *shared.stats.lock().unwrap();
if s != last {
last = s;
set_stats.call(s);
}
}
})
.ok();
}
});
// Each hook-using screen is mounted as its own component so its hooks are isolated from
// root's (root's own hooks above stay a stable prefix regardless of which screen renders).
let svc = Svc {
ctx: ctx.clone(),
set_screen: set_screen.clone(),
set_status: set_status.clone(),
};
match screen {
Screen::Hosts => component(hosts_page, HostsProps { svc, hosts, status }),
Screen::Connecting => vstack((
ProgressRing::indeterminate()
.width(48.0)
.height(48.0)
.horizontal_alignment(HorizontalAlignment::Center),
text_block("Connecting\u{2026}")
.font_size(16.0)
.horizontal_alignment(HorizontalAlignment::Center),
text_block(status.clone())
.foreground(ThemeRef::SecondaryText)
.horizontal_alignment(HorizontalAlignment::Center),
))
.spacing(16.0)
.horizontal_alignment(HorizontalAlignment::Center)
.vertical_alignment(VerticalAlignment::Center)
.into(),
// settings_page uses no hooks (it never touches `cx`), so calling it inline is sound.
Screen::Settings => settings_page(ctx, &set_screen),
Screen::Pair => component(pair_page, svc),
Screen::Stream => component(stream_page, StreamProps { svc, stats }),
}
}
fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
let ctx = &props.svc.ctx;
let hosts = props.hosts.as_slice();
let status = props.status.as_str();
let set_screen = &props.svc.set_screen;
let set_status = &props.svc.set_status;
let (manual, set_manual) = cx.use_state(String::new());
let known = KnownHosts::load();
let mut body: Vec<Element> = Vec::new();
// Header: title block + Settings button.
body.push(
grid((
vstack((
text_block("Punktfunk").font_size(30.0).bold(),
text_block("Stream from a host on your network.")
.foreground(ThemeRef::SecondaryText),
))
.spacing(2.0)
.grid_column(0)
.vertical_alignment(VerticalAlignment::Center),
button("Settings")
.on_click({
let ss = set_screen.clone();
move || ss.call(Screen::Settings)
})
.grid_column(1)
.vertical_alignment(VerticalAlignment::Center),
))
.columns([GridLength::Star(1.0), GridLength::Auto])
.margin(edges(0.0, 0.0, 0.0, 6.0))
.into(),
);
if !status.is_empty() {
body.push(card(text_block(status.to_string()).foreground(ThemeRef::SystemCritical)).into());
}
// Saved (trusted/paired) hosts.
if !known.hosts.is_empty() {
body.push(section("SAVED HOSTS"));
for k in &known.hosts {
let target = Target {
name: k.name.clone(),
addr: k.addr.clone(),
port: k.port,
fp_hex: Some(k.fp_hex.clone()),
pair_optional: false,
};
let (ctx2, ss, st) = (ctx.clone(), set_screen.clone(), set_status.clone());
body.push(host_card(
&k.name,
&format!("{}:{}", k.addr, k.port),
if k.paired { "Paired" } else { "Trusted" },
move || initiate(&ctx2, target.clone(), &ss, &st),
));
}
}
// Discovered hosts.
body.push(section("ON YOUR NETWORK"));
if hosts.is_empty() {
body.push(
card(
hstack((
ProgressRing::indeterminate().width(18.0).height(18.0),
text_block("Searching the LAN\u{2026}").foreground(ThemeRef::SecondaryText),
))
.spacing(12.0),
)
.into(),
);
} else {
for h in hosts {
let target = Target {
name: h.name.clone(),
addr: h.addr.clone(),
port: h.port,
fp_hex: (!h.fp_hex.is_empty()).then(|| h.fp_hex.clone()),
pair_optional: h.pair == "optional",
};
let (ctx2, ss, st) = (ctx.clone(), set_screen.clone(), set_status.clone());
let badge = if h.pair == "required" { "PIN" } else { "Open" };
body.push(host_card(
&h.name,
&format!("{}:{}", h.addr, h.port),
badge,
move || initiate(&ctx2, target.clone(), &ss, &st),
));
}
}
// Manual connection.
body.push(section("CONNECT MANUALLY"));
let connect_manual = {
let (ctx2, ss, st, text) = (
ctx.clone(),
set_screen.clone(),
set_status.clone(),
manual.clone(),
);
move || {
let text = text.trim();
if text.is_empty() {
return;
}
let (addr, port) = match text.rsplit_once(':') {
Some((a, p)) => (a.to_string(), p.parse().unwrap_or(9777)),
None => (text.to_string(), 9777),
};
initiate(
&ctx2,
Target {
name: addr.clone(),
addr,
port,
fp_hex: None,
pair_optional: false,
},
&ss,
&st,
);
}
};
body.push(
card(
grid((
text_box(manual)
.placeholder("host or host:port")
.on_changed(move |s| set_manual.call(s))
.grid_column(0)
.vertical_alignment(VerticalAlignment::Center),
button("Connect")
.accent()
.on_click(connect_manual)
.grid_column(1)
.margin(edges(8.0, 0.0, 0.0, 0.0)),
))
.columns([GridLength::Star(1.0), GridLength::Auto]),
)
.into(),
);
page(body)
}
/// The trust gate (mirrors the GTK client's `initiate_connect`): pinned fingerprint → silent
/// connect; known address → stored pin; advertised `pair=optional` → TOFU; otherwise → PIN
/// pairing.
fn initiate(
ctx: &Arc<AppCtx>,
target: Target,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
let known = KnownHosts::load();
let pin = target
.fp_hex
.as_ref()
.and_then(|fp| known.find_by_fp(fp).map(|_| fp.clone()))
.or_else(|| {
known
.find_by_addr(&target.addr, target.port)
.map(|k| k.fp_hex.clone())
})
.and_then(|fp| trust::parse_hex32(&fp));
if let Some(pin) = pin {
connect(ctx, &target, Some(pin), set_screen, set_status);
} else if target.pair_optional {
connect(ctx, &target, None, set_screen, set_status); // TOFU
} else {
*ctx.shared.target.lock().unwrap() = target;
set_screen.call(Screen::Pair);
}
}
fn connect(
ctx: &Arc<AppCtx>,
target: &Target,
pin: Option<[u8; 32]>,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
let s = ctx.settings.lock().unwrap().clone();
let mode = if s.width != 0 && s.refresh_hz != 0 {
Mode {
width: s.width,
height: s.height,
refresh_hz: s.refresh_hz,
}
} else {
Mode {
width: 1920,
height: 1080,
refresh_hz: 60,
}
};
let gamepad_pref = match GamepadPref::from_name(&s.gamepad) {
Some(GamepadPref::Auto) | None => ctx.gamepad.auto_pref(),
Some(explicit) => explicit,
};
let handle = session::start(SessionParams {
host: target.addr.clone(),
port: target.port,
mode,
compositor: CompositorPref::Auto,
gamepad: gamepad_pref,
bitrate_kbps: s.bitrate_kbps,
mic_enabled: s.mic_enabled,
pin,
identity: ctx.identity.clone(),
});
set_status.call(String::new());
set_screen.call(Screen::Connecting);
let tofu = pin.is_none();
let (shared, gamepad) = (ctx.shared.clone(), ctx.gamepad.clone());
let (ss, st) = (set_screen.clone(), set_status.clone());
let target = target.clone();
std::thread::spawn(move || loop {
match handle.events.recv_blocking() {
Ok(SessionEvent::Connected {
connector,
fingerprint,
..
}) => {
if tofu {
let mut k = KnownHosts::load();
k.upsert(KnownHost {
name: target.name.clone(),
addr: target.addr.clone(),
port: target.port,
fp_hex: trust::hex(&fingerprint),
paired: false,
});
let _ = k.save();
}
gamepad.attach(connector.clone());
*shared.stats.lock().unwrap() = Stats::default(); // clear any prior session's numbers
*shared.handoff.lock().unwrap() = Some((connector, handle.frames.clone()));
ss.call(Screen::Stream);
}
Ok(SessionEvent::Failed {
msg,
trust_rejected,
}) => {
st.call(msg);
gamepad.detach();
if trust_rejected {
// Pinned-fingerprint mismatch / pairing required → re-pair via the PIN screen.
*shared.target.lock().unwrap() = target.clone();
ss.call(Screen::Pair);
} else {
ss.call(Screen::Hosts);
}
break;
}
Ok(SessionEvent::Ended(err)) => {
st.call(err.unwrap_or_else(|| "Session ended".into()));
gamepad.detach();
ss.call(Screen::Hosts);
break;
}
Ok(SessionEvent::Stats(s)) => *shared.stats.lock().unwrap() = s,
Err(_) => {
gamepad.detach();
ss.call(Screen::Hosts);
break;
}
}
});
}
fn pair_page(props: &Svc, cx: &mut RenderCx) -> Element {
let ctx = &props.ctx;
let set_screen = &props.set_screen;
let set_status = &props.set_status;
let (code, set_code) = cx.use_state(String::new());
let target = ctx.shared.target.lock().unwrap().clone();
let pair_btn = {
let (ctx2, ss, st, code2, target2) = (
ctx.clone(),
set_screen.clone(),
set_status.clone(),
code.clone(),
target.clone(),
);
button("Pair & Connect").accent().on_click(move || {
let pin = code2.trim().to_string();
let (ctx3, ss, st, target3) = (ctx2.clone(), ss.clone(), st.clone(), target2.clone());
std::thread::spawn(move || {
let name =
std::env::var("COMPUTERNAME").unwrap_or_else(|_| "windows-client".into());
match NativeClient::pair(
&target3.addr,
target3.port,
(&ctx3.identity.0, &ctx3.identity.1),
&pin,
&name,
std::time::Duration::from_secs(90),
) {
Ok(fp) => {
let mut k = KnownHosts::load();
k.upsert(KnownHost {
name: target3.name.clone(),
addr: target3.addr.clone(),
port: target3.port,
fp_hex: trust::hex(&fp),
paired: true,
});
let _ = k.save();
connect(&ctx3, &target3, Some(fp), &ss, &st);
}
Err(e) => {
st.call(format!("Pairing failed: {e:?} (wrong PIN, or not armed?)"));
ss.call(Screen::Hosts);
}
}
});
})
};
let cancel_btn = {
let ss = set_screen.clone();
button("Cancel").on_click(move || ss.call(Screen::Hosts))
};
let content = card(vstack((
text_block(format!("Pair with {}", target.name))
.font_size(20.0)
.semibold(),
text_block(
"Arm pairing on the host (its console or web console), then enter the 4-digit PIN it \
shows.",
)
.foreground(ThemeRef::SecondaryText)
.max_width(440.0),
text_box(code)
.placeholder("PIN")
.on_changed(move |s| set_code.call(s)),
hstack((pair_btn, cancel_btn)).spacing(8.0),
))
.spacing(14.0))
.max_width(480.0)
.horizontal_alignment(HorizontalAlignment::Center)
.margin(edges(0.0, 80.0, 0.0, 0.0));
page(vec![content.into()])
}
fn settings_page(ctx: &Arc<AppCtx>, set_screen: &AsyncSetState<Screen>) -> Element {
let s = ctx.settings.lock().unwrap().clone();
let res_i = RESOLUTIONS
.iter()
.position(|&(w, h)| w == s.width && h == s.height)
.unwrap_or(0) as i32;
let hz_i = REFRESH.iter().position(|&r| r == s.refresh_hz).unwrap_or(0) as i32;
let res_names: Vec<String> = RESOLUTIONS
.iter()
.map(|&(w, h)| {
if w == 0 {
"Native display".into()
} else {
format!("{w} \u{00D7} {h}")
}
})
.collect();
let hz_names: Vec<String> = REFRESH
.iter()
.map(|&r| {
if r == 0 {
"Native".into()
} else {
format!("{r} Hz")
}
})
.collect();
let res_combo = {
let ctx = ctx.clone();
ComboBox::new(res_names)
.header("Resolution")
.selected_index(res_i)
.on_selection_changed(move |i: i32| {
let (w, h) = RESOLUTIONS[(i.max(0) as usize).min(RESOLUTIONS.len() - 1)];
let mut s = ctx.settings.lock().unwrap();
(s.width, s.height) = (w, h);
s.save();
})
};
let hz_combo = {
let ctx = ctx.clone();
ComboBox::new(hz_names)
.header("Refresh rate")
.selected_index(hz_i)
.on_selection_changed(move |i: i32| {
let mut s = ctx.settings.lock().unwrap();
s.refresh_hz = REFRESH[(i.max(0) as usize).min(REFRESH.len() - 1)];
s.save();
})
};
let mic_toggle = {
let ctx = ctx.clone();
check_box(s.mic_enabled)
.label("Stream microphone to the host")
.on_changed(move |on: bool| {
let mut s = ctx.settings.lock().unwrap();
s.mic_enabled = on;
s.save();
})
};
let header = grid((
text_block("Settings")
.font_size(30.0)
.bold()
.grid_column(0)
.vertical_alignment(VerticalAlignment::Center),
button("Back")
.accent()
.on_click({
let ss = set_screen.clone();
move || ss.call(Screen::Hosts)
})
.grid_column(1)
.vertical_alignment(VerticalAlignment::Center),
))
.columns([GridLength::Star(1.0), GridLength::Auto])
.margin(edges(0.0, 0.0, 0.0, 6.0));
let stream_card = card(
vstack((
text_block("Stream").font_size(15.0).semibold(),
text_block("The host creates a virtual display at exactly this mode.")
.font_size(12.0)
.foreground(ThemeRef::SecondaryText),
res_combo,
hz_combo,
))
.spacing(10.0),
);
let audio_card =
card(vstack((text_block("Audio").font_size(15.0).semibold(), mic_toggle)).spacing(10.0));
page(vec![
header.into(),
section("STREAM"),
stream_card.into(),
section("AUDIO"),
audio_card.into(),
])
}
// --- stream page --------------------------------------------------------------------------
fn present_newest(ctx: &mut PresentCtx) {
let mut newest = None;
while let Ok(f) = ctx.frames.try_recv() {
newest = Some(f);
}
let cpu = newest.as_ref().map(|DecodedFrame::Cpu(c)| c);
ctx.presenter.present(cpu);
}
fn stream_page(props: &StreamProps, cx: &mut RenderCx) -> Element {
let ctx = &props.svc.ctx;
// Take the connector + frames handoff once on mount; keep the connector alive (and for input)
// in a use_ref, stash frames for `on_ready`, install the input hooks (and remove on unmount).
let connector_ref = cx.use_ref::<Option<Arc<NativeClient>>>(None);
cx.use_effect_with_cleanup((), {
let shared = ctx.shared.clone();
let connector_ref = connector_ref.clone();
move || {
if let Some((connector, frames)) = shared.handoff.lock().unwrap().take() {
let mode = connector.mode();
connector_ref.set(Some(connector.clone()));
PENDING_FRAMES.with(|c| *c.borrow_mut() = Some(frames));
crate::input::install(connector, mode);
}
Some(crate::input::uninstall)
}
});
let rendering = cx.use_ref::<Option<Rendering>>(None);
cx.use_effect((), {
let rendering = rendering.clone();
move || {
if let Ok(r) = on_rendering(move || {
PRESENT.with(|cell| {
if let Some(ctx) = cell.borrow_mut().as_mut() {
present_newest(ctx);
}
});
}) {
rendering.set(Some(r));
}
}
});
let mode = connector_ref.borrow().as_ref().map(|c| c.mode());
grid((
swap_chain_panel()
.on_ready(|panel| match Presenter::new(1280, 720) {
Ok(p) => {
if let Err(e) = panel.set_swap_chain(p.swap_chain()) {
tracing::error!(error = %e, "set_swap_chain");
}
if let Some(frames) = PENDING_FRAMES.with(|c| c.borrow_mut().take()) {
PRESENT.with(|cell| {
*cell.borrow_mut() = Some(PresentCtx {
presenter: p,
frames,
});
});
tracing::info!("stream presenter bound to SwapChainPanel");
}
}
Err(e) => tracing::error!(error = %e, "create presenter"),
})
.on_resize(|w, h| {
PRESENT.with(|cell| {
if let Some(ctx) = cell.borrow_mut().as_mut() {
ctx.presenter.resize(w as u32, h as u32);
}
});
}),
hud_overlay(&props.stats, mode),
))
.into()
}
/// The streaming HUD overlay (top-right), mirroring the Apple client: mode + fps/throughput, the
/// capture→client latency + decode time, and the release-cursor hint. Layered over the
/// `SwapChainPanel` in the same grid cell.
fn hud_overlay(stats: &Stats, mode: Option<Mode>) -> Element {
let res = mode
.map(|m| format!("{}\u{00D7}{}@{}", m.width, m.height, m.refresh_hz))
.unwrap_or_else(|| "\u{2014}".into());
let line1 = format!("{res} {:.0} fps {:.1} Mb/s", stats.fps, stats.mbps);
let line2 = format!(
"capture\u{2192}client {:.1} ms p50 \u{00B7} decode {:.1} ms",
stats.latency_ms, stats.decode_ms
);
border(
vstack((
text_block(line1)
.font_size(12.0)
.foreground(Color::rgb(255, 255, 255)),
text_block(line2)
.font_size(11.0)
.foreground(Color::rgb(200, 200, 200)),
text_block("Ctrl+Alt+Shift+Q releases the mouse")
.font_size(11.0)
.foreground(Color::rgb(160, 160, 160)),
))
.spacing(2.0),
)
.background(Color::rgb(0, 0, 0))
.corner_radius(8.0)
.padding(uniform(10.0))
.opacity(0.82)
.horizontal_alignment(HorizontalAlignment::Right)
.vertical_alignment(VerticalAlignment::Top)
.margin(uniform(12.0))
.into()
}
+289
View File
@@ -0,0 +1,289 @@
//! Audio: playback (decoded PCM → a WASAPI shared-mode render stream) and the microphone
//! uplink (WASAPI capture → Opus → 0xCB datagrams, the inverse of the host's virtual mic).
//!
//! The WASAPI analogue of the Linux client's PipeWire backend. Playback mirrors the host's
//! virtual-mic producer's adaptive jitter buffer: the session pump pushes 5 ms Opus-decoded
//! chunks on the network clock; the WASAPI render thread pulls whole event-driven quanta on
//! the device clock. Prime to ~3 quanta before producing, cap the ring so latency stays
//! bounded, re-prime after a real drain.
//!
//! WASAPI objects are COM-apartment-bound and not `Send`, so they live on a dedicated thread
//! (the same discipline as the host's `wasapi_cap`); only the channel + stop flag + join
//! handle cross the boundary.
use anyhow::{anyhow, Context, Result};
use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{Receiver, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::Duration;
use wasapi::{DeviceEnumerator, Direction, SampleType, StreamMode, WaveFormat};
const SAMPLE_RATE: usize = 48_000;
const CHANNELS: usize = 2;
/// 48 kHz stereo f32: 2 channels * 4 bytes = 8 bytes per frame.
const BLOCK_ALIGN: usize = CHANNELS * 4;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
pub struct AudioPlayer {
pcm_tx: SyncSender<Vec<f32>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl AudioPlayer {
/// Spawn the WASAPI render thread. Failure (no render endpoint on this box) is
/// survivable — the caller streams video-only.
pub fn spawn() -> Result<AudioPlayer> {
// 64 × 5 ms = 320 ms of slack between the pump and the WASAPI loop.
let (pcm_tx, pcm_rx) = std::sync::mpsc::sync_channel::<Vec<f32>>(64);
let stop = Arc::new(AtomicBool::new(false));
let (ready_tx, ready_rx) = std::sync::mpsc::sync_channel::<Result<()>>(1);
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-audio".into())
.spawn(move || {
if let Err(e) = render_thread(pcm_rx, stop_t, ready_tx) {
tracing::warn!(error = format!("{e:#}"), "audio playback thread ended");
}
})
.context("spawn audio thread")?;
match ready_rx.recv_timeout(Duration::from_secs(3)) {
Ok(Ok(())) => {
tracing::info!("WASAPI render: 48 kHz stereo f32 (default endpoint)");
Ok(AudioPlayer {
pcm_tx,
stop,
thread: Some(thread),
})
}
Ok(Err(e)) => Err(e),
Err(_) => Err(anyhow!(
"wasapi render init timed out (no render endpoint?)"
)),
}
}
/// Queue one interleaved-stereo f32 chunk. Drops the chunk if the WASAPI side is wedged
/// (the renderer conceals the gap; never block the session pump).
pub fn push(&self, pcm: Vec<f32>) {
if let Err(TrySendError::Disconnected(_)) = self.pcm_tx.try_send(pcm) {
// Thread already dead — Drop will reap it; nothing to do per-chunk.
}
}
}
impl Drop for AudioPlayer {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
fn render_thread(
pcm_rx: Receiver<Vec<f32>>,
stop: Arc<AtomicBool>,
ready: SyncSender<Result<()>>,
) -> Result<()> {
if let Err(e) = wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")
{
let _ = ready.send(Err(e));
return Ok(());
}
let res = (|| -> Result<()> {
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Render)
.context("default render endpoint")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CHANNELS, None);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
autoconvert: true,
buffer_duration_hns: default_period,
};
audio_client
.initialize_client(&desired, &Direction::Render, &mode)
.context("initialize render client")?;
let h_event = audio_client.set_get_eventhandle().context("event handle")?;
let render_client = audio_client
.get_audiorenderclient()
.context("IAudioRenderClient")?;
audio_client.start_stream().context("start render stream")?;
let _ = ready.send(Ok(()));
// Adaptive jitter buffer, in f32-byte units (same shape as the host's virtual mic).
let mut ring: VecDeque<u8> = VecDeque::new();
let mut primed = false;
while !stop.load(Ordering::Relaxed) {
if h_event.wait_for_event(100).is_err() {
continue;
}
// Drain everything the pump has queued into the ring.
while let Ok(chunk) = pcm_rx.try_recv() {
for s in chunk {
ring.extend(s.to_le_bytes());
}
}
let avail_frames = audio_client
.get_available_space_in_frames()
.context("available space")? as usize;
if avail_frames == 0 {
continue;
}
let want_bytes = avail_frames * BLOCK_ALIGN;
// Prime to ~3 quanta; cap at ~1 quantum of slack beyond that; re-prime on drain.
let target = (3 * want_bytes).clamp(720 * BLOCK_ALIGN, 9600 * BLOCK_ALIGN);
while ring.len() > target.max(want_bytes) + want_bytes {
ring.pop_front();
}
if !primed && ring.len() >= target {
primed = true;
}
let mut out = vec![0u8; want_bytes];
if primed {
let n = ring.len().min(want_bytes);
for (dst, b) in out.iter_mut().zip(ring.drain(..n)) {
*dst = b;
}
}
if ring.is_empty() {
primed = false;
}
render_client
.write_to_device(avail_frames, &out, None)
.context("write_to_device")?;
}
audio_client.stop_stream().ok();
Ok(())
})();
if let Err(ref e) = res {
let _ = ready.send(Err(anyhow!("{e:#}")));
}
res
}
/// The microphone uplink: capture the default input device, Opus-encode 20 ms chunks, ship
/// them as 0xCB datagrams into the host's virtual mic source.
pub struct MicStreamer {
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl MicStreamer {
pub fn spawn(connector: Arc<NativeClient>) -> Result<MicStreamer> {
let stop = Arc::new(AtomicBool::new(false));
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, stop_t) {
tracing::warn!(error = format!("{e:#}"), "mic uplink thread ended");
}
})
.context("spawn mic thread")?;
Ok(MicStreamer {
stop,
thread: Some(thread),
})
}
}
impl Drop for MicStreamer {
fn drop(&mut self) {
self.stop.store(true, Ordering::SeqCst);
if let Some(t) = self.thread.take() {
let _ = t.join();
}
}
}
fn mic_thread(connector: &Arc<NativeClient>, stop: Arc<AtomicBool>) -> Result<()> {
wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")?;
let mut encoder = opus::Encoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
opus::Application::Voip,
)
.map_err(|e| anyhow!("opus encoder: {e}"))?;
let _ = encoder.set_bitrate(opus::Bitrate::Bits(64_000));
let device = DeviceEnumerator::new()
.context("DeviceEnumerator")?
.get_default_device(&Direction::Capture)
.context("default capture endpoint (no microphone?)")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CHANNELS, None);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
autoconvert: true,
buffer_duration_hns: default_period,
};
audio_client
.initialize_client(&desired, &Direction::Capture, &mode)
.context("initialize capture client")?;
let h_event = audio_client.set_get_eventhandle().context("event handle")?;
let capture_client = audio_client
.get_audiocaptureclient()
.context("IAudioCaptureClient")?;
audio_client
.start_stream()
.context("start capture stream")?;
let mut bytes: VecDeque<u8> = VecDeque::new();
let mut ring: VecDeque<f32> = VecDeque::new();
let mut out = vec![0u8; 4000];
let mut seq = 0u32;
while !stop.load(Ordering::Relaxed) {
if h_event.wait_for_event(100).is_err() {
continue;
}
loop {
match capture_client.get_next_packet_size() {
Ok(Some(0)) | Ok(None) => break,
Ok(Some(_n)) => {
capture_client
.read_from_device_to_deque(&mut bytes)
.context("read capture")?;
}
Err(e) => return Err(anyhow!("get_next_packet_size: {e}")),
}
}
let whole = (bytes.len() / 4) * 4;
for c in bytes.drain(..whole).collect::<Vec<u8>>().chunks_exact(4) {
ring.push_back(f32::from_le_bytes([c[0], c[1], c[2], c[3]]));
}
// Ship every complete 20 ms stereo frame.
while ring.len() >= MIC_FRAME * CHANNELS {
let pcm: Vec<f32> = ring.drain(..MIC_FRAME * CHANNELS).collect();
match encoder.encode_float(&pcm, &mut out) {
Ok(len) => {
let pts = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as u64)
.unwrap_or(0);
let _ = connector.send_mic(seq, pts, out[..len].to_vec());
seq = seq.wrapping_add(1);
}
Err(e) => tracing::debug!(error = %e, "opus mic encode"),
}
}
}
audio_client.stop_stream().ok();
Ok(())
}
+76
View File
@@ -0,0 +1,76 @@
//! LAN host discovery: browse the host's mDNS advert (`_punktfunk._udp`, TXT keys
//! `fp`/`pair`/`id` — see the host crate's `discovery.rs`) on a worker thread and stream
//! results to the UI. Ported verbatim from the GTK client (`mdns-sd` is cross-platform).
use mdns_sd::{ServiceDaemon, ServiceEvent};
#[derive(Clone, Debug, PartialEq)]
pub struct DiscoveredHost {
/// Stable row key: the advertised host id, falling back to the mDNS fullname.
pub key: String,
pub name: String,
pub addr: String,
pub port: u16,
/// Host certificate fingerprint to pin (lowercase hex), empty if not advertised.
pub fp_hex: String,
/// Pairing requirement: `"required"` or `"optional"`.
pub pair: String,
}
/// Browse continuously for the app's lifetime. The thread exits when the receiver is
/// dropped (the send fails) or the daemon dies.
pub fn browse() -> async_channel::Receiver<DiscoveredHost> {
let (tx, rx) = async_channel::unbounded();
std::thread::Builder::new()
.name("punktfunk-mdns".into())
.spawn(move || {
let daemon = match ServiceDaemon::new() {
Ok(d) => d,
Err(e) => {
tracing::warn!(error = %e, "mDNS daemon failed — discovery disabled");
return;
}
};
let receiver = match daemon.browse("_punktfunk._udp.local.") {
Ok(r) => r,
Err(e) => {
tracing::warn!(error = %e, "mDNS browse failed — discovery disabled");
return;
}
};
while let Ok(event) = receiver.recv() {
if let ServiceEvent::ServiceResolved(info) = event {
let props = info.get_properties();
let val = |k: &str| props.get_property_val_str(k).unwrap_or("").to_string();
let Some(addr) = info.get_addresses().iter().next().map(|a| a.to_string())
else {
continue;
};
let id = val("id");
let host = DiscoveredHost {
key: if id.is_empty() {
info.get_fullname().to_string()
} else {
id
},
name: info
.get_fullname()
.split('.')
.next()
.unwrap_or("?")
.to_string(),
addr,
port: info.get_port(),
fp_hex: val("fp"),
pair: val("pair"),
};
if tx.send_blocking(host).is_err() {
break; // UI gone — stop browsing
}
}
}
let _ = daemon.shutdown();
})
.expect("spawn mdns thread");
rx
}
+550
View File
@@ -0,0 +1,550 @@
//! App-lifetime gamepad service over SDL3 (mirrors the Swift/GTK clients' `GamepadManager` +
//! capture/feedback). Ported near-verbatim from the GTK Linux client — SDL3 is cross-platform,
//! so the only Windows change is the build (`sdl3` is compiled from source via the bundled
//! CMake, since there is no system SDL3).
//!
//! One worker thread owns SDL for the process lifetime: it tracks connected pads, selects the
//! ONE controller forwarded as pad 0 (user pin, else the most recently connected), and — while
//! a session is attached — forwards buttons/axes, DualSense touchpad contacts and motion
//! samples (0xCC), and renders feedback: rumble on every pad, lightbar via SDL, and on a real
//! DualSense the raw effects packet (adaptive-trigger blocks replayed verbatim, player LEDs).
//! Held state is zeroed on the wire when the active pad switches or the session detaches, so
//! nothing sticks down.
//!
//! This thread is also the single consumer of the rumble and HID-output pull planes.
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::GamepadPref;
use punktfunk_core::input::{gamepad as wire, InputEvent, InputKind};
use punktfunk_core::quic::{HidOutput, RichInput};
use std::collections::HashMap;
use std::sync::mpsc::{Receiver, Sender};
use std::sync::{Arc, Mutex};
use std::time::Duration;
/// Motion scale constants, shared convention with the other clients (`GamepadWire`): derived
/// from hid-playstation's math over the host's fixed calibration blob. SDL hands us gyro in
/// rad/s and accel in m/s²; the DualSense report wants raw LSBs.
const GYRO_LSB_PER_RAD_S: f32 = 20.0 * 180.0 / std::f32::consts::PI;
const ACCEL_LSB_PER_G: f32 = 10_000.0;
const G: f32 = 9.80665;
#[derive(Clone, Debug)]
pub struct PadInfo {
// `id`/`name` feed the settings GUI's pad list (a follow-up); the windowed client only
// reads `is_dualsense` (via `auto_pref`), so they're unused in reachable code for now.
#[allow(dead_code)]
pub id: u32,
#[allow(dead_code)]
pub name: String,
pub is_dualsense: bool,
}
enum Ctl {
Attach(Arc<NativeClient>),
Detach,
Pin(Option<u32>),
}
#[derive(Clone)]
pub struct GamepadService {
pads: Arc<Mutex<Vec<PadInfo>>>,
active: Arc<Mutex<Option<PadInfo>>>,
pinned: Arc<Mutex<Option<u32>>>,
// `Arc<Mutex<…>>` (not a bare `Sender`, which is `!Sync`) so the service is `Sync` — the
// WinUI app shares it across the UI thread and the session-pump thread (attach/detach).
ctl: Arc<Mutex<Sender<Ctl>>>,
}
impl GamepadService {
pub fn start() -> GamepadService {
let pads = Arc::new(Mutex::new(Vec::new()));
let active = Arc::new(Mutex::new(None));
let pinned = Arc::new(Mutex::new(None));
let (ctl, ctl_rx) = std::sync::mpsc::channel();
let (p, a, pin) = (pads.clone(), active.clone(), pinned.clone());
if let Err(e) = std::thread::Builder::new()
.name("punktfunk-gamepad".into())
.spawn(move || {
if let Err(e) = run(&p, &a, &pin, &ctl_rx) {
tracing::warn!(error = %e, "gamepad service ended — pads disabled");
}
})
{
tracing::warn!(error = %e, "gamepad service failed to start");
}
GamepadService {
pads,
active,
pinned,
ctl: Arc::new(Mutex::new(ctl)),
}
}
#[allow(dead_code)] // consumed by the settings GUI (follow-up)
pub fn pads(&self) -> Vec<PadInfo> {
self.pads.lock().unwrap().clone()
}
pub fn active(&self) -> Option<PadInfo> {
self.active.lock().unwrap().clone()
}
#[allow(dead_code)] // consumed by the settings GUI (follow-up)
pub fn pinned(&self) -> Option<u32> {
*self.pinned.lock().unwrap()
}
#[allow(dead_code)] // consumed by the settings GUI (follow-up)
pub fn set_pinned(&self, id: Option<u32>) {
let _ = self.ctl.lock().unwrap().send(Ctl::Pin(id));
}
pub fn attach(&self, connector: Arc<NativeClient>) {
let _ = self.ctl.lock().unwrap().send(Ctl::Attach(connector));
}
pub fn detach(&self) {
let _ = self.ctl.lock().unwrap().send(Ctl::Detach);
}
/// What "Automatic" resolves to right now — the virtual pad matching the physical one
/// (Swift parity); no pad connected leaves the host's own default.
pub fn auto_pref(&self) -> GamepadPref {
match self.active() {
Some(p) if p.is_dualsense => GamepadPref::DualSense,
Some(_) => GamepadPref::Xbox360,
None => GamepadPref::Auto,
}
}
}
fn send(connector: &NativeClient, kind: InputKind, code: u32, x: i32) {
let _ = connector.send_input(&InputEvent {
kind,
_pad: [0; 3],
code,
x,
y: 0,
flags: 0, // pad index 0 — single-pad model
});
}
fn button_bit(b: sdl3::gamepad::Button) -> Option<u32> {
use sdl3::gamepad::Button;
Some(match b {
Button::South => wire::BTN_A,
Button::East => wire::BTN_B,
Button::West => wire::BTN_X,
Button::North => wire::BTN_Y,
Button::Back => wire::BTN_BACK,
Button::Start => wire::BTN_START,
Button::Guide => wire::BTN_GUIDE,
Button::LeftStick => wire::BTN_LS_CLICK,
Button::RightStick => wire::BTN_RS_CLICK,
Button::LeftShoulder => wire::BTN_LB,
Button::RightShoulder => wire::BTN_RB,
Button::DPadUp => wire::BTN_DPAD_UP,
Button::DPadDown => wire::BTN_DPAD_DOWN,
Button::DPadLeft => wire::BTN_DPAD_LEFT,
Button::DPadRight => wire::BTN_DPAD_RIGHT,
Button::Touchpad => wire::BTN_TOUCHPAD,
_ => return None,
})
}
/// SDL axis → (wire axis id, wire value). SDL sticks are +y = down; the wire (XInput
/// convention) is +y = up. SDL triggers span 0..32767; the wire wants 0..255.
fn axis_value(axis: sdl3::gamepad::Axis, v: i16) -> (u32, i32) {
use sdl3::gamepad::Axis;
match axis {
Axis::LeftX => (wire::AXIS_LS_X, v as i32),
Axis::LeftY => (wire::AXIS_LS_Y, -(v as i32).max(-32767)),
Axis::RightX => (wire::AXIS_RS_X, v as i32),
Axis::RightY => (wire::AXIS_RS_Y, -(v as i32).max(-32767)),
Axis::TriggerLeft => (wire::AXIS_LT, (v as i32).clamp(0, 32767) >> 7),
Axis::TriggerRight => (wire::AXIS_RT, (v as i32).clamp(0, 32767) >> 7),
}
}
/// The DualSense effects packet (SDL `DS5EffectsState_t`, 47 bytes) — the same layout the host
/// parses off its virtual pad; the wire's 11-byte trigger blocks drop in verbatim. Enable bits
/// select only the fields each update touches, so rumble (driven separately through SDL) and
/// untouched fields keep their state.
#[derive(Default)]
struct Ds5Feedback;
impl Ds5Feedback {
const RIGHT_TRIGGER: usize = 10;
const LEFT_TRIGGER: usize = 21;
const PAD_LIGHTS: usize = 43;
const LED_RGB: usize = 44;
fn trigger_packet(which: u8, effect: &[u8]) -> [u8; 47] {
let mut p = [0u8; 47];
let (flag, off) = if which == 1 {
(0x04, Self::RIGHT_TRIGGER)
} else {
(0x08, Self::LEFT_TRIGGER)
};
p[0] = flag;
let n = effect.len().min(11);
p[off..off + n].copy_from_slice(&effect[..n]);
p
}
fn lightbar_packet(r: u8, g: u8, b: u8) -> [u8; 47] {
let mut p = [0u8; 47];
p[1] = 0x04; // lightbar enable
p[Self::LED_RGB] = r;
p[Self::LED_RGB + 1] = g;
p[Self::LED_RGB + 2] = b;
p
}
fn player_packet(bits: u8) -> [u8; 47] {
let mut p = [0u8; 47];
p[1] = 0x10; // player-LED enable
p[Self::PAD_LIGHTS] = bits & 0x1F;
p
}
}
struct Worker {
subsystem: sdl3::GamepadSubsystem,
opened: HashMap<u32, sdl3::gamepad::Gamepad>,
/// Connection order; the most recently connected is the auto selection.
order: Vec<u32>,
pinned: Option<u32>,
attached: Option<Arc<NativeClient>>,
/// Wire state of the active pad — zeroed on the wire at switch/detach.
last_axis: [i32; 6],
held_buttons: Vec<u32>,
last_accel: [i16; 3],
}
impl Worker {
fn active_id(&self) -> Option<u32> {
self.pinned
.filter(|id| self.opened.contains_key(id))
.or_else(|| self.order.last().copied())
}
fn pad_info(&self, id: u32) -> Option<PadInfo> {
let pad = self.opened.get(&id)?;
Some(PadInfo {
id,
name: pad.name().unwrap_or_else(|| "Controller".into()),
is_dualsense: matches!(
self.subsystem
.type_for_id(sdl3::sys::joystick::SDL_JoystickID(id)),
sdl3::gamepad::GamepadType::PS5
),
})
}
/// Zero everything the host believes is held — on pad switch and detach.
fn flush_held(&mut self) {
if let Some(c) = &self.attached {
for b in self.held_buttons.drain(..) {
send(c, InputKind::GamepadButton, b, 0);
}
for (id, v) in self.last_axis.iter_mut().enumerate() {
if *v != 0 && *v != i32::MIN {
send(c, InputKind::GamepadAxis, id as u32, 0);
}
*v = i32::MIN;
}
} else {
self.held_buttons.clear();
self.last_axis = [i32::MIN; 6];
}
}
/// Sensors stream only while a session wants them (they cost USB/BT bandwidth).
fn set_sensors(&mut self, enabled: bool) {
let Some(id) = self.active_id() else { return };
if let Some(pad) = self.opened.get_mut(&id) {
use sdl3::sensor::SensorType;
for s in [SensorType::Gyroscope, SensorType::Accelerometer] {
if unsafe { pad.has_sensor(s) } {
let _ = pad.sensor_set_enabled(s, enabled);
}
}
}
}
}
#[allow(clippy::too_many_lines)]
fn run(
pads_out: &Mutex<Vec<PadInfo>>,
active_out: &Mutex<Option<PadInfo>>,
pinned_out: &Mutex<Option<u32>>,
ctl: &Receiver<Ctl>,
) -> Result<(), String> {
// Off-main-thread + no video subsystem: keep SDL away from signals, poll pads on its own
// thread.
sdl3::hint::set("SDL_NO_SIGNAL_HANDLERS", "1");
sdl3::hint::set("SDL_JOYSTICK_THREAD", "1");
let sdl = sdl3::init().map_err(|e| e.to_string())?;
let subsystem = sdl.gamepad().map_err(|e| e.to_string())?;
let mut pump = sdl.event_pump().map_err(|e| e.to_string())?;
let mut w = Worker {
subsystem,
opened: HashMap::new(),
order: Vec::new(),
pinned: None,
attached: None,
last_axis: [i32::MIN; 6],
held_buttons: Vec::new(),
last_accel: [0; 3],
};
let publish = |w: &Worker| {
let mut list: Vec<PadInfo> = w.order.iter().filter_map(|&id| w.pad_info(id)).collect();
list.reverse(); // most recent first — the Settings list order
*pads_out.lock().unwrap() = list;
*active_out.lock().unwrap() = w.active_id().and_then(|id| w.pad_info(id));
*pinned_out.lock().unwrap() = w.pinned;
};
loop {
// Control plane from the UI thread.
loop {
match ctl.try_recv() {
Ok(Ctl::Attach(c)) => {
w.attached = Some(c);
w.last_axis = [i32::MIN; 6];
w.set_sensors(true);
}
Ok(Ctl::Detach) => {
w.flush_held();
w.set_sensors(false);
w.attached = None;
}
Ok(Ctl::Pin(id)) => {
let before = w.active_id();
w.pinned = id;
if w.active_id() != before {
w.flush_held();
if w.attached.is_some() {
w.set_sensors(true);
}
}
publish(&w);
}
Err(std::sync::mpsc::TryRecvError::Empty) => break,
Err(std::sync::mpsc::TryRecvError::Disconnected) => return Ok(()), // app gone
}
}
while let Some(event) = pump.poll_event() {
use sdl3::event::Event;
let active = w.active_id();
match event {
Event::ControllerDeviceAdded { which, .. } => {
if !w.opened.contains_key(&which) {
match w.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(which)) {
Ok(pad) => {
tracing::info!(
name = pad.name().unwrap_or_default(),
"gamepad attached"
);
w.opened.insert(which, pad);
w.order.push(which);
if w.attached.is_some() && w.active_id() == Some(which) {
w.set_sensors(true);
}
publish(&w);
}
Err(e) => tracing::warn!(error = %e, "gamepad open failed"),
}
}
}
Event::ControllerDeviceRemoved { which, .. } => {
if w.opened.remove(&which).is_some() {
w.order.retain(|&id| id != which);
if active == Some(which) {
w.flush_held();
}
tracing::info!("gamepad detached");
publish(&w);
}
}
Event::ControllerButtonDown { which, button, .. }
if active == Some(which) && w.attached.is_some() =>
{
if let Some(bit) = button_bit(button) {
w.held_buttons.push(bit);
send(
w.attached.as_ref().unwrap(),
InputKind::GamepadButton,
bit,
1,
);
}
}
Event::ControllerButtonUp { which, button, .. }
if active == Some(which) && w.attached.is_some() =>
{
if let Some(bit) = button_bit(button) {
w.held_buttons.retain(|&b| b != bit);
send(
w.attached.as_ref().unwrap(),
InputKind::GamepadButton,
bit,
0,
);
}
}
Event::ControllerAxisMotion {
which, axis, value, ..
} if active == Some(which) && w.attached.is_some() => {
let (id, v) = axis_value(axis, value);
if w.last_axis[id as usize] != v {
w.last_axis[id as usize] = v;
send(w.attached.as_ref().unwrap(), InputKind::GamepadAxis, id, v);
}
}
// DualSense touchpad → the rich-input plane, normalized 0..=65535.
Event::ControllerTouchpadDown {
which,
finger,
x,
y,
..
}
| Event::ControllerTouchpadMotion {
which,
finger,
x,
y,
..
} if active == Some(which) && w.attached.is_some() => {
let _ = w
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: true,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
}
Event::ControllerTouchpadUp {
which,
finger,
x,
y,
..
} if active == Some(which) && w.attached.is_some() => {
let _ = w
.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Touchpad {
pad: 0,
finger: finger as u8,
active: false,
x: (x.clamp(0.0, 1.0) * 65535.0) as u16,
y: (y.clamp(0.0, 1.0) * 65535.0) as u16,
});
}
// Motion: accel events update the cache; each gyro event ships a sample (the
// DualSense reports both at ~250 Hz). Scale convention shared with the other
// clients — sign/scale derived, not yet live-verified.
Event::ControllerSensorUpdated {
which,
sensor,
data,
..
} if active == Some(which) && w.attached.is_some() => {
use sdl3::sensor::SensorType;
match sensor {
SensorType::Accelerometer => {
for (i, v) in data.iter().enumerate() {
w.last_accel[i] =
(v / G * ACCEL_LSB_PER_G).clamp(-32768.0, 32767.0) as i16;
}
}
SensorType::Gyroscope => {
let mut gyro = [0i16; 3];
for (i, v) in data.iter().enumerate() {
gyro[i] = (v * GYRO_LSB_PER_RAD_S).clamp(-32768.0, 32767.0) as i16;
}
let _ =
w.attached
.as_ref()
.unwrap()
.send_rich_input(RichInput::Motion {
pad: 0,
gyro,
accel: w.last_accel,
});
}
_ => {}
}
}
_ => {}
}
}
// Feedback planes (this thread is their single consumer). The host re-sends rumble state
// periodically, so a generous duration with refresh-on-update is safe — a dropped stop
// heals within ~500 ms.
if let Some(connector) = w.attached.clone() {
while let Ok((pad, low, high)) = connector.next_rumble(Duration::ZERO) {
if pad == 0 {
if let Some(p) = w.active_id().and_then(|id| w.opened.get_mut(&id)) {
// Surface a failed SDL rumble write: a swallowed error here (DualSense not in
// the right HIDAPI mode, etc.) reads exactly like "rumble doesn't work". The
// host logs the send side on 0xCA, so the two together pinpoint host-game vs
// client-render.
if let Err(e) = p.set_rumble(low, high, 5_000) {
tracing::warn!(low, high, error = %e, "rumble: SDL set_rumble failed");
} else {
tracing::debug!(low, high, "rumble: rendered");
}
} else {
tracing::debug!(low, high, "rumble: received but no active pad to render");
}
}
}
while let Ok(hid) = connector.next_hidout(Duration::ZERO) {
let Some(id) = w.active_id() else { continue };
let is_ds = w.pad_info(id).is_some_and(|p| p.is_dualsense);
let Some(pad) = w.opened.get_mut(&id) else {
continue;
};
match hid {
HidOutput::Led { pad: 0, r, g, b } if is_ds => {
let _ = pad.send_effect(&Ds5Feedback::lightbar_packet(r, g, b));
}
HidOutput::Led { pad: 0, r, g, b } => {
let _ = pad.set_led(r, g, b);
}
HidOutput::PlayerLeds { pad: 0, bits } if is_ds => {
let _ = pad.send_effect(&Ds5Feedback::player_packet(bits));
}
HidOutput::Trigger {
pad: 0,
which,
ref effect,
} if is_ds => {
let _ = pad.send_effect(&Ds5Feedback::trigger_packet(which, effect));
}
_ => {}
}
}
}
std::thread::sleep(Duration::from_millis(if w.attached.is_some() {
2
} else {
30
}));
}
}
+333
View File
@@ -0,0 +1,333 @@
//! Stream input: Win32 low-level keyboard + mouse hooks forwarding to the host while the WinUI
//! window is focused and the pointer is captured.
//!
//! windows-reactor exposes no raw key-down/up or pointer-position/wheel events (only keyboard
//! *accelerators* and pointer button-state), which is insufficient for a game stream. So this
//! drops below XAML to `WH_KEYBOARD_LL` / `WH_MOUSE_LL`, installed on the UI thread when the
//! stream page mounts and removed when it unmounts.
//!
//! **Pointer lock.** While captured the cursor is *locked* the way a game-streaming client locks
//! it (Moonlight/Parsec): the OS cursor is hidden + confined to the window (`ClipCursor`), and
//! every physical move is turned into a **relative** delta (`InputKind::MouseMove`) — we read the
//! offset from the window centre, ship it (scaled screen→host through the Contain-fit factor, with
//! sub-pixel remainder carried so slow drags aren't lost), then warp the cursor back to centre so
//! it never reaches a screen edge. This is why the old absolute path froze: swallowing
//! `WM_MOUSEMOVE` pinned the OS cursor, so `pt` never travelled and the absolute coordinate
//! snapped to one point. Keys carry the native Windows VK directly (the wire contract).
//!
//! **Ctrl+Alt+Shift+Q** toggles capture — releasing the lock hands the cursor back to the local
//! desktop (and re-grabs on the next toggle). Losing foreground also releases the lock so the
//! cursor is never stranded.
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::Mode;
use punktfunk_core::input::{InputEvent, InputKind};
use std::collections::HashSet;
use std::sync::atomic::{AtomicIsize, Ordering};
use std::sync::{Arc, Mutex};
use windows::Win32::Foundation::{HWND, LPARAM, LRESULT, POINT, RECT, WPARAM};
use windows::Win32::Graphics::Gdi::ClientToScreen;
use windows::Win32::System::LibraryLoader::GetModuleHandleW;
use windows::Win32::UI::Input::KeyboardAndMouse::VK_Q;
use windows::Win32::UI::WindowsAndMessaging::{
CallNextHookEx, ClipCursor, GetClientRect, GetForegroundWindow, SetCursorPos,
SetWindowsHookExW, ShowCursor, UnhookWindowsHookEx, HC_ACTION, HHOOK, KBDLLHOOKSTRUCT,
LLMHF_INJECTED, MSLLHOOKSTRUCT, WH_KEYBOARD_LL, WH_MOUSE_LL, WM_KEYUP, WM_LBUTTONDOWN,
WM_LBUTTONUP, WM_MBUTTONDOWN, WM_MBUTTONUP, WM_MOUSEHWHEEL, WM_MOUSEMOVE, WM_MOUSEWHEEL,
WM_RBUTTONDOWN, WM_RBUTTONUP, WM_SYSKEYUP, WM_XBUTTONDOWN, WM_XBUTTONUP,
};
struct State {
connector: Arc<NativeClient>,
mode: Mode,
/// Our window handle, stored as the raw `isize` so `State` is `Send` (`HWND` is not).
hwnd: isize,
/// User intent: forward input to the host (toggled by Ctrl+Alt+Shift+Q).
captured: bool,
/// The OS pointer is currently locked (hidden + confined + recentering). Tracks the real
/// `ClipCursor`/`ShowCursor` state so we engage/disengage exactly once per transition.
locked: bool,
/// Lock centre in screen coordinates (the cursor is warped here after every move).
center_x: i32,
center_y: i32,
/// Sub-pixel remainder of the screen→host scale, carried so slow drags aren't truncated away.
acc_x: f32,
acc_y: f32,
/// Modifier state, tracked from the hook's own event stream (see `kbd_proc`).
ctrl: bool,
alt: bool,
shift: bool,
held_keys: HashSet<u8>,
held_buttons: HashSet<u32>,
}
// `State` carries no `!Send` handle (hwnd is an `isize`), so the static is sound. The hook procs
// run on the same UI thread that installs/removes the hooks, so the lock is uncontended.
static STATE: Mutex<Option<State>> = Mutex::new(None);
static KBD_HOOK: AtomicIsize = AtomicIsize::new(0);
static MOUSE_HOOK: AtomicIsize = AtomicIsize::new(0);
/// Install the hooks for a streaming session. Call from the UI thread once the window is shown.
pub fn install(connector: Arc<NativeClient>, mode: Mode) {
let hwnd = unsafe { GetForegroundWindow() };
let mut st = State {
connector,
mode,
hwnd: hwnd.0 as isize,
captured: true,
locked: false,
center_x: 0,
center_y: 0,
acc_x: 0.0,
acc_y: 0.0,
ctrl: false,
alt: false,
shift: false,
held_keys: HashSet::new(),
held_buttons: HashSet::new(),
};
// Lock immediately (the window is foreground at mount, like Moonlight grabbing on stream start).
set_locked(&mut st, true);
*STATE.lock().unwrap() = Some(st);
unsafe {
let hinst = GetModuleHandleW(None).ok();
if let Ok(h) = SetWindowsHookExW(WH_KEYBOARD_LL, Some(kbd_proc), hinst.map(Into::into), 0) {
KBD_HOOK.store(h.0 as isize, Ordering::SeqCst);
}
if let Ok(h) = SetWindowsHookExW(WH_MOUSE_LL, Some(mouse_proc), hinst.map(Into::into), 0) {
MOUSE_HOOK.store(h.0 as isize, Ordering::SeqCst);
}
}
tracing::info!(
"stream input hooks installed — pointer locked (Ctrl+Alt+Shift+Q toggles capture)"
);
}
/// Remove the hooks, release the pointer lock, and flush any held keys/buttons (so nothing
/// sticks down on the host).
pub fn uninstall() {
unsafe {
let k = KBD_HOOK.swap(0, Ordering::SeqCst);
if k != 0 {
let _ = UnhookWindowsHookEx(HHOOK(k as *mut _));
}
let m = MOUSE_HOOK.swap(0, Ordering::SeqCst);
if m != 0 {
let _ = UnhookWindowsHookEx(HHOOK(m as *mut _));
}
}
if let Some(mut st) = STATE.lock().unwrap().take() {
set_locked(&mut st, false); // hand the cursor back to the desktop
flush_held(&mut st);
}
}
/// Release every held key/button on the host, so nothing sticks down when capture is dropped
/// (toggled off) or the session ends.
fn flush_held(st: &mut State) {
let c = st.connector.clone();
for vk in st.held_keys.drain() {
send(&c, InputKind::KeyUp, vk as u32, 0, 0, 0);
}
for b in st.held_buttons.drain() {
send(&c, InputKind::MouseButtonUp, b, 0, 0, 0);
}
}
/// Engage or release the pointer lock: confine + hide + recentre on, free + show on off.
/// Guarded so the `ClipCursor`/`ShowCursor` calls stay balanced (one each per transition).
fn set_locked(st: &mut State, on: bool) {
if on == st.locked {
return;
}
let hwnd = HWND(st.hwnd as *mut _);
unsafe {
if on {
let mut rc = RECT::default();
if GetClientRect(hwnd, &mut rc).is_ok() {
let mut tl = POINT {
x: rc.left,
y: rc.top,
};
let mut br = POINT {
x: rc.right,
y: rc.bottom,
};
let _ = ClientToScreen(hwnd, &mut tl);
let _ = ClientToScreen(hwnd, &mut br);
let clip = RECT {
left: tl.x,
top: tl.y,
right: br.x,
bottom: br.y,
};
let _ = ClipCursor(Some(&clip as *const RECT));
st.center_x = (tl.x + br.x) / 2;
st.center_y = (tl.y + br.y) / 2;
let _ = SetCursorPos(st.center_x, st.center_y);
}
let _ = ShowCursor(false);
st.acc_x = 0.0;
st.acc_y = 0.0;
} else {
let _ = ClipCursor(None);
let _ = ShowCursor(true);
}
}
st.locked = on;
}
fn send(c: &NativeClient, kind: InputKind, code: u32, x: i32, y: i32, flags: u32) {
let _ = c.send_input(&InputEvent {
kind,
_pad: [0; 3],
code,
x,
y,
flags,
});
}
unsafe extern "system" fn kbd_proc(code: i32, wparam: WPARAM, lparam: LPARAM) -> LRESULT {
if code == HC_ACTION as i32 {
let kb = unsafe { &*(lparam.0 as *const KBDLLHOOKSTRUCT) };
let msg = wparam.0 as u32;
let up = msg == WM_KEYUP || msg == WM_SYSKEYUP;
let vk = kb.vkCode as u16;
let mut guard = STATE.lock().unwrap();
if let Some(st) = guard.as_mut() {
// Track modifier state from the hook's own event stream — reliable even while we
// swallow these keys (GetAsyncKeyState doesn't reflect keys suppressed by our own LL
// hook, which is why the shortcut never fired). Handles the generic + L/R vk codes.
match kb.vkCode {
0x11 | 0xA2 | 0xA3 => st.ctrl = !up, // (L/R)CONTROL
0x12 | 0xA4 | 0xA5 => st.alt = !up, // (L/R)MENU (Alt)
0x10 | 0xA0 | 0xA1 => st.shift = !up, // (L/R)SHIFT
_ => {}
}
let foreground = unsafe { GetForegroundWindow() }.0 as isize == st.hwnd;
if foreground {
// Capture toggle: Ctrl+Alt+Shift+Q (consumed locally, never forwarded).
if !up && vk == VK_Q.0 && st.ctrl && st.alt && st.shift {
let on = !st.captured;
st.captured = on;
set_locked(st, on); // grab/release the cursor immediately
if !on {
flush_held(st); // release held keys/buttons so nothing sticks on the host
}
tracing::info!(captured = on, "capture toggled (Ctrl+Alt+Shift+Q)");
return LRESULT(1);
}
if st.captured {
let v = vk as u8;
if up {
if st.held_keys.remove(&v) {
send(&st.connector, InputKind::KeyUp, v as u32, 0, 0, 0);
}
} else {
st.held_keys.insert(v);
send(&st.connector, InputKind::KeyDown, v as u32, 0, 0, 0);
}
return LRESULT(1); // swallow so it reaches the host, not the local OS
}
}
}
}
unsafe { CallNextHookEx(None, code, wparam, lparam) }
}
/// Client-area size in pixels (for the screen→host relative-motion scale).
fn client_size(hwnd: isize) -> (f32, f32) {
let mut rc = RECT::default();
if unsafe { GetClientRect(HWND(hwnd as *mut _), &mut rc) }.is_ok() {
(
(rc.right - rc.left).max(1) as f32,
(rc.bottom - rc.top).max(1) as f32,
)
} else {
(1.0, 1.0)
}
}
unsafe extern "system" fn mouse_proc(code: i32, wparam: WPARAM, lparam: LPARAM) -> LRESULT {
if code == HC_ACTION as i32 {
let ms = unsafe { &*(lparam.0 as *const MSLLHOOKSTRUCT) };
let msg = wparam.0 as u32;
let injected = (ms.flags & LLMHF_INJECTED) != 0;
let mut guard = STATE.lock().unwrap();
if let Some(st) = guard.as_mut() {
let foreground = unsafe { GetForegroundWindow() }.0 as isize == st.hwnd;
let want_lock = st.captured && foreground;
if want_lock != st.locked {
set_locked(st, want_lock); // sync to focus changes (e.g. lost foreground)
}
if st.locked {
// Skip the synthetic move our own SetCursorPos recentre generates.
if injected {
return unsafe { CallNextHookEx(None, code, wparam, lparam) };
}
let c = st.connector.clone();
match msg {
WM_MOUSEMOVE => {
let dx = (ms.pt.x - st.center_x) as f32;
let dy = (ms.pt.y - st.center_y) as f32;
if dx != 0.0 || dy != 0.0 {
// screen px → host px: the Contain-fit display scale's inverse, so the
// host cursor tracks the physical mouse 1:1 on screen at any window size.
let (ww, wh) = client_size(st.hwnd);
let (vw, vh) =
(st.mode.width.max(1) as f32, st.mode.height.max(1) as f32);
let s = (ww / vw).min(wh / vh).max(0.01);
st.acc_x += dx / s;
st.acc_y += dy / s;
let (hx, hy) = (st.acc_x.trunc() as i32, st.acc_y.trunc() as i32);
st.acc_x -= hx as f32;
st.acc_y -= hy as f32;
if hx != 0 || hy != 0 {
send(&c, InputKind::MouseMove, 0, hx, hy, 0);
}
}
let _ = unsafe { SetCursorPos(st.center_x, st.center_y) };
}
WM_LBUTTONDOWN => button(st, 1, true),
WM_LBUTTONUP => button(st, 1, false),
WM_RBUTTONDOWN => button(st, 3, true),
WM_RBUTTONUP => button(st, 3, false),
WM_MBUTTONDOWN => button(st, 2, true),
WM_MBUTTONUP => button(st, 2, false),
WM_XBUTTONDOWN => button(st, 3 + ((ms.mouseData >> 16) as u16 as u32), true),
WM_XBUTTONUP => button(st, 3 + ((ms.mouseData >> 16) as u16 as u32), false),
WM_MOUSEWHEEL => send(
&c,
InputKind::MouseScroll,
0,
(ms.mouseData >> 16) as i16 as i32,
0,
0,
),
WM_MOUSEHWHEEL => send(
&c,
InputKind::MouseScroll,
1,
(ms.mouseData >> 16) as i16 as i32,
0,
0,
),
_ => {}
}
return LRESULT(1); // swallow inside the locked window
}
}
}
unsafe { CallNextHookEx(None, code, wparam, lparam) }
}
fn button(st: &mut State, id: u32, down: bool) {
let c = st.connector.clone();
if down {
st.held_buttons.insert(id);
send(&c, InputKind::MouseButtonDown, id, 0, 0, 0);
} else if st.held_buttons.remove(&id) {
send(&c, InputKind::MouseButtonUp, id, 0, 0, 0);
}
}
+258
View File
@@ -0,0 +1,258 @@
//! `punktfunk-client` — the native Windows punktfunk/1 client.
//!
//! Pure Rust: `NativeClient` linked as a crate (no C ABI, like the GTK Linux client) · FFmpeg
//! decode · WASAPI audio · SDL3 gamepads · a **WinUI 3** shell (windows-reactor) with the video
//! on a `SwapChainPanel` bound to a D3D11 composition swapchain. The trust surface mirrors the
//! other native clients: persistent identity, trust-on-first-use, SPAKE2 PIN pairing — all in-app
//! (host list, settings, pairing). `--headless` keeps a CLI connect path for tests/measurement.
//!
//! Usage:
//! punktfunk-client (open the WinUI 3 window: host list, settings, pairing)
//! punktfunk-client --discover (list punktfunk hosts on the LAN)
//! punktfunk-client --headless --connect host[:port] [--pin HEX] [--pair PIN] [--mode WxHxHz]
//! [--bitrate MBPS] [--mic] (no window; count frames + print stats)
// Link as a GUI (windows) subsystem binary so the default windowed launch (MSIX / double-click)
// does NOT pop a console window. The CLI paths (--headless/--discover) reattach to the launching
// terminal's console at startup (see main), so their output is still visible when run from a shell.
#![cfg_attr(windows, windows_subsystem = "windows")]
#[cfg(windows)]
mod app;
#[cfg(windows)]
mod audio;
#[cfg(windows)]
mod discovery;
#[cfg(windows)]
mod gamepad;
#[cfg(windows)]
mod input;
#[cfg(windows)]
mod present;
#[cfg(windows)]
mod session;
#[cfg(windows)]
mod trust;
#[cfg(windows)]
mod video;
#[cfg(windows)]
fn main() {
// With #![windows_subsystem = "windows"] the process starts with no console, so the GUI/MSIX
// launch is window-free. AttachConsole only binds to an ALREADY-EXISTING parent console (it
// never creates one), so when launched from a terminal — `--headless`/`--discover` — stdout and
// the tracing writer below land in that terminal; from Explorer/MSIX it's a harmless no-op.
unsafe {
use windows::Win32::System::Console::{AttachConsole, ATTACH_PARENT_PROCESS};
let _ = AttachConsole(ATTACH_PARENT_PROCESS);
}
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env().unwrap_or_else(|_| "info".into()),
)
.init();
let args: Vec<String> = std::env::args().collect();
let flag = |name: &str| args.iter().any(|a| a == name);
if flag("--discover") {
discover_and_print();
return;
}
let identity = match trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
eprintln!("client identity: {e:#}");
std::process::exit(1);
}
};
if flag("--headless") {
run_headless_cli(&args, identity);
return;
}
// Windowed (default): the WinUI 3 app owns host selection, settings, and pairing.
let gamepad = gamepad::GamepadService::start();
if let Err(e) = app::run(identity, gamepad) {
tracing::error!(error = %e, "WinUI app failed");
std::process::exit(1);
}
}
/// `--headless --connect host[:port] …`: connect from the CLI, count frames, print stats — the
/// Windows analogue of `punktfunk-probe`.
#[cfg(windows)]
fn run_headless_cli(args: &[String], identity: (String, String)) {
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use std::time::{Duration, Instant};
let arg = |name: &str| -> Option<String> {
args.iter()
.position(|a| a == name)
.and_then(|i| args.get(i + 1))
.cloned()
};
let flag = |name: &str| args.iter().any(|a| a == name);
let Some(target) = arg("--connect") else {
eprintln!("--headless requires --connect host[:port]");
std::process::exit(2);
};
let (host, port) = match target.rsplit_once(':') {
Some((a, p)) => (a.to_string(), p.parse().unwrap_or(9777)),
None => (target.clone(), 9777u16),
};
let mode = arg("--mode")
.and_then(|m| {
let mut it = m.split(['x', 'X']);
Some(Mode {
width: it.next()?.parse().ok()?,
height: it.next()?.parse().ok()?,
refresh_hz: it.next()?.parse().ok()?,
})
})
.unwrap_or(Mode {
width: 1280,
height: 720,
refresh_hz: 60,
});
let bitrate_kbps = arg("--bitrate")
.and_then(|b| b.parse::<u32>().ok())
.map(|m| m * 1000)
.unwrap_or(0);
let known = trust::KnownHosts::load();
let mut pin = arg("--pin")
.and_then(|h| trust::parse_hex32(&h))
.or_else(|| {
known
.find_by_addr(&host, port)
.and_then(|k| trust::parse_hex32(&k.fp_hex))
});
if let Some(code) = arg("--pair") {
let name = std::env::var("COMPUTERNAME").unwrap_or_else(|_| "windows-client".into());
match punktfunk_core::client::NativeClient::pair(
&host,
port,
(&identity.0, &identity.1),
code.trim(),
&name,
Duration::from_secs(90),
) {
Ok(fp) => {
let mut k = trust::KnownHosts::load();
k.upsert(trust::KnownHost {
name: host.clone(),
addr: host.clone(),
port,
fp_hex: trust::hex(&fp),
paired: true,
});
let _ = k.save();
tracing::info!(fp = %trust::hex(&fp), "paired");
pin = Some(fp);
}
Err(e) => {
eprintln!("Pairing failed: {e:?}");
std::process::exit(1);
}
}
}
tracing::info!(%host, port, ?mode, tofu = pin.is_none(), "connecting (headless)");
let handle = session::start(session::SessionParams {
host,
port,
mode,
compositor: CompositorPref::Auto,
gamepad: GamepadPref::Auto,
bitrate_kbps,
mic_enabled: flag("--mic"),
pin,
identity,
});
let deadline = Instant::now() + Duration::from_secs(60);
let mut frames_seen = 0u64;
loop {
while let Ok(ev) = handle.events.try_recv() {
match ev {
session::SessionEvent::Connected {
mode, fingerprint, ..
} => tracing::info!(?mode, fp = %trust::hex(&fingerprint), "connected"),
session::SessionEvent::Stats(s) => tracing::info!(
fps = format!("{:.0}", s.fps),
mbps = format!("{:.1}", s.mbps),
decode_ms = format!("{:.2}", s.decode_ms),
lat_ms = format!("{:.2}", s.latency_ms),
frames_seen,
"stats"
),
session::SessionEvent::Failed { msg, .. } => {
tracing::error!(%msg, "connect failed");
return;
}
session::SessionEvent::Ended(err) => {
tracing::info!(reason = err.as_deref().unwrap_or("done"), "session ended");
return;
}
}
}
while handle.frames.try_recv().is_ok() {
frames_seen += 1;
}
if Instant::now() > deadline {
tracing::info!(frames_seen, "harness deadline — stopping");
handle.stop.store(true, std::sync::atomic::Ordering::SeqCst);
return;
}
std::thread::sleep(Duration::from_millis(2));
}
}
/// `--discover`: browse the LAN for punktfunk hosts (mDNS) and print them, then exit.
#[cfg(windows)]
fn discover_and_print() {
use std::time::{Duration, Instant};
println!("Browsing the LAN for punktfunk hosts (~5 s)…");
let rx = discovery::browse();
let deadline = Instant::now() + Duration::from_secs(5);
let mut seen = std::collections::HashSet::new();
while Instant::now() < deadline {
while let Ok(h) = rx.try_recv() {
if seen.insert(h.key.clone()) {
println!(
" {} {}:{} pair={} fp={}",
h.name,
h.addr,
h.port,
if h.pair.is_empty() {
"optional"
} else {
&h.pair
},
if h.fp_hex.is_empty() { "-" } else { &h.fp_hex },
);
}
}
std::thread::sleep(Duration::from_millis(100));
}
if seen.is_empty() {
println!(" (none found — is a host running with --native / punktfunk1-host?)");
}
}
/// WinUI 3 / Direct3D11 / WASAPI / SDL3 are Windows turf; this stub keeps `cargo build
/// --workspace` green on Linux/macOS (the other native clients live in
/// clients/linux and clients/apple).
#[cfg(not(windows))]
fn main() {
eprintln!(
"punktfunk-client-windows is Windows-only — the Linux client lives in \
clients/linux, the macOS client in clients/apple"
);
std::process::exit(2);
}
+444
View File
@@ -0,0 +1,444 @@
//! Direct3D11 presenter for a WinUI 3 `SwapChainPanel`: upload a decoded `CpuFrame` (RGBA)
//! into a dynamic texture and draw it Contain-fit into a **composition** flip-model swapchain,
//! which the reactor stream page binds to the panel via `SwapChainPanelHandle::set_swap_chain`.
//!
//! The device prefers a hardware adapter and falls back to **WARP** (the GPU-less dev box runs
//! the whole present path in software). The draw is a single full-screen triangle sampling the
//! video texture; a letterbox is produced by clearing the back buffer black and setting the
//! viewport to the Contain-fit rect (no per-frame vertex buffer).
//!
//! **HDR10**: when a frame is BT.2020 PQ (`CpuFrame::hdr`), the swapchain flips to
//! `R10G10B10A2` + `DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020` (+ HDR10 metadata) via
//! `ResizeBuffers`/`SetColorSpace1`; the decoded samples are already PQ-encoded so the shader is a
//! plain passthrough and the compositor maps PQ→display. SDR stays 8-bit B8G8R8A8.
//!
//! All `windows` types here come from the same windows-rs commit as `windows-reactor`, so the
//! `IDXGISwapChain1` handed to `set_swap_chain` satisfies reactor's `windows_core::Interface`.
use crate::video::CpuFrame;
use anyhow::{anyhow, Context, Result};
use windows::core::{Interface, PCSTR};
use windows::Win32::Graphics::Direct3D::Fxc::{D3DCompile, D3DCOMPILE_OPTIMIZATION_LEVEL3};
use windows::Win32::Graphics::Direct3D::{
ID3DBlob, D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_WARP, D3D_FEATURE_LEVEL_11_0,
D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST,
};
use windows::Win32::Graphics::Direct3D11::*;
use windows::Win32::Graphics::Dxgi::Common::*;
use windows::Win32::Graphics::Dxgi::*;
const SHADER_HLSL: &str = r#"
struct VSOut { float4 pos : SV_Position; float2 uv : TEXCOORD0; };
VSOut vs_main(uint vid : SV_VertexID) {
float2 uv = float2((vid << 1) & 2, vid & 2);
VSOut o;
o.pos = float4(uv * float2(2, -2) + float2(-1, 1), 0, 1);
o.uv = uv;
return o;
}
Texture2D tex : register(t0);
SamplerState smp : register(s0);
float4 ps_main(VSOut i) : SV_Target { return tex.Sample(smp, i.uv); }
"#;
pub struct Presenter {
device: ID3D11Device,
context: ID3D11DeviceContext,
vs: ID3D11VertexShader,
ps: ID3D11PixelShader,
sampler: ID3D11SamplerState,
swap: IDXGISwapChain1,
rtv: Option<ID3D11RenderTargetView>,
/// Video texture + SRV + dimensions; recreated when the decoded size changes.
tex: Option<(ID3D11Texture2D, ID3D11ShaderResourceView, u32, u32)>,
/// Panel (swapchain) size in pixels, updated on resize.
panel_w: u32,
panel_h: u32,
/// Whether the swapchain is currently in 10-bit HDR10 (R10G10B10A2 + ST.2084) mode; flipped
/// to match each frame's `hdr` flag.
hdr: bool,
}
impl Presenter {
/// Create the D3D11 device + composition swapchain + shaders, sized to the panel.
pub fn new(width: u32, height: u32) -> Result<Presenter> {
let (device, context) = create_device()?;
let (vs, ps, sampler) = build_pipeline(&device)?;
let swap = create_composition_swapchain(&device, width.max(1), height.max(1))?;
Ok(Presenter {
device,
context,
vs,
ps,
sampler,
swap,
rtv: None,
tex: None,
panel_w: width.max(1),
panel_h: height.max(1),
hdr: false,
})
}
/// The DXGI swapchain to hand to `SwapChainPanelHandle::set_swap_chain`.
pub fn swap_chain(&self) -> &IDXGISwapChain1 {
&self.swap
}
/// Resize the back buffers to the panel's new size (drops the stale RTV).
pub fn resize(&mut self, width: u32, height: u32) {
if width == 0 || height == 0 || (width == self.panel_w && height == self.panel_h) {
return;
}
self.rtv = None; // release all back-buffer refs before ResizeBuffers
unsafe {
let _ = self.swap.ResizeBuffers(
0,
width,
height,
DXGI_FORMAT_UNKNOWN,
DXGI_SWAP_CHAIN_FLAG(0),
);
}
self.panel_w = width;
self.panel_h = height;
}
/// Present one decoded frame (Contain-fit) — or, when `frame` is `None`, just re-present the
/// last texture (or black). Called from the reactor `on_rendering` per-frame callback.
pub fn present(&mut self, frame: Option<&CpuFrame>) {
if let Some(f) = frame {
if f.hdr != self.hdr {
self.set_hdr(f.hdr);
}
if let Err(e) = self.upload(f) {
tracing::warn!(error = %e, "frame upload failed");
}
}
let Ok(rtv) = self.rtv() else {
return;
};
let (pw, ph) = (self.panel_w, self.panel_h);
unsafe {
let c = &self.context;
c.ClearRenderTargetView(&rtv, &[0.0, 0.0, 0.0, 1.0]);
if let Some((_, srv, vw, vh)) = &self.tex {
// Contain-fit viewport: scale to the smaller axis, centre, letterbox the rest.
let (ww, wh, vfw, vfh) = (
pw as f32,
ph as f32,
(*vw).max(1) as f32,
(*vh).max(1) as f32,
);
let scale = (ww / vfw).min(wh / vfh);
let (dw, dh) = (vfw * scale, vfh * scale);
let (ox, oy) = ((ww - dw) / 2.0, (wh - dh) / 2.0);
c.OMSetRenderTargets(Some(&[Some(rtv.clone())]), None);
let vp = D3D11_VIEWPORT {
TopLeftX: ox,
TopLeftY: oy,
Width: dw,
Height: dh,
MinDepth: 0.0,
MaxDepth: 1.0,
};
c.RSSetViewports(Some(&[vp]));
c.IASetInputLayout(None);
c.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
c.VSSetShader(&self.vs, None);
c.PSSetShader(&self.ps, None);
c.PSSetShaderResources(0, Some(&[Some(srv.clone())]));
c.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
c.Draw(3, 0);
}
let _ = self.swap.Present(1, DXGI_PRESENT(0));
}
}
/// Switch the swapchain between 8-bit SDR (B8G8R8A8, sRGB/BT.709) and 10-bit HDR10
/// (R10G10B10A2, ST.2084 PQ BT.2020). `ResizeBuffers` can change the back-buffer format in
/// place, so the panel binding (`set_swap_chain`) stays valid — no rebind needed. The decoded
/// samples are already PQ-encoded BT.2020 (see `video::convert`), so the colour space is all the
/// compositor needs to map them to the display.
fn set_hdr(&mut self, on: bool) {
self.rtv = None; // release back-buffer refs before ResizeBuffers
self.tex = None; // texture format changes (R10G10B10A2 vs R8G8B8A8)
let format = if on {
DXGI_FORMAT_R10G10B10A2_UNORM
} else {
DXGI_FORMAT_B8G8R8A8_UNORM
};
unsafe {
if let Err(e) = self.swap.ResizeBuffers(
0,
self.panel_w,
self.panel_h,
format,
DXGI_SWAP_CHAIN_FLAG(0),
) {
tracing::warn!(error = %e, "ResizeBuffers for HDR switch failed");
return;
}
let colorspace = if on {
DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020
} else {
DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
};
if let Ok(sc3) = self.swap.cast::<IDXGISwapChain3>() {
// Only set a colour space the swapchain accepts for present (on an SDR desktop the
// DWM still tone-maps HDR10 → SDR, so leaving the default there is fine).
if let Ok(support) = sc3.CheckColorSpaceSupport(colorspace) {
if support & DXGI_SWAP_CHAIN_COLOR_SPACE_SUPPORT_FLAG_PRESENT.0 as u32 != 0 {
let _ = sc3.SetColorSpace1(colorspace);
}
}
}
if on {
if let Ok(sc4) = self.swap.cast::<IDXGISwapChain4>() {
let md = hdr10_metadata();
let bytes = std::slice::from_raw_parts(
&md as *const DXGI_HDR_METADATA_HDR10 as *const u8,
std::mem::size_of::<DXGI_HDR_METADATA_HDR10>(),
);
let _ = sc4.SetHDRMetaData(DXGI_HDR_METADATA_TYPE_HDR10, Some(bytes));
}
}
}
self.hdr = on;
tracing::info!(hdr = on, "swapchain colour mode switched");
}
fn upload(&mut self, frame: &CpuFrame) -> Result<()> {
let (w, h) = (frame.width, frame.height);
let need_new = !matches!(&self.tex, Some((_, _, tw, th)) if *tw == w && *th == h);
if need_new {
let format = if self.hdr {
DXGI_FORMAT_R10G10B10A2_UNORM
} else {
DXGI_FORMAT_R8G8B8A8_UNORM
};
let desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: format,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DYNAMIC,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
CPUAccessFlags: D3D11_CPU_ACCESS_WRITE.0 as u32,
MiscFlags: 0,
};
let texture = unsafe {
let mut t = None;
self.device
.CreateTexture2D(&desc, None, Some(&mut t))
.context("CreateTexture2D")?;
t.unwrap()
};
let srv = unsafe {
let mut s = None;
self.device
.CreateShaderResourceView(&texture, None, Some(&mut s))
.context("CreateShaderResourceView")?;
s.unwrap()
};
self.tex = Some((texture, srv, w, h));
}
let (texture, _, _, _) = self.tex.as_ref().unwrap();
unsafe {
let mut mapped = D3D11_MAPPED_SUBRESOURCE::default();
self.context
.Map(texture, 0, D3D11_MAP_WRITE_DISCARD, 0, Some(&mut mapped))
.context("Map video texture")?;
let dst = mapped.pData as *mut u8;
let dst_pitch = mapped.RowPitch as usize;
let src_pitch = frame.stride;
let row_bytes = (w as usize) * 4;
for y in 0..h as usize {
std::ptr::copy_nonoverlapping(
frame.pixels.as_ptr().add(y * src_pitch),
dst.add(y * dst_pitch),
row_bytes.min(src_pitch),
);
}
self.context.Unmap(texture, 0);
}
Ok(())
}
fn rtv(&mut self) -> Result<ID3D11RenderTargetView> {
if self.rtv.is_none() {
let back: ID3D11Texture2D = unsafe { self.swap.GetBuffer(0).context("GetBuffer")? };
let rtv = unsafe {
let mut v = None;
self.device
.CreateRenderTargetView(&back, None, Some(&mut v))
.context("CreateRenderTargetView")?;
v.unwrap()
};
self.rtv = Some(rtv);
}
Ok(self.rtv.clone().unwrap())
}
}
fn create_device() -> Result<(ID3D11Device, ID3D11DeviceContext)> {
for driver in [D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_WARP] {
let mut device = None;
let mut context = None;
let r = unsafe {
D3D11CreateDevice(
None,
driver,
None,
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
};
if r.is_ok() {
let name = if driver == D3D_DRIVER_TYPE_HARDWARE {
"hardware"
} else {
"WARP (software)"
};
tracing::info!(driver = name, "D3D11 device created");
return Ok((device.unwrap(), context.unwrap()));
}
}
Err(anyhow!(
"D3D11CreateDevice failed for both hardware and WARP"
))
}
/// A composition flip-model swapchain (no HWND) for binding to a XAML `SwapChainPanel`.
fn create_composition_swapchain(
device: &ID3D11Device,
width: u32,
height: u32,
) -> Result<IDXGISwapChain1> {
let dxdev: IDXGIDevice = device.cast().context("IDXGIDevice cast")?;
let factory: IDXGIFactory2 = unsafe {
let adapter = dxdev.GetAdapter().context("GetAdapter")?;
adapter.GetParent().context("GetParent (IDXGIFactory2)")?
};
let desc = DXGI_SWAP_CHAIN_DESC1 {
Width: width,
Height: height,
Format: DXGI_FORMAT_B8G8R8A8_UNORM,
Stereo: false.into(),
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
BufferUsage: DXGI_USAGE_RENDER_TARGET_OUTPUT,
BufferCount: 2,
Scaling: DXGI_SCALING_STRETCH,
SwapEffect: DXGI_SWAP_EFFECT_FLIP_SEQUENTIAL,
// IGNORE (opaque), not PREMULTIPLIED: the video fills the panel and the HDR `X2BGR10`
// upload leaves the 2 padding/alpha bits 0 — premultiplied alpha would then make HDR frames
// transparent. Opaque is correct for a full-frame video surface either way.
AlphaMode: DXGI_ALPHA_MODE_IGNORE,
Flags: 0,
};
unsafe {
factory
.CreateSwapChainForComposition(device, &desc, None)
.context("CreateSwapChainForComposition")
}
}
fn build_pipeline(
device: &ID3D11Device,
) -> Result<(ID3D11VertexShader, ID3D11PixelShader, ID3D11SamplerState)> {
let vs_blob = compile(SHADER_HLSL, "vs_main", "vs_5_0")?;
let ps_blob = compile(SHADER_HLSL, "ps_main", "ps_5_0")?;
unsafe {
let mut vs = None;
device
.CreateVertexShader(blob_bytes(&vs_blob), None, Some(&mut vs))
.context("CreateVertexShader")?;
let mut ps = None;
device
.CreatePixelShader(blob_bytes(&ps_blob), None, Some(&mut ps))
.context("CreatePixelShader")?;
let sdesc = D3D11_SAMPLER_DESC {
Filter: D3D11_FILTER_MIN_MAG_MIP_LINEAR,
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
MaxLOD: D3D11_FLOAT32_MAX,
..Default::default()
};
let mut sampler = None;
device
.CreateSamplerState(&sdesc, Some(&mut sampler))
.context("CreateSamplerState")?;
Ok((vs.unwrap(), ps.unwrap(), sampler.unwrap()))
}
}
fn compile(src: &str, entry: &str, target: &str) -> Result<ID3DBlob> {
let entry_c = std::ffi::CString::new(entry).unwrap();
let target_c = std::ffi::CString::new(target).unwrap();
let mut code = None;
let mut errors = None;
let r = unsafe {
D3DCompile(
src.as_ptr() as *const _,
src.len(),
PCSTR::null(),
None,
None,
PCSTR(entry_c.as_ptr() as *const u8),
PCSTR(target_c.as_ptr() as *const u8),
D3DCOMPILE_OPTIMIZATION_LEVEL3,
0,
&mut code,
Some(&mut errors),
)
};
if r.is_err() {
let msg = errors
.as_ref()
.map(|b| unsafe {
let p = b.GetBufferPointer() as *const u8;
let n = b.GetBufferSize();
String::from_utf8_lossy(std::slice::from_raw_parts(p, n)).to_string()
})
.unwrap_or_default();
return Err(anyhow!("D3DCompile {entry}: {msg}"));
}
code.ok_or_else(|| anyhow!("D3DCompile produced no bytecode"))
}
fn blob_bytes(blob: &ID3DBlob) -> &[u8] {
unsafe {
let p = blob.GetBufferPointer() as *const u8;
let n = blob.GetBufferSize();
std::slice::from_raw_parts(p, n)
}
}
/// Generic HDR10 mastering metadata: BT.2020 primaries + D65 white (0.00002 units), a 1000-nit
/// mastering display, MaxCLL 1000 / MaxFALL 400. The protocol doesn't carry the stream's real
/// mastering metadata yet (host follow-up), so these are sane defaults the display tone-maps from.
fn hdr10_metadata() -> DXGI_HDR_METADATA_HDR10 {
DXGI_HDR_METADATA_HDR10 {
RedPrimary: [35400, 14600],
GreenPrimary: [8500, 39850],
BluePrimary: [6550, 2300],
WhitePoint: [15635, 16450],
MaxMasteringLuminance: 1000,
MinMasteringLuminance: 1, // 0.0001-nit units → 0.0001 nits
MaxContentLightLevel: 1000,
MaxFrameAverageLightLevel: 400,
}
}
+272
View File
@@ -0,0 +1,272 @@
//! Session controller: one worker thread runs connect → pump (video pull + decode, audio
//! pull + Opus decode, stats), feeding the UI over channels. The UI keeps the
//! `Arc<NativeClient>` from the `Connected` event for direct input sends (no extra hop on
//! the input path) — `NativeClient` is `Sync`, planes stay one-consumer-per-thread:
//! video+audio here, rumble+hidout on the gamepad thread.
//!
//! Ported from the GTK Linux client; the platform-specific pieces are the video decoder
//! (software-only here) and the audio backend (WASAPI). The pump body is identical.
use crate::audio;
use crate::video::{DecodedFrame, Decoder};
use punktfunk_core::client::NativeClient;
use punktfunk_core::config::{CompositorPref, GamepadPref, Mode};
use punktfunk_core::PunktfunkError;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
pub struct SessionParams {
pub host: String,
pub port: u16,
pub mode: Mode,
pub compositor: CompositorPref,
pub gamepad: GamepadPref,
pub bitrate_kbps: u32,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Pinned host fingerprint; `None` = trust on first use (caller persists the observed one).
pub pin: Option<[u8; 32]>,
pub identity: (String, String),
}
#[derive(Clone, Copy, Default, PartialEq)]
pub struct Stats {
pub fps: f32,
pub mbps: f32,
pub decode_ms: f32,
/// Median capture→decoded latency over the last window (host-clock corrected).
pub latency_ms: f32,
}
pub enum SessionEvent {
Connected {
connector: Arc<NativeClient>,
mode: Mode,
fingerprint: [u8; 32],
},
/// `trust_rejected` is set when the connect failed the TLS trust check (a `Crypto`
/// error): for a pinned connect this is the fingerprint-changed signal, so the UI can
/// offer a re-pair (PIN) path rather than a dead-end error.
Failed {
msg: String,
trust_rejected: bool,
},
Ended(Option<String>),
Stats(Stats),
}
pub struct SessionHandle {
pub events: async_channel::Receiver<SessionEvent>,
pub frames: async_channel::Receiver<DecodedFrame>,
pub stop: Arc<AtomicBool>,
}
pub fn start(params: SessionParams) -> SessionHandle {
let (ev_tx, ev_rx) = async_channel::unbounded();
// Tiny frame queue, newest wins: force_send displaces the oldest when the UI lags.
let (frame_tx, frame_rx) = async_channel::bounded(2);
let stop = Arc::new(AtomicBool::new(false));
let stop_w = stop.clone();
std::thread::Builder::new()
.name("punktfunk-session".into())
.spawn(move || pump(params, ev_tx, frame_tx, stop_w))
.expect("spawn session thread");
SessionHandle {
events: ev_rx,
frames: frame_rx,
stop,
}
}
fn now_ns() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as u64)
.unwrap_or(0)
}
fn pump(
params: SessionParams,
ev_tx: async_channel::Sender<SessionEvent>,
frame_tx: async_channel::Sender<DecodedFrame>,
stop: Arc<AtomicBool>,
) {
let connector = match NativeClient::connect(
&params.host,
params.port,
params.mode,
params.compositor,
params.gamepad,
params.bitrate_kbps,
// Advertise 10-bit + HDR10: the presenter handles BT.2020 PQ (R10G10B10A2) frames, so the
// host may upgrade HDR content to a Main10/PQ stream (it still only does so for actual HDR
// content with its own 10-bit gate). 8-bit SDR is unaffected.
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR,
None, // launch: the Windows client has no library picker yet
params.pin,
Some(params.identity),
Duration::from_secs(15),
) {
Ok(c) => Arc::new(c),
Err(e) => {
let trust_rejected = matches!(e, PunktfunkError::Crypto);
let msg = match e {
PunktfunkError::Crypto => {
"Host identity rejected — wrong fingerprint, or the host requires pairing"
.to_string()
}
PunktfunkError::Timeout => "Connection timed out".to_string(),
other => format!("Connect failed: {other:?}"),
};
let _ = ev_tx.send_blocking(SessionEvent::Failed {
msg,
trust_rejected,
});
return;
}
};
let _ = ev_tx.send_blocking(SessionEvent::Connected {
connector: connector.clone(),
mode: connector.mode(),
fingerprint: connector.host_fingerprint,
});
let mut decoder = match Decoder::new() {
Ok(d) => d,
Err(e) => {
let _ = ev_tx.send_blocking(SessionEvent::Ended(Some(format!("video decoder: {e}"))));
return;
}
};
// Audio is best-effort: a session without it still streams. Gamepads are the
// app-lifetime service's job (the UI attaches it on Connected).
let player = audio::AudioPlayer::spawn()
.map_err(|e| tracing::warn!(error = %e, "audio disabled"))
.ok();
let mut opus_dec = opus::Decoder::new(48_000, opus::Channels::Stereo)
.map_err(|e| tracing::warn!(error = %e, "opus decoder failed — audio disabled"))
.ok();
let _mic = params
.mic_enabled
.then(|| {
audio::MicStreamer::spawn(connector.clone())
.map_err(|e| tracing::warn!(error = %e, "mic uplink disabled"))
.ok()
})
.flatten();
let clock_offset = connector.clock_offset_ns;
let mut total_frames = 0u64;
let mut window_start = Instant::now();
let mut frames_n = 0u32;
let mut bytes_n = 0u64;
let mut decode_us_sum = 0u64;
let mut lat_us: Vec<u64> = Vec::with_capacity(256);
let mut pcm = vec![0f32; 5760 * 2]; // decode scratch: max Opus frame (120 ms stereo)
// Loss recovery: watch the host→client unrecoverable-drop count and ask for an IDR when it climbs.
let mut last_dropped = connector.frames_dropped();
let mut last_kf_req: Option<Instant> = None;
let end: Option<String> = loop {
if stop.load(Ordering::SeqCst) {
break None;
}
match connector.next_frame(Duration::from_millis(4)) {
Ok(frame) => {
let t0 = Instant::now();
match decoder.decode(&frame.data) {
Ok(Some(decoded)) => {
total_frames += 1;
if total_frames == 1 {
let DecodedFrame::Cpu(c) = &decoded;
tracing::info!(
width = c.width,
height = c.height,
path = "software",
"first frame decoded"
);
}
// Latency: our wall clock expressed in the host's capture clock,
// minus the host-stamped capture pts (same math as client-rs).
let lat = (now_ns() as i128 + clock_offset as i128 - frame.pts_ns as i128)
.max(0) as u64;
if lat > 0 && lat < 10_000_000_000 {
lat_us.push(lat / 1000);
}
decode_us_sum += t0.elapsed().as_micros() as u64;
frames_n += 1;
bytes_n += frame.data.len() as u64;
let _ = frame_tx.force_send(decoded);
}
Ok(None) => {}
// Survivable (loss until the next IDR/RFI recovery) — keep feeding.
Err(e) => tracing::debug!(error = %e, "decode error (recovering)"),
}
}
Err(PunktfunkError::NoFrame) => {}
Err(PunktfunkError::Closed) => break Some("Host ended the session".to_string()),
Err(e) => break Some(format!("session: {e:?}")),
}
// Loss recovery: under infinite GOP the only recovery keyframe is one we request. The
// reassembler drops unrecoverable AUs (frames_dropped); the decoder conceals the
// reference-missing delta frames that follow and returns Ok, so keying off a decode error
// rarely fires. Request an IDR when the drop count climbs, throttled.
let dropped = connector.frames_dropped();
if dropped > last_dropped {
last_dropped = dropped;
let now = Instant::now();
if last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100)) {
last_kf_req = Some(now);
let _ = connector.request_keyframe();
tracing::debug!(dropped, "requested keyframe (loss recovery)");
}
}
// Drain audio between frames (packets land every 5 ms; the queue holds 320 ms).
while let Ok(pkt) = connector.next_audio(Duration::ZERO) {
if let (Some(player), Some(dec)) = (&player, opus_dec.as_mut()) {
match dec.decode_float(&pkt.data, &mut pcm, false) {
Ok(samples) => player.push(pcm[..samples * 2].to_vec()),
Err(e) => tracing::debug!(error = %e, "opus decode"),
}
}
}
if window_start.elapsed() >= Duration::from_secs(1) {
let secs = window_start.elapsed().as_secs_f32();
lat_us.sort_unstable();
let p50 = lat_us.get(lat_us.len() / 2).copied().unwrap_or(0);
tracing::debug!(
fps = frames_n,
lat_p50_us = p50,
total_frames,
"stream window"
);
let _ = ev_tx.try_send(SessionEvent::Stats(Stats {
fps: frames_n as f32 / secs,
mbps: bytes_n as f32 * 8.0 / 1e6 / secs,
decode_ms: if frames_n > 0 {
decode_us_sum as f32 / frames_n as f32 / 1000.0
} else {
0.0
},
latency_ms: p50 as f32 / 1000.0,
}));
window_start = Instant::now();
frames_n = 0;
bytes_n = 0;
decode_us_sum = 0;
lat_us.clear();
}
};
tracing::info!(
total_frames,
reason = end.as_deref().unwrap_or("user"),
"session ended"
);
stop.store(true, Ordering::SeqCst);
let _ = ev_tx.send_blocking(SessionEvent::Ended(end));
}
+170
View File
@@ -0,0 +1,170 @@
//! Client identity, the known-hosts (pinned fingerprint) store, and app settings.
//!
//! Ported near-verbatim from the GTK Linux client; the only platform change is the config
//! directory — `%APPDATA%\punktfunk` (the Windows analogue of `~/.config/punktfunk`), shared
//! with the Windows host's identity location. The identity files (`client-{cert,key}.pem`)
//! keep the same names so the trust model is identical across the native clients.
use anyhow::{anyhow, Context, Result};
use punktfunk_core::quic::endpoint;
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
pub fn config_dir() -> Result<PathBuf> {
let appdata = std::env::var("APPDATA").context("APPDATA unset")?;
Ok(PathBuf::from(appdata).join("punktfunk"))
}
/// This client's persistent identity, generated on first use — presented on every connect
/// so hosts can recognize it once paired.
pub fn load_or_create_identity() -> Result<(String, String)> {
let dir = config_dir()?;
let (cp, kp) = (dir.join("client-cert.pem"), dir.join("client-key.pem"));
if let (Ok(c), Ok(k)) = (std::fs::read_to_string(&cp), std::fs::read_to_string(&kp)) {
return Ok((c, k));
}
let (c, k) = endpoint::generate_identity().map_err(|e| anyhow!("generate identity: {e}"))?;
std::fs::create_dir_all(&dir)?;
std::fs::write(&cp, &c)?;
std::fs::write(&kp, &k)?;
tracing::info!(cert = %cp.display(), "generated client identity");
Ok((c, k))
}
pub fn hex(fp: &[u8; 32]) -> String {
fp.iter().map(|b| format!("{b:02x}")).collect()
}
pub fn parse_hex32(s: &str) -> Option<[u8; 32]> {
if s.len() != 64 {
return None;
}
let mut out = [0u8; 32];
for (i, b) in out.iter_mut().enumerate() {
*b = u8::from_str_radix(&s[2 * i..2 * i + 2], 16).ok()?;
}
Some(out)
}
/// One trusted host: its pinned certificate fingerprint plus how we got there (TOFU or a
/// PIN ceremony) and where we last reached it.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct KnownHost {
pub name: String,
pub addr: String,
pub port: u16,
/// SHA-256 of the host certificate, lowercase hex — the pin for every later connect.
pub fp_hex: String,
/// True if trust came from the SPAKE2 PIN ceremony (vs. trust-on-first-use).
pub paired: bool,
}
#[derive(Default, Serialize, Deserialize)]
pub struct KnownHosts {
pub hosts: Vec<KnownHost>,
}
impl KnownHosts {
fn path() -> Result<PathBuf> {
Ok(config_dir()?.join("client-known-hosts.json"))
}
pub fn load() -> KnownHosts {
Self::path()
.and_then(|p| Ok(std::fs::read_to_string(p)?))
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default()
}
pub fn save(&self) -> Result<()> {
let p = Self::path()?;
std::fs::create_dir_all(p.parent().unwrap())?;
std::fs::write(&p, serde_json::to_string_pretty(self)?)?;
Ok(())
}
// Used by the GUI host-list's pinned-fingerprint trust decision (the silent-reconnect
// path); the current CLI trust flow keys on address. Kept for parity with the other
// clients' known-hosts API — wired when the discovered-hosts UI lands.
#[allow(dead_code)]
pub fn find_by_fp(&self, fp_hex: &str) -> Option<&KnownHost> {
self.hosts.iter().find(|h| h.fp_hex == fp_hex)
}
pub fn find_by_addr(&self, addr: &str, port: u16) -> Option<&KnownHost> {
self.hosts.iter().find(|h| h.addr == addr && h.port == port)
}
/// Insert or refresh an entry, keyed by fingerprint. `paired` only ever upgrades
/// (a later TOFU connect must not demote a PIN-paired host).
pub fn upsert(&mut self, entry: KnownHost) {
if let Some(h) = self.hosts.iter_mut().find(|h| h.fp_hex == entry.fp_hex) {
h.name = entry.name;
h.addr = entry.addr;
h.port = entry.port;
h.paired |= entry.paired;
} else {
self.hosts.push(entry);
}
}
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)]
#[serde(default)]
pub struct Settings {
/// Stream mode; `0` = the native size/refresh of the monitor the window is on,
/// resolved at connect time.
pub width: u32,
pub height: u32,
pub refresh_hz: u32,
/// Requested encoder bitrate (kbps); 0 = host default.
pub bitrate_kbps: u32,
pub gamepad: String,
/// Which host compositor backend to request (advisory; the host falls back to
/// auto-detect when unavailable).
pub compositor: String,
/// Grab system shortcuts (Alt+Tab, Win…) while input is captured.
pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
}
impl Default for Settings {
fn default() -> Self {
Settings {
width: 0,
height: 0,
refresh_hz: 0,
bitrate_kbps: 0,
gamepad: "auto".into(),
compositor: "auto".into(),
inhibit_shortcuts: true,
mic_enabled: false,
}
}
}
impl Settings {
fn path() -> Result<PathBuf> {
Ok(config_dir()?.join("client-windows-settings.json"))
}
pub fn load() -> Settings {
Self::path()
.and_then(|p| Ok(std::fs::read_to_string(p)?))
.ok()
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default()
}
pub fn save(&self) {
let Ok(p) = Self::path() else { return };
let _ = std::fs::create_dir_all(p.parent().unwrap());
if let Ok(s) = serde_json::to_string_pretty(self) {
let _ = std::fs::write(&p, s);
}
}
}
+136
View File
@@ -0,0 +1,136 @@
//! Video decode: reassembled HEVC access units → frames for the D3D11 presenter.
//!
//! The dev box has no working GPU, so this ships the **software** backend first: libavcodec
//! on the CPU + swscale to RGBA, uploaded into a D3D11 texture by the presenter. It runs
//! `AV_CODEC_FLAG_LOW_DELAY` with slice threading only — the host encodes zero-reorder
//! streams (no B-frames, in-band parameter sets on every IDR), so decode is strictly
//! one-in/one-out and frame threading would only add latency.
//!
//! `DecodedFrame` is an enum so the real-GPU **D3D11VA** path (decode → `NV12`/`P010`
//! `ID3D11Texture2D`, zero-copy into the swapchain) can be added as a second variant without
//! touching the session pump or the presenter's frame contract.
use anyhow::{anyhow, Context as _, Result};
use ffmpeg::format::Pixel;
use ffmpeg::software::scaling;
use ffmpeg::util::frame::Video as AvFrame;
use ffmpeg_next as ffmpeg;
pub enum DecodedFrame {
Cpu(CpuFrame),
}
/// Packed 4-byte-per-pixel frame for a D3D11 texture upload (which takes a row pitch). The bytes
/// are `R8G8B8A8` for SDR and `X2BGR10` (== DXGI `R10G10B10A2`, R in the low 10 bits) for HDR.
pub struct CpuFrame {
pub width: u32,
pub height: u32,
/// Row stride in bytes (≥ width*4 — swscale pads rows for SIMD).
pub stride: usize,
pub pixels: Vec<u8>,
/// BT.2020 PQ HDR10 frame: `pixels` is `X2BGR10` and the presenter switches to a 10-bit
/// R10G10B10A2 + ST.2084 swapchain. `false` = ordinary 8-bit BT.709 SDR.
pub hdr: bool,
}
pub struct Decoder {
inner: SoftwareDecoder,
}
impl Decoder {
pub fn new() -> Result<Decoder> {
ffmpeg::init().context("ffmpeg init")?;
Ok(Decoder {
inner: SoftwareDecoder::new()?,
})
}
/// Feed one access unit; returns the decoded frame (the host's streams are
/// one-in/one-out). A decode error after packet loss is survivable — log upstream and
/// keep feeding; the host's IDR/RFI recovery resynchronizes on the next keyframe.
pub fn decode(&mut self, au: &[u8]) -> Result<Option<DecodedFrame>> {
Ok(self.inner.decode(au)?.map(DecodedFrame::Cpu))
}
}
struct SoftwareDecoder {
decoder: ffmpeg::decoder::Video,
/// Rebuilt whenever the decoded format/size **or output format** changes (mid-stream
/// `Reconfigure`, or an SDR↔HDR flip): `(ctx, src_fmt, w, h, dst_fmt)`.
sws: Option<(scaling::Context, Pixel, u32, u32, Pixel)>,
}
impl SoftwareDecoder {
fn new() -> Result<SoftwareDecoder> {
let codec =
ffmpeg::decoder::find(ffmpeg::codec::Id::HEVC).ok_or(anyhow!("no HEVC decoder"))?;
let mut ctx = ffmpeg::codec::Context::new_with_codec(codec);
unsafe {
let raw = ctx.as_mut_ptr();
(*raw).flags |= ffmpeg::ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
// Slice threading adds no frame delay (frame threading adds thread_count-1).
(*raw).thread_type = ffmpeg::ffi::FF_THREAD_SLICE;
(*raw).thread_count = 0; // auto
}
let decoder = ctx.decoder().video().context("open HEVC decoder")?;
Ok(SoftwareDecoder { decoder, sws: None })
}
fn decode(&mut self, au: &[u8]) -> Result<Option<CpuFrame>> {
let packet = ffmpeg::Packet::copy(au);
self.decoder
.send_packet(&packet)
.map_err(|e| anyhow!("send_packet: {e}"))?;
let mut frame = AvFrame::empty();
let mut out = None;
while self.decoder.receive_frame(&mut frame).is_ok() {
out = Some(self.convert(&frame)?);
}
Ok(out)
}
/// Convert the decoded YUV frame to a packed 4-byte format the presenter uploads directly:
/// SDR → `RGBA` (BT.709), HDR (SMPTE ST.2084 / PQ transfer) → `X2BGR10` (10-bit, == DXGI
/// R10G10B10A2) using the BT.2020 matrix. For HDR the PQ-encoded values pass through unchanged
/// (swscale only applies the YUV→RGB matrix + range, never the transfer) — exactly what an
/// HDR10/ST.2084 swapchain wants.
fn convert(&mut self, frame: &AvFrame) -> Result<CpuFrame> {
use ffmpeg::color::TransferCharacteristic;
let (fmt, w, h) = (frame.format(), frame.width(), frame.height());
let hdr = frame.color_transfer_characteristic() == TransferCharacteristic::SMPTE2084;
let dst = if hdr { Pixel::X2BGR10LE } else { Pixel::RGBA };
let rebuild = !matches!(&self.sws, Some((_, f, sw, sh, d)) if *f == fmt && *sw == w && *sh == h && *d == dst);
if rebuild {
let mut ctx = scaling::Context::get(fmt, w, h, dst, w, h, scaling::Flags::POINT)
.context("swscale context")?;
if hdr {
// BT.2020 non-constant-luminance YUV (limited range) → full-range RGB. swscale
// applies only the matrix + range here, so the samples stay PQ-encoded.
unsafe {
let coef = ffmpeg::ffi::sws_getCoefficients(ffmpeg::ffi::SWS_CS_BT2020);
ffmpeg::ffi::sws_setColorspaceDetails(
ctx.as_mut_ptr(),
coef,
0, // src range: limited (video)
coef,
1, // dst range: full
0,
1 << 16,
1 << 16, // brightness / contrast / saturation defaults (16.16)
);
}
}
self.sws = Some((ctx, fmt, w, h, dst));
}
let (sws, ..) = self.sws.as_mut().unwrap();
let mut conv = AvFrame::empty();
sws.run(frame, &mut conv).map_err(|e| anyhow!("sws: {e}"))?;
Ok(CpuFrame {
width: w,
height: h,
stride: conv.stride(0),
pixels: conv.data(0).to_vec(),
hdr,
})
}
}