feat(client-linux): native GTK4 client — stage 1, first light at 1080p60
ci / rust (push) Failing after 29s
ci / web (push) Failing after 35s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 3s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 18s
ci / docs-site (push) Failing after 38s
apple / swift (push) Successful in 1m15s
docker / deploy-docs (push) Successful in 17s

New crate crates/punktfunk-client-linux (binary punktfunk-client), the
native Linux client on the Option A architecture (2026-06-12 research):

- GTK4/libadwaita shell linking punktfunk-core directly (no C ABI):
  mDNS host list, TOFU fingerprint prompt, SPAKE2 PIN pairing dialog,
  preferences (mode/bitrate/gamepad/shortcut capture), stats overlay,
  --connect host[:port] for scripting.
- Video: FFmpeg software HEVC decode (LOW_DELAY, slice threads) ->
  RGBA -> GdkMemoryTexture inside GtkGraphicsOffload (the dmabuf
  subsurface path lights up when VAAPI lands; black-background keeps
  fullscreen scanout-eligible).
- Audio: Opus -> PipeWire playback stream, the host virtual-mic's
  adaptive jitter ring inverted.
- Input: keyboard as the exact inverse of the host VK table (evdev
  keycodes, layout-independent; unit-tested), absolute mouse through
  the Contain-fit transform, WHEEL_DELTA(120) scroll, compositor
  shortcut inhibition while streaming, Ctrl+Alt+Shift+Q release chord,
  F11 fullscreen. SDL3 gamepad capture (single pad-0 model) + rumble
  and DualSense lightbar feedback on the same thread.
- Session pump owns video+audio pulls; the gamepad thread owns
  rumble+hidout — possible because NativeClient's plane receivers are
  now mutexed, making it Sync (Arc-shared, compiler-verified per-plane
  contract instead of the ABI's manual assertion).
- Linux-gated deps + a stub main keep cargo build --workspace green on
  macOS.

Validated live against serve --native on this box: 1920x1080@60,
locked 60 fps, capture->decoded p50 ~6.4 ms (software decode, debug
build). Teardown keys off AdwNavigationPage::hidden — NavigationView
push fires a transient unmap/map cycle that must not end the session.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-06-12 20:16:30 +00:00
parent 99b4de32ee
commit 96a35ca84c
17 changed files with 2518 additions and 4 deletions
@@ -0,0 +1,86 @@
//! Video decode: reassembled HEVC access units → RGBA frames for the GTK presenter.
//!
//! Stage 1 is libavcodec software decode + swscale to RGBA (`GdkMemoryTexture` upload on
//! the UI side). The host encodes zero-reorder streams (no B-frames, in-band parameter
//! sets on every IDR), so with `AV_CODEC_FLAG_LOW_DELAY` the decoder is strictly
//! one-in/one-out with no hidden queue. Slice threading only — frame threading would add
//! a frame of latency per extra thread.
//!
//! Stage 1.5 (Intel/AMD boxes): VAAPI hwaccel → DRM-PRIME dmabuf → `GdkDmabufTexture`,
//! slotting in behind the same `decode()` signature. Stage 2 (NVIDIA): Vulkan Video in
//! the bespoke presenter (see the design notes in docs-site).
use anyhow::{anyhow, Context as _, Result};
use ffmpeg::format::Pixel;
use ffmpeg::software::scaling;
use ffmpeg::util::frame::Video as AvFrame;
use ffmpeg_next as ffmpeg;
/// One decoded frame, tightly enough packed for `GdkMemoryTexture` (which takes a stride).
pub struct DecodedFrame {
pub width: u32,
pub height: u32,
/// RGBA row stride in bytes (≥ width*4 — swscale pads rows for SIMD).
pub stride: usize,
pub rgba: Vec<u8>,
}
pub struct Decoder {
decoder: ffmpeg::decoder::Video,
/// Rebuilt whenever the decoded format/size changes (mid-stream `Reconfigure`).
sws: Option<(scaling::Context, Pixel, u32, u32)>,
}
impl Decoder {
pub fn new() -> Result<Decoder> {
ffmpeg::init().context("ffmpeg init")?;
let codec =
ffmpeg::decoder::find(ffmpeg::codec::Id::HEVC).ok_or(anyhow!("no HEVC decoder"))?;
let mut ctx = ffmpeg::codec::Context::new_with_codec(codec);
unsafe {
let raw = ctx.as_mut_ptr();
(*raw).flags |= ffmpeg::ffi::AV_CODEC_FLAG_LOW_DELAY as i32;
// Slice threading adds no frame delay (frame threading adds thread_count-1).
(*raw).thread_type = ffmpeg::ffi::FF_THREAD_SLICE;
(*raw).thread_count = 0; // auto
}
let decoder = ctx.decoder().video().context("open HEVC decoder")?;
Ok(Decoder { decoder, sws: None })
}
/// Feed one access unit; returns the decoded frame (the host's streams are
/// one-in/one-out). A decode error after packet loss is survivable — log upstream and
/// keep feeding; the host's RFI/IDR recovery resynchronizes the reference chain.
pub fn decode(&mut self, au: &[u8]) -> Result<Option<DecodedFrame>> {
let packet = ffmpeg::Packet::copy(au);
self.decoder
.send_packet(&packet)
.map_err(|e| anyhow!("send_packet: {e}"))?;
let mut frame = AvFrame::empty();
let mut out = None;
while self.decoder.receive_frame(&mut frame).is_ok() {
out = Some(self.convert_rgba(&frame)?);
}
Ok(out)
}
fn convert_rgba(&mut self, frame: &AvFrame) -> Result<DecodedFrame> {
let (fmt, w, h) = (frame.format(), frame.width(), frame.height());
let rebuild =
!matches!(&self.sws, Some((_, f, sw, sh)) if *f == fmt && *sw == w && *sh == h);
if rebuild {
let ctx = scaling::Context::get(fmt, w, h, Pixel::RGBA, w, h, scaling::Flags::POINT)
.context("swscale context")?;
self.sws = Some((ctx, fmt, w, h));
}
let (sws, ..) = self.sws.as_mut().unwrap();
let mut rgba = AvFrame::empty();
sws.run(frame, &mut rgba).map_err(|e| anyhow!("sws: {e}"))?;
Ok(DecodedFrame {
width: w,
height: h,
stride: rgba.stride(0),
rgba: rgba.data(0).to_vec(),
})
}
}