feat: M4 stage 1 — the SwiftUI client is real: compiles, tested, first light on glass
ci / rust (push) Has been cancelled
ci / rust (push) Has been cancelled
The clients/apple scaffold is now a working macOS client, validated live against this repo's host across the LAN: gamescope virtual output → NVENC HEVC → lumen/1 (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox → AVSampleBufferDisplayLayer at 720p60, mouse/keyboard flowing back as QUIC datagrams into the host's gamescope EIS injector (~3.7k events injected in one session). LumenKit: - LumenConnection: the predicted cbindgen compile fixes (C17 header spells the typedefs as integers while the enum constants import as a distinct Swift type — bridge by rawValue); close() is now safe from any thread (a close flag + pumpLock held across the blocking poll enforce the C contract "never close with a next_au in flight"; flag prevents lock-starvation by back-to-back polls). - StreamView: per-pump cancellation token (reconnects can't double-pump), flush + re-gate on the next in-band parameter sets when the layer fails, no stale enqueue after restart. - InputCapture: fractional-delta accumulation (sub-pixel motion isn't truncated away), pressed-state tracking with release-all on focus loss and stop() (nothing sticks down host-side), global-singleton ownership guard (GC has one handler slot per process), X1/X2 buttons, horizontal scroll, full keypad/CapsLock/ISO-102nd/PrintScreen/Menu VKs. - LumenClient app shell (swift run LumenClient): connect form, fps/Mb-s HUD, LUMEN_AUTOCONNECT/LUMEN_MODE for scripted first-light runs. - Tests: Annex-B byte-level units; real-codec round trip (VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → AnnexB → VTDecompressionSession → pixels); test-loopback.sh (Swift client vs a real local m3-host over loopback — the Swift twin of c_abi_connection_roundtrip); RemoteFirstLightTests (full pipeline over the LAN). Host/build fixes that fell out: - The workspace builds on non-Linux again: gamestream audio (opus) and sendmmsg batching are now platform-gated with stubs/fallback, per the crate's "compiles everywhere" rule. - Horizontal scroll was inverted end-to-end: the injectors negated BOTH axes onto the ei/wl axes, but GameStream's horizontal convention is positive = right (moonlight-qt/Sunshine pass it through unnegated) — only vertical flips now. This also un-inverts real Moonlight clients. - AnnexB drops all zeros preceding a start code (trailing_zero_8bits padding), ffmpeg's policy, instead of leaking them into the preceding NAL. - build-xcframework.sh: deployment targets pinned to the package floor + an otool guard — cargo does not fingerprint MACOSX_DEPLOYMENT_TARGET, so warm caches can silently ship too-new minos objects. Adversarially reviewed (5-dimension multi-agent pass, every finding refutation-verified): 14 confirmed findings, all fixed above; the send-while-polling core-contract gap flagged here is closed by the lumen/1 session-planes work (&self pulls + per-plane borrow slots). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -7,7 +7,26 @@
|
||||
//! the media streams follow (see the M2 task list / plan).
|
||||
|
||||
pub mod apps;
|
||||
#[cfg(target_os = "linux")]
|
||||
mod audio;
|
||||
/// Stub — the audio plane needs Linux (PipeWire capture + libopus); this keeps non-Linux
|
||||
/// dev builds compiling (crate doc: "the crate compiles everywhere"). Reports failure the
|
||||
/// same way the real stream thread does: by clearing `running`.
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
mod audio {
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
pub fn start(
|
||||
running: Arc<AtomicBool>,
|
||||
_gcm_key: [u8; 16],
|
||||
_rikeyid: i32,
|
||||
_audio_cap: Arc<Mutex<Option<Box<dyn crate::audio::AudioCapturer>>>>,
|
||||
) {
|
||||
tracing::error!("GameStream audio requires Linux (PipeWire + libopus)");
|
||||
running.store(false, Ordering::SeqCst);
|
||||
}
|
||||
}
|
||||
pub(crate) mod cert;
|
||||
mod control;
|
||||
mod crypto;
|
||||
|
||||
@@ -144,6 +144,7 @@ type PacketBatch = Vec<Vec<u8>>;
|
||||
|
||||
/// Send `pkts` with as few syscalls as possible (`sendmmsg`, up to 64 per call). The socket is
|
||||
/// connected, so no per-message address. Returns an error on the first send failure.
|
||||
#[cfg(target_os = "linux")]
|
||||
fn sendmmsg_all(sock: &UdpSocket, pkts: &[Vec<u8>]) -> std::io::Result<()> {
|
||||
use std::os::fd::AsRawFd;
|
||||
const CHUNK: usize = 64;
|
||||
@@ -179,6 +180,16 @@ fn sendmmsg_all(sock: &UdpSocket, pkts: &[Vec<u8>]) -> std::io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Portable fallback (non-Linux dev builds — GameStream hosting never ships there): one
|
||||
/// syscall per packet.
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
fn sendmmsg_all(sock: &UdpSocket, pkts: &[Vec<u8>]) -> std::io::Result<()> {
|
||||
for p in pkts {
|
||||
sock.send(p)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Dedicated send thread: one [`PacketBatch`] per frame arrives on `rx`; its packets go out in
|
||||
/// `sendmmsg` chunks, paced so the frame's data spreads over ~3/4 of the frame interval
|
||||
/// (microburst shaping at chunk granularity — a real link drops line-rate bursts; the encode
|
||||
|
||||
@@ -370,10 +370,12 @@ impl EiState {
|
||||
InputKind::MouseScroll => match slot.interface::<ei::Scroll>() {
|
||||
Some(s) => {
|
||||
// GameStream sends WHEEL_DELTA(120)-scaled deltas in `x`; ei scroll_discrete
|
||||
// uses the same 120-per-detent unit. Positive GameStream = up/left, which is
|
||||
// negative on the ei axis (matches wl_pointer).
|
||||
// uses the same 120-per-detent unit. Positive GameStream = up (vertical),
|
||||
// which is negative on the ei axis, but = RIGHT (horizontal), which is
|
||||
// already positive there (moonlight-qt/Sunshine pass horizontal through
|
||||
// unnegated) — only the vertical axis flips.
|
||||
if ev.code == SCROLL_HORIZONTAL {
|
||||
s.scroll_discrete(-ev.x, 0);
|
||||
s.scroll_discrete(ev.x, 0);
|
||||
} else {
|
||||
s.scroll_discrete(0, -ev.x);
|
||||
}
|
||||
|
||||
@@ -226,10 +226,17 @@ impl InputInjector for WlrootsInjector {
|
||||
wl_pointer::Axis::VerticalScroll
|
||||
};
|
||||
// GameStream sends WHEEL_DELTA(120)-scaled units; a notch ≈ 15px. Positive
|
||||
// GameStream = scroll up, which is negative on the Wayland axis.
|
||||
// GameStream = up (vertical), negative on the Wayland axis; but = RIGHT
|
||||
// (horizontal), already positive there (moonlight-qt/Sunshine pass
|
||||
// horizontal through unnegated) — only the vertical axis flips.
|
||||
let notches = event.x as f64 / 120.0;
|
||||
let sign = if event.code == SCROLL_HORIZONTAL {
|
||||
1.0
|
||||
} else {
|
||||
-1.0
|
||||
};
|
||||
self.pointer.axis_source(wl_pointer::AxisSource::Wheel);
|
||||
self.pointer.axis(t, axis, -notches * 15.0);
|
||||
self.pointer.axis(t, axis, sign * notches * 15.0);
|
||||
self.pointer.frame();
|
||||
}
|
||||
InputKind::KeyDown | InputKind::KeyUp => {
|
||||
|
||||
@@ -448,6 +448,7 @@ fn input_thread(rx: std::sync::mpsc::Receiver<InputEvent>, conn: quinn::Connecti
|
||||
/// The audio thread: desktop capture → Opus (48 kHz stereo, 5 ms, CBR — same tuning as the
|
||||
/// GameStream path) → `AUDIO_MAGIC` datagrams. QUIC already encrypts; no extra layer.
|
||||
/// The capturer comes from (and returns to) the persistent slot — see [`AudioCapSlot`].
|
||||
#[cfg(target_os = "linux")]
|
||||
fn audio_thread(conn: quinn::Connection, stop: Arc<AtomicBool>, audio_cap: AudioCapSlot) {
|
||||
use crate::audio::{CHANNELS, SAMPLE_RATE};
|
||||
const FRAME_MS: usize = 5;
|
||||
@@ -519,6 +520,15 @@ fn audio_thread(conn: quinn::Connection, stop: Arc<AtomicBool>, audio_cap: Audio
|
||||
}
|
||||
}
|
||||
|
||||
/// Stub — lumen/1 audio needs Linux (PipeWire capture + libopus); non-Linux dev builds
|
||||
/// run sessions without it, same as when the capturer fails to open.
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
fn audio_thread(_conn: quinn::Connection, _stop: Arc<AtomicBool>, _audio_cap: AudioCapSlot) {
|
||||
tracing::warn!(
|
||||
"lumen/1 audio requires Linux (PipeWire + libopus) — session continues without it"
|
||||
);
|
||||
}
|
||||
|
||||
fn synthetic_stream(session: &mut Session, frames: u32, stop: &AtomicBool) -> Result<()> {
|
||||
let interval = std::time::Duration::from_millis(1000 / 60);
|
||||
for idx in 0..frames {
|
||||
|
||||
Reference in New Issue
Block a user