feat: M3 — lumen/1 native streaming: real video at client mode + input over QUIC datagrams

The native protocol now does the real thing, end to end:

- Hello carries the client's requested mode; the host creates a NATIVE virtual output at
  exactly that size/refresh (same vdisplay backends as the GameStream path) and streams
  NVENC HEVC through the M1 Session (GF(2^16) Leopard FEC + AES-GCM, QUIC-negotiated).
- Input rides QUIC DATAGRAMS — encrypted, congestion-managed, no ENet retransmission
  spikes — decoded into lumen_core InputEvents and fed to the session's input injector.
- Frames are stamped with the capture wall clock; the reference client computes per-frame
  capture→reassembled latency percentiles and writes a playable .h265.
- m3-host gains --source synthetic|virtual + --seconds; the client gains --mode WxHxFPS,
  --out, --input-test (scripted mouse/keyboard datagrams).

VALIDATED live (gamescope session, xev nested): client requested 1280x720@120 → host
created gamescope at that mode → 1680/1680 frames over 14s, zero loss, valid HEVC;
pipeline latency p50 0.83ms / p95 1.2ms / p99 1.3ms (capture→encode→FEC→crypto→UDP→
reassembled, same-host clock); 176 input datagrams sent → injector (GamescopeEi) → 164
X events observed inside the nested session.

Known follow-on: slice-level sub-frame pipelining needs the NVENC SDK directly (libavcodec
emits whole AUs only) — the next big latency lever.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 06:56:47 +00:00
parent de3123038f
commit 5b0d84acd0
4 changed files with 429 additions and 84 deletions
+23 -5
View File
@@ -26,10 +26,12 @@ use crate::error::{LumenError, Result};
/// Protocol magic + version, first bytes of every message payload.
pub const MAGIC: &[u8; 4] = b"LMN1";
/// `client → host`: open the session.
/// `client → host`: open the session, requesting a display mode (the host creates its
/// virtual output at exactly this size/refresh — native resolution end to end).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Hello {
pub abi_version: u32,
pub mode: Mode,
}
/// `host → client`: the complete session offer.
@@ -56,18 +58,27 @@ pub struct Start {
impl Hello {
pub fn encode(&self) -> Vec<u8> {
let mut b = Vec::with_capacity(8);
let mut b = Vec::with_capacity(20);
b.extend_from_slice(MAGIC);
b.extend_from_slice(&self.abi_version.to_le_bytes());
b.extend_from_slice(&self.mode.width.to_le_bytes());
b.extend_from_slice(&self.mode.height.to_le_bytes());
b.extend_from_slice(&self.mode.refresh_hz.to_le_bytes());
b
}
pub fn decode(b: &[u8]) -> Result<Hello> {
if b.len() < 8 || &b[0..4] != MAGIC {
if b.len() < 20 || &b[0..4] != MAGIC {
return Err(LumenError::InvalidArg("bad Hello"));
}
let u32at = |o: usize| u32::from_le_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]]);
Ok(Hello {
abi_version: u32::from_le_bytes([b[4], b[5], b[6], b[7]]),
abi_version: u32at(4),
mode: Mode {
width: u32at(8),
height: u32at(12),
refresh_hz: u32at(16),
},
})
}
}
@@ -324,7 +335,14 @@ mod tests {
#[test]
fn hello_start_roundtrip() {
let h = Hello { abi_version: 1 };
let h = Hello {
abi_version: 1,
mode: Mode {
width: 1280,
height: 720,
refresh_hz: 120,
},
};
assert_eq!(Hello::decode(&h.encode()).unwrap(), h);
let s = Start {
client_udp_port: 1234,