520d7342dd
ci / rust (push) Has been cancelled
m3-host is now a real host, not a one-shot demo. Everything validated live on this box (two back-to-back sessions, pinned + TOFU, ~200 audio pkts/s, p50 0.84 ms at 720p60). lumen-core: - quic.rs: QUIC-datagram side planes demuxed by first byte — Opus audio 0xC9 ([magic][u32 seq][u64 pts_ns][opus], host→client) and rumble 0xCA ([magic][pad][low][high]). - Trust: endpoint::server_with_identity (persistent PEM identity) and endpoint::client_pinned — SHA-256 cert-fingerprint pinning with TOFU (observed fingerprint reported back for persisting). The verifier checks the TLS 1.3 CertificateVerify signature for real (an MITM replaying the host's public cert without its key is rejected; cert pinning alone would not prove key possession). - client.rs: NativeClient gains pin + host_fingerprint, audio/rumble receivers (next_audio / next_rumble); pull methods take &self so the C ABI's per-plane threads never alias a &mut (per-plane mutexed borrow slots in abi.rs). - abi.rs: lumen_connect(pin_sha256, observed_sha256_out) + lumen_connection_next_audio / next_rumble. input.rs: documented gamepad wire contract (GameStream buttonFlags bits, XInput axis conventions, +y = up) — exported as LUMEN_BTN_*/LUMEN_AXIS_* (bare BTN_* collides with <linux/input-event-codes.h> at different values). lumen-host (m3): - Persistent accept loop: sessions back to back on one endpoint (--max-sessions, 0 = forever); per-session failures log and the loop keeps serving; 10 s handshake deadline so a silent client can't wedge the sequential accept queue; teardown on every exit path (stop flag → conn.close → join audio+input threads). - Audio plane: desktop PipeWire capture → Opus 48 kHz stereo 5 ms CBR → datagrams; ONE capturer reused across sessions via an AudioCapSlot (PipeWire streams have no cheap teardown — per-session opens would leak a thread + core connection + live node each). - Gamepad routing: incremental GamepadButton/GamepadAxis datagrams accumulate into per-pad state feeding the uinput xpad manager; force feedback returns as rumble datagrams, with current state re-sent every 500 ms (idempotent-state healing for the lossy channel). QUIC endpoint serves the persistent ~/.config/lumen identity and logs the pinnable fingerprint. lumen-client-rs: --pin (malformed values abort — never silently downgrade to TOFU), TOFU fingerprint logging, audio/rumble datagram counters, gamepad events in --input-test. clients/apple: scaffold synced — pinSHA256/hostFingerprint (wrong-size pin throws, fail-closed), nextAudio/nextRumble, gamepad event constructors; README handoff updated (persistent listener, audio decode notes, trust UX). Adversarially reviewed (5-dimension multi-agent pass over the diff, 2-skeptic verification): fixed the MITM signature-check gap, a Y-axis contract inversion, header macro collisions, ABI aliasing UB, the PipeWire per-session leak, the missing handshake deadline, fail-open pin parsing, and teardown-on-error paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
191 lines
8.2 KiB
Swift
191 lines
8.2 KiB
Swift
// Swift wrapper around the lumen-core C ABI's lumen/1 connection API.
|
||
//
|
||
// Threading contract (mirrors the C header): one LumenConnection is used from a single
|
||
// pump thread for nextAU(); nextAudio() may run on its own (single) audio thread;
|
||
// sendInput() is enqueue-only and safe alongside both. The pointers inside an AU/audio
|
||
// packet are only valid until the next call of the same kind, so we copy into Data here —
|
||
// the copies are small and keep the Swift side memory-safe.
|
||
//
|
||
// Trust: pass the host's pinned certificate fingerprint (the host logs it at startup, and
|
||
// `hostFingerprint` reports what a trust-on-first-use connect observed — persist it, e.g.
|
||
// in UserDefaults keyed by host, and pin it from then on).
|
||
//
|
||
// SCAFFOLD: written on the Linux host, not yet compiled against Xcode — expect to fix
|
||
// trivial issues on first build (see README.md "Handoff").
|
||
|
||
import Foundation
|
||
import LumenCore
|
||
|
||
/// One reassembled, FEC-recovered, decrypted access unit (Annex-B HEVC from the host).
|
||
public struct AccessUnit: Sendable {
|
||
public let data: Data
|
||
public let ptsNs: UInt64
|
||
public let frameIndex: UInt32
|
||
public let flags: UInt32
|
||
}
|
||
|
||
/// One Opus audio packet (48 kHz stereo, 5 ms frames) — decode with AVAudioConverter
|
||
/// (`kAudioFormatOpus`) or libopus into an AVAudioEngine source node.
|
||
public struct AudioPacket: Sendable {
|
||
public let data: Data
|
||
public let ptsNs: UInt64
|
||
public let seq: UInt32
|
||
}
|
||
|
||
public enum LumenClientError: Error {
|
||
/// Connect failed — wrong host/port, timeout, or a certificate-pin mismatch.
|
||
case connectFailed
|
||
/// `pinSHA256` was non-nil but not exactly 32 bytes. Failing closed: connecting
|
||
/// unpinned when the caller asked for verification would be a silent trust downgrade.
|
||
case invalidPin
|
||
case closed
|
||
}
|
||
|
||
public final class LumenConnection {
|
||
private var handle: OpaquePointer?
|
||
|
||
/// Negotiated session mode (host-confirmed).
|
||
public private(set) var width: UInt32 = 0
|
||
public private(set) var height: UInt32 = 0
|
||
public private(set) var refreshHz: UInt32 = 0
|
||
|
||
/// SHA-256 fingerprint of the certificate the host presented (32 bytes). After a
|
||
/// trust-on-first-use connect, persist this and pass it as `pinSHA256` next time.
|
||
public private(set) var hostFingerprint: Data = Data()
|
||
|
||
/// Connect and start a session at the requested mode (the host creates a native virtual
|
||
/// output at exactly this size/refresh). Blocks up to `timeoutMs`.
|
||
///
|
||
/// `pinSHA256`: the host's expected certificate fingerprint (exactly 32 bytes, else
|
||
/// `invalidPin` is thrown — never silently downgraded); nil = trust on first use
|
||
/// (check `hostFingerprint` afterwards). A pinned mismatch throws.
|
||
public init(
|
||
host: String, port: UInt16 = 9777,
|
||
width: UInt32, height: UInt32, refreshHz: UInt32,
|
||
pinSHA256: Data? = nil,
|
||
timeoutMs: UInt32 = 10_000
|
||
) throws {
|
||
if let pin = pinSHA256, pin.count != 32 { throw LumenClientError.invalidPin }
|
||
var observed = [UInt8](repeating: 0, count: 32)
|
||
handle = host.withCString { cs in
|
||
if let pin = pinSHA256 {
|
||
return pin.withUnsafeBytes { p in
|
||
lumen_connect(
|
||
cs, port, width, height, refreshHz,
|
||
p.bindMemory(to: UInt8.self).baseAddress, &observed, timeoutMs)
|
||
}
|
||
}
|
||
return lumen_connect(cs, port, width, height, refreshHz, nil, &observed, timeoutMs)
|
||
}
|
||
guard handle != nil else { throw LumenClientError.connectFailed }
|
||
hostFingerprint = Data(observed)
|
||
var w: UInt32 = 0, h: UInt32 = 0, hz: UInt32 = 0
|
||
_ = lumen_connection_mode(handle, &w, &h, &hz)
|
||
self.width = w
|
||
self.height = h
|
||
self.refreshHz = hz
|
||
}
|
||
|
||
/// Pull the next access unit; nil on timeout, throws once the session is closed.
|
||
public func nextAU(timeoutMs: UInt32 = 100) throws -> AccessUnit? {
|
||
var frame = LumenFrame()
|
||
switch lumen_connection_next_au(handle, &frame, timeoutMs) {
|
||
case LUMEN_STATUS_OK:
|
||
let data = Data(bytes: frame.data, count: frame.len) // copy: ptr valid only until next call
|
||
return AccessUnit(
|
||
data: data, ptsNs: frame.pts_ns,
|
||
frameIndex: frame.frame_index, flags: frame.flags)
|
||
case LUMEN_STATUS_NO_FRAME:
|
||
return nil
|
||
case LUMEN_STATUS_CLOSED:
|
||
throw LumenClientError.closed
|
||
default:
|
||
throw LumenClientError.closed
|
||
}
|
||
}
|
||
|
||
/// Pull the next Opus audio packet; nil on timeout, throws once the session is closed.
|
||
/// Drain from a dedicated audio thread — packets arrive every 5 ms (320 ms buffered).
|
||
public func nextAudio(timeoutMs: UInt32 = 100) throws -> AudioPacket? {
|
||
var pkt = LumenAudioPacket()
|
||
switch lumen_connection_next_audio(handle, &pkt, timeoutMs) {
|
||
case LUMEN_STATUS_OK:
|
||
let data = Data(bytes: pkt.data, count: pkt.len) // copy: ptr valid only until next call
|
||
return AudioPacket(data: data, ptsNs: pkt.pts_ns, seq: pkt.seq)
|
||
case LUMEN_STATUS_NO_FRAME:
|
||
return nil
|
||
default:
|
||
throw LumenClientError.closed
|
||
}
|
||
}
|
||
|
||
/// Pull the next force-feedback update for the GCController haptics engine:
|
||
/// `(pad, lowFrequency, highFrequency)` with 0...0xFFFF amplitudes, (0, 0) = stop.
|
||
public func nextRumble(timeoutMs: UInt32 = 100) throws -> (pad: UInt16, low: UInt16, high: UInt16)? {
|
||
var pad: UInt16 = 0, low: UInt16 = 0, high: UInt16 = 0
|
||
switch lumen_connection_next_rumble(handle, &pad, &low, &high, timeoutMs) {
|
||
case LUMEN_STATUS_OK:
|
||
return (pad, low, high)
|
||
case LUMEN_STATUS_NO_FRAME:
|
||
return nil
|
||
default:
|
||
throw LumenClientError.closed
|
||
}
|
||
}
|
||
|
||
/// Send one input event (delivered to the host as a QUIC datagram).
|
||
public func send(_ event: LumenInputEvent) {
|
||
var ev = event
|
||
_ = lumen_connection_send_input(handle, &ev)
|
||
}
|
||
|
||
public func close() {
|
||
if let h = handle {
|
||
lumen_connection_close(h)
|
||
handle = nil
|
||
}
|
||
}
|
||
|
||
deinit { close() }
|
||
}
|
||
|
||
// Convenience constructors for the wire input events (field semantics match
|
||
// lumen_core::input::InputEvent; see lumen_core.h).
|
||
public extension LumenInputEvent {
|
||
static func mouseMove(dx: Int32, dy: Int32) -> LumenInputEvent {
|
||
LumenInputEvent(kind: LUMEN_INPUT_KIND_MOUSE_MOVE, _pad: (0, 0, 0), code: 0, x: dx, y: dy, flags: 0)
|
||
}
|
||
static func mouseButton(_ button: UInt32, down: Bool) -> LumenInputEvent {
|
||
LumenInputEvent(
|
||
kind: down ? LUMEN_INPUT_KIND_MOUSE_BUTTON_DOWN : LUMEN_INPUT_KIND_MOUSE_BUTTON_UP,
|
||
_pad: (0, 0, 0), code: button, x: 0, y: 0, flags: 0)
|
||
}
|
||
static func key(_ vk: UInt32, down: Bool) -> LumenInputEvent {
|
||
LumenInputEvent(
|
||
kind: down ? LUMEN_INPUT_KIND_KEY_DOWN : LUMEN_INPUT_KIND_KEY_UP,
|
||
_pad: (0, 0, 0), code: vk, x: 0, y: 0, flags: 0)
|
||
}
|
||
static func scroll(_ delta: Int32) -> LumenInputEvent {
|
||
LumenInputEvent(kind: LUMEN_INPUT_KIND_MOUSE_SCROLL, _pad: (0, 0, 0), code: 0, x: delta, y: 0, flags: 0)
|
||
}
|
||
|
||
// Gamepad (wire contract in lumen_core::input::gamepad): one transition per event,
|
||
// `pad` = controller index, accumulated host-side into a virtual Xbox 360 pad.
|
||
|
||
/// `button` is a GameStream buttonFlags bit (A=0x1000 B=0x2000 X=0x4000 Y=0x8000,
|
||
/// dpad=0x1/2/4/8, start=0x10 back=0x20 LS=0x40 RS=0x80 LB=0x100 RB=0x200 guide=0x400).
|
||
static func gamepadButton(_ button: UInt32, down: Bool, pad: UInt32 = 0) -> LumenInputEvent {
|
||
LumenInputEvent(
|
||
kind: LUMEN_INPUT_KIND_GAMEPAD_BUTTON,
|
||
_pad: (0, 0, 0), code: button, x: down ? 1 : 0, y: 0, flags: pad)
|
||
}
|
||
|
||
/// Axis ids: 0=LSX 1=LSY 2=RSX 3=RSY (−32768...32767, XInput convention: +y = UP —
|
||
/// `GCControllerDirectionPad.yAxis` already matches, no flip), 4=LT 5=RT (0...255).
|
||
static func gamepadAxis(_ axis: UInt32, value: Int32, pad: UInt32 = 0) -> LumenInputEvent {
|
||
LumenInputEvent(
|
||
kind: LUMEN_INPUT_KIND_GAMEPAD_AXIS,
|
||
_pad: (0, 0, 0), code: axis, x: value, y: 0, flags: pad)
|
||
}
|
||
}
|