feat: M4 groundwork — lumen/1 client connector in the C ABI + SwiftUI client scaffold
ci / rust (push) Has been cancelled

The shared-core architecture pays off: platform clients now link ONE Rust library that
does the entire lumen/1 protocol, and only add decode/present/input on top.

lumen-core:
- client.rs (quic feature): NativeClient — QUIC handshake + UDP data plane + input
  datagrams on internal threads; embedder surface = connect / next_frame / send_input.
- abi.rs: lumen_connect / lumen_connection_next_au (borrow-until-next-call, matching
  lumen_client_poll_frame semantics) / lumen_connection_send_input / lumen_connection_mode /
  lumen_connection_close. Guarded in the generated header by LUMEN_FEATURE_QUIC (cbindgen
  [defines] mapping), so the checked-in header is stable across feature sets.
- error.rs: append-only LumenStatus additions Timeout (-9) and Closed (-10).
- TESTED end-to-end through the C ABI: in-process lumen/1 host, lumen_connect pulls 25
  byte-verified frames, sends input, closes (m3.rs::c_abi_connection_roundtrip).

Apple client (clients/apple — SCAFFOLD, written on Linux, first Xcode build pending):
- scripts/build-xcframework.sh: cargo per Apple target → universal staticlib + header
  (LUMEN_FEATURE_QUIC pre-defined) + modulemap → LumenCore.xcframework.
- Package.swift (LumenKit) + Swift sources: LumenConnection (ABI wrapper), AnnexB
  (in-band VPS/SPS/PPS → CMVideoFormatDescription, Annex-B → AVCC CMSampleBuffers with
  DisplayImmediately), StreamView (SwiftUI over AVSampleBufferDisplayLayer — stage-1
  presenter that hardware-decodes compressed HEVC itself), InputCapture (GCMouse raw
  deltas + GCKeyboard HID→VK).
- README.md is the full handoff for the next (Mac-side) agent: build steps, ABI contract,
  first-light test recipe against the Linux host, stage-2 (VT+Metal pacing) plan, and the
  known host-side gaps (single-session m3-host, no lumen/1 audio yet, gamepad kinds not
  yet routed in m3's injector, seed-stage trust).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 07:28:41 +00:00
parent 2b4ffc3518
commit 3ea096ace9
17 changed files with 1147 additions and 26 deletions
@@ -0,0 +1,83 @@
// SwiftUI presentation: AVSampleBufferDisplayLayer fed straight from the lumen/1 connection.
//
// Stage-1 presenter (see README): the layer accepts *compressed* HEVC sample buffers and
// does hardware decode + display itself fastest path to pixels, IOSurface-backed
// zero-copy on Apple silicon. Stage 2 (explicit VTDecompressionSession + CAMetalLayer)
// replaces this when we start tuning frame pacing / measuring glass-to-glass.
//
// SCAFFOLD: written on the Linux host, not yet compiled against Xcode. macOS-first
// (NSViewRepresentable); the iOS variant is the same layer under UIViewRepresentable.
#if os(macOS)
import AVFoundation
import SwiftUI
public struct StreamView: NSViewRepresentable {
private let connection: LumenConnection
public init(connection: LumenConnection) {
self.connection = connection
}
public func makeNSView(context: Context) -> StreamLayerView {
let view = StreamLayerView()
view.start(connection: connection)
return view
}
public func updateNSView(_ view: StreamLayerView, context: Context) {}
}
public final class StreamLayerView: NSView {
private let displayLayer = AVSampleBufferDisplayLayer()
private var pump: Thread?
private var running = false
public override init(frame: NSRect) {
super.init(frame: frame)
wantsLayer = true
displayLayer.videoGravity = .resizeAspect
layer = displayLayer
}
public required init?(coder: NSCoder) { fatalError("not used") }
/// Pump thread: pull AUs from the connection, wrap, enqueue. The first IDR yields the
/// format description; non-IDR AUs before it are dropped (the host opens with an IDR).
public func start(connection: LumenConnection) {
guard !running else { return }
running = true
let layer = displayLayer
let thread = Thread { [weak self] in
var format: CMVideoFormatDescription?
while self?.running == true {
do {
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
if let f = AnnexB.formatDescription(fromIDR: au.data) {
format = f // refreshed on every IDR (mode changes included)
}
guard let f = format,
let sample = AnnexB.sampleBuffer(au: au, format: f)
else { continue }
if layer.status == .failed {
layer.flush()
}
layer.enqueue(sample)
} catch {
break // session closed
}
}
}
thread.name = "lumen-pump"
thread.qualityOfService = .userInteractive
pump = thread
thread.start()
}
public func stop() {
running = false
}
deinit { running = false }
}
#endif