feat: M4 stage 1 — the SwiftUI client is real: compiles, tested, first light on glass
ci / rust (push) Has been cancelled
ci / rust (push) Has been cancelled
The clients/apple scaffold is now a working macOS client, validated live against this repo's host across the LAN: gamescope virtual output → NVENC HEVC → lumen/1 (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox → AVSampleBufferDisplayLayer at 720p60, mouse/keyboard flowing back as QUIC datagrams into the host's gamescope EIS injector (~3.7k events injected in one session). LumenKit: - LumenConnection: the predicted cbindgen compile fixes (C17 header spells the typedefs as integers while the enum constants import as a distinct Swift type — bridge by rawValue); close() is now safe from any thread (a close flag + pumpLock held across the blocking poll enforce the C contract "never close with a next_au in flight"; flag prevents lock-starvation by back-to-back polls). - StreamView: per-pump cancellation token (reconnects can't double-pump), flush + re-gate on the next in-band parameter sets when the layer fails, no stale enqueue after restart. - InputCapture: fractional-delta accumulation (sub-pixel motion isn't truncated away), pressed-state tracking with release-all on focus loss and stop() (nothing sticks down host-side), global-singleton ownership guard (GC has one handler slot per process), X1/X2 buttons, horizontal scroll, full keypad/CapsLock/ISO-102nd/PrintScreen/Menu VKs. - LumenClient app shell (swift run LumenClient): connect form, fps/Mb-s HUD, LUMEN_AUTOCONNECT/LUMEN_MODE for scripted first-light runs. - Tests: Annex-B byte-level units; real-codec round trip (VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → AnnexB → VTDecompressionSession → pixels); test-loopback.sh (Swift client vs a real local m3-host over loopback — the Swift twin of c_abi_connection_roundtrip); RemoteFirstLightTests (full pipeline over the LAN). Host/build fixes that fell out: - The workspace builds on non-Linux again: gamestream audio (opus) and sendmmsg batching are now platform-gated with stubs/fallback, per the crate's "compiles everywhere" rule. - Horizontal scroll was inverted end-to-end: the injectors negated BOTH axes onto the ei/wl axes, but GameStream's horizontal convention is positive = right (moonlight-qt/Sunshine pass it through unnegated) — only vertical flips now. This also un-inverts real Moonlight clients. - AnnexB drops all zeros preceding a start code (trailing_zero_8bits padding), ffmpeg's policy, instead of leaking them into the preceding NAL. - build-xcframework.sh: deployment targets pinned to the package floor + an otool guard — cargo does not fingerprint MACOSX_DEPLOYMENT_TARGET, so warm caches can silently ship too-new minos objects. Adversarially reviewed (5-dimension multi-agent pass, every finding refutation-verified): 14 confirmed findings, all fixed above; the send-while-polling core-contract gap flagged here is closed by the lumen/1 session-planes work (&self pulls + per-plane borrow slots). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -7,3 +7,7 @@
|
||||
/tools/*/target
|
||||
node_modules/
|
||||
dist/
|
||||
# Swift package build artifacts + the locally-built xcframework (rebuild via scripts/build-xcframework.sh)
|
||||
clients/apple/.build/
|
||||
clients/apple/LumenCore.xcframework/
|
||||
clients/apple/.swiftpm/
|
||||
|
||||
@@ -44,11 +44,16 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
|
||||
|
||||
## What's left
|
||||
|
||||
1. **M4 — client decode + present**: the SwiftUI client is scaffolded and handed off —
|
||||
the lumen/1 connector is in the C ABI (`lumen_connect` & co., ABI-roundtrip-tested) with
|
||||
an xcframework build script + LumenKit Swift package; **see
|
||||
[`clients/apple/README.md`](clients/apple/README.md) for the Mac-side pickup**. Then
|
||||
glass-to-glass numbers via `tools/latency-probe` (scaffold). The Linux reference client
|
||||
1. **M4 — client decode + present: macOS stage 1 done, first light achieved
|
||||
(2026-06-10).** LumenKit compiles and is tested on macOS (AnnexB → VideoToolbox →
|
||||
`AVSampleBufferDisplayLayer`, GCMouse/GCKeyboard capture, `LumenClient` app shell);
|
||||
validated live Mac ↔ this box at 720p60 — vkcube on glass, input injected via gamescope
|
||||
EIS. Tests: `swift test` in `clients/apple` (unit + real-codec round trip),
|
||||
`test-loopback.sh` (Swift client vs synthetic m3-host on loopback — runs on macOS),
|
||||
`RemoteFirstLightTests` (full pipeline over the LAN). See
|
||||
[`clients/apple/README.md`](clients/apple/README.md). Next: stage 2 presenter
|
||||
(`VTDecompressionSession` + `CAMetalLayer` frame pacing), glass-to-glass numbers via
|
||||
`tools/latency-probe` (scaffold), iOS variant. The Linux reference client
|
||||
(`lumen-client-rs`) gets VAAPI + wgpu on the same connector later.
|
||||
2. **Sub-frame pipelining**: overlap encode and transmit within a frame. Requires a direct
|
||||
NVENC SDK wrapper (libavcodec only emits whole AUs) — the next big latency lever (~2–4 ms
|
||||
@@ -61,9 +66,9 @@ Low-latency desktop/game streaming stack, Linux-first, with a shared Rust protoc
|
||||
HDR/10-bit/AV1 negotiation, surround audio, reconnect-at-new-mode robustness.
|
||||
5. **Native clients** (`clients/{apple,android}` scaffolds) consuming `lumen_core.h`.
|
||||
6. **This box, one-time setup still pending**: `sudo cp scripts/60-lumen.rules
|
||||
/etc/udev/rules.d/` + user into `input` group (gamepads); `sudo ninja -C
|
||||
/tmp/gamescope-src/build install` (the fixed gamescope ≥ 3.16.22 — until then use
|
||||
`PATH=/tmp/gamescope-src/build/src:$PATH`); `apt install gnome-shell` (Mutter validation).
|
||||
/etc/udev/rules.d/` + user into `input` group (gamepads); `apt install gnome-shell`
|
||||
(Mutter validation). Done since last update: gamescope 3.16.22 is installed at
|
||||
`/usr/local/bin` — the `PATH=/tmp/gamescope-src/...` override is no longer needed.
|
||||
|
||||
## Build / test / run
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ negotiated extension. See [`docs/implementation-plan.md`](docs/implementation-pl
|
||||
| M2 — P1 host → stock Moonlight | 🟡 capture+encode landed in M0; pairing/RTSP/vdisplay pending |
|
||||
| M3 — measurement harness | 🟡 `tools/loss-harness` runs; `latency-probe` scaffolded |
|
||||
| M4 — P2 transport + Rust client | 🟡 GF(2¹⁶) core done; `lumen-client-rs` scaffolded |
|
||||
| M5 — Apple client | ⬜ scaffolded (`clients/apple`) |
|
||||
| M5 — Apple client | 🟡 macOS first light: HEVC on glass + input over `lumen/1` (`clients/apple`) |
|
||||
|
||||
`lumen-core` is complete and verified: it builds and its full test suite (FEC recovery,
|
||||
loopback round-trip under loss, property tests, and a **C ABI harness**) passes on
|
||||
|
||||
@@ -8,7 +8,8 @@ let package = Package(
|
||||
name: "LumenKit",
|
||||
platforms: [.macOS(.v14), .iOS(.v17)],
|
||||
products: [
|
||||
.library(name: "LumenKit", targets: ["LumenKit"])
|
||||
.library(name: "LumenKit", targets: ["LumenKit"]),
|
||||
.executable(name: "LumenClient", targets: ["LumenClient"]),
|
||||
],
|
||||
targets: [
|
||||
.binaryTarget(name: "LumenCore", path: "LumenCore.xcframework"),
|
||||
@@ -22,5 +23,8 @@ let package = Package(
|
||||
.linkedLibrary("resolv"),
|
||||
]
|
||||
),
|
||||
// Development app shell (swift run LumenClient): connect form → stream + input.
|
||||
.executableTarget(name: "LumenClient", dependencies: ["LumenKit"]),
|
||||
.testTarget(name: "LumenKitTests", dependencies: ["LumenKit"]),
|
||||
]
|
||||
)
|
||||
|
||||
+86
-76
@@ -1,102 +1,112 @@
|
||||
# lumen Apple client (SwiftUI) — handoff
|
||||
# lumen Apple client (SwiftUI)
|
||||
|
||||
The native macOS/iOS client for **`lumen/1`** (the post-GameStream protocol). All
|
||||
networking/protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM,
|
||||
input datagrams — lives in the shared Rust core and is **done and tested**; this package
|
||||
is the Swift shell: decode (VideoToolbox), present (SwiftUI), input capture.
|
||||
input datagrams, Opus audio, cert pinning — lives in the shared Rust core (statically
|
||||
linked as `LumenCore.xcframework`); this package is the Swift shell: decode
|
||||
(VideoToolbox), present (SwiftUI), input capture.
|
||||
|
||||
## What exists (built + tested on the Linux host)
|
||||
## Status — first light achieved (2026-06-10)
|
||||
|
||||
- **The connector**: `lumen_core::client::NativeClient` (Rust) exposed over the C ABI as
|
||||
`lumen_connect` / `lumen_connection_next_au` / `lumen_connection_next_audio` /
|
||||
`lumen_connection_next_rumble` / `lumen_connection_send_input` / `lumen_connection_mode`
|
||||
/ `lumen_connection_close` (see `include/lumen_core.h`, guarded by `LUMEN_FEATURE_QUIC`).
|
||||
**End-to-end tested through the C ABI** against an in-process host
|
||||
(`crates/lumen-host/src/m3.rs::tests::c_abi_connection_roundtrip` — three sequential
|
||||
sessions: TOFU, pinned reconnect, wrong-pin rejection).
|
||||
- **The host to test against**: `lumen-host m3-host --source virtual --seconds 60` on the
|
||||
Linux box — a **persistent listener** (sessions back to back, reconnect at will during
|
||||
development; `--max-sessions N` to bound it). It creates a native virtual output at
|
||||
whatever mode the client requests and streams HEVC + desktop **Opus audio**;
|
||||
`LUMEN_COMPOSITOR=gamescope LUMEN_GAMESCOPE_APP=vkcube` for moving content.
|
||||
- **This package (SCAFFOLD — written on Linux, never compiled in Xcode)**:
|
||||
- `LumenConnection.swift` — Swift wrapper over the C ABI (AUs/audio copied into `Data`;
|
||||
certificate pinning + TOFU fingerprint via `pinSHA256:`/`hostFingerprint`).
|
||||
Validated live, Mac ↔ Linux box over the LAN: gamescope virtual output → NVENC HEVC →
|
||||
`lumen/1` (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox →
|
||||
`AVSampleBufferDisplayLayer` on glass at 1280×720@60, with mouse/keyboard flowing back as
|
||||
QUIC datagrams into the host's gamescope EIS injector (thousands of events injected during
|
||||
the session). Headless variant of the same proof: `RemoteFirstLightTests` decoded 60/60
|
||||
received AUs spanning 983 ms of host capture clock.
|
||||
|
||||
The connector underneath (`lumen_core::client::NativeClient` over the C ABI) carries the
|
||||
full session: video AUs, **Opus audio** (`nextAudio()`), **rumble** (`nextRumble()`),
|
||||
input incl. gamepads, and **cert pinning + TOFU** (`pinSHA256:`/`hostFingerprint`) — see
|
||||
`m3.rs::tests::c_abi_connection_roundtrip` (three sequential sessions: TOFU, pinned
|
||||
reconnect, wrong-pin rejection). The host (`lumen-host m3-host`) is a persistent listener:
|
||||
reconnect at will during development.
|
||||
|
||||
What's here, all compiled and tested on macOS (Xcode 26.5 / Swift 6.3):
|
||||
|
||||
- **`LumenKit`** (library)
|
||||
- `LumenConnection.swift` — wrapper over the C ABI. AUs/audio are copied into `Data`
|
||||
(the C pointer is only valid until the next call of the same kind). `close()` is safe
|
||||
from any thread: per-plane locks enforce the C contract ("never close with a
|
||||
`next_au`/`next_audio` in flight") instead of leaving it to callers. Pinning + TOFU
|
||||
via `pinSHA256:`/`hostFingerprint`.
|
||||
- `AnnexB.swift` — in-band VPS/SPS/PPS → `CMVideoFormatDescription`; Annex-B → AVCC
|
||||
`CMSampleBuffer` with `DisplayImmediately` set.
|
||||
- `StreamView.swift` — SwiftUI `NSViewRepresentable` over `AVSampleBufferDisplayLayer`
|
||||
(stage-1 presenter: the layer hardware-decodes compressed HEVC itself).
|
||||
- `InputCapture.swift` — `GCMouse` raw deltas + `GCKeyboard` HID→VK mapping →
|
||||
`lumen_connection_send_input`.
|
||||
(stage-1 presenter: the layer hardware-decodes compressed HEVC itself). One pump
|
||||
thread per view, token-cancelled so reconnects can't double-pump.
|
||||
- `InputCapture.swift` — `GCMouse` raw deltas + `GCKeyboard` HID→VK mapping (the host's
|
||||
`vk_to_evdev` consumes Windows VKs), with fractional-delta accumulation so sub-pixel
|
||||
motion isn't truncated away. Buttons use GameStream ids (1=left … 5=X2); scroll is
|
||||
WHEEL_DELTA(120)-scaled.
|
||||
- **`LumenClient`** (development app shell): connect form → stream + input, fps/Mb-s HUD.
|
||||
(Audio playback and gamepad capture are not wired into the app yet — the connector
|
||||
surface is there; see notes 5–6.)
|
||||
- **Tests** (`swift test`): byte-level Annex-B units; a real-codec round trip
|
||||
(VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → `AnnexB` →
|
||||
VTDecompressionSession → pixels); loopback integration against a real local host
|
||||
(`test-loopback.sh`); the remote first-light test above.
|
||||
|
||||
## Build steps (on the Mac)
|
||||
## Build / run / test (on a Mac)
|
||||
|
||||
```sh
|
||||
rustup target add aarch64-apple-darwin x86_64-apple-darwin
|
||||
bash scripts/build-xcframework.sh # → clients/apple/LumenCore.xcframework
|
||||
open clients/apple/Package.swift # or add the package to an Xcode app project
|
||||
cd clients/apple
|
||||
swift build && swift test # loopback/remote tests self-skip without a host
|
||||
swift run LumenClient # the app; or open Package.swift in Xcode
|
||||
|
||||
bash test-loopback.sh # full loopback proof: builds lumen-host
|
||||
# (synthetic source — runs on macOS), streams
|
||||
# byte-verified frames into the Swift client
|
||||
|
||||
# against the real host (Linux box, see CLAUDE.md "Running on this box") — m3-host is a
|
||||
# persistent listener, reconnect at will:
|
||||
# LUMEN_COMPOSITOR=gamescope LUMEN_GAMESCOPE_APP=vkcube LUMEN_ZEROCOPY=1 \
|
||||
# cargo run -rp lumen-host -- m3-host --source virtual --seconds 60
|
||||
LUMEN_REMOTE_HOST=<box-ip> swift test --filter RemoteFirstLightTests # headless
|
||||
LUMEN_AUTOCONNECT=<box-ip> LUMEN_MODE=1280x720x60 swift run LumenClient # on glass
|
||||
```
|
||||
|
||||
Minimal app around it:
|
||||
## Notes for whoever picks this up next
|
||||
|
||||
```swift
|
||||
@main struct LumenApp: App {
|
||||
var body: some Scene { WindowGroup { ContentView() } }
|
||||
}
|
||||
struct ContentView: View {
|
||||
@State private var conn: LumenConnection?
|
||||
var body: some View {
|
||||
if let conn {
|
||||
StreamView(connection: conn)
|
||||
.onAppear { InputCapture(connection: conn).start() }
|
||||
} else {
|
||||
Button("Connect") {
|
||||
conn = try? LumenConnection(
|
||||
host: "192.168.1.70", width: 2560, height: 1440, refreshHz: 120)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Handoff — what the next agent needs to know
|
||||
|
||||
1. **Expect small compile fixes.** Every Swift file is flagged SCAFFOLD: API-checked from
|
||||
documentation, never run through Xcode. Likely friction: the imported C enum spellings
|
||||
(`LUMEN_STATUS_OK` etc. — cbindgen emits `QualifiedScreamingSnakeCase`), `LumenFrame()`
|
||||
zero-init, `_pad` tuple shape on `LumenInputEvent`.
|
||||
2. **ABI contract** (matches `lumen_core.h` docs): `next_au`'s pointer is valid only until
|
||||
the *next* call on that handle (we copy to `Data` immediately); one pump thread per
|
||||
connection, plus optionally one *separate* audio thread for `next_audio` (independent
|
||||
borrow slots); `send_input` is enqueue-only and thread-safe alongside both; `close`
|
||||
joins the Rust threads — never call it with a `next_au`/`next_audio` call in flight.
|
||||
3. **Decode flow**: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band,
|
||||
and recovery keyframes re-send them — so "wait for the first format description, refresh
|
||||
it on every IDR" (already what `StreamView` does) is sufficient; there is no out-of-band
|
||||
extradata, ever.
|
||||
4. **First-light test**: Linux box runs
|
||||
`PATH=/tmp/gamescope-src/build/src:$PATH LUMEN_COMPOSITOR=gamescope \
|
||||
LUMEN_GAMESCOPE_APP=vkcube LUMEN_ZEROCOPY=1 cargo run -rp lumen-host -- m3-host
|
||||
--source virtual --seconds 120`; Mac connects with the app. Success = the spinning
|
||||
vkcube on glass. Then mouse/keys should appear inside the gamescope session (verify
|
||||
with `LUMEN_GAMESCOPE_APP=xev` and the box-side log `/tmp/lumen-gamescope.log`).
|
||||
5. **Stage 2 (after first light)**: replace `AVSampleBufferDisplayLayer` with explicit
|
||||
`VTDecompressionSession` + `CAMetalLayer` for frame-pacing control (ProMotion/120 Hz),
|
||||
and add glass-to-glass measurement (`tools/latency-probe` is the scaffold; the host
|
||||
already stamps `pts_ns` with its capture wall clock — across machines you'll need a
|
||||
clock-offset estimate from the QUIC RTT, or the probe's visual timestamp loop).
|
||||
6. **Audio**: `nextAudio()` yields raw Opus packets (48 kHz stereo, one 5 ms frame each,
|
||||
1. **cbindgen import quirk** (the predicted "small compile fixes", now fixed): the
|
||||
C17-compatible header spells `LumenStatus`/`LumenInputKind` as integer typedefs while
|
||||
the enum *constants* import into Swift as a distinct same-named type — bridge with
|
||||
`.rawValue` (see the top of `LumenConnection.swift`). Don't fight the generated header.
|
||||
2. **ABI contract**: one video pump thread per connection, plus optionally one *separate*
|
||||
audio drain thread for `nextAudio()`/`nextRumble()` (the core keeps per-plane borrow
|
||||
slots, so the planes never alias); `send()` is enqueue-only and safe alongside all of
|
||||
them. The wrapper's per-plane locks make `close()` safe from anywhere (it waits out
|
||||
in-flight polls, ≤ their timeouts).
|
||||
3. **Decode flow**: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band
|
||||
and recovery keyframes re-send them — "refresh the format description on every IDR"
|
||||
(what `StreamView` does) is sufficient; there is no out-of-band extradata, ever.
|
||||
4. **Stage 2 (next)**: explicit `VTDecompressionSession` + `CAMetalLayer` for frame-pacing
|
||||
control (ProMotion/120 Hz), glass-to-glass measurement via `tools/latency-probe` (the
|
||||
host stamps `pts_ns` with its capture wall clock; across machines you need a clock
|
||||
offset estimate from the QUIC RTT).
|
||||
5. **Audio**: `nextAudio()` yields raw Opus packets (48 kHz stereo, one 5 ms frame each,
|
||||
sequence-numbered). Decode with libopus or `AVAudioConverter`/`kAudioFormatOpus` into an
|
||||
`AVAudioEngine` source node; conceal gaps (drop/dup) rather than blocking — the Rust
|
||||
side buffers 320 ms and drops the newest packet when the puller lags. Wall-clock
|
||||
`ptsNs` shares the host clock with video AUs for A/V sync.
|
||||
7. **Gamepads**: `GCController` → `.gamepadButton(...)`/`.gamepadAxis(...)` events (wire
|
||||
`ptsNs` shares the host clock with video AUs for A/V sync. Wiring this into
|
||||
`LumenClient` is the next app-side task.
|
||||
6. **Gamepads**: `GCController` → `.gamepadButton(...)`/`.gamepadAxis(...)` events (wire
|
||||
contract documented on the constructors; the host accumulates them into a virtual
|
||||
Xbox 360 pad). Poll `nextRumble()` and feed `GCDeviceHaptics` for force feedback.
|
||||
8. **Trust**: connect once with `pinSHA256: nil` (TOFU), persist `hostFingerprint` keyed
|
||||
Client-side capture isn't in `InputCapture` yet.
|
||||
7. **Trust**: connect once with `pinSHA256: nil` (TOFU), persist `hostFingerprint` keyed
|
||||
by host, pass it on every later connect — a mismatch throws `.connectFailed`. The host
|
||||
logs its fingerprint at startup ("clients pin this fingerprint") for out-of-band
|
||||
verification UX; a PIN-style pairing ceremony is a later lumen-core task.
|
||||
verification UX; a PIN-style pairing ceremony is a later lumen-core task. `LumenClient`
|
||||
doesn't persist fingerprints yet — add it alongside the "add host" UX.
|
||||
8. **Input capture caveats** (stage 1): GC handlers only fire while the app has focus —
|
||||
on focus loss `InputCapture` auto-releases everything still held (keys + buttons) so
|
||||
nothing sticks down host-side. Local shortcuts (⌘-anything) still also reach the host;
|
||||
a capture toggle is a small follow-up. One live capture per process (the GC mouse/
|
||||
keyboard singletons have a single handler slot — ownership is tracked so a stale
|
||||
capture's stop() can't clobber a newer one).
|
||||
9. **iOS**: same package (`BUILD_IOS=1` for the xcframework slice); `StreamView` needs the
|
||||
`UIViewRepresentable` twin and touch→input mapping.
|
||||
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
// Connect form ⇄ live stream. Stage-1 UX: pick host + mode, see frames, type/aim.
|
||||
|
||||
import AppKit
|
||||
import LumenKit
|
||||
import SwiftUI
|
||||
|
||||
struct ContentView: View {
|
||||
@StateObject private var model = SessionModel()
|
||||
@AppStorage("lumen.host") private var host = "192.168.1.70"
|
||||
@AppStorage("lumen.port") private var port = 9777
|
||||
@AppStorage("lumen.width") private var width = 1920
|
||||
@AppStorage("lumen.height") private var height = 1080
|
||||
@AppStorage("lumen.hz") private var hz = 60
|
||||
|
||||
var body: some View {
|
||||
Group {
|
||||
if let conn = model.connection {
|
||||
stream(conn)
|
||||
} else {
|
||||
connectForm
|
||||
}
|
||||
}
|
||||
.onAppear { autoConnectIfAsked() }
|
||||
.onDisappear { model.disconnect() } // window closed mid-session (Cmd+N spawns more)
|
||||
}
|
||||
|
||||
/// Development hook: LUMEN_AUTOCONNECT=host[:port] connects immediately at the saved
|
||||
/// (or LUMEN_MODE=WxHxHz) mode — lets scripts drive first-light runs. (IPv4/hostname
|
||||
/// only; an IPv6 literal would need bracket parsing.)
|
||||
private func autoConnectIfAsked() {
|
||||
guard let target = ProcessInfo.processInfo.environment["LUMEN_AUTOCONNECT"],
|
||||
!target.isEmpty, model.connection == nil, !model.connecting
|
||||
else { return }
|
||||
let parts = target.split(separator: ":")
|
||||
host = String(parts[0])
|
||||
if parts.count == 2, let p = Int(parts[1]) { port = p }
|
||||
if let mode = ProcessInfo.processInfo.environment["LUMEN_MODE"] {
|
||||
let dims = mode.split(separator: "x").compactMap { Int($0) }
|
||||
if dims.count == 3 {
|
||||
width = dims[0]
|
||||
height = dims[1]
|
||||
hz = dims[2]
|
||||
}
|
||||
}
|
||||
model.connect(
|
||||
host: host, port: UInt16(clamping: port),
|
||||
width: UInt32(clamping: width), height: UInt32(clamping: height),
|
||||
hz: UInt32(clamping: hz))
|
||||
}
|
||||
|
||||
private func stream(_ conn: LumenConnection) -> some View {
|
||||
StreamView(
|
||||
connection: conn,
|
||||
onFrame: { [meter = model.meter] au in meter.note(byteCount: au.data.count) },
|
||||
onSessionEnd: { [weak model] in
|
||||
Task { @MainActor in model?.sessionEnded() }
|
||||
}
|
||||
)
|
||||
.overlay(alignment: .topTrailing) { hud(conn) }
|
||||
.frame(minWidth: 640, minHeight: 360)
|
||||
.background(Color.black)
|
||||
}
|
||||
|
||||
private func hud(_ conn: LumenConnection) -> some View {
|
||||
VStack(alignment: .trailing, spacing: 4) {
|
||||
Text("\(conn.width)×\(conn.height)@\(conn.refreshHz) \(model.fps) fps \(model.mbps, specifier: "%.1f") Mb/s")
|
||||
.font(.system(.caption, design: .monospaced))
|
||||
Button("Disconnect") { model.disconnect() }
|
||||
.font(.caption)
|
||||
}
|
||||
.padding(8)
|
||||
.background(.black.opacity(0.5), in: RoundedRectangle(cornerRadius: 6))
|
||||
.foregroundStyle(.white)
|
||||
.padding(10)
|
||||
}
|
||||
|
||||
private var connectForm: some View {
|
||||
VStack(spacing: 14) {
|
||||
Text("lumen").font(.largeTitle.weight(.semibold))
|
||||
Form {
|
||||
TextField("Host", text: $host)
|
||||
TextField("Port", value: $port, format: .number.grouping(.never))
|
||||
HStack {
|
||||
TextField("Width", value: $width, format: .number.grouping(.never))
|
||||
Text("×")
|
||||
TextField("Height", value: $height, format: .number.grouping(.never))
|
||||
Text("@")
|
||||
TextField("Hz", value: $hz, format: .number.grouping(.never))
|
||||
}
|
||||
Button("Use this display's mode") { fillFromMainScreen() }
|
||||
.buttonStyle(.link)
|
||||
}
|
||||
.frame(width: 340)
|
||||
|
||||
if let error = model.errorMessage {
|
||||
Text(error)
|
||||
.font(.caption)
|
||||
.foregroundStyle(.red)
|
||||
.frame(width: 340)
|
||||
}
|
||||
|
||||
Button(model.connecting ? "Connecting…" : "Connect") {
|
||||
model.connect(
|
||||
host: host, port: UInt16(clamping: port),
|
||||
width: UInt32(clamping: width), height: UInt32(clamping: height),
|
||||
hz: UInt32(clamping: hz))
|
||||
}
|
||||
.keyboardShortcut(.defaultAction)
|
||||
.disabled(model.connecting || host.isEmpty)
|
||||
}
|
||||
.padding(28)
|
||||
.frame(minWidth: 420, minHeight: 320)
|
||||
}
|
||||
|
||||
private func fillFromMainScreen() {
|
||||
guard let screen = NSScreen.main else { return }
|
||||
let scale = screen.backingScaleFactor
|
||||
width = Int(screen.frame.width * scale)
|
||||
height = Int(screen.frame.height * scale)
|
||||
hz = screen.maximumFramesPerSecond
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
// LumenClient — development app shell around LumenKit (swift run LumenClient).
|
||||
// Connect form → StreamView (AVSampleBufferDisplayLayer HEVC) + InputCapture.
|
||||
|
||||
import AppKit
|
||||
import SwiftUI
|
||||
|
||||
@main
|
||||
struct LumenClientApp: App {
|
||||
@NSApplicationDelegateAdaptor(AppDelegate.self) private var appDelegate
|
||||
|
||||
var body: some Scene {
|
||||
WindowGroup("lumen") {
|
||||
ContentView()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
final class AppDelegate: NSObject, NSApplicationDelegate {
|
||||
func applicationDidFinishLaunching(_ notification: Notification) {
|
||||
// `swift run` launches an unbundled binary; promote it to a regular app so the
|
||||
// window fronts and receives keyboard/mouse focus (GameController needs focus).
|
||||
NSApp.setActivationPolicy(.regular)
|
||||
NSApp.activate(ignoringOtherApps: true)
|
||||
}
|
||||
|
||||
func applicationShouldTerminateAfterLastWindowClosed(_ sender: NSApplication) -> Bool {
|
||||
true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
// Session state for the app shell: owns the connection, the input capture, and the
|
||||
// pump-thread → main-actor stats relay.
|
||||
|
||||
import Foundation
|
||||
import LumenKit
|
||||
import SwiftUI
|
||||
|
||||
/// Pump-thread-side frame counters; a 1 Hz main-actor timer drains them into @Published
|
||||
/// values. NSLock instead of an actor — the writer is the (non-async) pump thread.
|
||||
final class FrameMeter: @unchecked Sendable {
|
||||
private let lock = NSLock()
|
||||
private var frames = 0
|
||||
private var bytes = 0
|
||||
private var totalFrames = 0
|
||||
|
||||
func note(byteCount: Int) {
|
||||
lock.lock()
|
||||
frames += 1
|
||||
bytes += byteCount
|
||||
totalFrames += 1
|
||||
lock.unlock()
|
||||
}
|
||||
|
||||
/// Returns and resets the per-interval counters (the running total stays).
|
||||
func drain() -> (frames: Int, bytes: Int, total: Int) {
|
||||
lock.lock()
|
||||
defer {
|
||||
frames = 0
|
||||
bytes = 0
|
||||
lock.unlock()
|
||||
}
|
||||
return (frames, bytes, totalFrames)
|
||||
}
|
||||
}
|
||||
|
||||
@MainActor
|
||||
final class SessionModel: ObservableObject {
|
||||
@Published var connection: LumenConnection?
|
||||
@Published var connecting = false
|
||||
@Published var errorMessage: String?
|
||||
@Published var fps = 0
|
||||
@Published var mbps = 0.0
|
||||
@Published var totalFrames = 0
|
||||
|
||||
let meter = FrameMeter()
|
||||
private var inputCapture: InputCapture?
|
||||
private var statsTimer: Timer?
|
||||
|
||||
func connect(host: String, port: UInt16, width: UInt32, height: UInt32, hz: UInt32) {
|
||||
guard !connecting else { return }
|
||||
connecting = true
|
||||
errorMessage = nil
|
||||
Task.detached(priority: .userInitiated) {
|
||||
// LumenConnection.init blocks on the QUIC handshake — keep it off the main actor.
|
||||
let result = Result { try LumenConnection(
|
||||
host: host, port: port, width: width, height: height, refreshHz: hz) }
|
||||
await MainActor.run { [weak self] in
|
||||
guard let self else { return }
|
||||
self.connecting = false
|
||||
switch result {
|
||||
case .success(let conn):
|
||||
self.connection = conn
|
||||
self.startInput(conn)
|
||||
self.startStatsTimer()
|
||||
case .failure:
|
||||
self.errorMessage = "Connection failed — is the host running? " +
|
||||
"(lumen-host m3-host on \(host):\(port))"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func disconnect() {
|
||||
inputCapture?.stop()
|
||||
inputCapture = nil
|
||||
statsTimer?.invalidate()
|
||||
statsTimer = nil
|
||||
if let conn = connection {
|
||||
// close() waits out an in-flight poll (≤100 ms) and joins the Rust worker
|
||||
// threads — keep that off the main actor.
|
||||
Task.detached { conn.close() }
|
||||
}
|
||||
connection = nil
|
||||
fps = 0
|
||||
mbps = 0
|
||||
}
|
||||
|
||||
/// Called (via the main actor) when the pump hits end-of-session.
|
||||
func sessionEnded() {
|
||||
guard connection != nil else { return }
|
||||
disconnect()
|
||||
errorMessage = "Session ended by host."
|
||||
}
|
||||
|
||||
private func startInput(_ conn: LumenConnection) {
|
||||
let capture = InputCapture(connection: conn)
|
||||
capture.start()
|
||||
inputCapture = capture
|
||||
}
|
||||
|
||||
private func startStatsTimer() {
|
||||
let timer = Timer(timeInterval: 1.0, repeats: true) { [weak self] _ in
|
||||
guard let self else { return }
|
||||
Task { @MainActor in
|
||||
let (frames, bytes, total) = self.meter.drain()
|
||||
self.fps = frames
|
||||
self.mbps = Double(bytes) * 8 / 1_000_000
|
||||
self.totalFrames = total
|
||||
}
|
||||
}
|
||||
// .common so the HUD keeps updating during window drags / menu tracking.
|
||||
RunLoop.main.add(timer, forMode: .common)
|
||||
statsTimer = timer
|
||||
}
|
||||
}
|
||||
@@ -12,6 +12,9 @@ import Foundation
|
||||
|
||||
public enum AnnexB {
|
||||
/// Split an Annex-B stream into NAL units (start codes 00 00 01 / 00 00 00 01 stripped).
|
||||
/// All zeros immediately preceding a start code are dropped: they're either the
|
||||
/// 4-byte-code prefix or `trailing_zero_8bits` padding, never NAL payload (emulation
|
||||
/// prevention keeps 00 00 0x out of conforming NAL bytes) — same policy as ffmpeg.
|
||||
public static func nalUnits(in data: Data) -> [Data] {
|
||||
var nals: [Data] = []
|
||||
let bytes = [UInt8](data)
|
||||
@@ -19,8 +22,11 @@ public enum AnnexB {
|
||||
var start = -1
|
||||
while i + 2 < bytes.count {
|
||||
if bytes[i] == 0, bytes[i + 1] == 0, bytes[i + 2] == 1 {
|
||||
let codeStart = (i > 0 && bytes[i - 1] == 0) ? i - 1 : i
|
||||
if start >= 0 {
|
||||
var codeStart = i
|
||||
while codeStart > 0, bytes[codeStart - 1] == 0 {
|
||||
codeStart -= 1
|
||||
}
|
||||
if start >= 0, start < codeStart {
|
||||
nals.append(Data(bytes[start..<codeStart]))
|
||||
}
|
||||
start = i + 3
|
||||
|
||||
@@ -4,26 +4,50 @@
|
||||
// injector expects for relative motion. GCKeyboard gives HID keycodes which we map to the
|
||||
// Windows VK space the host's vk_to_evdev table consumes (same space Moonlight uses).
|
||||
// Gamepads (GCController) come later — the host's uinput pads already speak the
|
||||
// GamepadButton/GamepadAxis event kinds.
|
||||
// GamepadButton/GamepadAxis event kinds, but m3's injector path doesn't route them yet.
|
||||
//
|
||||
// SCAFFOLD: written on the Linux host, not yet compiled against Xcode. The VK map covers
|
||||
// the common keys; extend alongside lumen-host/src/inject.rs::vk_to_evdev.
|
||||
// The wire carries integer deltas; GC hands us Floats. We accumulate the fractional
|
||||
// remainder per axis so slow, sub-pixel motion isn't truncated away.
|
||||
//
|
||||
// GC only delivers while the app is active, so anything held when focus leaves would
|
||||
// stick down on the host forever — we track pressed keys/buttons and release them all on
|
||||
// didResignActive and on stop(). All GC handlers and notifications fire on the main
|
||||
// queue (the framework default), so the mutable state here needs no locking.
|
||||
//
|
||||
// GCMouse.current/GCKeyboard.coalesced are process-global singletons with one handler
|
||||
// slot each: only one InputCapture can be live per process. `activeCapture` tracks
|
||||
// ownership so a stale capture's stop() can't clobber a newer one's handlers.
|
||||
|
||||
#if os(macOS)
|
||||
import AppKit
|
||||
import Foundation
|
||||
import GameController
|
||||
import LumenCore
|
||||
|
||||
public final class InputCapture {
|
||||
private static weak var activeCapture: InputCapture?
|
||||
|
||||
private let connection: LumenConnection
|
||||
private var observers: [NSObjectProtocol] = []
|
||||
private var mice: [GCMouse] = []
|
||||
private var keyboards: [GCKeyboard] = []
|
||||
|
||||
// Main-queue-only state (see header comment).
|
||||
private var residualX: Float = 0
|
||||
private var residualY: Float = 0
|
||||
private var residualScrollX: Float = 0
|
||||
private var residualScrollY: Float = 0
|
||||
private var pressedVKs: Set<UInt32> = []
|
||||
private var pressedButtons: Set<UInt32> = []
|
||||
|
||||
public init(connection: LumenConnection) {
|
||||
self.connection = connection
|
||||
}
|
||||
|
||||
/// Begin forwarding the current (and future) mouse/keyboard to the host.
|
||||
/// Begin forwarding the current (and future) mouse/keyboard to the host. Steals the
|
||||
/// global GC handler slots from any previous capture (one live capture per process).
|
||||
public func start() {
|
||||
Self.activeCapture = self
|
||||
if let mouse = GCMouse.current { attach(mouse: mouse) }
|
||||
if let keyboard = GCKeyboard.coalesced { attach(keyboard: keyboard) }
|
||||
observers.append(NotificationCenter.default.addObserver(
|
||||
@@ -36,44 +60,130 @@ public final class InputCapture {
|
||||
) { [weak self] n in
|
||||
if let k = n.object as? GCKeyboard { self?.attach(keyboard: k) }
|
||||
})
|
||||
// Focus loss: GC stops delivering, so release everything still held host-side.
|
||||
observers.append(NotificationCenter.default.addObserver(
|
||||
forName: NSApplication.didResignActiveNotification, object: nil, queue: .main
|
||||
) { [weak self] _ in
|
||||
self?.releaseAll()
|
||||
})
|
||||
}
|
||||
|
||||
public func stop() {
|
||||
releaseAll()
|
||||
observers.forEach(NotificationCenter.default.removeObserver(_:))
|
||||
observers.removeAll()
|
||||
// Don't clobber the handlers if a newer capture has taken the global devices.
|
||||
if Self.activeCapture === self || Self.activeCapture == nil {
|
||||
for mouse in mice {
|
||||
guard let input = mouse.mouseInput else { continue }
|
||||
input.mouseMovedHandler = nil
|
||||
input.leftButton.pressedChangedHandler = nil
|
||||
input.rightButton?.pressedChangedHandler = nil
|
||||
input.middleButton?.pressedChangedHandler = nil
|
||||
input.auxiliaryButtons?.forEach { $0.pressedChangedHandler = nil }
|
||||
input.scroll.valueChangedHandler = nil
|
||||
}
|
||||
for keyboard in keyboards {
|
||||
keyboard.keyboardInput?.keyChangedHandler = nil
|
||||
}
|
||||
Self.activeCapture = nil
|
||||
}
|
||||
mice.removeAll()
|
||||
keyboards.removeAll()
|
||||
}
|
||||
|
||||
deinit { stop() }
|
||||
|
||||
/// Send release events for everything currently held, and drop the motion residuals.
|
||||
private func releaseAll() {
|
||||
for vk in pressedVKs {
|
||||
connection.send(.key(vk, down: false))
|
||||
}
|
||||
for button in pressedButtons {
|
||||
connection.send(.mouseButton(button, down: false))
|
||||
}
|
||||
pressedVKs.removeAll()
|
||||
pressedButtons.removeAll()
|
||||
residualX = 0
|
||||
residualY = 0
|
||||
residualScrollX = 0
|
||||
residualScrollY = 0
|
||||
}
|
||||
|
||||
private func sendButton(_ button: UInt32, pressed: Bool) {
|
||||
if pressed {
|
||||
pressedButtons.insert(button)
|
||||
} else {
|
||||
pressedButtons.remove(button)
|
||||
}
|
||||
connection.send(.mouseButton(button, down: pressed))
|
||||
}
|
||||
|
||||
private func attach(mouse: GCMouse) {
|
||||
guard let input = mouse.mouseInput else { return }
|
||||
let conn = connection
|
||||
input.mouseMovedHandler = { _, dx, dy in
|
||||
guard let input = mouse.mouseInput,
|
||||
!mice.contains(where: { $0 === mouse }) // re-delivered on wake — attach once
|
||||
else { return }
|
||||
mice.append(mouse)
|
||||
input.mouseMovedHandler = { [weak self] _, dx, dy in
|
||||
guard let self else { return }
|
||||
// GC gives +y up; the host expects screen-space (+y down).
|
||||
conn.send(.mouseMove(dx: Int32(dx), dy: Int32(-dy)))
|
||||
let fx = dx + self.residualX
|
||||
let fy = -dy + self.residualY
|
||||
let ix = fx.rounded(.towardZero)
|
||||
let iy = fy.rounded(.towardZero)
|
||||
self.residualX = fx - ix
|
||||
self.residualY = fy - iy
|
||||
if ix != 0 || iy != 0 {
|
||||
self.connection.send(.mouseMove(dx: Int32(ix), dy: Int32(iy)))
|
||||
}
|
||||
}
|
||||
input.leftButton.pressedChangedHandler = { _, _, pressed in
|
||||
conn.send(.mouseButton(1, down: pressed))
|
||||
input.leftButton.pressedChangedHandler = { [weak self] _, _, pressed in
|
||||
self?.sendButton(1, pressed: pressed)
|
||||
}
|
||||
input.rightButton?.pressedChangedHandler = { _, _, pressed in
|
||||
conn.send(.mouseButton(3, down: pressed))
|
||||
input.rightButton?.pressedChangedHandler = { [weak self] _, _, pressed in
|
||||
self?.sendButton(3, pressed: pressed)
|
||||
}
|
||||
input.middleButton?.pressedChangedHandler = { _, _, pressed in
|
||||
conn.send(.mouseButton(2, down: pressed))
|
||||
input.middleButton?.pressedChangedHandler = { [weak self] _, _, pressed in
|
||||
self?.sendButton(2, pressed: pressed)
|
||||
}
|
||||
input.scroll.valueChangedHandler = { _, _, dy in
|
||||
if dy != 0 { conn.send(.scroll(Int32(dy * 120))) }
|
||||
// First two side buttons → GameStream X1/X2.
|
||||
if let aux = input.auxiliaryButtons {
|
||||
for (i, button) in aux.prefix(2).enumerated() {
|
||||
button.pressedChangedHandler = { [weak self] _, _, pressed in
|
||||
self?.sendButton(UInt32(4 + i), pressed: pressed)
|
||||
}
|
||||
}
|
||||
}
|
||||
input.scroll.valueChangedHandler = { [weak self] _, x, y in
|
||||
guard let self else { return }
|
||||
// WHEEL_DELTA(120) per notch; positive = up / right (Moonlight's convention).
|
||||
let fy = y * 120 + self.residualScrollY
|
||||
let fx = x * 120 + self.residualScrollX
|
||||
let iy = fy.rounded(.towardZero)
|
||||
let ix = fx.rounded(.towardZero)
|
||||
self.residualScrollY = fy - iy
|
||||
self.residualScrollX = fx - ix
|
||||
if iy != 0 { self.connection.send(.scroll(Int32(iy))) }
|
||||
if ix != 0 { self.connection.send(.scroll(Int32(ix), horizontal: true)) }
|
||||
}
|
||||
}
|
||||
|
||||
private func attach(keyboard: GCKeyboard) {
|
||||
let conn = connection
|
||||
keyboard.keyboardInput?.keyChangedHandler = { _, _, keyCode, pressed in
|
||||
if let vk = Self.hidToVK[keyCode.rawValue] {
|
||||
conn.send(.key(vk, down: pressed))
|
||||
guard !keyboards.contains(where: { $0 === keyboard }) else { return }
|
||||
keyboards.append(keyboard)
|
||||
keyboard.keyboardInput?.keyChangedHandler = { [weak self] _, _, keyCode, pressed in
|
||||
guard let self, let vk = Self.hidToVK[keyCode.rawValue] else { return }
|
||||
if pressed {
|
||||
self.pressedVKs.insert(vk)
|
||||
} else {
|
||||
self.pressedVKs.remove(vk)
|
||||
}
|
||||
self.connection.send(.key(vk, down: pressed))
|
||||
}
|
||||
}
|
||||
|
||||
/// HID usage (GCKeyCode raw) → Windows VK (the host maps VK → evdev).
|
||||
/// HID usage (GCKeyCode raw) → Windows VK (the host maps VK → evdev; every VK emitted
|
||||
/// here exists in lumen-host/src/inject.rs::vk_to_evdev — extend the two together).
|
||||
static let hidToVK: [Int: UInt32] = {
|
||||
var m: [Int: UInt32] = [:]
|
||||
// a–z: HID 0x04..0x1D → VK 'A'..'Z'.
|
||||
@@ -90,11 +200,23 @@ public final class InputCapture {
|
||||
m[0x2F] = 0xDB; m[0x30] = 0xDD; m[0x31] = 0xDC // [ ] backslash
|
||||
m[0x33] = 0xBA; m[0x34] = 0xDE; m[0x35] = 0xC0 // ; ' `
|
||||
m[0x36] = 0xBC; m[0x37] = 0xBE; m[0x38] = 0xBF // , . /
|
||||
m[0x39] = 0x14 // caps lock
|
||||
// F1..F12: HID 0x3A..0x45 → VK 0x70..0x7B.
|
||||
for i in 0..<12 { m[0x3A + i] = UInt32(0x70 + i) }
|
||||
m[0x46] = 0x2C; m[0x47] = 0x91; m[0x48] = 0x13 // printscreen scrolllock pause
|
||||
m[0x4F] = 0x27; m[0x50] = 0x25; m[0x51] = 0x28; m[0x52] = 0x26 // arrows R L D U
|
||||
m[0x49] = 0x2D; m[0x4A] = 0x24; m[0x4B] = 0x21 // insert home pageup
|
||||
m[0x4C] = 0x2E; m[0x4D] = 0x23; m[0x4E] = 0x22 // delete end pagedown
|
||||
// Keypad: NumLock, / * - +, Enter, 1..9, 0, decimal. KP Enter goes as
|
||||
// VK_SEPARATOR (0x6C) — this host maps it to KEY_KPENTER (Windows itself would
|
||||
// send VK_RETURN+extended, which vk_to_evdev can't distinguish).
|
||||
m[0x53] = 0x90
|
||||
m[0x54] = 0x6F; m[0x55] = 0x6A; m[0x56] = 0x6D; m[0x57] = 0x6B
|
||||
m[0x58] = 0x6C
|
||||
for i in 0..<9 { m[0x59 + i] = UInt32(0x61 + i) }
|
||||
m[0x62] = 0x60; m[0x63] = 0x6E
|
||||
m[0x64] = 0xE2 // ISO 102nd key (<> next to left shift on ISO layouts)
|
||||
m[0x65] = 0x5D // menu/application
|
||||
m[0xE0] = 0xA2; m[0xE1] = 0xA0; m[0xE2] = 0xA4; m[0xE3] = 0x5B // Lctrl Lshift Lalt Lcmd
|
||||
m[0xE4] = 0xA3; m[0xE5] = 0xA1; m[0xE6] = 0xA5; m[0xE7] = 0x5C // Rctrl Rshift Ralt Rcmd
|
||||
return m
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
// Swift wrapper around the lumen-core C ABI's lumen/1 connection API.
|
||||
//
|
||||
// Threading contract (mirrors the C header): one LumenConnection is used from a single
|
||||
// pump thread for nextAU(); nextAudio() may run on its own (single) audio thread;
|
||||
// sendInput() is enqueue-only and safe alongside both. The pointers inside an AU/audio
|
||||
// Threading contract (mirrors the C header): one LumenConnection is pumped from a single
|
||||
// video thread via nextAU(); nextAudio()/nextRumble() may each run on their own (single)
|
||||
// drain thread — the core keeps per-plane borrow slots, so the planes never alias;
|
||||
// send() is enqueue-only and safe alongside all of them. The pointers inside an AU/audio
|
||||
// packet are only valid until the next call of the same kind, so we copy into Data here —
|
||||
// the copies are small and keep the Swift side memory-safe.
|
||||
//
|
||||
@@ -10,12 +11,22 @@
|
||||
// `hostFingerprint` reports what a trust-on-first-use connect observed — persist it, e.g.
|
||||
// in UserDefaults keyed by host, and pin it from then on).
|
||||
//
|
||||
// SCAFFOLD: written on the Linux host, not yet compiled against Xcode — expect to fix
|
||||
// trivial issues on first build (see README.md "Handoff").
|
||||
// close() is safe from any thread: it flags the pullers to exit at their next poll
|
||||
// boundary, then takes the per-plane locks (each held across its blocking C poll), so the
|
||||
// handle is never freed under an in-flight call — the C contract ("never close with a
|
||||
// next_au/next_audio call in flight") is enforced here rather than left to callers. After
|
||||
// close, the pull methods throw `.closed` and the threads unwind on their own.
|
||||
|
||||
import Foundation
|
||||
import LumenCore
|
||||
|
||||
// cbindgen's C17-compatible header spells the typedefs as plain integers
|
||||
// (`typedef int32_t LumenStatus`, `typedef uint8_t LumenInputKind`) while the enum
|
||||
// constants import as a distinct same-named Swift type — bridge by raw value once here.
|
||||
private let statusOK: Int32 = LUMEN_STATUS_OK.rawValue
|
||||
private let statusNoFrame: Int32 = LUMEN_STATUS_NO_FRAME.rawValue
|
||||
private let statusClosed: Int32 = LUMEN_STATUS_CLOSED.rawValue
|
||||
|
||||
/// One reassembled, FEC-recovered, decrypted access unit (Annex-B HEVC from the host).
|
||||
public struct AccessUnit: Sendable {
|
||||
public let data: Data
|
||||
@@ -39,10 +50,22 @@ public enum LumenClientError: Error {
|
||||
/// unpinned when the caller asked for verification would be a silent trust downgrade.
|
||||
case invalidPin
|
||||
case closed
|
||||
case status(Int32)
|
||||
}
|
||||
|
||||
public final class LumenConnection {
|
||||
private var handle: OpaquePointer?
|
||||
/// Set by close() before it contends for the plane locks: the pullers see it at their
|
||||
/// next poll boundary and exit, so close() can't be starved by back-to-back polls
|
||||
/// (NSLock is not fair).
|
||||
private var closeRequested = false
|
||||
/// Serializes send()/close() against each other and guards `handle`/`closeRequested`.
|
||||
private let abiLock = NSLock()
|
||||
/// Held across the blocking next_au call; close() takes it (same plane-lock → abiLock
|
||||
/// order as the pullers) so it can never free the handle under an in-flight poll.
|
||||
private let pumpLock = NSLock()
|
||||
/// Same role for the audio/rumble drain thread (its own plane in the core).
|
||||
private let audioLock = NSLock()
|
||||
|
||||
/// Negotiated session mode (host-confirmed).
|
||||
public private(set) var width: UInt32 = 0
|
||||
@@ -86,87 +109,141 @@ public final class LumenConnection {
|
||||
self.refreshHz = hz
|
||||
}
|
||||
|
||||
/// Pull the next access unit; nil on timeout, throws once the session is closed.
|
||||
/// Pull the next access unit; nil on timeout, throws `.closed` once the session ended.
|
||||
/// Call from a single pump thread.
|
||||
public func nextAU(timeoutMs: UInt32 = 100) throws -> AccessUnit? {
|
||||
pumpLock.lock()
|
||||
defer { pumpLock.unlock() }
|
||||
guard let h = liveHandle() else { throw LumenClientError.closed }
|
||||
|
||||
var frame = LumenFrame()
|
||||
switch lumen_connection_next_au(handle, &frame, timeoutMs) {
|
||||
case LUMEN_STATUS_OK:
|
||||
let data = Data(bytes: frame.data, count: frame.len) // copy: ptr valid only until next call
|
||||
let rc = lumen_connection_next_au(h, &frame, timeoutMs)
|
||||
switch rc {
|
||||
case statusOK:
|
||||
guard let base = frame.data, frame.len > 0 else { return nil }
|
||||
let data = Data(bytes: base, count: Int(frame.len)) // copy: ptr valid only until next call
|
||||
return AccessUnit(
|
||||
data: data, ptsNs: frame.pts_ns,
|
||||
frameIndex: frame.frame_index, flags: frame.flags)
|
||||
case LUMEN_STATUS_NO_FRAME:
|
||||
case statusNoFrame:
|
||||
return nil
|
||||
case LUMEN_STATUS_CLOSED:
|
||||
case statusClosed:
|
||||
throw LumenClientError.closed
|
||||
default:
|
||||
throw LumenClientError.closed
|
||||
throw LumenClientError.status(rc)
|
||||
}
|
||||
}
|
||||
|
||||
/// Pull the next Opus audio packet; nil on timeout, throws once the session is closed.
|
||||
/// Drain from a dedicated audio thread — packets arrive every 5 ms (320 ms buffered).
|
||||
/// Pull the next Opus audio packet; nil on timeout, throws `.closed` once the session
|
||||
/// ended. Drain from a dedicated audio thread — packets arrive every 5 ms (the core
|
||||
/// buffers 320 ms and drops the newest when the puller lags).
|
||||
public func nextAudio(timeoutMs: UInt32 = 100) throws -> AudioPacket? {
|
||||
audioLock.lock()
|
||||
defer { audioLock.unlock() }
|
||||
guard let h = liveHandle() else { throw LumenClientError.closed }
|
||||
|
||||
var pkt = LumenAudioPacket()
|
||||
switch lumen_connection_next_audio(handle, &pkt, timeoutMs) {
|
||||
case LUMEN_STATUS_OK:
|
||||
let data = Data(bytes: pkt.data, count: pkt.len) // copy: ptr valid only until next call
|
||||
let rc = lumen_connection_next_audio(h, &pkt, timeoutMs)
|
||||
switch rc {
|
||||
case statusOK:
|
||||
guard let base = pkt.data, pkt.len > 0 else { return nil }
|
||||
let data = Data(bytes: base, count: Int(pkt.len)) // copy: ptr valid only until next call
|
||||
return AudioPacket(data: data, ptsNs: pkt.pts_ns, seq: pkt.seq)
|
||||
case LUMEN_STATUS_NO_FRAME:
|
||||
case statusNoFrame:
|
||||
return nil
|
||||
default:
|
||||
case statusClosed:
|
||||
throw LumenClientError.closed
|
||||
default:
|
||||
throw LumenClientError.status(rc)
|
||||
}
|
||||
}
|
||||
|
||||
/// Pull the next force-feedback update for the GCController haptics engine:
|
||||
/// `(pad, lowFrequency, highFrequency)` with 0...0xFFFF amplitudes, (0, 0) = stop.
|
||||
public func nextRumble(timeoutMs: UInt32 = 100) throws -> (pad: UInt16, low: UInt16, high: UInt16)? {
|
||||
/// Shares the audio drain thread's plane (call from that thread).
|
||||
public func nextRumble(timeoutMs: UInt32 = 0) throws -> (pad: UInt16, low: UInt16, high: UInt16)? {
|
||||
audioLock.lock()
|
||||
defer { audioLock.unlock() }
|
||||
guard let h = liveHandle() else { throw LumenClientError.closed }
|
||||
|
||||
var pad: UInt16 = 0, low: UInt16 = 0, high: UInt16 = 0
|
||||
switch lumen_connection_next_rumble(handle, &pad, &low, &high, timeoutMs) {
|
||||
case LUMEN_STATUS_OK:
|
||||
let rc = lumen_connection_next_rumble(h, &pad, &low, &high, timeoutMs)
|
||||
switch rc {
|
||||
case statusOK:
|
||||
return (pad, low, high)
|
||||
case LUMEN_STATUS_NO_FRAME:
|
||||
case statusNoFrame:
|
||||
return nil
|
||||
default:
|
||||
case statusClosed:
|
||||
throw LumenClientError.closed
|
||||
default:
|
||||
throw LumenClientError.status(rc)
|
||||
}
|
||||
}
|
||||
|
||||
/// Send one input event (delivered to the host as a QUIC datagram).
|
||||
/// Send one input event (delivered to the host as a QUIC datagram). Thread-safe;
|
||||
/// silently dropped after close.
|
||||
public func send(_ event: LumenInputEvent) {
|
||||
var ev = event
|
||||
_ = lumen_connection_send_input(handle, &ev)
|
||||
abiLock.lock()
|
||||
defer { abiLock.unlock() }
|
||||
guard let h = handle, !closeRequested else { return }
|
||||
_ = lumen_connection_send_input(h, &ev)
|
||||
}
|
||||
|
||||
/// Close the connection and free the handle. Safe from any thread, idempotent; waits
|
||||
/// for in-flight pulls (≤ their timeouts) before tearing down.
|
||||
public func close() {
|
||||
if let h = handle {
|
||||
lumen_connection_close(h)
|
||||
handle = nil
|
||||
abiLock.lock()
|
||||
closeRequested = true
|
||||
abiLock.unlock()
|
||||
pumpLock.lock() // pullers exit at their next poll boundary, releasing these
|
||||
audioLock.lock()
|
||||
abiLock.lock()
|
||||
let h = handle
|
||||
handle = nil
|
||||
abiLock.unlock()
|
||||
audioLock.unlock()
|
||||
pumpLock.unlock()
|
||||
if let h {
|
||||
lumen_connection_close(h) // joins the connection's internal Rust threads
|
||||
}
|
||||
}
|
||||
|
||||
deinit { close() }
|
||||
|
||||
/// Snapshot the handle unless close is pending (callers hold their plane lock).
|
||||
private func liveHandle() -> OpaquePointer? {
|
||||
abiLock.lock()
|
||||
defer { abiLock.unlock() }
|
||||
return closeRequested ? nil : handle
|
||||
}
|
||||
}
|
||||
|
||||
// Convenience constructors for the wire input events (field semantics match
|
||||
// lumen_core::input::InputEvent; see lumen_core.h).
|
||||
public extension LumenInputEvent {
|
||||
private static func make(
|
||||
_ kind: UInt32, code: UInt32, x: Int32, y: Int32, flags: UInt32 = 0
|
||||
) -> LumenInputEvent {
|
||||
LumenInputEvent(kind: UInt8(kind), _pad: (0, 0, 0), code: code, x: x, y: y, flags: flags)
|
||||
}
|
||||
static func mouseMove(dx: Int32, dy: Int32) -> LumenInputEvent {
|
||||
LumenInputEvent(kind: LUMEN_INPUT_KIND_MOUSE_MOVE, _pad: (0, 0, 0), code: 0, x: dx, y: dy, flags: 0)
|
||||
make(LUMEN_INPUT_KIND_MOUSE_MOVE.rawValue, code: 0, x: dx, y: dy)
|
||||
}
|
||||
/// GameStream button ids: 1=left 2=middle 3=right 4=X1 5=X2 (host maps to evdev BTN_*).
|
||||
static func mouseButton(_ button: UInt32, down: Bool) -> LumenInputEvent {
|
||||
LumenInputEvent(
|
||||
kind: down ? LUMEN_INPUT_KIND_MOUSE_BUTTON_DOWN : LUMEN_INPUT_KIND_MOUSE_BUTTON_UP,
|
||||
_pad: (0, 0, 0), code: button, x: 0, y: 0, flags: 0)
|
||||
make(
|
||||
(down ? LUMEN_INPUT_KIND_MOUSE_BUTTON_DOWN : LUMEN_INPUT_KIND_MOUSE_BUTTON_UP).rawValue,
|
||||
code: button, x: 0, y: 0)
|
||||
}
|
||||
/// `vk` is a Windows virtual-key code (the host's vk_to_evdev table consumes these).
|
||||
static func key(_ vk: UInt32, down: Bool) -> LumenInputEvent {
|
||||
LumenInputEvent(
|
||||
kind: down ? LUMEN_INPUT_KIND_KEY_DOWN : LUMEN_INPUT_KIND_KEY_UP,
|
||||
_pad: (0, 0, 0), code: vk, x: 0, y: 0, flags: 0)
|
||||
make((down ? LUMEN_INPUT_KIND_KEY_DOWN : LUMEN_INPUT_KIND_KEY_UP).rawValue, code: vk, x: 0, y: 0)
|
||||
}
|
||||
static func scroll(_ delta: Int32) -> LumenInputEvent {
|
||||
LumenInputEvent(kind: LUMEN_INPUT_KIND_MOUSE_SCROLL, _pad: (0, 0, 0), code: 0, x: delta, y: 0, flags: 0)
|
||||
/// WHEEL_DELTA(120)-scaled; positive = up (vertical) / right (horizontal) — the
|
||||
/// convention Moonlight/SDL use; the host maps onto the ei/wl axes.
|
||||
static func scroll(_ delta: Int32, horizontal: Bool = false) -> LumenInputEvent {
|
||||
make(LUMEN_INPUT_KIND_MOUSE_SCROLL.rawValue, code: horizontal ? 1 : 0, x: delta, y: 0)
|
||||
}
|
||||
|
||||
// Gamepad (wire contract in lumen_core::input::gamepad): one transition per event,
|
||||
@@ -175,16 +252,14 @@ public extension LumenInputEvent {
|
||||
/// `button` is a GameStream buttonFlags bit (A=0x1000 B=0x2000 X=0x4000 Y=0x8000,
|
||||
/// dpad=0x1/2/4/8, start=0x10 back=0x20 LS=0x40 RS=0x80 LB=0x100 RB=0x200 guide=0x400).
|
||||
static func gamepadButton(_ button: UInt32, down: Bool, pad: UInt32 = 0) -> LumenInputEvent {
|
||||
LumenInputEvent(
|
||||
kind: LUMEN_INPUT_KIND_GAMEPAD_BUTTON,
|
||||
_pad: (0, 0, 0), code: button, x: down ? 1 : 0, y: 0, flags: pad)
|
||||
make(
|
||||
LUMEN_INPUT_KIND_GAMEPAD_BUTTON.rawValue,
|
||||
code: button, x: down ? 1 : 0, y: 0, flags: pad)
|
||||
}
|
||||
|
||||
/// Axis ids: 0=LSX 1=LSY 2=RSX 3=RSY (−32768...32767, XInput convention: +y = UP —
|
||||
/// `GCControllerDirectionPad.yAxis` already matches, no flip), 4=LT 5=RT (0...255).
|
||||
static func gamepadAxis(_ axis: UInt32, value: Int32, pad: UInt32 = 0) -> LumenInputEvent {
|
||||
LumenInputEvent(
|
||||
kind: LUMEN_INPUT_KIND_GAMEPAD_AXIS,
|
||||
_pad: (0, 0, 0), code: axis, x: value, y: 0, flags: pad)
|
||||
make(LUMEN_INPUT_KIND_GAMEPAD_AXIS.rawValue, code: axis, x: value, y: 0, flags: pad)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,8 +5,8 @@
|
||||
// zero-copy on Apple silicon. Stage 2 (explicit VTDecompressionSession + CAMetalLayer)
|
||||
// replaces this when we start tuning frame pacing / measuring glass-to-glass.
|
||||
//
|
||||
// SCAFFOLD: written on the Linux host, not yet compiled against Xcode. macOS-first
|
||||
// (NSViewRepresentable); the iOS variant is the same layer under UIViewRepresentable.
|
||||
// macOS-first (NSViewRepresentable); the iOS variant is the same layer under
|
||||
// UIViewRepresentable.
|
||||
|
||||
#if os(macOS)
|
||||
import AVFoundation
|
||||
@@ -14,70 +14,130 @@ import SwiftUI
|
||||
|
||||
public struct StreamView: NSViewRepresentable {
|
||||
private let connection: LumenConnection
|
||||
private let onFrame: (@Sendable (AccessUnit) -> Void)?
|
||||
private let onSessionEnd: (@Sendable () -> Void)?
|
||||
|
||||
public init(connection: LumenConnection) {
|
||||
/// `onFrame`/`onSessionEnd` fire on the pump thread — hop to the main actor for UI.
|
||||
public init(
|
||||
connection: LumenConnection,
|
||||
onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
|
||||
onSessionEnd: (@Sendable () -> Void)? = nil
|
||||
) {
|
||||
self.connection = connection
|
||||
self.onFrame = onFrame
|
||||
self.onSessionEnd = onSessionEnd
|
||||
}
|
||||
|
||||
public func makeNSView(context: Context) -> StreamLayerView {
|
||||
let view = StreamLayerView()
|
||||
view.start(connection: connection)
|
||||
view.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
|
||||
return view
|
||||
}
|
||||
|
||||
public func updateNSView(_ view: StreamLayerView, context: Context) {}
|
||||
public func updateNSView(_ view: StreamLayerView, context: Context) {
|
||||
// SwiftUI reuses the NSView across state changes — repoint the pump only when the
|
||||
// connection identity actually changed.
|
||||
if view.connection !== connection {
|
||||
view.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
|
||||
}
|
||||
}
|
||||
|
||||
public static func dismantleNSView(_ view: StreamLayerView, coordinator: ()) {
|
||||
view.stop()
|
||||
}
|
||||
}
|
||||
|
||||
public final class StreamLayerView: NSView {
|
||||
/// Cancellation handle owned by exactly one pump thread — a restart hands the old pump
|
||||
/// its own token, so it can never be revived by a newer start().
|
||||
private final class PumpToken: @unchecked Sendable {
|
||||
private let lock = NSLock()
|
||||
private var live = true
|
||||
var isLive: Bool {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
return live
|
||||
}
|
||||
func cancel() {
|
||||
lock.lock()
|
||||
live = false
|
||||
lock.unlock()
|
||||
}
|
||||
}
|
||||
|
||||
private let displayLayer = AVSampleBufferDisplayLayer()
|
||||
private var pump: Thread?
|
||||
private var running = false
|
||||
private var token: PumpToken?
|
||||
public private(set) var connection: LumenConnection?
|
||||
|
||||
public override init(frame: NSRect) {
|
||||
super.init(frame: frame)
|
||||
wantsLayer = true
|
||||
displayLayer.videoGravity = .resizeAspect
|
||||
layer = displayLayer
|
||||
layer = displayLayer // layer-hosting: assign before wantsLayer
|
||||
wantsLayer = true
|
||||
}
|
||||
|
||||
public required init?(coder: NSCoder) { fatalError("not used") }
|
||||
|
||||
/// Pump thread: pull AUs from the connection, wrap, enqueue. The first IDR yields the
|
||||
/// format description; non-IDR AUs before it are dropped (the host opens with an IDR).
|
||||
public func start(connection: LumenConnection) {
|
||||
guard !running else { return }
|
||||
running = true
|
||||
public func start(
|
||||
connection: LumenConnection,
|
||||
onFrame: (@Sendable (AccessUnit) -> Void)? = nil,
|
||||
onSessionEnd: (@Sendable () -> Void)? = nil
|
||||
) {
|
||||
stop()
|
||||
let token = PumpToken()
|
||||
self.token = token
|
||||
self.connection = connection
|
||||
let layer = displayLayer
|
||||
let thread = Thread { [weak self] in
|
||||
layer.flush() // drop any frames a previous connection left queued
|
||||
|
||||
let thread = Thread {
|
||||
var format: CMVideoFormatDescription?
|
||||
while self?.running == true {
|
||||
while token.isLive {
|
||||
do {
|
||||
guard let au = try connection.nextAU(timeoutMs: 100) else { continue }
|
||||
onFrame?(au)
|
||||
if let f = AnnexB.formatDescription(fromIDR: au.data) {
|
||||
format = f // refreshed on every IDR (mode changes included)
|
||||
}
|
||||
guard let f = format,
|
||||
let sample = AnnexB.sampleBuffer(au: au, format: f)
|
||||
else { continue }
|
||||
if layer.status == .failed {
|
||||
// Decode wedged: flush and re-gate on the next in-band parameter
|
||||
// sets — resuming with a delta frame can't recover. (A
|
||||
// request-IDR channel on lumen/1 is a host-side TODO; with the
|
||||
// host's infinite GOP this may otherwise stay black until the
|
||||
// next recovery keyframe.)
|
||||
layer.flush()
|
||||
format = AnnexB.formatDescription(fromIDR: au.data)
|
||||
}
|
||||
guard let f = format,
|
||||
let sample = AnnexB.sampleBuffer(au: au, format: f),
|
||||
token.isLive // don't enqueue a stale frame after a restart
|
||||
else { continue }
|
||||
layer.enqueue(sample)
|
||||
} catch {
|
||||
if token.isLive {
|
||||
onSessionEnd?()
|
||||
}
|
||||
break // session closed
|
||||
}
|
||||
}
|
||||
}
|
||||
thread.name = "lumen-pump"
|
||||
thread.qualityOfService = .userInteractive
|
||||
pump = thread
|
||||
thread.start()
|
||||
}
|
||||
|
||||
/// Stop pumping (≤ one poll timeout). Does not close the connection — that stays with
|
||||
/// whoever owns it (LumenConnection.close() is safe alongside a draining pump).
|
||||
public func stop() {
|
||||
running = false
|
||||
token?.cancel()
|
||||
token = nil
|
||||
connection = nil
|
||||
}
|
||||
|
||||
deinit { running = false }
|
||||
deinit {
|
||||
token?.cancel()
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
// Unit tests for the Annex-B ⇄ AVCC plumbing (pure byte-level; no codec involved —
|
||||
// VideoToolboxRoundTripTests covers the real-bitstream path).
|
||||
|
||||
import XCTest
|
||||
@testable import LumenKit
|
||||
|
||||
final class AnnexBTests: XCTestCase {
|
||||
/// NAL with the given HEVC type in bits 1..6 of the first header byte.
|
||||
private func nal(type: UInt8, payload: [UInt8]) -> Data {
|
||||
Data([type << 1, 0x01] + payload)
|
||||
}
|
||||
|
||||
private let start4: [UInt8] = [0, 0, 0, 1]
|
||||
private let start3: [UInt8] = [0, 0, 1]
|
||||
|
||||
func testSplitMixedStartCodes() {
|
||||
let a = nal(type: 32, payload: [0xAA])
|
||||
let b = nal(type: 33, payload: [0xBB, 0xBC])
|
||||
let c = nal(type: 19, payload: [0xCC, 0xCD, 0xCE])
|
||||
var au = Data(start4)
|
||||
au.append(a)
|
||||
au.append(contentsOf: start3)
|
||||
au.append(b)
|
||||
au.append(contentsOf: start4)
|
||||
au.append(c)
|
||||
|
||||
let nals = AnnexB.nalUnits(in: au)
|
||||
XCTAssertEqual(nals, [a, b, c])
|
||||
XCTAssertEqual(nals.map(AnnexB.hevcNalType), [32, 33, 19])
|
||||
}
|
||||
|
||||
func testSplitSingleNalNoTrailingCode() {
|
||||
let v = nal(type: 34, payload: [1, 2, 3])
|
||||
let au = Data(start3) + v
|
||||
XCTAssertEqual(AnnexB.nalUnits(in: au), [v])
|
||||
}
|
||||
|
||||
func testSplitEmptyAndGarbage() {
|
||||
XCTAssertEqual(AnnexB.nalUnits(in: Data()), [])
|
||||
// No start code at all → no NALs.
|
||||
XCTAssertEqual(AnnexB.nalUnits(in: Data([9, 8, 7, 6])), [])
|
||||
}
|
||||
|
||||
func testSplitDropsTrailingZeroPadding() {
|
||||
// trailing_zero_8bits between NALs (and >2 zeros forming a long separator) must
|
||||
// not leak into the preceding NAL.
|
||||
let a = nal(type: 33, payload: [0xAA])
|
||||
let b = nal(type: 19, payload: [0xBB])
|
||||
var au = Data(start4)
|
||||
au.append(a)
|
||||
au.append(contentsOf: [0, 0, 0, 0, 0, 1]) // padding + start code
|
||||
au.append(b)
|
||||
XCTAssertEqual(AnnexB.nalUnits(in: au), [a, b])
|
||||
}
|
||||
|
||||
func testAvccDropsParameterSetsAndPrefixesLengths() {
|
||||
let vps = nal(type: 32, payload: [0xAA])
|
||||
let sps = nal(type: 33, payload: [0xBB])
|
||||
let pps = nal(type: 34, payload: [0xCC])
|
||||
let idr = nal(type: 19, payload: [0xDD, 0xDE, 0xDF, 0xE0])
|
||||
var au = Data()
|
||||
for n in [vps, sps, pps, idr] {
|
||||
au.append(contentsOf: start4)
|
||||
au.append(n)
|
||||
}
|
||||
|
||||
let avcc = AnnexB.avcc(from: au)
|
||||
// Only the IDR survives: 4-byte BE length, then the NAL bytes.
|
||||
var expected = Data([0, 0, 0, UInt8(idr.count)])
|
||||
expected.append(idr)
|
||||
XCTAssertEqual(avcc, expected)
|
||||
}
|
||||
|
||||
func testFormatDescriptionNilWithoutParameterSets() {
|
||||
let idr = nal(type: 19, payload: [0xDD])
|
||||
let au = Data(start4) + idr
|
||||
XCTAssertNil(AnnexB.formatDescription(fromIDR: au))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
// Integration: the Swift wrapper against a real lumen/1 host over QUIC + UDP on loopback —
|
||||
// the Swift twin of lumen-host's m3.rs::c_abi_connection_roundtrip, this time through the
|
||||
// statically linked xcframework. Driven by clients/apple/test-loopback.sh, which builds and
|
||||
// starts `lumen-host m3-host --source synthetic` and sets LUMEN_LOOPBACK_PORT.
|
||||
|
||||
import XCTest
|
||||
@testable import LumenKit
|
||||
|
||||
final class LoopbackIntegrationTests: XCTestCase {
|
||||
func testSyntheticStreamRoundTrip() throws {
|
||||
guard let portStr = ProcessInfo.processInfo.environment["LUMEN_LOOPBACK_PORT"],
|
||||
let port = UInt16(portStr)
|
||||
else {
|
||||
throw XCTSkip("needs a running m3-host — use clients/apple/test-loopback.sh")
|
||||
}
|
||||
|
||||
let conn = try LumenConnection(
|
||||
host: "127.0.0.1", port: port, width: 1280, height: 720, refreshHz: 60)
|
||||
XCTAssertEqual(conn.width, 1280)
|
||||
XCTAssertEqual(conn.height, 720)
|
||||
XCTAssertEqual(conn.refreshHz, 60)
|
||||
|
||||
// Pull 25 synthetic frames and byte-verify the documented pattern:
|
||||
// u32 LE frame index, then data[i] = (idx as u8) &+ (i as u8).
|
||||
var got = 0
|
||||
var lastIndex: UInt32 = 0
|
||||
let deadline = Date().addingTimeInterval(30)
|
||||
while got < 25 {
|
||||
XCTAssertLessThan(Date(), deadline, "timed out after \(got) frames")
|
||||
guard let au = try conn.nextAU(timeoutMs: 2000) else { continue }
|
||||
let idx = au.data.prefix(4).reversed().reduce(UInt32(0)) { ($0 << 8) | UInt32($1) }
|
||||
for (i, byte) in au.data.enumerated().dropFirst(4) {
|
||||
let expected = UInt8(truncatingIfNeeded: idx) &+ UInt8(truncatingIfNeeded: i)
|
||||
if byte != expected {
|
||||
XCTFail("frame \(idx) corrupt at offset \(i)")
|
||||
break
|
||||
}
|
||||
}
|
||||
XCTAssertGreaterThan(au.ptsNs, 0)
|
||||
lastIndex = idx
|
||||
got += 1
|
||||
}
|
||||
XCTAssertGreaterThanOrEqual(lastIndex, 24)
|
||||
|
||||
// Input goes the other way (enqueue-only; the host logs the count on close).
|
||||
conn.send(.mouseMove(dx: 1, dy: 2))
|
||||
conn.send(.key(0x41, down: true))
|
||||
conn.send(.key(0x41, down: false))
|
||||
|
||||
conn.close()
|
||||
XCTAssertThrowsError(try conn.nextAU(timeoutMs: 10)) { error in
|
||||
guard case LumenClientError.closed = error else {
|
||||
return XCTFail("expected .closed, got \(error)")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func testConnectFailureThrows() {
|
||||
// Nothing listens on this port; connect must fail within its timeout, not hang.
|
||||
XCTAssertThrowsError(
|
||||
try LumenConnection(
|
||||
host: "127.0.0.1", port: 9, width: 640, height: 480, refreshHz: 30,
|
||||
timeoutMs: 2000))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
// First light, headless: the full client pipeline against a REAL remote host — QUIC
|
||||
// handshake over the LAN, NVENC HEVC AUs through FEC + AES-GCM, AnnexB conversion, and a
|
||||
// real VTDecompressionSession turning them into pixels. Everything the GUI does except
|
||||
// putting the layer on glass.
|
||||
//
|
||||
// Run (host side, on the Linux box):
|
||||
// LUMEN_COMPOSITOR=gamescope LUMEN_GAMESCOPE_APP=vkcube LUMEN_ZEROCOPY=1 \
|
||||
// lumen-host m3-host --source virtual --seconds 120
|
||||
// Then here:
|
||||
// LUMEN_REMOTE_HOST=192.168.1.70 swift test --filter RemoteFirstLightTests
|
||||
|
||||
import CoreMedia
|
||||
import VideoToolbox
|
||||
import XCTest
|
||||
@testable import LumenKit
|
||||
|
||||
final class RemoteFirstLightTests: XCTestCase {
|
||||
func testRemoteStreamDecodesToPixels() throws {
|
||||
guard let host = ProcessInfo.processInfo.environment["LUMEN_REMOTE_HOST"] else {
|
||||
throw XCTSkip("set LUMEN_REMOTE_HOST (and start m3-host --source virtual there)")
|
||||
}
|
||||
let width: UInt32 = 1280
|
||||
let height: UInt32 = 720
|
||||
|
||||
let conn = try LumenConnection(
|
||||
host: host, width: width, height: height, refreshHz: 60)
|
||||
defer { conn.close() }
|
||||
XCTAssertEqual(conn.width, width)
|
||||
XCTAssertEqual(conn.height, height)
|
||||
|
||||
var format: CMVideoFormatDescription?
|
||||
var decoder: VTDecompressionSession?
|
||||
defer { decoder.map { VTDecompressionSessionInvalidate($0) } }
|
||||
var received = 0
|
||||
var decoded = 0
|
||||
var firstPtsNs: UInt64 = 0
|
||||
var lastPtsNs: UInt64 = 0
|
||||
let deadline = Date().addingTimeInterval(30)
|
||||
|
||||
while decoded < 60, Date() < deadline {
|
||||
guard let au = try conn.nextAU(timeoutMs: 2000) else { continue }
|
||||
received += 1
|
||||
if firstPtsNs == 0 { firstPtsNs = au.ptsNs }
|
||||
lastPtsNs = au.ptsNs
|
||||
|
||||
if let f = AnnexB.formatDescription(fromIDR: au.data) {
|
||||
format = f
|
||||
if decoder == nil {
|
||||
let dims = CMVideoFormatDescriptionGetDimensions(f)
|
||||
XCTAssertEqual(UInt32(dims.width), width)
|
||||
XCTAssertEqual(UInt32(dims.height), height)
|
||||
var session: VTDecompressionSession?
|
||||
XCTAssertEqual(
|
||||
VTDecompressionSessionCreate(
|
||||
allocator: nil, formatDescription: f, decoderSpecification: nil,
|
||||
imageBufferAttributes: nil, outputCallback: nil,
|
||||
decompressionSessionOut: &session),
|
||||
noErr)
|
||||
decoder = session
|
||||
}
|
||||
}
|
||||
guard let f = format, let dec = decoder,
|
||||
let sample = AnnexB.sampleBuffer(au: au, format: f)
|
||||
else { continue }
|
||||
|
||||
var gotPixels = false
|
||||
VTDecompressionSessionDecodeFrame(
|
||||
dec, sampleBuffer: sample, flags: [], infoFlagsOut: nil
|
||||
) { status, _, imageBuffer, _, _ in
|
||||
gotPixels = status == noErr && imageBuffer != nil
|
||||
}
|
||||
if gotPixels { decoded += 1 }
|
||||
}
|
||||
|
||||
XCTAssertGreaterThanOrEqual(decoded, 60, "decoded \(decoded)/\(received) received AUs")
|
||||
// The host stamps pts with its capture wall clock — 60 frames should span ~1 s.
|
||||
let spanMs = Double(lastPtsNs &- firstPtsNs) / 1_000_000
|
||||
print("first light: \(decoded) frames decoded, \(received) received, pts span \(Int(spanMs)) ms")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,176 @@
|
||||
// Real-bitstream proof of the decode-prep path: VTCompressionSession encodes HEVC, we
|
||||
// rebuild the host's wire shape (Annex-B AU with in-band VPS/SPS/PPS — exactly what
|
||||
// lumen-host emits on every IDR), run it through AnnexB, and hand the result to a real
|
||||
// VTDecompressionSession. Pixels out = the whole client decode path is sound.
|
||||
|
||||
import AVFoundation
|
||||
import CoreMedia
|
||||
import VideoToolbox
|
||||
import XCTest
|
||||
@testable import LumenKit
|
||||
|
||||
final class VideoToolboxRoundTripTests: XCTestCase {
|
||||
private let width = 320
|
||||
private let height = 240
|
||||
|
||||
func testEncodeAnnexBDecodeRoundTrip() throws {
|
||||
let (formatDesc, avccSample) = try encodeOneHEVCKeyframe()
|
||||
|
||||
// Rebuild the host's wire format: Annex-B AU, parameter sets in-band before the VCL.
|
||||
let annexB = try annexBAU(formatDesc: formatDesc, avccSample: avccSample)
|
||||
|
||||
// 1) Parameter-set extraction → format description.
|
||||
let rebuilt = try XCTUnwrap(
|
||||
AnnexB.formatDescription(fromIDR: annexB),
|
||||
"in-band VPS/SPS/PPS should yield a format description")
|
||||
let dims = CMVideoFormatDescriptionGetDimensions(rebuilt)
|
||||
XCTAssertEqual(Int(dims.width), width)
|
||||
XCTAssertEqual(Int(dims.height), height)
|
||||
|
||||
// 2) Annex-B → AVCC re-pack must reproduce the encoder's own sample bytes.
|
||||
XCTAssertEqual(AnnexB.avcc(from: annexB), avccSample)
|
||||
|
||||
// 3) Sample buffer → real decoder → pixels.
|
||||
let au = AccessUnit(data: annexB, ptsNs: 1_000_000, frameIndex: 0, flags: 0)
|
||||
let sample = try XCTUnwrap(AnnexB.sampleBuffer(au: au, format: rebuilt))
|
||||
|
||||
var session: VTDecompressionSession?
|
||||
XCTAssertEqual(
|
||||
VTDecompressionSessionCreate(
|
||||
allocator: nil, formatDescription: rebuilt, decoderSpecification: nil,
|
||||
imageBufferAttributes: nil, outputCallback: nil,
|
||||
decompressionSessionOut: &session),
|
||||
noErr)
|
||||
let decoder = try XCTUnwrap(session)
|
||||
defer { VTDecompressionSessionInvalidate(decoder) }
|
||||
|
||||
var decoded: CVImageBuffer?
|
||||
var decodeStatus: OSStatus = -1
|
||||
// No async flag → the handler runs before DecodeFrame returns.
|
||||
VTDecompressionSessionDecodeFrame(
|
||||
decoder, sampleBuffer: sample, flags: [], infoFlagsOut: nil
|
||||
) { status, _, imageBuffer, _, _ in
|
||||
decodeStatus = status
|
||||
decoded = imageBuffer
|
||||
}
|
||||
XCTAssertEqual(decodeStatus, noErr)
|
||||
let pixels = try XCTUnwrap(decoded) // CVImageBuffer and CVPixelBuffer are the same CF type
|
||||
XCTAssertEqual(CVPixelBufferGetWidth(pixels), width)
|
||||
XCTAssertEqual(CVPixelBufferGetHeight(pixels), height)
|
||||
}
|
||||
|
||||
// MARK: - encode helpers
|
||||
|
||||
/// One forced-IDR HEVC frame; returns its format description and raw AVCC sample bytes.
|
||||
private func encodeOneHEVCKeyframe() throws -> (CMVideoFormatDescription, Data) {
|
||||
var session: VTCompressionSession?
|
||||
let rc = VTCompressionSessionCreate(
|
||||
allocator: nil, width: Int32(width), height: Int32(height),
|
||||
codecType: kCMVideoCodecType_HEVC, encoderSpecification: nil,
|
||||
imageBufferAttributes: nil, compressedDataAllocator: nil,
|
||||
outputCallback: nil, refcon: nil, compressionSessionOut: &session)
|
||||
guard rc == noErr, let encoder = session else {
|
||||
throw XCTSkip("no HEVC encoder available (\(rc))")
|
||||
}
|
||||
defer { VTCompressionSessionInvalidate(encoder) }
|
||||
VTSessionSetProperty(encoder, key: kVTCompressionPropertyKey_RealTime, value: kCFBooleanTrue)
|
||||
VTSessionSetProperty(
|
||||
encoder, key: kVTCompressionPropertyKey_AllowFrameReordering, value: kCFBooleanFalse)
|
||||
|
||||
let lock = NSLock()
|
||||
var output: CMSampleBuffer?
|
||||
let done = expectation(description: "encoded")
|
||||
VTCompressionSessionEncodeFrame(
|
||||
encoder, imageBuffer: try gradientPixelBuffer(),
|
||||
presentationTimeStamp: CMTime(value: 0, timescale: 30),
|
||||
duration: CMTime(value: 1, timescale: 30),
|
||||
frameProperties: [kVTEncodeFrameOptionKey_ForceKeyFrame: kCFBooleanTrue] as CFDictionary,
|
||||
infoFlagsOut: nil
|
||||
) { status, _, sample in
|
||||
XCTAssertEqual(status, noErr)
|
||||
lock.lock()
|
||||
output = sample
|
||||
lock.unlock()
|
||||
done.fulfill()
|
||||
}
|
||||
VTCompressionSessionCompleteFrames(encoder, untilPresentationTimeStamp: .invalid)
|
||||
wait(for: [done], timeout: 10)
|
||||
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
let sample = try XCTUnwrap(output)
|
||||
let desc = try XCTUnwrap(CMSampleBufferGetFormatDescription(sample))
|
||||
let block = try XCTUnwrap(CMSampleBufferGetDataBuffer(sample))
|
||||
var bytes = Data(count: CMBlockBufferGetDataLength(block))
|
||||
try bytes.withUnsafeMutableBytes { raw in
|
||||
let rc = CMBlockBufferCopyDataBytes(
|
||||
block, atOffset: 0, dataLength: raw.count,
|
||||
destination: raw.baseAddress!)
|
||||
if rc != noErr { throw NSError(domain: "CMBlockBuffer", code: Int(rc)) }
|
||||
}
|
||||
return (desc, bytes)
|
||||
}
|
||||
|
||||
/// The host's wire shape: 4-byte start codes, VPS/SPS/PPS in-band, then the VCL NALs.
|
||||
private func annexBAU(formatDesc: CMVideoFormatDescription, avccSample: Data) throws -> Data {
|
||||
var au = Data()
|
||||
|
||||
var psCount = 0
|
||||
var nalHeaderLen: Int32 = 0
|
||||
XCTAssertEqual(
|
||||
CMVideoFormatDescriptionGetHEVCParameterSetAtIndex(
|
||||
formatDesc, parameterSetIndex: 0, parameterSetPointerOut: nil,
|
||||
parameterSetSizeOut: nil, parameterSetCountOut: &psCount,
|
||||
nalUnitHeaderLengthOut: &nalHeaderLen),
|
||||
noErr)
|
||||
XCTAssertEqual(nalHeaderLen, 4, "AnnexB.avcc assumes 4-byte NAL length prefixes")
|
||||
for i in 0..<psCount {
|
||||
var ptr: UnsafePointer<UInt8>?
|
||||
var size = 0
|
||||
XCTAssertEqual(
|
||||
CMVideoFormatDescriptionGetHEVCParameterSetAtIndex(
|
||||
formatDesc, parameterSetIndex: i, parameterSetPointerOut: &ptr,
|
||||
parameterSetSizeOut: &size, parameterSetCountOut: nil,
|
||||
nalUnitHeaderLengthOut: nil),
|
||||
noErr)
|
||||
au.append(contentsOf: [0, 0, 0, 1])
|
||||
au.append(Data(bytes: try XCTUnwrap(ptr), count: size))
|
||||
}
|
||||
|
||||
// AVCC sample (4-byte BE length per NAL) → start codes.
|
||||
var i = avccSample.startIndex
|
||||
while i + 4 <= avccSample.endIndex {
|
||||
let len = avccSample[i..<i + 4].reduce(0) { ($0 << 8) | Int($1) }
|
||||
let body = avccSample.index(i, offsetBy: 4)
|
||||
guard let end = avccSample.index(body, offsetBy: len, limitedBy: avccSample.endIndex)
|
||||
else { break }
|
||||
au.append(contentsOf: [0, 0, 0, 1])
|
||||
au.append(avccSample[body..<end])
|
||||
i = end
|
||||
}
|
||||
return au
|
||||
}
|
||||
|
||||
private func gradientPixelBuffer() throws -> CVPixelBuffer {
|
||||
var pb: CVPixelBuffer?
|
||||
let attrs = [kCVPixelBufferIOSurfacePropertiesKey: [:]] as CFDictionary
|
||||
XCTAssertEqual(
|
||||
CVPixelBufferCreate(nil, width, height, kCVPixelFormatType_32BGRA, attrs, &pb),
|
||||
kCVReturnSuccess)
|
||||
let buf = try XCTUnwrap(pb)
|
||||
CVPixelBufferLockBaseAddress(buf, [])
|
||||
defer { CVPixelBufferUnlockBaseAddress(buf, []) }
|
||||
let base = try XCTUnwrap(CVPixelBufferGetBaseAddress(buf))
|
||||
let stride = CVPixelBufferGetBytesPerRow(buf)
|
||||
for y in 0..<height {
|
||||
let row = base.advanced(by: y * stride).assumingMemoryBound(to: UInt8.self)
|
||||
for x in 0..<width {
|
||||
row[x * 4 + 0] = UInt8(x & 0xFF) // B
|
||||
row[x * 4 + 1] = UInt8(y & 0xFF) // G
|
||||
row[x * 4 + 2] = UInt8((x ^ y) & 0xFF) // R
|
||||
row[x * 4 + 3] = 0xFF
|
||||
}
|
||||
}
|
||||
return buf
|
||||
}
|
||||
}
|
||||
Executable
+17
@@ -0,0 +1,17 @@
|
||||
#!/usr/bin/env bash
|
||||
# Loopback integration: a real lumen/1 host (synthetic source — pure protocol, runs fine on
|
||||
# macOS) on 127.0.0.1, then the Swift integration tests against it through the xcframework.
|
||||
# The m3 host serves exactly one session and exits; the trap is just for failure paths.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/../.."
|
||||
|
||||
PORT="${LUMEN_LOOPBACK_PORT:-19778}"
|
||||
|
||||
cargo build --release -p lumen-host
|
||||
target/release/lumen-host m3-host --port "$PORT" --source synthetic --frames 300 &
|
||||
HOST_PID=$!
|
||||
trap 'kill "$HOST_PID" 2>/dev/null || true' EXIT
|
||||
sleep 1
|
||||
|
||||
cd clients/apple
|
||||
LUMEN_LOOPBACK_PORT="$PORT" swift test --filter LoopbackIntegrationTests
|
||||
@@ -7,7 +7,26 @@
|
||||
//! the media streams follow (see the M2 task list / plan).
|
||||
|
||||
pub mod apps;
|
||||
#[cfg(target_os = "linux")]
|
||||
mod audio;
|
||||
/// Stub — the audio plane needs Linux (PipeWire capture + libopus); this keeps non-Linux
|
||||
/// dev builds compiling (crate doc: "the crate compiles everywhere"). Reports failure the
|
||||
/// same way the real stream thread does: by clearing `running`.
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
mod audio {
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
pub fn start(
|
||||
running: Arc<AtomicBool>,
|
||||
_gcm_key: [u8; 16],
|
||||
_rikeyid: i32,
|
||||
_audio_cap: Arc<Mutex<Option<Box<dyn crate::audio::AudioCapturer>>>>,
|
||||
) {
|
||||
tracing::error!("GameStream audio requires Linux (PipeWire + libopus)");
|
||||
running.store(false, Ordering::SeqCst);
|
||||
}
|
||||
}
|
||||
pub(crate) mod cert;
|
||||
mod control;
|
||||
mod crypto;
|
||||
|
||||
@@ -144,6 +144,7 @@ type PacketBatch = Vec<Vec<u8>>;
|
||||
|
||||
/// Send `pkts` with as few syscalls as possible (`sendmmsg`, up to 64 per call). The socket is
|
||||
/// connected, so no per-message address. Returns an error on the first send failure.
|
||||
#[cfg(target_os = "linux")]
|
||||
fn sendmmsg_all(sock: &UdpSocket, pkts: &[Vec<u8>]) -> std::io::Result<()> {
|
||||
use std::os::fd::AsRawFd;
|
||||
const CHUNK: usize = 64;
|
||||
@@ -179,6 +180,16 @@ fn sendmmsg_all(sock: &UdpSocket, pkts: &[Vec<u8>]) -> std::io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Portable fallback (non-Linux dev builds — GameStream hosting never ships there): one
|
||||
/// syscall per packet.
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
fn sendmmsg_all(sock: &UdpSocket, pkts: &[Vec<u8>]) -> std::io::Result<()> {
|
||||
for p in pkts {
|
||||
sock.send(p)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Dedicated send thread: one [`PacketBatch`] per frame arrives on `rx`; its packets go out in
|
||||
/// `sendmmsg` chunks, paced so the frame's data spreads over ~3/4 of the frame interval
|
||||
/// (microburst shaping at chunk granularity — a real link drops line-rate bursts; the encode
|
||||
|
||||
@@ -370,10 +370,12 @@ impl EiState {
|
||||
InputKind::MouseScroll => match slot.interface::<ei::Scroll>() {
|
||||
Some(s) => {
|
||||
// GameStream sends WHEEL_DELTA(120)-scaled deltas in `x`; ei scroll_discrete
|
||||
// uses the same 120-per-detent unit. Positive GameStream = up/left, which is
|
||||
// negative on the ei axis (matches wl_pointer).
|
||||
// uses the same 120-per-detent unit. Positive GameStream = up (vertical),
|
||||
// which is negative on the ei axis, but = RIGHT (horizontal), which is
|
||||
// already positive there (moonlight-qt/Sunshine pass horizontal through
|
||||
// unnegated) — only the vertical axis flips.
|
||||
if ev.code == SCROLL_HORIZONTAL {
|
||||
s.scroll_discrete(-ev.x, 0);
|
||||
s.scroll_discrete(ev.x, 0);
|
||||
} else {
|
||||
s.scroll_discrete(0, -ev.x);
|
||||
}
|
||||
|
||||
@@ -226,10 +226,17 @@ impl InputInjector for WlrootsInjector {
|
||||
wl_pointer::Axis::VerticalScroll
|
||||
};
|
||||
// GameStream sends WHEEL_DELTA(120)-scaled units; a notch ≈ 15px. Positive
|
||||
// GameStream = scroll up, which is negative on the Wayland axis.
|
||||
// GameStream = up (vertical), negative on the Wayland axis; but = RIGHT
|
||||
// (horizontal), already positive there (moonlight-qt/Sunshine pass
|
||||
// horizontal through unnegated) — only the vertical axis flips.
|
||||
let notches = event.x as f64 / 120.0;
|
||||
let sign = if event.code == SCROLL_HORIZONTAL {
|
||||
1.0
|
||||
} else {
|
||||
-1.0
|
||||
};
|
||||
self.pointer.axis_source(wl_pointer::AxisSource::Wheel);
|
||||
self.pointer.axis(t, axis, -notches * 15.0);
|
||||
self.pointer.axis(t, axis, sign * notches * 15.0);
|
||||
self.pointer.frame();
|
||||
}
|
||||
InputKind::KeyDown | InputKind::KeyUp => {
|
||||
|
||||
@@ -448,6 +448,7 @@ fn input_thread(rx: std::sync::mpsc::Receiver<InputEvent>, conn: quinn::Connecti
|
||||
/// The audio thread: desktop capture → Opus (48 kHz stereo, 5 ms, CBR — same tuning as the
|
||||
/// GameStream path) → `AUDIO_MAGIC` datagrams. QUIC already encrypts; no extra layer.
|
||||
/// The capturer comes from (and returns to) the persistent slot — see [`AudioCapSlot`].
|
||||
#[cfg(target_os = "linux")]
|
||||
fn audio_thread(conn: quinn::Connection, stop: Arc<AtomicBool>, audio_cap: AudioCapSlot) {
|
||||
use crate::audio::{CHANNELS, SAMPLE_RATE};
|
||||
const FRAME_MS: usize = 5;
|
||||
@@ -519,6 +520,15 @@ fn audio_thread(conn: quinn::Connection, stop: Arc<AtomicBool>, audio_cap: Audio
|
||||
}
|
||||
}
|
||||
|
||||
/// Stub — lumen/1 audio needs Linux (PipeWire capture + libopus); non-Linux dev builds
|
||||
/// run sessions without it, same as when the capturer fails to open.
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
fn audio_thread(_conn: quinn::Connection, _stop: Arc<AtomicBool>, _audio_cap: AudioCapSlot) {
|
||||
tracing::warn!(
|
||||
"lumen/1 audio requires Linux (PipeWire + libopus) — session continues without it"
|
||||
);
|
||||
}
|
||||
|
||||
fn synthetic_stream(session: &mut Session, frames: u32, stop: &AtomicBool) -> Result<()> {
|
||||
let interval = std::time::Duration::from_millis(1000 / 60);
|
||||
for idx in 0..frames {
|
||||
|
||||
@@ -13,11 +13,13 @@ cd "$(dirname "$0")/.."
|
||||
TARGETS_MAC=(aarch64-apple-darwin x86_64-apple-darwin)
|
||||
BUILD_IOS="${BUILD_IOS:-0}" # BUILD_IOS=1 adds an iOS slice (requires the ios target installed)
|
||||
|
||||
# Deployment targets must match Package.swift's platforms, or every consumer link emits
|
||||
# "object file was built for newer macOS version" warnings.
|
||||
for t in "${TARGETS_MAC[@]}"; do
|
||||
cargo build --release -p lumen-core --features quic --target "$t"
|
||||
MACOSX_DEPLOYMENT_TARGET=14.0 cargo build --release -p lumen-core --features quic --target "$t"
|
||||
done
|
||||
if [[ "$BUILD_IOS" == "1" ]]; then
|
||||
cargo build --release -p lumen-core --features quic --target aarch64-apple-ios
|
||||
IPHONEOS_DEPLOYMENT_TARGET=17.0 cargo build --release -p lumen-core --features quic --target aarch64-apple-ios
|
||||
fi
|
||||
|
||||
STAGE="$(mktemp -d)"
|
||||
@@ -49,6 +51,18 @@ if [[ "$BUILD_IOS" == "1" ]]; then
|
||||
ARGS+=(-library target/aarch64-apple-ios/release/liblumen_core.a -headers "$STAGE/include")
|
||||
fi
|
||||
|
||||
# Cargo does NOT fingerprint MACOSX_DEPLOYMENT_TARGET — units cached from a build without
|
||||
# it keep their old minos forever. Refuse to ship anything newer than the package floor
|
||||
# (objects BELOW it, e.g. rustup's precompiled std at 11.0, are fine and unavoidable).
|
||||
for obj in "$STAGE"/macos/liblumen_core.a; do
|
||||
bad=$(otool -l "$obj" 2>/dev/null | awk '/minos/ {print $2}' | sort -uV | awk -F. '$1 > 14' | head -1)
|
||||
if [[ -n "$bad" ]]; then
|
||||
echo "ERROR: $obj contains objects built for macOS $bad (> 14.0)." >&2
|
||||
echo "Stale cache — rm -rf target/{aarch64,x86_64}-apple-darwin and rebuild." >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
rm -rf clients/apple/LumenCore.xcframework
|
||||
xcodebuild -create-xcframework "${ARGS[@]}" -output clients/apple/LumenCore.xcframework
|
||||
echo "OK: clients/apple/LumenCore.xcframework"
|
||||
|
||||
Reference in New Issue
Block a user