feat(host/windows): host→client Opus audio — vendored libopus on MSVC
apple / swift (push) Successful in 53s
android / android (push) Failing after 35s
ci / docs-site (push) Successful in 29s
ci / bench (push) Failing after 26s
decky / build-publish (push) Failing after 3s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Failing after 0s
ci / rust (push) Failing after 30s
ci / web (push) Successful in 27s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Failing after 0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Failing after 0s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Failing after 0s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Failing after 1s
docker / deploy-docs (push) Has been skipped
flatpak / build-publish (push) Failing after 0s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 0s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 0s
deb / build-publish (push) Failing after 47s

The `m3` audio_thread (desktop capture → Opus 48 kHz stereo 5 ms CBR →
AUDIO_MAGIC datagrams) now runs on Windows, fed by the WASAPI loopback
capturer. The `opus` crate vendors libopus via `audiopus_sys` + cmake
(no system lib / vcpkg), so it builds on MSVC — moved into a
`cfg(any(linux, windows))` deps table and widened the audio_thread cfg
to match (the stub now only covers other targets, e.g. macOS).

Build note: CMake 4 rejects libopus's old `cmake_minimum_required`;
set `CMAKE_POLICY_VERSION_MINIMUM=3.5` when building the host on Windows.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 07:30:00 +00:00
parent 8c8d576e52
commit b8a1b7e469
3 changed files with 12 additions and 12 deletions
+6 -2
View File
@@ -53,6 +53,12 @@ utoipa-scalar = { version = "0.3", features = ["axum"] }
tower = { version = "0.5", features = ["util"] } tower = { version = "0.5", features = ["util"] }
http-body-util = "0.1" http-body-util = "0.1"
# Opus stereo encode for the host->client audio plane. The `opus` crate vendors libopus via
# `audiopus_sys` (cmake-built from source — no system lib, no vcpkg), so it builds on Windows MSVC
# too (needs CMake + NASM, both on the box). Both platforms that have an audio-capture backend.
[target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies]
opus = "0.3"
[target.'cfg(target_os = "linux")'.dependencies] [target.'cfg(target_os = "linux")'.dependencies]
# `screencast` gates the ScreenCast portal module; `remote_desktop` adds the RemoteDesktop # `screencast` gates the ScreenCast portal module; `remote_desktop` adds the RemoteDesktop
# portal we use for libei input on KWin/GNOME; `tokio` is the default runtime. # portal we use for libei input on KWin/GNOME; `tokio` is the default runtime.
@@ -81,8 +87,6 @@ wayland-backend = "0.3"
serde_json = "1" serde_json = "1"
# Builds/validates the xkb keymap uploaded to the virtual keyboard + tracks modifier state. # Builds/validates the xkb keymap uploaded to the virtual keyboard + tracks modifier state.
xkbcommon = "0.8" xkbcommon = "0.8"
# Opus encode for the GameStream audio stream (links system libopus).
opus = "0.3"
# The safe `opus` crate is stereo-only; surround (5.1/7.1) needs the libopus *multistream* # The safe `opus` crate is stereo-only; surround (5.1/7.1) needs the libopus *multistream*
# encoder (`opus_multistream_encoder_*`). `audiopus_sys` is the sys layer `opus` already # encoder (`opus_multistream_encoder_*`). `audiopus_sys` is the sys layer `opus` already
# vendors (same libopus link), so this adds bindings, not a second copy of the library. # vendors (same libopus link), so this adds bindings, not a second copy of the library.
+3 -5
View File
@@ -1340,7 +1340,7 @@ fn input_thread(
/// The audio thread: desktop capture → Opus (48 kHz stereo, 5 ms, CBR — same tuning as the /// The audio thread: desktop capture → Opus (48 kHz stereo, 5 ms, CBR — same tuning as the
/// GameStream path) → `AUDIO_MAGIC` datagrams. QUIC already encrypts; no extra layer. /// GameStream path) → `AUDIO_MAGIC` datagrams. QUIC already encrypts; no extra layer.
/// The capturer comes from (and returns to) the persistent slot — see [`AudioCapSlot`]. /// The capturer comes from (and returns to) the persistent slot — see [`AudioCapSlot`].
#[cfg(target_os = "linux")] #[cfg(any(target_os = "linux", target_os = "windows"))]
fn audio_thread(conn: quinn::Connection, stop: Arc<AtomicBool>, audio_cap: AudioCapSlot) { fn audio_thread(conn: quinn::Connection, stop: Arc<AtomicBool>, audio_cap: AudioCapSlot) {
use crate::audio::{CHANNELS, SAMPLE_RATE}; use crate::audio::{CHANNELS, SAMPLE_RATE};
const FRAME_MS: usize = 5; const FRAME_MS: usize = 5;
@@ -1415,11 +1415,9 @@ fn audio_thread(conn: quinn::Connection, stop: Arc<AtomicBool>, audio_cap: Audio
/// Stub — punktfunk/1 audio needs Linux (PipeWire capture + libopus); non-Linux dev builds /// Stub — punktfunk/1 audio needs Linux (PipeWire capture + libopus); non-Linux dev builds
/// run sessions without it, same as when the capturer fails to open. /// run sessions without it, same as when the capturer fails to open.
#[cfg(not(target_os = "linux"))] #[cfg(not(any(target_os = "linux", target_os = "windows")))]
fn audio_thread(_conn: quinn::Connection, _stop: Arc<AtomicBool>, _audio_cap: AudioCapSlot) { fn audio_thread(_conn: quinn::Connection, _stop: Arc<AtomicBool>, _audio_cap: AudioCapSlot) {
tracing::warn!( tracing::warn!("punktfunk/1 audio requires Linux or Windows — session continues without it");
"punktfunk/1 audio requires Linux (PipeWire + libopus) — session continues without it"
);
} }
fn synthetic_stream( fn synthetic_stream(
+3 -5
View File
@@ -28,13 +28,9 @@ Every OS-touching backend is implemented behind the existing traits and **builds
| NVENC (D3D11, `--features nvenc`) | ✅ compiles+links | needs a GPU at runtime | | NVENC (D3D11, `--features nvenc`) | ✅ compiles+links | needs a GPU at runtime |
| Run host (serve/m3-host) | ✅ live | m3-host starts + listens; `c_abi_connection_roundtrip` passes | | Run host (serve/m3-host) | ✅ live | m3-host starts + listens; `c_abi_connection_roundtrip` passes |
| Gamepad (ViGEm) | ✅ done | compiles; live needs ViGEmBus + a physical pad; rumble back-channel TODO | | Gamepad (ViGEm) | ✅ done | compiles; live needs ViGEmBus + a physical pad; rumble back-channel TODO |
| Host→client audio wiring | ⬜ blocked | `opus` crate links *system* libopus (not on MSVC) | | Host→client audio wiring | ✅ done | builds on MSVC; `m3` `audio_thread` active on Windows (silent VM → no samples to send) |
**Remaining for full parity:** **Remaining for full parity:**
- **Host→client Opus audio** — the WASAPI capture backend is done + init-validated, but the `m3`
`audio_thread` stays Linux-gated because the `opus` crate links *system* libopus, absent on MSVC.
Finish by providing libopus on MSVC (vcpkg) or switching `audio_thread` to a vendoring Opus crate
(audiopus/magnum-opus build libopus from C source), then widen the `audio_thread` cfg.
- **ViGEm rumble back-channel** (`Xbox360Wired::request_notification`) — small; needs a physical pad. - **ViGEm rumble back-channel** (`Xbox360Wired::request_notification`) — small; needs a physical pad.
- **Live GPU/in-session validation** — SudoVDA monitor activation, DXGI capture, NVENC encode, and - **Live GPU/in-session validation** — SudoVDA monitor activation, DXGI capture, NVENC encode, and
SendInput injection all need a real GPU and an interactive (console) session, not SSH/Session-0. SendInput injection all need a real GPU and an interactive (console) session, not SSH/Session-0.
@@ -59,6 +55,8 @@ Every OS-touching backend is implemented behind the existing traits and **builds
(Gitea). Sync uncommitted files with **sftp** (`sftp -b - host`, `/C:/...` paths — scp and (Gitea). Sync uncommitted files with **sftp** (`sftp -b - host`, `/C:/...` paths — scp and
base64-over-ssh are unreliable here). Commit on Linux → `git reset --hard origin/main` on the VM. base64-over-ssh are unreliable here). Commit on Linux → `git reset --hard origin/main` on the VM.
Build env: `PATH` += cargo bin + NASM + CMake + LLVM (vcvars not needed — rustc/cc self-locate MSVC). Build env: `PATH` += cargo bin + NASM + CMake + LLVM (vcvars not needed — rustc/cc self-locate MSVC).
Set `CMAKE_POLICY_VERSION_MINIMUM=3.5` — CMake 4 rejects libopus's old `cmake_minimum_required` when
`audiopus_sys` (vendored by the `opus` crate) builds libopus from source for the host→client audio path.
## Decisions (locked 2026-06-14) ## Decisions (locked 2026-06-14)