Three more tvOS-isms, all the same lesson — let the focus engine own the chrome:
- Host cards drew their own material platter + accent ring INSIDE the .card button
style, muting the native grow/tilt focus motion. On tvOS the card style now owns the
platter outright (material/ring stay on the pointer platforms), and the grid gets
48 pt spacing so the focused card swells without overlapping siblings.
- Add Host and Settings no longer sit in the hosts row: they're a compact button row
below the grid (and the empty state gains a Settings button, since tvOS has no
toolbar).
- The Add Host and pairing dialogs drop Form entirely on tvOS — list rows added a
full-width focus fill plus a row platter behind every field's own pill (the
"second outer pill"). As standalone fields in a centered dialog over the dimmed
home, each input is exactly one pill with vertically centered text.
Verified by screenshot in the Apple TV simulator (home grid + Add Host dialog).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The inline iOS form widgets fought the tvOS focus system at every turn: focused fields
showed nested pills, rows darkened oddly and grew on activation, the Compositor picker
was not even focusable, and prefilled fields (port, client name) floated their label
inside the pill, shoving the value off-center.
- Settings is now a fully tv-native screen: NO inline text entry — the stream mode is
a preset picker (This TV native / 720p / 1080p / 4K, plus a Custom entry preserving
a mode set on another platform) and both pickers use .navigationLink style (pushed
selection lists, exactly like the system Settings app — and properly focusable; the
cover wraps in a NavigationStack for the pushes).
- Where text entry is unavoidable (Add Host, PIN pairing), the fields keep their stock
single-pill chrome (the grouped form style stays off tvOS — its row platters were
one of the nested pills) and prefilled fields hide their floating label so values
center vertically.
- All earlier row-clearing experiments reverted.
Verified by screenshot in the Apple TV simulator: Settings rows render as single
focus lozenges with chevrons; the Add Host pills are uniform with centered text.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tvOS forms/lists have CLEAR backgrounds and a fullScreenCover only shows what the
presented view paints, so Settings/Add Host/pairing rendered transparently over the
hosts grid. All three covers now sit on .thickMaterial edge to edge — the standard
tvOS blur-over-content panel look (verified in the Apple TV simulator).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The iOS chrome half-worked on tvOS: toolbar items rendered tiny with clipped labels
and could not even be focused (which is why "+" never opened the add-host form), and
sheet presentations are not a tvOS idiom (the Settings form looked broken).
- The toolbar is gone on tvOS. Add Host and Settings live IN the hosts grid as
full-size, focus-native tiles (.card style, same geometry as the host cards) — the
natural way actions work on television.
- Every modal (Add Host, Settings, PIN pairing) presents as a fullScreenCover on tvOS;
Settings gains a tvOS-only Done button (covers don't dismiss themselves).
- iOS/macOS keep their existing toolbar + sheets untouched.
Verified in the Apple TV simulator: title, host card and both action tiles render
full-size and focusable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Icon Composer doesn't cover tvOS — tvOS app icons are the older parallax format:
flat layers in an asset-catalog "App Icon & Top Shelf Image" brand asset. Generated
from the same Affinity layer exports the Icon Composer .icon uses, mirroring its
composition (violet automatic-gradient background → light circle → dark circle →
blob in front), via scripts/render-tvos-icon.swift (checked in for regeneration):
- App Icon.imagestack 400×240 @1x/@2x + App Icon - App Store.imagestack 1280×768,
four layers each so the focus engine gets real parallax depth.
- Top Shelf Image (1920×720) + Wide (2320×720) @1x/@2x as flat composites.
- ASSETCATALOG_COMPILER_APPICON_NAME = "App Icon & Top Shelf Image" on the tvOS
configs; verified on the Apple TV simulator home screen.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A bandwidth probe over the punktfunk/1 data path so a user-settable bitrate can be
informed by what the network actually sustains (throughput/loss/jitter), surfaced in
the client UI + web console. Reuses the existing Session/FEC plumbing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of "input doesn't work" on the unified host: a single fresh session
injects fine (EIS connects, "Gamescope Virtual Input" device added), but the
host-lifetime injector reused a STALE per-session EIS socket across sessions →
"connect EIS socket …: Connection refused". (Headless gamescope is EIS-only — it
ignores uinput — so libei/EIS is the one input path for both gamescope and KWin;
no second path needed.)
- connect_socket_file: re-READ the relay file and RETRY the connect on
refused/missing (the live gamescope's EIS appears shortly), bounded at 15s,
instead of connecting once and bubbling ECONNREFUSED.
- GamescopeProc::drop: clear the relayed EIS socket name on teardown so a dead
session can't hand a stale path to the next reconnect.
Validated: two sessions back-to-back each reconnect (EIS connected + device added).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The same app now runs on tvOS (target Punktfunk-tvOS, bundle io.unom.punktfunk.tvos),
validated live against the box: vkcube at 1280x720@60, 60 fps in the Apple TV 4K
simulator, glass HUD with a focusable Disconnect button.
- PunktfunkCore.xcframework grows tvOS device + universal-simulator slices. These are
TIER-3 Rust targets (no prebuilt std): BUILD_TVOS=1 builds them with nightly and
-Zbuild-std from rust-src — the full quic stack (quinn/rustls-ring/tokio) compiles
for tvOS unchanged.
- The UIKit stream view covers iOS AND tvOS, with pointer interaction, pointer lock,
touch forwarding and InputCapture gated to iOS — tvOS is view-only until gamepad
capture lands (the natural tvOS input).
- SessionAudio on tvOS: .playback session, no mic (no app-accessible microphone).
- App chrome gates: keyboardShortcut/textSelection/controlSize/statusBarHidden are
iOS/macOS-only; host cards use the focus-native .card button style on tvOS; the
Audio settings section hides (system-routed); mode seeding works from the TV screen
(1920x1080@60).
- Package platforms += .tvOS(.v17); new Xcode target + shared scheme
(TARGETED_DEVICE_FAMILY 3, local-network usage description included).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Working through the brand-color follow-ups:
- AccentColor gains a dark-appearance variant (#8678F5 — the brand violet lifted one
step toward the icon's light periwinkle) so tinted controls keep contrast on dark.
- Host cards remember sessions: StoredHost.lastConnected (set when a session reaches
streaming) renders as a "Connected … ago" relative-time line, and the most recent
host's card carries a subtle accent ring — the grid finally has hierarchy.
- The HUD swaps the pre-glass black-50% rectangle for .regularMaterial with an accent
live-dot; hint lines use semantic .secondary instead of opacity.
- Security moments: the trust card's lock.shield and the pairing sheet's header take
the brand tint; the PIN field is larger monospaced and uses the number pad on iOS.
Icon ↔ accent decision: the accent stays the exact brand #6656F2; the Icon Composer
layers keep their adjacent palette (#6C5BF3 family) — close enough to read as one
brand, and the icon remains the design-tool source of truth.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AccentColor color set + ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME on all four app
configurations — the platform-sanctioned global tint, so the host-card icons, prominent
buttons, toggles, pickers and links all carry the brand violet on macOS and iOS without
any per-view styling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 160 pt grid minimum packed five small cards per iPad row. iOS columns now use a
280 pt minimum (one full-width card on iPhone portrait, 3–4 generous cards on iPad)
and the card content scales with it: 56 pt icon, title3 name, taller padding. macOS
keeps its compact 180–240 pt cards.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Matches the bundle display name; was the lowercase project name "punktfunk" in the
home navigation title (iOS large title / macOS titlebar) and the WindowGroup title.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Streaming on iPad left the status bar up and the video boxed inside the safe areas, on
top of a 16:9 default mode letterboxing on the 4:3 screen, with the iPadOS cursor
hovering over the video. The session view is now immersive on iOS:
- .ignoresSafeArea + .statusBarHidden + .persistentSystemOverlays(.hidden) for the
session only (home gets its chrome back on disconnect).
- First run seeds the stream mode from the device's native screen
(UIScreen.nativeBounds + maximumFramesPerSecond) instead of 1920×1080 — verified
live: a fresh install negotiated the iPad's 2752×2064 with the host. macOS keeps the
1080p default (a desktop window is not the screen).
- The iPadOS cursor hides while over the video (UIPointerInteraction .hidden(),
re-resolved on capture toggles) — the host renders its own cursor from our deltas;
true pointer lock through UIHostingController remains the documented gap.
Found along the way (host-side, not fixed here): at very high modes a keyframe burst
can fill the UDP send buffer and m3 treats the sendmmsg WouldBlock as fatal
("session ended with error: submit_frame: WouldBlock") instead of backpressuring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
§8a (require native pairing by default, serve --open) shipped + deployed. §8b
(delegated approval) refined into §8b-1 (host pending-requests + mgmt endpoints +
web Approve/Deny — achievable now) and §8b-2 (peer push to a paired Device A —
needs the native/Apple client UI).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An open punktfunk/1 host any LAN device can trust-on-first-use and stream from is
insecure. The unified host now gates native sessions on pairing by DEFAULT: a client
must complete the SPAKE2 PIN ceremony (armed from the web console) before it's
admitted; paired devices persist. `serve --open` keeps the old TOFU behavior for
trusted single-user setups.
native_serve_opts now takes a NativeServe { port, require_pairing }; parse_serve
builds it with require_pairing = !--open. GameStream pairing (separate) is unchanged.
The require_pairing gate + ceremony are already covered by m3::pairing_ceremony_and_gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Icon Composer re-import after stripping the Affinity artboard rects (full-canvas
fill:none rects the exporter adds per layer) that caused rendering artifacts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Completes the web-UI native (punktfunk/1) pairing flow the unified host backs.
The Pairing page now leads with a native card that arms a window via the mgmt API
and DISPLAYS the host PIN (the SPAKE2 ceremony is host-mints / client-enters) with
a live countdown + Cancel, plus a paired-devices list with unpair — no journalctl.
The existing Moonlight PIN-submit moves into its own section below.
Uses the orval-generated `native` hooks (regenerated from the committed OpenAPI on
build) + en/de strings. Validated end-to-end through the web server's proxy + cookie
auth: login → status → arm (PIN shown) → clients. tsc + production build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "title looks off" report traced to the GRID, not the title: the Mac-tuned
adaptive(180–240) columns yielded a single max-width card, centered, so nothing aligned
with the leading large title. The header is now entirely stock primitives — default
.navigationTitle large-title behavior (the inlineLarge experiment is gone), default
.padding() so content sits on the system 16 pt margins — and the grid columns are
platform-tuned: iOS drops the max so columns FILL the width and the cards stay
edge-aligned with the title; macOS keeps the 180–240 cap (huge windows shouldn't grow
huge cards).
Verified in the iPhone 17 simulator with seeded hosts: pill top-right, large title at
system metrics, two full-width-filling cards flush with the title's leading edge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- README: replace the stale M0/M2-in-flight status with reality — M1 hardened, M2
GameStream host live to stock Moonlight, M3 punktfunk/1 validated, M4 Apple first
light, web console + unified host; FFmpeg 7/8; Bazzite-deployed. Layout adds
web/, packaging/, native_pairing, dualsense.
- CLAUDE: protocol-growth item now reflects the unified host + web-console native
pairing (done) and flags the next steps; layout updated.
- roadmap §7 Windows: de-risked via SudoVDA (the Sunshine Virtual Display Adapter) —
no self-signed kernel IDD needed; the virtual-display backend drops XL→M.
- roadmap §8 (new) Pairing & trust hardening: mandatory PIN pairing by default
(TOFU-open is insecure on a LAN) + delegated pairing approval (an already-paired
device approves a new one, no out-of-band PIN).
- windows-host.md: SudoVDA path throughout (status, table, phasing, effort M not L).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`serve --native` now runs the GameStream host AND the native punktfunk/1 (QUIC)
host in ONE process, sharing a single NativePairing handle with the management API
— so native pairing is operable from the web console instead of journalctl.
- gamestream::serve gains a native_port: spawns crate::m3::serve in the same
runtime and passes the shared NativePairing to mgmt::run. Validated live: one
process binds both RTSP 48010 and QUIC 9777.
- mgmt API: new `native` endpoints — GET /native/pair (status), POST
/native/pair/arm (mint a fresh, time-limited PIN to DISPLAY), DELETE /native/pair
(disarm), GET/DELETE /native/clients (list/unpair). GameStream-only hosts report
enabled:false. OpenAPI regenerated (checked-in doc + drift test).
- main.rs: serve --native / --native-port flags.
The native host arms pairing on demand (the operator reads the PIN from the
console; the SPAKE2 ceremony is host-shows-PIN). New mgmt + native_pairing tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Groundwork for web-UI-driven native (punktfunk/1) pairing. Replaces m3's fixed
startup PIN + local paired store with a shared `NativePairing` (new module):
arm-on-demand with a fresh, time-limited PIN (`arm(ttl)`), `current_pin()` read
per ceremony so a lapsed window stops pairing, plus the trust store (list/add/
remove/is_paired) and a `status()` snapshot. The management API (next commit) and
the QUIC accept loop share one handle. CLI `--allow-pairing`/`--require-pairing`
still arm at startup (no expiry, PIN logged) — back-compat. m3 pairing ceremony +
gate and the C-ABI roundtrip stay green; new unit tests for arm/expire/pair.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ToolbarSpacer split into separate circles was the wrong read — with the
inline-large title row in place, the expected header is the single grouped pill
(the system default for adjacent trailing items). Dropped the spacer and the
availability fork; the two trailing items now share one pill next to the title.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The home screen stacked the toolbar row above the large title; the modern (iOS 26
Liquid Glass) header puts the large title leading and the glass action circles trailing
on the SAME row. That's exactly .toolbarTitleDisplayMode(.inlineLarge) — applied on iOS
only, macOS keeps its window chrome untouched.
Verified in the iPhone 17 simulator: "punktfunk" large title left, gear/+ circles
right, one row.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The iOS chrome inherited macOS dialog sizing and read as undersized on a phone:
- Toolbar: the two trailing actions shared one compact glass pill; on iOS 26+ each now
gets its own full-size circle (explicit .topBarTrailing placements split by a fixed
ToolbarSpacer — the system-app look, e.g. Files), with the grouped-pill fallback on
iOS 17–18. The buttons are extracted so macOS keeps SettingsLink + .help untouched.
- Sheets and CTAs (AddHostSheet, PairSheet, trust card, empty-state Add Host) get
.controlSize(.large) on iOS — proper touch targets instead of macOS dialog buttons.
Verified in the iPhone 17 simulator: two ~44 pt glass circles matching the Files app's
toolbar sizing; macOS suite and app build unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The whole client now runs on iPadOS/iOS from the same sources, first-lit live in the
iPad simulator against the real host at 1280x720@60 (60 fps on the HUD, capture state
machine active, mic permission flow shown).
- PunktfunkCore.xcframework grows iOS device + universal-simulator slices
(BUILD_IOS=1; rustup targets aarch64-apple-ios{,-sim} + x86_64-apple-ios).
- The decode pump is extracted into a shared StreamPump (identical IDR re-gate logic on
both platforms); the iOS StreamView (StreamViewIOS.swift) has the same name/signature
as the macOS one, so ContentView & co. are byte-identical across platforms — hosted
in a UIViewController for prefersPointerLocked (the iPadOS cursor capture; see README
note 9 for the UIHostingController forwarding caveat).
- Touch is always forwarded: per-finger wire ids, coordinates mapped through the
aspect-fit letterbox into LIVE host-mode pixels (surface == host mode, identity
rescale host-side; follows mid-stream requestMode switches).
- InputCapture is cross-platform: GC works the same on iPadOS, ⌘⎋ is detected from the
HID stream there; stale-⌘ tracking after focus loss fixed on both platforms
(releaseAll now drops the modifier/latch state — a ⌘ released in another app
otherwise hijacked Esc forever).
- SessionAudio: AVAudioSession on iOS (.playAndRecord + .defaultToSpeaker — without it
iPhones route host audio to the EARPIECE; deactivated with
notifyOthersOnDeactivation on stop so interrupted background audio resumes); HAL
device pinning + the Settings pickers stay macOS-only.
- New Punktfunk-iOS app target (shared synchronized sources, generated Info.plist with
mic + local-network usage descriptions — QUIC to a LAN host trips local network
privacy on real devices — scene manifest + indirect input events for Stage Manager /
external displays), shared scheme, macOS min-window frames gated off iOS.
For the iPad-on-an-external-screen idea: with multiple scenes + indirect input enabled,
Stage Manager iPads can drag the punktfunk window onto the external display and drive
the PC with keyboard/mouse/touch. Known gaps (README note 9): the pointer-lock
preference isn't consulted through UIHostingController (relative mouse works, the local
cursor just stays visible) and AVAudioSession interruptions don't auto-restart audio.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
punktfunk-host builds unchanged against either FFmpeg 7.x (libavcodec 61) or 8.x
(libavcodec 62) — ffmpeg-sys-next auto-detects the system version, and the host's
ffmpeg FFI only touches long-stable APIs. Confirmed by building + running live on a
Bazzite F43 box (FFmpeg 7.1.3): full gamescope capture → zero-copy dmabuf→CUDA →
NVENC H.265 at 1280x720x60, p50 ~0.96 ms. Just doc/spec accuracy, no code change:
- encode/linux.rs + CLAUDE.md: drop the "FFmpeg 8 only" claim; note 7.x/8.x both work.
- rpm spec: add the missing zero-copy GPU build deps the link actually needs —
pkgconfig(gl) + pkgconfig(gbm) (mesa) — and document that -lcuda needs libcuda.so at
link time (NVIDIA host, or the CUDA toolkit stub on a headless COPR/koji builder).
Tracked for a proper fix: make the cuda/gbm/GL FFI dlopen-based like khronos-egl so
the RPM builds on a GPU-less host.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A step-by-step walkthrough for running the host on Bazzite (the immutable
Fedora-Atomic gaming distro): the two install paths (rpm-ostree layering vs the
bootc image), udev + the `input` group, host.env knobs (gamescope-default), the
systemd --user service, firewall ports, verification, and troubleshooting — all
grounded in the packaging/ files. Flags the operator-run COPR, the loopback-only
mgmt port, and that the bundled unit runs the GameStream `serve` host (not m3-host).
Linked from packaging/README.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PUNKTFUNK_GAMEPAD=dualsense now routes a session's gamepad through a real virtual
DualSense (UHID + hid-playstation) end to end:
- host: a `PadBackend` enum (m3.rs) selects `GamepadManager` (uinput xpad, default)
or the new `DualSenseManager` (dualsense.rs) per session. The manager keeps each
pad's full DsState so touchpad + motion (rich-input plane) persist across
button/stick frames, and services the !Send /dev/uhid fd only on the input thread
(which cycles <=4ms, so the GET_REPORT init handshake completes).
- feedback: `service()` now returns `DsFeedback { hidout, rumble }`. Motor rumble
stays on the universal 0xCA plane (so non-DualSense clients still feel it; manager
dedups change); lightbar / player LEDs / adaptive-trigger effects ride the new
0xCD HID-output plane (host->client) as `HidOutput`.
- rich input: touchpad contacts + motion ride the 0xCC plane (client->host) as
`RichInput`, applied via `DualSenseManager::apply_rich` (merged with button state;
touch normalized 0..65535 -> the touchpad resolution).
- connector + C ABI: `NativeClient::next_hidout` / `send_rich_input`, exported as
`punktfunk_connection_next_hidout` (-> PunktfunkHidOutput) and
`punktfunk_connection_send_rich_input` (<- PunktfunkRichInput); header regenerated.
- reference client: `--rich-input-test` drives the DualSense touchpad + motion and
logs the 0xCD feedback that comes back.
Validated live on-box: a synthetic-source m3-host + client-rs created the real
kernel DualSense, drove 0xCC, and decoded 12 live 0xCD events (the kernel's actual
lightbar/trigger init reports) with the data plane unaffected (600/600 frames).
Adversarial review fixes folded in: the input loop no longer skips the rich drain +
feedback pump on a dropped gamepad event, and the touch contact id is clamped to its
slot. Remaining: the Apple client renders triggers/rumble on a real DualSense.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
4-agent feasibility read converged on three independent walls, any one fatal:
- host capture needs a kernel rebuild (CONFIG_USB_DUMMY_HCD off → no UDC for an
f_uac2 composite gadget; everything else for the gadget IS present);
- near-zero Linux supply (only ~5-10 Proton titles via custom Wine patches emit
it; hid-playstation/Steam-Input/RPCS3 don't);
- Apple client can't faithfully replay PCM haptics (CoreHaptics is discrete
pattern-based; no public CoreAudio channel-3/4 routing).
Advanced haptics ride the DualSense USB *audio* interface, not HID, so the UHID
backend structurally can't carry them. Defer; the reachable 80% ("real DualSense
feel") is adaptive triggers over the HID 0x02 path we already parse + two-motor
rumble. New docs/dualsense-haptics.md records the walls + conditions for a future
go; roadmap §5 updated (HID DualSense backend built & live-validated).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both directions of the audio plane, on CoreAudio's built-in Opus codec
(kAudioFormatOpus — no bundled libopus; OpusCodec.swift, round trip unit-tested):
- Playback: a drain thread pulls nextAudio() packets, decodes, and writes a priming
jitter ring feeding an AVAudioSourceNode (~20 ms prefill, adaptive to the device's
render quantum so large-buffer devices don't oscillate prime/dropout; a high-water
clamp sheds stall backlog so one network hiccup can't permanently lag audio behind
video; underrun re-primes — one dip, not sustained crackle).
- Mic: a second engine taps the input device, resamples to 48 kHz stereo, Opus-encodes
20 ms chunks and sendMic()s them into the host's virtual PipeWire source. Permission
via AVCaptureDevice (NSMicrophoneUsageDescription added to the Xcode target).
- Settings: Speaker + Microphone pickers (CoreAudio HAL enumeration, persisted by
device UID — "System default" leaves the engine unpinned so it follows macOS device
changes) and a "Send microphone" toggle (default on). Applies from the next session.
- Audio starts with streaming, never during the trust prompt (no host sound — and no
mic uplink — before the user trusted the host); teardown stops audio before close().
Adversarial-review fixes baked in: stop() and the dangling mic-permission callback
share one lock+flag protocol (no hot mic with no owner), the connect-success handler
bails when the attempt was abandoned mid-handshake (no session/mic for a dead window),
SessionAudio gets a deinit backstop (a dropped instance can't pin the connection via
its drain thread), and the render scratch buffer is block-owned (was leaked per
session).
Verified live against the box: remote test decodes 100 host Opus packets to PCM and
the host opens its virtual mic on the first uplinked frame ("punktfunk/1 virtual mic
ready"); on-glass session runs with both engines up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Roadmap #5 (rich DualSense). A UHID device presents a real Sony DualSense to the kernel's
hid-playstation driver (matched by VID 054C/PID 0CE6), which exposes the full controller —
gamepad, motion sensors, touchpad, lightbar/player LEDs, adaptive triggers — unlike the
uinput X-Box-360 pad.
- inject/dualsense.rs: hand-rolled /dev/uhid codec (no bindgen) mirroring the uinput style;
the canonical inputtino 232-byte USB HID report descriptor + the feature-report replies
(calibration 0x05 / pairing 0x09 / firmware 0x20) — answering hid-playstation's GET_REPORTs
during init is REQUIRED or it creates no input devices. DsState::from_gamepad maps a
GameStream/XInput frame → the DualSense input report (buttons/sticks/triggers/dpad, +
touchpad/motion fields); service() answers GET_REPORTs and parses HID OUTPUT (rumble /
lightbar RGB / player LEDs / adaptive triggers) into quic::HidOutput.
- scripts/60-punktfunk.rules: grant /dev/uhid to the 'input' group (like /dev/uinput).
- `punktfunk-host dualsense-test`: standalone validation (no streaming session).
Validated live: `dualsense-test` → hid-playstation binds + loads ff_memless + led_class_
multicolor; the kernel creates "Punktfunk DualSense 0" (event/js gamepad + Motion Sensors +
Touchpad + Headset Jack) at VID 054c/PID 0ce6, plus the lightbar at /sys/class/leds/
input*:rgb:indicator; js shows the Cross button firing + the left-stick sweep. Clippy/fmt
clean, workspace tests green. Wiring into the session (pad-type select, touchpad/motion
routing, HID-output back-channel) is the next commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adopts the new ABI surface (still v2, additive):
- PunktfunkConnection.sendMic(_:seq:ptsNs:) — Opus mic frames (48 kHz) to the host's
virtual PipeWire source; enqueue-only, empty data = DTX silence. Wiring the actual
Mac microphone (AVAudioEngine input → Opus) into the app is the follow-up, alongside
audio playback (README note 5).
- PunktfunkInputEvent.touchDown/touchMove/touchUp — absolute pixels + surface size in
flags, host injects via libei ei_touchscreen. Built for the iOS variant; nothing on
macOS emits them yet.
- Loopback round trip now also sends touch events and mic frames (incl. a DTX frame)
through the wrapper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The new features were Linux-built only and broke the documented macOS gate
(cargo build/test/clippy --workspace) four ways, all fixed following the existing
platform-gating conventions:
- m3.rs: mic_service_thread split into the Linux worker and a non-Linux stub that
drains and drops (sessions still count the datagrams) — opus/PipeWire are
Linux-gated deps, same pattern as audio_thread.
- punktfunk-client-rs: the new `opus` dependency moved into the Linux target table and
--mic-test gated with a warn-and-skip stub (only the synthetic-tone test rig needs
the encoder; the mic uplink itself is portable).
- gamestream/audio.rs: SAMPLE_RATE import gated to any(linux, test) (the frame_sizing
test uses it everywhere, the data plane only on Linux).
- tests/c_abi.rs: the harness's macOS link flags gained Security + CoreFoundation —
the quic feature now pulls rustls's platform verifier into the staticlib.
Also: two clippy match-ref-pats lints in the new rich-input/HID-output decoders
(clippy -D warnings is the repo gate), the regenerated punktfunk_core.h committed (the
checked-in copy predated the rich-input/HID-output constants — CI fails on drift), and
web's inlang cache dir gitignored.
cargo build/test/clippy/fmt --workspace: green on macOS, 122 tests passing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Foundation for rich DualSense support (roadmap #5). The fixed 18-byte InputEvent (0xC8) can't
hold the DualSense touchpad/motion or HID feedback, so two new variable-length, kind-tagged
datagram families join the side-plane (mouse/keyboard/gamepad/touch keep the fixed InputEvent):
- RICH_INPUT_MAGIC 0xCC, client→host: `[0xCC][kind][fields]`
Touchpad{pad,finger,active,x,y} (x/y normalized 0..65535; host scales to the pad)
Motion{pad, gyro[3], accel[3]} (raw i16, straight into the DualSense report)
- HIDOUT_MAGIC 0xCD, host→client: `[0xCD][kind][pad][fields]` — the rich analog of the 0xCA
rumble datagram (rumble stays on 0xCA):
Led{rgb} PlayerLeds{bits} Trigger{which, effect} (adaptive-trigger params to replay)
`RichInput`/`HidOutput` enums with encode/decode; unknown kinds + truncation decode to None
(forward-compatible). +2 round-trip/disjointness tests; quic suite green, clippy/fmt clean.
Wiring (host UHID device, capture, C ABI, client) lands in following commits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Roadmap #5 (touch, ahead of the XL UHID DualSense work). Touch fits the existing 18-byte
InputEvent: code = touch id, x/y = client pixels, flags = (w<<16)|h — the same absolute
mapping as MouseMoveAbs.
- core: InputKind::{TouchDown=9, TouchMove=10, TouchUp=11} + from_u8 + roundtrip test.
- host inject/libei.rs: request the RemoteDesktop Touchscreen device type, bind the Touch
capability, and inject ei_touchscreen down/motion/up (one event = one frame, per the
protocol rule), mapping coordinates into the device region like the abs pointer. wlroots
has no virtual-touch protocol wired — no-ops there.
- client-rs --touch-test: drags a synthetic finger (touch id 0) in a circle.
Validated live on headless KWin: the portal GRANTS the Touchscreen device type
(Keyboard|Pointer|Touchscreen), proving the request path — but KWin's EIS server creates no
touchscreen *device*, so touch currently no-ops on this KWin (now logged once, not silent).
The injection code is correct and will land on a backend that exposes ei_touchscreen
(gamescope / a newer compositor / the real touch-client path). Workspace green, clippy/fmt
clean, +1 unit test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A 4-agent read of the host crate: a Windows host is an "add a backend" job, not a parallel
port — ~95% reuse (core/protocol/FEC/crypto/C-ABI, QUIC, GameStream, mgmt, m3/pipeline are all
platform-agnostic and already cfg-isolated). New cfg(windows) backends behind the existing
traits: DXGI Desktop Duplication (capture), Media Foundation / NVENC-SDK (encode), SendInput +
ViGEm (input), WASAPI loopback + virtual mic (audio). The blocker is the virtual-display
feature — no user-mode Windows API; it needs a signed kernel-mode IDD driver (XL).
docs/windows-host.md records the per-subsystem effort + a phased plan (Phase 0 = a "basic
Windows host" capturing an existing monitor, smallest surface). Deferred: large and unbuildable
on the Linux dev box, per the request to only take it on if manageable. roadmap.md marks
#1/#2/#4 done, #3 packaged, and adds #7 Windows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Roadmap #3 (install on other devices). Bazzite already ships gamescope + PipeWire + the
NVIDIA stack, so the host slots in with minimal new deps (ffmpeg-libs from RPM Fusion + opus
+ libei).
- packaging/rpm/punktfunk.spec — builds punktfunk-host from source (cargo), installs the
binary + udev rule + systemd user unit + headless helpers; Requires/Recommends mapped from
the Ubuntu bootstrap deps to Fedora.
- packaging/bootc/Containerfile — layer punktfunk into a bazzite-nvidia bootc image for
atomic, image-based installs.
- packaging/bazzite/host.env — gamescope-default appliance config (spawned per session).
- packaging/copr/ + packaging/README.md — COPR build-from-SCM settings + install docs
(rpm-ostree and bootc paths), and why not Flatpak.
- LICENSE-MIT + LICENSE-APACHE — materialize the declared `MIT OR Apache-2.0` (was unfiled);
the RPM ships them.
Not buildable on the Ubuntu dev box (no rpm tooling) — the COPR/Fedora build is operator-run;
all spec-referenced files verified present and the cargo build is green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a
new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire
Audio/Source its apps can record from (voice chat, etc.).
Protocol (punktfunk-core):
- MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram).
- NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task
(mirror of send_input); C ABI punktfunk_connection_send_mic for native clients.
Host:
- audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class=
Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM.
- MicService: host-lifetime owner of the source + Opus decoder (mirror of
InjectorService / the audio capturer slot); lazily opened, persists across sessions,
self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a
single read_datagram loop (two loops would race).
- Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting,
so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output
was silence; with it, glitch-free across consumer quanta.
Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic
uplink (opus dep added) for end-to-end validation without a real microphone.
Validated live on headless KWin: client tone → host source → pw-record shows the
punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001
elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms
consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A full Plasma login starts several pieces our bare headless session was missing, which
surfaced as three separate failures while streaming the KDE desktop:
- Steam (and other X11 apps) failed "can't open display": Xwayland runs, but KWin only
sets DISPLAY for its own children — apps launched via the plasma menu / D-Bus activation
never saw it. Detect the Xwayland display after KWin is ready and export it into the
systemd/D-Bus activation environment.
- Discover / PackageKit couldn't install apps: polkitd (the policy engine) was running but
no authentication *agent* (the prompt) was — so privileged installs got no authorization.
Start polkit-kde-authentication-agent-1 (forcing the Qt Wayland platform, or it exits).
- The streamed desktop showed app windows but no wallpaper/panels: plasmashell had crashed
and the old unsupervised `plasmashell &` never brought it back. Supervise it — restart for
as long as KWin lives, so the desktop shell self-heals.
Validated live on this box: DISPLAY=:0 now in the --user environment (xdpyinfo on :0 works),
the polkit agent registers ("Listener online"), and plasmashell stays up under the supervisor.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Under rapid client reconnects, KWin's libei/EIS input setup intermittently wedged
with "EIS setup timed out", causing total input loss for affected sessions. Root
cause: each punktfunk/1 session opened (and tore down) its own RemoteDesktop-portal
CreateSession for pointer/keyboard injection, and back-to-back reconnects raced a
prior session's portal teardown before it settled.
LibeiInjector is only a Send channel handle to a worker thread that owns the portal
session, so the injector can live for the whole host run instead of per session.
Adds InjectorService: one host-lifetime thread owns the (!Send) injector, opened
ONCE (lazily, on the first event) and reused across every session — the portal grant
is established a single time and held. Sessions forward pointer/keyboard events to it
over a clonable Send channel; gamepads stay per-session (uinput, no portal). The
service self-heals — reopen after a 2s backoff if open fails or the backend worker
dies (covers a gamescope EIS socket that respawns with its nested session).
Mirrors the existing host-lifetime audio-capturer slot; the audio capturer is Send
(a slot works), the injector is !Send (needs the owning thread + channel).
Validated live on headless KWin: 8 rapid back-to-back input sessions →
"input injector ready (host-lifetime)" exactly once, ZERO "EIS setup timed out",
8/8 sessions injected input. Tests green, clippy/fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Scroll was wired to GCMouse's scroll dpad, which only fires for plain HID wheel
deltas — trackpad and Magic Mouse scrolling are gesture events that never reach
GameController, so scrolling was dead on the default Mac setups. The stream view now
overrides scrollWheel (while captured the cursor is parked mid-view, so it receives
every scroll event) and feeds InputCapture.sendScroll: precise gesture deltas are
pixels (~0.1 notch/px, SDL's factor → ×12 for WHEEL_DELTA(120)), classic wheels are
lines (×120), fractional remainders accumulate, and the GC scroll handler is gone so
wheel mice can't double-deliver. Signs pass through as-is, preserving the local
(natural-)scrolling preference.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adopts punktfunk_connect_ex from the compositor-selection batch: a Compositor enum on
PunktfunkConnection (auto/kwin/wlroots/mutter/gamescope, with the host's name aliases
for env parsing), a "Host compositor" picker in Settings (default Automatic — a
concrete choice is honored only if that backend is available host-side), and
PUNKTFUNK_COMPOSITOR / PUNKTFUNK_REMOTE_COMPOSITOR pass-throughs for the autoconnect
dev hook and the remote first-light test. The wire change is backward-compatible
(optional trailing byte), so no behavior changes at the default.
Validated live against the box: host with no compositor env (auto-detect = KWin)
logged "honoring client compositor request compositor=gamescope" and streamed 60/60
decoded frames from the spawned gamescope.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A client can now request which compositor backend the host drives its virtual
output on (gamescope/KWin/Mutter/wlroots). The host honors the request if that
backend is available, else falls back to auto-detect and reports the resolved
choice back — wire-compatible both directions (no ABI bump).
Protocol (punktfunk-core):
- New CompositorPref (config.rs): Auto|Kwin|Wlroots|Mutter|Gamescope with
u8/name mappings. Appended as one optional byte to Hello (client preference)
and Welcome (host's resolved choice). Both decoders already tolerate trailing
bytes, so old↔new interop is preserved — ABI_VERSION stays 2. Round-trip +
back-compat (truncated-message) tests.
- C ABI: punktfunk_connect_ex(compositor) + PUNKTFUNK_COMPOSITOR_* constants;
punktfunk_connect delegates with AUTO, so the existing symbol is unchanged.
NativeClient::connect / worker_main thread the preference through.
Host:
- vdisplay::available() enumerates usable backends via cheap, side-effect-free
probes (KWin zkde global, gamescope binary+version, GNOME/Sway env), plus
Compositor id/label/as_pref/from_pref/all helpers.
- m3 handshake resolves the preference to a concrete backend during the
handshake (pick_compositor pure + resolved logging), reports it in Welcome,
and threads it into virtual_stream (replacing the unconditional detect()).
- mgmt GET /v1/compositors lists every backend with availability + the
auto-detected default (OpenAPI regenerated).
Client:
- punktfunk-client-rs --compositor NAME; logs the host's resolved choice from
the Welcome ("session offer … compositor=…").
Web console:
- Host page gains a Compositors card (availability + default badges) via the
codegen'd useListCompositors hook; en/de strings added.
Also fixes a pre-existing, env-dependent test-isolation bug:
mgmt::tests::paired_clients_list_and_unpair seeded the real
~/.config/punktfunk/paired.json (AppState::new loads it), so a real
GameStream-paired client leaked into body[0] on a dev box — now cleared first.
Live-validated against headless KWin: --compositor kwin honored, --compositor
mutter falls back to kwin (available=[kwin, gamescope]), resolved choice
round-trips to the client. Tests: +6 (wire/back-compat, resolution precedence,
endpoint); workspace green, clippy/fmt clean, C ABI harness PASS at abi_version=2,
web typecheck + build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hardens the virtual-display → capture → encode bring-up against the transient
failures that surfaced as black screens / wrong refresh on cold KDE sessions.
- m3: build_pipeline_with_retry wraps the initial vd.create() + first-frame with
bounded exponential backoff (4 attempts, 500ms→2s). is_permanent_build_error
classifies config/version/missing-tool failures so they fail fast instead of
burning the retry budget. Encoder + frame clock now pace to the *achieved*
refresh reported in VirtualOutput::preferred_mode, not the requested rate.
- capture/linux: PortalCapturer::Drop sends a pipewire channel quit and joins the
thread, so a dropped/failed/retried capturer releases its PipeWire thread + EGL/
CUDA context promptly instead of leaking it to process exit. First-frame timeout
now reports the node id and distinguishes "format never negotiated" from
"negotiated but no buffers arrived" via a negotiated flag set in param_changed.
- vdisplay/kwin: set_custom_refresh reads back the active mode from kscreen-doctor
and returns the refresh KWin actually gave us (a rejected custom mode silently
leaves the output at 60Hz); create() carries it into preferred_mode.
- vdisplay/gamescope: find_gamescope_node requires the Video/Source object (the
node.name=gamescope tag is on two objects; the other wedges the link); a version
check warns on <3.16.22 (the PipeWire-1.6 capture-deadlock signature).
Live-validated against headless KWin: 720p120 build with requested=120 achieved=120,
zero-copy CUDA frames, and no per-session thread accumulation across back-to-back
sessions. Tests: +3 unit (retry classifier, gamescope version parse); 49 host tests
green, clippy/fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture used to engage whenever the app became active, so the click that activates the
window — on the title bar (a drag) or a resize edge — got the cursor warped away
mid-gesture, and raw deltas kept streaming to the host while the user fought the window.
Reworked Moonlight-style, with capture as a deliberate, reversible state owned by
StreamLayerView:
- Engage: automatically once when the stream starts / trust is confirmed (one-shot, can
never fire surprisingly later), or by clicking into the video (that click's
press/release are suppressed toward the host; acceptsFirstMouse makes it one click
from another app). NEVER on app re-activation.
- Release: ⌘⎋ (toggles, key-window-scoped), focus loss — now including same-app window
switches (⌘, / ⌘N / ⌘M resign key without resigning the app; previously the new
window inherited a hidden frozen cursor and its typing was double-delivered to the
host) — and disconnect.
- While released: nothing is forwarded (InputCapture.forwarding gates the GC handlers;
held keys/buttons are flushed host-side so nothing sticks), the cursor is free, and
the HUD (now showing the capture state) is clickable.
- The no-beep behavior moved from the NSEvent monitor to first-responder key
consumption — swallowing at the monitor risked starving GC's own delivery (the
"input broken altogether" report). The monitor now only intercepts ⌘⎋.
- Adversarial-review fixes: a second session preempts the previous one cleanly instead
of leaving it captured with dead GC handlers (onPreempted); the engage click's
suppression latch can't outlive the click (mouseUp backstop); ⌘⎋'s physical Esc can't
type into the host in either toggle direction (suppressedVK latch + Esc-while-⌘
guard); capture callbacks defer out of the SwiftUI update pass.
Validated live against the box: 16185 input datagrams injected during a captured
session (gamescope EIS), title-bar drag/resize free while released, and visible
cursor + typing on a streamed KWin desktop, all user-confirmed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GCKeyboard reads the HID state directly, so the key NSEvents kept traveling the
responder chain unhandled — and an unhandled keyDown makes NSWindow play the
"invalid input" sound on every keystroke. InputCapture now installs a local event
monitor for its lifetime that swallows key events, except ⌘-combos, which still
reach the local app (the HUD's ⌘D disconnect, ⌘Q) in addition to the host.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pairing/renegotiation batch bumped the punktfunk/1 ABI to v2 and the host now
hard-rejects v1 Hellos (m3.rs), so streaming from the Mac was dead until the bundled
PunktfunkCore.xcframework is rebuilt — it is gitignored, so that is a per-checkout step:
bash scripts/build-xcframework.sh. The Swift wrapper itself was already adapted upstream;
this lands the app on top of it.
- ClientIdentityStore: persistent client identity in the login Keychain, presented on
every connect so paired hosts recognize this Mac. Keychain access failure throws
instead of regenerating (a fresh identity would silently un-pair this Mac from every
--require-pairing host); a lost first-run race resolves toward the stored identity;
pairing uses the strict loadForPairing() so a memory-only identity can't strand a
ceremony.
- PairSheet: the SPAKE2 PIN ceremony, reachable from a host card's context menu and from
the trust prompt's "Pair with PIN instead…" (which drops the live session first — the
host's accept loop is sequential). Success pins the verified fingerprint and connects;
an in-flight ceremony self-discards when the sheet is dismissed, so a late success
can't pin + auto-connect behind the user's back. Wrong PIN and Keychain failures get
distinct, actionable error text.
- Tests: identity unit tests; the full pairing ceremony + --require-pairing gate on
loopback (test-loopback.sh arms a second host, parses its PIN from the log, and gives
both hosts throwaway config homes — no more writes to the real ~/.config/punktfunk);
remote pairing + pinned stream over the LAN (PUNKTFUNK_REMOTE_PIN, _PORT).
Validated live against the box: SPAKE2 ceremony with the host's arming PIN → verified
fingerprint → pinned + identified 720p60 session (host persisted the client identity);
first light 60/60 AUs decoded to pixels; vkcube on glass through the app.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A synchronous systemctl try-restart of the portal chain (xdg-desktop-portal is Type=dbus,
waits for its bus name) blocked the script ~30-40s before plasmashell started. --no-block
queues the restart and returns immediately — the portal only needs to be ready before the
first client streams (seconds later), not before plasmashell. Validated: plasmashell up in
1s (was ~30s); a virtual capture session against the fresh session streamed 720/720 frames
@720p120, zero-copy CUDA, no black screen.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Headless KDE startup was a chain of timing-sensitive handoffs gated by a blind `sleep 2`,
the dominant source of black screens. Phase-1 fixes:
- New `punktfunk-host probe-compositor` subcommand: exits 0 iff the detected compositor is
up AND ready to create a virtual output now. KWin gets a real check (connect + registry
roundtrip + the privileged zkde_screencast global must be advertised — what the backend
needs); gamescope/Mutter/wlroots create on demand so the probe just confirms Linux.
(vdisplay::probe dispatcher + kwin::probe; reuses kwin.rs's existing roundtrip path.)
- run-headless-kde.sh: replace `sleep 2` with an active readiness wait (poll probe-compositor
until ready, 30s deadline, and bail with kwin's log if kwin_wayland exits during init).
Move the portal restart to AFTER readiness, and precede it with `systemctl --user
import-environment` + `dbus-update-activation-environment` (the missing env import — the
Sway script does this; without it a restarted portal inherits a stale/empty WAYLAND_DISPLAY,
which is the "streams but eats no input/audio" failure). kwin's stderr → a log file.
Validated: probe-compositor exits 0 "Kwin ready" against the live session, exit 1 with a
clear diagnostic when the compositor is absent. 114 tests green, clippy/fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler