Bazzite (and SteamOS-like hosts) run Steam Big Picture inside their OWN
gamescope-session-plus session. Nesting a second gamescope+Steam can't work — the
second Steam sees the first and exits, taking the nested gamescope down with it
(crash in its exit handlers), killing both video and input. The robust model is to
let punktfunk OWN that session: run gamescope-session-plus headless at the client's
resolution (full Steam Deck UI polish: MangoApp, VRR, controller config) and have
the host ATTACH to it rather than spawn its own.
The video half already existed (PUNKTFUNK_GAMESCOPE_NODE=<id> attaches to a
PipeWire node). This finishes it:
- PUNKTFUNK_GAMESCOPE_NODE=auto discovers the gamescope Video/Source node, so the
(dynamic) node id needn't be hand-wired.
- The attach path now also points the libei injector at the running session's EIS
socket: find_gamescope_eis_socket() scans XDG_RUNTIME_DIR for gamescope-<N>-ei,
connect()-probes each (stale dead-session sockets refuse), and writes the newest
live one to the relay file the injector reads. So input reaches the attached
session with zero manual config.
scripts/punktfunk-steam-session.service: a systemd --user unit that runs
gamescope-session-plus headless at a configured resolution, with the one-time
headless-appliance setup (linger + multi-user.target) documented inline.
Validated live on bazzite (RTX 4090): the full Steam Big Picture session streams
(1499 frames, p50 ~1ms) with mouse/keyboard injected into it (device resumed, all
caps, emitted=true), node + EIS socket both auto-detected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
UdpTransport sockets are non-blocking, so a momentarily-full kernel send buffer
makes socket.send return WouldBlock (EAGAIN). submit_frame propagated that as a
fatal error, tearing the whole punktfunk/1 session down — observed when attaching
to an already-running source (a headless Steam session) that emits frames at full
rate the instant capture connects: the first burst saturates the tx queue and the
session dies before a single frame reaches the client.
The data plane is lossy + Leopard-FEC-protected and runs infinite-GOP with RFI
keyframes, so the real-time-correct response to a full tx queue is to DROP the
packet (the next frame / FEC recovers) — exactly what the recv path already does
for WouldBlock. Blocking would queue stale frames and add latency. Loopback/M1
paths are unaffected (LoopbackTransport never blocks; M1 tests stay green).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The PIN now surfaces in the host's web admin UI (port 3000 → Pairing), which is where
users will actually read it — the pairing sheet's footer, field prompts, the tvOS
keyboard title, and the wrong-PIN/failure errors all reference the console instead of
the host log / --allow-pairing flag (the log mention stays in the README as the
secondary path).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Host-side logs proved the macOS client sent keyboard + scroll but ZERO relative
mouse-motion and ZERO button events for an entire session — the user was moving
the mouse the whole time. Root cause is client-side: GCMouse's
mouseMovedHandler/pressedChangedHandler silently never fired on the live Mac
(a documented GameController quirk) while GCKeyboard worked and scroll already
rode NSEvent. So motion/buttons were the only input on a GCMouse-only path, and
that path was dead.
macOS: stop relying on GCMouse for motion/buttons (compiled out with
#if !os(macOS)); drive them from a local NSEvent monitor installed only while
captured — the same channel scrollWheel already uses successfully. Under
CGAssociateMouseAndMouseCursorPosition(false) the mouseMoved/dragged deltaX/deltaY
ARE the relative motion (OS-acceleration-applied, exactly what Moonlight's macOS
client ships). All four motion event types are covered so motion keeps flowing
during a button-held drag; buttons map left/right/middle/X1/X2 through the
existing engage-click-suppression + release-on-blur logic. NSEvent deltaY is
already screen-space (+y down) so, unlike the GCMouse path, it is NOT negated.
iPad: the input failure there was a different cause — GCMouse only delivers
relative deltas while the scene holds a true pointer LOCK, which the system grants
only to a full-screen, frontmost iPad scene and which UIHostingController doesn't
consult for children. Gate prefersPointerLocked to iPad + captured, add
childViewControllerForPointerLock so a reparenting container forwards the lock
decision to this VC, and log the resolved lock state. Touch remains the
unconditional fallback.
Adds a PUNKTFUNK_INPUT_DEBUG=1 switch (os.Logger, throttled) so motion/buttons
being SENT is verifiable on-device without host-side logs. iOS GCMouse path
otherwise unchanged; GCKeyboard unchanged on both. Researched + adversarially
reviewed; Swift builds only on a Mac, so this is unverified-compiled here.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The slide now runs on UISpringTimingParameters (stiffness 300, damping 30 — a ~0.87
damping ratio: settles quickly with a hint of life, no overshoot ping-pong) via the
transition library's .interpolatingSpring animation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The navigationLink Picker's INTERNAL destination list renders its rows in the focused
(dark-text) style while the push animates — black text over the dark backdrop until
focus settles (present under the old fade too; a SwiftUI-on-tvOS quirk we don't
control). Settings now uses its own primitives instead:
- TVSelectionRow: label + current value, pushes…
- TVSelectionList: a Settings-app-style option list (plain button rows + checkmark,
selecting pops back) — ordinary button chrome, no focused-style pre-rendering.
The stream-mode and compositor pickers are gone on tvOS; the Settings screen itself is
a plain scroll of rows + footer (no Form), matching the rest of the tv UI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SwiftUI's NavigationStack on tvOS animates pushes as a bare crossfade with no public
customization — the system Settings app slides. The home stack now applies
.customNavigationTransition(.slide) on tvOS via davdroman/swiftui-navigation-transitions
(MIT, tvOS 13+), covering the top-level routes AND the settings pickers' drill-ins.
The dependency is referenced by the Xcode PROJECT only and linked solely by the
Punktfunk-tvOS target: its manifest (no macOS platform declared vs 10.15 deps) breaks
SwiftPM whole-graph validation for plain `swift build`, and the #if os(tvOS) import
never compiles in the macOS-only SwiftPM dev shell anyway. Headless builds need
xcodebuild -skipMacroValidation (the lib pulls Swift macro packages; in the Xcode UI
it's a one-time Trust & Enable prompt).
iOS/macOS keep their untouched system navigation animations.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add Host, Settings and PIN pairing were fullScreenCover overlays, which is why
navigating felt unlike the system Settings app (no push animation, no Menu-pops-a-level
semantics). They are now navigationDestination ROUTES pushed inside the home
NavigationStack:
- the system push/pop animation and Menu-button back navigation come for free;
- the Settings pickers' navigationLink pushes reuse the same stack (its inner
NavigationStack wrapper is gone, as is the tvOS Done row — Menu pops, like Settings);
- Add Host is a real full-screen page (system navigation title, Settings-style rows on
the standard backdrop) instead of a floating dialog, same for the pairing page;
- the thickMaterial cover backdrops became unnecessary and are gone. The system
keyboard entries stay as covers — that presentation is system-owned either way.
iOS/macOS keep their sheets. Verified by screenshot: Add Host renders as a pushed
full-screen route with the title top-center.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The host-lifetime libei injector could connect to a gamescope EIS socket whose
listen socket exists but whose server never drives the EI handshake — a stale
socket left by a SIGKILLed prior session, or one created early in a new
gamescope's startup before its libei server is ready. `UnixStream::connect` to a
socket *file* succeeds the moment the path exists, so the worker sailed past the
connect and then hung forever in `handshake_tokio` (or sat connected with no
device ever resumed). Because `LibeiInjector::inject` only enqueues onto a
channel (the !Send worker owns the connection), the send never errors, so
InjectorService never noticed the dead worker and never reopened — every input
event for the whole session was silently swallowed. The 30s setup timeout didn't
help: a typical session ends first, so input just died with no error logged.
Reconnecting made it worse (more stale sockets to land on).
Two self-heal bounds, both paths (gamescope socket + KWin/GNOME portal):
- Bound the EI handshake at 8s — a non-responding EIS server now errors instead
of hanging, so the worker exits and the next inject() reopens.
- Watchdog: if no input device resumes within 5s of connecting, treat the
connection as dead-on-arrival and exit (same reopen path). Healthy servers
add+resume a device within a beat of the handshake.
Verified on-box: clean gamescope + KWin paths connect/resume/emit unchanged; a
stale listener that accepts-but-never-handshakes now errors in 8s; two
back-to-back gamescope sessions both inject (session 2 reopens against the fresh
socket). Independently confirmed end-to-end delivery on KWin — a focused wev got
the injected motions/keys/buttons — i.e. injection itself was never broken, only
its recovery from a bad connection.
Also adds permanent low-volume diagnostics so the next "input dead" report is
instantly triageable: log each EIS device's capabilities on resume, the first of
each InputKind a client sends + whether it emitted, and no-resumed-device drops.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SwiftUI's inline TextField on tvOS is structurally wrong for television: it grows when
activated, shows a full-width editing surface behind the pill, and floats labels
off-center — none of it stylable into the Settings-app look. Per Apple's tvOS text
input guidance, real tvOS apps never edit inline: a field is a value ROW, and pressing
it raises the SYSTEM fullscreen keyboard.
- TVTextEntry (UIViewControllerRepresentable): a UITextField that becomesFirstResponder
on appear, presenting the standard tvOS fullscreen keyboard with the field's prompt;
done/dismiss commits the text. TVFieldRow is the Settings-style label+value lozenge.
- Add Host and PIN pairing on tvOS now use rows + keyboard covers exclusively (the
port row also fixes the off-center value text for good — it's a Text, not a field);
the port input validates 1...65535.
- No SwiftUI TextField remains in any tvOS code path.
Verified by screenshot: the dialog rows render exactly like the Settings app, and the
address row raises the system linear keyboard with prompt + done.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three more tvOS-isms, all the same lesson — let the focus engine own the chrome:
- Host cards drew their own material platter + accent ring INSIDE the .card button
style, muting the native grow/tilt focus motion. On tvOS the card style now owns the
platter outright (material/ring stay on the pointer platforms), and the grid gets
48 pt spacing so the focused card swells without overlapping siblings.
- Add Host and Settings no longer sit in the hosts row: they're a compact button row
below the grid (and the empty state gains a Settings button, since tvOS has no
toolbar).
- The Add Host and pairing dialogs drop Form entirely on tvOS — list rows added a
full-width focus fill plus a row platter behind every field's own pill (the
"second outer pill"). As standalone fields in a centered dialog over the dimmed
home, each input is exactly one pill with vertically centered text.
Verified by screenshot in the Apple TV simulator (home grid + Add Host dialog).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The inline iOS form widgets fought the tvOS focus system at every turn: focused fields
showed nested pills, rows darkened oddly and grew on activation, the Compositor picker
was not even focusable, and prefilled fields (port, client name) floated their label
inside the pill, shoving the value off-center.
- Settings is now a fully tv-native screen: NO inline text entry — the stream mode is
a preset picker (This TV native / 720p / 1080p / 4K, plus a Custom entry preserving
a mode set on another platform) and both pickers use .navigationLink style (pushed
selection lists, exactly like the system Settings app — and properly focusable; the
cover wraps in a NavigationStack for the pushes).
- Where text entry is unavoidable (Add Host, PIN pairing), the fields keep their stock
single-pill chrome (the grouped form style stays off tvOS — its row platters were
one of the nested pills) and prefilled fields hide their floating label so values
center vertically.
- All earlier row-clearing experiments reverted.
Verified by screenshot in the Apple TV simulator: Settings rows render as single
focus lozenges with chevrons; the Add Host pills are uniform with centered text.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tvOS forms/lists have CLEAR backgrounds and a fullScreenCover only shows what the
presented view paints, so Settings/Add Host/pairing rendered transparently over the
hosts grid. All three covers now sit on .thickMaterial edge to edge — the standard
tvOS blur-over-content panel look (verified in the Apple TV simulator).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The iOS chrome half-worked on tvOS: toolbar items rendered tiny with clipped labels
and could not even be focused (which is why "+" never opened the add-host form), and
sheet presentations are not a tvOS idiom (the Settings form looked broken).
- The toolbar is gone on tvOS. Add Host and Settings live IN the hosts grid as
full-size, focus-native tiles (.card style, same geometry as the host cards) — the
natural way actions work on television.
- Every modal (Add Host, Settings, PIN pairing) presents as a fullScreenCover on tvOS;
Settings gains a tvOS-only Done button (covers don't dismiss themselves).
- iOS/macOS keep their existing toolbar + sheets untouched.
Verified in the Apple TV simulator: title, host card and both action tiles render
full-size and focusable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Icon Composer doesn't cover tvOS — tvOS app icons are the older parallax format:
flat layers in an asset-catalog "App Icon & Top Shelf Image" brand asset. Generated
from the same Affinity layer exports the Icon Composer .icon uses, mirroring its
composition (violet automatic-gradient background → light circle → dark circle →
blob in front), via scripts/render-tvos-icon.swift (checked in for regeneration):
- App Icon.imagestack 400×240 @1x/@2x + App Icon - App Store.imagestack 1280×768,
four layers each so the focus engine gets real parallax depth.
- Top Shelf Image (1920×720) + Wide (2320×720) @1x/@2x as flat composites.
- ASSETCATALOG_COMPILER_APPICON_NAME = "App Icon & Top Shelf Image" on the tvOS
configs; verified on the Apple TV simulator home screen.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A bandwidth probe over the punktfunk/1 data path so a user-settable bitrate can be
informed by what the network actually sustains (throughput/loss/jitter), surfaced in
the client UI + web console. Reuses the existing Session/FEC plumbing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of "input doesn't work" on the unified host: a single fresh session
injects fine (EIS connects, "Gamescope Virtual Input" device added), but the
host-lifetime injector reused a STALE per-session EIS socket across sessions →
"connect EIS socket …: Connection refused". (Headless gamescope is EIS-only — it
ignores uinput — so libei/EIS is the one input path for both gamescope and KWin;
no second path needed.)
- connect_socket_file: re-READ the relay file and RETRY the connect on
refused/missing (the live gamescope's EIS appears shortly), bounded at 15s,
instead of connecting once and bubbling ECONNREFUSED.
- GamescopeProc::drop: clear the relayed EIS socket name on teardown so a dead
session can't hand a stale path to the next reconnect.
Validated: two sessions back-to-back each reconnect (EIS connected + device added).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The same app now runs on tvOS (target Punktfunk-tvOS, bundle io.unom.punktfunk.tvos),
validated live against the box: vkcube at 1280x720@60, 60 fps in the Apple TV 4K
simulator, glass HUD with a focusable Disconnect button.
- PunktfunkCore.xcframework grows tvOS device + universal-simulator slices. These are
TIER-3 Rust targets (no prebuilt std): BUILD_TVOS=1 builds them with nightly and
-Zbuild-std from rust-src — the full quic stack (quinn/rustls-ring/tokio) compiles
for tvOS unchanged.
- The UIKit stream view covers iOS AND tvOS, with pointer interaction, pointer lock,
touch forwarding and InputCapture gated to iOS — tvOS is view-only until gamepad
capture lands (the natural tvOS input).
- SessionAudio on tvOS: .playback session, no mic (no app-accessible microphone).
- App chrome gates: keyboardShortcut/textSelection/controlSize/statusBarHidden are
iOS/macOS-only; host cards use the focus-native .card button style on tvOS; the
Audio settings section hides (system-routed); mode seeding works from the TV screen
(1920x1080@60).
- Package platforms += .tvOS(.v17); new Xcode target + shared scheme
(TARGETED_DEVICE_FAMILY 3, local-network usage description included).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Working through the brand-color follow-ups:
- AccentColor gains a dark-appearance variant (#8678F5 — the brand violet lifted one
step toward the icon's light periwinkle) so tinted controls keep contrast on dark.
- Host cards remember sessions: StoredHost.lastConnected (set when a session reaches
streaming) renders as a "Connected … ago" relative-time line, and the most recent
host's card carries a subtle accent ring — the grid finally has hierarchy.
- The HUD swaps the pre-glass black-50% rectangle for .regularMaterial with an accent
live-dot; hint lines use semantic .secondary instead of opacity.
- Security moments: the trust card's lock.shield and the pairing sheet's header take
the brand tint; the PIN field is larger monospaced and uses the number pad on iOS.
Icon ↔ accent decision: the accent stays the exact brand #6656F2; the Icon Composer
layers keep their adjacent palette (#6C5BF3 family) — close enough to read as one
brand, and the icon remains the design-tool source of truth.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AccentColor color set + ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME on all four app
configurations — the platform-sanctioned global tint, so the host-card icons, prominent
buttons, toggles, pickers and links all carry the brand violet on macOS and iOS without
any per-view styling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 160 pt grid minimum packed five small cards per iPad row. iOS columns now use a
280 pt minimum (one full-width card on iPhone portrait, 3–4 generous cards on iPad)
and the card content scales with it: 56 pt icon, title3 name, taller padding. macOS
keeps its compact 180–240 pt cards.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Matches the bundle display name; was the lowercase project name "punktfunk" in the
home navigation title (iOS large title / macOS titlebar) and the WindowGroup title.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Streaming on iPad left the status bar up and the video boxed inside the safe areas, on
top of a 16:9 default mode letterboxing on the 4:3 screen, with the iPadOS cursor
hovering over the video. The session view is now immersive on iOS:
- .ignoresSafeArea + .statusBarHidden + .persistentSystemOverlays(.hidden) for the
session only (home gets its chrome back on disconnect).
- First run seeds the stream mode from the device's native screen
(UIScreen.nativeBounds + maximumFramesPerSecond) instead of 1920×1080 — verified
live: a fresh install negotiated the iPad's 2752×2064 with the host. macOS keeps the
1080p default (a desktop window is not the screen).
- The iPadOS cursor hides while over the video (UIPointerInteraction .hidden(),
re-resolved on capture toggles) — the host renders its own cursor from our deltas;
true pointer lock through UIHostingController remains the documented gap.
Found along the way (host-side, not fixed here): at very high modes a keyframe burst
can fill the UDP send buffer and m3 treats the sendmmsg WouldBlock as fatal
("session ended with error: submit_frame: WouldBlock") instead of backpressuring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
§8a (require native pairing by default, serve --open) shipped + deployed. §8b
(delegated approval) refined into §8b-1 (host pending-requests + mgmt endpoints +
web Approve/Deny — achievable now) and §8b-2 (peer push to a paired Device A —
needs the native/Apple client UI).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An open punktfunk/1 host any LAN device can trust-on-first-use and stream from is
insecure. The unified host now gates native sessions on pairing by DEFAULT: a client
must complete the SPAKE2 PIN ceremony (armed from the web console) before it's
admitted; paired devices persist. `serve --open` keeps the old TOFU behavior for
trusted single-user setups.
native_serve_opts now takes a NativeServe { port, require_pairing }; parse_serve
builds it with require_pairing = !--open. GameStream pairing (separate) is unchanged.
The require_pairing gate + ceremony are already covered by m3::pairing_ceremony_and_gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Icon Composer re-import after stripping the Affinity artboard rects (full-canvas
fill:none rects the exporter adds per layer) that caused rendering artifacts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Completes the web-UI native (punktfunk/1) pairing flow the unified host backs.
The Pairing page now leads with a native card that arms a window via the mgmt API
and DISPLAYS the host PIN (the SPAKE2 ceremony is host-mints / client-enters) with
a live countdown + Cancel, plus a paired-devices list with unpair — no journalctl.
The existing Moonlight PIN-submit moves into its own section below.
Uses the orval-generated `native` hooks (regenerated from the committed OpenAPI on
build) + en/de strings. Validated end-to-end through the web server's proxy + cookie
auth: login → status → arm (PIN shown) → clients. tsc + production build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "title looks off" report traced to the GRID, not the title: the Mac-tuned
adaptive(180–240) columns yielded a single max-width card, centered, so nothing aligned
with the leading large title. The header is now entirely stock primitives — default
.navigationTitle large-title behavior (the inlineLarge experiment is gone), default
.padding() so content sits on the system 16 pt margins — and the grid columns are
platform-tuned: iOS drops the max so columns FILL the width and the cards stay
edge-aligned with the title; macOS keeps the 180–240 cap (huge windows shouldn't grow
huge cards).
Verified in the iPhone 17 simulator with seeded hosts: pill top-right, large title at
system metrics, two full-width-filling cards flush with the title's leading edge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- README: replace the stale M0/M2-in-flight status with reality — M1 hardened, M2
GameStream host live to stock Moonlight, M3 punktfunk/1 validated, M4 Apple first
light, web console + unified host; FFmpeg 7/8; Bazzite-deployed. Layout adds
web/, packaging/, native_pairing, dualsense.
- CLAUDE: protocol-growth item now reflects the unified host + web-console native
pairing (done) and flags the next steps; layout updated.
- roadmap §7 Windows: de-risked via SudoVDA (the Sunshine Virtual Display Adapter) —
no self-signed kernel IDD needed; the virtual-display backend drops XL→M.
- roadmap §8 (new) Pairing & trust hardening: mandatory PIN pairing by default
(TOFU-open is insecure on a LAN) + delegated pairing approval (an already-paired
device approves a new one, no out-of-band PIN).
- windows-host.md: SudoVDA path throughout (status, table, phasing, effort M not L).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`serve --native` now runs the GameStream host AND the native punktfunk/1 (QUIC)
host in ONE process, sharing a single NativePairing handle with the management API
— so native pairing is operable from the web console instead of journalctl.
- gamestream::serve gains a native_port: spawns crate::m3::serve in the same
runtime and passes the shared NativePairing to mgmt::run. Validated live: one
process binds both RTSP 48010 and QUIC 9777.
- mgmt API: new `native` endpoints — GET /native/pair (status), POST
/native/pair/arm (mint a fresh, time-limited PIN to DISPLAY), DELETE /native/pair
(disarm), GET/DELETE /native/clients (list/unpair). GameStream-only hosts report
enabled:false. OpenAPI regenerated (checked-in doc + drift test).
- main.rs: serve --native / --native-port flags.
The native host arms pairing on demand (the operator reads the PIN from the
console; the SPAKE2 ceremony is host-shows-PIN). New mgmt + native_pairing tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Groundwork for web-UI-driven native (punktfunk/1) pairing. Replaces m3's fixed
startup PIN + local paired store with a shared `NativePairing` (new module):
arm-on-demand with a fresh, time-limited PIN (`arm(ttl)`), `current_pin()` read
per ceremony so a lapsed window stops pairing, plus the trust store (list/add/
remove/is_paired) and a `status()` snapshot. The management API (next commit) and
the QUIC accept loop share one handle. CLI `--allow-pairing`/`--require-pairing`
still arm at startup (no expiry, PIN logged) — back-compat. m3 pairing ceremony +
gate and the C-ABI roundtrip stay green; new unit tests for arm/expire/pair.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ToolbarSpacer split into separate circles was the wrong read — with the
inline-large title row in place, the expected header is the single grouped pill
(the system default for adjacent trailing items). Dropped the spacer and the
availability fork; the two trailing items now share one pill next to the title.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The home screen stacked the toolbar row above the large title; the modern (iOS 26
Liquid Glass) header puts the large title leading and the glass action circles trailing
on the SAME row. That's exactly .toolbarTitleDisplayMode(.inlineLarge) — applied on iOS
only, macOS keeps its window chrome untouched.
Verified in the iPhone 17 simulator: "punktfunk" large title left, gear/+ circles
right, one row.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The iOS chrome inherited macOS dialog sizing and read as undersized on a phone:
- Toolbar: the two trailing actions shared one compact glass pill; on iOS 26+ each now
gets its own full-size circle (explicit .topBarTrailing placements split by a fixed
ToolbarSpacer — the system-app look, e.g. Files), with the grouped-pill fallback on
iOS 17–18. The buttons are extracted so macOS keeps SettingsLink + .help untouched.
- Sheets and CTAs (AddHostSheet, PairSheet, trust card, empty-state Add Host) get
.controlSize(.large) on iOS — proper touch targets instead of macOS dialog buttons.
Verified in the iPhone 17 simulator: two ~44 pt glass circles matching the Files app's
toolbar sizing; macOS suite and app build unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The whole client now runs on iPadOS/iOS from the same sources, first-lit live in the
iPad simulator against the real host at 1280x720@60 (60 fps on the HUD, capture state
machine active, mic permission flow shown).
- PunktfunkCore.xcframework grows iOS device + universal-simulator slices
(BUILD_IOS=1; rustup targets aarch64-apple-ios{,-sim} + x86_64-apple-ios).
- The decode pump is extracted into a shared StreamPump (identical IDR re-gate logic on
both platforms); the iOS StreamView (StreamViewIOS.swift) has the same name/signature
as the macOS one, so ContentView & co. are byte-identical across platforms — hosted
in a UIViewController for prefersPointerLocked (the iPadOS cursor capture; see README
note 9 for the UIHostingController forwarding caveat).
- Touch is always forwarded: per-finger wire ids, coordinates mapped through the
aspect-fit letterbox into LIVE host-mode pixels (surface == host mode, identity
rescale host-side; follows mid-stream requestMode switches).
- InputCapture is cross-platform: GC works the same on iPadOS, ⌘⎋ is detected from the
HID stream there; stale-⌘ tracking after focus loss fixed on both platforms
(releaseAll now drops the modifier/latch state — a ⌘ released in another app
otherwise hijacked Esc forever).
- SessionAudio: AVAudioSession on iOS (.playAndRecord + .defaultToSpeaker — without it
iPhones route host audio to the EARPIECE; deactivated with
notifyOthersOnDeactivation on stop so interrupted background audio resumes); HAL
device pinning + the Settings pickers stay macOS-only.
- New Punktfunk-iOS app target (shared synchronized sources, generated Info.plist with
mic + local-network usage descriptions — QUIC to a LAN host trips local network
privacy on real devices — scene manifest + indirect input events for Stage Manager /
external displays), shared scheme, macOS min-window frames gated off iOS.
For the iPad-on-an-external-screen idea: with multiple scenes + indirect input enabled,
Stage Manager iPads can drag the punktfunk window onto the external display and drive
the PC with keyboard/mouse/touch. Known gaps (README note 9): the pointer-lock
preference isn't consulted through UIHostingController (relative mouse works, the local
cursor just stays visible) and AVAudioSession interruptions don't auto-restart audio.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
punktfunk-host builds unchanged against either FFmpeg 7.x (libavcodec 61) or 8.x
(libavcodec 62) — ffmpeg-sys-next auto-detects the system version, and the host's
ffmpeg FFI only touches long-stable APIs. Confirmed by building + running live on a
Bazzite F43 box (FFmpeg 7.1.3): full gamescope capture → zero-copy dmabuf→CUDA →
NVENC H.265 at 1280x720x60, p50 ~0.96 ms. Just doc/spec accuracy, no code change:
- encode/linux.rs + CLAUDE.md: drop the "FFmpeg 8 only" claim; note 7.x/8.x both work.
- rpm spec: add the missing zero-copy GPU build deps the link actually needs —
pkgconfig(gl) + pkgconfig(gbm) (mesa) — and document that -lcuda needs libcuda.so at
link time (NVIDIA host, or the CUDA toolkit stub on a headless COPR/koji builder).
Tracked for a proper fix: make the cuda/gbm/GL FFI dlopen-based like khronos-egl so
the RPM builds on a GPU-less host.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A step-by-step walkthrough for running the host on Bazzite (the immutable
Fedora-Atomic gaming distro): the two install paths (rpm-ostree layering vs the
bootc image), udev + the `input` group, host.env knobs (gamescope-default), the
systemd --user service, firewall ports, verification, and troubleshooting — all
grounded in the packaging/ files. Flags the operator-run COPR, the loopback-only
mgmt port, and that the bundled unit runs the GameStream `serve` host (not m3-host).
Linked from packaging/README.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PUNKTFUNK_GAMEPAD=dualsense now routes a session's gamepad through a real virtual
DualSense (UHID + hid-playstation) end to end:
- host: a `PadBackend` enum (m3.rs) selects `GamepadManager` (uinput xpad, default)
or the new `DualSenseManager` (dualsense.rs) per session. The manager keeps each
pad's full DsState so touchpad + motion (rich-input plane) persist across
button/stick frames, and services the !Send /dev/uhid fd only on the input thread
(which cycles <=4ms, so the GET_REPORT init handshake completes).
- feedback: `service()` now returns `DsFeedback { hidout, rumble }`. Motor rumble
stays on the universal 0xCA plane (so non-DualSense clients still feel it; manager
dedups change); lightbar / player LEDs / adaptive-trigger effects ride the new
0xCD HID-output plane (host->client) as `HidOutput`.
- rich input: touchpad contacts + motion ride the 0xCC plane (client->host) as
`RichInput`, applied via `DualSenseManager::apply_rich` (merged with button state;
touch normalized 0..65535 -> the touchpad resolution).
- connector + C ABI: `NativeClient::next_hidout` / `send_rich_input`, exported as
`punktfunk_connection_next_hidout` (-> PunktfunkHidOutput) and
`punktfunk_connection_send_rich_input` (<- PunktfunkRichInput); header regenerated.
- reference client: `--rich-input-test` drives the DualSense touchpad + motion and
logs the 0xCD feedback that comes back.
Validated live on-box: a synthetic-source m3-host + client-rs created the real
kernel DualSense, drove 0xCC, and decoded 12 live 0xCD events (the kernel's actual
lightbar/trigger init reports) with the data plane unaffected (600/600 frames).
Adversarial review fixes folded in: the input loop no longer skips the rich drain +
feedback pump on a dropped gamepad event, and the touch contact id is clamped to its
slot. Remaining: the Apple client renders triggers/rumble on a real DualSense.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
4-agent feasibility read converged on three independent walls, any one fatal:
- host capture needs a kernel rebuild (CONFIG_USB_DUMMY_HCD off → no UDC for an
f_uac2 composite gadget; everything else for the gadget IS present);
- near-zero Linux supply (only ~5-10 Proton titles via custom Wine patches emit
it; hid-playstation/Steam-Input/RPCS3 don't);
- Apple client can't faithfully replay PCM haptics (CoreHaptics is discrete
pattern-based; no public CoreAudio channel-3/4 routing).
Advanced haptics ride the DualSense USB *audio* interface, not HID, so the UHID
backend structurally can't carry them. Defer; the reachable 80% ("real DualSense
feel") is adaptive triggers over the HID 0x02 path we already parse + two-motor
rumble. New docs/dualsense-haptics.md records the walls + conditions for a future
go; roadmap §5 updated (HID DualSense backend built & live-validated).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both directions of the audio plane, on CoreAudio's built-in Opus codec
(kAudioFormatOpus — no bundled libopus; OpusCodec.swift, round trip unit-tested):
- Playback: a drain thread pulls nextAudio() packets, decodes, and writes a priming
jitter ring feeding an AVAudioSourceNode (~20 ms prefill, adaptive to the device's
render quantum so large-buffer devices don't oscillate prime/dropout; a high-water
clamp sheds stall backlog so one network hiccup can't permanently lag audio behind
video; underrun re-primes — one dip, not sustained crackle).
- Mic: a second engine taps the input device, resamples to 48 kHz stereo, Opus-encodes
20 ms chunks and sendMic()s them into the host's virtual PipeWire source. Permission
via AVCaptureDevice (NSMicrophoneUsageDescription added to the Xcode target).
- Settings: Speaker + Microphone pickers (CoreAudio HAL enumeration, persisted by
device UID — "System default" leaves the engine unpinned so it follows macOS device
changes) and a "Send microphone" toggle (default on). Applies from the next session.
- Audio starts with streaming, never during the trust prompt (no host sound — and no
mic uplink — before the user trusted the host); teardown stops audio before close().
Adversarial-review fixes baked in: stop() and the dangling mic-permission callback
share one lock+flag protocol (no hot mic with no owner), the connect-success handler
bails when the attempt was abandoned mid-handshake (no session/mic for a dead window),
SessionAudio gets a deinit backstop (a dropped instance can't pin the connection via
its drain thread), and the render scratch buffer is block-owned (was leaked per
session).
Verified live against the box: remote test decodes 100 host Opus packets to PCM and
the host opens its virtual mic on the first uplinked frame ("punktfunk/1 virtual mic
ready"); on-glass session runs with both engines up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Roadmap #5 (rich DualSense). A UHID device presents a real Sony DualSense to the kernel's
hid-playstation driver (matched by VID 054C/PID 0CE6), which exposes the full controller —
gamepad, motion sensors, touchpad, lightbar/player LEDs, adaptive triggers — unlike the
uinput X-Box-360 pad.
- inject/dualsense.rs: hand-rolled /dev/uhid codec (no bindgen) mirroring the uinput style;
the canonical inputtino 232-byte USB HID report descriptor + the feature-report replies
(calibration 0x05 / pairing 0x09 / firmware 0x20) — answering hid-playstation's GET_REPORTs
during init is REQUIRED or it creates no input devices. DsState::from_gamepad maps a
GameStream/XInput frame → the DualSense input report (buttons/sticks/triggers/dpad, +
touchpad/motion fields); service() answers GET_REPORTs and parses HID OUTPUT (rumble /
lightbar RGB / player LEDs / adaptive triggers) into quic::HidOutput.
- scripts/60-punktfunk.rules: grant /dev/uhid to the 'input' group (like /dev/uinput).
- `punktfunk-host dualsense-test`: standalone validation (no streaming session).
Validated live: `dualsense-test` → hid-playstation binds + loads ff_memless + led_class_
multicolor; the kernel creates "Punktfunk DualSense 0" (event/js gamepad + Motion Sensors +
Touchpad + Headset Jack) at VID 054c/PID 0ce6, plus the lightbar at /sys/class/leds/
input*:rgb:indicator; js shows the Cross button firing + the left-stick sweep. Clippy/fmt
clean, workspace tests green. Wiring into the session (pad-type select, touchpad/motion
routing, HID-output back-channel) is the next commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adopts the new ABI surface (still v2, additive):
- PunktfunkConnection.sendMic(_:seq:ptsNs:) — Opus mic frames (48 kHz) to the host's
virtual PipeWire source; enqueue-only, empty data = DTX silence. Wiring the actual
Mac microphone (AVAudioEngine input → Opus) into the app is the follow-up, alongside
audio playback (README note 5).
- PunktfunkInputEvent.touchDown/touchMove/touchUp — absolute pixels + surface size in
flags, host injects via libei ei_touchscreen. Built for the iOS variant; nothing on
macOS emits them yet.
- Loopback round trip now also sends touch events and mic frames (incl. a DTX frame)
through the wrapper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The new features were Linux-built only and broke the documented macOS gate
(cargo build/test/clippy --workspace) four ways, all fixed following the existing
platform-gating conventions:
- m3.rs: mic_service_thread split into the Linux worker and a non-Linux stub that
drains and drops (sessions still count the datagrams) — opus/PipeWire are
Linux-gated deps, same pattern as audio_thread.
- punktfunk-client-rs: the new `opus` dependency moved into the Linux target table and
--mic-test gated with a warn-and-skip stub (only the synthetic-tone test rig needs
the encoder; the mic uplink itself is portable).
- gamestream/audio.rs: SAMPLE_RATE import gated to any(linux, test) (the frame_sizing
test uses it everywhere, the data plane only on Linux).
- tests/c_abi.rs: the harness's macOS link flags gained Security + CoreFoundation —
the quic feature now pulls rustls's platform verifier into the staticlib.
Also: two clippy match-ref-pats lints in the new rich-input/HID-output decoders
(clippy -D warnings is the repo gate), the regenerated punktfunk_core.h committed (the
checked-in copy predated the rich-input/HID-output constants — CI fails on drift), and
web's inlang cache dir gitignored.
cargo build/test/clippy/fmt --workspace: green on macOS, 122 tests passing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Foundation for rich DualSense support (roadmap #5). The fixed 18-byte InputEvent (0xC8) can't
hold the DualSense touchpad/motion or HID feedback, so two new variable-length, kind-tagged
datagram families join the side-plane (mouse/keyboard/gamepad/touch keep the fixed InputEvent):
- RICH_INPUT_MAGIC 0xCC, client→host: `[0xCC][kind][fields]`
Touchpad{pad,finger,active,x,y} (x/y normalized 0..65535; host scales to the pad)
Motion{pad, gyro[3], accel[3]} (raw i16, straight into the DualSense report)
- HIDOUT_MAGIC 0xCD, host→client: `[0xCD][kind][pad][fields]` — the rich analog of the 0xCA
rumble datagram (rumble stays on 0xCA):
Led{rgb} PlayerLeds{bits} Trigger{which, effect} (adaptive-trigger params to replay)
`RichInput`/`HidOutput` enums with encode/decode; unknown kinds + truncation decode to None
(forward-compatible). +2 round-trip/disjointness tests; quic suite green, clippy/fmt clean.
Wiring (host UHID device, capture, C ABI, client) lands in following commits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Roadmap #5 (touch, ahead of the XL UHID DualSense work). Touch fits the existing 18-byte
InputEvent: code = touch id, x/y = client pixels, flags = (w<<16)|h — the same absolute
mapping as MouseMoveAbs.
- core: InputKind::{TouchDown=9, TouchMove=10, TouchUp=11} + from_u8 + roundtrip test.
- host inject/libei.rs: request the RemoteDesktop Touchscreen device type, bind the Touch
capability, and inject ei_touchscreen down/motion/up (one event = one frame, per the
protocol rule), mapping coordinates into the device region like the abs pointer. wlroots
has no virtual-touch protocol wired — no-ops there.
- client-rs --touch-test: drags a synthetic finger (touch id 0) in a circle.
Validated live on headless KWin: the portal GRANTS the Touchscreen device type
(Keyboard|Pointer|Touchscreen), proving the request path — but KWin's EIS server creates no
touchscreen *device*, so touch currently no-ops on this KWin (now logged once, not silent).
The injection code is correct and will land on a backend that exposes ei_touchscreen
(gamescope / a newer compositor / the real touch-client path). Workspace green, clippy/fmt
clean, +1 unit test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A 4-agent read of the host crate: a Windows host is an "add a backend" job, not a parallel
port — ~95% reuse (core/protocol/FEC/crypto/C-ABI, QUIC, GameStream, mgmt, m3/pipeline are all
platform-agnostic and already cfg-isolated). New cfg(windows) backends behind the existing
traits: DXGI Desktop Duplication (capture), Media Foundation / NVENC-SDK (encode), SendInput +
ViGEm (input), WASAPI loopback + virtual mic (audio). The blocker is the virtual-display
feature — no user-mode Windows API; it needs a signed kernel-mode IDD driver (XL).
docs/windows-host.md records the per-subsystem effort + a phased plan (Phase 0 = a "basic
Windows host" capturing an existing monitor, smallest surface). Deferred: large and unbuildable
on the Linux dev box, per the request to only take it on if manageable. roadmap.md marks
#1/#2/#4 done, #3 packaged, and adds #7 Windows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Roadmap #3 (install on other devices). Bazzite already ships gamescope + PipeWire + the
NVIDIA stack, so the host slots in with minimal new deps (ffmpeg-libs from RPM Fusion + opus
+ libei).
- packaging/rpm/punktfunk.spec — builds punktfunk-host from source (cargo), installs the
binary + udev rule + systemd user unit + headless helpers; Requires/Recommends mapped from
the Ubuntu bootstrap deps to Fedora.
- packaging/bootc/Containerfile — layer punktfunk into a bazzite-nvidia bootc image for
atomic, image-based installs.
- packaging/bazzite/host.env — gamescope-default appliance config (spawned per session).
- packaging/copr/ + packaging/README.md — COPR build-from-SCM settings + install docs
(rpm-ostree and bootc paths), and why not Flatpak.
- LICENSE-MIT + LICENSE-APACHE — materialize the declared `MIT OR Apache-2.0` (was unfiled);
the RPM ships them.
Not buildable on the Ubuntu dev box (no rpm tooling) — the COPR/Fedora build is operator-run;
all spec-referenced files verified present and the cargo build is green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a
new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire
Audio/Source its apps can record from (voice chat, etc.).
Protocol (punktfunk-core):
- MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram).
- NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task
(mirror of send_input); C ABI punktfunk_connection_send_mic for native clients.
Host:
- audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class=
Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM.
- MicService: host-lifetime owner of the source + Opus decoder (mirror of
InjectorService / the audio capturer slot); lazily opened, persists across sessions,
self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a
single read_datagram loop (two loops would race).
- Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting,
so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output
was silence; with it, glitch-free across consumer quanta.
Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic
uplink (opus dep added) for end-to-end validation without a real microphone.
Validated live on headless KWin: client tone → host source → pw-record shows the
punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001
elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms
consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A full Plasma login starts several pieces our bare headless session was missing, which
surfaced as three separate failures while streaming the KDE desktop:
- Steam (and other X11 apps) failed "can't open display": Xwayland runs, but KWin only
sets DISPLAY for its own children — apps launched via the plasma menu / D-Bus activation
never saw it. Detect the Xwayland display after KWin is ready and export it into the
systemd/D-Bus activation environment.
- Discover / PackageKit couldn't install apps: polkitd (the policy engine) was running but
no authentication *agent* (the prompt) was — so privileged installs got no authorization.
Start polkit-kde-authentication-agent-1 (forcing the Qt Wayland platform, or it exits).
- The streamed desktop showed app windows but no wallpaper/panels: plasmashell had crashed
and the old unsupervised `plasmashell &` never brought it back. Supervise it — restart for
as long as KWin lives, so the desktop shell self-heals.
Validated live on this box: DISPLAY=:0 now in the --user environment (xdpyinfo on :0 works),
the polkit agent registers ("Listener online"), and plasmashell stays up under the supervisor.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler