The slide now runs on UISpringTimingParameters (stiffness 300, damping 30 — a ~0.87
damping ratio: settles quickly with a hint of life, no overshoot ping-pong) via the
transition library's .interpolatingSpring animation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The navigationLink Picker's INTERNAL destination list renders its rows in the focused
(dark-text) style while the push animates — black text over the dark backdrop until
focus settles (present under the old fade too; a SwiftUI-on-tvOS quirk we don't
control). Settings now uses its own primitives instead:
- TVSelectionRow: label + current value, pushes…
- TVSelectionList: a Settings-app-style option list (plain button rows + checkmark,
selecting pops back) — ordinary button chrome, no focused-style pre-rendering.
The stream-mode and compositor pickers are gone on tvOS; the Settings screen itself is
a plain scroll of rows + footer (no Form), matching the rest of the tv UI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SwiftUI's NavigationStack on tvOS animates pushes as a bare crossfade with no public
customization — the system Settings app slides. The home stack now applies
.customNavigationTransition(.slide) on tvOS via davdroman/swiftui-navigation-transitions
(MIT, tvOS 13+), covering the top-level routes AND the settings pickers' drill-ins.
The dependency is referenced by the Xcode PROJECT only and linked solely by the
Punktfunk-tvOS target: its manifest (no macOS platform declared vs 10.15 deps) breaks
SwiftPM whole-graph validation for plain `swift build`, and the #if os(tvOS) import
never compiles in the macOS-only SwiftPM dev shell anyway. Headless builds need
xcodebuild -skipMacroValidation (the lib pulls Swift macro packages; in the Xcode UI
it's a one-time Trust & Enable prompt).
iOS/macOS keep their untouched system navigation animations.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add Host, Settings and PIN pairing were fullScreenCover overlays, which is why
navigating felt unlike the system Settings app (no push animation, no Menu-pops-a-level
semantics). They are now navigationDestination ROUTES pushed inside the home
NavigationStack:
- the system push/pop animation and Menu-button back navigation come for free;
- the Settings pickers' navigationLink pushes reuse the same stack (its inner
NavigationStack wrapper is gone, as is the tvOS Done row — Menu pops, like Settings);
- Add Host is a real full-screen page (system navigation title, Settings-style rows on
the standard backdrop) instead of a floating dialog, same for the pairing page;
- the thickMaterial cover backdrops became unnecessary and are gone. The system
keyboard entries stay as covers — that presentation is system-owned either way.
iOS/macOS keep their sheets. Verified by screenshot: Add Host renders as a pushed
full-screen route with the title top-center.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SwiftUI's inline TextField on tvOS is structurally wrong for television: it grows when
activated, shows a full-width editing surface behind the pill, and floats labels
off-center — none of it stylable into the Settings-app look. Per Apple's tvOS text
input guidance, real tvOS apps never edit inline: a field is a value ROW, and pressing
it raises the SYSTEM fullscreen keyboard.
- TVTextEntry (UIViewControllerRepresentable): a UITextField that becomesFirstResponder
on appear, presenting the standard tvOS fullscreen keyboard with the field's prompt;
done/dismiss commits the text. TVFieldRow is the Settings-style label+value lozenge.
- Add Host and PIN pairing on tvOS now use rows + keyboard covers exclusively (the
port row also fixes the off-center value text for good — it's a Text, not a field);
the port input validates 1...65535.
- No SwiftUI TextField remains in any tvOS code path.
Verified by screenshot: the dialog rows render exactly like the Settings app, and the
address row raises the system linear keyboard with prompt + done.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three more tvOS-isms, all the same lesson — let the focus engine own the chrome:
- Host cards drew their own material platter + accent ring INSIDE the .card button
style, muting the native grow/tilt focus motion. On tvOS the card style now owns the
platter outright (material/ring stay on the pointer platforms), and the grid gets
48 pt spacing so the focused card swells without overlapping siblings.
- Add Host and Settings no longer sit in the hosts row: they're a compact button row
below the grid (and the empty state gains a Settings button, since tvOS has no
toolbar).
- The Add Host and pairing dialogs drop Form entirely on tvOS — list rows added a
full-width focus fill plus a row platter behind every field's own pill (the
"second outer pill"). As standalone fields in a centered dialog over the dimmed
home, each input is exactly one pill with vertically centered text.
Verified by screenshot in the Apple TV simulator (home grid + Add Host dialog).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The inline iOS form widgets fought the tvOS focus system at every turn: focused fields
showed nested pills, rows darkened oddly and grew on activation, the Compositor picker
was not even focusable, and prefilled fields (port, client name) floated their label
inside the pill, shoving the value off-center.
- Settings is now a fully tv-native screen: NO inline text entry — the stream mode is
a preset picker (This TV native / 720p / 1080p / 4K, plus a Custom entry preserving
a mode set on another platform) and both pickers use .navigationLink style (pushed
selection lists, exactly like the system Settings app — and properly focusable; the
cover wraps in a NavigationStack for the pushes).
- Where text entry is unavoidable (Add Host, PIN pairing), the fields keep their stock
single-pill chrome (the grouped form style stays off tvOS — its row platters were
one of the nested pills) and prefilled fields hide their floating label so values
center vertically.
- All earlier row-clearing experiments reverted.
Verified by screenshot in the Apple TV simulator: Settings rows render as single
focus lozenges with chevrons; the Add Host pills are uniform with centered text.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tvOS forms/lists have CLEAR backgrounds and a fullScreenCover only shows what the
presented view paints, so Settings/Add Host/pairing rendered transparently over the
hosts grid. All three covers now sit on .thickMaterial edge to edge — the standard
tvOS blur-over-content panel look (verified in the Apple TV simulator).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The iOS chrome half-worked on tvOS: toolbar items rendered tiny with clipped labels
and could not even be focused (which is why "+" never opened the add-host form), and
sheet presentations are not a tvOS idiom (the Settings form looked broken).
- The toolbar is gone on tvOS. Add Host and Settings live IN the hosts grid as
full-size, focus-native tiles (.card style, same geometry as the host cards) — the
natural way actions work on television.
- Every modal (Add Host, Settings, PIN pairing) presents as a fullScreenCover on tvOS;
Settings gains a tvOS-only Done button (covers don't dismiss themselves).
- iOS/macOS keep their existing toolbar + sheets untouched.
Verified in the Apple TV simulator: title, host card and both action tiles render
full-size and focusable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Icon Composer doesn't cover tvOS — tvOS app icons are the older parallax format:
flat layers in an asset-catalog "App Icon & Top Shelf Image" brand asset. Generated
from the same Affinity layer exports the Icon Composer .icon uses, mirroring its
composition (violet automatic-gradient background → light circle → dark circle →
blob in front), via scripts/render-tvos-icon.swift (checked in for regeneration):
- App Icon.imagestack 400×240 @1x/@2x + App Icon - App Store.imagestack 1280×768,
four layers each so the focus engine gets real parallax depth.
- Top Shelf Image (1920×720) + Wide (2320×720) @1x/@2x as flat composites.
- ASSETCATALOG_COMPILER_APPICON_NAME = "App Icon & Top Shelf Image" on the tvOS
configs; verified on the Apple TV simulator home screen.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A bandwidth probe over the punktfunk/1 data path so a user-settable bitrate can be
informed by what the network actually sustains (throughput/loss/jitter), surfaced in
the client UI + web console. Reuses the existing Session/FEC plumbing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of "input doesn't work" on the unified host: a single fresh session
injects fine (EIS connects, "Gamescope Virtual Input" device added), but the
host-lifetime injector reused a STALE per-session EIS socket across sessions →
"connect EIS socket …: Connection refused". (Headless gamescope is EIS-only — it
ignores uinput — so libei/EIS is the one input path for both gamescope and KWin;
no second path needed.)
- connect_socket_file: re-READ the relay file and RETRY the connect on
refused/missing (the live gamescope's EIS appears shortly), bounded at 15s,
instead of connecting once and bubbling ECONNREFUSED.
- GamescopeProc::drop: clear the relayed EIS socket name on teardown so a dead
session can't hand a stale path to the next reconnect.
Validated: two sessions back-to-back each reconnect (EIS connected + device added).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The same app now runs on tvOS (target Punktfunk-tvOS, bundle io.unom.punktfunk.tvos),
validated live against the box: vkcube at 1280x720@60, 60 fps in the Apple TV 4K
simulator, glass HUD with a focusable Disconnect button.
- PunktfunkCore.xcframework grows tvOS device + universal-simulator slices. These are
TIER-3 Rust targets (no prebuilt std): BUILD_TVOS=1 builds them with nightly and
-Zbuild-std from rust-src — the full quic stack (quinn/rustls-ring/tokio) compiles
for tvOS unchanged.
- The UIKit stream view covers iOS AND tvOS, with pointer interaction, pointer lock,
touch forwarding and InputCapture gated to iOS — tvOS is view-only until gamepad
capture lands (the natural tvOS input).
- SessionAudio on tvOS: .playback session, no mic (no app-accessible microphone).
- App chrome gates: keyboardShortcut/textSelection/controlSize/statusBarHidden are
iOS/macOS-only; host cards use the focus-native .card button style on tvOS; the
Audio settings section hides (system-routed); mode seeding works from the TV screen
(1920x1080@60).
- Package platforms += .tvOS(.v17); new Xcode target + shared scheme
(TARGETED_DEVICE_FAMILY 3, local-network usage description included).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Working through the brand-color follow-ups:
- AccentColor gains a dark-appearance variant (#8678F5 — the brand violet lifted one
step toward the icon's light periwinkle) so tinted controls keep contrast on dark.
- Host cards remember sessions: StoredHost.lastConnected (set when a session reaches
streaming) renders as a "Connected … ago" relative-time line, and the most recent
host's card carries a subtle accent ring — the grid finally has hierarchy.
- The HUD swaps the pre-glass black-50% rectangle for .regularMaterial with an accent
live-dot; hint lines use semantic .secondary instead of opacity.
- Security moments: the trust card's lock.shield and the pairing sheet's header take
the brand tint; the PIN field is larger monospaced and uses the number pad on iOS.
Icon ↔ accent decision: the accent stays the exact brand #6656F2; the Icon Composer
layers keep their adjacent palette (#6C5BF3 family) — close enough to read as one
brand, and the icon remains the design-tool source of truth.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AccentColor color set + ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME on all four app
configurations — the platform-sanctioned global tint, so the host-card icons, prominent
buttons, toggles, pickers and links all carry the brand violet on macOS and iOS without
any per-view styling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 160 pt grid minimum packed five small cards per iPad row. iOS columns now use a
280 pt minimum (one full-width card on iPhone portrait, 3–4 generous cards on iPad)
and the card content scales with it: 56 pt icon, title3 name, taller padding. macOS
keeps its compact 180–240 pt cards.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Matches the bundle display name; was the lowercase project name "punktfunk" in the
home navigation title (iOS large title / macOS titlebar) and the WindowGroup title.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Streaming on iPad left the status bar up and the video boxed inside the safe areas, on
top of a 16:9 default mode letterboxing on the 4:3 screen, with the iPadOS cursor
hovering over the video. The session view is now immersive on iOS:
- .ignoresSafeArea + .statusBarHidden + .persistentSystemOverlays(.hidden) for the
session only (home gets its chrome back on disconnect).
- First run seeds the stream mode from the device's native screen
(UIScreen.nativeBounds + maximumFramesPerSecond) instead of 1920×1080 — verified
live: a fresh install negotiated the iPad's 2752×2064 with the host. macOS keeps the
1080p default (a desktop window is not the screen).
- The iPadOS cursor hides while over the video (UIPointerInteraction .hidden(),
re-resolved on capture toggles) — the host renders its own cursor from our deltas;
true pointer lock through UIHostingController remains the documented gap.
Found along the way (host-side, not fixed here): at very high modes a keyframe burst
can fill the UDP send buffer and m3 treats the sendmmsg WouldBlock as fatal
("session ended with error: submit_frame: WouldBlock") instead of backpressuring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
§8a (require native pairing by default, serve --open) shipped + deployed. §8b
(delegated approval) refined into §8b-1 (host pending-requests + mgmt endpoints +
web Approve/Deny — achievable now) and §8b-2 (peer push to a paired Device A —
needs the native/Apple client UI).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An open punktfunk/1 host any LAN device can trust-on-first-use and stream from is
insecure. The unified host now gates native sessions on pairing by DEFAULT: a client
must complete the SPAKE2 PIN ceremony (armed from the web console) before it's
admitted; paired devices persist. `serve --open` keeps the old TOFU behavior for
trusted single-user setups.
native_serve_opts now takes a NativeServe { port, require_pairing }; parse_serve
builds it with require_pairing = !--open. GameStream pairing (separate) is unchanged.
The require_pairing gate + ceremony are already covered by m3::pairing_ceremony_and_gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Icon Composer re-import after stripping the Affinity artboard rects (full-canvas
fill:none rects the exporter adds per layer) that caused rendering artifacts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Completes the web-UI native (punktfunk/1) pairing flow the unified host backs.
The Pairing page now leads with a native card that arms a window via the mgmt API
and DISPLAYS the host PIN (the SPAKE2 ceremony is host-mints / client-enters) with
a live countdown + Cancel, plus a paired-devices list with unpair — no journalctl.
The existing Moonlight PIN-submit moves into its own section below.
Uses the orval-generated `native` hooks (regenerated from the committed OpenAPI on
build) + en/de strings. Validated end-to-end through the web server's proxy + cookie
auth: login → status → arm (PIN shown) → clients. tsc + production build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "title looks off" report traced to the GRID, not the title: the Mac-tuned
adaptive(180–240) columns yielded a single max-width card, centered, so nothing aligned
with the leading large title. The header is now entirely stock primitives — default
.navigationTitle large-title behavior (the inlineLarge experiment is gone), default
.padding() so content sits on the system 16 pt margins — and the grid columns are
platform-tuned: iOS drops the max so columns FILL the width and the cards stay
edge-aligned with the title; macOS keeps the 180–240 cap (huge windows shouldn't grow
huge cards).
Verified in the iPhone 17 simulator with seeded hosts: pill top-right, large title at
system metrics, two full-width-filling cards flush with the title's leading edge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- README: replace the stale M0/M2-in-flight status with reality — M1 hardened, M2
GameStream host live to stock Moonlight, M3 punktfunk/1 validated, M4 Apple first
light, web console + unified host; FFmpeg 7/8; Bazzite-deployed. Layout adds
web/, packaging/, native_pairing, dualsense.
- CLAUDE: protocol-growth item now reflects the unified host + web-console native
pairing (done) and flags the next steps; layout updated.
- roadmap §7 Windows: de-risked via SudoVDA (the Sunshine Virtual Display Adapter) —
no self-signed kernel IDD needed; the virtual-display backend drops XL→M.
- roadmap §8 (new) Pairing & trust hardening: mandatory PIN pairing by default
(TOFU-open is insecure on a LAN) + delegated pairing approval (an already-paired
device approves a new one, no out-of-band PIN).
- windows-host.md: SudoVDA path throughout (status, table, phasing, effort M not L).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`serve --native` now runs the GameStream host AND the native punktfunk/1 (QUIC)
host in ONE process, sharing a single NativePairing handle with the management API
— so native pairing is operable from the web console instead of journalctl.
- gamestream::serve gains a native_port: spawns crate::m3::serve in the same
runtime and passes the shared NativePairing to mgmt::run. Validated live: one
process binds both RTSP 48010 and QUIC 9777.
- mgmt API: new `native` endpoints — GET /native/pair (status), POST
/native/pair/arm (mint a fresh, time-limited PIN to DISPLAY), DELETE /native/pair
(disarm), GET/DELETE /native/clients (list/unpair). GameStream-only hosts report
enabled:false. OpenAPI regenerated (checked-in doc + drift test).
- main.rs: serve --native / --native-port flags.
The native host arms pairing on demand (the operator reads the PIN from the
console; the SPAKE2 ceremony is host-shows-PIN). New mgmt + native_pairing tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Groundwork for web-UI-driven native (punktfunk/1) pairing. Replaces m3's fixed
startup PIN + local paired store with a shared `NativePairing` (new module):
arm-on-demand with a fresh, time-limited PIN (`arm(ttl)`), `current_pin()` read
per ceremony so a lapsed window stops pairing, plus the trust store (list/add/
remove/is_paired) and a `status()` snapshot. The management API (next commit) and
the QUIC accept loop share one handle. CLI `--allow-pairing`/`--require-pairing`
still arm at startup (no expiry, PIN logged) — back-compat. m3 pairing ceremony +
gate and the C-ABI roundtrip stay green; new unit tests for arm/expire/pair.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ToolbarSpacer split into separate circles was the wrong read — with the
inline-large title row in place, the expected header is the single grouped pill
(the system default for adjacent trailing items). Dropped the spacer and the
availability fork; the two trailing items now share one pill next to the title.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The home screen stacked the toolbar row above the large title; the modern (iOS 26
Liquid Glass) header puts the large title leading and the glass action circles trailing
on the SAME row. That's exactly .toolbarTitleDisplayMode(.inlineLarge) — applied on iOS
only, macOS keeps its window chrome untouched.
Verified in the iPhone 17 simulator: "punktfunk" large title left, gear/+ circles
right, one row.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The iOS chrome inherited macOS dialog sizing and read as undersized on a phone:
- Toolbar: the two trailing actions shared one compact glass pill; on iOS 26+ each now
gets its own full-size circle (explicit .topBarTrailing placements split by a fixed
ToolbarSpacer — the system-app look, e.g. Files), with the grouped-pill fallback on
iOS 17–18. The buttons are extracted so macOS keeps SettingsLink + .help untouched.
- Sheets and CTAs (AddHostSheet, PairSheet, trust card, empty-state Add Host) get
.controlSize(.large) on iOS — proper touch targets instead of macOS dialog buttons.
Verified in the iPhone 17 simulator: two ~44 pt glass circles matching the Files app's
toolbar sizing; macOS suite and app build unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The whole client now runs on iPadOS/iOS from the same sources, first-lit live in the
iPad simulator against the real host at 1280x720@60 (60 fps on the HUD, capture state
machine active, mic permission flow shown).
- PunktfunkCore.xcframework grows iOS device + universal-simulator slices
(BUILD_IOS=1; rustup targets aarch64-apple-ios{,-sim} + x86_64-apple-ios).
- The decode pump is extracted into a shared StreamPump (identical IDR re-gate logic on
both platforms); the iOS StreamView (StreamViewIOS.swift) has the same name/signature
as the macOS one, so ContentView & co. are byte-identical across platforms — hosted
in a UIViewController for prefersPointerLocked (the iPadOS cursor capture; see README
note 9 for the UIHostingController forwarding caveat).
- Touch is always forwarded: per-finger wire ids, coordinates mapped through the
aspect-fit letterbox into LIVE host-mode pixels (surface == host mode, identity
rescale host-side; follows mid-stream requestMode switches).
- InputCapture is cross-platform: GC works the same on iPadOS, ⌘⎋ is detected from the
HID stream there; stale-⌘ tracking after focus loss fixed on both platforms
(releaseAll now drops the modifier/latch state — a ⌘ released in another app
otherwise hijacked Esc forever).
- SessionAudio: AVAudioSession on iOS (.playAndRecord + .defaultToSpeaker — without it
iPhones route host audio to the EARPIECE; deactivated with
notifyOthersOnDeactivation on stop so interrupted background audio resumes); HAL
device pinning + the Settings pickers stay macOS-only.
- New Punktfunk-iOS app target (shared synchronized sources, generated Info.plist with
mic + local-network usage descriptions — QUIC to a LAN host trips local network
privacy on real devices — scene manifest + indirect input events for Stage Manager /
external displays), shared scheme, macOS min-window frames gated off iOS.
For the iPad-on-an-external-screen idea: with multiple scenes + indirect input enabled,
Stage Manager iPads can drag the punktfunk window onto the external display and drive
the PC with keyboard/mouse/touch. Known gaps (README note 9): the pointer-lock
preference isn't consulted through UIHostingController (relative mouse works, the local
cursor just stays visible) and AVAudioSession interruptions don't auto-restart audio.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
punktfunk-host builds unchanged against either FFmpeg 7.x (libavcodec 61) or 8.x
(libavcodec 62) — ffmpeg-sys-next auto-detects the system version, and the host's
ffmpeg FFI only touches long-stable APIs. Confirmed by building + running live on a
Bazzite F43 box (FFmpeg 7.1.3): full gamescope capture → zero-copy dmabuf→CUDA →
NVENC H.265 at 1280x720x60, p50 ~0.96 ms. Just doc/spec accuracy, no code change:
- encode/linux.rs + CLAUDE.md: drop the "FFmpeg 8 only" claim; note 7.x/8.x both work.
- rpm spec: add the missing zero-copy GPU build deps the link actually needs —
pkgconfig(gl) + pkgconfig(gbm) (mesa) — and document that -lcuda needs libcuda.so at
link time (NVIDIA host, or the CUDA toolkit stub on a headless COPR/koji builder).
Tracked for a proper fix: make the cuda/gbm/GL FFI dlopen-based like khronos-egl so
the RPM builds on a GPU-less host.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A step-by-step walkthrough for running the host on Bazzite (the immutable
Fedora-Atomic gaming distro): the two install paths (rpm-ostree layering vs the
bootc image), udev + the `input` group, host.env knobs (gamescope-default), the
systemd --user service, firewall ports, verification, and troubleshooting — all
grounded in the packaging/ files. Flags the operator-run COPR, the loopback-only
mgmt port, and that the bundled unit runs the GameStream `serve` host (not m3-host).
Linked from packaging/README.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PUNKTFUNK_GAMEPAD=dualsense now routes a session's gamepad through a real virtual
DualSense (UHID + hid-playstation) end to end:
- host: a `PadBackend` enum (m3.rs) selects `GamepadManager` (uinput xpad, default)
or the new `DualSenseManager` (dualsense.rs) per session. The manager keeps each
pad's full DsState so touchpad + motion (rich-input plane) persist across
button/stick frames, and services the !Send /dev/uhid fd only on the input thread
(which cycles <=4ms, so the GET_REPORT init handshake completes).
- feedback: `service()` now returns `DsFeedback { hidout, rumble }`. Motor rumble
stays on the universal 0xCA plane (so non-DualSense clients still feel it; manager
dedups change); lightbar / player LEDs / adaptive-trigger effects ride the new
0xCD HID-output plane (host->client) as `HidOutput`.
- rich input: touchpad contacts + motion ride the 0xCC plane (client->host) as
`RichInput`, applied via `DualSenseManager::apply_rich` (merged with button state;
touch normalized 0..65535 -> the touchpad resolution).
- connector + C ABI: `NativeClient::next_hidout` / `send_rich_input`, exported as
`punktfunk_connection_next_hidout` (-> PunktfunkHidOutput) and
`punktfunk_connection_send_rich_input` (<- PunktfunkRichInput); header regenerated.
- reference client: `--rich-input-test` drives the DualSense touchpad + motion and
logs the 0xCD feedback that comes back.
Validated live on-box: a synthetic-source m3-host + client-rs created the real
kernel DualSense, drove 0xCC, and decoded 12 live 0xCD events (the kernel's actual
lightbar/trigger init reports) with the data plane unaffected (600/600 frames).
Adversarial review fixes folded in: the input loop no longer skips the rich drain +
feedback pump on a dropped gamepad event, and the touch contact id is clamped to its
slot. Remaining: the Apple client renders triggers/rumble on a real DualSense.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
4-agent feasibility read converged on three independent walls, any one fatal:
- host capture needs a kernel rebuild (CONFIG_USB_DUMMY_HCD off → no UDC for an
f_uac2 composite gadget; everything else for the gadget IS present);
- near-zero Linux supply (only ~5-10 Proton titles via custom Wine patches emit
it; hid-playstation/Steam-Input/RPCS3 don't);
- Apple client can't faithfully replay PCM haptics (CoreHaptics is discrete
pattern-based; no public CoreAudio channel-3/4 routing).
Advanced haptics ride the DualSense USB *audio* interface, not HID, so the UHID
backend structurally can't carry them. Defer; the reachable 80% ("real DualSense
feel") is adaptive triggers over the HID 0x02 path we already parse + two-motor
rumble. New docs/dualsense-haptics.md records the walls + conditions for a future
go; roadmap §5 updated (HID DualSense backend built & live-validated).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both directions of the audio plane, on CoreAudio's built-in Opus codec
(kAudioFormatOpus — no bundled libopus; OpusCodec.swift, round trip unit-tested):
- Playback: a drain thread pulls nextAudio() packets, decodes, and writes a priming
jitter ring feeding an AVAudioSourceNode (~20 ms prefill, adaptive to the device's
render quantum so large-buffer devices don't oscillate prime/dropout; a high-water
clamp sheds stall backlog so one network hiccup can't permanently lag audio behind
video; underrun re-primes — one dip, not sustained crackle).
- Mic: a second engine taps the input device, resamples to 48 kHz stereo, Opus-encodes
20 ms chunks and sendMic()s them into the host's virtual PipeWire source. Permission
via AVCaptureDevice (NSMicrophoneUsageDescription added to the Xcode target).
- Settings: Speaker + Microphone pickers (CoreAudio HAL enumeration, persisted by
device UID — "System default" leaves the engine unpinned so it follows macOS device
changes) and a "Send microphone" toggle (default on). Applies from the next session.
- Audio starts with streaming, never during the trust prompt (no host sound — and no
mic uplink — before the user trusted the host); teardown stops audio before close().
Adversarial-review fixes baked in: stop() and the dangling mic-permission callback
share one lock+flag protocol (no hot mic with no owner), the connect-success handler
bails when the attempt was abandoned mid-handshake (no session/mic for a dead window),
SessionAudio gets a deinit backstop (a dropped instance can't pin the connection via
its drain thread), and the render scratch buffer is block-owned (was leaked per
session).
Verified live against the box: remote test decodes 100 host Opus packets to PCM and
the host opens its virtual mic on the first uplinked frame ("punktfunk/1 virtual mic
ready"); on-glass session runs with both engines up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Roadmap #5 (rich DualSense). A UHID device presents a real Sony DualSense to the kernel's
hid-playstation driver (matched by VID 054C/PID 0CE6), which exposes the full controller —
gamepad, motion sensors, touchpad, lightbar/player LEDs, adaptive triggers — unlike the
uinput X-Box-360 pad.
- inject/dualsense.rs: hand-rolled /dev/uhid codec (no bindgen) mirroring the uinput style;
the canonical inputtino 232-byte USB HID report descriptor + the feature-report replies
(calibration 0x05 / pairing 0x09 / firmware 0x20) — answering hid-playstation's GET_REPORTs
during init is REQUIRED or it creates no input devices. DsState::from_gamepad maps a
GameStream/XInput frame → the DualSense input report (buttons/sticks/triggers/dpad, +
touchpad/motion fields); service() answers GET_REPORTs and parses HID OUTPUT (rumble /
lightbar RGB / player LEDs / adaptive triggers) into quic::HidOutput.
- scripts/60-punktfunk.rules: grant /dev/uhid to the 'input' group (like /dev/uinput).
- `punktfunk-host dualsense-test`: standalone validation (no streaming session).
Validated live: `dualsense-test` → hid-playstation binds + loads ff_memless + led_class_
multicolor; the kernel creates "Punktfunk DualSense 0" (event/js gamepad + Motion Sensors +
Touchpad + Headset Jack) at VID 054c/PID 0ce6, plus the lightbar at /sys/class/leds/
input*:rgb:indicator; js shows the Cross button firing + the left-stick sweep. Clippy/fmt
clean, workspace tests green. Wiring into the session (pad-type select, touchpad/motion
routing, HID-output back-channel) is the next commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adopts the new ABI surface (still v2, additive):
- PunktfunkConnection.sendMic(_:seq:ptsNs:) — Opus mic frames (48 kHz) to the host's
virtual PipeWire source; enqueue-only, empty data = DTX silence. Wiring the actual
Mac microphone (AVAudioEngine input → Opus) into the app is the follow-up, alongside
audio playback (README note 5).
- PunktfunkInputEvent.touchDown/touchMove/touchUp — absolute pixels + surface size in
flags, host injects via libei ei_touchscreen. Built for the iOS variant; nothing on
macOS emits them yet.
- Loopback round trip now also sends touch events and mic frames (incl. a DTX frame)
through the wrapper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The new features were Linux-built only and broke the documented macOS gate
(cargo build/test/clippy --workspace) four ways, all fixed following the existing
platform-gating conventions:
- m3.rs: mic_service_thread split into the Linux worker and a non-Linux stub that
drains and drops (sessions still count the datagrams) — opus/PipeWire are
Linux-gated deps, same pattern as audio_thread.
- punktfunk-client-rs: the new `opus` dependency moved into the Linux target table and
--mic-test gated with a warn-and-skip stub (only the synthetic-tone test rig needs
the encoder; the mic uplink itself is portable).
- gamestream/audio.rs: SAMPLE_RATE import gated to any(linux, test) (the frame_sizing
test uses it everywhere, the data plane only on Linux).
- tests/c_abi.rs: the harness's macOS link flags gained Security + CoreFoundation —
the quic feature now pulls rustls's platform verifier into the staticlib.
Also: two clippy match-ref-pats lints in the new rich-input/HID-output decoders
(clippy -D warnings is the repo gate), the regenerated punktfunk_core.h committed (the
checked-in copy predated the rich-input/HID-output constants — CI fails on drift), and
web's inlang cache dir gitignored.
cargo build/test/clippy/fmt --workspace: green on macOS, 122 tests passing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Foundation for rich DualSense support (roadmap #5). The fixed 18-byte InputEvent (0xC8) can't
hold the DualSense touchpad/motion or HID feedback, so two new variable-length, kind-tagged
datagram families join the side-plane (mouse/keyboard/gamepad/touch keep the fixed InputEvent):
- RICH_INPUT_MAGIC 0xCC, client→host: `[0xCC][kind][fields]`
Touchpad{pad,finger,active,x,y} (x/y normalized 0..65535; host scales to the pad)
Motion{pad, gyro[3], accel[3]} (raw i16, straight into the DualSense report)
- HIDOUT_MAGIC 0xCD, host→client: `[0xCD][kind][pad][fields]` — the rich analog of the 0xCA
rumble datagram (rumble stays on 0xCA):
Led{rgb} PlayerLeds{bits} Trigger{which, effect} (adaptive-trigger params to replay)
`RichInput`/`HidOutput` enums with encode/decode; unknown kinds + truncation decode to None
(forward-compatible). +2 round-trip/disjointness tests; quic suite green, clippy/fmt clean.
Wiring (host UHID device, capture, C ABI, client) lands in following commits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Roadmap #5 (touch, ahead of the XL UHID DualSense work). Touch fits the existing 18-byte
InputEvent: code = touch id, x/y = client pixels, flags = (w<<16)|h — the same absolute
mapping as MouseMoveAbs.
- core: InputKind::{TouchDown=9, TouchMove=10, TouchUp=11} + from_u8 + roundtrip test.
- host inject/libei.rs: request the RemoteDesktop Touchscreen device type, bind the Touch
capability, and inject ei_touchscreen down/motion/up (one event = one frame, per the
protocol rule), mapping coordinates into the device region like the abs pointer. wlroots
has no virtual-touch protocol wired — no-ops there.
- client-rs --touch-test: drags a synthetic finger (touch id 0) in a circle.
Validated live on headless KWin: the portal GRANTS the Touchscreen device type
(Keyboard|Pointer|Touchscreen), proving the request path — but KWin's EIS server creates no
touchscreen *device*, so touch currently no-ops on this KWin (now logged once, not silent).
The injection code is correct and will land on a backend that exposes ei_touchscreen
(gamescope / a newer compositor / the real touch-client path). Workspace green, clippy/fmt
clean, +1 unit test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A 4-agent read of the host crate: a Windows host is an "add a backend" job, not a parallel
port — ~95% reuse (core/protocol/FEC/crypto/C-ABI, QUIC, GameStream, mgmt, m3/pipeline are all
platform-agnostic and already cfg-isolated). New cfg(windows) backends behind the existing
traits: DXGI Desktop Duplication (capture), Media Foundation / NVENC-SDK (encode), SendInput +
ViGEm (input), WASAPI loopback + virtual mic (audio). The blocker is the virtual-display
feature — no user-mode Windows API; it needs a signed kernel-mode IDD driver (XL).
docs/windows-host.md records the per-subsystem effort + a phased plan (Phase 0 = a "basic
Windows host" capturing an existing monitor, smallest surface). Deferred: large and unbuildable
on the Linux dev box, per the request to only take it on if manageable. roadmap.md marks
#1/#2/#4 done, #3 packaged, and adds #7 Windows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Roadmap #3 (install on other devices). Bazzite already ships gamescope + PipeWire + the
NVIDIA stack, so the host slots in with minimal new deps (ffmpeg-libs from RPM Fusion + opus
+ libei).
- packaging/rpm/punktfunk.spec — builds punktfunk-host from source (cargo), installs the
binary + udev rule + systemd user unit + headless helpers; Requires/Recommends mapped from
the Ubuntu bootstrap deps to Fedora.
- packaging/bootc/Containerfile — layer punktfunk into a bazzite-nvidia bootc image for
atomic, image-based installs.
- packaging/bazzite/host.env — gamescope-default appliance config (spawned per session).
- packaging/copr/ + packaging/README.md — COPR build-from-SCM settings + install docs
(rpm-ostree and bootc paths), and why not Flatpak.
- LICENSE-MIT + LICENSE-APACHE — materialize the declared `MIT OR Apache-2.0` (was unfiled);
the RPM ships them.
Not buildable on the Ubuntu dev box (no rpm tooling) — the COPR/Fedora build is operator-run;
all spec-referenced files verified present and the cargo build is green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a
new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire
Audio/Source its apps can record from (voice chat, etc.).
Protocol (punktfunk-core):
- MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram).
- NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task
(mirror of send_input); C ABI punktfunk_connection_send_mic for native clients.
Host:
- audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class=
Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM.
- MicService: host-lifetime owner of the source + Opus decoder (mirror of
InjectorService / the audio capturer slot); lazily opened, persists across sessions,
self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a
single read_datagram loop (two loops would race).
- Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting,
so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output
was silence; with it, glitch-free across consumer quanta.
Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic
uplink (opus dep added) for end-to-end validation without a real microphone.
Validated live on headless KWin: client tone → host source → pw-record shows the
punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001
elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms
consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A full Plasma login starts several pieces our bare headless session was missing, which
surfaced as three separate failures while streaming the KDE desktop:
- Steam (and other X11 apps) failed "can't open display": Xwayland runs, but KWin only
sets DISPLAY for its own children — apps launched via the plasma menu / D-Bus activation
never saw it. Detect the Xwayland display after KWin is ready and export it into the
systemd/D-Bus activation environment.
- Discover / PackageKit couldn't install apps: polkitd (the policy engine) was running but
no authentication *agent* (the prompt) was — so privileged installs got no authorization.
Start polkit-kde-authentication-agent-1 (forcing the Qt Wayland platform, or it exits).
- The streamed desktop showed app windows but no wallpaper/panels: plasmashell had crashed
and the old unsupervised `plasmashell &` never brought it back. Supervise it — restart for
as long as KWin lives, so the desktop shell self-heals.
Validated live on this box: DISPLAY=:0 now in the --user environment (xdpyinfo on :0 works),
the polkit agent registers ("Listener online"), and plasmashell stays up under the supervisor.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Under rapid client reconnects, KWin's libei/EIS input setup intermittently wedged
with "EIS setup timed out", causing total input loss for affected sessions. Root
cause: each punktfunk/1 session opened (and tore down) its own RemoteDesktop-portal
CreateSession for pointer/keyboard injection, and back-to-back reconnects raced a
prior session's portal teardown before it settled.
LibeiInjector is only a Send channel handle to a worker thread that owns the portal
session, so the injector can live for the whole host run instead of per session.
Adds InjectorService: one host-lifetime thread owns the (!Send) injector, opened
ONCE (lazily, on the first event) and reused across every session — the portal grant
is established a single time and held. Sessions forward pointer/keyboard events to it
over a clonable Send channel; gamepads stay per-session (uinput, no portal). The
service self-heals — reopen after a 2s backoff if open fails or the backend worker
dies (covers a gamescope EIS socket that respawns with its nested session).
Mirrors the existing host-lifetime audio-capturer slot; the audio capturer is Send
(a slot works), the injector is !Send (needs the owning thread + channel).
Validated live on headless KWin: 8 rapid back-to-back input sessions →
"input injector ready (host-lifetime)" exactly once, ZERO "EIS setup timed out",
8/8 sessions injected input. Tests green, clippy/fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Scroll was wired to GCMouse's scroll dpad, which only fires for plain HID wheel
deltas — trackpad and Magic Mouse scrolling are gesture events that never reach
GameController, so scrolling was dead on the default Mac setups. The stream view now
overrides scrollWheel (while captured the cursor is parked mid-view, so it receives
every scroll event) and feeds InputCapture.sendScroll: precise gesture deltas are
pixels (~0.1 notch/px, SDL's factor → ×12 for WHEEL_DELTA(120)), classic wheels are
lines (×120), fractional remainders accumulate, and the GC scroll handler is gone so
wheel mice can't double-deliver. Signs pass through as-is, preserving the local
(natural-)scrolling preference.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adopts punktfunk_connect_ex from the compositor-selection batch: a Compositor enum on
PunktfunkConnection (auto/kwin/wlroots/mutter/gamescope, with the host's name aliases
for env parsing), a "Host compositor" picker in Settings (default Automatic — a
concrete choice is honored only if that backend is available host-side), and
PUNKTFUNK_COMPOSITOR / PUNKTFUNK_REMOTE_COMPOSITOR pass-throughs for the autoconnect
dev hook and the remote first-light test. The wire change is backward-compatible
(optional trailing byte), so no behavior changes at the default.
Validated live against the box: host with no compositor env (auto-detect = KWin)
logged "honoring client compositor request compositor=gamescope" and streamed 60/60
decoded frames from the spawned gamescope.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A client can now request which compositor backend the host drives its virtual
output on (gamescope/KWin/Mutter/wlroots). The host honors the request if that
backend is available, else falls back to auto-detect and reports the resolved
choice back — wire-compatible both directions (no ABI bump).
Protocol (punktfunk-core):
- New CompositorPref (config.rs): Auto|Kwin|Wlroots|Mutter|Gamescope with
u8/name mappings. Appended as one optional byte to Hello (client preference)
and Welcome (host's resolved choice). Both decoders already tolerate trailing
bytes, so old↔new interop is preserved — ABI_VERSION stays 2. Round-trip +
back-compat (truncated-message) tests.
- C ABI: punktfunk_connect_ex(compositor) + PUNKTFUNK_COMPOSITOR_* constants;
punktfunk_connect delegates with AUTO, so the existing symbol is unchanged.
NativeClient::connect / worker_main thread the preference through.
Host:
- vdisplay::available() enumerates usable backends via cheap, side-effect-free
probes (KWin zkde global, gamescope binary+version, GNOME/Sway env), plus
Compositor id/label/as_pref/from_pref/all helpers.
- m3 handshake resolves the preference to a concrete backend during the
handshake (pick_compositor pure + resolved logging), reports it in Welcome,
and threads it into virtual_stream (replacing the unconditional detect()).
- mgmt GET /v1/compositors lists every backend with availability + the
auto-detected default (OpenAPI regenerated).
Client:
- punktfunk-client-rs --compositor NAME; logs the host's resolved choice from
the Welcome ("session offer … compositor=…").
Web console:
- Host page gains a Compositors card (availability + default badges) via the
codegen'd useListCompositors hook; en/de strings added.
Also fixes a pre-existing, env-dependent test-isolation bug:
mgmt::tests::paired_clients_list_and_unpair seeded the real
~/.config/punktfunk/paired.json (AppState::new loads it), so a real
GameStream-paired client leaked into body[0] on a dev box — now cleared first.
Live-validated against headless KWin: --compositor kwin honored, --compositor
mutter falls back to kwin (available=[kwin, gamescope]), resolved choice
round-trips to the client. Tests: +6 (wire/back-compat, resolution precedence,
endpoint); workspace green, clippy/fmt clean, C ABI harness PASS at abi_version=2,
web typecheck + build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hardens the virtual-display → capture → encode bring-up against the transient
failures that surfaced as black screens / wrong refresh on cold KDE sessions.
- m3: build_pipeline_with_retry wraps the initial vd.create() + first-frame with
bounded exponential backoff (4 attempts, 500ms→2s). is_permanent_build_error
classifies config/version/missing-tool failures so they fail fast instead of
burning the retry budget. Encoder + frame clock now pace to the *achieved*
refresh reported in VirtualOutput::preferred_mode, not the requested rate.
- capture/linux: PortalCapturer::Drop sends a pipewire channel quit and joins the
thread, so a dropped/failed/retried capturer releases its PipeWire thread + EGL/
CUDA context promptly instead of leaking it to process exit. First-frame timeout
now reports the node id and distinguishes "format never negotiated" from
"negotiated but no buffers arrived" via a negotiated flag set in param_changed.
- vdisplay/kwin: set_custom_refresh reads back the active mode from kscreen-doctor
and returns the refresh KWin actually gave us (a rejected custom mode silently
leaves the output at 60Hz); create() carries it into preferred_mode.
- vdisplay/gamescope: find_gamescope_node requires the Video/Source object (the
node.name=gamescope tag is on two objects; the other wedges the link); a version
check warns on <3.16.22 (the PipeWire-1.6 capture-deadlock signature).
Live-validated against headless KWin: 720p120 build with requested=120 achieved=120,
zero-copy CUDA frames, and no per-session thread accumulation across back-to-back
sessions. Tests: +3 unit (retry classifier, gamescope version parse); 49 host tests
green, clippy/fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler