feat(apple): iOS/iPadOS client — touch, pointer lock, shared SwiftUI shell
ci / rust (push) Has been cancelled
ci / rust (push) Has been cancelled
The whole client now runs on iPadOS/iOS from the same sources, first-lit live in the
iPad simulator against the real host at 1280x720@60 (60 fps on the HUD, capture state
machine active, mic permission flow shown).
- PunktfunkCore.xcframework grows iOS device + universal-simulator slices
(BUILD_IOS=1; rustup targets aarch64-apple-ios{,-sim} + x86_64-apple-ios).
- The decode pump is extracted into a shared StreamPump (identical IDR re-gate logic on
both platforms); the iOS StreamView (StreamViewIOS.swift) has the same name/signature
as the macOS one, so ContentView & co. are byte-identical across platforms — hosted
in a UIViewController for prefersPointerLocked (the iPadOS cursor capture; see README
note 9 for the UIHostingController forwarding caveat).
- Touch is always forwarded: per-finger wire ids, coordinates mapped through the
aspect-fit letterbox into LIVE host-mode pixels (surface == host mode, identity
rescale host-side; follows mid-stream requestMode switches).
- InputCapture is cross-platform: GC works the same on iPadOS, ⌘⎋ is detected from the
HID stream there; stale-⌘ tracking after focus loss fixed on both platforms
(releaseAll now drops the modifier/latch state — a ⌘ released in another app
otherwise hijacked Esc forever).
- SessionAudio: AVAudioSession on iOS (.playAndRecord + .defaultToSpeaker — without it
iPhones route host audio to the EARPIECE; deactivated with
notifyOthersOnDeactivation on stop so interrupted background audio resumes); HAL
device pinning + the Settings pickers stay macOS-only.
- New Punktfunk-iOS app target (shared synchronized sources, generated Info.plist with
mic + local-network usage descriptions — QUIC to a LAN host trips local network
privacy on real devices — scene manifest + indirect input events for Stage Manager /
external displays), shared scheme, macOS min-window frames gated off iOS.
For the iPad-on-an-external-screen idea: with multiple scenes + indirect input enabled,
Stage Manager iPads can drag the punktfunk window onto the external display and drive
the PC with keyboard/mouse/touch. Known gaps (README note 9): the pointer-lock
preference isn't consulted through UIHostingController (relative mouse works, the local
cursor just stays visible) and AVAudioSession interruptions don't auto-restart audio.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
+26
-3
@@ -97,7 +97,10 @@ signing, bundle id `io.unom.punktfunk`. Notes:
|
||||
- **Tests from Xcode**: the package tests run with `swift test`; to get them on ⌘U, add
|
||||
`PunktfunkKitTests` once via Edit Scheme → Test → + (Xcode persists it into the shared
|
||||
scheme — a hand-written package-test reference doesn't resolve headlessly).
|
||||
- `xcodebuild -project Punktfunk.xcodeproj -scheme Punktfunk build` works headlessly.
|
||||
- `xcodebuild -project Punktfunk.xcodeproj -scheme Punktfunk build` works headlessly;
|
||||
same for `-scheme Punktfunk-iOS -destination 'generic/platform=iOS Simulator'` (run it
|
||||
in a simulator via `xcrun simctl install/launch` — `SIMCTL_CHILD_PUNKTFUNK_AUTOCONNECT=…`
|
||||
passes the dev autoconnect env through).
|
||||
|
||||
## Notes for whoever picks this up next
|
||||
|
||||
@@ -170,8 +173,28 @@ signing, bundle id `io.unom.punktfunk`. Notes:
|
||||
while the app has focus, and focus loss also auto-releases everything held. One live capture per process (the GC
|
||||
mouse/keyboard singletons have a single handler slot — ownership is tracked so a stale
|
||||
capture's stop() can't clobber a newer one).
|
||||
9. **iOS**: same package (`BUILD_IOS=1` for the xcframework slice); `StreamView` needs the
|
||||
`UIViewRepresentable` twin and touch→input mapping.
|
||||
9. **iOS/iPadOS — ported and first-lit** (iPad simulator ↔ the real host, 60 fps).
|
||||
`BUILD_IOS=1 bash scripts/build-xcframework.sh` builds device + universal-simulator
|
||||
slices; the Xcode project has a second target, **Punktfunk-iOS**, sharing the same
|
||||
synchronized sources. The iOS `StreamView` (StreamViewIOS.swift — same name/signature
|
||||
as the macOS one, so the SwiftUI shell is identical) hosts the shared `StreamPump` in
|
||||
a view controller for `prefersPointerLocked`: with a hardware mouse/trackpad that is
|
||||
the iPadOS cursor capture (system honors it fullscreen-and-frontmost; in Stage
|
||||
Manager it degrades to both-cursors forwarding). Touch is always forwarded — every
|
||||
finger gets a wire touch id and coordinates map through the aspect-fit letterbox
|
||||
into host-mode pixels (surface == host mode, so the host rescale is the identity).
|
||||
`InputCapture` is cross-platform (GC works the same on iPadOS; ⌘⎋ is detected from
|
||||
the HID stream there); audio routes via `AVAudioSession` (the Settings device
|
||||
pickers are macOS-only). For the iPad-with-external-display setup: the target
|
||||
enables multiple scenes + indirect input events — on Stage Manager iPads, drag the
|
||||
punktfunk window onto the external screen and the stream runs there with full
|
||||
keyboard/mouse/touch. Known gaps: `prefersPointerLocked` is declared on the stream
|
||||
view controller but UIHostingController doesn't forward the preference from
|
||||
representable children, so the system cursor stays visible (relative-mouse
|
||||
forwarding works regardless — fixing it means putting the controller into the UIKit
|
||||
presentation chain, e.g. a full-screen UIKit presentation on session start); and
|
||||
AVAudioSession interruptions (calls, Siri) don't auto-restart the audio engines yet
|
||||
(reconnect recovers).
|
||||
|
||||
## Known limitations of the current host (relevant to client UX)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user