feat(apple): iOS/iPadOS client — touch, pointer lock, shared SwiftUI shell
ci / rust (push) Has been cancelled

The whole client now runs on iPadOS/iOS from the same sources, first-lit live in the
iPad simulator against the real host at 1280x720@60 (60 fps on the HUD, capture state
machine active, mic permission flow shown).

- PunktfunkCore.xcframework grows iOS device + universal-simulator slices
  (BUILD_IOS=1; rustup targets aarch64-apple-ios{,-sim} + x86_64-apple-ios).
- The decode pump is extracted into a shared StreamPump (identical IDR re-gate logic on
  both platforms); the iOS StreamView (StreamViewIOS.swift) has the same name/signature
  as the macOS one, so ContentView & co. are byte-identical across platforms — hosted
  in a UIViewController for prefersPointerLocked (the iPadOS cursor capture; see README
  note 9 for the UIHostingController forwarding caveat).
- Touch is always forwarded: per-finger wire ids, coordinates mapped through the
  aspect-fit letterbox into LIVE host-mode pixels (surface == host mode, identity
  rescale host-side; follows mid-stream requestMode switches).
- InputCapture is cross-platform: GC works the same on iPadOS, ⌘⎋ is detected from the
  HID stream there; stale-⌘ tracking after focus loss fixed on both platforms
  (releaseAll now drops the modifier/latch state — a ⌘ released in another app
  otherwise hijacked Esc forever).
- SessionAudio: AVAudioSession on iOS (.playAndRecord + .defaultToSpeaker — without it
  iPhones route host audio to the EARPIECE; deactivated with
  notifyOthersOnDeactivation on stop so interrupted background audio resumes); HAL
  device pinning + the Settings pickers stay macOS-only.
- New Punktfunk-iOS app target (shared synchronized sources, generated Info.plist with
  mic + local-network usage descriptions — QUIC to a LAN host trips local network
  privacy on real devices — scene manifest + indirect input events for Stage Manager /
  external displays), shared scheme, macOS min-window frames gated off iOS.

For the iPad-on-an-external-screen idea: with multiple scenes + indirect input enabled,
Stage Manager iPads can drag the punktfunk window onto the external display and drive
the PC with keyboard/mouse/touch. Known gaps (README note 9): the pointer-lock
preference isn't consulted through UIHostingController (relative mouse works, the local
cursor just stays visible) and AVAudioSession interruptions don't auto-restart audio.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-06-11 11:18:18 +02:00
parent 136390514d
commit e1af4d57c6
14 changed files with 766 additions and 73 deletions
@@ -8,7 +8,9 @@
// the only way into hosts running --require-pairing. Once pinned, reconnects are silent
// and a changed host identity refuses to connect.
#if os(macOS)
import AppKit
#endif
import PunktfunkKit
import SwiftUI
@@ -21,6 +23,9 @@ struct ContentView: View {
@AppStorage("punktfunk.compositor") private var compositor = 0
@State private var showAddHost = false
@State private var pairingTarget: StoredHost?
#if os(iOS)
@State private var showSettings = false
#endif
var body: some View {
Group {
@@ -70,7 +75,9 @@ struct ContentView: View {
trustCard(fp)
}
}
#if os(macOS)
.frame(minWidth: 640, minHeight: 360)
#endif
.background(Color.black)
}
@@ -106,17 +113,38 @@ struct ContentView: View {
.help("Add a host")
}
ToolbarItem {
#if os(macOS)
SettingsLink {
Label("Settings", systemImage: "gearshape")
}
.help("Stream mode and settings")
#else
Button {
showSettings = true
} label: {
Label("Settings", systemImage: "gearshape")
}
#endif
}
}
}
#if os(macOS)
.frame(minWidth: 480, minHeight: 360)
#endif
.sheet(isPresented: $showAddHost) {
AddHostSheet { store.add($0) }
}
#if os(iOS)
.sheet(isPresented: $showSettings) {
NavigationStack {
SettingsView()
.navigationTitle("Settings")
.toolbar {
Button("Done") { showSettings = false }
}
}
}
#endif
.alert(
"Connection failed",
isPresented: Binding(
@@ -239,7 +267,11 @@ struct ContentView: View {
model.rejectTrust()
pairingTarget = host
}
#if os(macOS)
.buttonStyle(.link)
#else
.buttonStyle(.borderless)
#endif
.font(.callout)
}
.padding(28)
@@ -287,11 +319,20 @@ struct ContentView: View {
.font(.system(.caption, design: .monospaced))
// While captured the cursor is hidden+frozen, so the button is keyboard-only
// ( or Cmd+Tab release the cursor; released, it's clickable again).
#if os(macOS)
Text(model.mouseCaptured
? "⌘⎋ releases the mouse"
: "Click the stream to capture input")
.font(.caption2)
.opacity(0.8)
#else
// Touch always plays directly; (hardware keyboard) toggles kb/mouse.
Text(model.mouseCaptured
? "⌘⎋ releases keyboard & mouse"
: "⌘⎋ captures keyboard & mouse")
.font(.caption2)
.opacity(0.8)
#endif
Button("Disconnect (⌘D)") { model.disconnect() }
.font(.caption)
.keyboardShortcut("d", modifiers: .command)