unom/punktfunk
6
0
Fork 1
Code Issues 3 Pull Requests 3 Actions 1 Packages Projects Releases 45 Wiki Activity

feat(client/apple): the phone's gyro can speak for a gyro-less pad

Browse Source 
Opt-in "Gyro from this device" (DefaultsKey.gyroFromDevice, off by default,
iOS only): while player 1's forwarded controller reports no rotation rate of
its own — no GCMotion, or the gravity-only motion an Xbox pad exposes — this
device's IMU sources pad 0's wire motion instead. The rumble-on-device
mirror's sibling, data flowing the other way: same session-scoped
UserDefaults read, same hardware-gated settings rows, same pad-0 rule.

DeviceGyro wraps CMDeviceMotion at the ~100 Hz CoreMotion ceiling on a
dedicated serial queue (not main — the controller path's main-queue delivery
is a known jitter source), converts with the shared GamepadWire constants,
and rotates each sample from the device's portrait frame into the controller
frame by interface orientation, so a phone clipped landscape yaws when the
player yaws instead of rolling. The remap matrix is derived and pinned by
DeviceGyroRemapTests.

GamepadCapture owns engage/stand-down (reconcile, suspend/resume, stop), and
suppresses pad 0's controller-motion forwarding while the mirror runs — two
writers on one pad's motion state would fight, and the accel-only stream
would stomp the mirror's gyro with zeros.

Also fixes the stale-motion latch from the gyro sweep on the controller
path: flush now parks motion at zero (keeping the last accel, so gravity
doesn't become free-fall), and the mirror's stop sends the same closing
zero. The host holds motion as state and re-emits it — a nonzero angular
velocity left behind read as endless rotation for as long as an overlay
(Control Center pull-down) kept the app inactive.
This commit is contained in:
enricobuehler
2026-08-07 11:38:13 +02:00
parent 3608de25ed
commit 1f54b75c5c
7 changed files with 378 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
clients/apple/Sources/PunktfunkClient/Settings/GamepadSettingsView.swift
+17
View File
@@ -85,6 +85,7 @@ struct GamepadSettingsView: View {
#endif
#if os(iOS)
@AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false
@AppStorage(DefaultsKey.gyroFromDevice) private var gyroFromDevice = false
#endif
@ObservedObject private var gamepads = GamepadManager.shared
/// The profile catalog (ProfileStore.shared, like every other surface that reads it) — the
@@ -649,6 +650,22 @@ struct GamepadSettingsView: View {
value: $rumbleOnDevice),
at: at + 1)
}
// The phone-gyro mirror sits beside the rumble mirror: same clip-on-pad audience,
// opposite data direction. Hidden where the device has no motion hardware; engages
// in-session only while player 1's controller reports no rotation rate of its own.
if DeviceGyro.isAvailable,
let anchor = list.firstIndex(where: { $0.id == "deviceRumble" })
?? list.firstIndex(where: { $0.id == "padType" }) {
list.insert(
toggleRow(
id: "deviceGyro", tab: .controller,
icon: "gyroscope",
label: "Gyro from this device",
detail: "When the controller has no gyro, send this device's motion "
+ "sensors as player 1's — for clip-on pads without one of their own.",
value: $gyroFromDevice),
at: anchor + 1)
}
#endif
return list + profileRows
}
clients/apple/Sources/PunktfunkClient/Settings/SettingsView+Sections.swift
+9
View File
@@ -712,6 +712,15 @@ extension SettingsView {
Toggle("Rumble on this iPhone", isOn: $rumbleOnDevice)
}
}
// The rumble mirror's sibling, data flowing the other way: hidden where the
// device has no motion hardware, engages only while the player-1 controller
// reports no rotation rate of its own.
if !inProfileScope, DeviceGyro.isAvailable {
described("When the controller has no gyro of its own, sends this device's "
+ "motion sensors as player 1's — for clip-on pads without one.") {
Toggle("Gyro from this device", isOn: $gyroFromDevice)
}
}
#endif
#if !os(tvOS)
if !inProfileScope {
clients/apple/Sources/PunktfunkClient/Settings/SettingsView.swift
+1
View File
@@ -91,6 +91,7 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.pointerCapture) var pointerCapture = true
@AppStorage(DefaultsKey.touchMode) var touchMode = TouchInputMode.trackpad.rawValue
@AppStorage(DefaultsKey.rumbleOnDevice) var rumbleOnDevice = false
@AppStorage(DefaultsKey.gyroFromDevice) var gyroFromDevice = false
// The sidebar selection drives the detail pane on iPad and the pushed sub-page on iPhone.
// Width class decides the initial value: nil on iPhone (show the category list first),
// General on iPad (a two-column layout should never open with an empty detail).
clients/apple/Sources/PunktfunkKit/Gamepad/DeviceGyro.swift
+201
View File
@@ -0,0 +1,201 @@
// The opt-in phone-gyro mirror (`DefaultsKey.gyroFromDevice`): when player 1's forwarded
// controller has no rotation sensor of its own, THIS device's IMU speaks for it on the wire's
// motion plane — for clip-on and third-party pads that ship without a gyro, where the phone
// body is rigidly attached to (or simply is) the thing in the player's hands. The sibling of
// `GamepadFeedback`'s rumble-on-device mirror, with the data flowing the other way.
//
// GamepadCapture owns the engage/stand-down decision (it knows the pad-0 slot and whether its
// controller reports a rotation rate); this class only turns CoreMotion on and off and converts
// samples. Two invariants it enforces itself:
// - one motion writer per pad: samples go out only between `start` and `stop`, and capture
// suppresses pad 0's controller-motion forwarding while this runs;
// - no stale rotation: `stop` sends a single zero-gyro sample after the last real one, so the
// host's virtual pad never keeps integrating an angular velocity this device stopped
// producing (the gyro-sweep "stale angular velocity re-sent forever" failure mode).
//
// Samples are CMDeviceMotion (sensor-fused: bias-corrected rotation rate, gravity split from
// user acceleration) at the ~100 Hz CoreMotion ceiling — below a DualSense's 250 Hz, but the
// host's motion plane is event-driven, not cadence-locked, so a slower producer just means
// fewer samples. Units and axis semantics match `GamepadCapture.forwardMotion` exactly (the
// `GamepadWire` constants; accel = gravity + user acceleration — the same convention, so a
// future sign/scale correction lands in one place for both sources). The one thing the phone
// adds is a frame remap: CoreMotion reports in the device's portrait frame, while the wire
// wants the controller frame the player sees (x right, y up, z out of the screen), so each
// sample is rotated by the current interface orientation — a phone clipped landscape must yaw
// when the player yaws, not roll.
#if os(iOS)
import CoreMotion
import Foundation
import UIKit
/// Device-frame → controller-frame axis remap for one interface orientation. CoreMotion's
/// frame is fixed to the portrait device (+x right edge, +y top, +z out of the screen); the
/// controller frame keeps +z (the screen always faces the player) and rotates x/y so they
/// mean "player's right" and "player's up". Derived, like the wire scale constants — pinned
/// by `DeviceGyroRemapTests`, correctable in one place if on-glass says otherwise.
/// File-scope rather than nested so the sample thread can use it without actor isolation.
enum DeviceGyroRemap {
case identity
/// Upside-down portrait: both in-plane axes flip.
case flipped
/// Landscape, device top to the player's LEFT (interface `.landscapeRight`):
/// player-right = device-bottom, player-up = device-right.
case topLeft
/// Landscape, device top to the player's RIGHT (interface `.landscapeLeft`).
case topRight
init(_ orientation: UIInterfaceOrientation) {
switch orientation {
case .portraitUpsideDown: self = .flipped
case .landscapeRight: self = .topLeft
case .landscapeLeft: self = .topRight
default: self = .identity
}
}
/// Rotate one device-frame vector (rotation rate or acceleration — both transform the
/// same way under an in-plane rotation) into the controller frame.
func apply(x: Float, y: Float, z: Float) -> (x: Float, y: Float, z: Float) {
switch self {
case .identity: return (x, y, z)
case .flipped: return (-x, -y, z)
case .topLeft: return (-y, x, z)
case .topRight: return (y, -x, z)
}
}
}
@MainActor
public final class DeviceGyro {
/// Whether this device can source motion at all — gates the settings rows (a device
/// without an IMU would make the toggle a silent no-op, the rumble mirror's rule).
/// One shared probe: Apple recommends a single `CMMotionManager` per app, and the
/// settings UI asking per-render must not allocate one each time.
public static let isAvailable: Bool = CMMotionManager().isDeviceMotionAvailable
/// Everything the sample thread touches, behind one lock: the orientation remap (written
/// on main when the device rotates), the last converted accel, and whether a real sample
/// went out (so `stop` knows it owes the wire a zero). Kept off the actor deliberately —
/// `forward` runs on the delivery queue.
private final class SampleState: @unchecked Sendable {
let lock = NSLock()
var remap: DeviceGyroRemap = .identity
var sentSample = false
/// Re-sent with the closing zero-gyro sample so "rotation stopped" doesn't also
/// overwrite a plausible gravity vector with free-fall.
var lastAccel: (Int16, Int16, Int16) = (0, 0, 0)
}
/// Ship one converted sample (wire pad 0). Must be thread-safe — invoked from the
/// delivery queue (`PunktfunkConnection.sendMotion` locks internally).
private let send: @Sendable (_ gyro: (Int16, Int16, Int16), _ accel: (Int16, Int16, Int16)) -> Void
private let motion = CMMotionManager()
/// Dedicated serial delivery queue — deliberately NOT main (the controller path's
/// main-queue delivery is a known jitter source; the mirror starts clean).
private let queue: OperationQueue = {
let q = OperationQueue()
q.name = "punktfunk.device-gyro"
q.maxConcurrentOperationCount = 1
return q
}()
private let state = SampleState()
private var orientationObserver: NSObjectProtocol?
/// Whether the mirror is between `start` and `stop` — read by GamepadCapture to keep the
/// controller path off pad 0's motion while this runs.
public private(set) var isRunning = false
public init(
send: @escaping @Sendable (_ gyro: (Int16, Int16, Int16), _ accel: (Int16, Int16, Int16)) -> Void
) {
self.send = send
}
/// Begin sourcing pad-0 motion from this device. Idempotent.
public func start() {
guard !isRunning, motion.isDeviceMotionAvailable else { return }
isRunning = true
updateRemap()
// Interface orientation only changes alongside a device-orientation notification, so
// this is the one signal needed; re-reading the scene keeps a rotation lock stable.
orientationObserver = NotificationCenter.default.addObserver(
forName: UIDevice.orientationDidChangeNotification, object: nil, queue: .main
) { [weak self] _ in
MainActor.assumeIsolated { self?.updateRemap() }
}
// CoreMotion's practical ceiling; requesting faster just clamps.
motion.deviceMotionUpdateInterval = 1.0 / 100.0
motion.startDeviceMotionUpdates(to: queue) { [state, send] m, _ in
guard let m else { return }
Self.forward(m, state: state, send: send)
}
}
/// Stop sourcing and, if anything was sent, park the host pad's rotation at zero. The
/// zero rides the same serial queue as the samples, so it is guaranteed last — without
/// blocking the caller.
public func stop() {
guard isRunning else { return }
isRunning = false
motion.stopDeviceMotionUpdates()
if let o = orientationObserver {
NotificationCenter.default.removeObserver(o)
orientationObserver = nil
}
queue.addOperation { [state, send] in
state.lock.lock()
let owed = state.sentSample
state.sentSample = false
let accel = state.lastAccel
state.lock.unlock()
if owed { send((0, 0, 0), accel) }
}
}
private func updateRemap() {
let o = UIApplication.shared.connectedScenes
.compactMap { $0 as? UIWindowScene }
.first?.interfaceOrientation ?? .portrait
state.lock.lock()
state.remap = DeviceGyroRemap(o)
state.lock.unlock()
}
/// Runs on the delivery queue: remap, scale, ship.
nonisolated private static func forward(
_ m: CMDeviceMotion, state: SampleState,
send: (_ gyro: (Int16, Int16, Int16), _ accel: (Int16, Int16, Int16)) -> Void
) {
state.lock.lock()
let r = state.remap
state.lock.unlock()
let rot = r.apply(
x: Float(m.rotationRate.x), y: Float(m.rotationRate.y), z: Float(m.rotationRate.z))
// Same total-acceleration convention as GamepadCapture.forwardMotion.
let acc = r.apply(
x: Float(m.gravity.x + m.userAcceleration.x),
y: Float(m.gravity.y + m.userAcceleration.y),
z: Float(m.gravity.z + m.userAcceleration.z))
let gs = GamepadWire.gyroLSBPerRadS
let as_ = GamepadWire.accelLSBPerG
let gyro = (
GamepadWire.motionRaw(rot.x, scale: gs),
GamepadWire.motionRaw(rot.y, scale: gs),
GamepadWire.motionRaw(rot.z, scale: gs)
)
let accel = (
GamepadWire.motionRaw(acc.x, scale: as_),
GamepadWire.motionRaw(acc.y, scale: as_),
GamepadWire.motionRaw(acc.z, scale: as_)
)
state.lock.lock()
state.lastAccel = accel
state.sentSample = true
state.lock.unlock()
send(gyro, accel)
}
}
#endif
clients/apple/Sources/PunktfunkKit/Gamepad/GamepadCapture.swift
+80 -12
View File
@@ -67,6 +67,13 @@ public final class GamepadCapture {
var axes: [Int32] = [0, 0, 0, 0, 0, 0]
var fingerActive: [Bool] = [false, false]
var lastMotionNs: UInt64 = 0
/// A motion sample went out on this pad — `flush` then owes the wire a zero-gyro
/// sample: the host holds motion as STATE and re-emits it, so a nonzero angular
/// velocity left behind reads as endless rotation (the gyro-sweep latch).
var motionSent = false
/// The last accel sent, re-used by the flush zero so "rotation stopped" doesn't
/// also replace a plausible gravity vector with free-fall.
var lastAccel: (Int16, Int16, Int16) = (0, 0, 0)
// Hold-Select→guide gesture state (pf-client-core's `SelectGesture`, adapted to
// this class's mask-diff model): a Select pressed ALONE is held out of the mask
// until it resolves into a tap (delivered on release) or — past `guideHold` — a
@@ -153,6 +160,15 @@ public final class GamepadCapture {
/// everywhere but macOS). See `guideHold`.
public let guideGesture: Bool
#if os(iOS)
/// Opt-in phone-gyro mirror (`DefaultsKey.gyroFromDevice`): while player 1's forwarded
/// controller has no rotation sensor, this device's IMU sources pad 0's motion instead —
/// for clip-on pads without a gyro. Session-scoped (the setting is read once here); nil
/// when off, unavailable, or forwarding is off (the mirror is wire-only, so with nothing
/// to send there is nothing to mirror). Engage/stand-down lives in `updateDeviceGyro`.
private let deviceGyro: DeviceGyro?
#endif
public init(
connection: PunktfunkConnection, manager: GamepadManager, forwarding: Bool = true,
systemForward: Bool = true, guideGesture: Bool = false
@@ -162,6 +178,17 @@ public final class GamepadCapture {
self.forwarding = forwarding
self.systemForward = systemForward
self.guideGesture = guideGesture
#if os(iOS)
if forwarding, DeviceGyro.isAvailable,
UserDefaults.standard.bool(forKey: DefaultsKey.gyroFromDevice) {
deviceGyro = DeviceGyro { [weak connection] gyro, accel in
// Thread-safe (sendMotion locks); pad 0 by the same rule as the rumble mirror.
connection?.sendMotion(pad: 0, gyro: gyro, accel: accel)
}
} else {
deviceGyro = nil
}
#endif
}
public func start() {
@@ -187,6 +214,9 @@ public final class GamepadCapture {
MainActor.assumeIsolated {
self?.suspended = true
self?.releaseAll()
// The mirror pauses with capture (its stop parks the host pad's rotation
// at zero — an overlay pull-down must not leave the game spinning).
self?.updateDeviceGyro()
}
})
observers.append(NotificationCenter.default.addObserver(
@@ -199,11 +229,15 @@ public final class GamepadCapture {
for slot in self.slots {
if let ext = slot.controller.extendedGamepad { self.sync(slot, ext) }
}
self.updateDeviceGyro()
}
})
}
public func stop() {
#if os(iOS)
deviceGyro?.stop()
#endif
closeAllSlots()
forwardedSub = nil
observers.forEach { NotificationCenter.default.removeObserver($0) }
@@ -224,6 +258,8 @@ public final class GamepadCapture {
}
// A chord-holding pad may have just unplugged — re-evaluate so a stale hold disarms.
updateEscapeChord()
// Pad 0 may have changed hands — re-evaluate whether this device's IMU speaks for it.
updateDeviceGyro()
}
/// Open one forwarded controller on its assigned wire index: attach GC handlers, claim its
@@ -561,6 +597,13 @@ public final class GamepadCapture {
private func forwardMotion(_ slot: Slot, _ m: GCMotion) {
guard !suspended else { return }
#if os(iOS)
// While the phone-gyro mirror speaks for pad 0, the controller's own motion —
// necessarily rotation-less, that's the engage condition — stays off the wire:
// two writers on one pad's motion state would fight, and this accel-only stream
// would keep stomping the mirror's gyro with zeros.
if slot.pad == 0, deviceGyro?.isRunning == true { return }
#endif
let now = DispatchTime.now().uptimeNanoseconds
guard now &- slot.lastMotionNs >= Self.motionIntervalNs else { return }
slot.lastMotionNs = now
@@ -579,18 +622,35 @@ public final class GamepadCapture {
}
let gs = GamepadWire.gyroLSBPerRadS
let as_ = GamepadWire.accelLSBPerG
wire?.sendMotion(
pad: UInt8(slot.pad),
gyro: (
GamepadWire.motionRaw(Float(m.rotationRate.x), scale: gs),
GamepadWire.motionRaw(Float(m.rotationRate.y), scale: gs),
GamepadWire.motionRaw(Float(m.rotationRate.z), scale: gs)
),
accel: (
GamepadWire.motionRaw(ax, scale: as_),
GamepadWire.motionRaw(ay, scale: as_),
GamepadWire.motionRaw(az, scale: as_)
))
let gyro = (
GamepadWire.motionRaw(Float(m.rotationRate.x), scale: gs),
GamepadWire.motionRaw(Float(m.rotationRate.y), scale: gs),
GamepadWire.motionRaw(Float(m.rotationRate.z), scale: gs)
)
let accel = (
GamepadWire.motionRaw(ax, scale: as_),
GamepadWire.motionRaw(ay, scale: as_),
GamepadWire.motionRaw(az, scale: as_)
)
if wire != nil {
slot.motionSent = true
slot.lastAccel = accel
}
wire?.sendMotion(pad: UInt8(slot.pad), gyro: gyro, accel: accel)
}
/// Engage or stand down the phone-gyro mirror: it speaks for pad 0 exactly while a
/// forwarded controller holds that index but can't rotate for itself — no `GCMotion`,
/// or a motion object without a rotation rate (gravity-only pads, e.g. an Xbox pad on
/// iOS). Re-evaluated on every reconcile and on suspend/resume; `DeviceGyro.stop`
/// parks the host pad's rotation at zero, so standing down never strands a spin.
private func updateDeviceGyro() {
#if os(iOS)
guard let gyro = deviceGyro else { return }
let pad0 = slots.first { $0.pad == 0 }
let wants = !suspended && pad0 != nil && pad0!.controller.motion?.hasRotationRate != true
if wants { gyro.start() } else { gyro.stop() }
#endif
}
/// Arm the disconnect timer when ANY forwarded pad holds the full escape chord, disarm the
@@ -634,6 +694,14 @@ public final class GamepadCapture {
wire?.sendTouchpad(pad: UInt8(slot.pad), finger: UInt8(f), active: false, x: 0, y: 0)
slot.fingerActive[f] = false
}
// Motion is host-side STATE, re-emitted until replaced — a nonzero angular velocity
// left behind reads as endless rotation (the gyro-sweep latch: Control Center
// pull-down froze the last sample for as long as the overlay stayed up). Rest means
// zero rotation; the last accel is kept so gravity doesn't become free-fall.
if slot.motionSent {
slot.motionSent = false
wire?.sendMotion(pad: UInt8(slot.pad), gyro: (0, 0, 0), accel: slot.lastAccel)
}
}
/// Flush every open slot's held state (app deactivation) — keeps the slots open (GC just stops
clients/apple/Sources/PunktfunkShared/DefaultsKeys.swift
+8
View File
@@ -193,6 +193,14 @@ public enum DefaultsKey {
/// once per session by `GamepadFeedback`. The toggle is shown only where the device actually
/// has a haptic actuator (no iPad/Mac/TV).
public static let rumbleOnDevice = "punktfunk.rumbleOnDevice"
/// Use this device's own gyroscope as player 1's motion when the forwarded controller has
/// none of its own — for clip-on and third-party pads without an IMU, where the device body
/// moves with the player's hands. The rumble mirror's sibling, data flowing the other way.
/// Off by default (opt-in); read once per session by `GamepadCapture`, whose `DeviceGyro`
/// mirror engages only while pad 0's controller reports no rotation rate (a real gyro pad
/// always wins). The toggle is shown only where the device has motion hardware
/// (`DeviceGyro.isAvailable`).
public static let gyroFromDevice = "punktfunk.gyroFromDevice"
/// Auto-wake on connect: when connecting to a saved host that isn't advertising on mDNS, fire
/// Wake-on-LAN and, if the dial fails, wait for it to come back before retrying (the "Waking…"
/// overlay). On by default. Turn off if a host that's already on just isn't seen on mDNS (a
clients/apple/Tests/PunktfunkKitTests/DeviceGyroRemapTests.swift
+62
View File
@@ -0,0 +1,62 @@
// Pins the phone-gyro mirror's device→controller frame remap (DeviceGyro.swift). The matrix is
// derived (like the wire scale constants), so these tests are the contract: if on-glass says an
// axis is wrong, fix the enum AND these expectations together.
#if os(iOS)
import UIKit
import XCTest
@testable import PunktfunkKit
final class DeviceGyroRemapTests: XCTestCase {
/// A distinct vector per axis so a swapped or flipped component can't cancel out.
private let v: (x: Float, y: Float, z: Float) = (1, 2, 3)
func testPortraitIsIdentity() {
let r = DeviceGyroRemap.identity.apply(x: v.x, y: v.y, z: v.z)
XCTAssertEqual([r.x, r.y, r.z], [1, 2, 3])
}
func testUpsideDownFlipsInPlane() {
let r = DeviceGyroRemap.flipped.apply(x: v.x, y: v.y, z: v.z)
XCTAssertEqual([r.x, r.y, r.z], [-1, -2, 3])
}
/// Device top to the player's LEFT: player-right = device-bottom (−y), player-up =
/// device-right (+x). z (out of the screen) never changes — the screen faces the player.
func testTopLeftLandscape() {
let r = DeviceGyroRemap.topLeft.apply(x: v.x, y: v.y, z: v.z)
XCTAssertEqual([r.x, r.y, r.z], [-2, 1, 3])
}
/// Device top to the player's RIGHT: player-right = device-top (+y), player-up =
/// device-left (−x).
func testTopRightLandscape() {
let r = DeviceGyroRemap.topRight.apply(x: v.x, y: v.y, z: v.z)
XCTAssertEqual([r.x, r.y, r.z], [2, -1, 3])
}
/// Interface orientation → remap: `.landscapeRight` means the Home edge is on the
/// player's right, i.e. the device top points LEFT (and vice versa).
func testOrientationMapping() {
XCTAssertEqual(DeviceGyroRemap(.portrait), .identity)
XCTAssertEqual(DeviceGyroRemap(.portraitUpsideDown), .flipped)
XCTAssertEqual(DeviceGyroRemap(.landscapeRight), .topLeft)
XCTAssertEqual(DeviceGyroRemap(.landscapeLeft), .topRight)
XCTAssertEqual(DeviceGyroRemap(.unknown), .identity)
}
/// Every remap must stay a proper rotation (right-handed): x̂ × ŷ = ẑ after mapping.
func testHandednessPreserved() {
for remap in [DeviceGyroRemap.identity, .flipped, .topLeft, .topRight] {
let x = remap.apply(x: 1, y: 0, z: 0)
let y = remap.apply(x: 0, y: 1, z: 0)
// Cross product of the two mapped in-plane basis vectors.
let cross = (
x: x.y * 0 - 0 * y.y, y: 0 * y.x - x.x * 0, z: x.x * y.y - x.y * y.x
)
XCTAssertEqual(cross.z, 1, "left-handed remap: \(remap)")
}
}
}
#endif