feat(apple): mic uplink + touch events in PunktfunkKit
ci / rust (push) Has been cancelled

Adopts the new ABI surface (still v2, additive):

- PunktfunkConnection.sendMic(_:seq:ptsNs:) — Opus mic frames (48 kHz) to the host's
  virtual PipeWire source; enqueue-only, empty data = DTX silence. Wiring the actual
  Mac microphone (AVAudioEngine input → Opus) into the app is the follow-up, alongside
  audio playback (README note 5).
- PunktfunkInputEvent.touchDown/touchMove/touchUp — absolute pixels + surface size in
  flags, host injects via libei ei_touchscreen. Built for the iOS variant; nothing on
  macOS emits them yet.
- Loopback round trip now also sends touch events and mic frames (incl. a DTX frame)
  through the wrapper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-06-11 09:08:04 +02:00
parent 6575dddac7
commit 3a51551f97
3 changed files with 52 additions and 2 deletions
+3 -1
View File
@@ -118,7 +118,9 @@ signing, bundle id `io.unom.punktfunk`. Notes:
host stamps `pts_ns` with its capture wall clock; across machines you need a clock
offset estimate from the QUIC RTT).
5. **Audio**: `nextAudio()` yields raw Opus packets (48 kHz stereo, one 5 ms frame each,
sequence-numbered). Decode with libopus or `AVAudioConverter`/`kAudioFormatOpus` into an
sequence-numbered). The inverse direction exists too: `sendMic(_:seq:ptsNs:)` uplinks
the client's mic as Opus frames into a virtual PipeWire source on the host (wire it
to AVAudioEngine input + an Opus encoder alongside playback). Decode with libopus or `AVAudioConverter`/`kAudioFormatOpus` into an
`AVAudioEngine` source node; conceal gaps (drop/dup) rather than blocking — the Rust
side buffers 320 ms and drops the newest packet when the puller lags. Wall-clock
`ptsNs` shares the host clock with video AUs for A/V sync. Wiring this into