Compare commits

..
Author SHA1 Message Date
enricobuehler 7c964e95c6 fix(host,audio): the rtkit boost was applied to a thread that does not run the capture callback
ci / bun-nix (pull_request) Successful in 20s
ci / rust-arm64 (pull_request) Successful in 1m18s
ci / web (pull_request) Successful in 4m10s
ci / docs-site (pull_request) Successful in 4m36s
ci / rust (pull_request) Successful in 4m50s
android / android (pull_request) Successful in 7m52s
The comment above it asserted "the stream's `process` callbacks run ON this
mainloop thread (we never hand PipeWire a separate data loop)". We do: the
stream is created with `RT_PROCESS`, so libpipewire runs `process()` on a data
loop it creates and schedules itself.

Measured in one live host process on 2026-08-15:

    punktfunk-pw-au   SCHED_OTHER  nice 0     <- the thread we boost
    data-loop.0       SCHED_RR     prio 20    <- the thread running process()

This is not a stale-comment nit. #232 shipped the rtkit boost to answer a field
report of audio stutter, its success line was read as evidence that the capture
callback had been prioritised, and the follow-up round concluded priorities were
"engaged but insufficient" — when they had never been applied to the thread in
question. Whether the capture callback is realtime decides whether a Wine shader
storm can deschedule it for tens of milliseconds at a 2.7 ms quantum, which is
the exact shape of the one field signature still unexplained (~2 stalls/s of
~30 ms with the node reporting itself continuously Streaming).

So the assumption is replaced by a measurement rather than a guess about which
thread to boost:

- `pf_frame::thread_qos::current_thread_sched()` reports the calling thread's
  policy, RT priority and nice. Three by-value syscalls, no allocation and no
  blocking, so it is safe to call from an RT callback.
- The capture callback reports its own scheduling once per open. Every future
  field log now states what the audio path actually runs as, instead of what we
  asked for somewhere else.

The boost itself is kept: this thread still dispatches state and format events,
and it IS the capture thread when `PUNKTFUNK_STREAM_SINK=0` selects the legacy
monitor path.

Deliberately not attempted here: boosting the data loop. rtkit is a blocking
D-Bus call and must never run inside an RT audio callback, and on the one host
that could be measured PipeWire already gives that thread SCHED_RR/20 — a nice
boost would be inert. Ship the instrument first; a host that reports
SCHED_OTHER here is the evidence that would justify the plumbing.

Gated in the amd64 CI container: fmt, clippy -D warnings on punktfunk-host and
pf-frame, pf-frame tests (incl. a non-vacuity test that the introspection
returns a policy the kernel could have named), and the full punktfunk-host suite
at 560 passed. The one failure, mgmt::tests::local_summary_is_loopback_only_and_
non_sensitive, is the recorded process-global parallel-test race: it passes when
run alone, which is the documented discriminator, and it is untouched by this.
2026-08-15 20:13:14 +02:00
enricobuehler 2563041e88 feat(host,audio): a 16-second outage and a starved stream reported the same numbers
Measured on a live host running this commit's parent: our PipeWire capture
stream spent 16.2 s of one window in `Paused`, and the line said

    delivered_pct=63 gaps=0 max_gap_ms=0

Every number is correct. `gaps` scores inter-callback deltas and a stream that
is not scheduled fires no callbacks, so there is nothing to score — that is
deliberate, and `a_paused_span_is_not_scored` pins it. But the outage did not
vanish; it moved into `delivered_pct`, because the reporting window is flushed
from the process callback and therefore STRETCHES by exactly the time we were
absent. The only explanation lived in the state DEBUG lines, which a field
journal at INFO does not carry.

So a shortfall had two possible causes and no way to tell them apart: a sink
nobody was rendering into (benign — the 2026-08-15 logs show ~40 s of it at
every session start, at peak_db=-120.0, i.e. digital silence), or a capture path
losing real audio under load (Skynet, 5-8 % through loud gameplay). Those want
opposite responses and cost days of investigation to separate by hand.

Three changes, one theme — make the line answer the question it invites:

- `pauses` / `paused_ms` beside the percentage they explain. `gaps` keeps its
  narrow meaning (holes inside a running stream); absence is counted separately,
  because a burst of sub-10 ms holes is a scheduling problem on the box and a
  multi-second pause is our node not being in the graph at all.

- The quantum now tracks what the graph is actually handing us instead of
  latching the first callback of the open. A graph re-plans whenever anything
  else on the box asks for a different latency, and the stale value silently
  corrupted the very threshold gaps are scored against. A new size must survive
  three callbacks before it is believed, so one short buffer cannot move it.

- `audio egress` — the send path had no periodic metric of any kind. Across five
  field logs it emitted 14 lines, all the same session banner, which made "the
  host paces audio badly" unfalsifiable and left it on the suspect list forever.
  It now reports sent/infilled/late/max_late_ms/max_spacing_ms/reanchors on the
  same 30 s window as capture, so the two read as a pair: holes at the tap with
  clean departures means the host delivered everything it had. Note `reanchors`
  in particular — the pacer forgives accumulated debt silently, and that is
  precisely the event that leaves no trace and then gets blamed on the network.

Windows keeps its own field set; its capture model differs (loopback stops
delivering while the endpoint idles) and zero-valued fields would imply it had
measured something it did not.

Gated in the amd64 CI container: fmt, clippy -D warnings, 18 capture-policy
tests (4 new, plus the pause pair sitting next to the test that pins the
blindness they answer).
2026-08-15 20:05:10 +02:00
enricobuehler 4ee095220f The in-flight budget tests still assumed buffer-only accounting (#254)
ci / bun-nix (push) Successful in 25s
ci / rust-arm64 (push) Successful in 1m29s
apple / swift (push) Successful in 2m8s
windows-client / client (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m4s
deb / build-publish (push) Successful in 4m31s
deb / build-publish-gamescope (push) Successful in 22s
ci / rust (push) Successful in 5m54s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 13s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 8s
ci / web (push) Successful in 6m37s
deb / build-publish-host (push) Successful in 5m9s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 31s
docker / builders (ci/gamescope-trixie.Dockerfile, punktfunk-gamescope-trixie) (push) Successful in 11s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
deb / build-publish-client-arm64 (push) Successful in 1m12s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 24s
ci / docs-site (push) Successful in 6m55s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 51s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m15s
docker / builders-arm64cross (push) Successful in 7s
docker / deploy-docs (push) Successful in 32s
windows-client / client (x64, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m40s
deb / smoke-install (push) Successful in 2m44s
arch / build-publish (push) Successful in 11m58s
android / android (push) Successful in 13m41s
apple / distribute (push) Successful in 11m30s
flatpak / build-publish (push) Successful in 7m48s
apple / screenshots (push) Successful in 9m28s
windows-host / package (push) Successful in 13m13s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 15s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m41s
2026-08-15 16:41:33 +00:00
enricobuehler d9d877f985 Merge pull request 'The shared clipboard now works on iPhone and iPad' (#250) from worktree-worktree-ios-clipboard into main
apple / swift (push) Successful in 1m55s
ci / bun-nix (push) Successful in 28s
ci / web (push) Successful in 1m16s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 12s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 11s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 8s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / builders (ci/gamescope-trixie.Dockerfile, punktfunk-gamescope-trixie) (push) Successful in 8s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
ci / docs-site (push) Successful in 1m28s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 21s
docker / builders-arm64cross (push) Successful in 7s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m11s
docker / deploy-docs (push) Successful in 34s
apple / distribute (push) Successful in 10m53s
ci / rust-arm64 (push) Successful in 12m27s
apple / screenshots (push) Successful in 9m34s
ci / rust (push) Canceled after 17m50s
Reviewed-on: #250
2026-08-15 16:15:12 +00:00
enricobuehler 8c4a41913f The pad sink wore a profile name GE-Proton matches nothing on, so neither haptics nor speaker could route (#252)
ci / rust-arm64 (push) Successful in 2m1s
ci / bun-nix (push) Successful in 2m38s
ci / rust (push) Canceled after 3m21s
ci / web (push) Canceled after 3m6s
ci / docs-site (push) Canceled after 3m6s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/gamescope-trixie.Dockerfile, punktfunk-gamescope-trixie) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
deb / build-publish-gamescope (push) Successful in 28s
deb / build-publish (push) Successful in 3m54s
deb / build-publish-client-arm64 (push) Successful in 3m8s
android / android (push) Successful in 7m50s
arch / build-publish (push) Successful in 11m51s
deb / build-publish-host (push) Successful in 8m53s
deb / smoke-install (push) Successful in 2m35s
windows-host / package (push) Successful in 16m8s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 13m5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 12m21s
Merged with ci/rust red. That check fails identically on unmodified main at the base commit f04da20f (run 18516) and reproduces locally with --test-threads=1, in punktfunk-core::packet::tests — a crate this branch does not touch. Pre-existing breakage from 59d8b8a6, tracked separately.
2026-08-15 16:11:45 +00:00
enricobuehler a7da3e4cb0 Merge pull request 'The host knew about every one of these faults and had no way to say so' (#253) from worktree-console-diagnostics into main
ci / rust-arm64 (push) Successful in 2m26s
ci / bun-nix (push) Successful in 55s
deb / build-publish (push) Successful in 4m33s
ci / web (push) Successful in 8m6s
ci / rust (push) Failing after 8m55s
deb / build-publish-gamescope (push) Successful in 1m58s
ci / docs-site (push) Successful in 8m31s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 49s
arch / build-publish (push) Successful in 11m38s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 1m26s
deb / build-publish-client-arm64 (push) Successful in 3m37s
deb / build-publish-host (push) Successful in 5m8s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 50s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 1m40s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 14s
docker / builders (ci/gamescope-trixie.Dockerfile, punktfunk-gamescope-trixie) (push) Successful in 34s
docker / builders-arm64cross (push) Successful in 18s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m8s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m17s
android / android (push) Canceled after 15m26s
docker / deploy-docs (push) Successful in 31s
deb / smoke-install (push) Canceled after 1m52s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 2m1s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 2m0s
windows-host / package (push) Canceled after 15m33s
windows-host / canary-manifest (push) Canceled after 0s
windows-host / winget-source (push) Canceled after 0s
Web-console diagnostics v1 (design/web-console-diagnostics.md WP1–WP3).

Merged with `ci / rust` red, deliberately: that job's Test step fails on two
punktfunk-core tests — packet::tests::in_flight_buffer_budget_bounds_allocation and
packet::tests::streamed_open_commits_its_own_extent_and_stays_bounded — which this
branch cannot affect (it touches zero files under crates/punktfunk-core).

The same two fail on main's own push run (18516, sha f04da20f) and reproduce locally
on a clean checkout. Root cause: 59d8b8a6 "meter per-block reassembly state against
the in-flight budget" changed reassemble.rs without updating packet/tests.rs, so both
tests still assert the pre-change budget arithmetic. Fixing those expectations is a
separate, focused change against that commit's intent — not something to guess at from
inside a console feature.

Every other gate is green on this branch: fmt, unsafe-hygiene, clippy -D warnings (both
the default and native-only trees), build, rust-arm64, web, docs-site, bun-nix.
2026-08-15 15:56:19 +00:00
enricobuehler 998e5d3379 docs: answer the "set the controller audio to Pro Audio" question our sink cannot offer a switch for
android / android (pull_request) Successful in 4m38s
ci / web (pull_request) Successful in 2m3s
ci / rust-arm64 (pull_request) Successful in 2m6s
ci / docs-site (pull_request) Successful in 1m50s
ci / bun-nix (pull_request) Successful in 4m0s
ci / rust (pull_request) Failing after 15m36s
The community fix for DualSense haptics on Linux is to open sound settings and flip the pad's
card profile to Pro Audio, and a field report compared that against our minted sink: theirs has a
Profile dropdown reading "Pro Audio", ours has no dropdown at all. That reads like something is
missing, and it is worth saying plainly that it is not — a real pad is a USB sound card and gets
a profile selector, ours is a software device with no card, and the layout the selector exists to
reach is the one we are already minted in.

New page covers the whole feature end to end: what a DualSense's speaker and voice coils actually
are (a four-channel audio stream, not rumble), the USB-only requirement on the client, GE-Proton
11-5 on the host, why every other card profile folds the coil channels away, the three host log
lines that say how far it got, and the per-game Proton launch options that force GE onto its most
direct route. That last one now has teeth: with the sink advertising a split parent, GE's
`PROTON_DUALSENSE_HAPTICS_PREFER_NON_EVENT=1` opens our node by name and writes the four channels
in with no remix anywhere in between.

Known limits are stated rather than left to be rediscovered: Bluetooth pads have no audio
interface, container-ID matching cannot work behind a uhid pad with no USB device to derive one
from, and a real pad plugged into the host itself can win the match against the one you are
streaming to.
2026-08-15 17:33:44 +02:00
enricobuehler bec2d193c4 fix(audio): the pad sink wore a profile name GE-Proton matches nothing on, so neither haptics nor speaker could route
Everything a title needs to find the DualSense's audio device on a Linux host is a substring of
the sink's name or a key in its proplist, and ours were chosen against an older reading of
GE-Proton. Three of them were wrong, and each one cost a different piece of the feature.

**The profile suffix.** We minted `-00.analog-surround-40`. GE's `is_dualsense_speaker_sink()` is
a substring test for `Speaker__sink`, and three things hang off it: the pad-SPEAKER (mono
controller-effect) streams only bind and retarget to a sink it accepts, its
`apply_windows_sony_audio_format()` forces the wine endpoint to the Windows 4x48 kHz
`KSAUDIO_SPEAKER_QUAD` layout DS5 titles probe for, and the endpoint lands on the identity
Spider-Man's working path used. `analog-surround-40` matched none of it — the speaker half of
this feature had nothing to attach to at all. Now `-00.HiFi__Speaker__sink`.

The channels stay Pro Audio's four raw AUX, which is deliberately not the same real-pad profile
the name comes from. Since alsa-ucm-conf gained `USB-Audio/Sony/DualSense-PS5` a real pad's
profiles are UCM SplitPCM views of one 4-channel PCM — a mono Speaker, a stereo Headphones, a
4-channel Direct "for wine compatibility" — and all of them are POSITIONED. GE renders haptics as
an `AUX0..AUX3` stream, so on every one of those the graph re-mixes and the voice-coil pair is
folded away; only Pro Audio's raw AUX node passes it through by index. That is the whole content
of the field advice "you only need the controller audio set to Pro Audio". No single real profile
satisfies both halves, so we take the name from one and the layout from the other.

**`api.alsa.split.name` was absent.** GE reads it off the sink it is about to render haptics into
and opens THAT node through its bundled pipewire-alsa plugin as `pipewire:NODE=<name>` with
`aux_channels=1`. Without the key `get_dualsense_haptic_target()` returns NULL and the leg cannot
engage — including for the titles GE auto-switches into "Windows Sony audio mode" after eight
format probes. We have no split, so the honest value is our own node name;
`PUNKTFUNK_PAD_SINK_SPLIT_NAME` drops or overrides it in the field. Carrying `Speaker__sink` and
a split parent at once is a real pad's shape rather than a contrivance: GE's
`is_dualsense_endpoint_speaker_sink` notes that "Edge speaker sinks may also carry raw haptic
metadata" and handles the pair by keeping them as routing targets while withholding the shared
mono endpoint id.

**The node name broke its own matcher.** It carried an invented `DualSense_` infix, but a plain
DualSense's USB iProduct string is just "Wireless Controller" — only the Edge has a model word —
and that infix split the contiguous `Sony_Interactive_Entertainment_Wireless_Controller` substring
the community WirePlumber rule and GE's own name-only fallback key on.

The proplist now also states the shape (`audio.position`, `audio.channels`) and the ALSA card
names; pipewire-pulse hands every one of these to a Proton client verbatim. Mint-time logging and
the `pad-sink-test` devtest print the strings a title has to match, plus the channel-map a
hand-driven test file needs to reach the coils at all.
2026-08-15 17:33:39 +02:00
enricobuehler 37d39295aa test(core): the in-flight budget tests still assumed buffer-only accounting
apple / swift (pull_request) Successful in 1m59s
apple / distribute (pull_request) Skipped
apple / screenshots (pull_request) Skipped
ci / rust-arm64 (pull_request) Successful in 1m41s
ci / web (pull_request) Successful in 1m26s
android / android (pull_request) Successful in 5m19s
ci / bun-nix (pull_request) Successful in 22s
ci / docs-site (pull_request) Successful in 1m25s
windows-client / client (arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (pull_request) Successful in 3m9s
windows-client / client (x64, , x86_64-pc-windows-msvc, C:\t) (pull_request) Successful in 6m33s
ci / rust (pull_request) Successful in 20m48s
`59d8b8a6` (security-review 2026-08-15 finding 11) started metering BlockState against the
in-flight budget, because both its vectors are sized from attacker-declared header fields and a
slice-streamed frame could otherwise mint thousands of unmetered blocks. That fix is right and
stays. What it missed is that two tests encode the OLD cost model as arithmetic in their
comments — "exactly 32 such frames fit; the 33rd must be refused", "four fit the budget, the
fifth must be refused" — so a stricter, more correct budget reads as a failure:

  in_flight_buffer_budget_bounds_allocation          7 drops, expected 1
  streamed_open_commits_its_own_extent_and_stays_bounded   2 drops, expected 1

Both numbers are exactly what the new metering predicts (a 512 B buffer + 104 B of block state
takes 26 of the 16384 B budget, not 32; a ceiling-claiming 4096 B open takes 3, not 4), so the
tests were measuring the hole rather than the firewall. main has been red on `ci / rust` since.

Derive the refusal boundary from the cost model instead of baking in a frame count: the tests now
ask block_state_bytes() how much a frame commits, push exactly one frame past what fits, and
assert the same property as before — everything under the budget accepted, the one past it
dropped. A future field on BlockState moves the boundary and the tests follow it, rather than
failing with an arithmetic puzzle that invites re-hardcoding whatever number CI last printed.

Also assert the invariant the counts were only ever a proxy for: `in_flight() <= budget` at the
end of each. That one catches a release site forgetting half the cost — the accounting-drift
failure `in_flight`'s own doc comment warns about, which surfaces in the field as a permanent
loss storm once the budget wedges.

IN_FLIGHT_BUF_FACTOR and block_state_bytes become pub(super) (the LOSS_WINDOW_NS precedent);
no production behaviour changes.

Verified: punktfunk-core 414/414 with --all-features (the superset of CI's failing target),
`cargo fmt --all --check` clean, clippy -D warnings clean. Mutation-checked: inflating
IN_FLIGHT_BUF_FACTOR 1000x makes both tests fail with 0 drops, so neither went vacuous.
2026-08-15 17:01:45 +02:00
enricobuehler 8740f48c92 feat(apple): the shared clipboard now works on iPhone and iPad
ci / web (pull_request) Successful in 1m11s
ci / docs-site (pull_request) Successful in 1m16s
apple / swift (pull_request) Successful in 2m2s
apple / distribute (pull_request) Skipped
apple / screenshots (pull_request) Skipped
ci / bun-nix (pull_request) Successful in 4m29s
ci / rust-arm64 (pull_request) Successful in 5m55s
ci / rust (pull_request) Failing after 14m20s
The clipboard bridge was written against NSPasteboard and gated `#if os(macOS)`,
so the iOS half of the same universal app had a per-host toggle it could not
show and a wire plane it never opened — even though the core's clipboard ABI is
in every slice of the framework and nothing below the pasteboard was
platform-specific.

Rather than keep a second copy of the subtle half (one drain thread, offer
sequence numbers, the pending fetches a blocked paste waits on, echo
suppression), that logic moves into a `ClipboardPasteboard` seam and stays
shared. What genuinely differs is small and lives in two adapters: AppKit
fulfils a paste by blocking a provider thread, UIKit by answering an
NSItemProvider load handler; AppKit transcodes images through NSImage, UIKit
through UIImage. macOS behaviour is unchanged — same poll interval, same
timeouts, same lock discipline.

Two things the iOS side needs that the Mac does not.

Backgrounding the app ends the session, so a lazy promise on the pasteboard
would outlive anything able to answer it and "copy on the host, switch to
Safari, paste" would hand Safari nothing. So a host offer still unpasted when
the sync tears down is pulled across then — bounded to 8 MiB and 3 seconds,
skipped entirely if it was already pasted. Everything else stays lazy: copy on
the host, stay in the app, paste nothing, and no clipboard bytes move.

And since iOS 14 reading pasteboard *contents* is a privacy event the user
sees, while reading its change count and type list is not. That maps onto the
lazy design exactly, so the announce poll stays silent however long a session
runs; the one real read happens when someone on the host pastes. It now runs
off the drain thread, because on iOS 16+ that read can sit waiting for the
user's answer, and the drain thread is what would deliver the host's cancel.

One bug fixed while moving the code: ownership of the pasteboard was recorded
from a write's resulting change count without checking the write had landed. A
dropped write would have made the sync read the user's own next copy as its own
echo and never announce it again.

tvOS has no pasteboard, so the guards became `#if !os(tvOS)` rather than
widening to every platform.

Verified: macOS `swift build` + 319 tests green (15 new), iOS and tvOS
typechecks via the `--triple` recipe, and `xcodebuild` of the shipping
Punktfunk-iOS scheme.
2026-08-15 15:56:44 +02:00
25 changed files with 1774 additions and 405 deletions
@@ -30,10 +30,10 @@ struct AddHostSheet: View {
@State private var pinnedIDs: Set<String>
@ObservedObject private var profiles = ProfileStore.shared
#endif
#if os(macOS)
/// Share the clipboard with this host (macOS sessions only; design
/// clipboard-and-file-transfer.md §5.3). Off by default; honored only when the host
/// advertises the capability at connect.
#if !os(tvOS)
/// Share the clipboard with this host (design clipboard-and-file-transfer.md §5.3). Off by
/// default; honored only when the host advertises the capability at connect. Absent on tvOS,
/// which has no pasteboard to share.
@State private var clipboardSync: Bool
#endif
#if os(tvOS)
@@ -72,7 +72,7 @@ struct AddHostSheet: View {
_port = State(initialValue: Int(existing?.port ?? 9777))
let stored = existing?.macAddresses ?? []
_mac = State(initialValue: (stored.isEmpty ? suggestedMacs : stored).joined(separator: ", "))
#if os(macOS)
#if !os(tvOS)
_clipboardSync = State(initialValue: existing?.clipboardSync ?? false)
#endif
#if !os(tvOS)
@@ -144,7 +144,7 @@ struct AddHostSheet: View {
#if os(iOS)
.textInputAutocapitalization(.never)
#endif
#if os(macOS)
#if !os(tvOS)
Toggle("Share clipboard with this host", isOn: $clipboardSync)
#endif
profileRows
@@ -200,11 +200,11 @@ struct AddHostSheet: View {
}
#if os(iOS)
/// Four fields + the action row a touch taller than the 3-field add sheet used to be. The
/// edit sheet's profile rows are the only thing that can outgrow it, and they say by how much;
/// a single fixed number is what clipped them.
/// Four fields, the clipboard toggle, and the action row. The edit sheet's profile rows are
/// the only thing that can outgrow it, and they say by how much; a single fixed number is what
/// clipped them.
private var sheetHeight: CGFloat {
var height: CGFloat = 392
var height: CGFloat = 392 + 44 // the fields and action row, plus the clipboard toggle
if showsProfileRows {
height += 116 // the Profile picker and its footnote
height += 96 + CGFloat(profiles.profiles.count) * 44 // the pins, their header + footer
@@ -282,7 +282,7 @@ struct AddHostSheet: View {
host.address = address.trimmingCharacters(in: .whitespaces)
host.port = UInt16(clamping: port)
host.macAddresses = Self.parseMacs(mac)
#if os(macOS)
#if !os(tvOS)
// nil when off: the key stays absent from the saved JSON (forward-compat, and "never
// opted in" and "opted out" read the same off).
host.clipboardSync = clipboardSync ? true : nil
@@ -250,14 +250,14 @@ final class SessionModel: ObservableObject {
private var audio: SessionAudio?
private var gamepadCapture: GamepadCapture?
private var gamepadFeedback: GamepadFeedback?
#if os(macOS)
#if !os(tvOS)
/// The live session's clipboard bridge (design/clipboard-and-file-transfer.md §5) created
/// by `beginStreaming` when the per-host toggle is on and the host advertises
/// `HOST_CAP_CLIPBOARD`; stopped (off-main, drain joined) in `disconnect`.
private var clipboardSync: ClipboardSync?
#endif
/// Whether clipboard sync is live (host-acked `ClipState.enabled`) drives the Stream menu
/// item's title and the settings footnote. Always false off-macOS.
/// item's title and the settings footnote. Always false on tvOS, which has no pasteboard.
@Published private(set) var clipboardEnabled = false
/// The host's last `ClipState.reason` (`CLIP_REASON_*`) why an enable was refused
/// (backend unavailable / policy disabled / ); 0 = OK.
@@ -768,7 +768,7 @@ final class SessionModel: ObservableObject {
#endif
let feedback = gamepadFeedback
gamepadFeedback = nil
#if os(macOS)
#if !os(tvOS)
let clipboard = clipboardSync
clipboardSync = nil
#endif
@@ -781,8 +781,11 @@ final class SessionModel: ObservableObject {
Task.detached {
audio?.stop()
feedback?.stop()
#if os(macOS)
clipboard?.stop() // disables sync on the wire while the connection is still up
#if !os(tvOS)
// Disables sync on the wire while the connection is still up and on iOS pulls a
// host offer the user has not pasted yet down to real bytes, which needs that
// connection, so it must stay ahead of the close below.
clipboard?.stop()
#endif
// Deliberate user quit tell the host to skip the keep-alive linger (must precede close).
if deliberate { conn.disconnectQuit() }
@@ -792,7 +795,7 @@ final class SessionModel: ObservableObject {
Task.detached {
audio?.stop()
feedback?.stop()
#if os(macOS)
#if !os(tvOS)
clipboard?.stop()
#endif
}
@@ -940,7 +943,7 @@ final class SessionModel: ObservableObject {
let feedback = GamepadFeedback(connection: conn, manager: .shared)
feedback.start()
gamepadFeedback = feedback
#if os(macOS)
#if !os(tvOS)
// Shared clipboard: opt-in per host AND host-advertised (older hosts / operator-disabled
// hosts never see a ClipControl) AND granted to this device (per-client access §5
// without the bit the host would refuse with CLIP_REASON_NOT_PERMITTED anyway; not
@@ -958,7 +961,7 @@ final class SessionModel: ObservableObject {
#endif
}
#if os(macOS)
#if !os(tvOS)
/// Create + start the session's clipboard bridge and route its host acks into the published
/// UI state. `ClipboardSync.start()` sends the enable; the host's `.state` answer flips
/// `clipboardEnabled` (or leaves it false with a `clipboardReason` the UI can explain).
@@ -977,9 +980,9 @@ final class SessionModel: ObservableObject {
/// Flip clipboard sync mid-session (the Stream menu). Off on requires the host cap; on
/// off tears the bridge down (off-main the drain join must not block the main actor) and
/// tells the host, which drops any selection we own there. No-op off-macOS or while idle.
/// tells the host, which drops any selection we own there. No-op on tvOS or while idle.
func toggleClipboardSync() {
#if os(macOS)
#if !os(tvOS)
guard let conn = connection, phase == .streaming else { return }
if let sync = clipboardSync {
clipboardSync = nil
@@ -22,8 +22,7 @@ import SwiftUI
/// `.focusedSceneValue` so the Scene-level commands can drive it.
struct SessionFocus {
var isStreaming: Bool
/// The connected host advertises `HOST_CAP_CLIPBOARD` (gates the Share Clipboard item
/// macOS-only UI, but the fact is platform-neutral).
/// The connected host advertises `HOST_CAP_CLIPBOARD` (gates the Share Clipboard item).
var clipboardAvailable: Bool
/// Clipboard sync is live (host-acked) drives the item's Stop/Share title.
var clipboardOn: Bool
@@ -78,14 +77,15 @@ struct StreamCommands: Commands {
}
.keyboardShortcut("a", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true || session?.micAvailable != true)
#if os(macOS)
// Mid-session clipboard flip (design/clipboard-and-file-transfer.md §5.3). Greyed
// when the host doesn't advertise the cap (older host / operator policy off).
// when the host doesn't advertise the cap (older host / operator policy off). On iPad
// there is no menu bar to show it in, but a hardware keyboard still reaches it.
Button(session?.clipboardOn == true ? "Stop Sharing Clipboard" : "Share Clipboard") {
session?.toggleClipboard()
}
.keyboardShortcut("c", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true || session?.clipboardAvailable != true)
#if os(macOS)
// Toggle the window's fullscreen. F is the macOS-standard fullscreen combo; here it's
// explicit so it's discoverable AND survives capture while streaming the stream view
// swallows keys, so InputCapture's monitor detects the same combo and posts the same
@@ -0,0 +1,76 @@
// The shared clipboard's format vocabulary (design/clipboard-and-file-transfer.md §3.5), stated
// once for AppKit and UIKit alike.
//
// Every Apple pasteboard type in the table IS a uniform type identifier, and both frameworks name
// them with the same strings `NSPasteboard.PasteboardType.png` and the UIPasteboard type
// `"public.png"` are the same bytes. Keeping the table as plain strings is therefore not a
// lowest-common-denominator compromise; it is the actual shared spelling, and it keeps the two
// platform adapters from drifting apart in what they announce.
#if !os(tvOS)
import Foundation
enum ClipboardFormats {
/// Wire MIME uniform type identifier, in announce order. Files
/// (`application/x-punktfunk-files`) ride Phase 2 and are absent here.
///
/// Original image formats sit beside the mandatory `image/png` floor rather than replacing it:
/// a copied JPEG never balloons into PNG and a GIF keeps its animation, while a peer that can
/// only place PNG still has something to take.
static let table: [(wire: String, uti: String)] = [
("text/plain;charset=utf-8", "public.utf8-plain-text"),
("text/rtf", "public.rtf"),
("text/html", "public.html"),
("image/png", "public.png"),
("image/jpeg", "public.jpeg"),
("image/gif", "com.compuserve.gif"),
]
/// Pasteboard marker types that must never cross the wire password managers mark secrets
/// with these (see nspasteboard.org). A Mac convention that costs nothing to honour on iOS:
/// the cross-platform managers set them there too, and a pasteboard that carries neither is
/// unaffected.
static let concealed = "org.nspasteboard.ConcealedType"
static let transient = "org.nspasteboard.TransientType"
/// Image types we do not announce verbatim but CAN serve `image/png` from by transcoding at
/// fetch time screenshots and Preview leave TIFF, the camera roll leaves HEIC.
static let pngSources = ["public.tiff", "public.heic"]
static func uti(forWire wire: String) -> String? {
table.first { $0.wire == wire }?.uti
}
static func wire(forUti uti: String) -> String? {
table.first { $0.uti == uti }?.wire
}
/// True when the pasteboard is carrying a secret and must be ignored entirely.
static func isConcealed(_ types: [String]) -> Bool {
types.contains(concealed) || types.contains(transient)
}
/// The format list to announce for a pasteboard holding `types` the lazy offer's whole
/// payload (§3.2). Empty means "nothing we sync", which legitimately clears the peer's side.
static func offerKinds(forTypes types: [String]) -> [PunktfunkConnection.ClipKind] {
var kinds = table
.filter { types.contains($0.uti) }
.map { PunktfunkConnection.ClipKind(mime: $0.wire) }
// PNG floor: announce the portable `image/png` whenever ANY convertible image is present
// native PNG, TIFF/HEIC, or a JPEG/GIF original already being offered verbatim above. The
// adapters convert at fetch time, so the fallback costs nothing unless a peer pastes it.
if !kinds.contains(where: { $0.mime == "image/png" }),
types.contains(where: { pngSources.contains($0) })
|| kinds.contains(where: { $0.mime.hasPrefix("image/") })
{
kinds.append(PunktfunkConnection.ClipKind(mime: "image/png"))
}
return kinds
}
/// The uniform types to place for a remote offer, in the table's order and skipping kinds this
/// client has no mapping for (files, and whatever a future host learns to offer).
static func placeableUtis(for kinds: [PunktfunkConnection.ClipKind]) -> [String] {
kinds.compactMap { uti(forWire: $0.mime) }
}
}
#endif
@@ -0,0 +1,115 @@
// The macOS half of the clipboard seam: `NSPasteboard.general`.
//
// AppKit's lazy-paste contract is a blocking one `provideDataForType` is called on a provider
// thread the moment a Mac app pastes, and whatever the item holds when that call returns is what
// the app gets. So this adapter is the one place that turns the asynchronous fetch into a wait.
#if os(macOS)
import AppKit
import Foundation
typealias SystemPasteboard = AppKitPasteboard
final class AppKitPasteboard: ClipboardPasteboard {
private let pb = NSPasteboard.general
private var activationObserver: NSObjectProtocol?
/// The provider backing the offer currently on the pasteboard. AppKit's own reference to it is
/// not something to rely on: nothing crosses if it is collected before the user pastes.
private var provider: BlockingOfferProvider?
var changeCount: Int { pb.changeCount }
var typeIdentifiers: [String] { (pb.types ?? []).map(\.rawValue) }
/// Read one wire format, converting where macOS stores a different native type: `image/png` is
/// served from a real `.png` entry when present, else converted from whatever image
/// representation the pasteboard holds (TIFF from screenshots and Preview, WebP/AVIF/GIF from
/// browsers `NSImage` decodes them all) into PNG at fetch time.
func read(wire: String) -> Data? {
guard wire == "image/png" else {
guard let uti = ClipboardFormats.uti(forWire: wire) else { return nil }
return pb.data(forType: NSPasteboard.PasteboardType(uti))
}
if let png = pb.data(forType: .png) {
return png
}
guard let img = NSImage(pasteboard: pb),
let tiff = img.tiffRepresentation,
let rep = NSBitmapImageRep(data: tiff)
else {
return nil
}
return rep.representation(using: .png, properties: [:])
}
func installLazy(utis: [String], fetch: @escaping ClipboardFetch) -> Int {
let provider = BlockingOfferProvider(fetch: fetch)
let item = NSPasteboardItem()
item.setDataProvider(provider, forTypes: utis.map { NSPasteboard.PasteboardType($0) })
pb.clearContents()
pb.writeObjects([item])
self.provider = provider
return pb.changeCount
}
/// Unused on macOS a promise here outlives any paste that might come, so there is never
/// cause to resolve one early. Implemented anyway so the seam has no platform-shaped hole.
func installResolved(_ items: [(uti: String, data: Data)]) -> Int {
let item = NSPasteboardItem()
for (uti, data) in items {
item.setData(data, forType: NSPasteboard.PasteboardType(uti))
}
pb.clearContents()
pb.writeObjects([item])
provider = nil
return pb.changeCount
}
func clear() -> Int {
pb.clearContents()
provider = nil
return pb.changeCount
}
/// A Mac keeps running after a session ends, but the promise dies with the sync regardless, so
/// there is nothing to be gained by spending a round-trip on it at teardown clearing leaves
/// the user exactly where they were.
let resolvesPendingOfferOnTeardown = false
func startObserving(onActivate: @escaping () -> Void) {
activationObserver = NotificationCenter.default.addObserver(
forName: NSApplication.didBecomeActiveNotification, object: nil, queue: nil
) { _ in onActivate() }
}
func stopObserving() {
if let activationObserver {
NotificationCenter.default.removeObserver(activationObserver)
self.activationObserver = nil
}
}
}
/// The lazy paste hook: AppKit calls `provideDataForType` only when a Mac app actually pastes; the
/// fetch then blocks this provider thread (never main) until the host's bytes arrive. On timeout
/// or a dead session it provides nothing, so the paste inserts nothing rather than hanging.
private final class BlockingOfferProvider: NSObject, NSPasteboardItemDataProvider {
private let fetch: ClipboardFetch
init(fetch: @escaping ClipboardFetch) {
self.fetch = fetch
}
func pasteboard(
_ pasteboard: NSPasteboard?, item: NSPasteboardItem,
provideDataForType type: NSPasteboard.PasteboardType
) {
guard let wire = ClipboardFormats.wire(forUti: type.rawValue) else { return }
let box = ClipboardResultBox()
fetch(wire) { box.settle($0) }
// The fetch enforces its own deadline and always completes; this is only a backstop
// against a lost completion wedging an AppKit thread forever.
guard let data = box.wait(timeout: ClipboardSync.fetchTimeout + 2) else { return }
item.setData(data, forType: type)
}
}
#endif
@@ -0,0 +1,121 @@
// The iOS/iPadOS half of the clipboard seam: `UIPasteboard.general`.
//
// Two things differ from AppKit in ways that shape the code here.
//
// **Laziness is asynchronous.** UIKit promises data with `NSItemProvider`, whose load handler is
// handed a completion rather than a return value, so this adapter passes the fetch straight
// through no thread is blocked waiting for a paste to resolve.
//
// **Reading the pasteboard is a privacy event.** Since iOS 14 the system tells the user when an
// app reads pasteboard *contents*, and since iOS 16 it asks first when the content came from
// another app. Reading *metadata* the change count, the list of type identifiers does not.
// That maps exactly onto the lazy design: the announce poll only ever looks at metadata, so it is
// silent no matter how long a session runs, and the one moment a read really happens is when
// someone on the host pastes, which is a deliberate act the user is present for.
#if !os(tvOS) && !os(macOS) && canImport(UIKit)
import Foundation
import UIKit
import UniformTypeIdentifiers
typealias SystemPasteboard = UIKitPasteboard
final class UIKitPasteboard: ClipboardPasteboard {
private let pb = UIPasteboard.general
private var activationObserver: NSObjectProtocol?
var changeCount: Int { pb.changeCount }
/// `types` reports the identifiers present without touching a single byte of content, so the
/// poll costs the user nothing and raises no banner.
var typeIdentifiers: [String] { pb.types }
/// Read one wire format, converting where iOS stores a different native type: `image/png` is
/// served from a real PNG entry when present, else re-encoded from whatever image the
/// pasteboard holds a photo copied out of Photos is HEIC, a screenshot may arrive as TIFF,
/// and neither is something a host can be expected to place.
///
/// This is the one call that reads contents, and on iOS 16+ it can put a permission alert in
/// front of the user and wait for their answer. `ClipboardSync` calls it off the drain thread
/// for exactly that reason.
func read(wire: String) -> Data? {
guard wire == "image/png" else {
guard let uti = ClipboardFormats.uti(forWire: wire) else { return nil }
if let data = pb.data(forPasteboardType: uti) {
return data
}
// UIPasteboard stores plain text as a string rather than a data representation often
// enough that the typed read comes back empty on content we can plainly see.
guard uti == UTType.utf8PlainText.identifier else { return nil }
return pb.string?.data(using: .utf8)
}
if let png = pb.data(forPasteboardType: UTType.png.identifier) {
return png
}
return pb.image?.pngData()
}
func installLazy(utis: [String], fetch: @escaping ClipboardFetch) -> Int {
let provider = NSItemProvider()
for uti in utis {
guard let type = UTType(uti), let wire = ClipboardFormats.wire(forUti: uti) else {
continue
}
provider.registerDataRepresentation(for: type, visibility: .all) { completion in
fetch(wire) { data in
if let data {
completion(data, nil)
} else {
completion(nil, ClipboardOfferError.unavailable)
}
}
return nil
}
}
// `localOnly`: these bytes do not exist on this device yet they are a promise against a
// session that is about to end. Handing that to Universal Clipboard would either force an
// eager pull of everything the host ever copies or strand another device with a promise
// nothing can answer.
pb.setItemProviders([provider], localOnly: true, expirationDate: nil)
return pb.changeCount
}
func installResolved(_ items: [(uti: String, data: Data)]) -> Int {
var representations: [String: Any] = [:]
for (uti, data) in items {
representations[uti] = data
}
pb.setItems([representations], options: [.localOnly: true])
return pb.changeCount
}
func clear() -> Int {
pb.items = []
return pb.changeCount
}
/// Backgrounding the app ends the session (see `ContentView`'s scenePhase driver), and with it
/// any hope of answering a promise so an offer the user has not pasted yet is pulled down to
/// real bytes while the connection is still open.
let resolvesPendingOfferOnTeardown = true
func startObserving(onActivate: @escaping () -> Void) {
activationObserver = NotificationCenter.default.addObserver(
forName: UIApplication.didBecomeActiveNotification, object: nil, queue: nil
) { _ in onActivate() }
}
func stopObserving() {
if let activationObserver {
NotificationCenter.default.removeObserver(activationObserver)
self.activationObserver = nil
}
}
}
/// What a lazy representation reports when the host cannot supply it a stale offer, a timed-out
/// fetch, or a session that ended. UIKit shows the paste as producing nothing, which is the same
/// outcome AppKit gets by providing no data.
enum ClipboardOfferError: Error {
case unavailable
}
#endif
@@ -0,0 +1,110 @@
// The platform seam under `ClipboardSync`: everything that actually touches NSPasteboard or
// UIPasteboard, and nothing else.
//
// The sync logic above this protocol the drain thread, the offer sequence numbers, the pending
// fetches a blocked paste waits on, echo suppression is identical on macOS and iOS and is worth
// having exactly one copy of. What genuinely differs is small and lives in the two adapters:
// AppKit fulfils a paste by BLOCKING a provider thread, UIKit by answering an asynchronous load
// handler; AppKit transcodes images through NSImage, UIKit through UIImage; and only UIKit has to
// worry about the process being suspended out from under an offer it promised to serve.
#if !os(tvOS)
import Foundation
/// Pulls the bytes of one lazily-offered wire format from the host. Called on whichever thread the
/// OS fulfils a paste on never the drain thread, which has to stay free to deliver the very
/// chunks this fetch is waiting for and answers asynchronously.
typealias ClipboardFetch = (_ wire: String, _ completion: @escaping (Data?) -> Void) -> Void
/// The system pasteboard, as much of it as the shared clipboard needs.
///
/// Calls arrive from the drain thread, the serve queue, and the thread tearing the sync down;
/// `ClipboardSync` serializes them with its own lock, so an adapter need not be internally
/// synchronized. It must not, however, block on the **main** queue: that lock is also taken from
/// main, and a `main.sync` under it would deadlock.
protocol ClipboardPasteboard: AnyObject {
/// Monotonic per pasteboard write, by anyone. Reading it must never count as reading the
/// pasteboard's *contents* on iOS that distinction is the difference between a silent poll
/// and a system paste banner on every tick.
var changeCount: Int { get }
/// The uniform type identifiers currently on the pasteboard. Must be answerable WITHOUT
/// reading contents, for the same reason.
var typeIdentifiers: [String] { get }
/// Bytes for one wire format, read from the live pasteboard and transcoded where the system
/// stores a different native type (`image/png` from a TIFF screenshot). Nil when the format
/// is not really there. This one DOES read contents.
func read(wire: String) -> Data?
/// Replace the pasteboard with a single item advertising `utis`, each backed by `fetch` the
/// bytes cross only if something actually pastes. Returns the resulting `changeCount`, which
/// the caller records so it can tell its own write apart from the user's next copy.
func installLazy(utis: [String], fetch: @escaping ClipboardFetch) -> Int
/// Replace the pasteboard with concrete bytes. Only iOS needs this (see
/// `ClipboardSync.resolvePendingOffer`); on macOS a lazy promise outlives any paste that
/// might come, so the adapter there never has cause to call it.
func installResolved(_ items: [(uti: String, data: Data)]) -> Int
/// Empty the pasteboard. Returns the resulting `changeCount`.
func clear() -> Int
/// Whether a host offer still sitting unresolved on the pasteboard should be pulled down to
/// concrete bytes as the sync is torn down, instead of being dropped.
///
/// This is the difference between the two platforms' idea of how long a promise lives. A Mac
/// keeps running long after a session ends, but the promise dies with the sync either way, so
/// AppKit clears it and the user loses nothing they had before. On iOS the teardown IS the
/// user leaving backgrounding ends the session and "copy on the host, then paste into
/// Safari" is the whole point of the feature on a tablet, so those bytes have to be made real
/// while the connection that can still supply them is open.
var resolvesPendingOfferOnTeardown: Bool { get }
/// Start watching for the user coming back to the app. The case that matters is "copied
/// elsewhere, now focusing the stream to paste" the offer must reach the host before their
/// V lands, which is sooner than the announce poll would get there on its own.
func startObserving(onActivate: @escaping () -> Void)
func stopObserving()
}
/// A fetch result handed between threads: the OS fulfils a paste on one thread and the drain
/// thread produces the bytes on another.
final class ClipboardResultBox: @unchecked Sendable {
private let ready = DispatchSemaphore(value: 0)
private let lock = NSLock()
private var value: Data?
private var settled = false
func settle(_ data: Data?) {
lock.lock()
guard !settled else {
lock.unlock()
return
}
settled = true
value = data
lock.unlock()
ready.signal()
}
/// Blocks until the bytes arrive, or gives up. Never call this from the drain thread it is
/// the drain thread that delivers what is being waited for.
func wait(timeout: TimeInterval) -> Data? {
guard ready.wait(timeout: .now() + timeout) == .success else { return nil }
lock.lock()
defer { lock.unlock() }
return value
}
}
extension ClipboardPasteboard {
/// The wire formats this pasteboard is currently carrying, honouring the concealed/transient
/// markers. Nil when the pasteboard holds a secret distinct from "holds nothing we sync",
/// which is an empty list and legitimately clears the peer.
var offerKinds: [PunktfunkConnection.ClipKind]? {
let types = typeIdentifiers
guard !ClipboardFormats.isConcealed(types) else { return nil }
return ClipboardFormats.offerKinds(forTypes: types)
}
}
#endif
@@ -1,51 +1,45 @@
// Shared clipboard, macOS client half (design/clipboard-and-file-transfer.md §5.2).
// Shared clipboard, client half (design/clipboard-and-file-transfer.md §5.2). One implementation
// for macOS and iOS/iPadOS; everything that touches an actual pasteboard sits behind
// `ClipboardPasteboard`.
//
// Bridges NSPasteboard.general to the session's QUIC clipboard plane, both directions lazy:
// Both directions are lazy:
//
// * **Local copy host**: a changeCount poll announces the *format list* (`clipOffer`); the
// bytes cross only when a host app pastes (a `.fetchRequest` event, answered from the live
// pasteboard by `clipServe`).
// * **Host copy local**: a `.remoteOffer` writes one NSPasteboardItem whose
// NSPasteboardItemDataProvider fires only when a Mac app actually pastes the provider then
// blocks (on its provider thread, never main) on a `clipFetch` round-trip.
// * **Local copy host**: a changeCount poll announces the *format list* (`clipOffer`); the bytes
// cross only when a host app pastes (a `.fetchRequest` event, answered from the live pasteboard
// by `clipServe`).
// * **Host copy local**: a `.remoteOffer` places a pasteboard item whose data provider fires only
// when a local app actually pastes the provider then pulls the bytes over a `clipFetch`.
//
// Password-manager respect: pasteboards marked `org.nspasteboard.ConcealedType` or
// `org.nspasteboard.TransientType` are never announced, never fetchable. Echo suppression: the
// changeCount of every write WE make is recorded so the announce poll skips it (§3.4).
//
// Phase 1 formats only (text / RTF / HTML / PNG). Files (NSFilePromiseProvider) ride Phase 2.
#if os(macOS)
import AppKit
// Phase 1 formats only (text / RTF / HTML / PNG / JPEG / GIF). Files ride Phase 2.
#if !os(tvOS)
import Foundation
/// One live session's clipboard bridge. Created by the session model when streaming begins on a
/// host that advertises `HOST_CAP_CLIPBOARD` and whose per-host toggle is on; `stop()` before the
/// connection closes. All pasteboard traffic runs on one dedicated drain thread plus the
/// AppKit-owned provider threads (paste fulfillment).
/// connection closes. All wire traffic runs on one dedicated drain thread, plus the OS-owned
/// threads that fulfil a paste.
public final class ClipboardSync: NSObject {
/// Wire MIME NSPasteboard type for the Phase-1 vocabulary (§3.5), in announce order.
private static let wireToPasteboard: [(wire: String, type: NSPasteboard.PasteboardType)] = [
("text/plain;charset=utf-8", .string),
("text/rtf", .rtf),
("text/html", .html),
("image/png", .png),
// Original image formats pass through VERBATIM beside the PNG floor a copied JPEG
// never balloons into PNG, a GIF keeps its animation; the destination picks the richest
// kind it can place.
("image/jpeg", NSPasteboard.PasteboardType("public.jpeg")),
("image/gif", NSPasteboard.PasteboardType("com.compuserve.gif")),
]
/// Pasteboard marker types that must never cross the wire (password managers mark secrets
/// with these see nspasteboard.org).
private static let concealed = NSPasteboard.PasteboardType("org.nspasteboard.ConcealedType")
private static let transient = NSPasteboard.PasteboardType("org.nspasteboard.TransientType")
/// How long a blocked paste waits for the host's bytes before providing nothing (§5.2).
private static let fetchTimeout: TimeInterval = 10
/// How long a paste waits for the host's bytes before giving up and providing nothing (§5.2).
/// Enforced here, so an adapter that has to block a thread can treat it as a guarantee.
static let fetchTimeout: TimeInterval = 10
/// Serve chunk size for host-side pastes of our data (bounds the per-call ABI copy).
private static let serveChunk = 4 << 20
/// Announce poll interval how stale a local copy may be before the host hears about it.
private static let announceInterval: TimeInterval = 0.5
/// Ceiling on what `resolvePendingOffer` will pull. Text and modest images are worth having on
/// the chance the user pastes them after the session ends; a 200 MB screenshot is not.
private static let resolveBudget = 8 << 20
/// And how long that may hold up teardown. Short on purpose it sits between the user
/// leaving and the connection closing, and a LAN round-trip for a few KB of text is
/// milliseconds. An offer that cannot be had in this long is one the user does without.
private static let resolveTimeout: TimeInterval = 3
private let connection: PunktfunkConnection
private let pasteboard: any ClipboardPasteboard
/// `CLIP_FLAG_*` sent with the enable (`CLIP_FLAG_FILES` when the session permits files
/// always 0 in Phase 1).
private let controlFlags: UInt8
@@ -53,72 +47,91 @@ public final class ClipboardSync: NSObject {
/// Host `.state` updates, delivered on the main queue drives the toggle/footnote UI.
public var onState: ((_ enabled: Bool, _ policy: UInt8, _ reason: UInt8) -> Void)?
// Drain-thread state (touched only on the drain thread once started).
// MARK: Offer bookkeeping
//
// Read by the drain thread, and written by the thread tearing the sync down too, so it is all
// under one lock. Nothing here is held across a fetch or a pasteboard read.
private let stateLock = NSLock()
private var offerSeq: UInt32 = 0
private var lastSeenChangeCount = 0
/// The changeCount of the last pasteboard write WE made (echo suppression + "do we still
/// own the pasteboard" on teardown/clear).
/// The changeCount of the last pasteboard write WE made (echo suppression, and "do we still
/// own the pasteboard" on teardown).
private var ownedChangeCount = -1
/// The host offer currently installed on the local pasteboard (nil = none).
private var installedRemoteSeq: UInt32?
/// The host offer currently placed on the local pasteboard (nil = none).
private var installedRemote: (seq: UInt32, kinds: [PunktfunkConnection.ClipKind])?
/// The offer already pulled down to concrete bytes, so teardown neither re-fetches it nor
/// takes it back off the pasteboard.
private var resolvedSeq: UInt32?
/// Outbound fetches a blocked paste is waiting on. Guarded by `fetchLock` appended by the
/// drain thread (`.data` events), consumed by AppKit's provider threads.
private struct PendingFetch {
// MARK: Outbound fetches
//
// Appended by whichever thread starts a fetch, completed by the drain thread as `.data`
// arrives. Guarded by `fetchLock`, which is never held while a completion runs.
private final class PendingFetch {
var buffer = Data()
let done = DispatchSemaphore(value: 0)
var failed = false
let completion: (Data?) -> Void
init(completion: @escaping (Data?) -> Void) { self.completion = completion }
}
private let fetchLock = NSLock()
private var pendingFetches: [UInt32: PendingFetch] = [:]
/// Fires the deadline that keeps a blocked paste from waiting forever on a host that went
/// quiet mid-transfer.
private let deadlines = DispatchQueue(label: "io.unom.punktfunk.clipboard.deadline")
/// Serves host pastes off the drain thread where a pasteboard read can await a user decision
/// (iOS's paste permission alert) the drain thread must keep running, it is the one that
/// would deliver the host's cancel.
private let serves = DispatchQueue(label: "io.unom.punktfunk.clipboard.serve")
private final class StopFlag: @unchecked Sendable {
private final class Flag: @unchecked Sendable {
private let lock = NSLock()
private var stopped = false
func stop() {
lock.lock()
stopped = true
lock.unlock()
}
var isStopped: Bool {
lock.lock()
defer { lock.unlock() }
return stopped
}
}
private let flag = StopFlag()
private let drainDone = DispatchSemaphore(value: 0)
private var started = false
/// Set by the app-activation observer, cleared by the drain loop: the user may have copied
/// elsewhere and is coming back to paste announce immediately instead of waiting out the
/// poll interval.
private final class OneShot: @unchecked Sendable {
private let lock = NSLock()
private var raised = false
private var value = false
func raise() {
lock.lock()
raised = true
value = true
lock.unlock()
}
func takeIfRaised() -> Bool {
var isRaised: Bool {
lock.lock()
defer { lock.unlock() }
let was = raised
raised = false
return value
}
/// Read-and-clear, for the one-shot "check the pasteboard now" nudge.
func take() -> Bool {
lock.lock()
defer { lock.unlock() }
let was = value
value = false
return was
}
}
private let checkNow = OneShot()
private var activationObserver: NSObjectProtocol?
private let stopped = Flag()
/// Raised by the activation observer, taken by the drain loop: the user may have copied
/// elsewhere and is coming back to paste announce now rather than waiting out the poll.
private let checkNow = Flag()
private let drainDone = DispatchSemaphore(value: 0)
private var started = false
public init(connection: PunktfunkConnection, allowFiles: Bool = false) {
/// - Parameter allowFiles: reserved for Phase 2; `CLIP_FLAG_FILES` is never set yet.
public convenience init(connection: PunktfunkConnection, allowFiles: Bool = false) {
self.init(connection: connection, pasteboard: SystemPasteboard(), allowFiles: allowFiles)
}
/// Designated init, taking the pasteboard so tests can drive the whole state machine against a
/// stub without an AppKit/UIKit pasteboard in the way.
init(
connection: PunktfunkConnection, pasteboard: any ClipboardPasteboard,
allowFiles: Bool = false
) {
self.connection = connection
self.pasteboard = pasteboard
self.controlFlags = 0 // CLIP_FLAG_FILES rides Phase 2
_ = allowFiles
super.init()
}
deinit { flag.stop() }
deinit { stopped.raise() }
// MARK: - Lifecycle
/// Enable sync with the host and start the drain thread. The host answers the enable with a
/// `.state` event (surfaced via `onState`) `BACKEND_UNAVAILABLE` et al. arrive there.
@@ -126,103 +139,105 @@ public final class ClipboardSync: NSObject {
guard !started else { return }
started = true
connection.clipControl(enabled: true, flags: controlFlags)
// Baseline: whatever is on the pasteboard when sync starts is announced immediately
// the "copy first, then connect and paste" flow must work.
// Baseline: whatever is on the pasteboard when sync starts is announced immediately the
// "copy first, then connect and paste" flow must work.
stateLock.lock()
lastSeenChangeCount = -1
activationObserver = NotificationCenter.default.addObserver(
forName: NSApplication.didBecomeActiveNotification, object: nil, queue: nil
) { [checkNow] _ in checkNow.raise() }
let connection = self.connection
let flag = self.flag
let thread = Thread { [weak self] in
var lastAnnounceCheck = Date.distantPast
while !flag.isStopped {
// Drain events (bounded burst so a chatty host can't starve the announce poll).
var drained = 0
while drained < 32, !flag.isStopped {
let ev: PunktfunkConnection.ClipEvent?
do {
ev = try connection.nextClipboard(timeoutMs: drained == 0 ? 200 : 0)
} catch {
flag.stop() // session closed
break
}
guard let ev else { break }
drained += 1
self?.handle(ev)
}
// Announce poll: every 500 ms, or immediately after app activation (§5.2).
let now = Date()
if now.timeIntervalSince(lastAnnounceCheck) >= 0.5
|| self?.checkNow.takeIfRaised() == true
{
lastAnnounceCheck = now
self?.announceIfChanged()
}
}
self?.drainDone.signal()
}
stateLock.unlock()
pasteboard.startObserving(onActivate: { [checkNow] in checkNow.raise() })
let thread = Thread { [weak self] in self?.drain() }
thread.name = "punktfunk-clipboard"
thread.qualityOfService = .utility
thread.start()
}
/// Disable sync and join the drain thread. Called off-main before `connection.close()`
/// (the same discipline as the audio/feedback drains). If the local pasteboard still holds
/// our remote-offer items, they are cleared their providers die with us.
/// Disable sync and join the drain thread. Called off-main before `connection.close()` (the
/// same discipline as the audio/feedback drains).
///
/// A host offer still sitting on the local pasteboard as a promise has to be dealt with here,
/// because after this returns nothing can answer it: either it is pulled down to real bytes
/// (iOS, where this teardown is the user walking away with something they copied) or it is
/// cleared, so a later paste comes up empty rather than silently doing nothing.
public func stop() {
guard started else { return }
started = false
if let obs = activationObserver {
NotificationCenter.default.removeObserver(obs)
activationObserver = nil
pasteboard.stopObserving()
// Before anything is torn down the drain thread has to still be running to deliver the
// chunks, and the connection still open to carry the fetch.
if pasteboard.resolvesPendingOfferOnTeardown {
resolvePendingOffer()
}
connection.clipControl(enabled: false, flags: 0)
flag.stop()
stopped.raise()
drainDone.wait()
// Fail every paste still blocked on us so no provider thread waits out its timeout.
fetchLock.lock()
for (_, pending) in pendingFetches {
pending.done.signal()
// Fail every paste still blocked on us so nothing waits out its timeout against a dead
// session.
settleAll(nil)
stateLock.lock()
let ownsUnresolvedOffer =
installedRemote != nil && resolvedSeq != installedRemote?.seq
&& pasteboard.changeCount == ownedChangeCount
installedRemote = nil
stateLock.unlock()
if ownsUnresolvedOffer {
_ = pasteboard.clear()
}
pendingFetches.removeAll()
fetchLock.unlock()
let pb = NSPasteboard.general
if installedRemoteSeq != nil, pb.changeCount == ownedChangeCount {
pb.clearContents()
}
private func drain() {
var lastAnnounceCheck = Date.distantPast
while !stopped.isRaised {
// Drain events (bounded burst so a chatty host can't starve the announce poll).
var drained = 0
while drained < 32, !stopped.isRaised {
let ev: PunktfunkConnection.ClipEvent?
do {
ev = try connection.nextClipboard(timeoutMs: drained == 0 ? 200 : 0)
} catch {
stopped.raise() // session closed
break
}
guard let ev else { break }
drained += 1
handle(ev)
}
let now = Date()
if now.timeIntervalSince(lastAnnounceCheck) >= Self.announceInterval
|| checkNow.take()
{
lastAnnounceCheck = now
announceIfChanged()
}
}
drainDone.signal()
}
// MARK: - Local copy host (announce)
/// Announce the local pasteboard's format list when it changed (skipping our own writes and
/// concealed/transient pasteboards). Runs on the drain thread.
/// Announce the local pasteboard's format list when it changed, skipping our own writes and
/// concealed/transient pasteboards. Runs on the drain thread.
private func announceIfChanged() {
let pb = NSPasteboard.general
let count = pb.changeCount
guard count != lastSeenChangeCount else { return }
lastSeenChangeCount = count
if count == ownedChangeCount { return } // our own write (a remote offer) never echo
installedRemoteSeq = nil // a local copy replaced the host's offer
let types = pb.types ?? []
if types.contains(Self.concealed) || types.contains(Self.transient) { return }
offerSeq &+= 1
var kinds = Self.wireToPasteboard
.filter { types.contains($0.type) }
.map { PunktfunkConnection.ClipKind(mime: $0.wire) }
// PNG floor: announce the portable `image/png` whenever ANY convertible image is present
// native PNG, TIFF/HEIC (screenshots, Preview), or a JPEG/GIF original already being
// offered verbatim above. `readWireData` converts at fetch time (lazy, §3.5), so the
// fallback costs nothing unless a peer actually pastes it.
if !kinds.contains(where: { $0.mime == "image/png" }),
types.contains(.tiff)
|| types.contains(NSPasteboard.PasteboardType("public.heic"))
|| kinds.contains(where: { $0.mime.hasPrefix("image/") })
{
kinds.append(PunktfunkConnection.ClipKind(mime: "image/png"))
let count = pasteboard.changeCount
stateLock.lock()
guard count != lastSeenChangeCount else {
stateLock.unlock()
return
}
lastSeenChangeCount = count
guard count != ownedChangeCount else {
stateLock.unlock() // our own write (a remote offer) never echo
return
}
installedRemote = nil // a local copy replaced the host's offer
stateLock.unlock()
guard let kinds = pasteboard.offerKinds else { return } // concealed never announced
stateLock.lock()
offerSeq &+= 1
let seq = offerSeq
stateLock.unlock()
// Empty = the pasteboard holds nothing we sync (or was cleared) clears the host side.
connection.clipOffer(seq: offerSeq, kinds: kinds)
connection.clipOffer(seq: seq, kinds: kinds)
}
// MARK: - Event handling (drain thread)
@@ -236,93 +251,166 @@ public final class ClipboardSync: NSObject {
case let .remoteOffer(seq, kinds):
installRemoteOffer(seq: seq, kinds: kinds)
case let .fetchRequest(reqId, seq, _, mime):
serveFetch(reqId: reqId, seq: seq, mime: mime)
serves.async { [weak self] in self?.serveFetch(reqId: reqId, seq: seq, mime: mime) }
case let .data(xferId, chunk, last):
fetchLock.lock()
if var pending = pendingFetches[xferId] {
pending.buffer.append(chunk)
pendingFetches[xferId] = pending
if last {
pendingFetches[xferId]?.done.signal()
}
}
let pending = pendingFetches[xferId]
pending?.buffer.append(chunk)
let finished = last ? pendingFetches.removeValue(forKey: xferId) : nil
fetchLock.unlock()
// Outside the lock: a completion may start the next fetch (or wake a thread that will).
if let finished {
finished.completion(finished.buffer)
}
case let .cancelled(id), let .error(id, _):
fetchLock.lock()
if var pending = pendingFetches[id] {
pending.failed = true
pendingFetches[id] = pending
pending.done.signal()
}
fetchLock.unlock()
settle(id, nil)
}
}
// MARK: - Host copy local (lazy install + blocked-paste fetch)
// MARK: - Host copy local (lazy placement + paste-time fetch)
/// Write one NSPasteboardItem advertising the host's formats, each backed by a lazy data
/// provider bytes cross only when a Mac app pastes. Empty `kinds` = the host cleared its
/// clipboard: drop our item if it's still current.
/// Place a pasteboard item advertising the host's formats, each backed by a lazy provider
/// bytes cross only when a local app pastes. Empty `kinds` = the host cleared its clipboard:
/// drop our item if it's still current.
private func installRemoteOffer(seq: UInt32, kinds: [PunktfunkConnection.ClipKind]) {
let pb = NSPasteboard.general
let types = kinds.compactMap { kind in
Self.wireToPasteboard.first(where: { $0.wire == kind.mime })?.type
}
guard !types.isEmpty else {
if installedRemoteSeq != nil, pb.changeCount == ownedChangeCount {
pb.clearContents()
ownedChangeCount = pb.changeCount
lastSeenChangeCount = pb.changeCount
let utis = ClipboardFormats.placeableUtis(for: kinds)
guard !utis.isEmpty else {
stateLock.lock()
let owned = installedRemote != nil && pasteboard.changeCount == ownedChangeCount
installedRemote = nil
resolvedSeq = nil
if owned {
let after = pasteboard.clear()
ownedChangeCount = after
lastSeenChangeCount = after
}
installedRemoteSeq = nil
stateLock.unlock()
return
}
let item = NSPasteboardItem()
item.setDataProvider(RemoteOfferProvider(sync: self, seq: seq), forTypes: types)
pb.clearContents()
pb.writeObjects([item])
installedRemoteSeq = seq
ownedChangeCount = pb.changeCount
lastSeenChangeCount = pb.changeCount
let fetch: ClipboardFetch = { [weak self] wire, done in
guard let self else {
done(nil)
return
}
self.fetch(seq: seq, wire: wire, completion: done)
}
let before = pasteboard.changeCount
let after = pasteboard.installLazy(utis: utis, fetch: fetch)
stateLock.lock()
installedRemote = (seq, kinds)
resolvedSeq = nil
// Only claim the pasteboard if the write actually landed. Recording a change count we did
// not cause is the one bookkeeping mistake with no recovery: the announce poll would read
// the user's own next copy as our echo and never tell the host about it again.
if after != before {
ownedChangeCount = after
lastSeenChangeCount = after
}
stateLock.unlock()
}
/// Blocked-paste fulfillment: fetch one wire format of host offer `seq` and wait (provider
/// thread) for the drain thread to assemble the chunks. Nil on timeout/cancel/error the
/// paste then provides nothing rather than hanging (§3.4).
///
/// `fetchLock` is held ACROSS the `clipFetch` so the pending entry exists before the drain
/// thread can process the first `.data` event (its `handle` takes `fetchLock` after
/// releasing the connection's clipboard lock no cycle).
fileprivate func fetchBlocking(seq: UInt32, wireMime: String) -> Data? {
/// Start pulling one wire format of host offer `seq`. `completion` runs exactly once with
/// the bytes, or with nil on a stale offer, a timeout, a cancel, or a closing session.
private func fetch(seq: UInt32, wire: String, completion: @escaping (Data?) -> Void) {
fetchLock.lock()
guard let xferId = connection.clipFetch(seq: seq, mime: wireMime) else {
guard !stopped.isRaised, let xferId = connection.clipFetch(seq: seq, mime: wire) else {
fetchLock.unlock()
return nil
completion(nil)
return
}
pendingFetches[xferId] = PendingFetch()
let done = pendingFetches[xferId]!.done
pendingFetches[xferId] = PendingFetch(completion: completion)
fetchLock.unlock()
let outcome = done.wait(timeout: .now() + Self.fetchTimeout)
deadlines.asyncAfter(deadline: .now() + Self.fetchTimeout) { [weak self] in
guard let self, self.settle(xferId, nil) else { return }
self.connection.clipCancel(id: xferId)
}
}
/// Complete one pending fetch if it hasn't been already. Returns whether this call is the one
/// that settled it, so a deadline knows whether it still has to cancel the transfer.
@discardableResult
private func settle(_ xferId: UInt32, _ data: Data?) -> Bool {
fetchLock.lock()
let pending = pendingFetches.removeValue(forKey: xferId)
fetchLock.unlock()
if outcome == .timedOut {
connection.clipCancel(id: xferId)
return nil
pending?.completion(data)
return pending != nil
}
private func settleAll(_ data: Data?) {
fetchLock.lock()
let all = pendingFetches
pendingFetches.removeAll()
fetchLock.unlock()
for (_, pending) in all {
pending.completion(data)
}
guard let pending, !pending.failed else { return nil }
return pending.buffer
}
// MARK: - Resolving a promise before it dies (iOS)
/// Pull the host's offer down to concrete bytes, replacing the promise on the pasteboard.
///
/// A lazy promise is only as good as the ability to answer it, and on iOS that ability ends
/// with the session: backgrounding the app disconnects it. Without this, "copy on the host,
/// then paste into Safari on the iPad" would hand Safari an empty promise. Everything else
/// about the design stays lazy a user who copies on the host, pastes nothing, and stays in
/// the app moves no clipboard bytes at all; this runs once, at the end, for content that is
/// still on the pasteboard and still unclaimed.
///
/// Runs on the thread calling `stop()` off-main by contract, and never the drain thread,
/// which has to keep running to deliver what this waits for.
private func resolvePendingOffer() {
stateLock.lock()
let offer = installedRemote
let unresolved = resolvedSeq != installedRemote?.seq
let stillOurs = pasteboard.changeCount == ownedChangeCount
stateLock.unlock()
guard let offer, unresolved, stillOurs, !stopped.isRaised else { return }
let deadline = Date().addingTimeInterval(Self.resolveTimeout)
var items: [(uti: String, data: Data)] = []
var budget = Self.resolveBudget
for kind in offer.kinds {
guard let uti = ClipboardFormats.uti(forWire: kind.mime) else { continue }
// A size hint of 0 means "unknown" try it, and let the byte count enforce the cap.
guard kind.sizeHint <= UInt64(budget), !stopped.isRaised else { continue }
let left = deadline.timeIntervalSinceNow
guard left > 0 else { break }
let box = ClipboardResultBox()
fetch(seq: offer.seq, wire: kind.mime) { box.settle($0) }
guard let data = box.wait(timeout: left), data.count <= budget else { continue }
budget -= data.count
items.append((uti, data))
}
guard !items.isEmpty else { return }
// Re-check: the host may have copied again, or the user may have copied locally, while we
// were pulling either way these bytes are no longer what belongs on the pasteboard.
stateLock.lock()
defer { stateLock.unlock() }
let before = pasteboard.changeCount
guard installedRemote?.seq == offer.seq, before == ownedChangeCount else { return }
let after = pasteboard.installResolved(items)
// As in `installRemoteOffer`: a write that did not land leaves the promise in place, so
// teardown should still take it back rather than believing these bytes are on the board.
guard after != before else { return }
resolvedSeq = offer.seq
ownedChangeCount = after
lastSeenChangeCount = after
}
// MARK: - Host paste of our data (serve)
/// Answer a host paste of our offered data from the live pasteboard. A stale `seq` (the
/// local clipboard changed since that announce) is cancelled never serve mismatched bytes.
/// Answer a host paste of our offered data from the live pasteboard. A stale `seq` (the local
/// clipboard changed since that announce) is cancelled never serve mismatched bytes.
///
/// Runs on `serves`, not the drain thread: reading the pasteboard can block on a user decision
/// (iOS's paste permission alert), and the drain thread has to stay live throughout.
private func serveFetch(reqId: UInt32, seq: UInt32, mime: String) {
let pb = NSPasteboard.general
guard seq == offerSeq, pb.changeCount == lastSeenChangeCount,
let data = Self.readWireData(pb, mime)
else {
stateLock.lock()
let fresh = seq == offerSeq && pasteboard.changeCount == lastSeenChangeCount
stateLock.unlock()
guard fresh, !stopped.isRaised, let data = pasteboard.read(wire: mime) else {
connection.clipCancel(id: reqId)
return
}
@@ -337,66 +425,5 @@ public final class ClipboardSync: NSObject {
connection.clipServe(reqId: reqId, data: Data(), last: true)
}
}
/// Read one wire format from the pasteboard, converting where macOS stores a different
/// native type: `image/png` is served from a real `.png` entry when present, else converted
/// from whatever image representation the pasteboard holds (TIFF from screenshots/Preview,
/// WebP/AVIF/GIF from browsers `NSImage` decodes them all) into PNG at fetch time.
private static func readWireData(_ pb: NSPasteboard, _ mime: String) -> Data? {
guard mime == "image/png" else {
guard let type = wireToPasteboard.first(where: { $0.wire == mime })?.type else {
return nil
}
return pb.data(forType: type)
}
if let png = pb.data(forType: .png) {
return png
}
// No native PNG: decode whatever image the pasteboard carries and re-encode.
guard let img = NSImage(pasteboard: pb),
let tiff = img.tiffRepresentation,
let rep = NSBitmapImageRep(data: tiff)
else {
return nil
}
return rep.representation(using: .png, properties: [:])
}
}
/// The lazy paste hook: AppKit calls `provideDataForType` only when a Mac app actually pastes;
/// the fetch then blocks this provider thread (never main) until the host's bytes arrive or the
/// timeout provides nothing. One provider per installed remote offer a dead sync (weak) or a
/// superseded offer provides nothing.
private final class RemoteOfferProvider: NSObject, NSPasteboardItemDataProvider {
private weak var sync: ClipboardSync?
private let seq: UInt32
init(sync: ClipboardSync, seq: UInt32) {
self.sync = sync
self.seq = seq
}
func pasteboard(
_ pasteboard: NSPasteboard?, item: NSPasteboardItem,
provideDataForType type: NSPasteboard.PasteboardType
) {
guard let sync,
let wire = wireMime(for: type),
let data = sync.fetchBlocking(seq: seq, wireMime: wire)
else { return }
item.setData(data, forType: type)
}
private func wireMime(for type: NSPasteboard.PasteboardType) -> String? {
switch type {
case .string: return "text/plain;charset=utf-8"
case .rtf: return "text/rtf"
case .html: return "text/html"
case .png: return "image/png"
case NSPasteboard.PasteboardType("public.jpeg"): return "image/jpeg"
case NSPasteboard.PasteboardType("com.compuserve.gif"): return "image/gif"
default: return nil
}
}
}
#endif
@@ -34,10 +34,10 @@ public struct StoredHost: Identifiable, Codable, Hashable, Sendable {
/// client can send a magic packet to wake the host later (when it's asleep and no longer
/// advertising). Optional (same forward-compat reason as `mgmtPort`); nil until first learned.
public var macAddresses: [String]?
/// Share the clipboard with this host (macOS sessions; design/clipboard-and-file-transfer.md
/// §5.3). Opt-in per host: nil/false = off (nil also keeps older saved JSON decoding same
/// forward-compat reason as `mgmtPort`). Honored only when the host advertises
/// `HOST_CAP_CLIPBOARD`.
/// Share the clipboard with this host (macOS and iOS sessions; tvOS has no pasteboard see
/// design/clipboard-and-file-transfer.md §5.3). Opt-in per host: nil/false = off (nil also
/// keeps older saved JSON decoding same forward-compat reason as `mgmtPort`). Honored only
/// when the host advertises `HOST_CAP_CLIPBOARD`.
public var clipboardSync: Bool?
/// This host's default settings profile (`StreamProfile.id`) what a plain click/tap uses.
/// nil, or an id whose profile was deleted, resolves as "Default settings", i.e. exactly
@@ -0,0 +1,154 @@
// The shared clipboard's format decisions what a given pasteboard gets announced as, and what a
// host offer gets placed as. Pure functions, and the half of the sync that macOS and iOS now
// genuinely share, so a change that suits one platform and breaks the other fails here.
import XCTest
@testable import PunktfunkKit
#if !os(tvOS)
final class ClipboardFormatsTests: XCTestCase {
// MARK: - Announcing what is on the local pasteboard
func testPlainTextAnnouncesOnlyText() {
let kinds = ClipboardFormats.offerKinds(forTypes: ["public.utf8-plain-text"])
XCTAssertEqual(kinds.map(\.mime), ["text/plain;charset=utf-8"])
}
func testRichTextAnnouncesEveryRepresentationInTableOrder() {
// A copy out of a word processor leaves all three; the host picks the richest it can place,
// so the order the table declares is the order the offer carries.
let kinds = ClipboardFormats.offerKinds(forTypes: [
"public.html", "public.utf8-plain-text", "public.rtf",
])
XCTAssertEqual(kinds.map(\.mime), ["text/plain;charset=utf-8", "text/rtf", "text/html"])
}
func testUnknownTypesAreNotAnnounced() {
// A pasteboard holding only things we have no wire vocabulary for announces nothing, which
// legitimately clears the host's side rather than offering something unfetchable.
let kinds = ClipboardFormats.offerKinds(forTypes: ["com.apple.mail.PasteboardTypeMessage"])
XCTAssertTrue(kinds.isEmpty)
}
// MARK: - The PNG floor
func testScreenshotTiffAnnouncesPngEvenThoughTiffIsNotOnTheWire() {
// Screenshots and Preview leave TIFF, which no host can place. The adapters transcode at
// fetch time, so announcing the PNG floor costs nothing until someone actually pastes.
let kinds = ClipboardFormats.offerKinds(forTypes: ["public.tiff"])
XCTAssertEqual(kinds.map(\.mime), ["image/png"])
}
func testPhotoHeicAnnouncesPng() {
// The iOS case: a photo copied out of Photos is HEIC.
let kinds = ClipboardFormats.offerKinds(forTypes: ["public.heic"])
XCTAssertEqual(kinds.map(\.mime), ["image/png"])
}
func testJpegCrossesVerbatimAndStillCarriesThePngFloor() {
// The original rides beside the floor rather than replacing it a copied JPEG must not
// balloon into a lossless PNG for peers that can take JPEG.
let kinds = ClipboardFormats.offerKinds(forTypes: ["public.jpeg"])
XCTAssertEqual(kinds.map(\.mime), ["image/jpeg", "image/png"])
}
func testGifCrossesVerbatimSoAnimationSurvives() {
let kinds = ClipboardFormats.offerKinds(forTypes: ["com.compuserve.gif"])
XCTAssertEqual(kinds.map(\.mime), ["image/gif", "image/png"])
}
func testNativePngIsNotAnnouncedTwice() {
let kinds = ClipboardFormats.offerKinds(forTypes: ["public.png", "public.tiff"])
XCTAssertEqual(kinds.map(\.mime), ["image/png"])
}
// MARK: - Secrets
func testConcealedAndTransientPasteboardsAreRecognized() {
// Password managers mark secrets with these; nothing so marked is ever announced.
XCTAssertTrue(
ClipboardFormats.isConcealed([
"public.utf8-plain-text", "org.nspasteboard.ConcealedType",
]))
XCTAssertTrue(
ClipboardFormats.isConcealed([
"public.utf8-plain-text", "org.nspasteboard.TransientType",
]))
XCTAssertFalse(ClipboardFormats.isConcealed(["public.utf8-plain-text"]))
}
func testAConcealedPasteboardOffersNothingRatherThanAnEmptyOffer() {
// The distinction matters: nil means "say nothing at all", where an empty list would tell
// the host to drop what it has.
let secret = StubPasteboard(types: ["public.utf8-plain-text", "org.nspasteboard.ConcealedType"])
XCTAssertNil(secret.offerKinds)
let ordinary = StubPasteboard(types: ["public.utf8-plain-text"])
XCTAssertEqual(ordinary.offerKinds?.map(\.mime), ["text/plain;charset=utf-8"])
let empty = StubPasteboard(types: [])
XCTAssertEqual(empty.offerKinds?.isEmpty, true)
}
// MARK: - Placing what the host offered
func testHostOfferMapsToUniformTypesAndSkipsWhatWeCannotPlace() {
// Files ride Phase 2 an offer carrying them places the rest and ignores that kind rather
// than failing the whole paste.
let kinds = [
PunktfunkConnection.ClipKind(mime: "text/plain;charset=utf-8"),
PunktfunkConnection.ClipKind(mime: "application/x-punktfunk-files"),
PunktfunkConnection.ClipKind(mime: "image/png"),
]
XCTAssertEqual(
ClipboardFormats.placeableUtis(for: kinds), ["public.utf8-plain-text", "public.png"])
}
func testWireAndUniformTypeMapBothWays() {
for (wire, uti) in ClipboardFormats.table {
XCTAssertEqual(ClipboardFormats.uti(forWire: wire), uti)
XCTAssertEqual(ClipboardFormats.wire(forUti: uti), wire)
}
XCTAssertNil(ClipboardFormats.uti(forWire: "application/x-punktfunk-files"))
XCTAssertNil(ClipboardFormats.wire(forUti: "public.tiff"))
}
// MARK: - Handing bytes between threads
func testResultBoxDeliversBytesToAWaiter() {
let box = ClipboardResultBox()
DispatchQueue.global().async { box.settle(Data("hello".utf8)) }
XCTAssertEqual(box.wait(timeout: 5), Data("hello".utf8))
}
func testResultBoxTimesOutRatherThanWaitingForever() {
// What a paste does when the host goes quiet mid-transfer: give up, insert nothing.
XCTAssertNil(ClipboardResultBox().wait(timeout: 0.05))
}
func testResultBoxKeepsTheFirstAnswer() {
// The fetch deadline and a late arrival can both fire; whichever settles first wins, and
// the loser must not overwrite it or signal a second time.
let box = ClipboardResultBox()
box.settle(nil)
box.settle(Data("late".utf8))
XCTAssertNil(box.wait(timeout: 1))
}
}
/// A pasteboard that holds nothing but a type list enough to exercise the announce decision
/// without an AppKit or UIKit pasteboard in the test process.
private final class StubPasteboard: ClipboardPasteboard {
private let types: [String]
init(types: [String]) { self.types = types }
var changeCount = 0
var typeIdentifiers: [String] { types }
func read(wire: String) -> Data? { nil }
func installLazy(utis: [String], fetch: @escaping ClipboardFetch) -> Int { 0 }
func installResolved(_ items: [(uti: String, data: Data)]) -> Int { 0 }
func clear() -> Int { 0 }
let resolvesPendingOfferOnTeardown = false
func startObserving(onActivate: @escaping () -> Void) {}
func stopObserving() {}
}
#endif
+69
View File
@@ -82,6 +82,51 @@ pub fn boost_thread_priority(critical: bool) {
}
}
/// What the OS is actually giving the CALLING thread: `(policy, rt_priority, nice)`.
///
/// Exists because a boost we *asked for* and a boost the hot thread *has* turned out to be
/// different questions. Callbacks handed to a library — PipeWire's `RT_PROCESS` streams above all
/// — run on a thread that library created and schedules, so a `boost_thread_priority` call in our
/// own setup path can log a cheerful success about a thread that never touches audio. A
/// 2026-08-15 measurement found exactly that shape: our loop thread at SCHED_OTHER/0 while the
/// data loop actually running the capture callback sat at SCHED_RR/20, both in the same process.
///
/// Report this from inside the hot callback, where "the calling thread" is the one that matters.
#[cfg(target_os = "linux")]
pub fn current_thread_sched() -> (&'static str, i32, i32) {
// SAFETY: all three calls take by-value integers (plus, for `sched_getparam`, a pointer to a
// fully-initialised local we own and outlive) and return integers. `0` means "the calling
// task" on Linux, so nothing outside this thread is read or written, and no allocation,
// locking or blocking happens — which is what makes this callable from an RT callback.
unsafe {
let policy = libc::sched_getscheduler(0);
let mut param: libc::sched_param = std::mem::zeroed();
let rt_priority = if libc::sched_getparam(0, &mut param) == 0 {
param.sched_priority
} else {
-1
};
// `getpriority` legitimately returns -1, so errno is the only way to tell a nice of -1
// from a failure.
*libc::__errno_location() = 0;
let nice = libc::getpriority(libc::PRIO_PROCESS, 0);
let nice = if *libc::__errno_location() == 0 {
nice
} else {
0
};
let policy = match policy {
libc::SCHED_FIFO => "SCHED_FIFO",
libc::SCHED_RR => "SCHED_RR",
libc::SCHED_OTHER => "SCHED_OTHER",
libc::SCHED_BATCH => "SCHED_BATCH",
libc::SCHED_IDLE => "SCHED_IDLE",
_ => "unknown",
};
(policy, rt_priority, nice)
}
}
/// RealtimeKit fallback for [`boost_thread_priority`]: ask the system-bus broker
/// (`org.freedesktop.RealtimeKit1`) to renice the calling thread when the direct
/// `setpriority` was refused. This is how PulseAudio/PipeWire clients get their boosts on a
@@ -121,3 +166,27 @@ mod linux_rtkit {
Ok(())
}
}
#[cfg(all(test, target_os = "linux"))]
mod tests {
/// Non-vacuity: the introspection has to come back with something the OS could actually have
/// said. A helper whose whole job is to be quoted in a field log is worthless if it can
/// quietly report a placeholder, and it only ever runs on hosts nobody can attach a debugger
/// to.
#[test]
fn current_thread_sched_reports_a_real_policy() {
let (policy, rt_priority, nice) = super::current_thread_sched();
assert!(
matches!(
policy,
"SCHED_OTHER" | "SCHED_RR" | "SCHED_FIFO" | "SCHED_BATCH" | "SCHED_IDLE"
),
"unrecognised policy {policy}"
);
assert!(
(0..=99).contains(&rt_priority),
"rt priority {rt_priority} outside the kernel's range"
);
assert!((-20..=19).contains(&nice), "nice {nice} outside PRIO range");
}
}
@@ -194,7 +194,7 @@ struct ReassemblyWindow {
/// partially-arrived frames of ACTUAL size (≪ max); without this cap, [`HARD_LOSS_WINDOW`]
/// max-sized declarations from one header-sized packet each could commit gigabytes — an
/// amplification the old sparse per-shard allocation didn't have.
const IN_FLIGHT_BUF_FACTOR: usize = 4;
pub(super) const IN_FLIGHT_BUF_FACTOR: usize = 4;
/// Recovery-shard buffer pool ceiling (shard-sized buffers): enough for several max-recovery
/// blocks in flight, small enough (~720 KB at a 1408-byte shard) to keep after a loss burst.
@@ -208,7 +208,12 @@ const RECOVERY_POOL_MAX: usize = 512;
/// can mint thousands of distinct-index blocks while its `FrameBuf::buf` stays pinned near zero —
/// they must be metered exactly like the buffer, or the firewall meters only half the allocation
/// (security-review 2026-08-15 finding 11).
fn block_state_bytes(data_shards: usize, recovery_shards: usize) -> usize {
///
/// `pub(super)` so the budget tests can locate the refusal boundary from the cost model itself
/// rather than from a baked-in frame count — [`BlockState`] gaining a field moves that boundary,
/// and a test that hard-codes it answers such a change with an arithmetic puzzle instead of the
/// question actually worth asking.
pub(super) fn block_state_bytes(data_shards: usize, recovery_shards: usize) -> usize {
std::mem::size_of::<BlockState>()
+ data_shards // have_data: Vec<bool>
+ recovery_shards * std::mem::size_of::<Option<Vec<u8>>>() // recovery slot table
+32 -9
View File
@@ -520,13 +520,19 @@ fn e2e_unrecoverable_loss_ages_out() {
/// gigabytes (the eager whole-frame buffer's amplification defense).
#[test]
fn in_flight_buffer_budget_bounds_allocation() {
let lim = limits(); // max_frame_bytes 4096, shards 16 B, ≤8 data shards × ≤4 blocks
// limits(): max_frame_bytes 4096, shards 16 B, ≤8 data shards × ≤4 blocks → budget 16384 B.
let lim = limits();
let budget = IN_FLIGHT_BUF_FACTOR * lim.max_frame_bytes;
// What ONE such frame commits: the largest geometry-consistent buffer (4 blocks × 8 shards
// × 16 B = 512 B) plus the state of the single block this first shard opens. Both are sized
// from header fields, so the firewall meters both — counting only the buffer is precisely
// the hole security-review 2026-08-15 #11 closed, and the boundary moved when it did.
let per_frame = 512 + block_state_bytes(8, 0);
let fits = budget / per_frame;
let mut r = Reassembler::new(lim);
let coder = coder_for(FecScheme::Gf8);
let stats = StatsCounters::default();
// Largest geometry-consistent frame: 4 blocks × 8 shards × 16 B = 512 B per buffer.
// Budget = 4 × 4096 = 16384 B → exactly 32 such frames fit; the 33rd must be refused.
for i in 0..33u32 {
for i in 0..=fits as u32 {
let mut h = base_header();
h.frame_index = i;
h.frame_bytes = 512;
@@ -539,6 +545,14 @@ fn in_flight_buffer_budget_bounds_allocation() {
1,
"the frame past the budget is dropped, everything under it accepted"
);
// The point of the whole exercise: whatever the geometry, the commitment stays under the
// ceiling. Asserted on the live figure, so a release site that forgets half the cost (the
// 0.23.0 accounting-drift lesson on `in_flight`) fails here and not in the field.
assert!(
r.in_flight() <= budget,
"in-flight commitment {} must never exceed the {budget} B budget",
r.in_flight(),
);
}
/// A header whose (data_shards, block_count) disagree with the geometry derived from its own
@@ -1519,11 +1533,15 @@ fn streamed_open_commits_its_own_extent_and_stays_bounded() {
);
// A SLICE sentinel whose wire base sits just under the ceiling really does commit a
// max-sized frame (base 3968 B + K 8 = 256 shards = 4096 B) — four fit the budget, the
// fifth must be refused.
let mut r = Reassembler::new(limits());
// max-sized frame (base 3968 B + K 8 = 256 shards = 4096 B), plus the state of the block it
// opens — so the budget takes fewer of these than the buffer alone would suggest, and the
// first one past it must be refused.
let lim = limits();
let budget = IN_FLIGHT_BUF_FACTOR * lim.max_frame_bytes;
let fits = budget / (4096 + block_state_bytes(8, 0));
let mut r = Reassembler::new(lim);
let stats = StatsCounters::default();
for fi in 0..5u32 {
for fi in 0..=fits as u32 {
let mut h = base_header();
h.user_flags = USER_FLAG_SLICE_STREAM;
h.block_count = 0;
@@ -1537,10 +1555,15 @@ fn streamed_open_commits_its_own_extent_and_stays_bounded() {
.unwrap()
.is_none());
}
assert!(
r.in_flight() <= budget,
"in-flight commitment {} must never exceed the {budget} B budget",
r.in_flight(),
);
assert_eq!(
stats.snapshot().packets_dropped,
1,
"the fifth ceiling-claiming open must be refused by the in-flight budget"
"the first ceiling-claiming open past the budget must be refused"
);
}
@@ -138,6 +138,22 @@ pub(crate) struct CaptureStats {
/// memory. Every one of these used to `return` silently, so a stream that fired its callback
/// on time and handed us nothing looked identical to a stream nobody was feeding.
pub(crate) missed_dequeues: u64,
/// Spans this window spent with the stream NOT in `Streaming`, and how long they totalled.
///
/// `gaps` deliberately cannot see these (see [`Self::observe_callback`]) — a paused stream
/// fires no callbacks at all, so there is no delta to score and the caller drops its cadence
/// stamp on every transition. The cost of that correct decision was that the outage went
/// somewhere else entirely: into `delivered_pct`, as an unattributed shortfall, because the
/// reporting window is flushed from the callback and therefore STRETCHES by exactly the time
/// we were not being scheduled.
///
/// Measured on a live host on 2026-08-15: a 16.2 s pause produced
/// `delivered_pct=63 gaps=0 max_gap_ms=0`. Every number was correct and the line still could
/// not say what happened — the explanation existed only in the state DEBUG lines, which a
/// field journal at INFO does not carry. These two fields are that explanation, at INFO,
/// beside the percentage they explain.
pub(crate) pauses: u64,
pub(crate) paused_us: u64,
}
impl CaptureStats {
@@ -157,9 +173,13 @@ impl CaptureStats {
///
/// `since_last` is `None` for the first callback of a stream — and, deliberately, for the
/// first after a state transition: the caller drops its stamp when the stream pauses, so a
/// legitimately Paused span is not scored as one enormous hole. (The Paused↔Streaming flaps
/// around a format renegotiation stay visible as the state DEBUG lines next to a small
/// post-resume gap, which is the honest reading of what happened.)
/// legitimately Paused span is not scored as one enormous hole.
///
/// That leaves this counter about ONE thing — holes inside a stream that is running — and
/// pushes the other kind onto [`Self::observe_pause`]. The split matters because the two want
/// opposite answers: a run of sub-10 ms holes is a scheduling problem on the box, whereas a
/// multi-second pause is our node not being in the graph at all. A single "gap" number that
/// mixed them would be worse than either.
///
/// `quantum` is the NEGOTIATED buffer duration, not the one we asked for: a graph handing us
/// 21.3 ms buffers is not gapping when its callbacks are 21.3 ms apart — it is doing exactly
@@ -183,6 +203,21 @@ impl CaptureStats {
self.max_gap_us / 1_000
}
/// Record one span the stream spent away from `Streaming`.
///
/// Called on the transition BACK, so the whole span lands in the window that is flushed after
/// the resume — which is the same window whose `delivered_pct` the span diluted. Keeping the
/// two together is the entire point: apart, neither is interpretable.
pub(crate) fn observe_pause(&mut self, span: Duration) {
self.pauses += 1;
self.paused_us += span.as_micros() as u64;
}
/// Total time away from `Streaming` this window, in whole ms.
pub(crate) fn paused_ms(&self) -> u64 {
self.paused_us / 1_000
}
/// `(peak dBFS, rms dBFS, delivered %)` for this window. Silence reports -120 dB rather than
/// -inf so the log line stays parseable.
pub(crate) fn summary(&self, elapsed: Duration, sample_rate: u32) -> (f64, f64, f64) {
@@ -199,6 +234,76 @@ impl CaptureStats {
}
}
/// A departure this far past its slot is a slip worth counting rather than ordinary jitter: one
/// whole protocol frame, so a frame that merely rounds late never scores.
const LATE_DEPARTURE: Duration = Duration::from_millis(FRAME_MS as u64);
/// One reporting window of AUDIO EGRESS vitals (WP-C).
///
/// Capture has been instrumented since WP-A2 and the send path has not, so a field log could show
/// audio arriving at the tap and say nothing whatsoever about how it left. That asymmetry is not
/// neutral: it made "the host paces audio badly" unfalsifiable, and an unfalsifiable suspect stays
/// on the list forever. Across five 2026-08-15 field logs the entire egress path emitted 14 lines,
/// all of them the same session-open banner.
///
/// The point of these counters is to be *boring*. If departures are clean while capture reports
/// holes, the pacing rework introduced in v0.25 is acquitted permanently and the search moves
/// upstream for good.
#[derive(Default)]
pub(crate) struct SendStats {
pub(crate) sent: u64,
/// Frames synthesized to cover a capture hole. Wire continuity and captured continuity are
/// different claims and a log that conflates them cannot be used to judge either.
pub(crate) infilled: u64,
/// Departures that missed their paced slot by at least [`LATE_DEPARTURE`].
pub(crate) late: u64,
/// The worst such miss, µs — kept even when the count is zero, because "never late" and
/// "never late by a whole frame" are different statements.
pub(crate) max_late_us: u64,
/// Widest gap between two consecutive departures, µs. The number a client-side starvation
/// complaint is actually about: the wire going quiet, whatever the reason.
pub(crate) max_spacing_us: u64,
/// Times the schedule fell more than `PACE_REANCHOR` behind and was re-anchored instead of
/// chased. Each one silently forgives accumulated debt, which is exactly the kind of event
/// that leaves no trace and then gets blamed on the network.
pub(crate) reanchors: u64,
}
impl SendStats {
/// Score one frame leaving the host. `late` is how far past its paced slot it went (zero when
/// the schedule is unanchored), `since_prev` the spacing from the previous departure.
pub(crate) fn observe_departure(
&mut self,
late: Duration,
since_prev: Option<Duration>,
infilled: bool,
) {
self.sent += 1;
if infilled {
self.infilled += 1;
}
self.max_late_us = self.max_late_us.max(late.as_micros() as u64);
if late >= LATE_DEPARTURE {
self.late += 1;
}
if let Some(gap) = since_prev {
self.max_spacing_us = self.max_spacing_us.max(gap.as_micros() as u64);
}
}
pub(crate) fn observe_reanchor(&mut self) {
self.reanchors += 1;
}
pub(crate) fn max_late_ms(&self) -> u64 {
self.max_late_us / 1_000
}
pub(crate) fn max_spacing_ms(&self) -> u64 {
self.max_spacing_us / 1_000
}
}
/// How long a capture hole may run before the wire starts covering it. Two protocol frames: long
/// enough that ordinary quantum jitter never trips it, short enough that the client's ring never
/// notices the hole.
@@ -525,4 +630,127 @@ mod tests {
assert_eq!(s.gaps, 0);
assert_eq!(s.max_gap_ms(), 0);
}
/// The companion to the test above, and the reason it is safe: a pause stays out of `gaps`,
/// but it does NOT stay out of the log line. Numbers are the ones measured on a live host on
/// 2026-08-15, where a 16.2 s pause reported `delivered_pct=63 gaps=0 max_gap_ms=0` and no
/// field in the line could say why.
#[test]
fn a_paused_span_is_reported_even_though_it_is_not_a_gap() {
let mut s = CaptureStats::default();
s.observe_callback(Some(Duration::from_millis(5)), Q);
s.observe_pause(Duration::from_millis(16_214));
s.observe_callback(None, Q); // resumed
s.observe_callback(Some(Duration::from_millis(5)), Q);
assert_eq!(s.gaps, 0, "a pause is still not a delivery gap");
assert_eq!(s.max_gap_ms(), 0);
assert_eq!(s.pauses, 1, "…but it is now countable");
assert_eq!(s.paused_ms(), 16_214);
}
/// One long outage and a burst of short flaps must not read alike — the same argument that
/// makes `gaps` and `max_gap_ms` two fields instead of one. The triple here is the shape every
/// Skynet and AVALON session start produced: three dwells, no format actually changing.
#[test]
fn pause_spans_accumulate_and_stay_countable() {
let mut long = CaptureStats::default();
long.observe_pause(Duration::from_millis(38_400));
let mut flappy = CaptureStats::default();
for ms in [12_534, 17_030, 8_765] {
flappy.observe_pause(Duration::from_millis(ms));
}
assert_eq!(long.pauses, 1);
assert_eq!(flappy.pauses, 3);
assert_eq!(flappy.paused_ms(), 38_329);
assert!(
long.paused_ms().abs_diff(flappy.paused_ms()) < 100,
"near-identical dead time, and the count is the only thing that separates them"
);
}
/// The discriminator the field logs needed. A stream that is running and starved reports gaps
/// and NO pause; a stream that was never scheduled reports the mirror image. Both dilute
/// `delivered_pct` identically, which is exactly why neither can be diagnosed from it alone.
#[test]
fn starvation_and_absence_are_told_apart() {
let mut starved = CaptureStats::default();
for _ in 0..60 {
starved.observe_callback(Some(Duration::from_millis(30)), Q);
}
let mut absent = CaptureStats::default();
absent.observe_pause(Duration::from_millis(1_800));
assert_eq!(starved.gaps, 60);
assert_eq!(starved.pauses, 0, "a running stream was never absent");
assert_eq!(absent.gaps, 0);
assert_eq!(absent.pauses, 1, "an absent stream never got to be slow");
assert_eq!(absent.paused_ms(), 1_800);
}
/// The acquittal case, and the whole reason [`SendStats`] exists: a pacer doing its job must
/// produce a line a reader can dismiss at a glance.
#[test]
fn a_healthy_pacer_reports_nothing_alarming() {
let mut s = SendStats::default();
let frame = Duration::from_millis(FRAME_MS as u64);
for i in 0..200 {
s.observe_departure(Duration::ZERO, (i > 0).then_some(frame), false);
}
assert_eq!(s.sent, 200);
assert_eq!(s.late, 0);
assert_eq!(s.reanchors, 0);
assert_eq!(s.infilled, 0);
assert_eq!(s.max_late_ms(), 0);
assert_eq!(s.max_spacing_ms(), FRAME_MS as u64);
}
/// Lateness under one frame is jitter, not a slip — but it must still be *visible*, or
/// "never late" and "never late by a whole frame" become the same report.
#[test]
fn sub_frame_lateness_is_measured_without_being_counted() {
let mut s = SendStats::default();
s.observe_departure(Duration::from_micros(3_400), None, false);
assert_eq!(s.late, 0, "3.4 ms has not slipped a whole 5 ms slot");
assert_eq!(s.max_late_ms(), 3, "…and it is still on the record");
}
/// A slot missed by a whole frame or more is the event the field logs could never show.
#[test]
fn a_slipped_slot_is_counted_and_its_worst_case_kept() {
let mut s = SendStats::default();
s.observe_departure(Duration::from_millis(6), None, false);
s.observe_departure(
Duration::from_millis(41),
Some(Duration::from_millis(47)),
false,
);
s.observe_departure(Duration::ZERO, Some(Duration::from_millis(5)), false);
s.observe_reanchor();
assert_eq!(s.late, 2);
assert_eq!(s.max_late_ms(), 41);
assert_eq!(s.max_spacing_ms(), 47, "the wire's worst quiet stretch");
assert_eq!(s.reanchors, 1);
}
/// Wire continuity is not captured continuity. A window whose frames were all synthesized
/// looks perfect on every other counter, and must not be readable as healthy audio.
#[test]
fn synthesized_frames_stay_distinguishable_from_captured_ones() {
let mut s = SendStats::default();
let frame = Duration::from_millis(FRAME_MS as u64);
for _ in 0..100 {
s.observe_departure(Duration::ZERO, Some(frame), true);
}
assert_eq!(s.sent, 100);
assert_eq!(
s.infilled, 100,
"every one of these was silence we invented"
);
assert_eq!(s.late, 0);
}
}
+135 -41
View File
@@ -395,6 +395,11 @@ const MIC_STALE: Duration = Duration::from_secs(1);
/// against the same number the ask used.
const CAPTURE_QUANTUM_FRAMES: u32 = 240;
/// Callbacks that must agree on a new buffer size before it replaces the one gaps are scored
/// against. Three is enough to reject a boundary artefact and still adopt a genuine re-plan
/// within ~15 ms.
const QUANTUM_CONFIRM: u8 = 3;
fn mic_pw_thread(
pcm_rx: Receiver<(std::time::Instant, Vec<f32>)>,
quit_rx: pipewire::channel::Receiver<Terminate>,
@@ -686,9 +691,19 @@ fn pw_thread(
use pw::{properties::properties, spa};
use spa::param::audio::{AudioFormat, AudioInfoRaw};
use spa::pod::Pod;
// The stream's `process` callbacks run ON this mainloop thread (we never hand PipeWire a
// separate data loop), so PipeWire's own client `module-rt` boost of its data loops does not
// cover it — the ~2.7 ms capture quantum lives or dies by this thread's scheduling.
// ⚠ This boosts the MAINLOOP thread, which is NOT where the capture callback runs.
//
// The previous comment here asserted the opposite ("we never hand PipeWire a separate data
// loop"), and it was wrong: we pass `RT_PROCESS` below, so libpipewire runs `process()` on a
// data loop it creates and schedules itself. Measured in one live host process on 2026-08-15
// — this thread at SCHED_OTHER/nice 0, `data-loop.0` at SCHED_RR/20. That mattered more than
// a stale comment usually does: a field investigation read the boost's success line as
// evidence that the audio callback was prioritised, and spent a round concluding priorities
// were "engaged but insufficient" when they had never been applied to the thread in question.
//
// The boost is kept — this thread still dispatches state and format events, and it IS the
// capture thread when `PUNKTFUNK_STREAM_SINK=0` selects the legacy monitor path. What replaces
// the assumption is a measurement: the callback reports its own scheduling on first entry.
pf_frame::thread_qos::boost_thread_priority(true);
// Setup errors funnel through the ready handshake (mirrors mic_pw_thread's IIFE).
@@ -782,12 +797,17 @@ fn pw_thread(
channels: u32,
stats: crate::audio::capture_policy::CaptureStats,
last_stats: std::time::Instant,
/// Whether this OPEN has reported its negotiated buffer size yet. Per-open, not the
/// process-wide `static AtomicBool` this replaces: a host runs for days across many
/// sessions, so the old form reported the very first capture of the process and then
/// never again — the one number that identifies a clamped quantum, invisible on every
/// Frames per callback the graph is currently handing us, `0` until the first is
/// confirmed. Per-open, not a process-wide latch: a host runs for days across many
/// sessions, so a process-wide form reported the very first capture and then never
/// again — the one number that identifies a clamped quantum, invisible on every
/// subsequent open (including every reopen after a device change).
reported_quantum: bool,
quantum_frames: usize,
/// A buffer size seen but not yet believed, with how many callbacks in a row have
/// agreed on it. Stops one short buffer from moving the gap threshold.
quantum_candidate: Option<(usize, u8)>,
/// Whether this open has reported the scheduling of the thread running `process()`.
reported_sched: bool,
/// When the callback last ran (WP-A2), so its CADENCE can be scored and not just its
/// content. Cleared across a state transition — a deliberate Paused span must not
/// read as one enormous hole. Lives here rather than in `stats` because the stats
@@ -804,19 +824,25 @@ fn pw_thread(
/// Shared with the capturer — see [`PwAudioCapturer::active`]. Read on every
/// failed hand-off to keep parked-capturer backpressure out of the drop count.
active: Arc<AtomicBool>,
/// When the stream last left `Streaming`, so the span can be charged to the window
/// that the span itself stretched. `None` while streaming.
paused_since: Option<std::time::Instant>,
}
let ud = CapUd {
tx,
channels,
stats: Default::default(),
last_stats: std::time::Instant::now(),
reported_quantum: false,
quantum_frames: 0,
quantum_candidate: None,
reported_sched: false,
last_cb: None,
quantum: Duration::from_micros(
CAPTURE_QUANTUM_FRAMES as u64 * 1_000_000 / SAMPLE_RATE as u64,
),
negotiated: None,
active,
paused_since: None,
};
let _listener = stream
.add_local_listener_with_user_data(ud)
@@ -829,6 +855,22 @@ fn pw_thread(
// existing. Scoring it would report one huge hole per renegotiation and bury
// the sub-10 ms ones the field log is actually about (WP-A2).
ud.last_cb = None;
// …but it still has to be reported, because the reporting window is flushed
// from the process callback and therefore stretches by the whole span. Charge
// it to the window flushed after the resume — the same window it diluted.
// Without this the line says `delivered_pct=4 gaps=0` and cannot say whether
// that is a dead capture path or a sink nobody was rendering into; the
// 2026-08-15 field logs are 40 s of exactly that ambiguity per session start.
match new {
pw::stream::StreamState::Streaming => {
if let Some(since) = ud.paused_since.take() {
ud.stats.observe_pause(since.elapsed());
}
}
_ => {
ud.paused_since.get_or_insert_with(std::time::Instant::now);
}
}
// A stream error is unrecoverable for this instance — exit so the sessions'
// reopen path builds a fresh one (same contract as the core-error path above).
if matches!(new, pw::stream::StreamState::Error(_)) {
@@ -888,6 +930,24 @@ fn pw_thread(
ud.last_cb = Some(now);
ud.stats.observe_callback(since_last, ud.quantum);
if !ud.reported_sched {
ud.reported_sched = true;
// Say what the thread that ACTUALLY runs this callback is scheduled as.
// Whether the capture callback is realtime decides whether a Wine shader
// storm can deschedule it for tens of ms at a 2.7 ms quantum, and until
// now no log anywhere carried the answer — only that we had asked for a
// boost, on a different thread. Once per open, off the hot path after
// that.
let (policy, rt_priority, nice) =
pf_frame::thread_qos::current_thread_sched();
tracing::info!(
policy,
rt_priority,
nice,
"audio capture callback scheduling"
);
}
let Some(mut buffer) = stream.dequeue_buffer() else {
ud.stats.missed_dequeues += 1;
return;
@@ -913,42 +973,70 @@ fn pw_thread(
let region = &buf[offset..(offset + size).min(buf.len())];
// Negotiated as F32LE; reinterpret the byte region as interleaved f32.
let n = region.len() / 4;
if !ud.reported_quantum {
ud.reported_quantum = true;
// What we ASKED for vs what PipeWire actually handed us. Stating only the
// result ("samples=2048") reads as a fact about the device; stating it
// next to the request is what makes a clamp legible. A VM is the common
// cause — stock `pipewire.conf` raises `default.clock.min-quantum` to
// 1024 whenever `cpu.vm.name` is set, so a 5 ms ask silently becomes
// 21.3 ms and the audio plane starts arriving in bursts. That cost a
// whole field investigation to find; it should cost one log line.
let frames = n / (ud.channels.max(1) as usize);
let want = CAPTURE_QUANTUM_FRAMES as usize;
// What a gap is measured against from here on — see `CapUd::quantum`.
if frames > 0 {
// Track the quantum the graph is ACTUALLY handing us, not merely the first one
// it ever did. The graph re-plans whenever anything else on the box asks for a
// different latency, and latching the first callback of the open left every
// subsequent gap scored against a buffer size that no longer existed — a
// silent corruption of the one metric this whole diagnosis rests on. A new
// size has to survive `QUANTUM_CONFIRM` callbacks before it is believed,
// because one short buffer at a boundary is not a new deal.
let frames = n / (ud.channels.max(1) as usize);
if frames > 0 && frames != ud.quantum_frames {
let streak = match ud.quantum_candidate {
Some((f, c)) if f == frames => c.saturating_add(1),
_ => 1,
};
if streak < QUANTUM_CONFIRM {
ud.quantum_candidate = Some((frames, streak));
} else {
let was = ud.quantum_frames;
ud.quantum_frames = frames;
ud.quantum_candidate = None;
// What a gap is measured against from here on — see `CapUd::quantum`.
ud.quantum = Duration::from_micros(
frames as u64 * 1_000_000 / SAMPLE_RATE as u64,
);
let want = CAPTURE_QUANTUM_FRAMES as usize;
let negotiated_ms =
format!("{:.1}", frames as f32 * 1000.0 / SAMPLE_RATE as f32);
if was != 0 {
// A mid-open change. Rare, and worth a line of its own: it moves
// the gap threshold under a reader who is comparing windows.
tracing::info!(
previous_frames = was,
negotiated_frames = frames,
negotiated_ms,
"the audio graph re-planned our quantum mid-stream"
);
} else if frames > want {
// What we ASKED for vs what PipeWire actually handed us. Stating
// only the result ("samples=2048") reads as a fact about the
// device; stating it next to the request is what makes a clamp
// legible. A VM is the common cause — stock `pipewire.conf` raises
// `default.clock.min-quantum` to 1024 whenever `cpu.vm.name` is
// set, so a 5 ms ask silently becomes 21.3 ms and the audio plane
// starts arriving in bursts. That cost a whole field
// investigation to find; it should cost one log line.
tracing::warn!(
requested_frames = want,
negotiated_frames = frames,
negotiated_ms,
"the audio graph refused our low-latency quantum — capture \
arrives in bursts this size, and the client must buffer at \
least that much to play them smoothly. On a VM this is \
PipeWire's `default.clock.min-quantum = 1024` rule; check \
`pw-metadata -n settings`"
);
} else {
tracing::info!(
requested_frames = want,
negotiated_frames = frames,
"audio capture quantum negotiated"
);
}
}
if frames > want {
tracing::warn!(
requested_frames = want,
negotiated_frames = frames,
negotiated_ms =
format!("{:.1}", frames as f32 * 1000.0 / SAMPLE_RATE as f32),
"the audio graph refused our low-latency quantum — capture arrives \
in bursts this size, and the client must buffer at least that \
much to play them smoothly. On a VM this is PipeWire's \
`default.clock.min-quantum = 1024` rule; check \
`pw-metadata -n settings`"
);
} else {
tracing::info!(
requested_frames = want,
negotiated_frames = frames,
"audio capture quantum negotiated"
);
}
} else if frames == ud.quantum_frames {
ud.quantum_candidate = None;
}
let mut samples = Vec::with_capacity(n);
for i in 0..n {
@@ -991,6 +1079,12 @@ fn pw_thread(
// percentage and mean entirely different things.
gaps = ud.stats.gaps,
max_gap_ms = ud.stats.max_gap_ms(),
// The OTHER thing a shortfall can be (see `CaptureStats::pauses`):
// time our node was not in the graph at all. `gaps` deliberately
// cannot see it, so without these two a paused span and a starved
// stream are the same number.
pauses = ud.stats.pauses,
paused_ms = ud.stats.paused_ms(),
missed_dequeues = ud.stats.missed_dequeues,
dropped_chunks = ud.stats.dropped_chunks,
"desktop audio capture"
+187 -27
View File
@@ -2,8 +2,8 @@
//! DualSense-family pad, wearing the identity DS5-native titles and GE-Proton's
//! controller-audio routing match on — so a game that renders voice-coil haptics or pad-speaker
//! audio finds "the controller's audio device" and plays into us. We own the sink, so the
//! `process()` callback IS the capture: 4-ch F32 48 kHz (FL FR RL RR — front pair = speaker,
//! back pair = voice coils, the same quad layout the Windows endpoint is stamped with) lands
//! `process()` callback IS the capture: 4-ch F32 48 kHz (AUX0..AUX3 — front pair = speaker,
//! back pair = voice coils, the same quad *order* the Windows endpoint is stamped with) lands
//! directly in the chunk channel that feeds the 0xD1 lanes (`native/pad_audio.rs`).
//!
//! Modeled on the stream-sink mode of [`super::PwAudioCapturer`] (same MainLoop-on-a-thread,
@@ -11,17 +11,34 @@
//! differences: **no default-sink claim** (nothing may auto-route here — games target it BY
//! IDENTITY) and a low `priority.session` so WirePlumber never elects it against real hardware.
//!
//! **Identity** (design `dualsense-audio-haptics-and-speaker.md` §3/§5): GE-Proton 11-2+
//! matches layered — pulse proplist (`device.bus == "usb"`, `device.vendor.id == 0x054c`,
//! **Identity** (design `dualsense-audio-haptics-and-speaker.md` §3/§5): GE-Proton matches
//! layered — pulse proplist (`device.bus == "usb"`, `device.vendor.id == 0x054c`,
//! `device.product.id ∈ {0x0ce6, 0x0df2}`), then name substrings
//! (`Sony_Interactive_Entertainment…Wireless_Controller`, `DualSense`); the community
//! WirePlumber rule keys on the node-name substring and sets `node.description =
//! "Wireless Controller"` (we mint it that way from the start). A pure PipeWire node cannot
//! satisfy wine's ContainerId derivation (udev walk to a `usb_device` parent → `GUID_NULL`)
//! nor GE's raw-ALSA fast path — both fall back to the Pulse-routed leg, which winepulse
//! serves from exactly this node (it enumerates sinks). Every identity string has an env
//! override for field debugging (`PUNKTFUNK_PAD_SINK_NAME` / `PUNKTFUNK_PAD_SINK_DESC`, with
//! `{pad}` / `{mac}` placeholders).
//! "Wireless Controller"` (we mint it that way from the start).
//!
//! We wear a real pad's **name** and Pro Audio's **channel layout** — deliberately not the same
//! profile for both, because no single real-pad profile satisfies GE on its own.
//!
//! Since alsa-ucm-conf gained `USB-Audio/Sony/DualSense-PS5` (2026-08-03) a real pad's profiles
//! are UCM SplitPCM views of one 4-channel PCM: a mono `Speaker__sink`, a stereo `Headphones`
//! sink, and a 4-channel `Direct__Direct__sink` (added "for wine compatibility"), plus ACP's
//! always-present Pro Audio. GE renders haptics as an `AUX0..AUX3` stream, so on every
//! *positioned* profile the graph re-mixes and the voice-coil pair is folded away — that is the
//! whole content of the field advice "you only need the controller audio set to Pro Audio", and
//! it is why this sink is one flat AUX quad rather than an emulation of the split topology. But
//! the pad-SPEAKER half of GE only binds to a sink whose name says `Speaker__sink`, and its
//! Windows 4-channel format forcing hangs off the same test. So the name says `Speaker__sink`
//! and the channels are Pro Audio's. GE explicitly supports that combination on real hardware
//! (see [`split_target`] and the node-name comment).
//!
//! What a pure PipeWire node still cannot satisfy is wine's ContainerId derivation (udev walk to
//! a `usb_device` parent → `GUID_NULL`; our pad is uhid and has no USB parent at all) and GE's
//! raw-ALSA leg (`snd_pcm_open` on an `api.alsa.path` that must be a real card). Its
//! `pipewire:NODE=` leg we *can* satisfy — see [`split_target`]. Every identity string has an
//! env override for field debugging (`PUNKTFUNK_PAD_SINK_NAME` / `PUNKTFUNK_PAD_SINK_DESC` with
//! `{pad}` / `{mac}` placeholders, `PUNKTFUNK_PAD_SINK_SPLIT_NAME`).
use anyhow::{anyhow, Context, Result};
use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError};
@@ -91,43 +108,118 @@ struct PadSinkIdentity {
serial: String,
product_id: &'static str,
product_name: &'static str,
card_name: &'static str,
long_card_name: String,
/// GE-Proton's `api.alsa.split.name` — the node name it opens as `pipewire:NODE=…` for the
/// haptic stream. Empty disables the key. See [`split_target`].
split_name: String,
}
/// GE-Proton reads `api.alsa.split.name` off the sink it is about to render haptics into and,
/// on its preferred leg, opens *that* node through its bundled pipewire-alsa plugin as
/// `pipewire:NODE=<name>` with `aux_channels=1` (patches 0114/0115/0116 of `proton-ds5-haptic`).
/// On a real pad the key names the **hidden 4-channel parent** WirePlumber mints for the UCM
/// SplitPCM profile — the public mono `Speaker__sink` is only a 1-channel split of it, so
/// rendering four channels at the public sink would lose the voice-coil pair.
///
/// We have no split: the sink IS the four-channel AUX node, so the honest value of the key is
/// our own `node.name` — GE then targets us directly instead of falling back to a leg that was
/// written to work around a topology we do not have. Without the key that leg cannot engage at
/// all (`get_dualsense_haptic_target` returns NULL), which is why titles GE auto-switches into
/// "Windows Sony audio mode" (the 8-format-probe games: Assassin's Creed, Death Stranding DC,
/// MH Wilds) never reached our sink.
///
/// `PUNKTFUNK_PAD_SINK_SPLIT_NAME` is the field lever: `0`/`false`/`off` drops the key (GE then
/// takes its Pulse leg, which also works for us because our channel positions already match its
/// forced `AUX0..AUX3` map), any other value overrides the target verbatim.
fn split_target(node_name: &str) -> String {
resolve_split_target(
node_name,
std::env::var("PUNKTFUNK_PAD_SINK_SPLIT_NAME").ok(),
)
}
/// [`split_target`]'s decision, with the environment lifted out so it is testable.
fn resolve_split_target(node_name: &str, override_var: Option<String>) -> String {
match override_var.as_deref().map(str::trim) {
Some("0" | "false" | "off" | "no") => String::new(),
Some(v) if !v.is_empty() => v.to_string(),
_ => node_name.to_string(),
}
}
impl PadSinkIdentity {
fn new(pad: u8, edge: bool) -> PadSinkIdentity {
let mac = pad_mac(pad);
let mac_bare: String = mac.chars().filter(|c| *c != ':').collect();
let (model, product_id, product_name) = if edge {
// The pad's USB `iProduct` string verbatim — a plain DualSense reports "Wireless
// Controller" with NO model word (only the Edge carries one). Getting this wrong is not
// cosmetic: udev builds the ALSA name out of manufacturer+product, so an invented
// `DualSense_` infix broke the contiguous `Sony_Interactive_Entertainment_Wireless_
// Controller` substring that the community WirePlumber rule and GE-Proton's
// `alsa_output.usb-Sony_Interactive_Entertainment_…` matchers key on.
let (usb_product, product_id, product_name, card_name) = if edge {
(
"DualSense_Edge",
"DualSense_Edge_Wireless_Controller",
"0df2",
"DualSense Edge Wireless Controller",
"DualSense Edge Wireless Controller",
)
} else {
("DualSense", "0ce6", "DualSense Wireless Controller")
(
"Wireless_Controller",
"0ce6",
"DualSense Wireless Controller",
"Wireless Controller",
)
};
// The ALSA-style name a REAL pad's card gets from udev (vendor_product_serial), which
// is what every known name-substring matcher was written against. `-00.analog-surround-40`
// = card profile suffix for the quad layout.
// udev's `ID_SERIAL`: manufacturer_product_serial. A real pad has no USB serial, so ALSA
// falls back to the card index; we carry the pad's virtual MAC there instead, which keeps
// multi-pad sinks distinct without disturbing the matched prefix.
let serial = format!("Sony_Interactive_Entertainment_{usb_product}_{mac_bare}");
// The `…-00.<verb>__Speaker__sink` suffix is LOAD-BEARING, not decoration. GE-Proton's
// `is_dualsense_speaker_sink()` is a pure substring test for `Speaker__sink` (plus the
// USB ids, or the `alsa_output.usb-Sony_Interactive_Entertainment_` + `Wireless_Controller`
// pair we also carry), and three things hang off it: `apply_windows_sony_audio_format()`
// forces the wine endpoint to the Windows 4×48 kHz `KSAUDIO_SPEAKER_QUAD` layout DS5
// titles probe for, the pad-SPEAKER (mono controller-effect) streams will only bind and
// retarget to a sink it accepts, and the whole controller-audio endpoint lands on the
// identity Spider-Man's working path used. A suffix naming any other profile — the
// `analog-surround-40` we used to mint, or a truthful `pro-output-0` — matches none of
// it, which left the speaker half of this feature with nothing to attach to.
//
// Carrying `Speaker__sink` AND [`split_target`] at once is a real pad's shape, not a
// contrivance: GE's own `is_dualsense_endpoint_speaker_sink` notes that "Edge speaker
// sinks may also carry raw haptic metadata", and handles the pair by keeping them as
// routing targets while withholding the *shared* mono endpoint id (a Spider-Man
// enumeration crash). The exclusion GE once had in `is_dualsense_speaker_sink` itself is
// gone. What we do NOT copy is a real pad's mono channel count: the sink stays four raw
// AUX channels — which is exactly what that endpoint is forced to advertise anyway.
let node_name = match std::env::var("PUNKTFUNK_PAD_SINK_NAME") {
Ok(t) if !t.trim().is_empty() => expand(&t, pad, &mac_bare),
_ => format!(
"alsa_output.usb-Sony_Interactive_Entertainment_{model}_Wireless_Controller_{mac_bare}-00.analog-surround-40"
),
_ => format!("alsa_output.usb-{serial}-00.HiFi__Speaker__sink"),
};
// What the community WirePlumber rule renames real pads TO — minted that way directly.
// Deliberately NOT the udev/hwdb description a real card gets ("DualSense wireless
// controller (PS5)"): wine hands `node.description` straight to the endpoint's
// `PKEY_Device_FriendlyName`, and the title matchers do a case-sensitive
// `wcsstr(name, L"Wireless Controller")` (FF14, FF7R) that a lowercase "wireless" fails.
let description = match std::env::var("PUNKTFUNK_PAD_SINK_DESC") {
Ok(t) if !t.trim().is_empty() => expand(&t, pad, &mac),
_ => "Wireless Controller".to_string(),
};
let split_name = split_target(&node_name);
PadSinkIdentity {
long_card_name: format!(
"Sony Interactive Entertainment {card_name} at usb-punktfunk-pad{pad}, full speed"
),
node_name,
description,
serial: format!(
"Sony_Interactive_Entertainment_{model}_Wireless_Controller_{mac_bare}"
),
serial,
product_id,
product_name,
card_name,
split_name,
}
}
}
@@ -141,6 +233,8 @@ pub struct PadSinkCapturer {
quit: pipewire::channel::Sender<Terminate>,
/// The minted node name, for logs and the devtest.
pub node_name: String,
/// What GE-Proton will read as `api.alsa.split.name`; empty when the key is suppressed.
pub split_name: String,
}
impl PadSinkCapturer {
@@ -149,6 +243,7 @@ impl PadSinkCapturer {
pub fn open(pad: u8, edge: bool) -> Result<PadSinkCapturer> {
let identity = PadSinkIdentity::new(pad, edge);
let node_name = identity.node_name.clone();
let split_name = identity.split_name.clone();
let (tx, rx) = sync_channel::<Vec<f32>>(64);
let (quit_tx, quit_rx) = pipewire::channel::channel::<Terminate>();
// Bring-up handshake (the session capturer's discipline): a PipeWire that isn't running
@@ -167,10 +262,26 @@ impl PadSinkCapturer {
Ok(Err(e)) => return Err(e),
Err(_) => return Err(anyhow!("pipewire pad-sink init timed out")),
}
// The identity a title has to match, in the log a field report will carry. Cheap once
// per pad, and it is the only place the negotiated strings are visible without a live
// `pactl` on the box.
let split_log = if split_name.is_empty() {
"(suppressed)"
} else {
split_name.as_str()
};
tracing::info!(
pad,
edge,
node_name = %node_name,
split_name = %split_log,
"pad-audio sink minted (Pro Audio shape: 4ch AUX0..AUX3, ch0/1 speaker, ch2/3 coils)"
);
Ok(PadSinkCapturer {
chunks: rx,
quit: quit_tx,
node_name,
split_name,
})
}
}
@@ -268,11 +379,22 @@ fn pad_sink_thread(
// this sink BY IDENTITY, nothing auto-routes here (no stream_sink claim either).
"priority.session" => "50",
// The pulse-proplist leg of GE-Proton's match (§3): bus + vendor/product ids, plus
// the human-readable pair pavucontrol and the game view show.
// the human-readable pair pavucontrol and the game view show. Every one of these
// reaches a wine/Proton client verbatim — pipewire-pulse fills a sink's proplist
// from the node's own props (`fill_sink_info_proplist`), it does not curate them.
"device.bus" => "usb",
"device.vendor.id" => "054c",
"device.vendor.name" => "Sony Interactive Entertainment",
"device.form_factor" => "gamepad",
"device.icon_name" => "audio-card-analog-usb",
// The shape, stated as props and not only as a negotiated format: four raw AUX
// channels — ch0/1 speaker, ch2/3 voice coils — which is what "Pro Audio" means on a
// real pad's card and the only layout that survives GE-Proton's AUX0..AUX3 stream
// map unfolded.
"audio.channels" => "4",
"audio.position" => "AUX0,AUX1,AUX2,AUX3",
"api.alsa.pcm.stream" => "playback",
"alsa.driver_name" => "snd_usb_audio",
};
props.insert(*pw::keys::NODE_NAME, identity.node_name.as_str());
props.insert(*pw::keys::NODE_DESCRIPTION, identity.description.as_str());
@@ -280,6 +402,13 @@ fn pad_sink_thread(
props.insert("device.serial", identity.serial.as_str());
props.insert("device.product.id", identity.product_id);
props.insert("device.product.name", identity.product_name);
props.insert("alsa.card_name", identity.card_name);
props.insert("alsa.long_card_name", identity.long_card_name.as_str());
// GE-Proton's preferred haptic leg; see `split_target`. Omitted (not empty) when the
// field lever turns it off, so `pa_proplist_gets` misses rather than returning "".
if !identity.split_name.is_empty() {
props.insert("api.alsa.split.name", identity.split_name.as_str());
}
let stream = pw::stream::StreamBox::new(&core, "punktfunk-pad-audio", props)
.context("pw pad-sink Stream")?;
@@ -428,22 +557,53 @@ mod tests {
#[test]
fn identity_carries_every_match_surface() {
let id = PadSinkIdentity::new(0, false);
// The name-substring matchers (GE-Proton + the community WirePlumber rule).
// GE-Proton's `string_contains_dualsense_name` legs, each checked separately.
assert!(id.node_name.contains("Sony_Interactive_Entertainment"));
assert!(id.node_name.contains("Wireless_Controller"));
assert!(id.node_name.contains("DualSense"));
assert!(id.node_name.ends_with("-00.analog-surround-40"));
// …and the CONTIGUOUS form the community WirePlumber rule and GE's
// `alsa_output.usb-Sony_Interactive_Entertainment_` prefix test want. An invented
// `DualSense_` infix used to split this in two and miss both.
assert!(id
.node_name
.starts_with("alsa_output.usb-Sony_Interactive_Entertainment_Wireless_Controller_"));
// The suffix GE's `is_dualsense_speaker_sink` substring-tests for — the pad-speaker
// binding and the Windows 4ch format forcing both hang off it (never `analog-*`, which
// matches nothing of GE's and names a positioned profile we do not wear).
assert!(id.node_name.ends_with("-00.HiFi__Speaker__sink"));
assert!(id.node_name.contains("Speaker__sink"));
// No colons in a udev-style serial/name.
assert!(!id.node_name.contains(':'));
// Case-sensitive `wcsstr(FriendlyName, L"Wireless Controller")` (FF14, FF7R).
assert_eq!(id.description, "Wireless Controller");
assert_eq!(id.product_id, "0ce6");
assert_eq!(id.card_name, "Wireless Controller");
assert!(id.long_card_name.contains("Sony Interactive Entertainment"));
let edge = PadSinkIdentity::new(1, true);
assert!(edge.node_name.contains("DualSense_Edge"));
// GE tests the Edge with the full `DualSense_Edge_Wireless_Controller` substring.
assert!(edge
.node_name
.contains("DualSense_Edge_Wireless_Controller"));
assert!(edge.node_name.contains("Speaker__sink"));
assert_eq!(edge.product_id, "0df2");
// Distinct pads mint distinct names (the serial octet).
assert_ne!(id.node_name, PadSinkIdentity::new(1, false).node_name);
}
#[test]
fn split_target_points_at_the_node_itself_unless_overridden() {
// No split on our side: the sink IS the four-channel parent GE wants to open as
// `pipewire:NODE=…`, so the honest target is our own name.
let id = PadSinkIdentity::new(0, false);
assert_eq!(id.split_name, id.node_name);
// The field lever, both ways — through the pure form, so no test mutates the process
// environment out from under a parallel test runner.
assert_eq!(resolve_split_target("n", None), "n");
assert_eq!(resolve_split_target("n", Some(" ".into())), "n");
assert!(resolve_split_target("n", Some("0".into())).is_empty());
assert!(resolve_split_target("n", Some("off".into())).is_empty());
assert_eq!(resolve_split_target("n", Some(" other ".into())), "other");
}
#[test]
fn template_expansion() {
assert_eq!(expand("pad{pad}-{mac}", 2, "AABB"), "pad2-AABB");
+12 -3
View File
@@ -256,9 +256,18 @@ pub fn pad_sink_test(args: &[String]) -> Result<()> {
let mut cap = crate::audio::pad_sink::PadSinkCapturer::open(pad, edge)
.context("mint pad-audio sink (is PipeWire running in this session?)")?;
println!(
"pad sink minted: node.name = {}\n inspect: pactl list sinks | grep -A20 punktfunk-pad\n \
drive it: pw-play --target '{}' <48k-file>\nCapturing for {secs}s",
cap.node_name, cap.node_name
"pad sink minted: node.name = {}\n api.alsa.split.name = {} (what GE-Proton opens as \
pipewire:NODE=)\n inspect: pactl list sinks | grep -A25 Speaker__sink\n \
drive it: pw-play --target '{}' --channel-map 'AUX0,AUX1,AUX2,AUX3' <48k-file>\n \
(a POSITIONED wav folds into the speaker pair and never reaches the coils the \
channel-map is not optional)\nCapturing for {secs}s",
cap.node_name,
if cap.split_name.is_empty() {
"(suppressed)"
} else {
cap.split_name.as_str()
},
cap.node_name
);
let deadline = Instant::now() + Duration::from_secs(secs);
let (mut chunks, mut samples) = (0u64, 0u64);
+42 -2
View File
@@ -218,6 +218,12 @@ pub(super) fn audio_thread(
// frame. `sent_any` is what keeps the seed from ever reaching the wire.
let mut next_pts_ns: u64 = 0;
let mut pace_due: Option<std::time::Instant> = None;
// WP-C — what the wire actually did, as opposed to what the tap handed us. See [`SendStats`]:
// until this existed the send path was the one stage of the audio pipeline that could not be
// ruled in or out from a field log.
let mut send_stats = crate::audio::capture_policy::SendStats::default();
let mut last_send_stats = std::time::Instant::now();
let mut last_departure: Option<std::time::Instant> = None;
if capturer.is_some() {
tracing::info!(
channels = want,
@@ -315,12 +321,20 @@ pub(super) fn audio_thread(
// send-time debt.
loop {
let now = std::time::Instant::now();
// How far past its slot this frame is leaving. Measured before the re-anchor arm can
// erase the evidence — that arm is the one that forgives debt silently (WP-C).
let mut late = std::time::Duration::ZERO;
match pace_due {
Some(due) if due > now => break, // this frame's slot has not arrived yet
Some(due) if now.duration_since(due) > PACE_REANCHOR => pace_due = None,
_ => {}
Some(due) if now.duration_since(due) > PACE_REANCHOR => {
send_stats.observe_reanchor();
pace_due = None;
}
Some(due) => late = now.duration_since(due),
None => {}
}
frame_buf.clear();
let mut infilled = false;
if acc.len() >= frame_len {
frame_buf.extend(acc.drain(..frame_len));
} else if !sent_any {
@@ -328,6 +342,7 @@ pub(super) fn audio_thread(
} else {
match infill.decide(last_chunk_at.elapsed()) {
crate::audio::capture_policy::Infill::Silence => {
infilled = true;
// Pad the partial frame out with silence and send THAT, rather than
// leaving it for post-gap samples to complete: one frame carrying audio
// from both sides of a hole is a click, and its pts is a lie about when
@@ -366,6 +381,15 @@ pub(super) fn audio_thread(
prev_frame.extend_from_slice(opus);
}
seq = seq.wrapping_add(1);
// Score the departure against its slot and against the previous one. `now` is
// from the top of this iteration — microseconds earlier and one clock read
// cheaper, 200 times a second.
send_stats.observe_departure(
late,
last_departure.map(|t| now.duration_since(t)),
infilled,
);
last_departure = Some(now);
// From here there is a continuity worth protecting, and `next_pts_ns` has a
// real anchor to continue from — both preconditions for synthesizing anything.
sent_any = true;
@@ -382,6 +406,22 @@ pub(super) fn audio_thread(
}
}
}
if last_send_stats.elapsed() >= crate::audio::capture_policy::STATS_EVERY {
// Deliberately the same window as the capture line, so the two can be read as a pair:
// holes at the tap with clean departures means the host delivered everything it had,
// and the search belongs upstream of us.
tracing::info!(
sent = send_stats.sent,
infilled = send_stats.infilled,
late = send_stats.late,
max_late_ms = send_stats.max_late_ms(),
max_spacing_ms = send_stats.max_spacing_ms(),
reanchors = send_stats.reanchors,
"audio egress"
);
send_stats = Default::default();
last_send_stats = std::time::Instant::now();
}
}
// Park the live capturer for the next session (None if it died and never reopened),
// releasing its session-scoped routing claim (Linux: the default sink moves back;
+3 -3
View File
@@ -208,9 +208,9 @@ reach, which app honours which of them, the two mouse modes, the three touch mod
are all on [Mouse, touch and pen](/docs/input#getting-your-input-back).
Copying between the two machines is a separate opt-in: the host operator allows it in `host.env`
and you turn it on for that one host in your client. Content crosses today from the macOS, Windows
and Android apps — the Linux client has the switch but no bridge behind it yet, and iOS, iPadOS and
tvOS have neither. See [Shared clipboard](/docs/clipboard).
and you turn it on for that one host in your client. Content crosses today from the macOS, iOS,
iPadOS, Windows and Android apps — the Linux client has the switch but no bridge behind it yet, and
tvOS has no pasteboard to share. See [Shared clipboard](/docs/clipboard).
## Which should I use?
+19 -9
View File
@@ -64,6 +64,7 @@ about *that* machine. You set it in the host's edit sheet, and it is deliberatel
| Client | Where the switch is | Label | Default |
|---|---|---|---|
| macOS | Host card menu → **Edit…** | **Share clipboard with this host** | Off |
| iOS, iPadOS | Host card menu → **Edit…** | **Share clipboard with this host** | Off |
| Windows | Host tile menu → **Edit…** | **Share clipboard with this host** | Off |
| Linux (GTK) | Host card menu → **Edit…** | **Share clipboard** | Off |
| Android (touch) | Host card menu → **Edit…** | **Shared clipboard** | **On** |
@@ -75,10 +76,12 @@ no clipboard row, so there is nowhere to change it there. It stays on, which is
The setting is read when a session starts, so if you change it while streaming, reconnect.
macOS can also flip it mid-session: **Stream ▸ Share Clipboard** (⌃⌥⇧C), which becomes **Stop
Sharing Clipboard** once the host has acknowledged it.
Sharing Clipboard** once the host has acknowledged it. On an iPad with a hardware keyboard the same
combo works, though there is no menu bar to show it in — and only while the pointer is released, as
a captured session sends the keys to the host instead.
iOS, iPadOS, tvOS and a Steam Deck in Gaming Mode have no clipboard switch — neither the Decky
panel nor the client's console home has a host edit sheet — see
tvOS and a Steam Deck in Gaming Mode have no clipboard switch — the Apple TV has no pasteboard to
share at all, and neither the Decky panel nor the client's console home has a host edit sheet — see
[what each client does](#which-hosts-and-clients-support-it) below.
## Nothing crosses until something pastes
@@ -91,16 +94,22 @@ That holds for everything you copy on your own machine, and for both directions
does **not** hold for a host copy arriving at a Windows or Android client: those two fetch the
content straight away and put it on your local clipboard, whether or not you ever paste. On Windows
that is because the lazy path needs Windows delayed rendering, which the client doesn't implement
yet; on Android there is no way to satisfy a paste from the network at all. The macOS client is
lazy in both directions.
yet; on Android there is no way to satisfy a paste from the network at all. The macOS and iOS
clients are lazy in both directions.
On iOS there is one deliberate exception. Backgrounding the app ends the session, and a promise
nobody can answer is worse than no promise at all — so if the host copied something and you have not
pasted it yet, those bytes are pulled across as the session ends, up to 8 MiB. That is what makes
"copy on the host, switch to Safari, paste" work on an iPad. Nothing is fetched if you never leave
the app, or if you already pasted.
A single transfer is capped at 64 MiB. Nothing else limits size, so a very large host-side copy can
cross to a Windows or Android client for a paste that never happens.
What you copy on the **macOS or Windows client** is filtered for secrets: content marked
`org.nspasteboard.ConcealedType` or `org.nspasteboard.TransientType` on macOS, or
What you copy on the **macOS, iOS or Windows client** is filtered for secrets: content marked
`org.nspasteboard.ConcealedType` or `org.nspasteboard.TransientType` on the Apple clients, or
`ExcludeClipboardContentFromMonitorProcessing` on Windows — what password managers set — is never
announced and never served. That check exists only in those two clients. The Android client has no
announced and never served. That check exists only in those clients. The Android client has no
equivalent, and neither does the host, so a password copied **on the host** is announced to your
client like anything else.
@@ -128,10 +137,11 @@ when a host application pastes.
| Client | What crosses |
|---|---|
| macOS | Plain text, rich text (RTF), HTML, and PNG, JPEG and GIF images |
| iOS, iPadOS | Plain text, rich text (RTF), HTML, and PNG, JPEG and GIF images |
| Windows | Plain text, and PNG images |
| Android, Android TV | **Plain text only** |
| Linux (GTK), Steam Deck | Nothing yet — see below |
| iOS, iPadOS, tvOS | Not implemented |
| tvOS | Not implemented — tvOS has no pasteboard |
The **Linux client has the switch but no working clipboard bridge**: it enables the plane and then
has no code to read or write the desktop's own clipboard, so nothing is announced and nothing is
+2 -1
View File
@@ -146,9 +146,10 @@ See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set t
|---|---|---|
| `PUNKTFUNK_GAMEPAD` | `xbox360` · `xboxone` · `dualsense` · `dualsenseedge` · `dualshock4` · `steamdeck` · `switchpro` · `steamcontroller` · `steamcontroller2` (aliases: `ps5`, `edge`, `ps4`, `deck`, `switch`, `sc2`, `ibex`, …) | The virtual pad the host creates. Usually **auto-resolved from the client's physical controller** — set this only to force a type. `xbox360` (XInput) is the universal fallback. `dualsenseedge` gives the client's back paddles native buttons; `switchpro` gives Nintendo-family pads correct glyphs/layout + gyro. `steamcontroller2` (the 2026 Steam Controller) is passed through **as-is** — the host presents a real SC2 (`28DE:1302`) that Steam Input drives directly, mirroring the physical pad's raw reports (Linux only). DualSense (Edge)/DualShock 4 work on Linux (UHID) and Windows (UMDF); the Steam Deck pad too (Windows via the promoted UMDF identity); Switch Pro and the classic Steam Controller need Linux UHID. Unsupported choices fold to Xbox 360. |
| `PUNKTFUNK_STEAM_GADGET` | `1` · `0` | Force the raw USB-gadget virtual Steam Deck on/off. **On by default on SteamOS**, off elsewhere. Lets Steam promote the virtual Deck to full Steam Input. |
| `PUNKTFUNK_PAD_AUDIO` | `1` · `0` *(default on)* | Controller audio: what a game plays through the DualSense's built-in speaker and voice-coil haptics is streamed to the client's physical pad as its own low-latency plane. On by default and free while idle — silence is never encoded or sent; `0` turns it off host-wide. On Windows the pad's audio device is a pre-provisioned virtual endpoint; on Linux it is a per-pad PipeWire sink minted with the DualSense identity games match on. |
| `PUNKTFUNK_PAD_AUDIO` | `1` · `0` *(default on)* | Controller audio: what a game plays through the DualSense's built-in speaker and voice-coil haptics is streamed to the client's physical pad as its own low-latency plane. On by default and free while idle — silence is never encoded or sent; `0` turns it off host-wide. On Windows the pad's audio device is a pre-provisioned virtual endpoint; on Linux it is a per-pad PipeWire sink minted with the DualSense identity games match on — see [Controller speaker and haptics](/docs/controller-audio). |
| `PUNKTFUNK_PAD_AUDIO_SLOTS` | `1``4` *(default: Windows `1`, Linux `4`)* | How many controllers can have their own audio at once. On Windows each slot is a pre-provisioned virtual endpoint, so the default stays at one; a Linux sink is minted lazily and costs nothing idle, so every slot is on. |
| `PUNKTFUNK_PAD_SINK_NAME` / `PUNKTFUNK_PAD_SINK_DESC` | templates | **(Linux, field debugging)** Override the minted pad sink's `node.name` / `node.description`. `{pad}` and `{mac}` expand per pad. Only for chasing a title whose device matcher wants different strings — the defaults carry every known match surface. |
| `PUNKTFUNK_PAD_SINK_SPLIT_NAME` | node name · `0` *(default: the sink's own name)* | **(Linux, field debugging)** The `api.alsa.split.name` the pad sink advertises. GE-Proton opens that node as `pipewire:NODE=…` with AUX channels for its preferred haptic path; on a real pad it names the hidden 4-channel parent behind the mono speaker split, and our sink has no split, so it names itself. `0` drops the key, which pushes GE onto its Pulse-routed leg instead. |
## Audio / microphone
+121
View File
@@ -0,0 +1,121 @@
---
title: Controller speaker and haptics
description: DualSense voice-coil haptics and the pad's built-in speaker, streamed from the host to the controller in your hands — what to enable, and what "set it to Pro Audio" means on a Linux host.
---
A DualSense is partly an audio device. Its little speaker and its two voice-coil motors — the
actuators that make a PS5 pad feel like sand, rain or a bowstring instead of a buzzing phone —
are all driven by a four-channel audio stream, not by rumble commands. Games that support them
write PCM into "the controller's audio device".
Punktfunk gives that device to the game on the host, captures what the game writes, and streams
it to the controller physically in your hands, on its own low-latency plane. Channels 12 are the
pad's speaker, channels 34 are the voice coils.
## What you need
- **A DualSense or DualSense Edge plugged in over USB** on the client. Bluetooth pads expose no
audio interface at all, so they fall back to ordinary rumble — this is a limit of the
controller, not of Punktfunk.
- On the client, **Controller haptics** is on by default. **Controller speaker** is opt-in: turn
it on if you want game audio coming out of the pad as well as your speakers.
- On a **Linux host**, a game that speaks DualSense — which in practice means running it under
**GE-Proton 11-5 or newer**. Stock Proton does not route controller audio.
- On the host, controller audio is on by default (`PUNKTFUNK_PAD_AUDIO`).
Nothing is sent while the pad is quiet, so leaving it on costs nothing.
## "Set the controller audio to Pro Audio" — you don't have to
If you have looked into DualSense haptics on Linux before, you have probably run into this
advice: plug the pad into the Linux box, open your sound settings, find *DualSense wireless
controller (PS5)*, and switch its **Profile** to **Pro Audio**. That advice is real and it is
correct — for a pad plugged directly into the host.
The reason is channel layout. A pad's other profiles present it as a mono speaker, a stereo
headphone jack, or a positioned four-channel "surround" device. Games write their haptics as four
*unpositioned* channels, so on any of those profiles the audio system helpfully re-mixes them into
the speaker pair and the voice-coil channels are folded away. You feel nothing. Pro Audio is the
one profile that hands the four channels through untouched, in order.
**Punktfunk's controller audio device is already in that shape.** It is created as four raw
channels with no re-mixing, which is exactly what Pro Audio produces — so there is nothing to
switch, and no switch to make.
That is also why it looks different in your sound settings. A real pad is a USB sound card, so it
gets a **Profile** dropdown; Punktfunk's is a software device, so it has no card and no dropdown.
Seeing **Wireless Controller** with a volume slider and no profile selector is what a correctly
minted controller-audio device looks like. It is not a sign that something is missing.
## Checking it is working
On the host, one line per pad is logged when the device is created:
```
pad-audio sink minted (Pro Audio shape: 4ch AUX0..AUX3, ch0/1 speaker, ch2/3 coils)
```
and, once a client that can render it connects:
```
pad audio streaming (0xD1, Opus 48 kHz, silence-gated)
```
When a game actually starts driving the actuators, the pad's own driver reports it:
```
DS5 title asserted haptics-select (audio haptics) pad=0
```
That last line is the one that matters: it means a title recognised the controller as an audio
device and switched the pad out of plain rumble. If you see it and still feel nothing, the problem
is downstream — on the client or the pad. If you never see it, the game never found the device.
You can also look at the device directly:
```sh
pactl list sinks | grep -A25 Speaker__sink
```
The line to check is `audio.position = "AUX0,AUX1,AUX2,AUX3"` — four unpositioned channels is the
layout that reaches the voice coils. Anything positioned (`FL,FR,RL,RR`) would not.
## If a game does not find it
Games identify the controller's audio device by name and by USB ids, and different titles check
different things. GE-Proton has several routes to the pad, and a couple of them are opt-in per
game. Add these as launch options if a title is not cooperating:
```
PROTON_DUALSENSE_HAPTICS_PREFER_NON_EVENT=1 %command%
```
This forces GE onto its most direct route — it opens Punktfunk's controller-audio device by name
and writes the four channels straight into it, with no re-mixing anywhere in between. It is the
first thing to try.
Some titles additionally want:
```
PROTON_SONY_WINDOWS_DEVICE_NAMES=1 PROTON_KEEP_SONY_AUDIO_ENDPOINT_VISIBLE=1 %command%
```
and *Death Stranding Director's Cut* has its own:
```
PROTON_DUALSENSE_SPLIT_AUDIO=1 %command%
```
To see which route GE took, launch the game with `WINEDEBUG=+pulse` and look for a line beginning
`Routing DualSense`. It names the device it chose and how it opened it.
## Known limits
- **Bluetooth client pads get rumble, not haptics.** No audio interface exists over BT.
- **Titles that match the controller by container ID** — a Windows notion of "these devices are
the same physical thing" — will not recognise the pairing on a Linux host, because the virtual
pad has no USB device behind it to derive one from. Titles that match by name or by USB ids are
unaffected, which is most of them.
- **A pad plugged into the host itself can steal the audio.** If a real DualSense is connected to
the host while you are streaming to a different one, some titles will find the local pad's sound
card first. Unplug it, or stream from a host that has no pad attached.
+1
View File
@@ -42,6 +42,7 @@
"moonlight",
"---Using Punktfunk---",
"input",
"controller-audio",
"client-settings",
"profiles-and-links",
"game-library",
+2 -2
View File
@@ -38,8 +38,8 @@ than de-escalating and staying degraded. The next structural lever is **sub-fram
overlapping encode and transmit inside a single frame via a direct slice path — which matters most
at high resolutions.
**Finishing the clipboard.** Text crosses today from a Windows, macOS or Android client to a host
whose operator turned the feature on, with images on the first two. Two pieces are genuinely
**Finishing the clipboard.** Text crosses today from a Windows, macOS, iOS, iPadOS or Android
client to a host whose operator turned the feature on, with images on all but Android. Two pieces are genuinely
unfinished: the **Linux client's** side of the bridge is a stub, so a Linux client offers and
applies nothing; and **file transfer** has a wire format and a host-side policy but no client that
offers files, so a copied file never crosses. See [Clipboard](/docs/clipboard).
+7 -5
View File
@@ -504,7 +504,7 @@ text from an IME), are covered in [Input](/docs/input).
| Linux desktop | ✅ ¹ | ⚠️ ¹³ | ✅ ² | ✅ | ❌ ³ | ✅ |
| Windows desktop | ✅ ¹ | ⚠️ ¹³ | ✅ ² | ✅ | ⚠️ ⁴ | ✅ |
| macOS | ⚠️ ⁵ | ⚠️ ⁶ | ✅ ² | ✅ | ✅ ⁷ | ✅ |
| iPhone · iPad | ⚠️ ⁵ | ⚠️ ⁶ | ✅ ² | ✅ | ❌ ⁸ | ✅ |
| iPhone · iPad | ⚠️ ⁵ | ⚠️ ⁶ | ✅ ² | ✅ | ✅ ⁷ | ✅ |
| Apple TV | ⚠️ ⁵ | ⚠️ ⁶ | ✅ ² | ❌ ⁹ | ❌ ⁸ | ✅ |
| Android · Android TV | ⚠️ ⁵ | ❌ | ✅ ² | ✅ | ⚠️ ¹⁰ | ✅ |
| Moonlight | ⚠️ ¹¹ | ❌ | ✅ ² | ❌ | ❌ | ❓ ¹² |
@@ -523,8 +523,10 @@ text from an IME), are covered in [Input](/docs/input).
full-chroma colour conversion, so 4:4:4 needs no encoder probe and resolves on any vendor. See
[Client settings](/docs/client-settings).
7. The richest implementation: text, RTF, HTML and images, fetched lazily in both directions.
Concealed and transient pasteboard items are skipped.
8. The clipboard bridge is macOS-only within the Apple app.
Concealed and transient pasteboard items are skipped. On iOS one thing is not lazy: because
backgrounding the app ends the session, an unpasted host offer is pulled across (up to 8 MiB)
as the session ends, so it survives to be pasted in another app.
8. tvOS has no pasteboard, so there is nothing to share.
9. tvOS gives apps no microphone input at all.
10. Text only, by design — and unlike every other client, Android's per-host **Shared clipboard**
switch starts **on** (the desktop clients and the Apple app default it off). Nothing crosses
@@ -550,7 +552,7 @@ These are negotiated, and either side can be the reason it did not happen:
host without the clipboard protocol (a gamescope session, for instance). The per-host "Share
clipboard" switch is edited before you connect, so it is never greyed out — on Linux, Windows,
Android and the Apple add-host sheet it stays settable against a host that will refuse, and just
does nothing. Only macOS reflects the refusal live: the Stream ▸ Share Clipboard menu item is
does nothing. Only the Apple app reflects the refusal live: the Stream ▸ Share Clipboard item is
disabled when the connected host has not advertised the capability.
- **Pen input** — the host advertises it only if it can really inject: a usable `/dev/uinput` on
Linux, or synthetic pointer devices on Windows 10 1809+. Without it, clients fold pen into touch.
@@ -574,7 +576,7 @@ capability.
| **Windows host** | Newest large surface, shipping as an installer with its own virtual-display driver — both signed with Punktfunk's own certificates rather than a publicly trusted one, so Windows warns on install (see [Windows host](/docs/windows-host#about-the-signatures)). NVENC is well trodden; the AMD (AMF) path was validated on real hardware in mid-2026 (Ryzen 7000 iGPU, 1080p120 HDR P010) and the Intel (QSV) path on Arc, but both see far less field time than NVENC — several of QSV's newer arms are still marked unvalidated in the code. One structural constraint that catches people: it must run in the interactive console session, not session 0. |
| **GameStream / Moonlight plane** | Works, and whether it is on depends on how you installed. Every Linux package (deb, RPM, Arch, the Bazzite sysext) and the SteamOS installer ship the unit as `serve --gamestream`, so GameStream is **on** there; NixOS defaults it on too. The Windows installer's checkbox is unticked, so it is **off** unless you asked for it, and a bare `punktfunk-host serve` is off. It pairs over plain HTTP with weaker legacy encryption — trusted LAN only, and worth turning off if you don't use Moonlight (see [Security](/docs/security#gamestream--moonlight-compatibility-is-the-weak-crypto-path)). It is a compatibility surface, so Punktfunk-only features (profiles, links, clipboard, microphone) are not on it. |
| **Linux and Windows desktop clients** | Packaged and current. They are one codebase: the same session binary streams for both, and for the Decky plugin and the `punktfunk` CLI. |
| **Apple client** (macOS · iOS · iPadOS · tvOS) | One universal build, distributed as a **TestFlight beta**; the Mac also has a notarized DMG. Feature-complete apart from the platform gaps named above (no microphone on tvOS, clipboard on macOS only). |
| **Apple client** (macOS · iOS · iPadOS · tvOS) | One universal build, distributed as a **TestFlight beta**; the Mac also has a notarized DMG. Feature-complete apart from the platform gaps named above (no microphone or clipboard on tvOS). |
| **Android client** (phone · TV) | Published on **Google Play** as a public listing for releases, with an invite-only Internal testing track for canary, plus a sideloadable APK. The same app in leanback mode is the TV client. |
| **Decky plugin** (Steam Deck) | Ships through install-from-URL rather than the Decky store, and keeps itself and the client it launches up to date. It is a launcher, not a second client: it starts the Linux client rather than streaming itself, and holds no settings, no library and no host editor of its own — its **Open Punktfunk** button hands all of that to the client's console home. |
| **Web console** | The full management surface — dashboard and sessions, pairing, library, displays, plugins and the plugin store, logs, stats, settings, and host updates. It cannot yet run a speed test or set a bitrate; the client apps can. |