Compare commits

...
Author SHA1 Message Date
enricobuehlerandClaude Opus 5 362595b20f fix(vdisplay/kwin): the streamed output declares it mirrors nothing, so a stored replicationSource can't clone a panel into the stream
apple / swift (pull_request) Successful in 1m25s
ci / rust-arm64 (pull_request) Successful in 1m27s
apple / screenshots (pull_request) Skipped
android / android (pull_request) Successful in 3m13s
ci / rust (pull_request) Successful in 4m22s
ci / docs-site (pull_request) Failing after 18m2s
ci / web (pull_request) Failing after 18m8s
ci / rust (push) Failing after 3s
ci / rust-arm64 (push) Failing after 5s
deb / build-publish-host (push) Failing after 3s
deb / build-publish-client-arm64 (push) Failing after 3s
apple / swift (push) Successful in 1m22s
ci / docs-site (push) Successful in 1m24s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 6s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 31s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 23s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 4m14s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 4m13s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 4m15s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 2s
docker / builders-arm64cross (push) Successful in 21s
android / android (push) Successful in 6m2s
docker / deploy-docs (push) Successful in 1m14s
apple / screenshots (push) Successful in 6m3s
arch / build-publish (push) Successful in 8m11s
deb / build-publish (push) Successful in 5m4s
ci / web (push) Failing after 10m16s
windows-host / package (push) Successful in 11m15s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 17s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 10m25s
KWin stores output configuration per *setup* — the exact set of connected
outputs, matched by EDID/connector — in `kwinoutputconfig.json`, and
`replicationSource` is one of the fields it saves and restores
(`OutputConfigurationStore::storeConfig` / `setupToConfig`). Our virtual output
carries a STABLE name on purpose, so once any setup has an entry making
`Virtual-punktfunk` a mirror of a physical head, KWin re-applies it to OUR
output on every session that reproduces that same monitor set — and only that
set, which is why the failure looks environment-dependent: a field report has
the stream cloning the panel whenever exactly one monitor is live, and behaving
normally the moment the others come back (a different setup key, a different
stored entry).

A mirroring output is not a desktop. KWin's `applyMirroring` overrides its scale
and render offset to the source's, so the stream carries the physical screen's
viewport at the physical screen's size instead of the mode the client
negotiated. The protocol says the rest out loud on `priority`: "an output may
not be in the output order if it's disabled or mirroring another screen" — so
the primary assertion this module works so hard to verify silently stops meaning
anything too.

Nothing we sent ever contradicted the stored value. The topology config enabled
our output, took priority 1 and disabled the others, but never stated the one
property that decides whether the thing is its own screen. Now it does:
`set_replication_source(ours, "")` rides along in the config we already build
(free, idempotent — an empty source is exactly what KWin resolves to "mirrors
nothing"), gated on management v13 where the request appeared, since wayland-rs
does not range-check requests and an out-of-range opcode would kill the
connection.

`extend`/`auto` issue no topology calls by design — the streamed output is meant
to join the desk without rearranging it — but a mirror is not an arrangement, it
is a broken source under every topology. So they get `clear_replication_source`,
which enumerates and applies ONLY when our output really is mirroring.

The device's `replication_source` event is now read, so the state is visible: a
mirrored streamed output names its source in a warn instead of leaving "the
stream just shows my monitor" as something only the reporter can see.

Verified on 192.168.1.25 (Ubuntu, cargo 1.96): `cargo test -p pf-vdisplay` 128
pass (7 in `kwin_output_mgmt`), `cargo clippy -p pf-vdisplay --all-targets
--locked -D warnings` clean, `scripts/xcheck.sh linux` clean, fmt clean. NOT yet
on-glass — no KDE box here reproduces a stored mirror; the reporter's setup is
the real test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 11:55:34 +02:00
enricobuehlerandClaude Fable 5 caa47e28e6 fix(client/decode): AV1 hardware decode stops silently opening libdav1d
apple / swift (push) Successful in 1m27s
ci / rust (push) Successful in 5m28s
android / android (push) Successful in 5m59s
ci / rust-arm64 (push) Successful in 2m24s
ci / web (push) Successful in 1m55s
arch / build-publish (push) Successful in 9m23s
ci / docs-site (push) Successful in 1m45s
deb / build-publish-client-arm64 (push) Successful in 2m23s
windows-host / package (push) Successful in 12m12s
windows-host / winget-source (push) Skipped
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 37s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 9s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
release / apple (push) Successful in 12m6s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
deb / build-publish (push) Successful in 5m54s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
deb / build-publish-host (push) Successful in 5m19s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 1m0s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m43s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 27s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m44s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
windows-host / canary-manifest (push) Successful in 24s
docker / deploy-docs (push) Successful in 45s
docker / builders-arm64cross (push) Successful in 8s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m7s
apple / screenshots (push) Successful in 5m51s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m17s
flatpak / build-publish (push) Successful in 5m38s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 2m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m7s
avcodec_find_decoder(id) returns the registry's FIRST decoder for the id, and
upstream orders the native av1 decoder LAST on purpose ("hwaccel hooks only,
so prefer external decoders" — allcodecs.c). All three hardware backends
selected by id, so every AV1 session opened libdav1d: a software decoder that
silently ignores hw_device_ctx and never calls get_format. Each frame then
failed the backend's hw-format guard and the session burned the demotion
ladder MID-STREAM — field-logged as 68 Vulkan fails → D3D11VA → 102 fails →
software, ~3 s of black — with "hardware decode active" already printed and
the D3D11 profile/pool probes all green. H.264/HEVC never hit this only
because their native decoders happen to be registered first.

Selection is now by capability: find_hw_decoder walks av_codec_iterate and
takes the first decoder whose avcodec_get_hw_config advertises the backend's
surface via HW_DEVICE_CTX, so a build without a usable hw decoder fails at
OPEN in milliseconds and the ladder runs there — the idiom the D3D11 probes
already follow. Registry order still wins among capable decoders, so
H.264/HEVC select exactly what they always did. The software path keeps the
id lookup on purpose: libdav1d is the fastest CPU AV1, and the native av1
decoder has no software path at all.

Every decode log now carries the selected decoder's name — decoder="av1" vs
decoder="libdav1d" is the whole diagnosis, and no log line said it. The
session log names the WIRE codec and drops the FFmpeg id for PyroWave
(ffmpeg_codec_id's fallthrough claimed codec_id=HEVC for wavelet sessions
that never touch FFmpeg).

The CPU lane also stops passing raw PQ off as a tone-map: software-decoded
frames deliberately never take the HDR10 swapchain, but a PQ stream there was
then shown UNtonemapped (washed out) with no warning — the pq-downgrade warn
keys off the swapchain answer — while the Detailed OSD badge claimed the
"HDR→SDR" tone-map that only the hardware lane's CSC runs. The presenter now
warns once when a PQ CpuFrame arrives, and the badge distinguishes
"HDR→SDR (raw)" (no tone-map pass) from the hardware lane's real "HDR→SDR".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:54:43 +02:00
enricobuehlerandClaude Fable 5 652abeb397 fix(host/audio): an unsatisfiable wiring plan waits for an endpoint change instead of hammering
Field case 2026-08: the display isolate invalidated the only real render
endpoint; the mic held the Steam Streaming Microphone, the Speakers were
blacklisted, and the capture loop re-ran the full wiring pass — three
IPolicyConfig SetDefaultEndpoint writes included — every 2 s for 8+
minutes, retrying a verdict that could never change.

- wiring_plan: a plan with no loopback is a typed structural verdict
  (Wiring::loopback_unsatisfiable + an endpoint-set fingerprint); the
  dead leftover() tier (byte-identical to real_hw()) becomes a real last
  resort that accepts ONLY the Steam Streaming Speakers, flagged
  loopback_last_resort — a known-silent-loopback QUALITY risk, never the
  cable/VoiceMeeter echo CORRECTNESS risks. excluded_from_loopback stays
  untouched (judge_default's mid-stream snap-back semantics).
- wasapi_cap: an unsatisfiable plan errors ONCE per topology with the
  render inventory, per-endpoint rejection reasons and only the remedies
  not already taken, then parks on a cheap enumerate-and-hash poll and
  re-plans the instant the set changes; transient failures get a real
  capped exponential backoff (2 s → 60 s, reset on success or set
  change); a last-resort capture re-plans on any set change and promotes
  the 30 s zero-packet breadcrumb to warn.
- audio_control: the recording default is asserted only when the plan
  changed or the default drifted — set_default_endpoint fires all three
  SetDefaultEndpoint roles unconditionally, so the 2 s loop silently
  stomped any operator recording-device change; the "attach one, or let
  the host install the Steam Streaming pair" warn (already satisfied in
  the field case) is replaced by the same diagnosis.

Verified: 19/19 wiring_plan tests (native rustc --test and the Linux CI
image via docker); both Windows files type-check and clippy clean
against wasapi 0.23.0 / windows 0.62.2 for x86_64-pc-windows-msvc via an
xcheck-style stub harness (the in-tree target check dies in
openh264-sys2's build script on macOS, as scripts/xcheck.sh documents).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:53:36 +02:00
enricobuehlerandClaude Fable 5 48511d1267 fix(abr): probe throughput is measured over the client's receive interval
The capacity probe divided client-side bytes by the HOST's burst
duration — a window wrong on both edges (base snapshotted before the
burst reached the host, frozen only when the ProbeResult landed, while
the host's clock stops the moment ITS send window closes, before the
switch/kernel queue finishes draining toward the client). On a 1 GbE
link a 2 Gbps burst target "measured" 1266 Mbps and set an 886 Mbps
climb ceiling the link could never carry — permanent for the session,
because set_ceiling never lowers.

The reassembler now stamps probe-scoped counters (bytes, packets,
first/last arrival, monotonic ns) at its FLAG_PROBE routing, so video
around the burst contaminates neither numerator nor denominator; the
throughput divisor is the client's first→last arrival interval, with
the host duration kept as the fallback when fewer than two probe
packets arrived. The user-facing speed test shares the corrected
computation (ProbeOutcome/PunktfunkProbeResult layouts unchanged;
elapsed_ms docs updated to the new semantics).

Two guards ride along:
- PUNKTFUNK_ABR_MAX_MBPS clamps inside set_ceiling — the one funnel
  every learned ceiling passes through — so a user cap binds no matter
  what any probe concludes.
- The controller latches decode_cap_kbps when two CONSECUTIVE backoffs
  carry decode-severe evidence (deep decode excursion or jump-to-live
  flush) at a similar pre-backoff rate, mirroring host_cap_kbps for the
  client decoder: a knee below the link ceiling was a permanent 30-60 s
  sawtooth costing a flush + dropped-frame burst per cycle (1440p120
  HEVC field case, knee ~490 Mbps). One spurious flush never latches;
  the cap re-probes on the CAP_REPROBE_WINDOWS clock, so it lifts when
  the decoder recovers.

Also rights the three stale "3 Gbps" probe-clamp comments (the host
constant has been 10 Gbps since MAX_PROBE_KBPS moved).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:53:36 +02:00
enricobuehlerandClaude Fable 5 3e649d372e fix(capture/stall): the ETW witness testifies in QPC, so compose-silence stops convicting content
The stall classifier's present witness never worked: the consumer was opened
without PROCESS_TRACE_MODE_RAW_TIMESTAMP, so ProcessTrace converted every
event's TimeStamp to FILETIME regardless of the session's ClientContext=1 —
FILETIME ticks (100 ns since 1601) are ~4 orders of magnitude above QPC, so
every ts <= to_q comparison was false. summary() always printed etw=none,
window_counts() always returned presents=0/queue_adds=0 while present_history
was still true (satisfied by the unfiltered ring), and classify() therefore
convicted EVERY compose-silence hole as CONTENT-SILENCE; FRAME-GENERATION —
the class the program exists to catch — was unreachable. Two comments
asserted the wrong contract ("TimeStamp IS a QPC value"); both now state the
real one: ClientContext selects the session clock, RAW_TIMESTAMP is what
stops the FILETIME conversion on delivery.

Three adjacent defects fixed with it:

- summary() and window_counts() each took their own ring lock and their own
  (Instant::now(), qpc_now()) anchor, with OpenProcess syscalls between the
  two calls — the prose and the verdict could disagree about the same hole.
  Merged into window_report(): one snapshot, one anchor, both halves; the
  summary keeps its 300 ms lead-in, the counts keep the gap-only window, and
  the etw=/etw_presents=/etw_queue_adds= log fields are unchanged.

- present_history/queue_history meant "an event EVER sat in the ring" —
  satisfied by events arriving after the hole, or by a previous session's
  leftovers in the never-cleared static RING. Both flags now mean witness
  LIVENESS: at least one event inside a 5 s LOOKBACK ending at the hole's
  start, i.e. the witness demonstrably worked before the hole opened. The
  ring is cleared when a new session starts, so a dead session's events can
  never pose as the next one's history.

- window_counts() accepted only BltQueueAddEntry (1071) as queue history
  while summary() also took BltQueueCompleteIndirectPresent (1068); either
  proves the queue witness works, so the merged reader takes both.

The windowing math is factored into a pure count_window() (plain i64 tick
arithmetic) with unit tests, and the classify() matrix gains the live-witness
zero-presents case. Conviction thresholds are untouched.

Verified: scripts/xcheck.sh windows clippy (-D warnings, --all-targets) green
for pf-frame/pf-win-display/pf-capture/pf-vdisplay; native cargo check green.
The new Windows-gated tests type-check but need a Windows box to run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:53:36 +02:00
enricobuehlerandClaude Fable 5 0d004c4680 fix(host/handshake): the 4:4:4 gate names the encoder backend, not the capturer
capture_supports_444 was an encoder-backend fact (direct NVENC or PyroWave)
logged under a capture-ish name — a field report burned real time hunting a
capture problem because of it. The 'encode chroma' line now says
ingest_chain_supports_444, a requested-but-declined session logs WHICH gate
lost, and the console UI's Full chroma explainer names the real requirement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:53:29 +02:00
enricobuehlerandClaude Fable 5 8d7e273a96 feat(client/present): PUNKTFUNK_PRESENT_MODE gains explicit mailbox and fifo_relaxed arms, and the docs stop guessing
The env knob silently folded 'mailbox' and every typo into the default arm,
FIFO_RELAXED was not reachable at all, and clients/session/README.md claimed
the default is FIFO (it is MAILBOX with a FIFO fallback). An AMD-on-Windows
client always lands on FIFO because that driver offers no MAILBOX — now
documented at the picker and in the docs-site client table, next to the ABR
probe/ceiling knobs a field report went looking for and couldn't find.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 09:53:28 +02:00
enricobuehlerandClaude Opus 5 e726542f96 docs(winget): the vhost belongs in unom/infra, not on the box
ci / rust-arm64 (push) Successful in 1m23s
ci / web (push) Successful in 1m42s
ci / rust (push) Successful in 4m45s
ci / docs-site (push) Successful in 1m30s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 13s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 9s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 6s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
docker / builders-arm64cross (push) Successful in 6s
docker / deploy-docs (push) Successful in 28s
Step 2 told you to add the Caddy vhost by hand on unom-1. That instruction is
what broke the source: ~/caddy/Caddyfile is a copy that deploy-all.sh rsyncs
over from unom/infra, with no .git there to warn you, so the hand-added block
survived until the 2026-07-31 hardening commit rewrote the file from the repo's
own copy and deleted it.

Point step 2 at unom/infra and record how the failure presents, since it does
not look like an ingress problem from the client side: no vhost means no
certificate for that SNI, so Caddy answers with TLS internal_error (alert 80)
before sending one, and winget surfaces that as
WINHTTP_CALLBACK_STATUS_FLAG_SECURITY_CHANNEL_ERROR / 0x8a15003b.

Also note that port 80 is useless for diagnosing it — Caddy 308s every Host to
https including names it has never heard of — and give the SNI probe that does
work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 10:43:39 +02:00
enricobuehlerandClaude Opus 5 e0427a3bb6 feat(ci/android): a release tag without Play notes fails before it builds
ci / rust (push) Canceled after 5m55s
ci / rust-arm64 (push) Canceled after 0s
ci / web (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
android / android (push) Successful in 6m28s
Play does not show an empty "What's new" when the file is missing — it carries
the PREVIOUS release's text onto the new version. So the store listing ends up
describing a build nobody is getting, and nothing surfaces it except reading
the listing. That is the shape of the v0.22.3 notes announcing a feature the
tag never contained, and a soft warning in a log nobody reads does not prevent
it.

The gate runs FIRST in the job, before the ten-minute build: a miss costs a
second and leaves nothing half-published — no build, no assets on the Gitea
release, nothing on Play. It rejects three things: a missing file, a file
byte-identical to another release's (the same bug reached by copy-paste rather
than omission), and an empty or over-500-char one.

Length is checked here as well as in play-upload.py on purpose. The uploader
stays the last line of defence and is the only check android-promote.yml gets,
but it runs at step 9; this catches an unedited TEMPLATE copy at step 1. It
counts CHARACTERS, not bytes — Play's cap is 500 chars and `•` is three bytes
in UTF-8, so a `wc -c` check would have called the 356-char v0.23.0 notes 365
and can reject a legal file.

whatsnew/TEMPLATE.txt gives the file a starting point and says what the gate
does and does not enforce: it cannot tell whether the prose was ever edited, so
a copy that still reads "<The headline change>" ships exactly as written.

Canary stays exempt — no curated notes, and Play reusing text for internal
testers costs nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 10:34:32 +02:00
enricobuehlerandClaude Fable 5 02a5bdb965 fix(gamepad): the virtual DualSense stops demanding a firmware update it cannot take
windows-drivers / probe-and-proto (push) Successful in 53s
apple / swift (push) Successful in 1m25s
ci / rust (push) Failing after 2m11s
windows-drivers / driver-build (push) Successful in 1m44s
ci / rust-arm64 (push) Successful in 1m38s
ci / web (push) Successful in 1m4s
android / android (push) Successful in 5m58s
ci / docs-site (push) Successful in 1m11s
apple / screenshots (push) Successful in 6m6s
arch / build-publish (push) Successful in 9m7s
deb / build-publish-client-arm64 (push) Successful in 1m3s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 15s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 4s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
deb / build-publish (push) Successful in 4m57s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
deb / build-publish-host (push) Successful in 4m52s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / builders-arm64cross (push) Successful in 8s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 18s
docker / deploy-docs (push) Successful in 26s
windows-host / package (push) Successful in 11m30s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 14s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m35s
The emulated pad's firmware-info feature report (0x20) advertised update
version 0x0154 — a 2021-era number. PlayStation Accessories compares it
against Sony's latest (0x0630 as of 2026-08) and offers an Update that can
only end in "can't complete the update", since the virtual pad speaks no DFU;
libScePad titles (Stellar Blade) surface the same nag in-game. A real pad
plugged in directly reads up to date, which made the prompt look like
punktfunk corrupting the controller.

The old value was chosen to keep the kernel and SDL on the flag0
COMPATIBLE_VIBRATION convention, but parse_ds_output has since learned the
firmware-≥2.24 COMPATIBLE_VIBRATION2 flag as well, so nothing depends on
looking old anymore. Advertise 0x0999 — above anything Sony has shipped and
comfortably ahead of their ~yearly cadence — instead of chasing their exact
latest, which would resurrect the prompt on every Sony release. Writers that
read the version now use the v2 flag; both conventions land in the same
rumble plane. Bumped in both copies of the blob (host uhid + Windows driver);
the DualSense Edge shares them, and its own versioning (0x0217 latest) sits
below the new value too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 10:24:20 +02:00
enricobuehlerandClaude Opus 5 09b9ee8f53 feat(ci/android): a release tag publishes to Play production, not alpha
ci / rust (push) Successful in 5m15s
ci / rust-arm64 (push) Successful in 2m3s
ci / web (push) Successful in 1m15s
ci / docs-site (push) Successful in 1m43s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 8s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7s
android / android (push) Successful in 5m55s
docker / builders-arm64cross (push) Successful in 13s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 51s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m10s
docker / deploy-docs (push) Successful in 36s
Production access came through on 2026-08-01. Until now a `vX.Y.Z` tag could
only reach `alpha` and someone had to promote it by hand in the Console; it now
goes to `production` at 100% (`completed`). Canary is unchanged on `internal`,
and its run-number versionCodes always outrank production, so testers keep
getting the newer build.

A tag therefore reaches real users with no further click. What keeps that
honest: the tag is only pushed once every platform is green, and Play reviews
each production release before it ships. Ramping instead is `--status
inProgress --user-fraction 0.2` on the upload step.

Play's "What's new" gets its own file, docs/releases/whatsnew/vX.Y.Z.txt — the
vX.Y.Z.md body is ~34 KB against a 500-char cap, so it cannot be reused. Only
tags have one; canary is a moving target and Play carrying the previous text
over is fine for internal testers. Same freeze rule as the notes: once the tag
exists, the file describes what that versionCode shipped.

android-promote.yml is the lever for everything that is not a fresh tag —
promote a tested build, halt a rollout, or roll production back onto an older
versionCode. It is separate from android.yml because promotion must not
rebuild, and an `if:` on all ten build steps is worse than one small workflow.
dry_run defaults to true, so a mis-typed versionCode validates and deletes the
edit instead of publishing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 09:29:20 +02:00
enricobuehlerandClaude Opus 5 43e3c7b69f feat(ci/android): play-upload can attach release notes and promote a build
Two things it could not do, both needed now that a tag ships to production.

Release notes: it never sent `releaseNotes`, so Play's "What's new" was whatever
the previous release said. It now takes --release-notes-file, and refuses text
over Play's 500-char-per-language cap with the actual count — that check has to
happen before the upload, because the API only rejects it at commit, by which
point the AAB is already on Play.

Promotion: --promote assigns a versionCode that is already on Play instead of
uploading, so what reaches production is the byte-identical artifact the testers
ran. Rebuilding would mint a fresh versionCode from possibly-newer sources and
ship something nobody tested. --promote-from asserts the code really is on that
track (a typo'd versionCode now fails before it touches production) and clears
that track in the SAME edit, so the build is never active on both at once.

--user-fraction comes along because --status inProgress is an API error without
it; it is validated as strictly between 0 and 1 rather than left to Google.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 09:29:04 +02:00
enricobuehler ea469162f9 fix(docs): three doc comments start a markdown list they never meant to
apple / swift (push) Successful in 1m14s
ci / rust-arm64 (push) Successful in 3m31s
ci / web (push) Successful in 1m27s
ci / docs-site (push) Successful in 1m53s
apple / screenshots (push) Successful in 5m57s
ci / rust (push) Successful in 8m9s
android-screenshots / screenshots (push) Successful in 1m54s
android / android (push) Successful in 6m53s
deb / build-publish-host (push) Successful in 5m17s
decky / build-publish (push) Successful in 42s
deb / build-publish-client-arm64 (push) Successful in 1m33s
release / apple (push) Successful in 9m50s
arch / build-publish (push) Successful in 11m34s
deb / build-publish (push) Successful in 11m20s
sbom / sbom (push) Successful in 24s
linux-client-screenshots / screenshots (push) Successful in 5m18s
docker / builders-arm64cross (push) Successful in 7s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 12s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 10s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 6s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m42s
flatpak / build-publish (push) Successful in 6m44s
docker / deploy-docs (push) Successful in 1m25s
web-screenshots / screenshots (push) Successful in 3m57s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 3m6s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 4m43s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m2s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m51s
windows-host / package (push) Successful in 11m36s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Successful in 21s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m38s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m1s
Windows clippy on the v0.23.0 tag: `doc_lazy_continuation` in
crates/pf-client-core/src/audio_wasapi.rs:37. The cause is one line break —
`+ wire cost.` begins a line, so the markdown parser reads `+` as a bullet
marker and the following line becomes a lazy continuation of that list item.

Fixed by reflowing so the `+` is mid-line rather than by taking clippy's
suggested indent: indenting would keep the accidental bullet in the rendered
docs, which is the actual defect. Same treatment for the two siblings a sweep
of every `///` line found, both invisible to the Linux gate for their own
reasons:

- gamestream/audio.rs:237 — `+ libopus;` at line start, on the
  cfg(not(linux/windows)) stub, so only a macOS clippy would ever see it.
- mgmt/tests.rs:1653 — `404.` at line start IS an ordered-list marker
  (CommonMark: 1-9 digits + `.`), and it is behind cfg(test), so only an
  --all-targets run sees it.

This is the [[Windows clippy sees what the Linux gate structurally cannot]]
shape again: audio_wasapi.rs is cfg(windows), so no amount of Linux CI would
have caught it.

Verified: a scanner over every .rs doc comment in the tree now reports zero
line-initial list markers with an unindented continuation; rustfmt clean (it
does not reflow doc comments, so these edits are stable).
2026-08-01 01:30:45 +02:00
enricobuehler 213b353dad docs(release): the 0.23.0 notes link to the docs site that exists
ci / rust (push) Canceled after 1m18s
ci / rust-arm64 (push) Canceled after 0s
ci / web (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
Both links pointed at https://punktfunk.unom.io/docs/... — the marketing host,
which 404s (verified live; the docs site serves from docs.punktfunk.unom.io, as
README.md has used throughout). The Updating link was the one a reader following
the one-click update section would actually click.

The echo link additionally pointed at the docs ROOT rather than the page it
names; both now resolve to their real slugs, confirmed against
docs-site/content/docs/{echo,updating}.md and by fetching them (200/200, vs 404
for the old host).

Release body re-synced by body-only PATCH.
2026-08-01 01:25:01 +02:00
enricobuehler 23ec0822d8 docs(release): the 0.23.0 notes stop calling gamescope HDR a handheld feature
ci / web (push) Canceled after 0s
ci / rust (push) Canceled after 0s
ci / rust-arm64 (push) Canceled after 0s
ci / docs-site (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
Two wording fixes to the shipped v0.23.0 notes, which docs/releases/README.md
explicitly allows after the tag — the file stays authoritative and the announce
step re-syncs from it.

"HDR turns itself on for Steam Deck and other gamescope handhelds" framed a
LINUX HOST change as a device one. What 19392918 actually did is default
PUNKTFUNK_GAMESCOPE_HDR on and get the patched gamescope onto the Linux install
routes (Bazzite + Arch sysext, the nix module, the Deck's on-device build
script) — which covers any host running its games through gamescope: a Bazzite
or SteamOS box in Game Mode, an HTPC, a desktop, not only a handheld. The
lead-in carried the same framing and mattered more, since everything above the
first `##` is what the Discord embed shows.

Also rewords the mic-mute lead-in ("stop the room being heard" read oddly).

No claim changes: same features, same scope, same release. The live release body
needs a re-sync — done via a body-only PATCH rather than an announce dispatch,
since announcing also posts to Discord and publishes the stable update manifest,
neither of which should fire before the fleet is green.
2026-08-01 01:21:54 +02:00
enricobuehler 49bbdcf4ef chore(release): bump workspace version to 0.23.0
audit / bun-audit (plugin-kit) (push) Successful in 30s
audit / bun-audit (sdk) (push) Successful in 31s
audit / bun-audit (web) (push) Successful in 33s
audit / pnpm-audit (push) Successful in 28s
audit / cargo-audit (push) Successful in 1m17s
audit / docs-site-audit (push) Successful in 1m2s
ci / web (push) Successful in 1m46s
ci / docs-site (push) Successful in 1m52s
apple / swift (push) Successful in 1m34s
audit / license-gate (push) Successful in 5m54s
ci / rust-arm64 (push) Successful in 6m26s
android-screenshots / screenshots (push) Successful in 3m2s
android / android (push) Successful in 4m59s
ci / rust (push) Canceled after 18m52s
decky / build-publish (push) Successful in 29s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 6s
deb / build-publish (push) Successful in 5m57s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 6s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 17s
deb / build-publish-host (push) Successful in 4m26s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
deb / build-publish-client-arm64 (push) Successful in 2m7s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 41s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Failing after 13s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Failing after 12s
windows / build (aarch64-pc-windows-msvc) (push) Failing after 11s
release / apple (push) Successful in 10m31s
windows-host / winget-source (push) Skipped
sbom / sbom (push) Successful in 19s
linux-client-screenshots / screenshots (push) Successful in 3m59s
arch / build-publish (push) Failing after 44s
windows-host / canary-manifest (push) Successful in 20s
windows-host / package (push) Successful in 15m12s
docker / builders-arm64cross (push) Successful in 13s
docker / deploy-docs (push) Successful in 34s
apple / screenshots (push) Canceled after 4m13s
web-screenshots / screenshots (push) Successful in 4m56s
flatpak / build-publish (push) Canceled after 11m15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 11m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 11m11s
A minor bump: 133 commits since v0.22.3 across 400-odd files. The wire grew two
negotiated abilities (slice-streamed access units, phase-locked capture); the
Android presenter was rebuilt; the microphone path was rebuilt end to end on
every client; the web console moved from polling to the host's event stream;
gamescope HDR is on by default; HDR and 4:4:4 stopped being mutually exclusive
on Windows; and the one-click update apply that missed the 0.22.3 cut ships here.
The canary base is already 0.23 — release.yml derives it as one minor ahead of
the latest stable tag — so this is the version the canary channel has been
publishing against all along.

Lock touched for the 32 workspace members only, via `cargo update --workspace`
rather than a sed: `wasapi` is itself at 0.23.0 and `rustls` at 0.23.41, so the
version space we are moving into is occupied by third-party crates this time.
Diff against origin/main is versions-only, 32 insertions and 32 deletions;
`cargo metadata --locked` resolves; `cargo fmt --all --check` clean in both the
main and the packaging/windows/drivers workspaces.

Notes at docs/releases/v0.23.0.md, per docs/releases/README.md — authored with
the bump so CI's ensure_release seeds the body at tag creation.
2026-08-01 01:01:16 +02:00
enricobuehler 3c509d48c9 feat(core/abi): report_phase earns its version — C ABI 13 -> 14
`punktfunk_connection_report_phase` (fa822744, coherence tail 1d31e4c5) and the
`PUNKTFUNK_CLIENT_CAP_PHASE_LOCK` mirror const (7cf71dd2) grew the embeddable C
surface without moving ABI_VERSION. Every prior additive entry in that doc list
bumped it — v3's wake_on_lan, v5's next_rumble2, v8's clipboard block, v13's
send_pen — precisely so an embedder can ask punktfunk_abi_version() whether the
function it wants to link is there. Left at 13, the one number that answers that
question said "no report_phase" about a core that has one.

The header was already regenerated with both symbols, so it carried the new
surface under the old number; the regen here changes exactly the #define and its
doc block and nothing else, which also confirms the committed header was
otherwise current.

Additive and capability-gated: the host arms on report receipt, the wire grows
only PhaseReport (0x32) — a control message an old host never reads — and a
strict-prefix append on the 0xCF host-timing tail, so WIRE_VERSION stays 2. No
in-tree caller compares ABI_VERSION against a literal; mgmt/tests.rs asserts
against the symbol.

Verified: cargo build -p punktfunk-core regenerates include/punktfunk_core.h to
exactly this diff; punktfunk-core lib suite 134/134; rustfmt clean. The C ABI
harness cannot run on this Mac (`ld: library 'opus' not found`, the documented
pre-existing local linker gap) — CI's Linux leg is the gate for it.
2026-08-01 00:58:11 +02:00
enricobuehler b8d987b145 docs(release): the v0.22.3 notes stop claiming one-click updating it never shipped
The v0.22.3 tag is `1c836afc`, cut 14:36. The one-click apply work landed on
main between 15:01 and 16:28, and this file was then edited at 17:23 (5790a3e3)
to announce it — three "New"/"Under the hood" claims about a build that does not
contain them. The live release body is still the pre-edit text, so nothing wrong
has been published yet; but `announce.yml` re-asserts this file over the release
on every announce, and 0.22.3 has not been announced. Announcing it would have
published the false version and put its lead-in ("can install it for you where
the platform allows") into the Discord embed.

Verified against the tag rather than the commit graph: `update.available` is in
`1c836afc`, `update.applied` is not, and `mgmt/tests.rs` there literally probes
`/api/v1/update/apply-does-not-exist-yet` while `auth.rs` carries the comment
"today it is only a check". The Updates card itself (b275e6d3, cc015626) IS in
the tag, so that bullet stays; only the apply half goes.

The removed material is not lost — it is in docs/releases/v0.23.0.md, which is
the release that actually ships it.
2026-08-01 00:58:11 +02:00
enricobuehler 2d3f9f8690 Merge remote-tracking branch 'origin/main' into audio/mic-latency-echo
ci / docs-site (push) Successful in 1m28s
ci / web (push) Successful in 2m36s
ci / rust (push) Failing after 2m43s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 8s
deb / build-publish (push) Successful in 3m30s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 56s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 15s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
ci / rust-arm64 (push) Successful in 4m10s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m35s
deb / build-publish-client-arm64 (push) Successful in 3m38s
docker / builders-arm64cross (push) Successful in 5s
docker / deploy-docs (push) Successful in 30s
arch / build-publish (push) Successful in 7m0s
android / android (push) Canceled after 7m27s
apple / swift (push) Canceled after 0s
apple / screenshots (push) Canceled after 0s
deb / build-publish-host (push) Canceled after 5m44s
release / apple (push) Canceled after 0s
flatpak / build-publish (push) Canceled after 3m12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 3m11s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 1m40s
windows-host / package (push) Canceled after 7m13s
windows-host / canary-manifest (push) Canceled after 0s
windows-host / winget-source (push) Canceled after 0s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 0s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
2026-08-01 00:54:20 +02:00
enricobuehler 951bcec650 Merge remote-tracking branch 'origin/main' into audio/mic-latency-echo 2026-08-01 00:47:50 +02:00
enricobuehlerandClaude Fable 5 badda070ef docs(android): the stats-array KDoc counts the doubles it actually returns
ci / rust-arm64 (push) Successful in 1m25s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 8s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 17s
ci / web (push) Successful in 2m12s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 17s
ci / docs-site (push) Successful in 2m1s
docker / builders-arm64cross (push) Successful in 8s
docker / deploy-docs (push) Successful in 31s
android / android (push) Canceled after 6m2s
ci / rust (push) Canceled after 5m55s
nativeVideoStats grew to 33 with the decode split and the overflow counter, but
its own KDoc still promised 30 and StatsOverlay still said 26 — a count that
was already two extensions stale before this one. Both now list the full index
set, with the JNI KDoc named as the authoritative one so the next extension has
a single place to update.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:47:04 +02:00
enricobuehlerandClaude Fable 5 46bcfc3041 fix(scripts/windows): the installer-run scripts go back to pure ASCII
CI's guard fired on build-web.ps1: an em-dash in the header comment. The
rule exists because PowerShell 5.1 mis-parses non-UTF-8-locale files, and
the check covers every script the installer can run, comments included.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:41:07 +02:00
enricobuehlerandClaude Fable 5 f3a39df7b3 test(host/audio): prove a reopen recovered with a live uplink, not one frame
`reopens_after_push_death` failed about one run in nine, and widening its
timeout did not help — the earlier commit blamed the backoff and was wrong.

The pump drops whatever queued while it was down: audio from before the
device came back is stale, so a fresh instance drains the channel right
after opening. The harness counts `opens` from the START of the open, so
the moment the test sees the counter move, the pump has not reached that
drain yet. The single frame it then sent landed inside the drain window and
was discarded exactly as designed, leaving the test waiting for audio that
was never going to arrive.

So the test now keeps feeding, which is what a real uplink does and what
the drain assumes. The sequence advances each time or the de-jitter reads
the repeats as duplicates and drops them for a second, correct reason.

Production behaviour is unchanged: this was the test asserting something
the pump never promised.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:41:07 +02:00
enricobuehlerandClaude Fable 5 f9c56eaf5c feat(android): Automatic prefers AV1 where the silicon says it should
ci / docs-site (push) Successful in 2m12s
ci / web (push) Successful in 2m36s
ci / rust-arm64 (push) Successful in 3m20s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 8s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 6s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 6s
android / android (push) Successful in 6m13s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 6s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 42s
ci / rust (push) Canceled after 6m49s
docker / builders-arm64cross (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
deb / build-publish-client-arm64 (push) Successful in 3m42s
arch / build-publish (push) Successful in 7m46s
deb / build-publish (push) Successful in 5m58s
deb / build-publish-host (push) Successful in 5m58s
apple / swift (push) Successful in 6m9s
apple / screenshots (push) Canceled after 3m57s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 6m53s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 6m21s
windows-host / package (push) Canceled after 0s
windows-host / canary-manifest (push) Canceled after 0s
windows-host / winget-source (push) Canceled after 0s
The P3 format A/B (NP3 ↔ RTX 4090, identical conditions) measured AV1 ~1.2 ms
faster end-to-end than HEVC with slightly better codec-pure decode time. Under
"Automatic" the client now sends AV1 as its soft preference when this device
hardware-decodes it (the advertised AV1 bit is already gated on a real,
non-blocked hardware decoder) AND it lacks FEATURE_PartialFrame — a
partial-frame device keeps HEVC, whose slice-progressive overlap AV1 cannot
ride (no slices, the chunked poll never arms). The host honors the preference
only inside its probed shared codec set, so an AV1-less encoder still resolves
HEVC, and an explicit user choice wins unchanged. The codec picker caption
mirrors the same rule so "Automatic" says what it does on this device.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:40:58 +02:00
enricobuehlerandClaude Fable 5 b69ef02f4d fix(android): a cold-start connect no longer loses HDR or the native mode
A punktfunk:// deep link can reach the connect before the activity is attached
to its display; context.display then throws and the display probes silently
fell to their worst answers — displaySupportsHdr advertised SDR (the whole
session pinned to 8-bit BT.709) and nativeDisplayMode fell back to 1080p60.
Seen live on the NP3: one cold connect advertised hdr=false, the warm retry
true, nothing in the log either way.

Both probes now share probeDisplay: the context display when attached, else
DisplayManager DEFAULT_DISPLAY — which IS the panel on phones and TVs; the
activity-display distinction only matters on multi-display setups, where the
attached path still wins whenever available. Each fallback leg logs itself, so
a downgraded session can never again be silent about why.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:40:58 +02:00
enricobuehlerandClaude Fable 5 4d45a96ff9 feat(encode/windows): sub-frame readback defaults on where the GPU supports it
Linux parity, validated by the .173 on-glass A/B (no regression; the win goes
to clients that actually consume slice-progressive parts): the caps probe now
reads NV_ENC_CAPS_SUPPORT_SUBFRAME_READBACK and seeds resolve_subframe with it
instead of a hard false, so PUNKTFUNK_NVENC_SUBFRAME becomes the tri-state
escape it already is on Linux, and the split×sub-frame arbitration hears the
real forced flag for its log severity.

The A/B also caught the default path opening every session with a WARN: the
submit-time idr_hint missed that NVENC emits the session-opening frame as an
IDR regardless of pic flags, so frame 1's early chunks went out unflagged and
the divergence check fired at every start. The hint now carries the Linux
twin's `opening` term.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:40:58 +02:00
enricobuehlerandClaude Fable 5 849baea881 feat(android): the stream re-votes its refresh rate and touches keep their curvature
surfaceChanged re-asserts the frame-rate vote (FIXED_SOURCE; ALWAYS only on the
TV low-latency path, mirroring the native hint) — a buffer-geometry change on
some OEM builds silently drops the 120 Hz pin mid-stream. Touch passthrough and
direct-pointer moves forward the MotionEvent historical samples before the
current point, so a fast swipe lands with its real shape; the trackpad path
keeps summed deltas on purpose — its acceleration curve is tuned for per-frame
dt and historicals would change the feel, not the sum.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:40:58 +02:00
enricobuehlerandClaude Fable 5 c767a904d2 feat(android): the decode stage answers where its time goes, HUD on or off
P3 decode science: every AU is stamped as its last piece enters the codec, so
the decode stage splits into feed (received→queued: hand-off + input-slot wait)
and codec (queued→decoded: the decoder alone — a slice head start would show
here). The split + an always-on capture→decoded e2e ride the 1 Hz pf-present
line, so a wireless HUD-off A/B reads everything from logcat; the HUD equation
gains the split (indices 30/31), the skipped counter tells benign newest-wins
pacing from parked-AU overflow (32), and a −2-refresh Apple-HUD-equivalent twin
makes iPhone comparisons honest (Apple shaves its OS floor; Android shows raw).

Connect now logs the per-mime decoder picks + FEATURE_PartialFrame verdicts
(tag pf.caps) — on the NP3 all three c2.qti low-latency decoders say no, so
parts delivery never arms and P2d is inert there; a debug.punktfunk.force_parts
sysprop overrides the probe for the on-glass question the API cannot answer.
Forced on glass: c2.qti accepts PARTIAL_FRAME pieces without erroring but only
assembles them — codec time unchanged, so the overlap is dead on SM8735 either
way.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:40:58 +02:00
enricobuehlerandClaude Opus 5 0985726415 fix(host,web): an empty update channel stops looking like a broken host
android / android (push) Canceled after 36s
apple / swift (push) Canceled after 0s
apple / screenshots (push) Canceled after 0s
arch / build-publish (push) Canceled after 37s
ci / rust (push) Canceled after 37s
ci / rust-arm64 (push) Canceled after 36s
ci / web (push) Canceled after 34s
ci / docs-site (push) Canceled after 34s
deb / build-publish (push) Canceled after 19s
deb / build-publish-host (push) Canceled after 0s
deb / build-publish-client-arm64 (push) Canceled after 0s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Canceled after 0s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Canceled after 0s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Canceled after 0s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Canceled after 0s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Canceled after 0s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Canceled after 0s
docker / builders-arm64cross (push) Canceled after 0s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Canceled after 0s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 0s
docker / deploy-docs (push) Canceled after 0s
windows-host / canary-manifest (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 0s
windows-host / package (push) Canceled after 33s
windows-host / winget-source (push) Canceled after 0s
flatpak / build-publish (push) Successful in 5m43s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m59s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m9s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 2m19s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
A channel nobody has published to answers `manifest.json` with a 404, and the
check reported that the same way it reports a dead registry or a bad signature:
"Last check failed: feed returned HTTP 404". Every host on the stable channel
shows it today, because the stable manifest only publishes when someone
dispatches `announce` for a release tag — so the first thing an operator sees
from the new Updates card is a red failure caused by nothing being wrong.

The shared checker now distinguishes the two. `feed::fetch_manifest_blocking`
returns a typed `FeedError` instead of a string, and only a 404 on the manifest
ITSELF becomes `NotPublished` — a 404 on the detached signature still fails
loudly, because that is the half-published pair the manifest-then-signature
upload order can produce, and it must stay fail-closed.

The host carries that through as `UpdateStatus.not_published`, mutually
exclusive with `last_error`. It is benign only while no manifest has ever been
seen for the channel: once a check has succeeded, the same 404 means the feed
LOST a document it used to serve, which stays an error. The console then shows a
plain sentence naming the channel instead of the failure banner, and "None
published yet" rather than "Not checked yet".

The Linux client makes the same distinction but deliberately NOT the same
choice: `--check-update` keeps exiting 1 and keeps `error` set, because its
consumer is a shell script and an empty channel is the absence of evidence that
this build is current — not a confirmation that it is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 00:37:56 +02:00
enricobuehler 23f1debe69 Merge remote-tracking branch 'origin/main' into audio/mic-latency-echo 2026-08-01 00:21:24 +02:00
enricobuehlerandClaude Fable 5 10a1863cc7 test(host/audio): give the reopen tests longer than the backoff they wait for
`wait_until` allowed 200 × 10 ms — exactly the 2 s `backoff_start` the reopen
path spends before it can succeed. On a warm, idle machine it wins the race;
on a cold binary or a loaded box it does not, and `reopens_after_push_death`
failed 3 of ~5 cold runs while gating this branch. CI is always cold.

Six seconds costs nothing when the test passes and only delays a genuine
failure, so the budget now clears the backoff with room to spare.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:15:54 +02:00
enricobuehlerandClaude Fable 5 a7143a6510 Merge origin/main into audio/mic-latency-echo
Main moved through the same surfaces while the audio work was in flight, so
three files needed hand-resolution:

- clients/windows/src/app/settings.rs — main gave Windows its speaker and
  microphone endpoint pickers, the gap this branch could only report. Both
  keep their rows: the pickers, then Echo cancellation, and the microphone
  description keeps the sentence naming the mute chord.
- crates/pf-console-ui/src/screens/settings.rs — both sides grew the couch
  row list. Main's seven new rows and Echo cancellation are all reachable
  in Gaming Mode; the count is 22 and the rationale comment names echo
  cancellation among the fields that would otherwise be unreachable there.
- crates/pf-presenter/src/run.rs — both sides added a stats test at the same
  line. Both are kept.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 00:04:34 +02:00
enricobuehlerandClaude Fable 5 8e877ad25f fix(android): a mute is a pause, not a hole the host tries to conceal
The Android uplink kept advancing `seq` while muted, so the first frame
after an unmute looked to the host like loss the width of the mute. The
de-jitter reads that as a gap: up to five concealment frames of stale
voice, and a seq gap counted in the uplink-health line. Past 600 ms the
pump's stale flush resets the chain first and hides it, which is why the
usual long mute looks fine — a quick toggle does not.

Freeze `seq` while muted, as the desktop uplink already does, so the
frame after an unmute continues the chain. `reset_stream` says it
plainly: a pause is not loss, and must not conceal or count a gap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 23:15:03 +02:00
enricobuehler 0c4a543831 Merge branch 'audio/desktop-mute-and-aec-toggle' into audio/mic-latency-echo 2026-07-31 23:14:09 +02:00
enricobuehler 4c98e2f788 Merge branch 'audio/android-mic-mute' into audio/mic-latency-echo 2026-07-31 23:14:09 +02:00
enricobuehler 6be8e5d6fa Merge branch 'audio/apple-mic-mute' into audio/mic-latency-echo 2026-07-31 23:14:09 +02:00
enricobuehlerandClaude Fable 5 7ec3107b4a feat(client): the mic has an off switch, and echo cancellation has a switch too
Ctrl+Alt+Shift+V mutes and unmutes the microphone mid-stream — V for
voice, since M and S were taken. The uplink keeps running while muted:
`MicStreamer::spawn` takes a shared AtomicBool the capture callback reads
every quantum, and a muted callback drains whole frames and sends
nothing. Stopping the stream instead would have re-primed the device
buffers and, on Linux, re-run source selection on every unmute — a
second of glitch for a key people press mid-sentence. The sequence
counter deliberately does NOT advance while muted, so the host sees one
continuous sequence with a pause rather than a gap the size of the mute,
which its de-jitter would try to conceal frame by frame (its 600 ms
stale-flush covers the rest).

The mute lives on SessionHandle as a MicControl with two flags, not one:
`live` is raised by the pump only once the uplink is actually running, so
a session with the mic off in Settings — or whose capture device wouldn't
open — reports "nothing to mute", the chord says so in the log, and no
indicator appears. Per session, never persisted.

Muted state draws as a persistent "Microphone muted" badge in the stream's
top-right corner, off `FrameCtx::mic_muted` rather than the stats text: it
has to be there with the stats overlay Off, which is where most people
leave it. The Detailed mic line still reads throughput, so it simply falls
to zero — the badge is what answers "am I muted".

Echo cancellation stops being an env-only lever. `Settings::echo_cancel`
(default on, `#[serde(default)]` so every stored file loads with it on)
now gates the same hooks PUNKTFUNK_NO_AEC gated: the echo-cancelled
PipeWire source preference and WASAPI's Communications stream category.
The env var still wins, one-way — it can only turn AEC off, never back on
— and both `aec_enabled` helpers say so. The row ships in the GTK, WinUI
and console settings, under the microphone toggle and greyed out while it
is off, matching what Apple and Android shipped in wave 1.

SettingsOverlay grows `echo_cancel` as a first-class field — apply,
absorb, clear, is_empty — instead of riding the `extra` passthrough, where
`clear_override("echo_cancel")` answered false. The JSON key is the one
Apple and Android already write, so one catalog round-trips through all
three.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 23:07:30 +02:00
enricobuehlerandClaude Fable 5 253e0bbe7c feat(android): the mic has an off switch you can reach mid-stream
Wave 1 gave the Android client a mic worth using. It gave it no way to stop
talking: leaving the stream, or digging through Settings to turn the whole
feature off, were the only ways to stop the room being heard. Now a tap (or
Select + Y on a pad) mutes it, and the screen says so while it lasts.

How muting gates the capture, and why that way. The AAudio input stream is
never stopped: a stop/start would re-run the input-preset fallback ladder and
re-prime the buffers on every toggle — hundreds of milliseconds, and possibly
a landing on a different rung, silently losing the HAL echo canceller wave 1
went to some trouble to get. Instead the encode loop reads an AtomicBool per
10 ms frame and, while it is set, drains the frame out of its ring and drops
it there — the last point before it would have become an Opus packet. Nothing
is encoded, nothing is sent, and the realtime capture callback is untouched,
so its allocation-free discipline and the queue policy stay exactly as wave 1
verified them. A toggle costs one atomic store and takes effect on the next
10 ms boundary.

The frame counter keeps advancing across a mute, because it numbers the
captured 10 ms TIMELINE rather than the datagrams. The gap the host then sees
is exactly the audio that never came: its de-jitter conceals at most a few
frames of it before the pump's 600 ms stale-gap flush resets the chain
outright, which is the right reading of a mute. Encoding silence instead
would have kept a pointless uplink and a host-side ring alive for its whole
duration.

Mute is per session and nothing is persisted — a new stream always starts
unmuted, and no new setting exists. The flag lives on the session handle
rather than on the capture, so the mic stop/start a surface recreate performs
brings the user's choice back with it, with no window in which the fresh
capture could send an unmuted frame.

The control is offered on the evidence that a capture is actually running
(nativeMicActive), not on the setting: with the mic disabled, RECORD_AUDIO
denied, or every AAudio input rung refused, there is nothing on screen to
lie about. On touch it is a pill in the corner the stats HUD doesn't use —
the one in-stream control, so it sits above the gesture layer to take its own
taps — dim while live, a red "Muted" badge while it isn't. On TV that badge
is the indicator alone: Select + Y is the control there, and a focusable
button would fight the game for the D-pad. Y is deliberately not one of the
exit chord's buttons, so neither chord can be reached through the other.

One honest consequence of keeping the stream open: the platform's recording
indicator stays lit while muted, because the mic really is still open. What
stops is the encode and the send.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:41:09 +02:00
enricobuehlerandClaude Fable 5 766991cf6a feat(apple): the mic has an off switch you can reach mid-stream
Until now the only way to stop sending the room was to end the session and turn
"Send microphone to the host" off in Settings — a per-app setting for something
that is really a per-conversation act. The mic now mutes from inside the stream:
a button on the HUD card, the Stream menu's ⌃⌥⇧A (macOS menu bar and iPad
hardware keyboards), the same chord while input is captured (both platforms
detect it in InputCapture, where the reserved ⌃⌥⇧Q/D/S already live — ⌃⌥⇧M is
the mouse-model flip, cross-client, so the mic gets A), and on iPhone/iPad a mic
disc beside the touch exit for the stats tiers whose HUD carries no buttons.

Muting is local and instant: it gates capture on this device, the host is never
asked and never told. The muted state gets its own badge over the stream —
independent of the stats overlay, because "am I muted?" is not a statistic and
the overlay is exactly what a player turns off. The badge is also the way back:
tapping it unmutes, which is the guaranteed path for a touch user who muted with
the overlay off.

Mute is session state and is deliberately not persisted. Every stream starts live
if the mic is enabled at all, rather than carrying a mute nobody remembers making
into a call three days later.

The mechanism is the one wave 1 built. `SessionAudio.setMicMuted` — which mutes
the voice processor's input on the combined engine and pauses the capture engine
on the split one — stays the single muting path; what changes is that it now
takes an EFFECTIVE mute the session composes from its two reasons: the user's
mute and the background keep-alive's privacy mute. Neither can clear the other,
so a user who muted before pocketing their phone comes back still muted, and
backgrounding no longer un-mutes anyone on return. It also latches the state, so
a mute made while the microphone permission prompt is still open lands on the
engine that grant creates instead of being lost.

The control is offered only where there is something to mute: the session's
resolved `micEnabled` (a profile can turn the mic on or off), a platform with an
app-accessible input (never tvOS), and a TCC grant the OS hasn't refused. Absent
rather than greyed on the HUD and the touch discs, greyed on the menu, and the
macOS start-of-stream banner only teaches ⌃⌥⇧A when the session actually sends a
microphone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:35:48 +02:00
enricobuehler 41e7035441 Merge branch 'audio/apple-mic' into audio/mic-latency-echo 2026-07-31 22:12:52 +02:00
enricobuehler 27861b52b8 Merge branch 'audio/android-mic' into audio/mic-latency-echo 2026-07-31 22:12:52 +02:00
enricobuehler c846b165ae Merge branch 'audio/core-desktop-uplink' into audio/mic-latency-echo 2026-07-31 22:12:51 +02:00
enricobuehler d3870294d0 Merge branch 'audio/host-mic-jitter' into audio/mic-latency-echo 2026-07-31 22:12:51 +02:00
enricobuehlerandClaude Fable 5 e861565e27 docs: "Why do I hear myself" — the four echo loops, and which knob stops each
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:11:19 +02:00
enricobuehlerandClaude Fable 5 79856d2c50 fix(host/audio): a VoiceMeeter box can no longer loop the mic into the stream
wiring_plan could hand the mic Voicemeeter Input and the loopback
Voicemeeter Aux Input — two strips of the same internal mixer, i.e. a
digital feedback loop with no acoustic path to break it, because
leftover() never asked virtualish() and the exclusion list only knew
"cable" and the Steam Speakers. Now every VoiceMeeter or generically
"virtual" render is excluded from loopback, and the last-resort tier
refuses anything virtual: a box with only mixer endpoints gets
loopback=None, honest like the cable-only case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:11:17 +02:00
enricobuehlerandClaude Fable 5 efac33fd33 feat(host/audio): the virtual mic buffers for the client it has, not the worst one ever
The mic pump grows a real de-jitter (audio/mic_jitter.rs): a two-frame
reorder window in front of the decoder, libopus concealment on sequence
gaps (up to 5 frames — a lost datagram no longer drains the ring into a
silence + re-prime crackle), and an adaptive target depth measured from
inter-arrival jitter, clamped to 10–60 ms. Both backend rings now prime
at one consumer quantum + that target: the old bursty Mac client still
measures ~42 ms and lands where the fixed 48 ms prime protected it, a
modern 10 ms-cadence client settles at ~25–35 ms, and a 2048-frame
recorder on Linux stops buying 128 ms of latency from the 3-quanta
clamp. Depth stuck above target sheds near-silent frames a few ms per
100 ms — never speech, never a hard clear.

PUNKTFUNK_MIC_LEGACY_BUFFER=1 (documented) is the one-release escape
hatch back to the fixed constants, and a "mic uplink health" line every
30 s (depth/target, cadence, gaps, conceals, reorders, drops, re-primes)
finally says which side of the link a bad mic lives on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 22:11:08 +02:00
enricobuehlerandClaude Fable 5 f3e122b0d0 feat(apple): the mic hears you, not the game — voice processing joins the engines
The Apple client on a loudspeaker was the primary reported echo source:
two AVAudioEngines by design (arbitrary mic/speaker pairs, two clocks)
meant no unit ever saw render and capture together, which is exactly
what setVoiceProcessingEnabled needs. With the mic enabled and the new
"Echo cancellation" toggle on (both defaults), playback and capture now
share ONE engine with the system voice processor engaged — AEC, noise
suppression and AGC — and the input node's other-audio ducking pinned
to .min with advanced ducking off, because this is a game stream, not a
call: the host's audio must never dip under the outgoing voice.

The two-engine path survives where it is the right answer: echo cancel
off, macOS sessions with a hand-picked speaker/mic UID or input channel
(the voice processor only follows the system default devices, and its
capture side is its own mono mix — a per-channel pick can't survive
it), and any machine whose voice processor refuses to engage. Every
failure falls back to a working configuration rather than silence. The
capture chain reads the input format after the processor is enabled,
so its mono/lower-rate output flows through the existing converter
unchanged; a first-run permission grant swaps the playback-only engine
for the combined one in place, reusing the same jitter ring and drain
thread; background mic muting mutes the processor's input instead of
pausing the shared engine (playback keeps running).

echo_cancel rides the full settings plumbing micEnabled has — defaults
key, effective settings, profile overlay (serialized as `echo_cancel`;
the Rust catalog carries it via unknown-key passthrough until it grows
the field), a captioned toggle beside the Microphone row, and a gamepad
settings row. tvOS behavior is untouched (playback only, as before).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:58:31 +02:00
enricobuehlerandClaude Fable 5 e54258b8ac feat(apple): the mic uplink drops to mono 10 ms packets and stops shoveling its FIFO
The capture path carried three latencies that had nothing to do with the
network: a 2048-frame tap request (42.7 ms bursts before the first
sample could even be sliced), 20 ms Opus framing, and a mono signal
duplicated into both stereo channels because the encoder was configured
like the downlink. CoreAudio's Opus encoder takes mFramesPerPacket=480
and mChannelsPerFrame=1 just fine — probed empirically: the converter
truly emits 10 ms mono CELT packets (TOC config 30), one per 480-frame
chunk, and the stereo-shaped decoder upmixes them with the tone intact —
so the uplink now asks for 10 ms tap buffers and encodes 48 kbps mono
10 ms packets, with 960 kept only as an init-time fallback. The host
decodes any Opus frame ≤120 ms, so nothing changes on the wire's far
end.

The tap thread also stops burning cycles per callback: the mono fold and
resampler scratch buffers are allocated once (regrown only if a larger
device quantum ever arrives), and the chunk slicer walks a head index
instead of removeFirst — which memmoved the entire backlog for every
packet on a render-adjacent thread. iOS additionally asks the session
for 5 ms IO quanta at 48 kHz when the mic is on (best-effort; the
hardware decides).

Verified: swift build (macOS arm64), swift test OpusCodecTests +
AudioChannelFoldTests, and an iOS arm64 cross-build; the 480/mono
behavior confirmed by TOC inspection on macOS 15.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:57:48 +02:00
enricobuehlerandClaude Fable 5 a681de7e5c feat(android): the mic goes through the echo canceller, and the host stops hearing itself
The capture stream opened under AAudio's default VoiceRecognition input
preset, which deliberately bypasses the HAL's acoustic echo canceller —
so a phone playing the game audio out of its own speaker fed that audio
straight back to the host. Two layers fix it, both behind a new "Echo
cancellation" setting (default ON, next to the Microphone toggle in the
touch and console settings, per-profile like every tier-P setting):

- Native: the mic opens under the VoiceCommunication preset (HAL AEC/NS
  on the capture path) and allocates an audio session id. The open
  ladder is Exclusive+voice → Shared+voice → Exclusive → Shared — some
  HALs refuse the preset or a session id outright, and a mic without
  echo cancellation still beats no mic; the last rungs are exactly the
  preset-less open this always did.

- Kotlin backstop: nativeStartMic now returns the allocated session id
  (0 = none), and StreamScreen hangs the Java AcousticEchoCanceler +
  NoiseSuppressor off it (guarded by isAvailable), releasing them on
  every mic-stop path — the surface teardown and the final dispose — so
  a surface recreate re-attaches instead of leaking effect engines.

The playback stream is untouched: retagging it voice/communication
would route it through the phone-call chain and regress quality.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:53:01 +02:00
enricobuehlerandClaude Fable 5 6286f91f89 feat(host/audio): the mic uplink keeps its sequence, and a backlog heals itself
The 0xCB datagrams always carried a seq + pts, and the ingest threw both
away one line after decoding them — the pump saw an anonymous byte pile.
Frames now travel as MicFrame {seq, pts_ns, opus} so the de-jitter that
follows can reorder, conceal and measure.

The shared queue also stops being a latency reservoir: cap 64 → 12, and
a pump that wakes to a backlog deeper than 6 frames jumps to the newest
4 instead of replaying the pile — before this, a scheduling stall could
park up to 1.28 s of standing mic delay that only a >600 ms silence gap
ever flushed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:50:29 +02:00
enricobuehlerandClaude Fable 5 55e01b4dcb feat(client): the stats overlay learns whether your mic frames arrive
NativeClient::mic_stats reaches the per-second stream window: sent and
dropped (queue-full + stale-shed) frame deltas join the tracing line and
the Stats event, and the Detailed OSD tier renders a mic line while the
uplink is live — a healthy 10 ms-frame mic reads ~100 f/s, and a drop
term means the client is shedding backlog, not the network eating audio.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:49:12 +02:00
enricobuehlerandClaude Fable 5 d17e942db0 feat(client/audio): the mic speaks 10 ms mono and asks the OS to cancel the echo
Uplink format on both desktops: 10 ms mono frames (was 20 ms stereo),
Opus Voip at 48 kbps with in-band FEC against 10 % assumed loss — half
the frame-fill latency, half the samples, and a lost datagram's audio
now rides in its successor. One datagram per frame, unchanged wire.

Linux: the capture stream finally asks for its own quantum
(NODE_LATENCY 480/48000) instead of inheriting the graph's 1024-2048
sample bursts, and when the user picked no mic it prefers an existing
echo-cancel source over the default (PUNKTFUNK_NO_AEC=1 opts out;
loading module-echo-cancel ourselves needs a load_module the pipewire
crate doesn't expose yet). Windows: the capture client declares
AudioCategory_Communications before Initialize so an endpoint's
communications APO (the system AEC) can engage; capture stays stereo
via autoconvert — the proven path — and downmixes to mono in code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:49:05 +02:00
enricobuehlerandClaude Fable 5 a7d4213778 feat(android): the mic uplink drops to mono 10 ms frames, and stops hoarding stale audio
Latency, three ways, all inside mic.rs:

- 48 kHz stereo 20 ms becomes mono 10 ms: speech gains nothing from a
  second channel, the shorter frame shaves a buffering interval off the
  uplink, and the host already decodes any Opus frame <= 120 ms with its
  stereo decoder (mono packets upmix) — no protocol change. The encoder
  follows: 48 kbps, complexity 5, in-band FEC at an assumed 10% loss so
  a dropped datagram reconstructs from its successor instead of a hole.

- The latency ratchet is gone: the capture callback drops the NEWEST
  chunk when the hand-off channel fills, so an encode-side stall used to
  convert into standing mic delay that never drained. The encode loop
  now drains the whole backlog in one lump and, past ~60 ms, jumps to
  the newest ~20 ms (one audible blip, live again), counting what it
  shed in the periodic log line. The realtime callback stays exactly as
  allocation-free as it was.

- The encode thread registers with the client's hot-thread set, so the
  ADPF session keeps mic encode on a fast core alongside audio decode.

No .frames_per_data_callback() pin: AAudio's own docs say leaving it
unset is the lowest-latency path (the callback runs at the device's
optimal burst), and the encode side re-chunks to 10 ms frames anyway.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:46:44 +02:00
enricobuehlerandClaude Fable 5 15392bd707 fix(core/client): the mic queue stops hoarding a second of your voice
MIC_QUEUE claimed 64 was ~320 ms of 5 ms frames; the frames were 20 ms, so
it really allowed 1.28 s — and since a full tokio mpsc can only refuse the
FRESH frame, one worker stall turned the whole backlog into permanent
standing mic latency. The queue shrinks to 12 and the pump's mic task now
sheds oldest-first past a ~60 ms backlog, so a stall costs a short dropout
and heals itself. New per-stage counters (sent / dropped-full /
dropped-stale) surface through NativeClient::mic_stats for the stats HUDs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-31 21:46:02 +02:00
123 changed files with 6276 additions and 833 deletions
+95
View File
@@ -0,0 +1,95 @@
# Move a versionCode that is ALREADY on Google Play between tracks — no rebuild.
#
# Why this is separate from android.yml: promotion must not rebuild. A rebuild produces a fresh
# versionCode (github.run_number) from possibly-newer sources, so it ships something nobody tested;
# promoting assigns the byte-identical artifact the testers already ran. Bolting this onto
# android.yml would mean an `if:` on all ten of its build steps.
#
# What it is for:
# * promote a tested build up a track (alpha -> production)
# * roll production back by re-pointing it at an older versionCode (to_track=production,
# version_code=<the good one>, from_track blank)
# * halt a rollout (status=halted)
#
# Defaults are deliberately the safe ones: dry_run starts TRUE, so a mis-typed versionCode
# validates and deletes the edit instead of publishing. Flip it to false only when the dry run
# printed what you meant.
name: android-promote
# Two concurrent promotions would race on the same Play edit; the loser fails with a stale-edit
# error. One at a time, and never cancel one mid-flight — a half-applied track change is worse
# than a queued one.
concurrency:
group: android-promote
cancel-in-progress: false
on:
workflow_dispatch:
inputs:
version_code:
description: 'versionCode already on Play (e.g. 10816)'
required: true
to_track:
description: 'destination track'
required: true
default: 'production'
from_track:
description: 'track to verify it is on, then clear (blank = touch nothing else)'
required: false
default: 'alpha'
notes_tag:
description: "tag whose docs/releases/whatsnew/<tag>.txt to attach, e.g. v0.23.0 (blank = none)"
required: false
default: ''
status:
description: 'completed (100%) | inProgress (needs user_fraction) | halted | draft'
required: true
default: 'completed'
user_fraction:
description: 'staged rollout fraction for inProgress, e.g. 0.2 (blank otherwise)'
required: false
default: ''
dry_run:
description: 'validate only, publish nothing'
required: true
default: 'true'
jobs:
promote:
runs-on: ubuntu-24.04
# Same image as android.yml purely for python3 + openssl (play-upload.py's only deps); it is
# already warm on the runner. Nothing here builds.
container:
image: 192.168.1.58:5010/punktfunk-android-ci:latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- name: Promote
env:
SERVICE_ACCOUNT_JSON: ${{ secrets.SERVICE_ACCOUNT_JSON }}
VERSION_CODE: ${{ inputs.version_code }}
TO_TRACK: ${{ inputs.to_track }}
FROM_TRACK: ${{ inputs.from_track }}
NOTES_TAG: ${{ inputs.notes_tag }}
STATUS: ${{ inputs.status }}
USER_FRACTION: ${{ inputs.user_fraction }}
DRY_RUN: ${{ inputs.dry_run }}
run: |
set -- --package io.unom.punktfunk \
--promote "$VERSION_CODE" \
--track "$TO_TRACK" --status "$STATUS"
# Explicit `if`, not `[ ] && …`: under `sh -e` a false AND-OR list that ends up LAST in
# the script aborts the step, and these get reordered.
if [ -n "$FROM_TRACK" ]; then set -- "$@" --promote-from "$FROM_TRACK"; fi
if [ -n "$USER_FRACTION" ]; then set -- "$@" --user-fraction "$USER_FRACTION"; fi
if [ -n "$NOTES_TAG" ]; then
NOTES="docs/releases/whatsnew/${NOTES_TAG}.txt"
# Fail loudly rather than silently publishing with the PREVIOUS release's text still
# showing on the store listing.
[ -f "$NOTES" ] || { echo "ERROR: no such notes file: $NOTES"; exit 1; }
set -- "$@" --release-notes-file "$NOTES"
fi
if [ "$DRY_RUN" = "true" ]; then set -- "$@" --no-commit; fi
echo "promoting versionCode=$VERSION_CODE -> $TO_TRACK (dry_run=$DRY_RUN)"
python3 clients/android/ci/play-upload.py "$@"
+76 -10
View File
@@ -34,9 +34,10 @@ on:
- 'rust-toolchain.toml'
- 'scripts/ci/**'
- '.gitea/workflows/android.yml'
# Single project version: a `vX.Y.Z` tag is THE release (uploads to Play's `alpha` closed
# track for manual promotion + attaches the .aab/.apk to the unified Gitea Release). A main
# push is canary (Play `internal`).
# Single project version: a `vX.Y.Z` tag is THE release (publishes to Play `production` at
# 100% + attaches the .aab/.apk to the unified Gitea Release). A main push is canary
# (Play `internal`). Production access was granted 2026-08-01; before that a tag could only
# reach `alpha` and someone had to promote it by hand in the Console.
tags: ['v*']
pull_request:
paths:
@@ -75,6 +76,56 @@ jobs:
steps:
- uses: actions/checkout@v4
# FIRST, because it costs a second and everything after it costs ten minutes.
#
# A release tag MUST carry its own Play "What's new". If the file is absent Play does not
# show nothing — it carries the PREVIOUS release's text onto this version, so production
# users read notes for a build they are not getting. That is the same defect the v0.22.3
# notes shipped (see docs/releases/README.md), and it is invisible until someone reads the
# store listing. Failing here also means a missing file cannot leave a half-published
# release: nothing is built, nothing is attached to the Gitea release, nothing reaches Play.
#
# Canary is exempt on purpose: it has no curated notes, and Play reusing text for internal
# testers costs nothing.
- name: Play release notes gate (tags only)
if: startsWith(github.ref, 'refs/tags/v')
run: |
NOTES="docs/releases/whatsnew/${GITHUB_REF_NAME}.txt"
if [ ! -f "$NOTES" ]; then
echo "ERROR: $NOTES does not exist."
echo "A production release needs its own Play 'What's new' (<=500 chars, written for"
echo "phone/TV users). Without it Play reuses the previous release's text."
echo "See docs/releases/README.md; copy docs/releases/whatsnew/TEMPLATE.txt."
exit 1
fi
# A verbatim copy of another release's file is the same bug wearing a hat: the store
# listing still describes the wrong build. Cheap to check, and only ever trips on an
# actual copy-paste that was never edited.
for other in docs/releases/whatsnew/*.txt; do
if [ "$other" != "$NOTES" ] && [ "$other" != "docs/releases/whatsnew/TEMPLATE.txt" ]; then
if cmp -s "$NOTES" "$other"; then
echo "ERROR: $NOTES is byte-identical to $other."
echo "Write notes describing THIS release, not the one before it."
exit 1
fi
fi
done
# Length is checked here as well as in play-upload.py. Not redundant: the uploader is
# the last line of defence (and the only one android-promote.yml gets), but it runs at
# step 9 — this catches an unedited TEMPLATE copy at step 1 instead of after the build.
# Must count CHARACTERS, not bytes: Play's cap is 500 chars and `•` is 3 bytes in UTF-8,
# so `wc -c` would reject a file that is comfortably legal.
python3 - "$NOTES" <<'PY'
import sys
path = sys.argv[1]
text = open(path, encoding="utf-8").read().strip()
if not text:
sys.exit(f"ERROR: {path} is empty.")
if len(text) > 500:
sys.exit(f"ERROR: {path} is {len(text)} chars; Play allows 500. Trim it.")
print(f"Play release notes OK: {path} ({len(text)}/500 chars)")
PY
# Everything below the checkout used to be four download steps (JDK, SDK,
# NDK+CMake, cargo-ndk — the flakiest, heaviest part of the job); it is all baked
# into the image now. This guard only re-asserts the Android targets so a
@@ -122,11 +173,20 @@ jobs:
run: |
eval "$(bash scripts/ci/pf-version.sh)" # -> PF_BASE (one minor ahead of the latest stable tag)
case "$GITHUB_REF" in
refs/tags/v*) VN="${GITHUB_REF_NAME#v}"; TRACK="alpha" ;; # alpha = built-in closed testing
refs/tags/v*) VN="${GITHUB_REF_NAME#v}"; TRACK="production" ;;
*) VN="${PF_BASE}-ci${GITHUB_RUN_NUMBER}"; TRACK="internal" ;;
esac
echo "VERSION_NAME=$VN" >> "$GITHUB_ENV"
echo "PLAY_TRACK=$TRACK" >> "$GITHUB_ENV"
# Play's own "What's new" (500-char cap, its own file — the vX.Y.Z.md body is ~34 KB).
# On a tag the gate step above already proved this exists, so the else branch is only
# ever the canary path. See docs/releases/README.md.
NOTES="docs/releases/whatsnew/${GITHUB_REF_NAME}.txt"
if [ -f "$NOTES" ]; then
echo "PLAY_NOTES=$NOTES" >> "$GITHUB_ENV"
else
echo "no Play release notes at $NOTES (canary — Play keeps the previous text)"
fi
echo "android version $VN -> Play track '$TRACK'"
- name: Build Release (signed AAB + universal APK)
@@ -199,15 +259,21 @@ jobs:
# Direct Publishing-API upload instead of r0adkll/upload-google-play — that action hides the
# real API error behind "Unknown error occurred."; this prints it. stdlib + openssl only (no
# pip), reuses SERVICE_ACCOUNT_JSON (raw JSON or base64), auto-handles changesNotSentForReview.
# Track: canary main -> `internal`; a vX.Y.Z release -> `alpha` (closed testing) for manual
# promotion to production in the Play console.
# Track: canary main -> `internal`; a vX.Y.Z release -> `production` at 100% (`completed`).
#
# A tag therefore ships to real users with no further click. Two things keep that honest:
# the tag is only pushed once every platform is green, and Play reviews each production
# release before it reaches anyone. To ramp instead of going straight to 100%, this is
# `--status inProgress --user-fraction 0.2`; to undo a bad one, halt or roll back from the
# Console (or `android-promote.yml`, which can re-point production at an older versionCode).
- name: Upload to Google Play
if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
env:
SERVICE_ACCOUNT_JSON: ${{ secrets.SERVICE_ACCOUNT_JSON }}
run: |
echo "uploading to Play track '$PLAY_TRACK'"
python3 clients/android/ci/play-upload.py \
--package io.unom.punktfunk \
--aab clients/android/app/build/outputs/bundle/release/app-release.aab \
--track "$PLAY_TRACK" --status completed
set -- --package io.unom.punktfunk \
--aab clients/android/app/build/outputs/bundle/release/app-release.aab \
--track "$PLAY_TRACK" --status completed
if [ -n "${PLAY_NOTES:-}" ]; then set -- "$@" --release-notes-file "$PLAY_NOTES"; fi
python3 clients/android/ci/play-upload.py "$@"
Generated
+32 -32
View File
@@ -947,7 +947,7 @@ dependencies = [
[[package]]
name = "cursor-probe"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"pf-capture",
@@ -1036,7 +1036,7 @@ dependencies = [
[[package]]
name = "display-disturb"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
@@ -2221,7 +2221,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.22.3"
version = "0.23.0"
[[package]]
name = "lazy_static"
@@ -2326,7 +2326,7 @@ dependencies = [
[[package]]
name = "libvpl-sys"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"bindgen",
"cmake",
@@ -2361,7 +2361,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"punktfunk-core",
]
@@ -2850,7 +2850,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-capture"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2871,7 +2871,7 @@ dependencies = [
[[package]]
name = "pf-client-core"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"ash",
@@ -2897,7 +2897,7 @@ dependencies = [
[[package]]
name = "pf-clipboard"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"ashpd",
@@ -2915,7 +2915,7 @@ dependencies = [
[[package]]
name = "pf-console-ui"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"ash",
@@ -2936,7 +2936,7 @@ dependencies = [
[[package]]
name = "pf-encode"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"ash",
@@ -2960,7 +2960,7 @@ dependencies = [
[[package]]
name = "pf-ffvk"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"ash",
"bindgen",
@@ -2969,7 +2969,7 @@ dependencies = [
[[package]]
name = "pf-frame"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"libc",
@@ -2981,7 +2981,7 @@ dependencies = [
[[package]]
name = "pf-gpu"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"pf-host-config",
@@ -2995,11 +2995,11 @@ dependencies = [
[[package]]
name = "pf-host-config"
version = "0.22.3"
version = "0.23.0"
[[package]]
name = "pf-inject"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"ashpd",
@@ -3028,14 +3028,14 @@ dependencies = [
[[package]]
name = "pf-paths"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"ash",
@@ -3050,7 +3050,7 @@ dependencies = [
[[package]]
name = "pf-update"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"serde",
"serde_json",
@@ -3058,7 +3058,7 @@ dependencies = [
[[package]]
name = "pf-update-check"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"base64",
@@ -3070,7 +3070,7 @@ dependencies = [
[[package]]
name = "pf-vdisplay"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"ashpd",
@@ -3103,7 +3103,7 @@ dependencies = [
[[package]]
name = "pf-win-display"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"pf-paths",
@@ -3115,7 +3115,7 @@ dependencies = [
[[package]]
name = "pf-zerocopy"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"ash",
@@ -3323,7 +3323,7 @@ dependencies = [
[[package]]
name = "punktfunk-cli"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"pf-client-core",
"punktfunk-core",
@@ -3334,7 +3334,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"android_logger",
"jni",
@@ -3350,7 +3350,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"async-channel",
@@ -3367,7 +3367,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-session"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"pf-client-core",
@@ -3382,7 +3382,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"async-channel",
"ffmpeg-next",
@@ -3402,7 +3402,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"aes-gcm",
"bytes",
@@ -3434,7 +3434,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"aes",
"aes-gcm",
@@ -3519,7 +3519,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3533,7 +3533,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"anyhow",
"ksni",
@@ -3556,7 +3556,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]]
name = "pyrowave-sys"
version = "0.22.3"
version = "0.23.0"
dependencies = [
"bindgen",
"cmake",
+1 -1
View File
@@ -53,7 +53,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package]
version = "0.22.3"
version = "0.23.0"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+7 -2
View File
@@ -3495,7 +3495,7 @@
"operationId": "forceUpdateCheck",
"responses": {
"200": {
"description": "Refreshed update-check state (`last_error` carries a failed check)",
"description": "Refreshed update-check state (`last_error` carries a failed check; `not_published` an empty channel, which is not one)",
"content": {
"application/json": {
"schema": {
@@ -7372,7 +7372,8 @@
"apply",
"channel_hint",
"check_disabled",
"available"
"available",
"not_published"
],
"properties": {
"apply": {
@@ -7452,6 +7453,10 @@
}
]
},
"not_published": {
"type": "boolean",
"description": "The check reached the feed and found this channel has **no release published yet** —\nan expected state (a channel nobody has announced to answers with a 404), not a\nfailure. Mutually exclusive with `last_error`, so a UI can say \"nothing published yet\"\ninstead of painting an empty feed as a broken host. Never set once a manifest has been\nseen for this channel: a feed that loses a document it used to serve stays an error."
},
"opt_in_hint": {
"type": [
"string",
@@ -395,6 +395,11 @@ private fun buildSettingsRows(
"mic", null, "Microphone", "Send this device's microphone to the host's virtual mic.",
s.micEnabled,
) { update(s.copy(micEnabled = it)) },
toggle(
"echoCancel", null, "Echo cancellation",
"Filter the stream's own audio out of the mic pickup. Applies while the microphone is on.",
s.echoCancel,
) { update(s.copy(echoCancel = it)) },
choice(
"padType", "Controllers", "Controller type",
@@ -2,6 +2,7 @@ package io.unom.punktfunk
import android.content.Context
import android.os.Build
import android.util.Log
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.VideoDecoders
@@ -45,18 +46,40 @@ suspend fun connectToHost(
// Transport-level half of "Low-latency mode (experimental)" (DSCP marking on the media
// sockets) — must be applied before connect, since sockets are tagged at creation.
NativeBridge.nativeSetLowLatencyMode(settings.lowLatencyMode)
val multiSlice = VideoDecoders.multiSliceTolerant()
val partialFrame = VideoDecoders.partialFrameCapable()
// Slice-progressive delivery: decoder truth AND the async decode loop — the legacy
// sync loop feeds whole AUs only, so parts must never arrive when it is selected.
val frameParts = settings.lowLatencyMode && partialFrame
val codecBits = VideoDecoders.decodableCodecBits()
// Automatic codec (P5, measured NP3 ↔ RTX 4090): AV1 beat HEVC by ~1.2 ms end-to-end at
// identical conditions, so under "Automatic" this device prefers AV1 when it hardware-
// decodes it (the AV1 bit is only ever set for a real, non-blocked hardware decoder) AND
// it lacks FEATURE_PartialFrame — a partial-frame device keeps HEVC, whose slice overlap
// AV1 cannot ride (AV1 has no slices; the host's chunked poll never arms). The host
// honors the preference only inside the probed shared codec set, so an AV1-less encoder
// still resolves HEVC. An explicit user choice always wins unchanged.
val preferredCodec = settings.preferredCodec().takeIf { it != 0 }
?: if (codecBits and 4 != 0 && !partialFrame) 4 else 0
// The connect-time capability readout (`adb logcat -s pf.caps`): the P2 slice pipeline
// is client-inert unless BOTH probes pass — this line says which decoder failed one.
Log.i(
"pf.caps",
VideoDecoders.capsReport() +
" → multiSlice=$multiSlice parts=$frameParts prefer=$preferredCodec" +
" (lowLatency=${settings.lowLatencyMode})",
)
NativeBridge.nativeConnect(
host, port, w, h, hz,
identity.certPem, identity.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, VideoDecoders.multiSliceTolerant(),
// Slice-progressive delivery: decoder truth AND the async decode loop — the legacy
// sync loop feeds whole AUs only, so parts must never arrive when it is selected.
settings.lowLatencyMode && VideoDecoders.partialFrameCapable(),
hdrEnabled, multiSlice,
frameParts,
settings.audioChannels,
// What this device can decode (H.264|HEVC always, AV1 when a real decoder exists) +
// the user's soft codec preference — the host resolves the emitted codec from both.
VideoDecoders.decodableCodecBits(), settings.preferredCodec(), timeoutMs,
// the soft codec preference (user choice, or the Automatic AV1 rule above) — the
// host resolves the emitted codec from both.
codecBits, preferredCodec, timeoutMs,
launch,
// The host's approval-list / trust-store label for this device — the same
// Build.MODEL convention the pairing dialogs use for nativePair.
@@ -38,6 +38,7 @@ data class SettingsOverlay(
val compositor: Int? = null,
val audioChannels: Int? = null,
val micEnabled: Boolean? = null,
val echoCancel: Boolean? = null,
val touchMode: TouchMode? = null,
val mouseMode: MouseMode? = null,
val invertScroll: Boolean? = null,
@@ -70,6 +71,7 @@ data class SettingsOverlay(
compositor = compositor ?: base.compositor,
audioChannels = audioChannels ?: base.audioChannels,
micEnabled = micEnabled ?: base.micEnabled,
echoCancel = echoCancel ?: base.echoCancel,
touchMode = touchMode ?: base.touchMode,
mouseMode = mouseMode ?: base.mouseMode,
invertScroll = invertScroll ?: base.invertScroll,
@@ -103,6 +105,7 @@ data class SettingsOverlay(
compositor = if (after.compositor != before.compositor) after.compositor else compositor,
audioChannels = if (after.audioChannels != before.audioChannels) after.audioChannels else audioChannels,
micEnabled = if (after.micEnabled != before.micEnabled) after.micEnabled else micEnabled,
echoCancel = if (after.echoCancel != before.echoCancel) after.echoCancel else echoCancel,
touchMode = if (after.touchMode != before.touchMode) after.touchMode else touchMode,
mouseMode = if (after.mouseMode != before.mouseMode) after.mouseMode else mouseMode,
invertScroll = if (after.invertScroll != before.invertScroll) after.invertScroll else invertScroll,
@@ -128,6 +131,7 @@ data class SettingsOverlay(
"compositor" -> copy(compositor = null)
"audio_channels" -> copy(audioChannels = null)
"mic_enabled" -> copy(micEnabled = null)
"echo_cancel" -> copy(echoCancel = null)
"touch_mode" -> copy(touchMode = null)
"mouse_mode" -> copy(mouseMode = null)
"invert_scroll" -> copy(invertScroll = null)
@@ -150,6 +154,7 @@ data class SettingsOverlay(
if (compositor != null) add("compositor")
if (audioChannels != null) add("audio_channels")
if (micEnabled != null) add("mic_enabled")
if (echoCancel != null) add("echo_cancel")
if (touchMode != null) add("touch_mode")
if (mouseMode != null) add("mouse_mode")
if (invertScroll != null) add("invert_scroll")
@@ -180,6 +185,7 @@ data class SettingsOverlay(
compositor?.let { j.put("compositor", it) }
audioChannels?.let { j.put("audio_channels", it) }
micEnabled?.let { j.put("mic_enabled", it) }
echoCancel?.let { j.put("echo_cancel", it) }
touchMode?.let { j.put("touch_mode", it.name) }
mouseMode?.let { j.put("mouse_mode", it.storedName) }
invertScroll?.let { j.put("invert_scroll", it) }
@@ -198,9 +204,9 @@ data class SettingsOverlay(
/** Keys this build models; everything else in a stored overlay is carried through. */
private val KNOWN = setOf(
"width", "height", "refresh_hz", "bitrate_kbps", "render_scale", "codec",
"hdr_enabled", "compositor", "audio_channels", "mic_enabled", "touch_mode",
"mouse_mode", "invert_scroll", "gamepad", "stats_verbosity", "low_latency_mode",
"present_priority", "smooth_buffer",
"hdr_enabled", "compositor", "audio_channels", "mic_enabled", "echo_cancel",
"touch_mode", "mouse_mode", "invert_scroll", "gamepad", "stats_verbosity",
"low_latency_mode", "present_priority", "smooth_buffer",
)
internal fun fromJson(j: JSONObject): SettingsOverlay = SettingsOverlay(
@@ -214,6 +220,7 @@ data class SettingsOverlay(
compositor = j.optIntOrNull("compositor"),
audioChannels = j.optIntOrNull("audio_channels"),
micEnabled = j.optBooleanOrNull("mic_enabled"),
echoCancel = j.optBooleanOrNull("echo_cancel"),
touchMode = j.optStringOrNull("touch_mode")
?.let { n -> TouchMode.entries.firstOrNull { it.name == n } },
mouseMode = j.optStringOrNull("mouse_mode")
@@ -1,6 +1,7 @@
package io.unom.punktfunk
import android.content.Context
import android.hardware.display.DisplayManager
import android.os.Build
import android.util.Log
import android.view.Display
@@ -41,6 +42,15 @@ data class Settings(
* the host resolves (AV1 is only advertised/offered when the device has a real AV1 decoder). */
val codec: String = "auto",
val micEnabled: Boolean = false,
/**
* Cancel acoustic echo on the mic uplink (plus noise suppression): the capture opens under
* the VoiceCommunication preset so the HAL's own AEC/NS process it, with the Java effects
* attached as a backstop where available. On by default — a phone/tablet plays the game audio
* out of the same device its mic hears, so without this the host hears its own stream back.
* Turn off for a headset-only setup where the untouched full-band capture sounds better.
* Only meaningful while [micEnabled] is on.
*/
val echoCancel: Boolean = true,
/**
* How much the in-stream stats overlay shows — see [StatsVerbosity]. Defaults to
* [StatsVerbosity.NORMAL] (the res/fps line + latency headline + reliability counters); the full
@@ -209,6 +219,7 @@ class SettingsStore(context: Context) {
audioChannels = prefs.getInt(K_AUDIO_CH, 2),
codec = prefs.getString(K_CODEC, "auto") ?: "auto",
micEnabled = prefs.getBoolean(K_MIC, false),
echoCancel = prefs.getBoolean(K_ECHO_CANCEL, true),
statsVerbosity = prefs.getString(K_STATS_VERBOSITY, null)
?.let { name -> StatsVerbosity.entries.firstOrNull { it.name == name } }
// Migration from the pre-tier Boolean "stats_hud_enabled": an explicit OFF stays off;
@@ -254,6 +265,7 @@ class SettingsStore(context: Context) {
.putInt(K_AUDIO_CH, s.audioChannels)
.putString(K_CODEC, s.codec)
.putBoolean(K_MIC, s.micEnabled)
.putBoolean(K_ECHO_CANCEL, s.echoCancel)
.putString(K_STATS_VERBOSITY, s.statsVerbosity.name)
.putString(K_TOUCH_MODE, s.touchMode.name)
.putBoolean(K_GAMEPAD_UI, s.gamepadUiEnabled)
@@ -282,6 +294,7 @@ class SettingsStore(context: Context) {
const val K_AUDIO_CH = "audio_channels"
const val K_CODEC = "codec"
const val K_MIC = "mic_enabled"
const val K_ECHO_CANCEL = "echo_cancel"
const val K_STATS_VERBOSITY = "stats_verbosity"
/** Pre-tier Boolean the [K_STATS_VERBOSITY] enum replaced — read once for migration, never
@@ -319,14 +332,31 @@ class SettingsStore(context: Context) {
}
}
/**
* The display to probe for capability/mode queries: the context's own display when it is already
* associated with one, else the DEFAULT display via [DisplayManager]. A `punktfunk://` deep-link
* COLD start can reach the connect before the activity is attached to its display —
* `context.display` then throws, and the old `false`/1080p60 fallbacks silently downgraded the
* whole session (no HDR advertised / non-native mode) with nothing in the log. The default
* display IS the panel on phones and TVs; the activity-display distinction only matters on
* multi-display setups, where the attached path still wins whenever it is available.
*/
private fun probeDisplay(context: Context): Display? =
runCatching { context.display }.getOrNull()
?: runCatching {
context.getSystemService(DisplayManager::class.java)
?.getDisplay(Display.DEFAULT_DISPLAY)
}.getOrNull().also {
if (it != null) Log.i("punktfunk", "display probe: context unattached — using DEFAULT_DISPLAY")
}
/**
* The device's native display mode as a landscape `(width, height, hz)` — the long edge is the
* width, since we stream a desktop. Falls back to 1920×1080@60 if the display can't be read.
* [context] must be a visual (Activity) context.
* width, since we stream a desktop. Falls back to 1920×1080@60 if no display can be read at all
* (see [probeDisplay] for the cold-start fallback that makes that a last resort).
*/
fun nativeDisplayMode(context: Context): Triple<Int, Int, Int> {
// getDisplay() throws on a non-visual context rather than returning null — guard it.
val display = runCatching { context.display }.getOrNull() ?: return Triple(1920, 1080, 60)
val display = probeDisplay(context) ?: return Triple(1920, 1080, 60)
val mode = display.mode
val w = mode.physicalWidth
val h = mode.physicalHeight
@@ -341,7 +371,12 @@ fun nativeDisplayMode(context: Context): Triple<Int, Int, Int> {
* capability gate the Apple/Windows clients apply.
*/
fun displaySupportsHdr(context: Context): Boolean {
val display = runCatching { context.display }.getOrNull() ?: return false
val display = probeDisplay(context)
if (display == null) {
// Distinguishable from a real SDR verdict — a silent `false` here cost an HDR session.
Log.w("punktfunk", "display HDR probe: no display reachable — advertising SDR")
return false
}
val types = buildSet {
// API 34+: the sanctioned per-mode query (Display.Mode.getSupportedHdrTypes). The
// deprecated Display-level hdrCapabilities can return EMPTY on Android 14+ devices
@@ -673,12 +673,22 @@ private fun DisplaySettings(s: Settings, update: (Settings) -> Unit, context: an
// Only codecs this device can actually decode are offered — a preference the client never
// advertises would be a dead setting (see [codecOptionsFor]).
val av1Capable = remember { VideoDecoders.pickDecoder("video/av01") != null }
// Mirror the Automatic AV1 rule in HostConnect (hardware AV1 AND no partial-frame
// support) so the picker says what "Automatic" actually does on THIS device.
val autoPrefersAv1 = remember {
VideoDecoders.decodableCodecBits() and 4 != 0 && !VideoDecoders.partialFrameCapable()
}
SettingDropdown(
label = "Video codec",
options = codecOptionsFor(s.codec, av1Capable),
selected = s.codec,
field = "codec",
caption = "A preference — the host falls back if it can't encode this one.",
caption = if (autoPrefersAv1) {
"A preference — the host falls back if it can't encode this one. " +
"Automatic prefers AV1 on this device."
} else {
"A preference — the host falls back if it can't encode this one."
},
) { c -> update(s.copy(codec = c)) }
// HDR is only meaningful on a panel that can present HDR10; on an SDR display the toggle is
@@ -794,6 +804,14 @@ private fun AudioSettings(s: Settings, update: (Settings) -> Unit, onMicChange:
field = "mic_enabled",
onCheckedChange = onMicChange,
)
ToggleRow(
title = "Echo cancellation",
subtitle = "Filters the stream's own audio out of the mic pickup",
checked = s.echoCancel,
enabled = s.micEnabled,
field = "echo_cancel",
onCheckedChange = { on -> update(s.copy(echoCancel = on)) },
)
}
}
@@ -18,11 +18,13 @@ import kotlin.math.roundToInt
* The live stats overlay the unified HUD (`design/stats-unification.md`): headline is
* `capturedisplayed` tiled by `host+network` + `decode` + `display` when the platform delivered
* OnFrameRendered render callbacks this window (`dispValid`), falling back to the v1
* `capturedecoded` headline without the `display` term when it didn't. Reads the 26-double
* layout from [NativeBridge.nativeVideoStats]:
* `capturedecoded` headline without the `display` term when it didn't. Reads the 33-double
* layout from [NativeBridge.nativeVideoStats] (that KDoc is the authoritative index list):
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skew, w, h, hz, lostTotal, bitDepth, colorPrimaries,
* colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms, netP50Ms, lost, skipped,
* fec, frames, dispValid, displayP50Ms, e2eDispP50Ms, e2eDispP95Ms]`.
* fec, frames, dispValid, displayP50Ms, e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms,
* presentsWindow, presenterActive, feedP50Ms, codecP50Ms, skippedOverflowWindow]`. Every read
* is length-guarded, so an older native lib simply omits the lines it can't feed.
*
* [verbosity] selects how many lines render (each tier a superset of the last see
* [StatsVerbosity]):
@@ -129,10 +131,30 @@ internal fun StatsOverlay(
} else {
""
}
// P3 decode split (s[30]/s[31]): `feed` = received→queued (hand-off + input-slot
// wait) + `codec` = queued→decoded (codec-pure) — rendered when a sample landed.
val decodeTerm = if (s.size >= 33 && (s[30] > 0 || s[31] > 0)) {
"decode ${"%.1f".format(s[15])} " +
"(feed ${"%.1f".format(s[30])} + codec ${"%.1f".format(s[31])})"
} else {
"decode ${"%.1f".format(s[15])}"
}
statLine(
"= $hostTerms + decode ${"%.1f".format(s[15])}$displayTerm$presents",
"= $hostTerms + $decodeTerm$displayTerm$presents",
Color.White,
)
// Metric fairness: the Apple client's HUD shaves ~2 refresh periods of OS
// pipeline floor off its shown display/end-to-end; Android shows raw. This twin
// applies the same shave so iPhone↔Android HUD numbers compare directly.
if (dispValid && hz > 0) {
val shave = 2000.0 / hz
statLine(
"≈ Apple-HUD equiv: end-to-end " +
"${"%.1f".format((s[24] - shave).coerceAtLeast(0.0))} · display " +
"${"%.1f".format((s[23] - shave).coerceAtLeast(0.0))} (2 refresh)",
Color(0xFFA8D8B8),
)
}
}
}
counterLine(s, lost)?.let { statLine(it, Color(0xFFFFB0B0)) }
@@ -178,12 +200,17 @@ private fun counterLine(s: DoubleArray, lostTotal: Long): String? {
val fec = s[20].toLong()
val frames = s[21].toLong()
if (lost == 0L && skipped == 0L && fec == 0L) return null
// The overflow subset of `skipped` (s[32]): whole AUs dropped before feeding — the decoder
// fell behind. Absent (0 / old layout) the plain count keeps meaning benign pacing drops.
val overflow = if (s.size >= 33) s[32].toLong() else 0L
return buildList {
if (lost > 0) {
val pct = 100.0 * lost / (frames + lost).coerceAtLeast(1)
add("lost $lost (${"%.1f".format(pct)}%)")
}
if (skipped > 0) add("skipped $skipped")
if (skipped > 0) {
add(if (overflow > 0) "skipped $skipped (⚠ $overflow overflow)" else "skipped $skipped")
}
if (fec > 0) add("FEC $fec")
}.joinToString(" · ")
}
@@ -9,11 +9,15 @@ import android.content.IntentFilter
import android.content.pm.ActivityInfo
import android.content.pm.PackageManager
import android.hardware.usb.UsbManager
import android.media.audiofx.AcousticEchoCanceler
import android.media.audiofx.AudioEffect
import android.media.audiofx.NoiseSuppressor
import android.net.wifi.WifiManager
import android.os.Build
import android.text.InputType
import android.util.Log
import android.view.KeyEvent
import android.view.Surface
import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.View
@@ -25,12 +29,20 @@ import android.view.inputmethod.InputMethodManager
import android.widget.Toast
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.aspectRatio
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Mic
import androidx.compose.material.icons.filled.MicOff
import androidx.compose.material3.Icon
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
@@ -41,6 +53,7 @@ import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.input.pointer.pointerInput
import androidx.compose.ui.platform.LocalContext
@@ -97,6 +110,38 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
Manifest.permission.RECORD_AUDIO,
) == PackageManager.PERMISSION_GRANTED
// The Java AEC/NS pair backstopping the native VoiceCommunication capture preset, hung off the
// audio session id `nativeStartMic` returns. Attached in surfaceCreated (where the mic starts)
// and released on every path that stops the mic — the surface teardown AND the final dispose —
// so a surface recreate re-attaches to the fresh stream instead of leaking effect engines.
// All three touch points run on the main thread; a plain list is race-free.
val micEffects = remember { mutableListOf<AudioEffect>() }
// In-stream mic mute. Per SESSION and never persisted (no setting backs it): a new stream
// always starts unmuted. The authoritative flag lives on the native handle, which is why a mute
// survives the mic stop/start a surface recreate performs — this state is the UI's mirror of
// it, and survives the same recreate because the composition outlives the surface.
var micMuted by remember(handle) { mutableStateOf(false) }
// Whether a capture is actually RUNNING, not merely wanted — set from surfaceCreated on what
// nativeMicActive reports. A device that refused every AAudio input rung gets no mute control
// rather than one that lies about a mic being heard.
var micRunning by remember(handle) { mutableStateOf(false) }
// Transient confirmation of a mic-chord toggle (null = nothing showing). Only the gamepad path
// needs it: the touch button confirms itself by changing under the finger, but a chord has no
// on-screen state of its own, and "did that register?" is exactly the doubt to answer.
var micHint by remember { mutableStateOf<String?>(null) }
LaunchedEffect(micHint) {
if (micHint != null) {
delay(1600)
micHint = null
}
}
// The one place mute is toggled — Compose state + the native flag, always together.
val setMicMuted = { muted: Boolean ->
micMuted = muted
NativeBridge.nativeSetMicMuted(handle, muted)
}
// Live decode stats for the HUD. `statsOn` (verbosity != OFF) gates the whole native pipeline:
// the per-frame sampling (nativeSetVideoStatsEnabled — a hidden HUD costs one atomic load per
// frame) AND the 1 s poll loop, which only runs while the overlay is visible. Enabling resets
@@ -287,6 +332,16 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
// Show a "hold to quit" hint the moment the chord completes (the router debounces the actual
// exit); it clears when the buttons release early or the hold elapses. Runs on the main thread.
router.onExitArmed = { armed -> exitArming = armed }
// Select + Y toggles the mic — the couch reach for the on-screen mute button, which a
// gamepad/TV user has no pointer for. Ignored when no capture is running (there is nothing
// to mute, and claiming otherwise would be the lie the control exists to avoid).
router.onMicChord = {
if (micRunning) {
val next = !micMuted
setMicMuted(next)
micHint = if (next) "Microphone muted" else "Microphone live"
}
}
// Physical mouse: uncaptured hover/click/wheel forwards as absolute pointing; captured
// (setting or the Ctrl+Alt+Shift+Q chord) raw deltas forward as relative mouse-look.
// The local cursor is hidden over the stream — the host's own cursor, composited into
@@ -483,6 +538,7 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
dsUsbReceiver?.let { runCatching { context.unregisterReceiver(it) } }
ds?.stop() // rumble-stop on the physical pad + release the USB link + free the wire slot
router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down
router.onMicChord = null // same: no mute toggle on buttons released during teardown
router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener
activity?.gamepadRouter = null
// Mouse/remote-pointer teardown: lift held buttons, drop the grab, restore the cursor.
@@ -519,6 +575,7 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
activity?.requestedOrientation =
priorOrientation ?: ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED
// Leaving the stream: stop the mic + audio + decode threads and tear down the session.
releaseMicEffects(micEffects)
NativeBridge.nativeStopMic(handle)
NativeBridge.nativeStopAudio(handle)
NativeBridge.nativeStopVideo(handle)
@@ -609,10 +666,38 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
.roundToInt(),
)
NativeBridge.nativeStartAudio(handle, lowLatencyMode)
if (micWanted) NativeBridge.nativeStartMic(handle)
if (micWanted) {
val sessionId =
NativeBridge.nativeStartMic(handle, initialSettings.echoCancel)
if (initialSettings.echoCancel) {
attachMicEffects(sessionId, micEffects)
}
// Did a capture actually open? That — not the setting — is what
// puts the mute control on screen. A restart after a surface
// recreate comes back already muted if the user muted: the flag
// lives on the session handle, so nothing has to be re-applied.
micRunning = NativeBridge.nativeMicActive(handle)
}
}
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {}
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {
// Re-assert the frame-rate vote: a buffer-geometry change can reset
// the surface's frame-rate setting on some OEM builds, silently
// dropping the 120 Hz pin mid-stream. Mirrors the native hint's
// policy (FIXED_SOURCE; ALWAYS only on the TV low-latency path —
// phones stay seamless so a re-hint can never force a mode flicker).
if (streamHz > 0) runCatching {
holder.surface.setFrameRate(
streamHz.toFloat(),
Surface.FRAME_RATE_COMPATIBILITY_FIXED_SOURCE,
if (isTv && lowLatencyMode) {
Surface.CHANGE_FRAME_RATE_ALWAYS
} else {
Surface.CHANGE_FRAME_RATE_ONLY_IF_SEAMLESS
},
)
}
}
override fun surfaceDestroyed(holder: SurfaceHolder) {
// Surface gone (backgrounding, or on the way out). Stop the threads that
@@ -620,7 +705,12 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
// DisposableEffect has closed it, the handle is freed; dereferencing it
// here is the use-after-free that crashed on back-navigation.
if (!closed.get()) {
releaseMicEffects(micEffects)
NativeBridge.nativeStopMic(handle)
// No capture, no control — but the MUTE state is deliberately left
// standing (native keeps it on the handle), so the restart in
// surfaceCreated brings the user's choice back with it.
micRunning = false
NativeBridge.nativeStopAudio(handle)
NativeBridge.nativeStopVideo(handle)
}
@@ -695,9 +785,106 @@ fun StreamScreen(session: ActiveSession, onDisconnect: () -> Unit) {
}
},
)
// Mic mute, LAST in the stack — the one in-stream control, so unlike the purely visual
// overlays above it has to sit on top of the gesture layer to receive its own taps (it
// costs the stream that small corner of touch area, which is why it exists only while a
// capture actually runs). On TV it is the indicator alone: the Select + Y chord is the
// control there, and a focusable button would fight the game for the D-pad.
if (micRunning && (micMuted || !isTv)) {
MicMuteControl(
muted = micMuted,
onToggle = if (isTv) null else ({ setMicMuted(!micMuted) }),
modifier = Modifier.align(Alignment.TopEnd).padding(12.dp),
)
}
// Chord confirmation (gamepad/TV) — the counterpart to the button changing under a finger.
micHint?.let { MicChordHint(it, Modifier.align(Alignment.TopCenter).padding(top = 16.dp)) }
}
}
/**
* Attach the Java echo-canceller + noise-suppressor pair to the mic stream's audio session the
* backstop for HALs whose VoiceCommunication capture path doesn't cancel on its own (the native
* side already opened the stream under that preset). [sessionId] `<= 0` means native allocated no
* session (echo cancellation off, or the preset fell back to the plain open), so there is nothing
* to hang an effect on. Created effects land in [into] for [releaseMicEffects]; `create()`
* returning null (unsupported / claimed) is quietly nothing the HAL preset still does its part.
* Needs no extra permission: the effect APIs attach to our own recording session.
*/
private fun attachMicEffects(sessionId: Int, into: MutableList<AudioEffect>) {
if (sessionId <= 0) return
if (AcousticEchoCanceler.isAvailable()) {
AcousticEchoCanceler.create(sessionId)?.let { it.setEnabled(true); into.add(it) }
}
if (NoiseSuppressor.isAvailable()) {
NoiseSuppressor.create(sessionId)?.let { it.setEnabled(true); into.add(it) }
}
}
/** Release every attached mic effect engine. Idempotent the list is cleared, and both stop
* paths (surface teardown, final dispose) may call it in either order. */
private fun releaseMicEffects(effects: MutableList<AudioEffect>) {
effects.forEach { runCatching { it.release() } }
effects.clear()
}
/**
* The in-stream mic control and its muted indicator, in one element: a dim mic glyph while the
* uplink is live, a red **Muted** badge while it isn't so the state that matters is the loud one,
* readable at couch distance and impossible to mistake for the stream's own picture.
*
* [onToggle] `null` makes it a pure indicator (the TV/gamepad surface, where the Select + Y chord
* is the control); non-null makes the badge itself the touch target. Rendering it at all is the
* caller's decision it means a capture is genuinely running.
*/
@Composable
private fun MicMuteControl(muted: Boolean, onToggle: (() -> Unit)?, modifier: Modifier = Modifier) {
val shape = RoundedCornerShape(10.dp)
Row(
modifier = modifier
.clip(shape)
.background(if (muted) Color(0xE0B3261E) else Color.Black.copy(alpha = 0.45f))
.then(if (onToggle != null) Modifier.clickable(onClick = onToggle) else Modifier)
.padding(horizontal = 12.dp, vertical = 10.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Icon(
imageVector = if (muted) Icons.Filled.MicOff else Icons.Filled.Mic,
// Spoken state first, then the action — a talkback user needs to know they are muted
// before they need to know how to stop being muted.
contentDescription = if (muted) {
"Microphone muted. Activate to unmute."
} else {
"Microphone live. Activate to mute."
},
tint = Color.White,
modifier = Modifier.size(20.dp),
)
if (muted) {
Spacer(Modifier.width(6.dp))
Text("Muted", color = Color.White, fontSize = 14.sp)
}
}
}
/**
* Transient confirmation that the mic chord (Select + Y) registered. The badge above already says
* *muted*, but nothing on screen says *un*muted and "did that press do anything?" is the whole
* doubt a chord with no button under the finger creates. Same pill vocabulary as the other
* in-stream cues; the caller clears it after a beat.
*/
@Composable
private fun MicChordHint(text: String, modifier: Modifier = Modifier) {
Text(
text,
modifier = modifier
.background(Color.Black.copy(alpha = 0.55f), RoundedCornerShape(8.dp))
.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White,
fontSize = 15.sp,
)
}
/**
* The "hold to quit" cue shown while the gamepad exit chord (Select + Start + L1 + R1) is held. The
* chord no longer quits on a quick press the router debounces it on a ~1 s hold so this confirms
@@ -105,8 +105,20 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long, styl
NativeBridge.nativeSendTouch(handle, it, 2, 0, 0, sw, sh)
}
c.positionChanged() ->
ids[c.id]?.let {
NativeBridge.nativeSendTouch(handle, it, 1, x, y, sw, sh)
ids[c.id]?.let { id ->
// Batched MotionEvents coalesce intermediate points into the
// historical list — forward them in order so a fast swipe keeps
// its real curvature on the host (usually empty during a stream:
// unbuffered dispatch is requested, so this costs nothing).
for (hs in c.historical) {
NativeBridge.nativeSendTouch(
handle, id, 1,
hs.position.x.roundToInt().coerceIn(0, sw - 1),
hs.position.y.roundToInt().coerceIn(0, sh - 1),
sw, sh,
)
}
NativeBridge.nativeSendTouch(handle, id, 1, x, y, sw, sh)
}
}
c.consume()
@@ -289,7 +301,10 @@ internal suspend fun PointerInputScope.streamTouchInput(
accY -= outY
}
} else {
moveAbs(p.position.x, p.position.y) // direct: cursor follows the finger
// Direct: cursor follows the finger — historical points first (batched
// MotionEvent samples), so the host cursor traces the finger's real path.
for (hs in p.historical) moveAbs(hs.position.x, hs.position.y)
moveAbs(p.position.x, p.position.y)
}
}
ev.changes.forEach { it.consume() }
+95 -16
View File
@@ -6,13 +6,23 @@ Why hand-rolled: stdlib + `openssl` only (no pip on the runner), and it prints G
error at the stage it fails instead of a catch-all. Reuses the SERVICE_ACCOUNT_JSON secret and
tolerates it being raw JSON *or* base64-encoded JSON.
Usage:
Usage (upload a new build):
SERVICE_ACCOUNT_JSON='<raw-or-base64 SA key>' \
python3 play-upload.py --package io.unom.punktfunk \
--aab path/to/app-release.aab --track internal --status completed [--no-commit]
--no-commit: do insert -> upload -> track-update -> validate, then delete the edit (publishes
nothing). Use it to dry-run the credentials/AAB without touching the live track.
Usage (promote a build that is already on Play, no rebuild):
python3 play-upload.py --package io.unom.punktfunk \
--promote 10816 --promote-from alpha --track production
Promotion assigns an EXISTING versionCode to another track, so what ships to production is the
byte-identical artifact the testers ran rebuilding would burn a fresh versionCode and ship
something nobody has tested. --promote-from additionally asserts the code really is on that track
(catches a typo'd versionCode before it reaches production) and empties it in the SAME edit, so
the move is atomic: testers are never left pinned to a code that production also serves.
--no-commit: do insert -> upload/assign -> track-update -> validate, then delete the edit
(publishes nothing). Use it to dry-run the credentials/AAB/notes without touching the live track.
"""
import argparse, base64, json, os, subprocess, sys, tempfile, time
import urllib.request, urllib.parse, urllib.error
@@ -104,18 +114,79 @@ def access_token(sa) -> str:
return tok["access_token"]
# Play's "What's new" is capped at 500 characters per language. The cap lives in the Console
# (the REST reference does not state it) and the API rejects longer text at commit — i.e. AFTER
# the AAB has uploaded — so check it up front and print the actual count.
NOTES_MAX = 500
def load_release_notes(path, language):
with open(path, encoding="utf-8") as f:
text = f.read().strip()
if not text:
sys.exit(f"ERROR: release-notes file is empty: {path}")
if len(text) > NOTES_MAX:
sys.exit(f"ERROR: release notes are {len(text)} chars, Play allows {NOTES_MAX}: {path}")
print(f"release notes: {len(text)}/{NOTES_MAX} chars ({language})")
return [{"language": language, "text": text}]
def put_track(app, edit, tok, track, version_codes, status, user_fraction=None, notes=None):
"""PUT one track. An empty version_codes list clears the track (what promotion does to the
track it promoted OUT of)."""
release = {"status": status, "versionCodes": [str(v) for v in version_codes]}
if user_fraction is not None:
release["userFraction"] = user_fraction
if notes:
release["releaseNotes"] = notes
# Clearing a track means "no active releases", not "an empty release".
body = {"track": track, "releases": [release] if version_codes else []}
call("PUT", f"{app}/edits/{edit}/tracks/{track}", token=tok,
data=json.dumps(body).encode(), content_type="application/json")
def assert_on_track(app, edit, tok, track, vc):
"""Fail before anything is written if --promote names a versionCode that is not actually on
the track we claim to be promoting out of."""
got = call("GET", f"{app}/edits/{edit}/tracks/{track}", token=tok)
live = [c for r in got.get("releases", []) for c in r.get("versionCodes", [])]
if str(vc) not in live:
sys.exit(f"ERROR: versionCode {vc} is not on track '{track}' (it has: {live or 'nothing'})")
print(f"verified versionCode={vc} is live on '{track}'")
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--package", required=True)
ap.add_argument("--aab", required=True)
ap.add_argument("--aab", help="upload this bundle (mutually exclusive with --promote)")
ap.add_argument("--promote", type=int, metavar="VERSIONCODE",
help="assign an already-uploaded versionCode instead of uploading")
ap.add_argument("--promote-from", metavar="TRACK",
help="with --promote: assert the code is on TRACK, then clear TRACK")
ap.add_argument("--track", default="internal")
ap.add_argument("--status", default="completed")
ap.add_argument("--user-fraction", type=float,
help="staged rollout fraction, 0<f<1; required by --status inProgress")
ap.add_argument("--release-notes-file", help="Play 'What's new' text (<=500 chars)")
ap.add_argument("--release-notes-language", default="en-US")
ap.add_argument("--no-commit", action="store_true")
a = ap.parse_args()
if not os.path.isfile(a.aab):
if bool(a.aab) == bool(a.promote):
sys.exit("ERROR: pass exactly one of --aab (upload) or --promote (assign an existing code)")
if a.promote_from and not a.promote:
sys.exit("ERROR: --promote-from only applies to --promote")
# inProgress without a fraction is an API error; halted/completed with one is also rejected.
if a.status == "inProgress" and a.user_fraction is None:
sys.exit("ERROR: --status inProgress requires --user-fraction")
if a.user_fraction is not None and not (0 < a.user_fraction < 1):
sys.exit(f"ERROR: --user-fraction must be strictly between 0 and 1 (got {a.user_fraction})")
if a.aab and not os.path.isfile(a.aab):
sys.exit(f"ERROR: AAB not found: {a.aab}")
notes = load_release_notes(a.release_notes_file, a.release_notes_language) \
if a.release_notes_file else None
sa = load_sa()
tok = access_token(sa)
print(f"authenticated as {sa['client_email']} (project {sa.get('project_id')})")
@@ -123,17 +194,25 @@ def main():
try:
edit = call("POST", f"{app}/edits", token=tok)["id"]
with open(a.aab, "rb") as f:
blob = f.read()
print(f"uploading {a.aab} ({len(blob)} bytes) ...")
vc = call("POST", f"{UPLOAD}/{a.package}/edits/{edit}/bundles?uploadType=media",
token=tok, data=blob, content_type="application/octet-stream")["versionCode"]
print(f"uploaded versionCode={vc}")
call("PUT", f"{app}/edits/{edit}/tracks/{a.track}", token=tok,
data=json.dumps({"track": a.track,
"releases": [{"status": a.status, "versionCodes": [str(vc)]}]}).encode(),
content_type="application/json")
print(f"assigned versionCode={vc} -> track={a.track} status={a.status}")
if a.promote:
vc = a.promote
if a.promote_from:
assert_on_track(app, edit, tok, a.promote_from, vc)
else:
with open(a.aab, "rb") as f:
blob = f.read()
print(f"uploading {a.aab} ({len(blob)} bytes) ...")
vc = call("POST", f"{UPLOAD}/{a.package}/edits/{edit}/bundles?uploadType=media",
token=tok, data=blob, content_type="application/octet-stream")["versionCode"]
print(f"uploaded versionCode={vc}")
put_track(app, edit, tok, a.track, [vc], a.status, a.user_fraction, notes)
print(f"assigned versionCode={vc} -> track={a.track} status={a.status}"
+ (f" userFraction={a.user_fraction}" if a.user_fraction is not None else ""))
# Same edit as the assignment above, so the code is never active on both tracks at once.
if a.promote_from:
put_track(app, edit, tok, a.promote_from, [], a.status)
print(f"cleared track '{a.promote_from}'")
if a.no_commit:
call("POST", f"{app}/edits/{edit}:validate", token=tok)
@@ -65,6 +65,16 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
*/
var onExitArmed: ((armed: Boolean) -> Unit)? = null
/**
* Invoked (main thread) each time the mic-mute chord ([MIC_CHORD], Select + Y) is COMPLETED on
* a pad the couch equivalent of the stream's on-screen mute button, which a gamepad user
* cannot reach. `StreamScreen` wires it to the mute toggle. Unlike the exit chord this fires
* immediately: muting is the kind of thing you want to have already happened, and the on-screen
* indicator makes an accidental toggle self-evident. The buttons still go to the host the
* chord adds a meaning to them rather than swallowing them, exactly as the exit chord does.
*/
var onMicChord: (() -> Unit)? = null
private val mainHandler = Handler(Looper.getMainLooper())
/** The pending exit-chord hold timer, or null when the chord isn't currently armed. */
private var pendingExit: Runnable? = null
@@ -108,14 +118,23 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
/**
* One button transition on [slot] the shared body behind [onButton] and an [ExternalPad]'s
* transitions: forward the wire event, track held state, and arm/disarm the exit chord.
* transitions: forward the wire event, track held state, arm/disarm the exit chord, and fire
* the mic-mute chord ([MIC_CHORD]).
*/
private fun slotButton(slot: Slot, bit: Int, down: Boolean, send: Boolean) {
if (down) {
if (send) NativeBridge.nativeSendGamepadButton(handle, bit, true, slot.index)
val wasHeld = slot.held
slot.held = slot.held or bit
// Full chord now held on this pad → start the hold countdown (idempotent while held).
if (slot.held and EXIT_CHORD == EXIT_CHORD) armExit()
// Mic mute, edge-triggered on the button that COMPLETES the chord: a genuine press
// (`wasHeld` lacks the bit, so an auto-repeat DOWN can't re-fire it) of a chord member
// that leaves the whole chord held. Any other button pressed while Select + Y are down
// fails the middle test, so the toggle happens once per chord, not once per press.
if (wasHeld and bit == 0 && bit and MIC_CHORD != 0 && slot.held and MIC_CHORD == MIC_CHORD) {
onMicChord?.invoke()
}
} else {
if (send) NativeBridge.nativeSendGamepadButton(handle, bit, false, slot.index)
slot.held = slot.held and bit.inv()
@@ -351,6 +370,14 @@ class GamepadRouter(context: Context, private val handle: Long, private val sett
*/
const val EXIT_HOLD_MS = 1000L
/**
* Mic-mute chord: Select + Y. Y is deliberately NOT one of [EXIT_CHORD]'s buttons, so no
* way of reaching the exit chord can pass through this one on the way (and vice versa)
* and Select is a menu button rather than a twitch action, which makes the pair unlikely
* to occur inside real play.
*/
const val MIC_CHORD = Gamepad.BTN_BACK or Gamepad.BTN_Y
/** Synthetic slot-key base for [ExternalPad]s — below every real (positive) InputDevice id. */
const val EXTERNAL_ID_BASE = -1000
}
@@ -248,11 +248,12 @@ object NativeBridge {
/**
* Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs.
* Returns 30 doubles (unified stats spec, `design/stats-unification.md`):
* Returns 33 doubles (unified stats spec, `design/stats-unification.md`):
* `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
* bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
* netP50Ms, lostWindow, skippedWindow, fecWindow, framesWindow, dispValid, displayP50Ms,
* e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive]`
* e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive,
* feedP50Ms, codecP50Ms, skippedOverflowWindow]`
* (the flags are 1.0/0.0; indexes 2/3 are the end-to-end capturedecoded headline; 1013
* describe the negotiated video feed bit depth 8/10, CICP primaries/transfer, and the HEVC
* chroma_format_idc 1=4:2:0 / 3=4:4:4; 14/15 are the stage p50s tiling the headline
@@ -263,7 +264,12 @@ object NativeBridge {
* `display` stage from the OnFrameRendered render timestamps when `dispValid` is 1.0 the
* headline becomes the directly-measured capturedisplayed pair at 24/25, tiled by
* `host+network` + `decode` + `display` (23), and when 0.0 the HUD falls back to the
* capturedecoded headline at 2/3 without the `display` term).
* capturedecoded headline at 2/3 without the `display` term; 2629 split the `display`
* term the timeline presenter owns `pace` = decodedrelease, `latch` = releasedisplayed,
* the window's on-glass confirm count, and whether the presenter is active at all; 30/31
* split `decode` (15) the same way `feed` = receivedqueued (hand-off + input-slot wait),
* `codec` = queueddecoded, the decoder's own time; 32 is the parked-AU overflow subset of
* `skipped` (19), i.e. the decoder falling behind rather than benign newest-wins pacing).
* Poll ~1 Hz; each call resets the measurement window.
*/
external fun nativeVideoStats(handle: Long): DoubleArray?
@@ -288,15 +294,53 @@ object NativeBridge {
external fun nativeStopAudio(handle: Long)
/**
* Start mic uplink: AAudio input Opus (48 kHz stereo, 20 ms) host (`send_mic` / 0xCB), all in
* Rust. No-op if already running. The caller MUST hold RECORD_AUDIO; otherwise the AAudio input
* stream fails to open and the rest of the session keeps streaming.
* Start mic uplink: AAudio input Opus (48 kHz mono, 10 ms) host (`send_mic` / 0xCB), all in
* Rust. [echoCancel] opens the capture under the VoiceCommunication preset (the HAL's own echo
* canceller / noise suppressor) and allocates an audio session id; the return value is that id
* (`> 0`) so the caller can attach the Java [android.media.audiofx.AcousticEchoCanceler] /
* [android.media.audiofx.NoiseSuppressor] as a backstop `0` when none was allocated
* (echoCancel off, the device refused the preset and the open fell back to the plain path, or
* the mic failed entirely). No-op if already running (returns the running capture's id). The
* caller MUST hold RECORD_AUDIO; otherwise the AAudio input stream fails to open and the rest
* of the session keeps streaming.
*/
external fun nativeStartMic(handle: Long)
external fun nativeStartMic(handle: Long, echoCancel: Boolean): Int
/** Stop + join the mic thread and close the AAudio input stream. No-op on `0`. */
/**
* Stop + join the mic thread and close the AAudio input stream. No-op on `0`. Leaves the
* session's mute state ([nativeSetMicMuted]) alone a surface recreate stops and restarts the
* mic, and a user who muted must stay muted through it.
*/
external fun nativeStopMic(handle: Long)
/**
* Mute/unmute the mic uplink mid-stream. Muting does NOT stop the capture: the AAudio input
* stream, the input preset it settled on and its primed buffers stay as they are, and the
* encode loop drops each 10 ms frame instead of encoding + sending it so room audio is never
* encoded and nothing goes on the wire, while a toggle costs an atomic store and takes effect
* on the next 10 ms boundary (a stop/start would re-run the preset fallback ladder and re-prime
* buffers every time).
*
* Sticky for the SESSION the flag lives on the handle, not on the capture so the mic
* restart a surface recreate performs comes back muted, with no window for an unmuted frame to
* escape; a fresh session always starts unmuted. Nothing here is persisted. No-op on `0`.
* Cheap (one atomic store); UI-safe.
*
* One honest consequence of keeping the stream open: the platform's own recording indicator
* stays lit while muted, because the mic really is still open. What stops is the encode and the
* send no captured audio leaves the process.
*/
external fun nativeSetMicMuted(handle: Long, muted: Boolean)
/**
* Is a mic capture actually RUNNING i.e. did [nativeStartMic] open a stream, and has
* [nativeStopMic] not been called since? Offer the in-stream mute control on THIS rather than
* on the user's setting: a device that refused every AAudio input rung (or a missing
* RECORD_AUDIO grant) then shows no control instead of a lie about a mic being heard. `false`
* on a `0` handle. Cheap; UI-safe.
*/
external fun nativeMicActive(handle: Long): Boolean
// ---- Input: Kotlin captures, Rust forwards to the host (send_input) ----
/** Relative mouse move; dx/dy are device-pixel deltas (screen +y down). */
@@ -100,6 +100,34 @@ object VideoDecoders {
}
}
/**
* One-line per-mime probe readout for the connect log (`adb logcat -s pf.caps`): which
* decoder each advertised mime resolves to and whether it declares `FEATURE_PartialFrame`
* the P2 slice-pipeline gate that is otherwise invisible until a stream behaves differently.
*/
fun capsReport(): String {
val mimes = buildList {
add("video/avc")
add("video/hevc")
if (decodableCodecBits() and 4 != 0) add("video/av01")
}
val infos = runCatching { MediaCodecList(MediaCodecList.REGULAR_CODECS).codecInfos }
.getOrNull() ?: return "codec list unavailable"
return mimes.joinToString(" ") { mime ->
val pick = pickDecoder(mime)?.name
val partial = pick?.let { p ->
infos.firstOrNull { it.name == p }?.let { info ->
runCatching {
info.getCapabilitiesForType(mime)
.isFeatureSupported(CodecCapabilities.FEATURE_PartialFrame)
}.getOrNull()
}
}
"${mime.removePrefix("video/")}=${pick ?: "platform-default"}" +
" partialFrame=${partial ?: "?"}"
}
}
fun pickDecoder(mime: String): DecoderChoice? {
if (mime.isEmpty()) return null
val infos = runCatching { MediaCodecList(MediaCodecList.REGULAR_CODECS).codecInfos }
+54 -16
View File
@@ -16,7 +16,7 @@ use std::time::{Duration, Instant};
use super::display::{
apply_hdr_dataspace, install_render_callback, release_render_callback, DisplayTracker,
};
use super::latency::{note_decoded_pts, now_realtime_ns, take_flags};
use super::latency::{note_decoded_pts, now_realtime_ns, take_flags, take_stamp};
use super::presenter::{presenter_disabled_by_sysprop, PresentMeter, PresentPriority, Presenter};
use super::setup::{
android_hdr_static_info, boost_hot_threads, boost_thread_priority, codec_mime,
@@ -280,6 +280,10 @@ pub(super) fn run_async(
let mut oversized_dropped: u64 = 0;
// Slice-progressive continuity ledger (see `PartFeed`).
let mut part_open: Option<PartFeed> = None;
// Queued-instant stamps (pts → realtime ns at the AU's LAST piece entering the codec) — the
// P3 decode-split ledger: `feed` = received→queued, `codec` = queued→decoded. Always on
// (one vDSO clock read per AU); consumed by `present_ready`.
let mut queued_stamps: VecDeque<(u64, i128)> = VecDeque::new();
// Freeze-until-reanchor gate (see the sync loop for the rationale). Armed on a frame-index gap
// (the feeder's Au verdict), a parked-AU overflow drop, a dropped-count climb, or a recoverable
// codec error; `recovery_flags` carries each AU's user_flags from `dispatch_event` (feed) to
@@ -349,7 +353,7 @@ pub(super) fn run_async(
p.on_vsync();
}
}
stats.note_skipped(aus_dropped); // parked-AU overflow drops are client-side skips too
stats.note_skipped_overflow(aus_dropped); // parked-AU overflow: skips, flagged as such
if fmt_dirty {
apply_hdr_dataspace(&codec, &window, &mut applied_ds);
}
@@ -361,6 +365,7 @@ pub(super) fn run_async(
&mut fed,
&mut oversized_dropped,
&mut part_open,
&mut queued_stamps,
&mut gate,
);
let had_output = !ready.is_empty();
@@ -372,6 +377,8 @@ pub(super) fn run_async(
&mut ready,
&stats,
&in_flight,
&mut queued_stamps,
&meter,
clock_offset.load(Ordering::Relaxed),
&tracker,
&mut presenter,
@@ -762,6 +769,7 @@ pub(super) struct PartFeed {
/// [`BUFFER_FLAG_PARTIAL_FRAME`] except the AU's last, all at the AU's pts. `part_open` is the
/// continuity ledger — any break (gap, orphan, oversize) abandons the AU per [`PartFeed::pts_us`]'s
/// close contract and re-syncs at the next `first`.
#[allow(clippy::too_many_arguments)] // one call site; the split ledger threads through like the gate
fn feed_ready(
codec: &MediaCodec,
client: &NativeClient,
@@ -770,6 +778,7 @@ fn feed_ready(
fed: &mut u64,
oversized_dropped: &mut u64,
part_open: &mut Option<PartFeed>,
queued_stamps: &mut VecDeque<(u64, i128)>,
gate: &mut ReanchorGate,
) {
while !pending_aus.is_empty() && !free_inputs.is_empty() {
@@ -859,9 +868,15 @@ fn feed_ready(
}
} else {
// `fed` counts ACCESS UNITS toward the HUD's fed/decoded balance — the closing
// piece (or a whole AU) bumps it.
// piece (or a whole AU) bumps it. The queued stamp marks the same instant (the AU
// is fully in the codec's hands): the P3 decode split measures `codec` from here,
// so a slice-progressive head start shows up as codec-pure shrink.
if last {
*fed += 1;
queued_stamps.push_back((pts_us, now_realtime_ns()));
if queued_stamps.len() > IN_FLIGHT_CAP {
queued_stamps.pop_front(); // stale — codec never echoed it back
}
}
*part_open = if last {
None
@@ -876,7 +891,8 @@ fn feed_ready(
}
}
/// Route the ready outputs toward glass. With the timeline presenter (default): fold each output
/// Route the ready outputs toward glass, recording each one's decode-split + e2e first. With the
/// timeline presenter (default): fold each output
/// through the re-anchor gate in pts order, hand the approved ones to the presenter's store
/// (newest-wins / smoothing FIFO — the actual release happens in `Presenter::pump`, budgeted and
/// timeline-timed), and release withheld concealment unrendered. Legacy (`arrival` sysprop):
@@ -892,6 +908,8 @@ fn present_ready(
ready: &mut Vec<OutputReady>,
stats: &crate::stats::VideoStats,
in_flight: &Mutex<VecDeque<(u64, i128)>>,
queued_stamps: &mut VecDeque<(u64, i128)>,
meter: &PresentMeter,
clock_offset: i64,
tracker: &DisplayTracker,
presenter: &mut Option<Presenter>,
@@ -903,22 +921,42 @@ fn present_ready(
if ready.is_empty() {
return;
}
// Pair each output's decode stage (feeds the ABR decode signal always; the HUD histogram only
// while visible) — both consume the receipt map, so enter for either.
if stats.enabled() || measure_decode {
// Pair each output's decode stage (the ABR decode signal + the HUD histogram consume the
// receipt map; the P3 split's codec-pure half needs only the queued stamp, so it records
// even with both off — that keeps the 1 Hz pf.present mirror HUD-off readable).
{
let want_stage = stats.enabled() || measure_decode;
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
for o in ready.iter() {
note_decoded_pts(
client,
measure_decode,
stats,
&mut g,
clock_offset,
o.pts_us,
o.decoded_ns,
);
let received_ns = if want_stage {
note_decoded_pts(
client,
measure_decode,
stats,
&mut g,
clock_offset,
o.pts_us,
o.decoded_ns,
)
} else {
None
};
let queued = take_stamp(queued_stamps, o.pts_us);
let codec_us = queued.map(|q| ((o.decoded_ns - q).max(0) / 1000) as u64);
let feed_us = match (queued, received_ns) {
(Some(q), Some(r)) => Some(((q - r).max(0) / 1000) as u64),
_ => None,
};
// Always-on e2e for the 1 Hz pf.present mirror (same formula + clamp as the HUD's
// capture→decoded headline in `note_decoded_pts`).
let e2e_ns = o.decoded_ns + clock_offset as i128 - o.pts_us as i128 * 1000;
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
meter.note_decode(feed_us, codec_us, e2e_us);
if let Some(c) = codec_us {
stats.note_decode_split(feed_us, c);
}
}
}
// Fold EVERY output through the gate in pts (== decode) order — even the ones newest-wins discards —
+21 -2
View File
@@ -20,7 +20,8 @@ pub(super) fn now_realtime_ns() -> i128 {
/// entries older than it are evicted (decode order == input order here — low-latency, no
/// B-frames — so anything before it was dropped inside the codec or stamped before a flush).
/// `decoded_ns` is the availability instant: the dequeue (sync loop) or the output callback's
/// stamp (async loop).
/// stamp (async loop). Returns the receipt stamp it paired (if any) so the caller can split the
/// `decode` stage further (feed wait vs codec-pure) without re-walking the map.
pub(super) fn note_decoded_pts(
client: &NativeClient,
measure_decode: bool,
@@ -29,7 +30,7 @@ pub(super) fn note_decoded_pts(
clock_offset: i64,
pts_us: u64,
decoded_ns: i128,
) {
) -> Option<i128> {
// Pair the echoed pts back to its receipt stamp, evicting stale (older) entries as we go.
let mut received_ns = None;
while let Some(&(p, r)) = in_flight.front() {
@@ -61,6 +62,24 @@ pub(super) fn note_decoded_pts(
let e2e_us = (e2e_ns > 0 && e2e_ns < 10_000_000_000).then_some((e2e_ns / 1000) as u64);
stats.note_decoded(e2e_us, decode_us);
}
received_ns
}
/// The queued-instant stamp for a decoded output, keyed by the echoed `presentationTimeUs` — the
/// same monotonic evict-as-you-go pairing as [`take_flags`], over an `(pts_us, realtime_ns)` map
/// (the feed side stamps each AU as its last piece enters the codec). A miss returns `None` —
/// the split is simply not recorded for that frame.
pub(super) fn take_stamp(map: &mut VecDeque<(u64, i128)>, pts_us: u64) -> Option<i128> {
while let Some(&(p, t)) = map.front() {
if p > pts_us {
break; // future frame — leave it for its own output buffer
}
map.pop_front();
if p == pts_us {
return Some(t);
}
}
None
}
/// The AU `user_flags` for a decoded output, keyed by the echoed `presentationTimeUs`. Recovery
+67 -5
View File
@@ -126,6 +126,15 @@ pub(super) struct PresentMeter {
struct PresentMeterInner {
latch_us: Vec<u64>,
displays: u64,
/// The `decode` stage's feed split, received→queued µs (P3 science: hand-off + input-slot
/// wait). Empty when no receipt stamp matched (HUD off and ABR not measuring decode).
feed_us: Vec<u64>,
/// The codec-pure half, queued→decoded µs, measured from the AU's LAST piece — always on,
/// so a HUD-off logcat A/B still reads the decoder's own time.
codec_us: Vec<u64>,
/// Capture→decoded end-to-end µs (skew-corrected, clamped) — always on for the same reason:
/// the wireless A/B's headline without having to reach the on-screen HUD.
e2e_us: Vec<u64>,
}
impl PresentMeter {
@@ -134,6 +143,9 @@ impl PresentMeter {
inner: Mutex::new(PresentMeterInner {
latch_us: Vec::with_capacity(256),
displays: 0,
feed_us: Vec::with_capacity(256),
codec_us: Vec::with_capacity(256),
e2e_us: Vec::with_capacity(256),
}),
}
}
@@ -152,14 +164,51 @@ impl PresentMeter {
}
}
fn drain(&self) -> (Vec<u64>, u64) {
/// One decoded frame's always-on measurements: the `decode`-stage split (feed =
/// received→queued when a receipt stamp matched; codec = queued→decoded when the queued
/// stamp did) and the capture→decoded end-to-end, µs. Decode thread; poison-proof.
pub(super) fn note_decode(
&self,
feed_us: Option<u64>,
codec_us: Option<u64>,
e2e_us: Option<u64>,
) {
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(f) = feed_us {
if g.feed_us.len() < 4096 {
g.feed_us.push(f);
}
}
if let Some(c) = codec_us {
if g.codec_us.len() < 4096 {
g.codec_us.push(c);
}
}
if let Some(e) = e2e_us {
if g.e2e_us.len() < 4096 {
g.e2e_us.push(e);
}
}
}
#[allow(clippy::type_complexity)] // one caller unpacks it in place; a struct would be noise
fn drain(&self) -> (Vec<u64>, u64, Vec<u64>, Vec<u64>, Vec<u64>) {
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let displays = g.displays;
g.displays = 0;
(std::mem::take(&mut g.latch_us), displays)
(
std::mem::take(&mut g.latch_us),
displays,
std::mem::take(&mut g.feed_us),
std::mem::take(&mut g.codec_us),
std::mem::take(&mut g.e2e_us),
)
}
}
@@ -386,7 +435,9 @@ impl Presenter {
/// `released` (to glass) / `displays` (OnFrameRendered confirms) / `paced` (policy drops) /
/// `noBudget` (waits on the closed budget) / `forced` (stale force-opens — 0 when healthy) /
/// `qDry` (FIFO underflows) / `pace` (decoded→release) / `latch` (release→displayed) /
/// `vsync` (the measured panel period).
/// `feed`+`codec` (the decode stage split: received→queued hand-off/slot wait + the
/// codec-pure queued→decoded time) / `e2e` (capture→decoded, skew-corrected — the wireless
/// A/B headline) / `vsync` (the measured panel period).
///
/// Returns this window's CIRCULAR latch statistics `(vector-mean latch ns mod panel period,
/// coherence ‰)` when a window actually flushed — the phase-lock reporter's v2 error signal
@@ -400,11 +451,14 @@ impl Presenter {
return None;
}
self.last_flush = Instant::now();
let (latch, displays) = meter.drain();
let (latch, displays, feed, codec, e2e) = meter.drain();
if self.released == 0 && displays == 0 {
return None; // idle stream — nothing worth a line
}
let (pace_p50, pace_max) = p50_max_ms(std::mem::take(&mut self.pace_us));
let (feed_p50, feed_max) = p50_max_ms(feed);
let (codec_p50, codec_max) = p50_max_ms(codec);
let (e2e_p50, e2e_max) = p50_max_ms(e2e);
let circ = clock.and_then(|c| {
punktfunk_core::phase::circular_latch(&latch, c.panel_period_ns().max(c.period_ns()))
});
@@ -416,7 +470,9 @@ impl Presenter {
log::info!(
target: "pf.present",
"released={} displays={} paced={} noBudget={} forced={} qDry={} \
paceMs p50={:.2} max={:.2} latchMs p50={:.2} max={:.2} circ={:.2}ms coh={} \
paceMs p50={:.2} max={:.2} latchMs p50={:.2} max={:.2} \
feedMs p50={:.2} max={:.2} codecMs p50={:.2} max={:.2} \
e2eMs p50={:.2} max={:.2} circ={:.2}ms coh={} \
vsyncMs={:.2} panelMs={:.2}",
self.released,
displays,
@@ -428,6 +484,12 @@ impl Presenter {
pace_max,
latch_p50,
latch_max,
feed_p50,
feed_max,
codec_p50,
codec_max,
e2e_p50,
e2e_max,
circ.map(|(m, _)| m as f64 / 1e6).unwrap_or(0.0),
circ.map(|(_, c)| c).unwrap_or(0),
period_ms,
+178 -46
View File
@@ -1,16 +1,22 @@
//! Android microphone uplink (android-only): capture mic PCM via AAudio (LowLatency **input**),
//! Opus-encode 20 ms stereo frames, and push them to the host over the connector's mic plane
//! Opus-encode 10 ms mono frames, and push them to the host over the connector's mic plane
//! (`send_mic` → 0xCB datagram). The mirror of [`crate::audio`] in reverse: AAudio's realtime input
//! callback hands captured interleaved f32 to a channel; a worker thread we own does the Opus
//! encode + send (encoding is too heavy for the realtime callback, exactly as decode is on the
//! playback side). Like the playback path, the realtime callback is allocation-free: captured
//! bursts are copied into pre-allocated buffers from a recycle free-list (pool empty = drop the
//! chunk, never allocate on the capture thread). Format matches the host decoder + the Linux
//! client: 48 kHz **stereo**, 20 ms, Opus VOIP.
//! callback hands captured f32 to a channel; a worker thread we own does the Opus encode + send
//! (encoding is too heavy for the realtime callback, exactly as decode is on the playback side).
//! Like the playback path, the realtime callback is allocation-free: captured bursts are copied
//! into pre-allocated buffers from a recycle free-list (pool empty = drop the chunk, never
//! allocate on the capture thread). Format: 48 kHz **mono**, 10 ms, Opus VOIP with in-band FEC —
//! the host decodes any Opus frame ≤ 120 ms with its stereo decoder (mono packets upmix), so this
//! needs no protocol change; speech gains nothing from stereo, and the shorter frame shaves a
//! buffering interval off the uplink.
//!
//! **Mute** is a flag the encode loop reads per 10 ms frame, never a stream teardown: the AAudio
//! input stream, the input-preset ladder it settled on and its primed buffers all survive a
//! mute/unmute untouched, so toggling costs an atomic load and nothing else.
use ndk::audio::{
AudioCallbackResult, AudioDirection, AudioFormat, AudioPerformanceMode, AudioSharingMode,
AudioStream, AudioStreamBuilder,
AudioCallbackResult, AudioDirection, AudioFormat, AudioInputPreset, AudioPerformanceMode,
AudioSharingMode, AudioStream, AudioStreamBuilder, SessionId,
};
use punktfunk_core::client::NativeClient;
use std::collections::VecDeque;
@@ -20,31 +26,57 @@ use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, SyncSender, TryS
use std::sync::Arc;
use std::time::{Duration, SystemTime, UNIX_EPOCH};
const CHANNELS: usize = 2;
const CHANNELS: usize = 1;
const SAMPLE_RATE: i32 = 48_000;
/// 20 ms per channel @ 48 kHz — the Linux client's frame; the host accepts ≤ 120 ms.
const FRAME_SAMPLES: usize = 960;
/// 10 ms per channel @ 48 kHz — half the desktop clients' 20 ms frame, trading a little Opus
/// header overhead for one less buffered interval; the host accepts ≤ 120 ms.
const FRAME_SAMPLES: usize = 480;
/// Captured-chunk hand-off depth (each ~ one burst); drops on overflow (best-effort uplink).
/// Bursts are sized in frames, so the wall-time depth is unchanged by the stereo→mono move.
const RING_CHUNKS: usize = 64;
/// Free-list buffer capacity, in interleaved f32 samples: comfortably above a LowLatency input
/// burst (typically ≤ ~480 frames). A device with larger bursts costs each buffer a one-time grow
/// on the capture thread, after which the steady state is allocation-free again.
const CHUNK_CAP_SAMPLES: usize = 1920; // 20 ms stereo
/// Opus VOIP target bitrate (speech; tunable).
const MIC_BITRATE: i32 = 64_000;
/// burst (typically ≤ ~480 frames — mono, so samples = frames). A device with larger bursts costs
/// each buffer a one-time grow on the capture thread, after which the steady state is
/// allocation-free again.
const CHUNK_CAP_SAMPLES: usize = 960; // 20 ms mono — the same wall-time as the old stereo value
/// Opus VOIP target bitrate (mono speech; tunable).
const MIC_BITRATE: i32 = 48_000;
/// Encode-side self-heal threshold, in queued 10 ms frames (~60 ms): waking to more than this
/// means the uplink stalled — and because the capture callback drops the NEWEST chunk when the
/// channel is full, a stall otherwise converts to standing mic delay that never drains (real-time
/// playback host-side never makes time back up). Skip to the newest few frames instead.
const BACKLOG_MAX_FRAMES: usize = 6;
/// What a self-heal keeps: ~20 ms of the freshest audio (one audible blip, live again).
const BACKLOG_KEEP_FRAMES: usize = 2;
/// Owned by [`crate::session::SessionHandle`]: the live AAudio input stream + the encode thread.
pub struct MicCapture {
_stream: AudioStream, // dropping it stops + closes the AAudio input stream
/// The audio-session id AAudio allocated (`> 0`) when echo cancellation asked for one — the
/// hook Kotlin hangs the Java `AcousticEchoCanceler`/`NoiseSuppressor` on. `0` = none.
session_id: i32,
shutdown: Arc<AtomicBool>,
join: Option<std::thread::JoinHandle<()>>,
}
impl MicCapture {
/// Open AAudio (LowLatency, 48 kHz/stereo/f32) for **input** with a realtime callback that
/// forwards captured PCM to a channel, then spawn the Opus encode + uplink thread. `None` on
/// failure (the caller leaves the rest of the session streaming).
pub fn start(client: Arc<NativeClient>) -> Option<MicCapture> {
/// Open AAudio (LowLatency, 48 kHz/mono/f32) for **input** with a realtime callback that
/// forwards captured PCM to a channel, then spawn the Opus encode + uplink thread. With
/// `echo_cancel` the stream opens under the `VoiceCommunication` input preset the HAL's own
/// echo canceller / noise suppressor on the capture path (the default `VoiceRecognition`
/// preset deliberately bypasses them, which is why the host used to hear its own stream back
/// from a speaker-playing phone) — and allocates an audio session id for Kotlin's Java-effect
/// backstop. `None` on failure (the caller leaves the rest of the session streaming).
///
/// `muted` is the SESSION's live mic-mute flag (owned by `SessionHandle`, not by this capture),
/// honoured per frame by [`encode_loop`]. Sharing it rather than owning it is what makes mute
/// survive the mic stop/start a surface recreate performs — and means a capture started while
/// muted never encodes its first frame, so there is no window for one to escape.
pub fn start(
client: Arc<NativeClient>,
echo_cancel: bool,
muted: Arc<AtomicBool>,
) -> Option<MicCapture> {
let captured = Arc::new(AtomicU64::new(0));
// Chunks discarded on the capture thread (free-list empty / encoder lagging); logged
// throttled from the encode worker.
@@ -52,7 +84,9 @@ impl MicCapture {
// One open attempt at a given sharing mode (same pattern as [`crate::audio`]: `open_stream`
// consumes the builder AND the callback, so each try rebuilds the channels it captures).
let try_open = |sharing: AudioSharingMode| -> ndk::audio::Result<(
let try_open = |sharing: AudioSharingMode,
voice: bool|
-> ndk::audio::Result<(
AudioStream,
Receiver<Vec<f32>>,
SyncSender<Vec<f32>>,
@@ -99,13 +133,25 @@ impl MicCapture {
AudioCallbackResult::Continue
};
let stream = AudioStreamBuilder::new()?
// NOTE: no `.frames_per_data_callback(...)`: AAudio's own docs call leaving it unset
// the lowest-latency path (the callback then runs at the device's optimal burst,
// while pinning a size inserts an adaptation buffer), and the encode side re-chunks
// to 10 ms frames regardless of how the bursts arrive.
let mut builder = AudioStreamBuilder::new()?
.direction(AudioDirection::Input)
.sample_rate(SAMPLE_RATE)
.channel_count(CHANNELS as i32)
.format(AudioFormat::PCM_Float)
.performance_mode(AudioPerformanceMode::LowLatency)
.sharing_mode(sharing)
.sharing_mode(sharing);
if voice {
// VoiceCommunication routes the capture through the HAL's AEC/NS; the allocated
// session id (`None` = allocate) is what Kotlin attaches the Java effects to.
builder = builder
.input_preset(AudioInputPreset::VoiceCommunication)
.session_id(None);
}
let stream = builder
.data_callback(Box::new(callback))
.error_callback(Box::new(|_s, e| {
log::warn!("mic: AAudio error (device reroute/disconnect?): {e:?}");
@@ -114,21 +160,52 @@ impl MicCapture {
Ok((stream, rx, free_tx))
};
// Exclusive first — MMAP-exclusive is AAudio's lowest-latency path — falling back to Shared
// when the device refuses (no MMAP, mic claimed, …). The started-log below prints the mode
// the device actually GRANTED (`share=`).
let (stream, rx, free_tx) = match try_open(AudioSharingMode::Exclusive) {
Ok(opened) => opened,
Err(e) => {
log::info!("mic: Exclusive open failed ({e}) — retrying Shared");
match try_open(AudioSharingMode::Shared) {
Ok(opened) => opened,
Err(e) => {
log::error!("mic: open_stream (RECORD_AUDIO granted?): {e}");
return None;
}
// Exclusive first — MMAP-exclusive is AAudio's lowest-latency path — falling back to
// Shared when the device refuses (no MMAP, mic claimed, …); and each sharing mode with
// the voice preset before without it, because some HALs reject VoiceCommunication (or a
// session id) outright and a mic without echo cancellation still beats no mic. The
// ladder's last rungs are exactly the preset-less open this always did. The started-log
// below prints what the device actually GRANTED (`share=`/`session=`).
let attempts: &[(AudioSharingMode, bool)] = if echo_cancel {
&[
(AudioSharingMode::Exclusive, true),
(AudioSharingMode::Shared, true),
(AudioSharingMode::Exclusive, false),
(AudioSharingMode::Shared, false),
]
} else {
&[
(AudioSharingMode::Exclusive, false),
(AudioSharingMode::Shared, false),
]
};
let mut opened = None;
for &(sharing, voice) in attempts {
match try_open(sharing, voice) {
Ok(o) => {
opened = Some(o);
break;
}
Err(e) => log::info!(
"mic: open {sharing:?}{} failed ({e}) — trying the next fallback",
if voice { "+VoiceCommunication" } else { "" },
),
}
}
let (stream, rx, free_tx) = match opened {
Some(o) => o,
None => {
log::error!("mic: open_stream (RECORD_AUDIO granted?): every mode refused");
return None;
}
};
// The session id AAudio actually allocated (only a voice rung asks for one): `> 0` is the
// handle Kotlin hangs the Java AcousticEchoCanceler/NoiseSuppressor off as the HAL
// preset's backstop; `0` = none, nothing to attach.
let session_id = match stream.session_id() {
SessionId::Allocated(id) => id.get(),
SessionId::None => 0,
};
if let Err(e) = stream.request_start() {
@@ -136,7 +213,7 @@ impl MicCapture {
return None;
}
log::info!(
"mic: AAudio input started rate={} ch={} fmt={:?} share={:?}",
"mic: AAudio input started rate={} ch={} fmt={:?} share={:?} session={session_id}",
stream.sample_rate(),
stream.channel_count(),
stream.format(),
@@ -147,15 +224,21 @@ impl MicCapture {
let sd = shutdown.clone();
let join = std::thread::Builder::new()
.name("pf-mic".into())
.spawn(move || encode_loop(client, rx, free_tx, sd, captured, dropped))
.spawn(move || encode_loop(client, rx, free_tx, sd, muted, captured, dropped))
.ok();
Some(MicCapture {
_stream: stream,
session_id,
shutdown,
join,
})
}
/// The audio-session id AAudio allocated (`> 0`; see [`MicCapture::start`]), `0` = none.
pub fn session_id(&self) -> i32 {
self.session_id
}
}
impl Drop for MicCapture {
@@ -168,20 +251,29 @@ impl Drop for MicCapture {
}
}
/// Consumer: drain captured f32 → accumulate → Opus `encode_float` 20 ms stereo frames → `send_mic`.
/// Consumer: drain captured f32 → accumulate → Opus `encode_float` 10 ms mono frames → `send_mic`.
/// Drained chunk buffers go back to the callback's free-list; the encode scratch is reused across
/// frames (only the packet Vec handed to `send_mic` is allocated per frame — it's sent away owned).
///
/// While `muted` is set a formed frame is dropped instead of encoded (see the frame loop) — the
/// capture side keeps running exactly as it does unmuted, so nothing about the stream, its ring or
/// its backlog behaviour changes across a toggle.
fn encode_loop(
client: Arc<NativeClient>,
rx: Receiver<Vec<f32>>,
free_tx: SyncSender<Vec<f32>>,
shutdown: Arc<AtomicBool>,
muted: Arc<AtomicBool>,
captured: Arc<AtomicU64>,
dropped: Arc<AtomicU64>,
) {
// Fold this Opus-encode/uplink thread into the client's hot-thread set so the ADPF session the
// decode thread opens keeps mic encode on a fast core too (the playback side's decode_loop
// does the same). No-op below API 33.
client.register_hot_thread();
let mut enc = match opus::Encoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
opus::Channels::Mono,
opus::Application::Voip,
) {
Ok(e) => e,
@@ -191,13 +283,21 @@ fn encode_loop(
}
};
let _ = enc.set_bitrate(opus::Bitrate::Bits(MIC_BITRATE));
// Speech tuning: complexity 5 roughly halves encode cost for no audible loss at this rate,
// and in-band FEC at an assumed 10% loss lets the host's decoder reconstruct a dropped
// datagram from its successor instead of playing a hole (the uplink is fire-and-forget).
let _ = enc.set_complexity(5);
let _ = enc.set_inband_fec(true);
let _ = enc.set_packet_loss_perc(10);
let frame = FRAME_SAMPLES * CHANNELS;
let mut ring: VecDeque<f32> = VecDeque::with_capacity(frame * 4);
let mut pcm = vec![0f32; frame]; // reusable encode scratch (one 20 ms frame)
let mut out = vec![0u8; 4000]; // max Opus packet for a 20 ms frame fits easily
let mut pcm = vec![0f32; frame]; // reusable encode scratch (one 10 ms frame)
let mut out = vec![0u8; 4000]; // max Opus packet for a 10 ms frame fits easily
let mut seq: u32 = 0;
let mut sent: u64 = 0;
let mut stale: u64 = 0; // frames shed by the backlog self-heal (see BACKLOG_MAX_FRAMES)
let mut muted_frames: u64 = 0; // frames dropped unencoded because the user muted
let mut peak = 0f32; // loudest |sample| since the last log — tells speech from silence
while !shutdown.load(Ordering::Relaxed) {
@@ -207,11 +307,41 @@ fn encode_loop(
// callback's free-list (dropped only if the pool is momentarily full).
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
// Drain whatever else queued while we were away, so a post-stall backlog lands as
// ONE lump the self-heal below can size up — chunk-at-a-time it would be encoded
// (and inflicted on the host as standing delay) before it ever looked deep.
while let Ok(mut chunk) = rx.try_recv() {
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
}
}
Err(RecvTimeoutError::Timeout) => continue, // wake to re-check shutdown
Err(RecvTimeoutError::Disconnected) => break,
}
// Self-heal the latency ratchet: a stall (scheduler hiccup, a slow send) queues stale
// audio, and every ms of it would ride the stream as mic delay for the rest of the
// session. Jump to the newest ~20 ms (one audible blip), counting the shed.
if ring.len() > BACKLOG_MAX_FRAMES * frame {
let excess = ring.len() - BACKLOG_KEEP_FRAMES * frame;
ring.drain(..excess);
stale += (excess / frame) as u64;
}
while ring.len() >= frame {
// Muted: drop the frame at the last point before it would become an Opus packet —
// room audio is never encoded and nothing goes on the wire. `seq` does NOT advance:
// it numbers the datagrams the host de-jitters, and that side reads a seq jump as
// loss (conceal + a counted gap) where a mute is a pause. Freezing it means the
// frame after an unmute continues the chain, which is what the host's own
// `reset_stream` doc calls for and what the desktop uplink does. (Encoding silence
// instead would keep a pointless uplink and a host-side ring alive for the whole
// mute.) `peak` is the loudest sample the UPLINK carried since the last log, so a
// dropped frame resets rather than raises it.
if muted.load(Ordering::Relaxed) {
ring.drain(..frame);
muted_frames += 1;
peak = 0.0;
continue;
}
for (dst, src) in pcm.iter_mut().zip(ring.drain(..frame)) {
*dst = src;
}
@@ -227,9 +357,10 @@ fn encode_loop(
let _ = client.send_mic(seq, pts, out[..len].to_vec());
seq = seq.wrapping_add(1);
sent += 1;
if sent % 250 == 0 {
if sent % 500 == 0 {
log::info!(
"mic: sent={sent} captured_frames={} dropped_chunks={} peak={peak:.3}",
"mic: sent={sent} captured_frames={} dropped_chunks={} \
stale_frames={stale} muted_frames={muted_frames} peak={peak:.3}",
captured.load(Ordering::Relaxed),
dropped.load(Ordering::Relaxed),
);
@@ -241,7 +372,8 @@ fn encode_loop(
}
}
log::info!(
"mic: stopped (sent={sent} captured_frames={} dropped_chunks={})",
"mic: stopped (sent={sent} captured_frames={} dropped_chunks={} stale_frames={stale} \
muted_frames={muted_frames})",
captured.load(Ordering::Relaxed),
dropped.load(Ordering::Relaxed),
);
+46 -3
View File
@@ -83,6 +83,28 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSetLowLaten
punktfunk_core::transport::set_dscp_default(enabled != 0);
}
/// `debug.punktfunk.force_parts` = 1: arm slice-progressive parts delivery even when the
/// Kotlin `FEATURE_PartialFrame` probe said no — the rebuild-free on-glass experiment for a
/// decoder that may accept `BUFFER_FLAG_PARTIAL_FRAME` without declaring the feature (the NP3's
/// c2.qti decoders declare nothing). Android-only; everywhere else the probe verdict stands.
#[cfg(target_os = "android")]
fn force_parts_sysprop() -> bool {
let mut buf = [0u8; 92]; // PROP_VALUE_MAX
// SAFETY: __system_property_get with a valid name + PROP_VALUE_MAX buffer is always safe.
let n = unsafe {
libc::__system_property_get(
c"debug.punktfunk.force_parts".as_ptr(),
buf.as_mut_ptr().cast(),
)
};
n > 0 && std::str::from_utf8(&buf[..n as usize]).unwrap_or("").trim() == "1"
}
#[cfg(not(target_os = "android"))]
fn force_parts_sysprop() -> bool {
false
}
/// `NativeBridge.nativeConnect(host, port, w, h, hz, certPem, keyPem, pinHex, bitrateKbps,
/// compositorPref, gamepadPref, hdrEnabled, audioChannels, preferredCodec, timeoutMs, launch,
/// deviceName): Long`.
@@ -155,6 +177,24 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
} else {
Some((cert, key))
};
// Slice-progressive parts, by decoder truth (Kotlin's FEATURE_PartialFrame probe) — with a
// sysprop escape hatch for the on-glass science question the probe can't answer: does the
// decoder ACTUALLY choke on BUFFER_FLAG_PARTIAL_FRAME input, or does it merely not declare
// the feature? (`adb shell setprop debug.punktfunk.force_parts 1` + stream restart; a codec
// that can't take parts errors recoverably and the reanchor gate + keyframe path recovers.)
let force_parts = force_parts_sysprop();
let frame_parts = frame_parts_ok != 0 || force_parts;
// The connect-time capability readout (`adb logcat -s pf.caps`): the P2 slice pipeline is
// inert client-side unless BOTH probes pass — this line is the one place that says which.
log::info!(
target: "pf.caps",
"decoder caps: multi_slice={} partial_frame={}{} hdr={} codec_bits={:#x}",
multi_slice_ok != 0,
frame_parts_ok != 0,
if force_parts { " (FORCED by sysprop)" } else { "" },
hdr_enabled != 0,
video_codecs,
);
let pin: Option<[u8; 32]> = if pin_hex.is_empty() {
None
} else {
@@ -230,9 +270,10 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
// should say what the client does).
punktfunk_core::quic::CLIENT_CAP_PHASE_LOCK,
// Slice-progressive delivery, by decoder truth (Kotlin probes FEATURE_PartialFrame on
// every decoder this device would use): AU prefixes then arrive as `Frame::part`
// pieces and the decode loop feeds them with BUFFER_FLAG_PARTIAL_FRAME.
frame_parts_ok != 0,
// every decoder this device would use; `debug.punktfunk.force_parts` overrides for the
// on-glass experiment): AU prefixes then arrive as `Frame::part` pieces and the decode
// loop feeds them with BUFFER_FLAG_PARTIAL_FRAME.
frame_parts,
launch, // a store-qualified library id to boot into a game, or None for the desktop
device_name, // Kotlin's Build.MODEL — the host's approval-list / trust-store label
pin, // Some → Crypto on host-fp mismatch
@@ -250,6 +291,8 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
audio: Mutex::new(None),
#[cfg(target_os = "android")]
mic: Mutex::new(None),
// A fresh session is never muted (mute is per-session UI state, not a setting).
mic_muted: Arc::new(std::sync::atomic::AtomicBool::new(false)),
};
Box::into_raw(Box::new(handle)) as jlong
}
@@ -61,6 +61,13 @@ pub(crate) struct SessionHandle {
audio: Mutex<Option<crate::audio::AudioPlayback>>,
#[cfg(target_os = "android")]
mic: Mutex<Option<crate::mic::MicCapture>>,
/// In-stream mic mute, set via `nativeSetMicMuted` and read per 10 ms frame by the mic's
/// encode loop ([`crate::mic`]). Session-lifetime rather than per-[`crate::mic::MicCapture`]
/// for the same reason the stats gate is: the mic stops and restarts across a surface
/// recreate, and a mute the user set must come back with it — with no window in which the
/// fresh capture could send an unmuted frame. Per session and never persisted: a new session
/// starts unmuted.
pub mic_muted: Arc<AtomicBool>,
}
struct VideoThread {
+98 -15
View File
@@ -177,11 +177,12 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
}
/// `NativeBridge.nativeVideoStats(handle): DoubleArray?` — drain ~1 s of decode stats for the HUD
/// (unified stats spec, `design/stats-unification.md`). Returns 26 doubles
/// (unified stats spec, `design/stats-unification.md`). Returns 33 doubles
/// `[fps, mbps, e2eP50Ms, e2eP95Ms, latValid, skewCorrected, width, height, refreshHz, framesLost,
/// bitDepth, colorPrimaries, colorTransfer, chromaFormatIdc, hostNetP50Ms, decodeP50Ms, hostP50Ms,
/// netP50Ms, lostWindow, skippedWindow, fecWindow, framesWindow, dispValid, displayP50Ms,
/// e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive]`
/// e2eDispP50Ms, e2eDispP95Ms, paceP50Ms, latchP50Ms, presentsWindow, presenterActive,
/// feedP50Ms, codecP50Ms, skippedOverflowWindow]`
/// (the flags are 1.0/0.0; indexes 021 match the previous 22-double layout — 013 the original
/// 14-double one with the latency pair re-based to the end-to-end capture→decoded headline, 14/15
/// the stage p50s tiling it: `host+network` = capture→received, `decode` = received→decoded; 16/17
@@ -198,7 +199,11 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopVideo(
/// term — `pace` = decoded→release (store + glass budget) p50 at 26, `latch` =
/// release→displayed (SurfaceFlinger) p50 at 27, the window's on-glass confirm count at 28
/// (`presents` vs `fps` is the presenter-health pair), and 29 = 1.0 while the timeline presenter
/// is active this session), or `null` when no decode thread is running.
/// is active this session; 30/31 are the `decode` stage's split p50s — `feed` =
/// received→queued (hand-off + input-slot wait) at 30 and `codec` = queued→decoded (codec-pure,
/// from the AU's last piece) at 31, both 0.0 when no sample landed (sync loop); 32 is the
/// parked-AU overflow subset of the window's `skipped` at 19 (decoder fell behind, vs benign
/// newest-wins pacing)), or `null` when no decode thread is running.
/// Poll ~1 Hz from the UI; each call
/// resets the measurement window. Not android-gated — pure `jni` + connector reads, so it links on
/// the host build too (Kotlin only ever calls it on device).
@@ -222,7 +227,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
.drain(h.client.frames_dropped(), h.client.fec_recovered_shards());
let mode = h.client.mode();
let color = h.client.color;
let buf: [f64; 30] = [
let buf: [f64; 33] = [
snap.fps,
snap.mbps,
snap.e2e_p50_ms,
@@ -270,6 +275,12 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoStats(
snap.latch_p50_ms,
snap.presents as f64,
if h.stats.presenter_active() { 1.0 } else { 0.0 },
// The `decode` stage's split (P3 science): feed = received→queued (hand-off +
// input-slot wait), codec = queued→decoded (codec-pure) — and the parked-AU
// overflow subset of `skipped` (decoder-health vs benign pacing drops).
snap.feed_p50_ms,
snap.codec_p50_ms,
snap.skipped_overflow as f64,
];
let arr = match env.new_double_array(buf.len() as jsize) {
Ok(a) => a,
@@ -390,33 +401,49 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopAudio(
})
}
/// `NativeBridge.nativeStartMic(handle)` — start mic capture (AAudio input → Opus → host `send_mic`).
/// No-op if already running or on a `0` handle. Caller MUST hold RECORD_AUDIO; a failure (e.g. no
/// permission) leaves the rest of the session streaming.
/// `NativeBridge.nativeStartMic(handle, echoCancel): Int` — start mic capture (AAudio input →
/// Opus → host `send_mic`). `echoCancel` opens the capture under the `VoiceCommunication` preset
/// (the HAL's echo canceller / noise suppressor) and allocates an audio session id; the return
/// value is that id (`> 0`), so Kotlin can attach the Java `AcousticEchoCanceler`/`NoiseSuppressor`
/// as a backstop — `0` when none was allocated (echoCancel off, the preset fell back to the plain
/// open, a `0` handle, or the mic failed entirely). Already running (a surface recreate) returns
/// the running capture's id. Caller MUST hold RECORD_AUDIO; a failure (e.g. no permission) leaves
/// the rest of the session streaming.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartMic(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
echo_cancel: jboolean,
) -> jni::sys::jint {
if handle == 0 {
return;
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let mut guard = h.mic.lock().unwrap();
if guard.is_some() {
return; // already capturing
if let Some(m) = guard.as_ref() {
return m.session_id(); // already capturing — same stream, same session
}
match crate::mic::MicCapture::start(h.client.clone()) {
Some(m) => *guard = Some(m),
None => log::error!("nativeStartMic: mic init failed (RECORD_AUDIO? — session unaffected)"),
// The capture SHARES the session's mute flag, so one started while muted stays muted (and
// sends nothing) from its very first frame — see `SessionHandle::mic_muted`.
match crate::mic::MicCapture::start(h.client.clone(), echo_cancel != 0, h.mic_muted.clone()) {
Some(m) => {
let session_id = m.session_id();
*guard = Some(m);
session_id
}
None => {
log::error!("nativeStartMic: mic init failed (RECORD_AUDIO? — session unaffected)");
0
}
}
}
/// `NativeBridge.nativeStopMic(handle)` — stop + join the mic thread and close the AAudio input
/// stream (without closing the session). No-op on `0`.
/// stream (without closing the session). No-op on `0`. Leaves the session's mute state alone: a
/// surface recreate stops and restarts the mic, and a user who muted must stay muted through it.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopMic(
@@ -432,3 +459,59 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStopMic(
}
})
}
/// `NativeBridge.nativeSetMicMuted(handle, muted)` — mute/unmute the mic uplink mid-stream.
///
/// Muting deliberately does NOT stop the capture: the AAudio input stream, the input-preset rung
/// it settled on and its primed buffers all stay exactly as they are, and the encode loop simply
/// drops each 10 ms frame instead of encoding + sending it. A stop/start would re-run the preset
/// fallback ladder and re-prime buffers on every toggle — hundreds of ms, and possibly a different
/// rung (echo cancellation silently lost). This way a toggle costs one atomic store here and one
/// relaxed load per frame there, and takes effect on the very next 10 ms boundary.
///
/// Sticky for the SESSION (the flag lives on the handle, not on the capture), so the mic restart a
/// surface recreate performs comes back muted with no window for an unmuted frame to escape; a
/// fresh session always starts unmuted. No-op on `0`. Not android-gated — pure `jni` + an atomic
/// store, so it links on the host build too.
///
/// One honest consequence of keeping the stream open: the platform's own recording indicator stays
/// lit while muted, because the mic really is still open. What stops is the encode and the send —
/// no captured audio leaves the process.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSetMicMuted(
_env: JNIEnv,
_this: JObject,
handle: jlong,
muted: jboolean,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
h.mic_muted
.store(muted != 0, std::sync::atomic::Ordering::Relaxed);
}
})
}
/// `NativeBridge.nativeMicActive(handle): Boolean` — is a mic capture actually RUNNING? `true` only
/// between a `nativeStartMic` that opened a stream and the matching `nativeStopMic`. The in-stream
/// mute control is offered on this evidence rather than on the user's setting, so a device that
/// refused every AAudio input rung (or a missing RECORD_AUDIO grant) shows no control instead of a
/// lie about a mic that is being heard. `false` on a `0` handle. Cheap (one uncontended lock).
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeMicActive(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
jni_guard(0, || {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
jboolean::from(h.mic.lock().unwrap().is_some())
})
}
+70
View File
@@ -76,6 +76,12 @@ struct Inner {
/// The other half of the split, release→displayed (SurfaceFlinger's latch + scanout), µs —
/// from the `OnFrameRendered` render timestamps. `pace + latch ≈ display` per frame.
latch_us: Vec<u64>,
/// The `decode` stage's feed split, received→queued (hand-off + input-slot wait), µs. Empty
/// when no receipt stamp matched (HUD off and ABR not measuring decode).
feed_us: Vec<u64>,
/// The other half, queued→decoded (codec-pure: the decoder's own time on the AU, measured
/// from its LAST piece so a slice-progressive head start shows up as a shrink here), µs.
codec_us: Vec<u64>,
/// Frames confirmed on glass this window (`OnFrameRendered` callbacks) — the `presents`-vs-
/// `fps` health pair: presents ≪ fps means the presenter is dropping/serializing; an fps
/// deficit is upstream.
@@ -83,6 +89,10 @@ struct Inner {
/// Client-side newest-wins/pacing drops this window (decoded frames released without
/// rendering, or parked AUs dropped on overflow) — the spec's `skipped` counter.
skipped: u64,
/// The subset of `skipped` that was parked-AU OVERFLOW (the decoder fell behind and whole
/// AUs were dropped before feeding) — a decoder-health signal, vs the benign newest-wins
/// pacing majority. Always ≤ `skipped`.
skipped_overflow: u64,
/// Baselines for windowing the session-cumulative connector counters: the unrecoverable-drop
/// and FEC-recovered totals as of the last drain (or the enable that opened the window), so
/// each snapshot reports only THIS window's `lost` / `FEC` (spec line 4).
@@ -119,6 +129,11 @@ pub struct Snapshot {
/// path / no render callbacks).
pub pace_p50_ms: f64,
pub latch_p50_ms: f64,
/// The `decode` stage's split p50s (ms): `feed` = received→queued (hand-off + input-slot
/// wait), `codec` = queued→decoded (codec-pure, from the AU's last piece). 0.0 when no
/// sample landed (sync loop / no receipt stamps).
pub feed_p50_ms: f64,
pub codec_p50_ms: f64,
/// Frames confirmed on glass this window (`OnFrameRendered` callbacks).
pub presents: u64,
/// Phase-2 `host` / `network` split p50s (ms) — 0.0 when no 0xCF timing matched this window
@@ -135,6 +150,8 @@ pub struct Snapshot {
pub lost: u64,
/// Client-side newest-wins/pacing drops this window (spec `skipped`).
pub skipped: u64,
/// The parked-AU overflow subset of `skipped` (decoder fell behind; ≤ `skipped`).
pub skipped_overflow: u64,
/// FEC shards recovered this window (spec `FEC`, windowed from the cumulative counter).
pub fec: u64,
}
@@ -167,8 +184,11 @@ impl VideoStats {
e2e_disp_us: Vec::with_capacity(256),
pace_us: Vec::with_capacity(256),
latch_us: Vec::with_capacity(256),
feed_us: Vec::with_capacity(256),
codec_us: Vec::with_capacity(256),
presents: 0,
skipped: 0,
skipped_overflow: 0,
last_dropped_total: 0,
last_fec_total: 0,
skew_corrected: false,
@@ -219,8 +239,11 @@ impl VideoStats {
g.e2e_disp_us.clear();
g.pace_us.clear();
g.latch_us.clear();
g.feed_us.clear();
g.codec_us.clear();
g.presents = 0;
g.skipped = 0;
g.skipped_overflow = 0;
g.last_dropped_total = dropped_total;
g.last_fec_total = fec_total;
}
@@ -314,6 +337,45 @@ impl VideoStats {
g.skipped += n;
}
/// Record parked-AU OVERFLOW drops (whole AUs dropped before feeding — the decoder fell
/// behind). Counts into `skipped` too, plus the overflow-only counter, so the HUD can tell
/// benign newest-wins pacing from a decoder that can't keep up.
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_skipped_overflow(&self, n: u64) {
if n == 0 || !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.skipped += n;
g.skipped_overflow += n;
}
/// Record one decoded frame's `decode`-stage split: `feed` = received→queued (hand-off +
/// input-slot wait; absent when no receipt stamp matched) and `codec` = queued→decoded
/// (codec-pure, measured from the AU's LAST piece — a slice-progressive head start shows
/// as a shrink here), both µs.
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn note_decode_split(&self, feed_us: Option<u64>, codec_us: u64) {
if !self.enabled.load(Ordering::Relaxed) {
return; // HUD hidden — skip the lock
}
// Poison-proof for the same reason as `note_received`.
let mut g = self
.inner
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(f) = feed_us {
g.feed_us.push(f);
}
g.codec_us.push(codec_us);
}
/// Record one decoded output frame: its capture→decoded `end-to-end` sample and its
/// received→decoded `decode` stage sample (either may be absent — e.g. the receipt stamp for
/// this pts predates the HUD being shown).
@@ -408,6 +470,8 @@ impl VideoStats {
g.e2e_disp_us.sort_unstable();
g.pace_us.sort_unstable();
g.latch_us.sort_unstable();
g.feed_us.sort_unstable();
g.codec_us.sort_unstable();
let snap = Snapshot {
fps,
mbps,
@@ -421,6 +485,8 @@ impl VideoStats {
disp_valid: !g.e2e_disp_us.is_empty(),
pace_p50_ms: pctl_ms(&g.pace_us, 0.50),
latch_p50_ms: pctl_ms(&g.latch_us, 0.50),
feed_p50_ms: pctl_ms(&g.feed_us, 0.50),
codec_p50_ms: pctl_ms(&g.codec_us, 0.50),
presents: g.presents,
host_p50_ms: pctl_ms(&g.host_us, 0.50),
net_p50_ms: pctl_ms(&g.net_us, 0.50),
@@ -429,6 +495,7 @@ impl VideoStats {
frames: g.frames,
lost: dropped_total.saturating_sub(g.last_dropped_total),
skipped: g.skipped,
skipped_overflow: g.skipped_overflow,
fec: fec_total.saturating_sub(g.last_fec_total),
};
g.window_start = Instant::now();
@@ -443,8 +510,11 @@ impl VideoStats {
g.e2e_disp_us.clear();
g.pace_us.clear();
g.latch_us.clear();
g.feed_us.clear();
g.codec_us.clear();
g.presents = 0;
g.skipped = 0;
g.skipped_overflow = 0;
g.last_dropped_total = dropped_total;
g.last_fec_total = fec_total;
snap
@@ -315,7 +315,15 @@ struct ContentView: View {
clipboardAvailable: model.connection?.hostSupportsClipboard == true,
clipboardOn: model.clipboardEnabled,
toggleClipboard: { model.toggleClipboardSync() },
micAvailable: model.micAvailable,
micMuted: model.micMuted,
toggleMicMute: { model.toggleMicMute() },
disconnect: { model.disconnect() }))
// A fired while input was CAPTURED (InputCapture's chord path posts it the menu's
// identical equivalent can't reach a captured stream). Same toggle either way.
.onReceive(NotificationCenter.default.publisher(for: .punktfunkToggleMicMute)) { _ in
model.toggleMicMute()
}
#endif
#if os(macOS)
// Fullscreen only while a session is up (incl. the trust prompt over the blurred stream),
@@ -724,31 +732,48 @@ struct ContentView: View {
}
.animation(.smooth(duration: 0.28), value: statsVerbosity)
}
#if os(macOS) || os(tvOS)
// The start-of-stream shortcut banner (Windows-client parity): the platform's
// reserved controls on a glass pill, bottom-centre, for the first 6 seconds of
// every session independent of the stats HUD, so the keys are discoverable
// even with statistics off. The banner's own task drops it (cancelled cleanly
// if the session view goes away first). On tvOS it carries the ONLY exits
// Menu/B is swallowed during a session (the `.onExitCommand {}` in the tvOS
// session branch), so the hold gestures must be told to the user.
// The bottom-centre stack: the muted-microphone badge over the start-of-stream
// shortcut banner. ONE overlay for both, so the two can never land on top of each
// other in the seconds where they overlap.
.overlay(alignment: .bottom) {
if captureEnabled && showShortcutHint {
Text(Self.shortcutHintText)
.font(.geist(Self.shortcutHintFont, relativeTo: .caption))
.foregroundStyle(.secondary)
.padding(.horizontal, 14)
.padding(.vertical, 8)
.glassBackground(Capsule())
.padding(.bottom, 24)
.transition(.opacity)
.task {
try? await Task.sleep(for: .seconds(6))
withAnimation(.easeOut(duration: 0.6)) { showShortcutHint = false }
}
VStack(spacing: 8) {
#if !os(tvOS)
// Shown for as long as the mic is muted, at every stats tier and with the
// overlay off see MicMutedBadge. tvOS has no microphone to mute.
if captureEnabled && model.micMuted {
MicMutedBadge { model.setMicMuted(false) }
.transition(.opacity.combined(with: .scale(scale: 0.9)))
}
#endif
#if os(macOS) || os(tvOS)
// The start-of-stream shortcut banner (Windows-client parity): the
// platform's reserved controls on a glass pill for the first 6 seconds of
// every session independent of the stats HUD, so the keys are
// discoverable even with statistics off. The banner's own task drops it
// (cancelled cleanly if the session view goes away first). On tvOS it
// carries the ONLY exits Menu/B is swallowed during a session (the
// `.onExitCommand {}` in the tvOS session branch), so the hold gestures
// must be told to the user.
if captureEnabled && showShortcutHint {
Text(shortcutHintText)
.font(.geist(Self.shortcutHintFont, relativeTo: .caption))
.foregroundStyle(.secondary)
.padding(.horizontal, 14)
.padding(.vertical, 8)
.glassBackground(Capsule())
.transition(.opacity)
.task {
try? await Task.sleep(for: .seconds(6))
withAnimation(.easeOut(duration: 0.6)) {
showShortcutHint = false
}
}
}
#endif
}
.padding(.bottom, 24)
.animation(.easeOut(duration: 0.2), value: model.micMuted)
}
#endif
#if os(iOS)
// Touch users have no menu / D, so when the HUD's Disconnect button isn't on
// screen the overlay off, or the compact pill (which carries no button)
@@ -766,21 +791,24 @@ struct ContentView: View {
.overlay(alignment: .topLeading) {
if captureEnabled,
statsVerbosity == .compact || (statsVerbosity == .off && showTouchExit) {
Button { model.disconnect() } label: {
Image(systemName: "xmark")
.font(.headline.weight(.semibold))
.frame(width: 36, height: 36)
// Floating glass disc over the frame (26+, material fallback).
// interactive: the disc IS the tap target, so the glass reacts
// to press.
.glassBackground(Circle(), interactive: true)
// Match the hit region to the visible disc so every tap also
// triggers the interactive-glass press highlight.
.contentShape(Circle())
HStack(spacing: 10) {
Button { model.disconnect() } label: { touchDisc("xmark") }
.buttonStyle(.plain)
.accessibilityLabel("Disconnect")
// The mic toggle rides the same discs, for the same reason: in these
// tiers the HUD carries no buttons (compact is a stat pill, off is
// nothing), so this is a touch-only user's ONLY way to mute. Absent
// not greyed when the session sends no microphone at all.
if model.micAvailable {
Button { model.toggleMicMute() } label: {
touchDisc(model.micMuted ? "mic.slash.fill" : "mic.fill")
}
.buttonStyle(.plain)
.accessibilityLabel(
model.micMuted ? "Unmute microphone" : "Mute microphone")
}
}
.buttonStyle(.plain)
.padding(12)
.accessibilityLabel("Disconnect")
.transition(.opacity)
.task {
guard statsVerbosity == .off else { return }
@@ -794,14 +822,34 @@ struct ContentView: View {
}
}
#if os(iOS)
/// One touch-control disc: an SF Symbol on a floating glass disc over the frame (26+,
/// material fallback), sized as a comfortable tap target. `interactive`: the disc IS the tap
/// target, so the glass reacts to press, and the hit region is matched to the visible disc so
/// every tap triggers that press highlight.
private func touchDisc(_ symbol: String) -> some View {
Image(systemName: symbol)
.font(.headline.weight(.semibold))
.frame(width: 36, height: 36)
.glassBackground(Circle(), interactive: true)
.contentShape(Circle())
}
#endif
#if os(macOS)
private static let shortcutHintText =
"Click the stream to capture · ⌃⌥⇧Q releases the mouse · ⌃⌥⇧D disconnects · ⌃⌥⇧S stats"
/// The reserved combos, told once per session. The mute segment appears only when the session
/// actually sends a microphone teaching a shortcut for a mic that isn't on would be a lie.
private var shortcutHintText: String {
let base =
"Click the stream to capture · ⌃⌥⇧Q releases the mouse · ⌃⌥⇧D disconnects · ⌃⌥⇧S stats"
return model.micAvailable ? base + " · ⌃⌥⇧A mutes the mic" : base
}
private static let shortcutHintFont: CGFloat = 12
#elseif os(tvOS)
private static let shortcutHintText =
private var shortcutHintText: String {
"Hold the remote's Back button — or L1+R1+Start+Select on a controller — to disconnect"
+ " · Touch surface moves the pointer · press clicks · Play/Pause right-clicks"
}
private static let shortcutHintFont: CGFloat = 22 // read from the couch
#endif
@@ -1,6 +1,9 @@
// Session state for the app shell: owns the connection, the input capture, the trust
// handshake phase, and the pump-thread main-actor stats relay.
// AVFoundation: AVCaptureDevice.authorizationStatus (the mic TCC grant behind `micAvailable`)
// and, on tvOS, AVPlayer.eligibleForHDRPlayback (the TV-capability HDR gate).
import AVFoundation
import Foundation
import os
import PunktfunkKit
@@ -11,9 +14,6 @@ import SwiftUI
#elseif canImport(UIKit)
import UIKit
#endif
#if os(tvOS)
import AVFoundation // AVPlayer.eligibleForHDRPlayback the TV-capability HDR gate
#endif
/// 1 Hz latency-stage line mirrored to the unified log so the stages can be read WITHOUT the
/// on-screen HUD (Console.app, wirelessly on an iPad/Apple TV). The HUD is not a neutral
@@ -137,6 +137,14 @@ final class SessionModel: ObservableObject {
/// Mirrors StreamView's capture state (it owns the input capture; this drives the
/// HUD's "click to capture" / " releases" hint).
@Published var mouseCaptured = false
/// The USER's in-stream mic mute (the HUD button, the Stream menu's A, the captured-state
/// chord, the iOS mic disc) session state, deliberately NOT persisted: a mute is for the
/// people in the room right now, so every new session starts live if the mic is on at all.
/// One of the two inputs to the effective mute; `isBackgrounded` is the other, and
/// `applyMicMute` composes them a user mute survives a trip through the background, and the
/// background's privacy mute never clears the user's choice. Local and instant: it gates
/// capture on this device, nothing is sent to the host.
@Published private(set) var micMuted = false
/// Resize overlay (design/midstream-resolution-resize.md client resize UX): true from the
/// instant a Match-window resize starts steering toward a new size until a frame at that size
/// decodes (or a safety timeout). Drives the blur+spinner so the unavoidable host-rebuild delay
@@ -440,7 +448,7 @@ final class SessionModel: ObservableObject {
guard phase == .streaming, let conn = connection, !isBackgrounded else { return }
isBackgrounded = true
conn.setVideoDropped(true)
audio?.setMicMuted(true)
applyMicMute() // now muted for privacy on top of the user's own mute, not instead of it
// Non-deliberate on fire (keep the host linger) so a user who returns late reconnects fast,
// exactly like today's network-drop path. min 1 minute guards a nonsense setting.
let minutes = max(1, timeoutMinutes)
@@ -465,13 +473,57 @@ final class SessionModel: ObservableObject {
backgroundDeadline = nil
backgroundTimer?.cancel()
backgroundTimer = nil
audio?.setMicMuted(false)
applyMicMute() // back to the user's own choice which may well still be "muted"
if let conn = connection {
conn.setVideoDropped(false)
conn.requestKeyframe()
}
}
// MARK: - Microphone mute (in-stream, per session)
/// Whether this session has a mic uplink there is any point in muting: the mic must be on in
/// the session's RESOLVED settings (a profile can turn it on or off), the platform must have
/// an app-accessible input at all, and the OS must not have refused us one. Drives whether the
/// mute control is offered a live-looking mute button over a session that sends no
/// microphone would be a lie. Same three conditions `SessionAudio` starts an uplink on
/// (`.notDetermined` counts: the prompt is pending and a grant starts the uplink mid-session).
var micAvailable: Bool {
#if os(tvOS)
return false // no app-accessible microphone SessionAudio never opens an uplink either
#else
guard settings.micEnabled else { return false }
switch AVCaptureDevice.authorizationStatus(for: .audio) {
case .authorized, .notDetermined: return true
default: return false // denied / restricted there is no uplink to mute
}
#endif
}
/// Flip the user's mute. The in-stream surfaces (HUD button, Stream menu, A while
/// captured, the iOS mic disc) all land here.
func toggleMicMute() {
setMicMuted(!micMuted)
}
/// Set the user's mute directly (the badge's tap-to-unmute). Ignored when the session has no
/// microphone, so a stale surface can't leave a phantom "muted" badge over a session that was
/// never sending anything.
func setMicMuted(_ muted: Bool) {
guard micAvailable, micMuted != muted else { return }
micMuted = muted
applyMicMute()
}
/// Push the EFFECTIVE mute the user's choice OR the background keep-alive's privacy mute
/// onto the audio engine. The two reasons are composed here and nowhere else: whichever one
/// changed, the other still holds, so returning from the background can't un-mute a user who
/// muted mid-stream, and a user unmuting while backgrounded (Live Activity, another window)
/// doesn't open the mic behind their back.
private func applyMicMute() {
audio?.setMicMuted(micMuted || isBackgrounded)
}
/// Follow a live stats-overlay cycle (S, the three-finger tap, the Stream menu). Those
/// surfaces write the GLOBAL setting as they always have; this moves the session's own tier
/// with it, so cycling still works in a session a profile put on a different tier.
@@ -509,6 +561,9 @@ final class SessionModel: ObservableObject {
backgroundTimer = nil
isBackgrounded = false
backgroundDeadline = nil
// The mic mute is per-session and never persisted: the next stream starts live (if the
// mic is enabled), rather than silently carrying a mute nobody remembers making.
micMuted = false
let audio = self.audio
self.audio = nil
// Gamepad capture is main-actor (releases held buttons on the wire while the
@@ -609,7 +664,8 @@ final class SessionModel: ObservableObject {
speakerUID: settings.speakerUID,
micUID: settings.micUID,
micChannel: settings.micChannel,
micEnabled: settings.micEnabled)
micEnabled: settings.micEnabled,
echoCancel: settings.echoCancel)
self.audio = audio
// Gamepads: forward every controller GamepadManager selected each on its own wire pad
// index (a pin forwards only one, Automatic forwards all) and render the host's feedback
@@ -1,7 +1,8 @@
// The app's "Stream" menu (macOS menu bar + iPad hardware-keyboard shortcuts). These live at
// the Scene level so they keep working when the HUD overlay is hidden. The shortcuts are the
// CROSS-CLIENT set every punktfunk client reserves Ctrl+Alt+Shift+Q (release the captured
// mouse) / +D (disconnect) / +S (stats) and the menu is their discoverable surface on macOS
// mouse) / +D (disconnect) / +S (stats), plus +A (mute the microphone), the Apple clients'
// addition to it and the menu is their discoverable surface on macOS
// (the Linux client has its GTK Shortcuts window, Windows its start-of-stream banner). While
// input is CAPTURED these key equivalents never reach the menu (the stream view swallows
// keys); InputCapture's monitor detects the same combos there and performs the same actions
@@ -27,6 +28,12 @@ struct SessionFocus {
/// Clipboard sync is live (host-acked) drives the item's Stop/Share title.
var clipboardOn: Bool
var toggleClipboard: () -> Void
/// The session has a mic uplink at all (its resolved `micEnabled`) gates the mute item, so
/// it is never an enabled control over a session that sends no microphone.
var micAvailable: Bool
/// The user's mic mute is engaged drives the item's Mute/Unmute title.
var micMuted: Bool
var toggleMicMute: () -> Void
var disconnect: () -> Void
}
@@ -60,6 +67,17 @@ struct StreamCommands: Commands {
}
.keyboardShortcut("q", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true)
// Mic mute, local and instant (it gates capture on this device the host is never
// asked). Per SESSION: it starts off every time, so this item is a live toggle, not a
// setting. Greyed when the session sends no microphone at all (Settings mic off, or
// a profile that turns it off) rather than pretending there is something to mute.
// Captured, the combo is handled by InputCapture's chord path before menus see it;
// this item is the released-state path and the shortcut's documentation.
Button(session?.micMuted == true ? "Unmute Microphone" : "Mute Microphone") {
session?.toggleMicMute()
}
.keyboardShortcut("a", modifiers: [.control, .option, .shift])
.disabled(session?.isStreaming != true || session?.micAvailable != true)
#if os(macOS)
// Mid-session clipboard flip (design/clipboard-and-file-transfer.md §5.3). Greyed
// when the host doesn't advertise the cap (older host / operator policy off).
@@ -179,6 +179,18 @@ struct StreamHUDView: View {
.foregroundStyle(.secondary)
}
#endif
// Mic mute the in-stream toggle, on the same card as the other in-overlay action.
// Absent (not greyed) when the session sends no microphone: the HUD is a status card,
// and a dead control on it would read as "there is a mic, and it is on". The muted
// STATE is not this button's job the badge over the stream says that at every tier
// and with the overlay off entirely. tvOS gets no control: no microphone, and a
// focusable one would steal the controller's A press from the host.
#if !os(tvOS)
if model.micAvailable {
Button(micButtonTitle) { model.toggleMicMute() }
.font(.geist(12, relativeTo: .caption))
}
#endif
// D lives on the app's Stream menu (so it still works when the HUD is hidden)
// and in InputCapture's monitor while captured; this button is the in-overlay,
// click-to-disconnect affordance. tvOS deliberately gets NEITHER a button (a
@@ -195,6 +207,19 @@ struct StreamHUDView: View {
}
}
#if !os(tvOS)
/// The mute button's wording. macOS names the chord, exactly as its Disconnect button does;
/// iOS/iPadOS spells the action out (the HUD's buttons there carry no shortcuts, even where a
/// hardware keyboard could fire one the Stream menu is that keyboard's surface).
private var micButtonTitle: String {
#if os(macOS)
return model.micMuted ? "Unmute Mic (⌃⌥⇧A)" : "Mute Mic (⌃⌥⇧A)"
#else
return model.micMuted ? "Unmute Microphone" : "Mute Microphone"
#endif
}
#endif
// MARK: - Card metrics
/// The card's inner content padding. Roomier on tvOS the stat text auto-scales for the
@@ -242,6 +267,42 @@ struct StreamHUDView: View {
}
}
#if !os(tvOS)
/// The muted-microphone badge the mute STATE, as opposed to the buttons that flip it. It rides
/// over the stream whenever the mic is muted, INDEPENDENT of the stats overlay (which the user
/// may have cycled off, and which the compact tier reduces to a stat line): "am I muted?" is not a
/// statistic, and a mute you can't see is how people talk to nobody for a minute. Same glass
/// language as the HUD, sized like the start-of-stream banner it shares the bottom edge with.
///
/// It is also a control: tapping it unmutes. That is the guaranteed way back for a touch user who
/// muted with the overlay off, and it costs the badge nothing (it is on screen either way).
struct MicMutedBadge: View {
let onUnmute: () -> Void
var body: some View {
Button(action: onUnmute) {
HStack(spacing: 7) {
Image(systemName: "mic.slash.fill")
.font(.system(size: 13, weight: .semibold))
.foregroundStyle(.red)
Text("Microphone muted")
.font(.geist(12, .medium, relativeTo: .caption))
.foregroundStyle(.white.opacity(0.9))
}
.padding(.horizontal, 14)
.padding(.vertical, 8)
// interactive: the badge IS the tap target, so the glass reacts to press.
.glassBackground(Capsule(), interactive: true)
.contentShape(Capsule())
}
.buttonStyle(.plain)
.environment(\.colorScheme, .dark) // reads over any frame, like the resize overlay
.accessibilityLabel("Microphone muted")
.accessibilityHint("Unmutes the microphone")
}
}
#endif
#if os(iOS)
/// Device display geometry the overlay needs but UIKit doesn't expose publicly.
enum DeviceMetrics {
@@ -32,6 +32,7 @@ struct GamepadSettingsView: View {
@AppStorage(DefaultsKey.enable444) private var enable444 = false
@AppStorage(DefaultsKey.codec) private var codec = "auto"
@AppStorage(DefaultsKey.micEnabled) private var micEnabled = true
@AppStorage(DefaultsKey.echoCancel) private var echoCancel = true
// The overlay tier's raw string (rows tag by rawValue); the absent-key default runs the
// legacy-hudEnabled migration (same pattern as ContentView/SettingsView).
@AppStorage(DefaultsKey.statsVerbosity) private var statsVerbosityRaw
@@ -316,6 +317,11 @@ struct GamepadSettingsView: View {
id: "mic", icon: "mic", label: "Microphone",
detail: "Send this device's microphone to the host's virtual mic.",
value: $micEnabled),
toggleRow(
id: "echoCancel", icon: "waveform", label: "Echo cancellation",
detail: "Cancel the audio this device plays out of the mic signal — stops "
+ "speaker setups feeding the game back to the host.",
value: $echoCancel),
choiceRow(
id: "pad", header: "Controller", icon: "gamecontroller", label: "Use controller",
@@ -98,6 +98,10 @@ enum SettingsFields {
.init(name: "mic_enabled", key: DefaultsKey.micEnabled,
overlay: \.micEnabled, effective: \.micEnabled)
}
static var echoCancel: SettingsField<Bool> {
.init(name: "echo_cancel", key: DefaultsKey.echoCancel,
overlay: \.echoCancel, effective: \.echoCancel)
}
static var touchMode: SettingsField<String> {
.init(name: "touch_mode", key: DefaultsKey.touchMode,
overlay: \.touchMode, effective: \.touchMode)
@@ -175,6 +179,7 @@ extension SettingsView {
base.compositor = compositor
base.audioChannels = audioChannels
base.micEnabled = micEnabled
base.echoCancel = echoCancel
base.gamepadType = gamepadType
base.statsVerbosity = statsVerbosityRaw
base.fullscreenWhileStreaming = fullscreenWhileStreaming
@@ -581,6 +581,10 @@ extension SettingsView {
field: "mic_enabled") {
Toggle("Send microphone to the host", isOn: scoped(SettingsFields.micEnabled))
}
described(echoCancelCaption, field: "echo_cancel") {
Toggle("Echo cancellation", isOn: scoped(SettingsFields.echoCancel))
.disabled(!effective.micEnabled)
}
#if os(macOS)
if !inProfileScope {
Picker("Microphone", selection: $micUID) {
@@ -619,6 +623,20 @@ extension SettingsView {
}
}
/// Honest about the macOS escape hatch: the voice processor only follows the system
/// default devices, so hand-picked endpoints silently keep the raw path (see
/// SessionAudio's topology note) better said here than discovered mid-call.
private var echoCancelCaption: String {
let base = "Voice processing cancels the audio this device plays out of the mic "
+ "signal, so a speaker setup doesn't feed the game back to the host."
#if os(macOS)
return base + " Follows the system default devices — a hand-picked speaker, "
+ "microphone or input channel streams the raw mic instead."
#else
return base
#endif
}
// MARK: - Controllers
@ViewBuilder var controllersSection: some View {
@@ -65,6 +65,7 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.libraryEnabled) var libraryEnabled = true
@AppStorage(DefaultsKey.fullscreenWhileStreaming) var fullscreenWhileStreaming = true
@AppStorage(DefaultsKey.micEnabled) var micEnabled = true
@AppStorage(DefaultsKey.echoCancel) var echoCancel = true
@AppStorage(DefaultsKey.audioChannels) var audioChannels = 2
@AppStorage(DefaultsKey.codec) var codec = "auto"
// The overlay tier's raw string (the pickers tag by rawValue); the absent-key default runs
@@ -1,7 +1,9 @@
// Opus PCM through CoreAudio's built-in codec (kAudioFormatOpus, macOS 10.13+ / iOS
// 11+) no bundled libopus. The host's audio plane is raw Opus packets (48 kHz stereo,
// one frame per packet); AVAudioConverter handles them as single-packet
// AVAudioCompressedBuffers with explicit packet descriptions.
// one frame per packet); the mic uplink is 48 kHz MONO packets (one microphone bus
// the host's decoder upmixes, so duplicating it into a second channel only cost bits).
// AVAudioConverter handles both as single-packet AVAudioCompressedBuffers with explicit
// packet descriptions.
//
// Both classes are single-threaded by contract (one per direction, owned by their
// drain/capture pipelines).
@@ -14,16 +16,16 @@ enum OpusCodecError: Error {
case convertFailed(String)
}
/// 48 kHz stereo float32 interleaved the PCM side of both converters and the layout
/// of the playback ring buffer.
/// 48 kHz stereo float32 interleaved the decoder's PCM side (the host plane's shape).
func opusPCMFormat() -> AVAudioFormat? {
AVAudioFormat(
commonFormat: .pcmFormatFloat32, sampleRate: 48_000, channels: 2, interleaved: true)
}
/// The compressed side: raw Opus, `framesPerPacket` nominal samples per packet at 48 kHz
/// (240 = the host's 5 ms audio plane; 960 = the 20 ms packets the encoder emits).
private func opusFormat(framesPerPacket: UInt32) -> AVAudioFormat? {
/// (240 = the host's 5 ms audio plane; 480 = the 10 ms packets the encoder emits) and
/// `channels` (2 = the host plane, 1 = the mic uplink).
private func opusFormat(framesPerPacket: UInt32, channels: UInt32) -> AVAudioFormat? {
var desc = AudioStreamBasicDescription(
mSampleRate: 48_000,
mFormatID: kAudioFormatOpus,
@@ -31,7 +33,7 @@ private func opusFormat(framesPerPacket: UInt32) -> AVAudioFormat? {
mBytesPerPacket: 0,
mFramesPerPacket: framesPerPacket,
mBytesPerFrame: 0,
mChannelsPerFrame: 2,
mChannelsPerFrame: channels,
mBitsPerChannel: 0,
mReserved: 0)
return AVAudioFormat(streamDescription: &desc)
@@ -45,7 +47,8 @@ final class OpusDecoder {
/// `framesPerPacket`: the sender's packet duration in samples (host audio = 240).
init(framesPerPacket: UInt32) throws {
guard let pcm = opusPCMFormat(), let opus = opusFormat(framesPerPacket: framesPerPacket),
guard let pcm = opusPCMFormat(),
let opus = opusFormat(framesPerPacket: framesPerPacket, channels: 2),
let converter = AVAudioConverter(from: opus, to: pcm)
else { throw OpusCodecError.unavailable }
self.converter = converter
@@ -90,24 +93,43 @@ final class OpusDecoder {
}
final class OpusEncoder {
/// The encoder's packet duration: 960 samples = 20 ms, CoreAudio's default Opus
/// framing. The host's mic service decodes any Opus frame size up to 120 ms.
static let framesPerPacket: AVAudioFrameCount = 960
/// The encoder's packet duration in samples: 480 = 10 ms, halving the packetization
/// latency of the old 20 ms framing. CoreAudio honors it mFramesPerPacket 480/mono
/// creates a converter that truly emits 10 ms CELT packets (TOC config 30), one per
/// 480-frame chunk, verified by inspection of the emitted TOC bytes and per-packet
/// frame accounting. 960 stays as the fallback should an older codec refuse 480.
/// The host's mic service decodes any Opus frame size up to 120 ms, so either is
/// wire-compatible.
let framesPerPacket: AVAudioFrameCount
/// 48 kHz MONO float32 interleaved the uplink carries one microphone bus.
let pcmFormat: AVAudioFormat
private let converter: AVAudioConverter
private let outBuf: AVAudioCompressedBuffer
let pcmFormat: AVAudioFormat
init() throws {
guard let pcm = opusPCMFormat(),
let opus = opusFormat(framesPerPacket: UInt32(Self.framesPerPacket)),
let converter = AVAudioConverter(from: pcm, to: opus)
guard let pcm = AVAudioFormat(
commonFormat: .pcmFormatFloat32, sampleRate: 48_000, channels: 1,
interleaved: true)
else { throw OpusCodecError.unavailable }
converter.bitRate = 96_000
self.converter = converter
self.pcmFormat = pcm
var made: (converter: AVAudioConverter, fpp: AVAudioFrameCount)?
for fpp: AVAudioFrameCount in [480, 960] {
if let opus = opusFormat(framesPerPacket: UInt32(fpp), channels: 1),
let converter = AVAudioConverter(from: pcm, to: opus) {
made = (converter, fpp)
break
}
}
guard let made else { throw OpusCodecError.unavailable }
// 48 kbps: transparent for mono voice the old 96 kbps budget was sized for
// the duplicated-stereo framing this encoder no longer emits.
made.converter.bitRate = 48_000
converter = made.converter
framesPerPacket = made.fpp
pcmFormat = pcm
outBuf = AVAudioCompressedBuffer(
format: opus, packetCapacity: 4, maximumPacketSize: 1500)
format: made.converter.outputFormat, packetCapacity: 4, maximumPacketSize: 1500)
}
/// Encode exactly `framesPerPacket` frames of `pcmFormat` audio; returns the encoded
@@ -5,15 +5,22 @@
// AVAudioSourceNode pulls from the ring (silence on underrun with re-priming, so a
// network gap costs one dip, not permanent crackle).
//
// mic host: a second AVAudioEngine taps the input device, folds it to one mono bus (the
// chosen channel of a multi-channel interface, or a sum of all channels), resamples to 48 kHz
// stereo, slices 20 ms chunks, Opus-encodes, and sendMic()s each packet the host feeds them
// into a virtual PipeWire source.
// mic host: a tap on the input node folds the capture to one mono bus (the chosen channel
// of a multi-channel interface, or a sum of all channels), resamples to 48 kHz mono, slices
// 10 ms chunks, Opus-encodes, and sendMic()s each packet the host feeds them into a
// virtual PipeWire source.
//
// Engine topology. With the mic enabled and echo cancellation on (both defaults), BOTH
// directions run on ONE AVAudioEngine with the system voice processor engaged
// (`setVoiceProcessingEnabled`) AEC needs render and capture on the same unit so it can
// subtract what the speaker is playing from what the mic hears; without it, a loudspeaker
// client feeds the host's own game audio straight back to it (the primary reported echo
// source). The voice processor can only follow the system DEFAULT devices, so explicit
// endpoint choices fall back to the old two-engine topology see `wantsCombined` for the
// exact decision, and `startCapture` for why two engines handle arbitrary device pairs.
//
// Devices are chosen by UID ("" = system default: the engine is then never pinned to a
// concrete device and follows default-device changes). Two engines, not one a single
// AVAudioEngine ties input+output to one aggregate clock, separate engines keep
// arbitrary mic/speaker combinations trivial.
// concrete device and follows default-device changes).
import AVFoundation
import os
@@ -40,7 +47,21 @@ public final class SessionAudio {
private let stateLock = NSLock()
private var playbackEngine: AVAudioEngine?
private var captureEngine: AVAudioEngine?
/// The one engine running BOTH directions when the voice processor is engaged;
/// `playbackEngine`/`captureEngine` stay nil while this is set.
private var combinedEngine: AVAudioEngine?
private var drainStarted = false
/// The mute LATCH: the effective mute the owner last asked for (see `setMicMuted`). Held
/// because the uplink engine can appear LATER than the request the mic permission prompt
/// is answered at the user's leisure, and the engine is built on the grant so a mute set
/// in the meantime must be waiting for it. Applied by whichever start path wins the race.
/// Guarded by `stateLock`, like the engines it applies to.
private var micMuted = false
/// The playback jitter ring created by whichever engine starts playback first and KEPT
/// across an engine rebuild (the permission-grant upgrade in `startEngines` swaps engines,
/// not the ring, so the drain thread never has to be re-pointed). Main-thread confined,
/// like every start path.
private var ring: AudioRing?
#if !os(macOS)
/// AVAudioSession `setCategory`/`setActive` are synchronous and block on the audio server, so
/// they must not run on the main thread (UI stall AVFoundation warns about it). PROCESS-WIDE
@@ -69,11 +90,15 @@ public final class SessionAudio {
/// ASYNCHRONOUS: it activates the AVAudioSession off the main thread, then starts the engines on
/// a later main-queue hop (gated by `!flag.isStopped`) so playback is live shortly after, not
/// on return. The mic may start later still if the permission prompt is pending.
public func start(speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool) {
/// `echoCancel` picks the engine topology see the header note and `wantsCombined`.
public func start(
speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool, echoCancel: Bool
) {
#if os(macOS)
// No AVAudioSession on macOS start the engines directly (caller's thread, as before).
startEngines(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel, micEnabled: micEnabled)
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel,
micEnabled: micEnabled, echoCancel: echoCancel)
#else
// Configure + activate the session OFF the main thread (it blocks on the audio server),
// then start the engines back on the main thread once it's active engine routing/format
@@ -85,7 +110,7 @@ public final class SessionAudio {
guard let self, !self.flag.isStopped else { return }
self.startEngines(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel,
micEnabled: micEnabled)
micEnabled: micEnabled, echoCancel: echoCancel)
}
}
#endif
@@ -104,6 +129,12 @@ public final class SessionAudio {
try session.setCategory(
.playAndRecord, mode: .default,
options: [.allowBluetoothA2DP, .defaultToSpeaker])
// Uplink latency: ask for 5 ms IO quanta at the wire rate (the default ~10-23 ms
// quantum is most of the mic path's burst latency). Best-effort the hardware
// has the final word (a Bluetooth route will ignore both), and whatever quantum
// is actually granted, the capture tap handles the buffers it gets.
try? session.setPreferredIOBufferDuration(0.005)
try? session.setPreferredSampleRate(48_000)
} else {
try session.setCategory(.playback, mode: .default)
}
@@ -117,32 +148,80 @@ public final class SessionAudio {
}
#endif
/// Build + start the playback engine (and the mic uplink when enabled + authorized). Main
/// thread (engine setup); on iOS/tvOS the session is already active by the time this runs.
/// Build + start the engines combined (voice-processed) or split, per `wantsCombined`
/// with the mic uplink only when enabled + authorized. Main thread (engine setup); on
/// iOS/tvOS the session is already active by the time this runs.
private func startEngines(
speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool
speakerUID: String, micUID: String, micChannel: Int, micEnabled: Bool, echoCancel: Bool
) {
startPlayback(speakerUID: speakerUID)
#if os(tvOS)
// No app-accessible microphone input on tvOS playback only.
startPlayback(speakerUID: speakerUID)
#else
guard micEnabled else { return }
guard micEnabled else {
startPlayback(speakerUID: speakerUID)
return
}
let combined = wantsCombined(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel,
echoCancel: echoCancel)
switch AVCaptureDevice.authorizationStatus(for: .audio) {
case .authorized:
startCapture(micUID: micUID, micChannel: micChannel)
if combined {
startCombined(speakerUID: speakerUID, micUID: micUID, micChannel: micChannel)
} else {
startPlayback(speakerUID: speakerUID)
startCapture(micUID: micUID, micChannel: micChannel)
}
case .notDetermined:
// Playback must not wait out the permission prompt (the user answers at their
// leisure) start it now, and on a grant either bolt the capture engine on
// (split) or swap the playback engine for the combined one (the ring and its
// drain thread carry over see `makePlaybackChain`).
startPlayback(speakerUID: speakerUID)
AVCaptureDevice.requestAccess(for: .audio) { [weak self] granted in
DispatchQueue.main.async {
guard let self, granted, !self.flag.isStopped else { return }
self.startCapture(micUID: micUID, micChannel: micChannel)
if combined {
self.stateLock.lock()
let playback = self.playbackEngine
self.playbackEngine = nil
self.stateLock.unlock()
playback?.stop()
self.startCombined(
speakerUID: speakerUID, micUID: micUID, micChannel: micChannel)
} else {
self.startCapture(micUID: micUID, micChannel: micChannel)
}
}
}
default:
startPlayback(speakerUID: speakerUID)
log.warning("microphone access denied — mic uplink disabled (System Settings → Privacy)")
}
#endif
}
#if !os(tvOS)
/// One engine or two: the voice processor requires render + capture on one unit, and that
/// unit can only follow the system DEFAULT devices so echo cancellation gets the combined
/// engine only while nothing is explicitly pinned. On macOS a chosen speaker/mic UID or a
/// picked input channel (the voice processor's capture side is its own mono mix a
/// per-channel pick can't survive it) keeps today's two-engine path, AEC-less but honoring
/// the exact endpoints the user named. On iOS routes are session-managed and the UIDs are
/// ignored, so the toggle alone decides.
private func wantsCombined(
speakerUID: String, micUID: String, micChannel: Int, echoCancel: Bool
) -> Bool {
guard echoCancel else { return false }
#if os(macOS)
return speakerUID.isEmpty && micUID.isEmpty && micChannel == 0
#else
return true
#endif
}
#endif
/// Stop both directions. Safe from any thread; waits the drain thread out ( its
/// poll timeout) so the caller can close the connection right after.
public func stop() {
@@ -152,6 +231,8 @@ public final class SessionAudio {
captureEngine = nil
let playback = playbackEngine
playbackEngine = nil
let combined = combinedEngine
combinedEngine = nil
let wasDraining = drainStarted
drainStarted = false
stateLock.unlock()
@@ -160,6 +241,10 @@ public final class SessionAudio {
capture.stop()
}
playback?.stop()
if let combined {
combined.inputNode.removeTap(onBus: 0)
combined.stop()
}
#if !os(macOS)
// Release the session so audio we interrupted (Music, podcasts) gets its resume cue. Like
// activation, setActive is synchronous/blocking run it on the shared serial session queue
@@ -180,15 +265,38 @@ public final class SessionAudio {
}
}
/// Background keep-alive: silence the mic uplink while backgrounded (privacy no room audio
/// leaves the device) and restore it on return. Pauses/resumes the capture engine; a no-op when
/// there's no uplink (playback-only / tvOS / mic disabled). The audio SESSION stays active for
/// background playback, so iOS may keep showing the recording indicator until a full reconfigure
/// this stops the actual capture, which is the privacy-relevant part. Main thread.
/// Silence the mic uplink (no room audio leaves the device) or restore it. THE one muting
/// mechanism: the owner composes its reasons the user's in-stream mute and the background
/// keep-alive's privacy mute into one effective state and passes that here, so neither can
/// clear the other (see `SessionModel.applyMicMute`).
///
/// Two-engine sessions pause/resume the capture engine; a combined session instead mutes the
/// voice processor's input (playback shares that engine and must keep running, so the engine
/// itself never pauses the mute zeroes the mic at the IO unit, and the tap encodes silence).
/// Local and instant either way: nothing is negotiated with the host, and the packets that do
/// leave carry silence. A no-op when there's no uplink (playback-only / tvOS / mic disabled),
/// except that the state is LATCHED for an uplink that starts later. The audio SESSION stays
/// active for background playback, so iOS may keep showing the recording indicator until a
/// full reconfigure either path stops room audio leaving the device, which is the
/// privacy-relevant part. Main thread.
public func setMicMuted(_ muted: Bool) {
stateLock.lock()
micMuted = muted
let capture = captureEngine
let combined = combinedEngine
stateLock.unlock()
apply(micMuted: muted, capture: capture, combined: combined)
}
/// Push the latched mute onto whichever engine carries the uplink. Split out from
/// `setMicMuted` because the start paths call it too, with the engine they just started
/// that's how a mute requested before the permission grant lands on the engine the grant
/// creates. Never resumes a stopped session's engine.
private func apply(micMuted muted: Bool, capture: AVAudioEngine?, combined: AVAudioEngine?) {
if let combined {
combined.inputNode.isVoiceProcessingInputMuted = muted
return
}
guard let capture else { return }
if muted {
capture.pause()
@@ -199,28 +307,21 @@ public final class SessionAudio {
// MARK: - Playback (host speaker)
private func startPlayback(speakerUID: String) {
/// The playback jitter ring + the source node draining it shared by the plain playback
/// engine and the combined voice-processing engine, and REUSED across an engine rebuild
/// (same session, same ring: the drain thread keeps writing right through the swap). nil
/// when the host's channel layout can't be expressed (already logged). Main thread.
private func makePlaybackChain()
-> (ring: AudioRing, source: AVAudioSourceNode, format: AVAudioFormat)?
{
// Build the playback layout from the host-RESOLVED channel count (never the request):
// 2 = stereo / 6 = 5.1 / 8 = 7.1, canonical wire order FL FR FC LFE RL RR SL SR.
let channels = Int(connection.resolvedAudioChannels)
// 1 s interleaved capacity, ~20 ms prefill (four 5 ms host packets of jitter absorption
// before the first sample plays), both scaled by the channel count.
let ring = AudioRing(
let ring = self.ring ?? AudioRing(
capacity: 48_000 * channels, prefill: 960 * channels, channels: channels)
let engine = AVAudioEngine()
#if os(macOS)
if !speakerUID.isEmpty {
if let dev = AudioDevices.deviceID(forUID: speakerUID),
let unit = engine.outputNode.audioUnit {
if !Self.setDevice(dev, on: unit) {
log.error("could not select speaker \(speakerUID) — using default")
}
} else {
log.warning("speaker \(speakerUID) not present — using default")
}
}
#endif
self.ring = ring
// Engine-native deinterleaved float; the render block deinterleaves from the ring. Surround
// uses an explicit wire-order channel layout; the mixer downmixes to the output device when
@@ -234,7 +335,7 @@ public final class SessionAudio {
}
guard let format else {
log.error("could not build \(channels)-channel audio format — audio disabled")
return
return nil
}
let scratch = ScratchBuffer() // block-owned; freed with the closure
let source = AVAudioSourceNode(format: format) { _, _, frameCount, abl -> OSStatus in
@@ -252,6 +353,24 @@ public final class SessionAudio {
}
return noErr
}
return (ring, source, format)
}
private func startPlayback(speakerUID: String) {
guard let (ring, source, format) = makePlaybackChain() else { return }
let engine = AVAudioEngine()
#if os(macOS)
if !speakerUID.isEmpty {
if let dev = AudioDevices.deviceID(forUID: speakerUID),
let unit = engine.outputNode.audioUnit {
if !Self.setDevice(dev, on: unit) {
log.error("could not select speaker \(speakerUID) — using default")
}
} else {
log.warning("speaker \(speakerUID) not present — using default")
}
}
#endif
engine.attach(source)
engine.connect(source, to: engine.mainMixerNode, format: format)
engine.prepare()
@@ -272,8 +391,14 @@ public final class SessionAudio {
startDrain(into: ring)
}
/// Idempotent the permission-grant engine swap reaches here a second time with the
/// drain thread already feeding the (carried-over) ring.
private func startDrain(into ring: AudioRing) {
stateLock.lock()
if drainStarted {
stateLock.unlock()
return
}
drainStarted = true
stateLock.unlock()
let thread = Thread { [connection, flag, drainDone] in
@@ -308,6 +433,80 @@ public final class SessionAudio {
// MARK: - Mic (mic host)
#if !os(tvOS)
/// One engine, both directions: engage the system voice processor on the shared IO unit
/// (AEC + noise suppression + AGC), hang the playback source off its render side and the
/// mic tap off its capture side. Every failure falls back to a WORKING configuration
/// the split path (no AEC) when the voice processor won't engage, plain playback when the
/// mic chain can't be built a session never loses audio to the echo-cancel feature.
private func startCombined(speakerUID: String, micUID: String, micChannel: Int) {
let engine = AVAudioEngine()
let input = engine.inputNode
do {
// Before anything reads the input's format: the voice processor changes it (often
// to its own mono mix, sometimes at a lower rate) installMicTap reads the format
// AFTER this, so the converter chain adapts to whatever the processor emits.
try input.setVoiceProcessingEnabled(true)
} catch {
log.warning("""
voice processing unavailable (\(error.localizedDescription)) separate \
engines, no echo cancellation
""")
startPlayback(speakerUID: speakerUID)
startCapture(micUID: micUID, micChannel: micChannel)
return
}
// Symmetric enable for the render side; with both directions on one engine the
// input-node enable already covers it, so a refusal here is not a failure.
try? engine.outputNode.setVoiceProcessingEnabled(true)
// This is a game stream, not a call: never duck the host's audio under the outgoing
// voice. .min is the closest to "off" the API offers, and advanced (selective)
// ducking stays off with it.
input.voiceProcessingOtherAudioDuckingConfiguration = .init(
enableAdvancedDucking: false, duckingLevel: .min)
guard let (ring, source, format) = makePlaybackChain() else {
// Playback impossible (logged) keep the uplink alive, as the split path would.
startCapture(micUID: micUID, micChannel: micChannel)
return
}
engine.attach(source)
engine.connect(source, to: engine.mainMixerNode, format: format)
guard installMicTap(on: input, micUID: micUID, micChannel: micChannel) else {
// Mic chain unavailable (logged) keep the session audible on the plain playback
// engine rather than playing through an idle voice processor.
startPlayback(speakerUID: speakerUID)
return
}
engine.prepare()
do {
try engine.start()
} catch {
log.error("combined engine failed to start: \(error.localizedDescription)")
input.removeTap(onBus: 0)
startPlayback(speakerUID: speakerUID) // no echo cancellation beats no audio
return
}
stateLock.lock()
if flag.isStopped {
stateLock.unlock()
input.removeTap(onBus: 0)
engine.stop() // stop() already ran don't strand a started engine (or a hot mic)
return
}
combinedEngine = engine
let muted = micMuted // latched before this engine existed (a mute during the prompt)
stateLock.unlock()
apply(micMuted: muted, capture: nil, combined: engine)
startDrain(into: ring)
log.info("audio engines joined — voice processing (echo cancellation) active")
}
/// The split path: capture on its OWN engine, playback on another the pre-echo-cancel
/// topology, kept verbatim. Two engines, not one a single AVAudioEngine ties
/// input+output to one aggregate clock, separate engines keep arbitrary mic/speaker
/// combinations trivial. That freedom is exactly why the voice processor can't ride this
/// path (AEC needs both directions on one unit) and why explicitly pinned endpoints land
/// here see `wantsCombined`.
private func startCapture(micUID: String, micChannel: Int) {
let engine = AVAudioEngine()
let input = engine.inputNode
@@ -322,11 +521,44 @@ public final class SessionAudio {
}
}
#endif
guard installMicTap(on: input, micUID: micUID, micChannel: micChannel) else { return }
engine.prepare()
do {
try engine.start()
} catch {
log.error("capture engine failed to start: \(error.localizedDescription)")
input.removeTap(onBus: 0)
return
}
stateLock.lock()
if flag.isStopped {
// stop() ran while we were starting (the permission prompt resolves at the
// user's leisure) tear the engine down ourselves, nobody else owns it now.
stateLock.unlock()
input.removeTap(onBus: 0)
engine.stop()
return
}
captureEngine = engine
let muted = micMuted // latched before this engine existed (a mute during the prompt)
stateLock.unlock()
apply(micMuted: muted, capture: engine, combined: nil)
log.info("mic uplink started (\(micUID.isEmpty ? "default input" : micUID))")
}
/// Resolve the input's live format + fold plan, build the monoOpus chain, and install the
/// capture tap on `input` everything mic except engine ownership, shared verbatim by the
/// combined and split topologies. Reads `input.outputFormat(forBus:)` at call time, so the
/// chain follows whatever the node emits: the raw device format, or the voice processor's
/// own mix when that's enabled. False (logged) when no input is usable or the encoder
/// can't be built; the tap is installed on true.
private func installMicTap(
on input: AVAudioInputNode, micUID: String, micChannel: Int
) -> Bool {
let inFormat = input.outputFormat(forBus: 0)
guard inFormat.sampleRate > 0, inFormat.channelCount > 0 else {
log.error("no usable input device — mic uplink disabled")
return
return false
}
// Multi-channel-interface handling. A pro interface exposes N discrete inputs with the mic
@@ -378,22 +610,38 @@ public final class SessionAudio {
#endif
// Encode a single mono bus (folded from `inFormat` in the tap): the resampler goes
// mono@inputSR the encoder's 48 kHz stereo, so it handles both the rate change and the
// monostereo duplication, and the wrong-channel downmix never happens.
// mono@inputSR the encoder's 48 kHz mono, so it handles the rate change and the
// wrong-channel downmix never happens. Mono end to end the host's decoder upmixes,
// so the old duplicate-into-stereo step only cost bits and cycles.
//
// `mono`/`staging` are the per-callback scratch buffers, preallocated HERE (grown only
// if a larger-than-expected device quantum ever arrives) the steady-state tap path
// allocates nothing.
let scratchFrames: AVAudioFrameCount = 8192
let stagingCapacity = { (frames: AVAudioFrameCount) -> AVAudioFrameCount in
AVAudioFrameCount(
(Double(frames) * 48_000 / inFormat.sampleRate).rounded(.up)) + 64
}
guard let monoFormat = AVAudioFormat(
commonFormat: .pcmFormatFloat32, sampleRate: inFormat.sampleRate,
channels: 1, interleaved: false),
let encoder = try? OpusEncoder(),
let resampler = AVAudioConverter(from: monoFormat, to: encoder.pcmFormat),
let chunk = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: OpusEncoder.framesPerPacket)
pcmFormat: encoder.pcmFormat, frameCapacity: encoder.framesPerPacket),
let monoScratch = AVAudioPCMBuffer(
pcmFormat: monoFormat, frameCapacity: scratchFrames),
let stagingScratch = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: stagingCapacity(scratchFrames))
else {
log.error("Opus encoder unavailable — mic uplink disabled")
return
return false
}
// Tap-thread-confined state: resample into `staging`, accumulate in `fifo`,
// slice 960-frame chunks for the encoder.
// Tap-thread-confined state: fold into `mono`, resample into `staging`, accumulate in
// `fifo`, slice `framesPerPacket` (10 ms) chunks for the encoder.
var mono = monoScratch
var staging = stagingScratch
var fifo: [Float] = []
fifo.reserveCapacity(48_000)
var seq: UInt32 = 0
@@ -412,14 +660,26 @@ public final class SessionAudio {
var inputPeak: Float = 0
var levelReported = false
input.installTap(onBus: 0, bufferSize: 2048, format: inFormat) { buffer, _ in
// 480 frames = 10 ms, matching the packet duration. Advisory CoreAudio delivers the
// device quantum whatever we ask (the old 2048 request came back as 42.7 ms bursts, most
// of the uplink's latency) but where the system honors it, the tap fires per-packet.
input.installTap(onBus: 0, bufferSize: 480, format: inFormat) { buffer, _ in
if flag.isStopped { return }
let frames = Int(buffer.frameLength)
guard frames > 0, let src = buffer.floatChannelData,
let mono = AVAudioPCMBuffer(
pcmFormat: monoFormat, frameCapacity: buffer.frameLength),
let dst = mono.floatChannelData?[0]
else { return }
guard frames > 0, let src = buffer.floatChannelData else { return }
if frames > Int(mono.frameCapacity) {
// A quantum larger than the scratch (bufferSize is advisory both ways) regrow
// once to the new high-water mark; the steady state stays allocation-free.
guard let biggerMono = AVAudioPCMBuffer(
pcmFormat: monoFormat, frameCapacity: buffer.frameLength),
let biggerStaging = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat,
frameCapacity: stagingCapacity(buffer.frameLength))
else { return }
mono = biggerMono
staging = biggerStaging
}
guard let dst = mono.floatChannelData?[0] else { return }
mono.frameLength = buffer.frameLength
// Fold the multi-channel input down to the one mono bus we encode.
@@ -451,11 +711,6 @@ public final class SessionAudio {
}
}
let ratio = 48_000 / inFormat.sampleRate
let outCapacity = AVAudioFrameCount((Double(frames) * ratio).rounded(.up) + 64)
guard let staging = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: outCapacity)
else { return }
var fed = false
var convError: NSError?
let status = resampler.convert(to: staging, error: &convError) { _, outStatus in
@@ -469,16 +724,20 @@ public final class SessionAudio {
}
guard status != .error, let p = staging.floatChannelData?[0] else { return }
fifo.append(contentsOf: UnsafeBufferPointer(
start: p, count: Int(staging.frameLength) * 2))
start: p, count: Int(staging.frameLength)))
let samplesPerChunk = Int(OpusEncoder.framesPerPacket) * 2
while fifo.count >= samplesPerChunk {
chunk.frameLength = OpusEncoder.framesPerPacket
// Consume whole chunks through a head index, then drop the eaten prefix in ONE
// move of the sub-chunk remainder. The old per-chunk removeFirst memmoved the
// entire backlog for every packet O(n) on the render-adjacent tap thread.
let samplesPerChunk = Int(encoder.framesPerPacket)
var head = 0
while fifo.count - head >= samplesPerChunk {
chunk.frameLength = encoder.framesPerPacket
fifo.withUnsafeBufferPointer { src in
chunk.floatChannelData![0].update(
from: src.baseAddress!, count: samplesPerChunk)
from: src.baseAddress! + head, count: samplesPerChunk)
}
fifo.removeFirst(samplesPerChunk)
head += samplesPerChunk
guard let packets = try? encoder.encode(chunk) else { continue }
for packet in packets {
connection.sendMic(
@@ -486,28 +745,9 @@ public final class SessionAudio {
seq &+= 1
}
}
if head > 0 { fifo.removeFirst(head) } // keeps capacity no realloc
}
engine.prepare()
do {
try engine.start()
} catch {
log.error("capture engine failed to start: \(error.localizedDescription)")
input.removeTap(onBus: 0)
return
}
stateLock.lock()
if flag.isStopped {
// stop() ran while we were starting (the permission prompt resolves at the
// user's leisure) tear the engine down ourselves, nobody else owns it now.
stateLock.unlock()
input.removeTap(onBus: 0)
engine.stop()
return
}
captureEngine = engine
stateLock.unlock()
log.info("mic uplink started (\(micUID.isEmpty ? "default input" : micUID))")
return true
}
/// Fold `channels` of input (`floatChannelData` layout: `interleaved` one buffer strided by
@@ -128,11 +128,19 @@ public final class InputCapture {
/// carries the same key equivalents for discoverability) can't see them, so the monitor is the
/// captured-state delivery path; released, the events pass through and the menu handles them.
/// Q releases the captured mouse/keyboard; D disconnects; S cycles the stats
/// overlay tier (off compact normal detailed). Main queue.
/// overlay tier (off compact normal detailed). A (`onToggleMicMute`, below) rides
/// the same path. Main queue.
public var onReleaseCapture: (() -> Void)?
public var onDisconnect: (() -> Void)?
public var onCycleStats: (() -> Void)?
/// Fired on A mute/unmute the microphone uplink, the one in-stream control a captured
/// session can't otherwise reach (the HUD's button is behind a grabbed cursor). Same delivery
/// rule as the combos above: only WHILE FORWARDING, because that's when the menu's identical
/// key equivalent can't fire. M the obvious letter is long since the mouse-model flip
/// (cross-client), so A ("audio in") is the mic's. Main queue.
public var onToggleMicMute: (() -> Void)?
/// Fired on F (macOS) toggle the streaming window in/out of fullscreen. Detected in the
/// monitor only WHILE FORWARDING, for the same reason as the combos: a captured stream view
/// swallows keys, so the Stream menu's identical F equivalent never reaches it; released, the
@@ -140,13 +148,13 @@ public final class InputCapture {
public var onToggleFullscreen: (() -> Void)?
#if os(iOS)
/// Windows VKs of the three modifier classes in the Q release chord, both L/R sides:
/// Windows VKs of the three modifier classes in the chords, both L/R sides:
/// control (0xA2/0xA3), option (0xA4/0xA5), shift (0xA0/0xA1). Used to sift the HID key stream.
private static let chordModifierVKs: Set<UInt32> = [0xA2, 0xA3, 0xA4, 0xA5, 0xA0, 0xA1]
/// Whether Control AND Option AND Shift are all currently held (either side of each counts)
/// the modifier precondition for the iPad Q release chord.
private var hasReleaseChordModifiers: Bool {
/// the modifier precondition for the iPad chords (Q releases capture, A mutes the mic).
private var hasChordModifiers: Bool {
let m = chordModifiersDown
return (m.contains(0xA2) || m.contains(0xA3)) // control
&& (m.contains(0xA4) || m.contains(0xA5)) // option
@@ -284,6 +292,10 @@ public final class InputCapture {
self.suppressedVK = 0x53
self.onCycleStats?()
return nil
case 0 /* A */:
self.suppressedVK = 0x41
self.onToggleMicMute?()
return nil
default:
break
}
@@ -704,7 +716,7 @@ public final class InputCapture {
}
}
#if os(iOS)
// Track Control/Option/Shift for the Q release chord below in both forwarding
// Track Control/Option/Shift for the chords below in both forwarding
// states (like `cmdKeysDown`) so a modifier held before capture engaged still counts.
if Self.chordModifierVKs.contains(vk) {
if pressed { self.chordModifiersDown.insert(vk) } else { self.chordModifiersDown.remove(vk) }
@@ -732,11 +744,19 @@ public final class InputCapture {
// otherwise). The Q is latched (`suppressedVK`) so its keyUp can't type into the host;
// the modifiers were forwarded as they went down and are flushed by the release
// path (setCaptured(false) releaseAll). VK 0x51 is layout-independent (physical Q).
if pressed, vk == 0x51, self.hasReleaseChordModifiers {
if pressed, vk == 0x51, self.hasChordModifiers {
self.suppressedVK = 0x51
self.onReleaseCapture?()
return
}
// A mutes/unmutes the mic uplink same detection, same latching, and needed here
// for the same reason as on macOS: a captured iPad swallows the Stream menu's
// identical key equivalent. VK 0x41 is layout-independent (physical A).
if pressed, vk == 0x41, self.hasChordModifiers {
self.suppressedVK = 0x41
self.onToggleMicMute?()
return
}
#endif
// Release direction of the toggle: GC's Esc-down can beat the NSEvent
// monitor never type Esc into the host while is held ( is reserved).
@@ -881,6 +881,12 @@ public final class StreamLayerView: NSView {
guard self?.window?.isKeyWindow == true else { return }
NotificationCenter.default.post(name: .punktfunkToggleFullscreen, object: nil)
}
capture.onToggleMicMute = { [weak self] in
// Session-level state the view doesn't own post to the app (same routing as the
// fullscreen chord), so the captured and released paths end at one toggle.
guard self?.window?.isKeyWindow == true else { return }
NotificationCenter.default.post(name: .punktfunkToggleMicMute, object: nil)
}
capture.onCycleStats = { [weak self] in
guard self?.window?.isKeyWindow == true else { return }
// Advance the shared tier setting directly every @AppStorage reader (the HUD's
@@ -424,6 +424,12 @@ public final class StreamViewController: StreamViewControllerBase {
capture.onReleaseCapture = { [weak self] in
self?.setCaptured(false)
}
// A mutes/unmutes the mic uplink. Session state this controller doesn't own, so it
// posts to the app exactly as the macOS chord does the Stream menu's identical
// equivalent (which a captured scene swallows) ends at the same toggle.
capture.onToggleMicMute = {
NotificationCenter.default.post(name: .punktfunkToggleMicMute, object: nil)
}
capture.onPreempted = { [weak self] in
self?.setCaptured(false)
}
@@ -42,6 +42,13 @@ public enum DefaultsKey {
/// falls back. Drives the decoder via `Welcome.codec`.
public static let codec = "punktfunk.codec"
public static let micEnabled = "punktfunk.micEnabled"
/// Echo cancellation for the mic uplink (on by default): playback + capture share ONE
/// audio engine so the system voice processor can subtract what this device is playing
/// from what its mic hears without it a loudspeaker client feeds the game audio straight
/// back to the host. Off = the raw two-engine capture path. macOS: an explicitly pinned
/// speaker/mic or mic channel also bypasses it (the voice processor only follows the
/// system default devices) see SessionAudio's topology note.
public static let echoCancel = "punktfunk.echoCancel"
public static let speakerUID = "punktfunk.speakerUID"
public static let micUID = "punktfunk.micUID"
/// macOS: which input channel of the chosen mic device feeds the host. 0 = "Auto" (sum every
@@ -193,6 +200,12 @@ extension Notification.Name {
/// state. macOS only.
public static let punktfunkToggleFullscreen = Notification.Name("io.unom.punktfunk.toggle-fullscreen")
/// Posted by InputCapture's chord path (A) when the combo fires while input is CAPTURED
/// the state in which the Stream menu's identical key equivalent never reaches the app. The
/// live session's owner (ContentView) flips the session's mic mute. Released, the menu item
/// handles the same combo directly; both end at `SessionModel.toggleMicMute`.
public static let punktfunkToggleMicMute = Notification.Name("io.unom.punktfunk.toggle-mic-mute")
/// Posted by the Live Activity's / Shortcuts' End-stream intent (`EndStreamIntent.perform`,
/// which runs in the app's process): the app tears the active session down deliberately
/// (quit-close the host). Same cross-process-signal pattern as `punktfunkReleaseCapture`
@@ -29,6 +29,7 @@ public struct EffectiveSettings: Equatable, Sendable {
public var compositor = 0
public var audioChannels = 2
public var micEnabled = true
public var echoCancel = true
public var touchMode = "trackpad"
public var mouseMode = "capture"
public var invertScroll = false
@@ -87,6 +88,7 @@ public struct EffectiveSettings: Equatable, Sendable {
compositor = int(DefaultsKey.compositor, compositor)
audioChannels = int(DefaultsKey.audioChannels, audioChannels)
micEnabled = bool(DefaultsKey.micEnabled, micEnabled)
echoCancel = bool(DefaultsKey.echoCancel, echoCancel)
touchMode = str(DefaultsKey.touchMode, touchMode)
mouseMode = str(DefaultsKey.mouseMode, mouseMode)
invertScroll = bool(DefaultsKey.invertScroll, invertScroll)
@@ -133,6 +135,7 @@ public struct EffectiveSettings: Equatable, Sendable {
if let v = overlay.compositor { s.compositor = v }
if let v = overlay.audioChannels { s.audioChannels = v }
if let v = overlay.micEnabled { s.micEnabled = v }
if let v = overlay.echoCancel { s.echoCancel = v }
if let v = overlay.touchMode { s.touchMode = v }
if let v = overlay.mouseMode { s.mouseMode = v }
if let v = overlay.invertScroll { s.invertScroll = v }
@@ -105,6 +105,7 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
public var compositor: Int?
public var audioChannels: Int?
public var micEnabled: Bool?
public var echoCancel: Bool?
public var touchMode: String?
public var mouseMode: String?
public var invertScroll: Bool?
@@ -145,6 +146,7 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
case compositor
case audioChannels = "audio_channels"
case micEnabled = "mic_enabled"
case echoCancel = "echo_cancel"
case touchMode = "touch_mode"
case mouseMode = "mouse_mode"
case invertScroll = "invert_scroll"
@@ -177,6 +179,7 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
compositor = int(.compositor)
audioChannels = int(.audioChannels)
micEnabled = bool(.micEnabled)
echoCancel = bool(.echoCancel)
touchMode = str(.touchMode)
mouseMode = str(.mouseMode)
invertScroll = bool(.invertScroll)
@@ -211,6 +214,7 @@ public struct SettingsOverlay: Codable, Equatable, Sendable {
try c.encodeIfPresent(compositor, forKey: AnyKey(Key.compositor.rawValue))
try c.encodeIfPresent(audioChannels, forKey: AnyKey(Key.audioChannels.rawValue))
try c.encodeIfPresent(micEnabled, forKey: AnyKey(Key.micEnabled.rawValue))
try c.encodeIfPresent(echoCancel, forKey: AnyKey(Key.echoCancel.rawValue))
try c.encodeIfPresent(touchMode, forKey: AnyKey(Key.touchMode.rawValue))
try c.encodeIfPresent(mouseMode, forKey: AnyKey(Key.mouseMode.rawValue))
try c.encodeIfPresent(invertScroll, forKey: AnyKey(Key.invertScroll.rawValue))
@@ -262,6 +266,7 @@ public enum OverlayField {
case "compositor": overlay.compositor = nil
case "audio_channels": overlay.audioChannels = nil
case "mic_enabled": overlay.micEnabled = nil
case "echo_cancel": overlay.echoCancel = nil
case "touch_mode": overlay.touchMode = nil
case "mouse_mode": overlay.mouseMode = nil
case "invert_scroll": overlay.invertScroll = nil
@@ -296,6 +301,7 @@ public enum OverlayField {
case "compositor": return o.compositor != nil
case "audio_channels": return o.audioChannels != nil
case "mic_enabled": return o.micEnabled != nil
case "echo_cancel": return o.echoCancel != nil
case "touch_mode": return o.touchMode != nil
case "mouse_mode": return o.mouseMode != nil
case "invert_scroll": return o.invertScroll != nil
@@ -9,32 +9,34 @@ import XCTest
@testable import PunktfunkKit
final class OpusCodecTests: XCTestCase {
/// Encode a 440 Hz stereo tone, decode it back, and require the result to be
/// recognizably the same signal (Opus is lossy check correlation, not bytes).
/// Encode a 440 Hz mono tone (the uplink's shape), decode it back through the
/// STEREO-configured decoder (the host-plane shape Opus upmixes mono packets), and
/// require the result to be recognizably the same signal (Opus is lossy check
/// correlation, not bytes).
func testEncodeDecodeRoundTripPreservesTone() throws {
let encoder = try OpusEncoder()
let decoder = try OpusDecoder(framesPerPacket: UInt32(OpusEncoder.framesPerPacket))
let decoder = try OpusDecoder(framesPerPacket: UInt32(encoder.framesPerPacket))
let pcmFormat = encoder.pcmFormat
let frames = OpusEncoder.framesPerPacket
let frames = encoder.framesPerPacket
var packets: [Data] = []
var phase: Float = 0
let step = 2 * Float.pi * 440 / 48_000
// 50 packets = 1 s of tone.
for _ in 0..<50 {
// 1 s of tone, whatever packet duration the encoder chose (10 ms 100 chunks).
let chunks = Int(48_000 / frames)
for _ in 0..<chunks {
let buf = AVAudioPCMBuffer(pcmFormat: pcmFormat, frameCapacity: frames)!
buf.frameLength = frames
let p = buf.floatChannelData![0] // interleaved: one plane, L R L R
let p = buf.floatChannelData![0] // mono: one plane
for f in 0..<Int(frames) {
let s = sin(phase) * 0.5
p[f] = sin(phase) * 0.5
phase += step
p[f * 2] = s
p[f * 2 + 1] = s
}
packets.append(contentsOf: try encoder.encode(buf))
}
XCTAssertGreaterThanOrEqual(packets.count, 45, "encoder must emit ~one packet per buffer")
XCTAssertGreaterThanOrEqual(
packets.count, chunks - 5, "encoder must emit ~one packet per buffer")
XCTAssertTrue(packets.allSatisfy { !$0.isEmpty })
var decoded: [Float] = []
@@ -62,29 +62,30 @@ final class RemoteFirstLightTests: XCTestCase {
host: host, port: port, width: 1280, height: 720, refreshHz: 60)
defer { conn.close() }
// Mic uplink: 2 s of 440 Hz tone (the host's mic service opens its virtual
// Mic uplink: 2 s of 440 Hz mono tone (the host's mic service opens its virtual
// source on the first frame check its log).
let encoder = try OpusEncoder()
let chunk = AVAudioPCMBuffer(
pcmFormat: encoder.pcmFormat, frameCapacity: OpusEncoder.framesPerPacket)!
pcmFormat: encoder.pcmFormat, frameCapacity: encoder.framesPerPacket)!
var phase: Float = 0
let step = 2 * Float.pi * 440 / 48_000
var seq: UInt32 = 0
for _ in 0..<100 {
chunk.frameLength = OpusEncoder.framesPerPacket
let p = chunk.floatChannelData![0]
for f in 0..<Int(OpusEncoder.framesPerPacket) {
let s = sin(phase) * 0.25
let chunks = 2 * 48_000 / Int(encoder.framesPerPacket)
let packetNs = UInt64(encoder.framesPerPacket) * 1_000_000_000 / 48_000
for _ in 0..<chunks {
chunk.frameLength = encoder.framesPerPacket
let p = chunk.floatChannelData![0] // mono: one plane
for f in 0..<Int(encoder.framesPerPacket) {
p[f] = sin(phase) * 0.25
phase += step
p[f * 2] = s
p[f * 2 + 1] = s
}
for packet in try encoder.encode(chunk) {
conn.sendMic(packet, seq: seq, ptsNs: UInt64(seq) * 20_000_000)
conn.sendMic(packet, seq: seq, ptsNs: UInt64(seq) * packetNs)
seq &+= 1
}
}
XCTAssertGreaterThanOrEqual(seq, 95, "mic encoder must emit ~one packet per chunk")
XCTAssertGreaterThanOrEqual(
seq, UInt32(chunks - 5), "mic encoder must emit ~one packet per chunk")
// Downlink: pull host audio packets and decode them (the host streams its sink
// monitor silence still produces packets).
+6
View File
@@ -1019,6 +1019,12 @@ pub fn shortcuts_window(parent: &adw::ApplicationWindow) -> gtk::ShortcutsWindow
<property name="accelerator">&lt;Control&gt;&lt;Alt&gt;&lt;Shift&gt;s</property>
</object>
</child>
<child>
<object class="GtkShortcutsShortcut">
<property name="title">Mute or unmute your microphone (only while the stream sends one)</property>
<property name="accelerator">&lt;Control&gt;&lt;Alt&gt;&lt;Shift&gt;v</property>
</object>
</child>
</object>
</child>
</object>
+17 -2
View File
@@ -480,8 +480,23 @@ fn headless_check_update() -> glib::ExitCode {
"installed {} ({}, {})",
status.current, status.kind, status.channel
);
println!("available {}", status.latest);
if let Some(err) = &status.error {
// `latest` falls back to `current` when the check couldn't run — printing that as
// "available" would read as a confirmed answer we don't have.
if status.error.is_some() {
println!("available unknown");
} else {
println!("available {}", status.latest);
}
if status.not_published {
// Says what it is, in words, instead of a raw HTTP status. The exit code still
// reports "could not tell" (see the doc comment above): an empty channel is the
// absence of evidence that this build is current, and a mistyped
// PUNKTFUNK_UPDATE_FEED is indistinguishable from one out here.
println!(
"update nothing published on the {} channel yet",
status.channel
);
} else if let Some(err) = &status.error {
eprintln!("check-update: {err}");
} else if status.update_available {
println!("update yes");
+29 -2
View File
@@ -607,6 +607,9 @@ fn commit_profile(active: &StreamProfile, touched: &Touched, values: &Settings)
if touched.has("mic_enabled") {
o.mic_enabled = Some(values.mic_enabled);
}
if touched.has("echo_cancel") {
o.echo_cancel = Some(values.echo_cancel);
}
if touched.has("touch_mode") {
o.touch_mode = Some(values.touch_mode.clone());
}
@@ -1301,7 +1304,11 @@ pub fn show_scoped(
);
let mic_row = adw::SwitchRow::builder()
.title("Stream microphone")
.subtitle("Sends your microphone to the host's virtual mic")
.subtitle("Sends your microphone to the host's virtual mic — Ctrl+Alt+Shift+V mutes it mid-stream")
.build();
let echo_row = adw::SwitchRow::builder()
.title("Echo cancellation")
.subtitle("Keeps the host's audio, playing from this machine's speakers, out of the uplink")
.build();
// Endpoint pickers (from the PipeWire probe): visible labels are descriptions, the
// stored value is the node name. Hidden when the probe found nothing; a saved
@@ -1345,12 +1352,23 @@ pub fn show_scoped(
"Microphone",
"The input that feeds the host's virtual mic",
);
// The device pick only matters while the mic streams at all.
// The device pick and the echo canceller only matter while the mic streams at all — both
// follow it. One handler each (the pickers are optional, the echo row never is), and the
// initial state is set here because the seed block further down fires these too.
//
// Insensitivity covers the whole row, including the per-row Reset a profile scope adds:
// an echo_cancel override can only be reset while the mic row is on. Turn it on, reset,
// turn it back off — the alternative is a control that looks live and isn't.
if let Some(r) = &micdev_row {
let w = r.widget().clone();
w.set_sensitive(mic_row.is_active());
mic_row.connect_active_notify(move |m| w.set_sensitive(m.is_active()));
}
{
let w = echo_row.clone();
w.set_sensitive(mic_row.is_active());
mic_row.connect_active_notify(move |m| w.set_sensitive(m.is_active()));
}
// ---- Controllers ----
// Controller forwarding: Automatic forwards EVERY real controller, each as its own pad
@@ -1453,6 +1471,7 @@ pub fn show_scoped(
inhibit_row.set_active(s.inhibit_shortcuts);
invert_row.set_active(s.invert_scroll);
mic_row.set_active(s.mic_enabled);
echo_row.set_active(s.echo_cancel);
hdr_row.set_active(s.hdr_enabled);
chroma_row.set_active(s.enable_444);
library_row.set_active(s.library_enabled);
@@ -1673,6 +1692,12 @@ pub fn show_scoped(
invert_scroll
);
toggle!(mic_row, "mic_enabled", o.mic_enabled.is_some(), mic_enabled);
toggle!(
echo_row,
"echo_cancel",
o.echo_cancel.is_some(),
echo_cancel
);
{
let revert = {
let (row, globals, touched) =
@@ -1781,6 +1806,7 @@ pub fn show_scoped(
audio_group.add(r.widget());
}
audio_group.add(&mic_row);
audio_group.add(&echo_row);
if let (Some(r), false) = (&micdev_row, profile_mode) {
audio_group.add(r.widget());
}
@@ -1890,6 +1916,7 @@ pub fn show_scoped(
s.inhibit_shortcuts = inhibit_row.is_active();
s.invert_scroll = invert_row.is_active();
s.mic_enabled = mic_row.is_active();
s.echo_cancel = echo_row.is_active();
s.hdr_enabled = hdr_row.is_active();
s.enable_444 = chroma_row.is_active();
s.audio_channels = match surround_row.selected() {
+2 -1
View File
@@ -61,6 +61,7 @@ default ≈1000 nits). The host still gates the upgrade behind its `PUNKTFUNK_10
policy.
Debug/bisect knobs: `PUNKTFUNK_DECODER=vulkan|vaapi|d3d11va|software`, `PUNKTFUNK_PRESENT_MODE=
mailbox|immediate` (default FIFO), `PUNKTFUNK_VK_DEVICE=<index>` (multi-GPU), and
mailbox|fifo|immediate|fifo_relaxed` (default MAILBOX, FIFO where the surface offers no
MAILBOX — AMD on Windows), `PUNKTFUNK_VK_DEVICE=<index>` (multi-GPU), and
`PUNKTFUNK_HW_FAULT=import` (fault every VAAPI dmabuf import — proves the three-strike
demotion to software on healthy hardware).
+1
View File
@@ -290,6 +290,7 @@ mod session_main {
// compositors only).
cursor_forward: settings.mouse_mode() == trust::MouseMode::Desktop,
mic_enabled: settings.mic_enabled,
echo_cancel: settings.echo_cancel,
clipboard,
// The Settings preference (auto → VAAPI where it exists; the presenter
// demotes to software on boxes whose Vulkan can't import the dmabufs).
+4
View File
@@ -21,6 +21,10 @@ const STREAM_SHORTCUTS: &[(&str, &str)] = &[
"Ctrl+Alt+Shift+S",
"Cycle the statistics overlay (off \u{00B7} compact \u{00B7} normal \u{00B7} detailed)",
),
(
"Ctrl+Alt+Shift+V",
"Mute or unmute your microphone (only while the stream sends one)",
),
(
"LB+RB+Start+Back",
"Controller: release input / leave fullscreen \u{2014} hold to disconnect",
+32 -1
View File
@@ -439,6 +439,7 @@ struct OverrideFlags {
compositor: bool,
audio_channels: bool,
mic_enabled: bool,
echo_cancel: bool,
touch_mode: bool,
mouse_mode: bool,
invert_scroll: bool,
@@ -466,6 +467,7 @@ impl OverrideFlags {
compositor: o.compositor.is_some(),
audio_channels: o.audio_channels.is_some(),
mic_enabled: o.mic_enabled.is_some(),
echo_cancel: o.echo_cancel.is_some(),
touch_mode: o.touch_mode.is_some(),
mouse_mode: o.mouse_mode.is_some(),
invert_scroll: o.invert_scroll.is_some(),
@@ -954,6 +956,12 @@ pub(crate) fn settings_page(
};
let speaker_combo = dev_combo(&s.speaker_device, &speakers, |s, v| s.speaker_device = v);
let mic_dev_combo = dev_combo(&s.mic_device, &mics, |s, v| s.mic_device = v);
// Echo cancellation is meaningless without an uplink, so it greys out with the mic above
// it. Every commit bumps `rev` and re-renders this screen, so the two stay in step live.
let echo_toggle = setting_toggle(ctx, scope, (rev, set_rev), s.echo_cancel, |s, on| {
s.echo_cancel = on
})
.enabled(s.mic_enabled);
let (hud_names, hud_i) = presets(STATS_TIERS, |v| *v == s.stats_verbosity());
let hud_combo = setting_combo(ctx, scope, (rev, set_rev), hud_names, hud_i, |s, i| {
@@ -1281,7 +1289,8 @@ pub(crate) fn settings_page(
"Stream microphone to the host",
over.mic_enabled,
mic_toggle,
"This device\u{2019}s microphone feeds the host\u{2019}s virtual mic.",
"This device\u{2019}s microphone feeds the host\u{2019}s virtual mic. \
Ctrl+Alt+Shift+V mutes and unmutes it during a stream.",
)),
(!profile_mode)
.then(|| {
@@ -1294,6 +1303,17 @@ pub(crate) fn settings_page(
})
})
.flatten(),
Some(described_overridable(
(rev, set_rev),
scope,
"echo_cancel",
"Echo cancellation",
over.echo_cancel,
echo_toggle,
"Keeps the host\u{2019}s audio, playing from this machine\u{2019}s \
speakers, from being picked up and sent straight back. Turn it off if \
your microphone already does its own processing.",
)),
]
.into_iter()
.flatten()
@@ -1696,5 +1716,16 @@ mod tests {
..Default::default()
};
assert!(OverrideFlags::of(Some(&p2)).resolution);
// The audio pair: the mic and its echo canceller are separate overrides, so a profile
// can pin one without claiming the other.
let mut p3 = StreamProfile::new("t3".to_string());
p3.overrides = SettingsOverlay {
echo_cancel: Some(false),
..Default::default()
};
let f3 = OverrideFlags::of(Some(&p3));
assert!(f3.echo_cancel);
assert!(!f3.mic_enabled);
}
}
+32 -14
View File
@@ -1670,6 +1670,22 @@ impl IddPushCapturer {
// the running correlated/total tally — lives on `StallWatch` (sweep Phase 5.4). It was
// ~65 lines of log prose inside `try_consume`, which is the hot loop, and its two
// counters were capturer fields that nothing else touched.
// One ETW read serves both evidence fields: the prose summary spans the gap plus
// the same 300 ms lead-in the report's OS-event correlation uses (the disturbance
// that CAUSED the hole lands just before it), while the discriminator counts span
// the GAP ONLY — no lead-in: presents from the healthy flow right before the hole
// would falsely acquit the content (the stall-ending frame's own present lands at
// the window edge and stays well under the acquit bar). Both halves must come from
// the same ring snapshot under the same clock anchor, or the prose and the verdict
// can disagree about the same hole.
let (etw, etw_counts) = self
.etw
.as_ref()
.and_then(|w| {
now.checked_sub(stall.gap)
.map(|from| w.window_report(from, now, Duration::from_millis(300)))
})
.unzip();
let evidence = StallEvidence {
// A publisher re-attach restarts `offered_total` near zero; a ring recreate resets
// the stall watch before that can matter, but guard the delta anyway (a restarted
@@ -1682,24 +1698,14 @@ impl IddPushCapturer {
}
}),
max_heartbeat_age_ms: self.max_hb_age_us / 1_000,
// The probe + ETW reads span the same window the report's OS-event correlation
// uses (the gap plus a lead-in for the disturbance that CAUSED it).
// The probe read spans the same window the report's OS-event correlation uses
// (the gap plus a lead-in for the disturbance that CAUSED it).
probes: now
.checked_sub(stall.gap + Duration::from_millis(300))
.zip(self.probes.as_deref())
.map(|(from, p)| p.window(from, now)),
etw: self.etw.as_ref().and_then(|w| {
now.checked_sub(stall.gap + Duration::from_millis(300))
.map(|from| w.summary(from, now))
}),
// The discriminator counts span the GAP ONLY — no lead-in: presents from the
// healthy flow right before the hole would falsely acquit the content. The
// stall-ending frame's own present lands at the window edge and stays well
// under the acquit bar.
etw_counts: self.etw.as_ref().and_then(|w| {
now.checked_sub(stall.gap)
.map(|from| w.window_counts(from, now))
}),
etw,
etw_counts,
};
self.stall_watch.report(&stall, now, &evidence);
}
@@ -2453,6 +2459,18 @@ mod tests {
),
StallClass::ContentSilence
);
// A LIVE witness (history true = it demonstrably worked just before the hole) reading
// an exact zero is the strongest content conviction — the zero is a measurement, not
// an absence.
assert_eq!(
classify(
gap,
&StallVerdict::ComposeSilence,
Some(&probes(Some(16_000), Some(20_000), Some(30_000))),
Some(&counts(0, 0))
),
StallClass::ContentSilence
);
// The present witness does NOT overrule the driver's own verdicts or the harder
// classes — it only refines compose-silence.
assert_eq!(
@@ -18,7 +18,7 @@
//! A second provider rides the same session: `Microsoft-Windows-DXGI` (user-mode), filtered to
//! `Present`/`PresentMultiplaneOverlay` starts (ids 42/55) — one event per swapchain present,
//! stamped with the PRESENTING process id. Together they are the compose-silence discriminator
//! ([`EtwWatch::window_counts`]): DXGI presents flowing while `BltQueueAddEntry` gaps = the OS
//! ([`EtwWatch::window_report`]): DXGI presents flowing while `BltQueueAddEntry` gaps = the OS
//! display path dropped composed frames (the real display-path bug); BOTH silent = the content
//! stopped presenting (benign pause — menus/loading/game hitch). The predecessor witnesses are
//! retired for cause: DxgKrnl id 184 `Present` never fires on the modern redirected path, and
@@ -48,7 +48,8 @@ use windows::Win32::System::Diagnostics::Etw::{
EVENT_CONTROL_CODE_ENABLE_PROVIDER, EVENT_FILTER_DESCRIPTOR, EVENT_FILTER_TYPE_EVENT_ID,
EVENT_RECORD, EVENT_TRACE_CONTROL_STOP, EVENT_TRACE_LOGFILEW, EVENT_TRACE_PROPERTIES,
EVENT_TRACE_REAL_TIME_MODE, PROCESSTRACE_HANDLE, PROCESS_TRACE_MODE_EVENT_RECORD,
PROCESS_TRACE_MODE_REAL_TIME, TRACE_LEVEL_INFORMATION, WNODE_FLAG_TRACED_GUID,
PROCESS_TRACE_MODE_RAW_TIMESTAMP, PROCESS_TRACE_MODE_REAL_TIME, TRACE_LEVEL_INFORMATION,
WNODE_FLAG_TRACED_GUID,
};
use windows::Win32::System::Performance::{QueryPerformanceCounter, QueryPerformanceFrequency};
use windows::Win32::System::Threading::{
@@ -122,8 +123,13 @@ fn qpc_freq() -> i64 {
})
}
/// The consumer's per-event callback — record id + QPC timestamp (the session's `ClientContext`
/// is 1, so `TimeStamp` IS a QPC value) and return; runs on the consumer thread.
/// The consumer's per-event callback — record id + timestamp + pid into the ring and return;
/// runs on the consumer thread. `TimeStamp` is a raw QPC value only because BOTH halves of the
/// clock contract hold: `ClientContext = 1` makes QPC the session clock, and the consumer is
/// opened with `PROCESS_TRACE_MODE_RAW_TIMESTAMP`, which is what stops ProcessTrace converting
/// every event's timestamp to FILETIME (100 ns units since 1601) on delivery. Without the flag
/// the conversion happens REGARDLESS of the session clock, and every `ts <= to_q` comparison
/// downstream is against the wrong clock — never true, a witness that silently reads empty.
unsafe extern "system" fn on_event(record: *mut EVENT_RECORD) {
if record.is_null() {
return;
@@ -150,10 +156,10 @@ pub(super) struct EtwWatch {
}
// SAFETY: both fields are plain kernel handle VALUES (u64 wrappers) owned by this watch; every
// operation on them (summary reads the static ring; Drop stops/closes) is thread-safe by the ETW
// API contract, and the singleton hands out only `Arc<EtwWatch>`.
// operation on them (window_report reads the static ring; Drop stops/closes) is thread-safe by
// the ETW API contract, and the singleton hands out only `Arc<EtwWatch>`.
unsafe impl Send for EtwWatch {}
// SAFETY: as above — `&EtwWatch` exposes only `summary` (static-ring reads).
// SAFETY: as above — `&EtwWatch` exposes only `window_report` (static-ring reads).
unsafe impl Sync for EtwWatch {}
static WATCH: Mutex<Weak<EtwWatch>> = Mutex::new(Weak::new());
@@ -201,8 +207,10 @@ impl EtwWatch {
let mut session = CONTROLTRACE_HANDLE::default();
// SAFETY: `buf` is a live, zeroed allocation of base + name bytes; every write below is a
// field of the properties struct at its head; `LoggerNameOffset = base` points at the
// appended name space (ETW copies the name there itself). ClientContext 1 = QPC clock —
// what makes event timestamps comparable to our probe windows.
// appended name space (ETW copies the name there itself). ClientContext 1 selects QPC as
// the SESSION clock — necessary but not sufficient for QPC comparisons: ProcessTrace
// still converts every event's timestamp to FILETIME on delivery unless the consumer is
// opened with PROCESS_TRACE_MODE_RAW_TIMESTAMP (set below).
let rc = unsafe {
let props = buf.as_mut_ptr().cast::<EVENT_TRACE_PROPERTIES>();
(*props).Wnode.BufferSize = buf.len() as u32;
@@ -224,6 +232,11 @@ impl EtwWatch {
);
return None;
}
// A fresh session gets a fresh ring: the static [`RING`] outlives any `EtwWatch`, so
// whatever is in it belongs to a DEAD session — leaking it forward would let a previous
// session's presents pose as this session's witness history. Race-free here: the
// consumer thread that repopulates it is spawned below.
RING.lock().unwrap().clear();
// Enable DxgKrnl with a kernel-side event-id filter — the whole point: the provider's
// vblank/DPC keywords never reach us. Fatal on failure (the DDI families + queue
@@ -240,7 +253,7 @@ impl EtwWatch {
return None;
}
// The DXGI (user-mode) present witness rides the same session. Degraded-not-fatal: a
// refusal only costs the per-process present counts — `window_counts` then reports
// refusal only costs the per-process present counts — `window_report` then reports
// no present history and classification stays honest (Unattributed, never a guess).
if !enable_provider(session, &DXGI, &DXGI_FILTER_IDS) {
tracing::debug!(
@@ -252,8 +265,12 @@ impl EtwWatch {
LoggerName: PWSTR(name.as_ptr() as *mut _),
..Default::default()
};
log.Anonymous1.ProcessTraceMode =
PROCESS_TRACE_MODE_REAL_TIME | PROCESS_TRACE_MODE_EVENT_RECORD;
// RAW_TIMESTAMP is load-bearing: it stops ProcessTrace converting `EVENT_HEADER.TimeStamp`
// to FILETIME on delivery, so events arrive stamped in the session clock (QPC, per the
// ClientContext above) — the only clock the window edges are computed in.
log.Anonymous1.ProcessTraceMode = PROCESS_TRACE_MODE_REAL_TIME
| PROCESS_TRACE_MODE_EVENT_RECORD
| PROCESS_TRACE_MODE_RAW_TIMESTAMP;
log.Anonymous2.EventRecordCallback = Some(on_event);
// SAFETY: `log` is a fully-initialized local; `name` outlives the call (OpenTrace copies
// what it needs before returning).
@@ -303,14 +320,34 @@ impl EtwWatch {
Some(Self { session, consumer })
}
/// Summarize the DDI activity inside `[from, to]` — the correlation line a stall report
/// carries. Brackets that merely SPAN the window count too (a freeze-long `SetPowerState`
/// has both edges outside the hole it caused). `"none"` when the window is clean.
pub(super) fn summary(&self, from: Instant, to: Instant) -> String {
// Instant → QPC: anchor both clocks now and offset backwards.
/// One stall window's ETW evidence, both halves from a SINGLE ring snapshot under a SINGLE
/// `(Instant::now(), qpc_now())` anchor: the DDI/present prose summary a stall report
/// carries, and the structured discriminator counts the classifier folds in. The summary
/// covers `[hole_from - lead_in, hole_to]` — the disturbance that CAUSED a hole lands just
/// before DWM stops delivering, so the prose needs the lead-in. The counts cover
/// `[hole_from, hole_to]` ONLY — presents from the healthy flow inside the lead-in would
/// falsely acquit the content. Two separate reads (two locks, two anchors, syscalls in
/// between) would let events arriving between them make the prose and the verdict disagree
/// about the same hole — hence one method returning both.
///
/// Brackets that merely SPAN the summary window count too (a freeze-long `SetPowerState`
/// has both edges outside the hole it caused). The summary reads `"none"` when the window
/// is clean.
pub(super) fn window_report(
&self,
hole_from: Instant,
hole_to: Instant,
lead_in: Duration,
) -> (String, EtwWindowCounts) {
// Instant → QPC: anchor both clocks once and offset backwards; every window edge below
// derives from this one anchor.
let (now_i, now_q, freq) = (Instant::now(), qpc_now(), qpc_freq());
let to_q = now_q - duration_qpc(now_i.saturating_duration_since(to), freq);
let from_q = now_q - duration_qpc(now_i.saturating_duration_since(from), freq);
let to_q = now_q - duration_qpc(now_i.saturating_duration_since(hole_to), freq);
let from_q = now_q - duration_qpc(now_i.saturating_duration_since(hole_from), freq);
let summary_from_q = from_q - duration_qpc(lead_in, freq);
// One snapshot, then the lock drops: everything below — including the OpenProcess
// syscalls behind `process_name` — runs off the copy, so the consumer callback never
// queues behind a stall report.
let events: Vec<(i64, u16, u32)> = {
let ring = RING.lock().unwrap();
ring.iter()
@@ -318,6 +355,7 @@ impl EtwWatch {
.copied()
.collect()
};
let counts = count_window(&events, from_q, to_q, duration_qpc(LOOKBACK, freq));
let ms = |dq: i64| dq.max(0) * 1_000 / freq;
let mut parts = Vec::new();
for (start_id, stop_id, label) in [
@@ -335,7 +373,7 @@ impl EtwWatch {
} else if id == stop_id {
if let Some(s) = open.take() {
// The bracket [s, ts] counts when it intersects the window.
if s <= to_q && ts >= from_q {
if s <= to_q && ts >= summary_from_q {
count += 1;
max_ms = max_ms.max(ms(ts - s));
}
@@ -362,43 +400,34 @@ impl EtwWatch {
] {
let count = events
.iter()
.filter(|(ts, i, _)| *i == id && *ts >= from_q && *ts <= to_q)
.filter(|(ts, i, _)| *i == id && *ts >= summary_from_q && *ts <= to_q)
.count();
if count > 0 {
parts.push(format!("{label}×{count}"));
}
}
// Present + queue accounting (DXGI 42/55 + BltQueueAddEntry/Complete): total presents
// inside the window plus the top presenters, NAMED — the line that splits a
// inside the summary window plus the top presenters, NAMED — the line that splits a
// compose-silence hole into "the content stopped presenting" (no presents anywhere)
// versus "presents flowed and the display path dropped them" (presents at rate while
// the queue starves). "Present×0" is printed explicitly when the stream has history
// but the window is empty — silence is a finding, not an absence.
// the queue starves). "Present×0" is printed explicitly when the witness was LIVE
// before the hole ([`LOOKBACK`]) but the window is empty — silence is a finding, not
// an absence; a dead witness's window prints nothing rather than a fake zero.
let mut per_pid: Vec<(u32, u32)> = Vec::new();
let mut have_present_history = false;
let (mut adds, mut completes) = (0u32, 0u32);
let mut have_queue_history = false;
for &(ts, id, pid) in &events {
if ts < summary_from_q || ts > to_q {
continue;
}
match id {
DXGI_PRESENT_ID | DXGI_PRESENT_MPO_ID => {
have_present_history = true;
if ts >= from_q && ts <= to_q {
match per_pid.iter_mut().find(|(p, _)| *p == pid) {
Some((_, c)) => *c += 1,
None => per_pid.push((pid, 1)),
}
}
}
BLT_ADD_ID | BLT_COMPLETE_ID => {
have_queue_history = true;
if ts >= from_q && ts <= to_q {
if id == BLT_ADD_ID {
adds += 1;
} else {
completes += 1;
}
match per_pid.iter_mut().find(|(p, _)| *p == pid) {
Some((_, c)) => *c += 1,
None => per_pid.push((pid, 1)),
}
}
BLT_ADD_ID => adds += 1,
BLT_COMPLETE_ID => completes += 1,
_ => {}
}
}
@@ -415,61 +444,80 @@ impl EtwWatch {
.collect::<Vec<_>>()
.join(",");
parts.push(format!("Present×{total}({top})"));
} else if have_present_history {
} else if counts.present_history {
parts.push("Present×0".to_string());
}
if have_queue_history {
if counts.queue_history || adds > 0 || completes > 0 {
parts.push(format!("blt-queue add×{adds} complete×{completes}"));
}
if parts.is_empty() {
let summary = if parts.is_empty() {
"none".to_string()
} else {
parts.join(" ")
}
}
/// The structured discriminator read for `[from, to]` (the stall classifier's evidence):
/// how many swapchain presents (DXGI 42/55, any process) and how many virtual-display
/// queue entries (`BltQueueAddEntry`) landed in the window, plus whether each stream has
/// EVER produced an event (distinguishing a true zero from a witness that is not working —
/// e.g. the DXGI enable was refused, or an OS build renumbered the BltQueue events).
pub(super) fn window_counts(&self, from: Instant, to: Instant) -> EtwWindowCounts {
let (now_i, now_q, freq) = (Instant::now(), qpc_now(), qpc_freq());
let to_q = now_q - duration_qpc(now_i.saturating_duration_since(to), freq);
let from_q = now_q - duration_qpc(now_i.saturating_duration_since(from), freq);
let ring = RING.lock().unwrap();
let mut out = EtwWindowCounts::default();
for &(ts, id, _) in ring.iter() {
match id {
DXGI_PRESENT_ID | DXGI_PRESENT_MPO_ID => {
out.present_history = true;
if ts >= from_q && ts <= to_q {
out.presents += 1;
}
}
BLT_ADD_ID => {
out.queue_history = true;
if ts >= from_q && ts <= to_q {
out.queue_adds += 1;
}
}
_ => {}
}
}
out
};
(summary, counts)
}
}
/// [`EtwWatch::window_counts`]'s read: the compose-silence discriminator's structured evidence.
/// Witness-liveness lookback: the [`EtwWindowCounts`] history flags are true only when the
/// stream produced at least one event inside the `LOOKBACK` window ENDING at the hole's start.
/// "Ever produced an event" would be wrong in both directions: an event that arrived only AFTER
/// the hole (the resume burst, the stall-ending frame) proves nothing about whether the witness
/// was working DURING it, and a provider that died mid-session (or whose events aged out of the
/// ring) would keep flying a stale known-working flag forever. Demonstrated life immediately
/// BEFORE the hole is the claim the classifier actually needs; 5 s is far longer than any
/// pre-stall active-flow gate, so a genuinely working witness cannot blink false across a
/// frame-time lull.
const LOOKBACK: Duration = Duration::from_secs(5);
/// The discriminator's windowing math, factored pure (plain i64 QPC-tick arithmetic, no ETW,
/// no clock reads) so the ring→counts contract is unit-testable without a session: presents
/// (DXGI 42/55, any process) and queue entries (`BltQueueAddEntry`) inside `[from_q, to_q]`,
/// witness liveness from `[from_q - lookback_q, from_q]` (see [`LOOKBACK`]). A
/// `BltQueueCompleteIndirectPresent` proves the queue witness works exactly as an add does —
/// both ride the same provider enable — so either satisfies `queue_history`.
fn count_window(
events: &[(i64, u16, u32)],
from_q: i64,
to_q: i64,
lookback_q: i64,
) -> EtwWindowCounts {
let mut out = EtwWindowCounts::default();
for &(ts, id, _) in events {
let in_window = ts >= from_q && ts <= to_q;
let in_lookback = ts >= from_q.saturating_sub(lookback_q) && ts <= from_q;
match id {
DXGI_PRESENT_ID | DXGI_PRESENT_MPO_ID => {
out.present_history |= in_lookback;
if in_window {
out.presents += 1;
}
}
BLT_ADD_ID => {
out.queue_history |= in_lookback;
if in_window {
out.queue_adds += 1;
}
}
BLT_COMPLETE_ID => out.queue_history |= in_lookback,
_ => {}
}
}
out
}
/// [`EtwWatch::window_report`]'s structured half: the compose-silence discriminator's evidence.
#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
pub(super) struct EtwWindowCounts {
/// Swapchain presents (any process — the game AND dwm both count) inside the window.
pub(super) presents: u32,
/// `BltQueueAddEntry` events (frames entering the virtual display's kernel queue) inside it.
pub(super) queue_adds: u32,
/// The present stream has produced at least one event EVER (witness known-working).
/// The present stream demonstrated liveness inside [`LOOKBACK`] BEFORE the hole opened — a
/// working witness whose in-window zero is a reading, not a dead one whose zero is noise.
pub(super) present_history: bool,
/// The queue stream has produced at least one event EVER (witness known-working).
/// Queue-stream liveness inside [`LOOKBACK`] before the hole (`BltQueueAddEntry` or
/// `BltQueueCompleteIndirectPresent` — either proves the witness works).
pub(super) queue_history: bool,
}
@@ -561,3 +609,72 @@ impl Drop for EtwWatch {
}
}
}
// The module only compiles on Windows (lib.rs gates `mod windows`), so plain `cfg(test)` here
// already means "Windows tests" — and [`count_window`] itself is pure tick math, no session.
#[cfg(test)]
mod tests {
use super::*;
/// [`count_window`]'s contract: counts come from the hole window `[from, to]`; liveness
/// comes ONLY from the lookback window ending at the hole's start. An event after the hole
/// (the resume burst) or older than the lookback (a dead provider's leftovers) must not fly
/// the known-working flag — those are exactly the shapes that used to convict every
/// compose-silence hole as content.
#[test]
fn count_window_liveness_and_windowing() {
// Hole [1000, 2000], lookback 500 → liveness window [500, 1000]. Plain ticks.
let (from, to, lb) = (1_000i64, 2_000i64, 500i64);
let ev = |ts: i64, id: u16| (ts, id, 42u32);
// The healthy shape: liveness demonstrated before the hole, activity inside it.
let events = [
ev(600, DXGI_PRESENT_ID), // lookback → present witness live
ev(700, BLT_COMPLETE_ID), // lookback → queue witness live (completes count)
ev(1_100, DXGI_PRESENT_ID), // in-window present
ev(1_200, DXGI_PRESENT_MPO_ID), // in-window present (MPO path)
ev(1_300, BLT_ADD_ID), // in-window queue add
ev(1_400, 430), // non-witness id: never counted here
];
assert_eq!(
count_window(&events, from, to, lb),
EtwWindowCounts {
presents: 2,
queue_adds: 1,
present_history: true,
queue_history: true,
}
);
// In-window events count but do NOT confer liveness — the witness must have worked
// BEFORE the hole for its zeros elsewhere to mean anything.
let window_only = [ev(1_500, DXGI_PRESENT_ID), ev(1_600, BLT_ADD_ID)];
let c = count_window(&window_only, from, to, lb);
assert_eq!((c.presents, c.queue_adds), (1, 1));
assert!(!c.present_history && !c.queue_history);
// An event only AFTER the hole proves nothing about the witness during it.
let after_only = [ev(2_100, DXGI_PRESENT_ID), ev(2_200, BLT_ADD_ID)];
assert_eq!(
count_window(&after_only, from, to, lb),
EtwWindowCounts::default()
);
// Events that aged past the lookback (a previous session's leftovers) don't either.
let stale = [ev(499, DXGI_PRESENT_ID), ev(1, BLT_ADD_ID)];
assert_eq!(
count_window(&stale, from, to, lb),
EtwWindowCounts::default()
);
// Both lookback edges are inclusive; the hole-start event is both liveness and count.
let edges = [ev(500, DXGI_PRESENT_ID), ev(1_000, BLT_ADD_ID)];
let c = count_window(&edges, from, to, lb);
assert!(c.present_history && c.queue_history);
assert_eq!((c.presents, c.queue_adds), (0, 1));
// A lookback reaching below tick 0 saturates instead of wrapping.
let c = count_window(&[ev(0, DXGI_PRESENT_ID)], 3, to, i64::MAX);
assert!(c.present_history);
}
}
@@ -57,7 +57,7 @@ pub(super) struct StallEvidence {
/// The DxgKrnl DDI activity inside the window (Phase A.3 ETW summary); `None` when the
/// session is unavailable (non-admin dev run).
pub(super) etw: Option<String>,
/// The structured present-vs-queue counts for the window ([`EtwWatch::window_counts`]) —
/// The structured present-vs-queue counts for the window ([`EtwWatch::window_report`]) —
/// the compose-silence discriminator: presents flowing while the queue starves = the OS
/// display path dropped composed frames; both silent = the content stopped presenting.
/// `None` when the ETW session is unavailable.
+112 -22
View File
@@ -14,9 +14,12 @@ use std::sync::mpsc::{Receiver, SyncSender, TrySendError};
use std::sync::Arc;
const SAMPLE_RATE: u32 = 48_000;
const CHANNELS: usize = 2;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
/// Mic capture is MONO: voice is mono at the source, the host accepts any Opus channel
/// layout (its stereo decoder upmixes), and half the samples halve the encode + wire cost.
const MIC_CHANNELS: usize = 1;
/// Mic frames are 10 ms (480 mono samples) — any size ≤ 120 ms is fine host-side; 10 ms
/// halves the frame-fill share of mouth-to-ear latency vs the old 20 ms.
const MIC_FRAME: usize = 480;
struct Terminate;
@@ -327,20 +330,31 @@ fn pw_thread(
Ok(())
}
/// The microphone uplink: capture the default input device, Opus-encode 20 ms chunks,
/// ship them as 0xCB datagrams into the host's virtual PipeWire source.
/// The microphone uplink: capture the default input device (or the picked / echo-cancelled
/// source), Opus-encode 10 ms mono chunks, ship them as 0xCB datagrams into the host's
/// virtual PipeWire source.
pub struct MicStreamer {
quit_tx: pipewire::channel::Sender<Terminate>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl MicStreamer {
pub fn spawn(connector: Arc<NativeClient>) -> Result<MicStreamer> {
/// `muted` is the in-stream mute (B4), shared live with the capture callback: set, the
/// callback keeps pulling and discarding whole frames but sends nothing. Muting by
/// STOPPING the stream was rejected — it re-primes the device buffers and re-runs the
/// source selection below on every unmute, so the first second back is glitchy.
///
/// `echo_cancel` is the Settings toggle; `PUNKTFUNK_NO_AEC=1` overrides it off.
pub fn spawn(
connector: Arc<NativeClient>,
muted: Arc<std::sync::atomic::AtomicBool>,
echo_cancel: bool,
) -> Result<MicStreamer> {
let (quit_tx, quit_rx) = pipewire::channel::channel::<Terminate>();
let thread = std::thread::Builder::new()
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, quit_rx) {
if let Err(e) = mic_thread(&connector, quit_rx, muted, echo_cancel) {
tracing::warn!(error = %e, "mic uplink thread ended");
}
})
@@ -361,19 +375,76 @@ impl Drop for MicStreamer {
}
}
/// Capture-side state: accumulated PCM and the Opus encoder (encoding a 20 ms frame is
/// ~100 µs — fine inside the process callback).
/// Capture-side state: accumulated PCM and the Opus encoder (encoding a 10 ms frame is
/// well under 100 µs — fine inside the process callback).
struct MicData {
connector: Arc<NativeClient>,
ring: VecDeque<f32>,
encoder: opus::Encoder,
seq: u32,
out: Vec<u8>,
/// The in-stream mute (B4), flipped by the session's chord. Read per callback.
muted: Arc<std::sync::atomic::AtomicBool>,
}
/// Whether the mic echo-cancellation hooks run this session: the `echo_cancel` setting, with
/// `PUNKTFUNK_NO_AEC=1` as a one-way override OFF. The env var wins — it is the escape hatch
/// for a box whose canceller misbehaves, and it predates the setting; nothing turns AEC back
/// on once it is set. Here the hook is the echo-cancelled-source preference below; the WASAPI
/// twin gates its Communications stream category the same way.
fn aec_enabled(echo_cancel: bool) -> bool {
echo_cancel && !std::env::var("PUNKTFUNK_NO_AEC").is_ok_and(|v| !v.is_empty() && v != "0")
}
/// The capture stream's `target.object`, in preference order: the Settings microphone pick
/// (`Settings::mic_device` via session main's `PUNKTFUNK_AUDIO_SOURCE`) verbatim, else — so a
/// desktop that already runs `module-echo-cancel` stops feeding its own downlink audio back
/// into the host's virtual mic — the first echo-cancelled source in the graph. `None` = the
/// user picked nothing and no such source exists: PipeWire's default routing, as before.
///
/// Preference-only by design: loading `libpipewire-module-echo-cancel` ourselves needs
/// `pw_context_load_module`, which the pipewire crate (0.9) doesn't expose safely — until it
/// does, we only ever target processing the user (or their session) already set up.
fn mic_capture_target(echo_cancel: bool) -> Option<String> {
if let Ok(target) = std::env::var("PUNKTFUNK_AUDIO_SOURCE") {
if !target.is_empty() {
return Some(target);
}
}
if !aec_enabled(echo_cancel) {
return None;
}
let name = echo_cancel_source()?;
tracing::info!(
source = %name,
"mic capture targets the echo-cancelled source (Echo cancellation off, or \
PUNKTFUNK_NO_AEC=1, disables this)"
);
Some(name)
}
/// Find an existing echo-cancelled capture node: the first `Audio/Source` whose `node.name`
/// or description says echo-cancel (`module-echo-cancel`'s convention — `echo-cancel-*`
/// nodes, "Echo-Cancel …" descriptions; PulseAudio-compat setups match too). One registry
/// roundtrip via [`devices`]; any failure reads as "none".
fn echo_cancel_source() -> Option<String> {
let (_, sources) = devices().ok()?;
sources.into_iter().find_map(|d| {
let name = d.name.to_ascii_lowercase();
let desc = d.description.to_ascii_lowercase();
(name.contains("echo-cancel")
|| name.contains("echo_cancel")
|| desc.contains("echo-cancel")
|| desc.contains("echo cancel"))
.then_some(d.name)
})
}
fn mic_thread(
connector: &Arc<NativeClient>,
quit_rx: pipewire::channel::Receiver<Terminate>,
muted: Arc<std::sync::atomic::AtomicBool>,
echo_cancel: bool,
) -> Result<()> {
use pipewire as pw;
use pw::{properties::properties, spa};
@@ -384,9 +455,14 @@ fn mic_thread(
PW_INIT.call_once(pw::init);
let mut encoder =
opus::Encoder::new(SAMPLE_RATE, opus::Channels::Stereo, opus::Application::Voip)
opus::Encoder::new(SAMPLE_RATE, opus::Channels::Mono, opus::Application::Voip)
.map_err(|e| anyhow::anyhow!("opus encoder: {e}"))?;
let _ = encoder.set_bitrate(opus::Bitrate::Bits(64_000));
// Voice tuning: 48 kbps mono is transparent for speech; in-band FEC + an assumed 10 %
// loss let the host's decoder rebuild a lost 0xCB datagram from its successor instead
// of concealing (datagrams are fire-and-forget — this FEC is the only redundancy).
let _ = encoder.set_bitrate(opus::Bitrate::Bits(48_000));
let _ = encoder.set_inband_fec(true);
let _ = encoder.set_packet_loss_perc(10);
let mainloop = pw::main_loop::MainLoopRc::new(None).context("pw mic MainLoop")?;
let context = pw::context::ContextRc::new(&mainloop, None).context("pw mic Context")?;
@@ -405,14 +481,15 @@ fn mic_thread(
*pw::keys::MEDIA_ROLE => "Communication",
*pw::keys::NODE_NAME => "punktfunk-mic-capture",
*pw::keys::NODE_DESCRIPTION => "Punktfunk Microphone",
// ~10 ms quantum (one mic frame). Without it the capture stream inherits the graph
// quantum — commonly 10242048 samples, so the mic arrived in 2143 ms bursts that
// sat ahead of the encoder as latency (the playback stream always asked for 5 ms).
*pw::keys::NODE_LATENCY => "480/48000",
};
// The Settings microphone pick (`Settings::mic_device` via session main).
if let Ok(target) = std::env::var("PUNKTFUNK_AUDIO_SOURCE") {
if !target.is_empty() {
// Raw key: the `keys::TARGET_OBJECT` constant is feature-gated on a newer
// libpipewire than we require; the wire name is stable.
props.insert("target.object", target);
}
if let Some(target) = mic_capture_target(echo_cancel) {
// Raw key: the `keys::TARGET_OBJECT` constant is feature-gated on a newer
// libpipewire than we require; the wire name is stable.
props.insert("target.object", target);
}
let stream = pw::stream::StreamBox::new(&core, "punktfunk-mic-capture", props)
.context("pw mic Stream")?;
@@ -423,6 +500,7 @@ fn mic_thread(
encoder,
seq: 0,
out: vec![0u8; 4000],
muted,
};
let _listener = stream
@@ -447,9 +525,20 @@ fn mic_thread(
.push_back(f32::from_le_bytes([s[0], s[1], s[2], s[3]]));
}
}
// Ship every complete 20 ms stereo frame.
while ud.ring.len() >= MIC_FRAME * CHANNELS {
let pcm: Vec<f32> = ud.ring.drain(..MIC_FRAME * CHANNELS).collect();
// Muted (B4): the stream stays open and the device keeps its primed buffers —
// only the sending stops. Whole frames are discarded so the ring can't grow,
// and `seq` deliberately does NOT advance: the host sees one continuous
// sequence with a silent pause in the middle rather than a gap the size of the
// mute, which its de-jitter would try to conceal frame by frame.
if ud.muted.load(std::sync::atomic::Ordering::Relaxed) {
let whole =
(ud.ring.len() / (MIC_FRAME * MIC_CHANNELS)) * (MIC_FRAME * MIC_CHANNELS);
ud.ring.drain(..whole);
return;
}
// Ship every complete 10 ms mono frame.
while ud.ring.len() >= MIC_FRAME * MIC_CHANNELS {
let pcm: Vec<f32> = ud.ring.drain(..MIC_FRAME * MIC_CHANNELS).collect();
match ud.encoder.encode_float(&pcm, &mut ud.out) {
Ok(len) => {
let pts = std::time::SystemTime::now()
@@ -473,7 +562,8 @@ fn mic_thread(
let mut info = AudioInfoRaw::new();
info.set_format(AudioFormat::F32LE);
info.set_rate(SAMPLE_RATE);
info.set_channels(CHANNELS as u32);
// Mono: the stream's adapter downmixes whatever layout the source really has.
info.set_channels(MIC_CHANNELS as u32);
let obj = pw::spa::pod::Object {
type_: pw::spa::utils::SpaTypes::ObjectParamFormat.as_raw(),
id: pw::spa::param::ParamType::EnumFormat.as_raw(),
+92 -20
View File
@@ -23,14 +23,22 @@ use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{Receiver, SyncSender, TrySendError};
use std::sync::Arc;
use std::time::Duration;
use wasapi::{DeviceEnumerator, Direction, SampleType, StreamMode, WaveFormat};
use wasapi::{
AudioClientProperties, DeviceEnumerator, Direction, SampleType, StreamCategory, StreamMode,
WaveFormat,
};
const SAMPLE_RATE: usize = 48_000;
/// The microphone uplink stays stereo (the host's virtual mic is stereo). The render path is
/// multichannel — its channel count + block align are runtime, driven by the host-resolved layout.
const CHANNELS: usize = 2;
/// Mic frames are 20 ms (960 samples/channel) — any size ≤ 120 ms is fine host-side.
const MIC_FRAME: usize = 960;
/// Mic capture requests STEREO from WASAPI (autoconvert matrixes any endpoint layout down to
/// it — the proven path; `read_from_device_to_deque` then delivers our requested format) and
/// downmixes to MONO in code before the encoder: voice is mono at the source, the host accepts
/// any Opus channel layout (its stereo decoder upmixes), and half the samples halve the
/// encode + wire cost. The render path is multichannel — its channel count + block align are
/// runtime, driven by the host-resolved layout.
const CAPT_CHANNELS: usize = 2;
/// Mic frames are 10 ms (480 mono samples) — any size ≤ 120 ms is fine host-side; 10 ms
/// halves the frame-fill share of mouth-to-ear latency vs the old 20 ms.
const MIC_FRAME: usize = 480;
/// A selectable WASAPI endpoint for the settings pickers.
#[derive(Clone, Debug)]
@@ -302,21 +310,31 @@ fn render_thread(
res
}
/// The microphone uplink: capture the default input device, Opus-encode 20 ms chunks, ship
/// them as 0xCB datagrams into the host's virtual mic source.
/// The microphone uplink: capture the default input device, Opus-encode 10 ms mono chunks,
/// ship them as 0xCB datagrams into the host's virtual mic source.
pub struct MicStreamer {
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl MicStreamer {
pub fn spawn(connector: Arc<NativeClient>) -> Result<MicStreamer> {
/// `muted` is the in-stream mute (B4), shared live with the capture loop: set, the loop
/// keeps reading the endpoint and discarding whole frames but sends nothing. Muting by
/// STOPPING the client was rejected — an `IAudioClient` stop/start re-primes the endpoint
/// buffers and re-runs the category negotiation below on every unmute.
///
/// `echo_cancel` is the Settings toggle; `PUNKTFUNK_NO_AEC=1` overrides it off.
pub fn spawn(
connector: Arc<NativeClient>,
muted: Arc<AtomicBool>,
echo_cancel: bool,
) -> Result<MicStreamer> {
let stop = Arc::new(AtomicBool::new(false));
let stop_t = stop.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-mic".into())
.spawn(move || {
if let Err(e) = mic_thread(&connector, stop_t) {
if let Err(e) = mic_thread(&connector, stop_t, muted, echo_cancel) {
tracing::warn!(error = %format!("{e:#}"), "mic uplink thread ended");
}
})
@@ -337,24 +355,58 @@ impl Drop for MicStreamer {
}
}
fn mic_thread(connector: &Arc<NativeClient>, stop: Arc<AtomicBool>) -> Result<()> {
/// Whether the mic echo-cancellation hooks run this session: the `echo_cancel` setting, with
/// `PUNKTFUNK_NO_AEC=1` as a one-way override OFF. The env var wins — it is the escape hatch
/// for a box whose canceller misbehaves, and it predates the setting; nothing turns AEC back
/// on once it is set. Here the hook is the Communications stream category below; the PipeWire
/// twin gates its echo-cancelled-source preference the same way.
fn aec_enabled(echo_cancel: bool) -> bool {
echo_cancel && !std::env::var("PUNKTFUNK_NO_AEC").is_ok_and(|v| !v.is_empty() && v != "0")
}
fn mic_thread(
connector: &Arc<NativeClient>,
stop: Arc<AtomicBool>,
muted: Arc<AtomicBool>,
echo_cancel: bool,
) -> Result<()> {
wasapi::initialize_mta()
.ok()
.context("CoInitializeEx (MTA)")?;
let mut encoder = opus::Encoder::new(
SAMPLE_RATE as u32,
opus::Channels::Stereo,
opus::Channels::Mono,
opus::Application::Voip,
)
.map_err(|e| anyhow!("opus encoder: {e}"))?;
let _ = encoder.set_bitrate(opus::Bitrate::Bits(64_000));
// Voice tuning: 48 kbps mono is transparent for speech; in-band FEC + an assumed 10 %
// loss let the host's decoder rebuild a lost 0xCB datagram from its successor instead
// of concealing (datagrams are fire-and-forget — this FEC is the only redundancy).
let _ = encoder.set_bitrate(opus::Bitrate::Bits(48_000));
let _ = encoder.set_inband_fec(true);
let _ = encoder.set_packet_loss_perc(10);
let enumerator = DeviceEnumerator::new().context("DeviceEnumerator")?;
let device = pick_device(&enumerator, &Direction::Capture, "PUNKTFUNK_AUDIO_SOURCE")
.context("capture endpoint (no microphone?)")?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CHANNELS, None);
// Communications category → the endpoint's communications signal-processing chain. A
// driver/APO stack with an echo canceller only engages it for communications-category
// streams; the default (Other) category never did, so the downlink audio playing on
// this box fed straight back into the host's virtual mic. Must precede Initialize
// (SetClientProperties is a pre-init call; the wasapi crate QIs IAudioClient2 inside).
// Best-effort: an endpoint without IAudioClient2 just keeps the default category.
// The "Echo cancellation" setting opts out, and PUNKTFUNK_NO_AEC=1 overrides that off
// (same lever as the Linux echo-cancel-source preference) — see `aec_enabled`.
if aec_enabled(echo_cancel) {
if let Err(e) = audio_client.set_properties(
AudioClientProperties::new().set_category(StreamCategory::Communications),
) {
tracing::debug!(error = %e, "mic capture: Communications category not set");
}
}
let desired = WaveFormat::new(32, 32, &SampleType::Float, SAMPLE_RATE, CAPT_CHANNELS, None);
let (default_period, _min_period) =
audio_client.get_device_period().context("device period")?;
let mode = StreamMode::EventsShared {
@@ -392,13 +444,33 @@ fn mic_thread(connector: &Arc<NativeClient>, stop: Arc<AtomicBool>) -> Result<()
Err(e) => return Err(anyhow!("get_next_packet_size: {e}")),
}
}
let whole = (bytes.len() / 4) * 4;
for c in bytes.drain(..whole).collect::<Vec<u8>>().chunks_exact(4) {
ring.push_back(f32::from_le_bytes([c[0], c[1], c[2], c[3]]));
// One stereo capture frame (8 bytes) → one mono sample: average L/R. Autoconvert
// already matrixed the endpoint's real layout (mono/stereo/array mic) into the
// stereo stream we initialized, so this is the only downmix left to do.
let stereo_frame = 4 * CAPT_CHANNELS;
let whole = (bytes.len() / stereo_frame) * stereo_frame;
for c in bytes
.drain(..whole)
.collect::<Vec<u8>>()
.chunks_exact(stereo_frame)
{
let l = f32::from_le_bytes([c[0], c[1], c[2], c[3]]);
let r = f32::from_le_bytes([c[4], c[5], c[6], c[7]]);
ring.push_back((l + r) * 0.5);
}
// Ship every complete 20 ms stereo frame.
while ring.len() >= MIC_FRAME * CHANNELS {
let pcm: Vec<f32> = ring.drain(..MIC_FRAME * CHANNELS).collect();
// Muted (B4): the capture client stays started and keeps its primed buffers — only
// the sending stops. Whole frames are discarded so the ring can't grow, and `seq`
// deliberately does NOT advance: the host sees one continuous sequence with a silent
// pause in the middle rather than a gap the size of the mute, which its de-jitter
// would try to conceal frame by frame.
if muted.load(Ordering::Relaxed) {
let drop_n = (ring.len() / MIC_FRAME) * MIC_FRAME;
ring.drain(..drop_n);
continue;
}
// Ship every complete 10 ms mono frame.
while ring.len() >= MIC_FRAME {
let pcm: Vec<f32> = ring.drain(..MIC_FRAME).collect();
match encoder.encode_float(&pcm, &mut out) {
Ok(len) => {
let pts = std::time::SystemTime::now()
+44
View File
@@ -62,6 +62,8 @@ pub struct SettingsOverlay {
#[serde(skip_serializing_if = "Option::is_none")]
pub mic_enabled: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub echo_cancel: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub touch_mode: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub mouse_mode: Option<String>,
@@ -122,6 +124,9 @@ impl SettingsOverlay {
if let Some(v) = self.mic_enabled {
s.mic_enabled = v;
}
if let Some(v) = self.echo_cancel {
s.echo_cancel = v;
}
if let Some(v) = &self.touch_mode {
s.touch_mode = v.clone();
}
@@ -197,6 +202,9 @@ impl SettingsOverlay {
if after.mic_enabled != before.mic_enabled {
self.mic_enabled = Some(after.mic_enabled);
}
if after.echo_cancel != before.echo_cancel {
self.echo_cancel = Some(after.echo_cancel);
}
if after.touch_mode != before.touch_mode {
self.touch_mode = Some(after.touch_mode.clone());
}
@@ -243,6 +251,7 @@ impl SettingsOverlay {
"compositor" => self.compositor = None,
"audio_channels" => self.audio_channels = None,
"mic_enabled" => self.mic_enabled = None,
"echo_cancel" => self.echo_cancel = None,
"touch_mode" => self.touch_mode = None,
"mouse_mode" => self.mouse_mode = None,
"invert_scroll" => self.invert_scroll = None,
@@ -437,6 +446,7 @@ mod tests {
compositor: Some("gamescope".into()),
audio_channels: Some(6),
mic_enabled: Some(true),
echo_cancel: Some(false),
touch_mode: Some("pointer".into()),
mouse_mode: Some("desktop".into()),
invert_scroll: Some(true),
@@ -457,6 +467,7 @@ mod tests {
assert_eq!(out.compositor, "gamescope");
assert_eq!(out.audio_channels, 6);
assert!(out.mic_enabled);
assert!(!out.echo_cancel);
assert_eq!(out.touch_mode, "pointer");
assert_eq!(out.mouse_mode, "desktop");
assert!(out.invert_scroll);
@@ -529,6 +540,39 @@ mod tests {
assert_eq!(o2, o);
}
/// `echo_cancel` is a first-class overlay field, not an `extra` passenger: it applies,
/// absorbs, clears, and serialises under the `echo_cancel` key the Apple and Android
/// clients write — one catalog has to round-trip through all three.
#[test]
fn echo_cancel_is_a_first_class_override() {
let base = Settings::default();
assert!(base.echo_cancel, "the setting ships on");
let mut o = SettingsOverlay::default();
let before = o.apply(&base);
let mut after = before.clone();
after.echo_cancel = false;
o.absorb(&before, &after);
assert_eq!(o.echo_cancel, Some(false));
assert!(!o.apply(&base).echo_cancel);
assert!(
o.extra.is_empty(),
"modelled fields must never land in the passthrough"
);
// Serialised under the shared key, and read back from a foreign client's file.
let text = serde_json::to_string(&o).unwrap();
assert!(text.contains("\"echo_cancel\":false"), "{text}");
let from_apple: SettingsOverlay =
serde_json::from_str(r#"{"mic_enabled":true,"echo_cancel":false}"#).unwrap();
assert_eq!(from_apple.echo_cancel, Some(false));
assert!(from_apple.extra.is_empty());
assert!(o.clear("echo_cancel"));
assert_eq!(o.echo_cancel, None);
assert!(o.is_empty());
}
/// `clear` is the explicit way back to inheriting, including the resolution tri-state.
#[test]
fn clear_drops_one_override() {
+137 -7
View File
@@ -41,6 +41,9 @@ pub struct SessionParams {
pub display_hdr: Option<punktfunk_core::quic::HdrMeta>,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Run the uplink through the platform's echo cancellation ([`Settings::echo_cancel`]).
/// Ignored when `mic_enabled` is false; `PUNKTFUNK_NO_AEC=1` overrides it off.
pub echo_cancel: bool,
/// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The
/// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`.
pub clipboard: bool,
@@ -145,6 +148,13 @@ pub struct Stats {
/// received+lost (%). The OSD renders the counter line only when nonzero.
pub lost: u32,
pub lost_pct: f32,
/// Mic uplink frames this window: handed to the QUIC datagram send, and shed anywhere
/// client-side (queue-full at the producer + the pump's stale-oldest backlog governor —
/// see [`NativeClient::mic_stats`]). Both stay 0 while the mic is off OR muted (a mute
/// stops the sending, not the capture), so the OSD renders the mic line only while voice
/// is actually going out — the muted case has its own badge, which does not need stats on.
pub mic_sent: u32,
pub mic_dropped: u32,
/// The decode path frames actually took this window (`"vaapi"`/`"software"`, empty
/// until the first frame) — the OSD's trailing tag; tracks a mid-session fallback.
pub decoder: &'static str,
@@ -199,10 +209,61 @@ pub enum SessionEvent {
Stats(Stats),
}
/// The in-stream microphone mute (B4), shared between the embedder's toggle (a keyboard chord
/// in the presenter) and the capture callback that reads it every quantum.
///
/// Two flags, not one, so the indicator can never lie: `live` is raised by the pump only once
/// the uplink is actually running, so a session whose mic is off in Settings — or whose capture
/// device failed to open — reports "no mic here" and the chord is a documented no-op instead of
/// silently latching a mute nothing implements. Per session by design: the mute is a moment
/// ("don't send the doorbell"), not a preference, so it is never persisted and every new
/// session starts unmuted.
#[derive(Clone, Default)]
pub struct MicControl {
muted: Arc<AtomicBool>,
live: Arc<AtomicBool>,
}
impl MicControl {
/// True when this session has a running uplink to mute at all.
pub fn live(&self) -> bool {
self.live.load(Ordering::Relaxed)
}
/// True when the user has muted a uplink that exists — what the OSD indicator draws.
pub fn muted(&self) -> bool {
self.live() && self.muted.load(Ordering::Relaxed)
}
/// Flip the mute. `Some(now_muted)` when it applied, `None` when this session has no
/// uplink (the caller says so rather than pretending something happened).
pub fn toggle(&self) -> Option<bool> {
if !self.live() {
return None;
}
let next = !self.muted.load(Ordering::Relaxed);
self.muted.store(next, Ordering::Relaxed);
Some(next)
}
/// The capture side's handle on the flag (the streamer reads it per quantum).
fn flag(&self) -> Arc<AtomicBool> {
self.muted.clone()
}
/// The pump's report that the uplink came up (or went away).
fn set_live(&self, live: bool) {
self.live.store(live, Ordering::Relaxed);
}
}
pub struct SessionHandle {
pub events: async_channel::Receiver<SessionEvent>,
pub frames: async_channel::Receiver<DecodedFrame>,
pub stop: Arc<AtomicBool>,
/// The in-stream mic mute. Inert (`live()` false) until the pump has the uplink running,
/// and for the whole session when the mic is off in Settings.
pub mic: MicControl,
/// The pump thread. A Vulkan-Video pump SUBMITS to the shared device's decode
/// queue — the presenter must join this before any `vkDeviceWaitIdle`/teardown
/// (external-sync rule over every device queue).
@@ -215,14 +276,17 @@ pub fn start(params: SessionParams) -> SessionHandle {
let (frame_tx, frame_rx) = async_channel::bounded(2);
let stop = Arc::new(AtomicBool::new(false));
let stop_w = stop.clone();
let mic = MicControl::default();
let mic_w = mic.clone();
let thread = std::thread::Builder::new()
.name("punktfunk-session".into())
.spawn(move || pump(params, ev_tx, frame_tx, stop_w))
.spawn(move || pump(params, ev_tx, frame_tx, stop_w, mic_w))
.expect("spawn session thread");
SessionHandle {
events: ev_rx,
frames: frame_rx,
stop,
mic,
thread: Some(thread),
}
}
@@ -275,6 +339,7 @@ fn pump(
ev_tx: async_channel::Sender<SessionEvent>,
frame_tx: async_channel::Sender<DecodedFrame>,
stop: Arc<AtomicBool>,
mic: MicControl,
) {
// PUNKTFUNK_PREFER_PYROWAVE=1 — the Phase-2 lab opt-in for the wired-LAN wavelet codec
// (a Settings toggle is the Phase-3 productization). Riding `preferred_codec` is exactly
@@ -359,11 +424,31 @@ fn pump(
// Build the decoder for the codec the host resolved (never assume HEVC), honoring the
// Settings backend preference (auto/vaapi/software).
let codec_id = crate::video::ffmpeg_codec_id(connector.codec);
tracing::info!(
?codec_id,
welcome_codec = connector.codec,
"negotiated video codec"
);
// The WIRE codec is the negotiated truth; the FFmpeg id is meaningful only where
// FFmpeg decodes it. `ffmpeg_codec_id`'s fallthrough maps every unknown wire bit —
// PyroWave included — to HEVC, so logging it unconditionally claimed
// `codec_id=HEVC` for wavelet sessions that never touch FFmpeg at all.
let codec = match connector.codec {
punktfunk_core::quic::CODEC_H264 => "H264",
punktfunk_core::quic::CODEC_HEVC => "HEVC",
punktfunk_core::quic::CODEC_AV1 => "AV1",
punktfunk_core::quic::CODEC_PYROWAVE => "PyroWave",
_ => "unknown",
};
if connector.codec == punktfunk_core::quic::CODEC_PYROWAVE {
tracing::info!(
codec,
welcome_codec = connector.codec,
"negotiated video codec"
);
} else {
tracing::info!(
codec,
?codec_id,
welcome_codec = connector.codec,
"negotiated video codec"
);
}
// A negotiated PyroWave session decodes on the presenter's device, no FFmpeg —
// reachable only through the explicit preference above (resolve_codec never
// auto-picks the bit), so failing loudly here is failing an opted-in experiment.
@@ -430,14 +515,18 @@ fn pump(
.ok()
})
.flatten();
// The uplink, and with it the mute the embedder's chord drives. `set_live` is what makes
// the chord (and its indicator) real: a mic turned off in Settings, or a capture device
// that wouldn't open, leaves it false and the chord stays an honest no-op.
let _mic = params
.mic_enabled
.then(|| {
audio::MicStreamer::spawn(connector.clone())
audio::MicStreamer::spawn(connector.clone(), mic.flag(), params.echo_cancel)
.map_err(|e| tracing::warn!(error = %e, "mic uplink disabled"))
.ok()
})
.flatten();
mic.set_live(_mic.is_some());
// Live host↔client clock offset: loaded per frame (Relaxed) so mid-stream re-syncs (an NTP
// step, drift) keep the capture-clock latency stats honest — never cached at session start.
@@ -492,6 +581,9 @@ fn pump(
let mut dec_path: &'static str = "";
// The stats window keeps its own drop cursor — the OSD shows the per-window delta.
let mut window_dropped = connector.frames_dropped();
// Mic uplink cursor (same per-window diffing): a healthy 10 ms-frame mic reads ~100
// sent/s; a nonzero drop delta is the queue shedding backlog (see NativeClient::mic_stats).
let mut window_mic = connector.mic_stats();
let mut last_kf_req: Option<Instant> = None;
// Freeze-until-reanchor: the shared post-loss gate ([`punktfunk_core::reanchor::ReanchorGate`]).
// Armed on any loss signal (frame-index gap, dropped-count climb, decoder wedge/demotion), it
@@ -898,6 +990,12 @@ fn pump(
let (pace_p50, _) = window_percentiles(&mut pace_us_win);
let lost = dropped.saturating_sub(window_dropped) as u32;
window_dropped = dropped;
let mic_now = connector.mic_stats();
let mic_sent = mic_now.sent.saturating_sub(window_mic.sent) as u32;
let mic_dropped = (mic_now.dropped_full + mic_now.dropped_stale)
.saturating_sub(window_mic.dropped_full + window_mic.dropped_stale)
as u32;
window_mic = mic_now;
tracing::debug!(
fps = frames_n,
hostnet_p50_us = hn_p50,
@@ -909,6 +1007,8 @@ fn pump(
pace_p50_us = pace_p50,
decode_p50_us = dec_p50,
lost,
mic_sent,
mic_dropped,
total_frames,
"stream window"
);
@@ -931,6 +1031,8 @@ fn pump(
} else {
0.0
},
mic_sent,
mic_dropped,
decoder: dec_path,
target_kbps: connector.current_bitrate_kbps(),
auto_rate,
@@ -957,6 +1059,10 @@ fn pump(
"session ended"
);
stop.store(true, Ordering::SeqCst);
// The uplink is about to be dropped with the rest of this frame — stop claiming a mute
// surface, so an embedder still holding the handle through its end path (browse mode
// returns to the console with it) can't draw a muted mic that no longer exists.
mic.set_live(false);
if let Some(t) = audio_thread {
let _ = t.join(); // exits within its 100 ms pull timeout once `stop` is set
}
@@ -1067,4 +1173,28 @@ mod tests {
assert!(parse_debug_reconfigure(bad).is_none(), "{bad:?} parsed");
}
}
/// The mute is inert until the pump reports a live uplink — a session without a mic must
/// answer "nothing to mute" rather than latching a mute and drawing the indicator.
#[test]
fn mic_mute_is_a_no_op_without_an_uplink() {
let mic = MicControl::default();
assert!(!mic.live());
assert_eq!(mic.toggle(), None, "no uplink, nothing to toggle");
assert!(!mic.muted(), "and nothing to show");
mic.set_live(true);
assert_eq!(mic.toggle(), Some(true));
assert!(mic.muted());
// The capture side reads the same flag the toggle writes.
assert!(mic.flag().load(Ordering::Relaxed));
assert_eq!(mic.toggle(), Some(false));
assert!(!mic.muted());
// A mute that outlives its uplink stops being shown (session end clears `live`).
assert_eq!(mic.toggle(), Some(true));
mic.set_live(false);
assert!(!mic.muted());
assert_eq!(mic.toggle(), None);
}
}
+13
View File
@@ -835,6 +835,15 @@ pub struct Settings {
pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source.
pub mic_enabled: bool,
/// Run the mic uplink through the platform's echo cancellation (the Apple/Android clients'
/// "Echo cancellation" toggle, same `echo_cancel` key). On Linux that means preferring an
/// echo-cancelled PipeWire source; on Windows, asking WASAPI for the Communications stream
/// category so the endpoint's own canceller engages. Default ON — without it, a laptop
/// speaker playing the host's audio is heard by this device's mic and sent straight back.
/// Only meaningful while `mic_enabled`. `PUNKTFUNK_NO_AEC=1` overrides it off (see
/// `audio::aec_enabled`). `default` so pre-existing stores load with it on.
#[serde(default = "default_true")]
pub echo_cancel: bool,
/// Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
/// can capture; the resolved count drives the decoder + playback layout.
pub audio_channels: u8,
@@ -991,6 +1000,7 @@ impl Default for Settings {
mouse_mode: "capture".into(),
inhibit_shortcuts: true,
mic_enabled: false,
echo_cancel: true,
audio_channels: 2,
codec: "auto".into(),
decoder: "auto".into(),
@@ -1177,6 +1187,9 @@ mod tests {
assert_eq!(s.forward_pad, "");
assert!(s.fullscreen_on_stream);
assert!(!s.library_enabled);
// Echo cancellation post-dates every stored file: it must load ON, or an upgrade
// would silently turn a user's echo protection off.
assert!(s.echo_cancel);
}
/// Stats-tier resolution: a pre-tier store falls back to `show_stats` (off → Off,
+15 -1
View File
@@ -99,6 +99,18 @@ pub struct Status {
/// Why the check couldn't complete. `update_available` is always false when set.
#[serde(skip_serializing_if = "Option::is_none")]
pub error: Option<String>,
/// The feed answered, but this channel has **no release published yet** — an expected
/// state rather than a malfunction, so a caller can say so plainly instead of showing a
/// raw "HTTP 404".
///
/// Deliberately NOT symmetric with the host's `UpdateStatus`, which clears `last_error`
/// for this case: there the consumer is a human reading a console, and a red "last check
/// failed" on an empty feed is the bug being fixed. Here the consumer is a shell script
/// reading an exit code, so `error` stays set and `--check-update` keeps returning 1.
/// An empty channel is not evidence that this build is current, and a mistyped
/// `PUNKTFUNK_UPDATE_FEED` is indistinguishable from one out here.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub not_published: bool,
}
/// The Ed25519 keys trusted for update manifests — pinned once in [`pf_update_check`] so the
@@ -302,6 +314,7 @@ pub fn check(current: &str) -> Status {
opt_in_hint: opt_in_would_help(kind, caps).then(opt_in_hint),
notes_url: String::new(),
error: None,
not_published: false,
};
let (apply, applier) = apply_route(kind, caps);
status.apply = apply;
@@ -320,7 +333,8 @@ pub fn check(current: &str) -> Status {
) {
Ok(m) => m,
Err(e) => {
status.error = Some(e);
status.not_published = e.is_not_published();
status.error = Some(e.to_string());
return status;
}
};
+97 -2
View File
@@ -321,6 +321,88 @@ pub fn ffmpeg_codec_id(wire: u8) -> ffmpeg::codec::Id {
}
}
/// Select a decoder for `codec_id` that can actually drive `hw_pix_fmt` through
/// `hw_device_ctx` — the open-time capability check every hardware backend needs.
///
/// `avcodec_find_decoder(id)` is NOT that: it returns the registry's FIRST decoder for
/// the id, and upstream orders the native `av1` decoder LAST on purpose ("hwaccel hooks
/// only, so prefer external decoders" — allcodecs.c), behind libdav1d/libaom. The ID
/// lookup therefore hands every AV1 session a pure software decoder that silently
/// ignores `hw_device_ctx` and never calls `get_format`; each frame then fails the
/// backend's hw-format guard and the session burns the demotion ladder MID-STREAM
/// (~1 s per rung — field-logged as 68 Vulkan fails → D3D11VA → 102 fails → software,
/// ~3 s of black) instead of failing here at open in milliseconds. H.264/HEVC never hit
/// this only because their native decoders happen to be registered first.
///
/// The walk mirrors what `avcodec_find_decoder` would do, restricted to decoders whose
/// `avcodec_get_hw_config` advertises the wanted surface via
/// `AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX` — registry order still wins among those,
/// so H.264/HEVC keep selecting exactly the decoder they always did. The error names
/// the decoders that WERE found, so a log reader can tell "this build has no AV1
/// hwaccel at all" from "no AV1 decoder exists, period".
pub(crate) fn find_hw_decoder(
codec_id: ffmpeg::codec::Id,
hw_pix_fmt: ffmpeg::ffi::AVPixelFormat,
) -> Result<*const ffmpeg::ffi::AVCodec> {
use ffmpeg::ffi;
let want: ffi::AVCodecID = codec_id.into();
let mut found: Vec<String> = Vec::new();
// SAFETY: `av_codec_iterate` walks libav's static codec registry (`opaque` is its
// cursor) and returns static `AVCodec`s; `avcodec_get_hw_config` only reads the
// codec's own static hw-config table, NULL-terminated by returning null past the end.
unsafe {
let mut opaque = std::ptr::null_mut();
loop {
let codec = ffi::av_codec_iterate(&mut opaque);
if codec.is_null() {
break;
}
if (*codec).id != want || ffi::av_codec_is_decoder(codec) == 0 {
continue;
}
for i in 0.. {
let cfg = ffi::avcodec_get_hw_config(codec, i);
if cfg.is_null() {
break;
}
if (*cfg).methods & ffi::AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX as i32 != 0
&& (*cfg).pix_fmt == hw_pix_fmt
{
return Ok(codec);
}
}
found.push(
std::ffi::CStr::from_ptr((*codec).name)
.to_string_lossy()
.into_owned(),
);
}
}
if found.is_empty() {
bail!("no {codec_id:?} decoder in this FFmpeg build");
}
bail!(
"no {codec_id:?} decoder in this FFmpeg build can drive {hw_pix_fmt:?} via \
hw_device_ctx (found: {})",
found.join(", ")
);
}
/// The name of a registry `AVCodec` (`(*codec).name`), owned — the field every decode
/// log carries so `decoder="av1"` vs `decoder="libdav1d"` is one glance, not a debugger.
///
/// # Safety
/// `codec` must point to a registered `AVCodec` (their `name` is a static NUL-terminated
/// string, valid for the process).
pub(crate) unsafe fn codec_name(codec: *const ffmpeg::ffi::AVCodec) -> String {
// SAFETY: caller guarantees a registered AVCodec; `name` is its static C string.
unsafe {
std::ffi::CStr::from_ptr((*codec).name)
.to_string_lossy()
.into_owned()
}
}
/// The `quic` codec bitfield this client can decode — whatever FFmpeg has a decoder for (HEVC/H.264
/// always; AV1 when built in). Advertised to the host so it never emits a codec we can't decode.
pub fn decodable_codecs() -> u8 {
@@ -435,7 +517,11 @@ impl Decoder {
vaapi_tried = true;
match VaapiDecoder::new(codec_id) {
Ok(v) => {
tracing::info!(?codec_id, "VAAPI hardware decode active (zero-copy dmabuf)");
tracing::info!(
?codec_id,
decoder = v.name(),
"VAAPI hardware decode active (zero-copy dmabuf)"
);
return done(Backend::Vaapi(v));
}
Err(e) => {
@@ -470,6 +556,7 @@ impl Decoder {
Ok(d) => {
tracing::info!(
?codec_id,
decoder = d.name(),
"D3D11VA hardware decode active (shared-texture hand-off)"
);
return done(Backend::D3d11va(d));
@@ -490,6 +577,7 @@ impl Decoder {
Ok(v) => {
tracing::info!(
?codec_id,
decoder = v.name(),
"Vulkan Video hardware decode active (presenter-shared device)"
);
return done(Backend::Vulkan(v));
@@ -520,7 +608,11 @@ impl Decoder {
if choice != "software" && choice != "vulkan" && !vaapi_tried {
match VaapiDecoder::new(codec_id) {
Ok(v) => {
tracing::info!(?codec_id, "VAAPI hardware decode active (zero-copy dmabuf)");
tracing::info!(
?codec_id,
decoder = v.name(),
"VAAPI hardware decode active (zero-copy dmabuf)"
);
return done(Backend::Vaapi(v));
}
Err(e) => {
@@ -548,6 +640,7 @@ impl Decoder {
Ok(d) => {
tracing::info!(
?codec_id,
decoder = d.name(),
"D3D11VA hardware decode active (shared-texture hand-off)"
);
return done(Backend::D3d11va(d));
@@ -724,6 +817,7 @@ impl Decoder {
match VaapiDecoder::new(self.codec_id) {
Ok(v) => {
tracing::warn!(error = %e, fails = self.vaapi_fails,
decoder = v.name(),
"Vulkan Video decode failing repeatedly — demoting to VAAPI");
self.backend = Backend::Vaapi(v);
self.vaapi_fails = 0;
@@ -745,6 +839,7 @@ impl Decoder {
) {
Ok(d) => {
tracing::warn!(error = %e, fails = self.vaapi_fails,
decoder = d.name(),
"Vulkan Video decode failing repeatedly — demoting to D3D11VA");
self.backend = Backend::D3d11va(d);
self.vaapi_fails = 0;
+31 -5
View File
@@ -552,6 +552,10 @@ pub(crate) struct D3d11vaDecoder {
/// ([`crate::video::VulkanDecodeDevice::d3d11_hdr10`]) — PQ streams get the HDR
/// pass-through ring; without it they keep the tonemap-to-sRGB ring.
hdr10_out: bool,
/// The selected decoder's registry name (`(*codec).name`) — `"av1"` vs `"libdav1d"`
/// is the difference between hardware decode and a silent CPU fallback, so every
/// log a field report leans on carries it.
name: String,
}
// SAFETY: the libav pointers are this decoder's own allocations (freed once in `Drop`) and the COM
@@ -609,10 +613,16 @@ impl D3d11vaDecoder {
if !d3d11va_decode_supported(hw_device.as_ptr()) {
bail!("GPU can't create the D3D11VA decode surface pool");
}
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
bail!("no {codec_id:?} decoder");
}
// NOT `avcodec_find_decoder`: the ID lookup returns the registry's FIRST
// decoder, and for AV1 that is libdav1d (upstream orders the hwaccel-only
// native decoder last) — a software decoder that silently ignores
// `hw_device_ctx` and fails every frame's D3D11-format guard mid-stream,
// even when the DXVA profile + pool probes above all passed. Select by
// capability instead: the first decoder that can drive AV_PIX_FMT_D3D11
// via hw_device_ctx, or fail here at open.
let codec =
crate::video::find_hw_decoder(codec_id, ffi::AVPixelFormat::AV_PIX_FMT_D3D11)?;
let name = crate::video::codec_name(codec);
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device.as_ptr());
(*ctx).get_format = Some(get_format_d3d11);
@@ -638,10 +648,16 @@ impl D3d11vaDecoder {
video_context1,
ring: None,
hdr10_out,
name,
})
}
}
/// The selected decoder's registry name (e.g. `"av1"`) — see the field doc.
pub(crate) fn name(&self) -> &str {
&self.name
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<D3d11Frame>> {
use ffmpeg::ffi;
// SAFETY: `packet`/`frame`/`ctx` are this decoder's own allocations, live for its whole
@@ -830,6 +846,7 @@ impl D3d11vaDecoder {
src_desc.Height,
index,
color.is_pq(),
&self.name,
);
Ok(D3d11Frame {
width,
@@ -883,7 +900,15 @@ impl Drop for D3d11vaDecoder {
/// One-time dump of the first decoded surface's layout — the forensics for a new GPU/driver.
/// `tex_*` is the DXVA-aligned decode surface (>= the frame); the gap is the padding the
/// stream source rect excludes.
fn log_layout_once(width: u32, height: u32, tex_w: u32, tex_h: u32, index: u32, pq: bool) {
fn log_layout_once(
width: u32,
height: u32,
tex_w: u32,
tex_h: u32,
index: u32,
pq: bool,
decoder: &str,
) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
if ONCE.swap(false, Ordering::Relaxed) {
@@ -894,6 +919,7 @@ fn log_layout_once(width: u32, height: u32, tex_w: u32, tex_h: u32, index: u32,
tex_h,
slice = index,
pq,
decoder,
"D3D11VA first frame"
);
}
@@ -34,6 +34,12 @@ impl SoftwareDecoder {
(*raw).thread_count = 0; // auto
}
let decoder = ctx.decoder().video().context("open video decoder")?;
// Every construction site (session open, preference, mid-stream demotion) says
// which decoder actually opened: for AV1 the ID lookup means libdav1d here —
// deliberately (fastest CPU path; the native `av1` decoder has no software
// path at all) — and the name in the log is what keeps that distinguishable
// from the hardware lanes' capability-selected decoders.
tracing::info!(?codec_id, decoder = codec.name(), "software decoder opened");
Ok(SoftwareDecoder { decoder, sws: None })
}
+22 -6
View File
@@ -46,6 +46,10 @@ pub(crate) struct VaapiDecoder {
hw_device: AvBuffer,
packet: *mut ffmpeg::ffi::AVPacket,
frame: *mut ffmpeg::ffi::AVFrame,
/// The selected decoder's registry name (`(*codec).name`) — `"av1"` vs `"libdav1d"`
/// is the difference between hardware decode and a silent CPU fallback, so every
/// log a field report leans on carries it.
name: String,
}
// SAFETY: the three raw pointers (`ctx`, `packet`, `frame`) are allocations this decoder makes in
@@ -80,11 +84,15 @@ impl VaapiDecoder {
// Owned from here: every `bail!` below drops it, so none of them unref by hand.
let hw_device = AvBuffer::from_raw(hw_device)
.context("av_hwdevice_ctx_create(VAAPI) gave no device")?;
// The negotiated codec's decoder id (av_codec_id maps 1:1 from ffmpeg::codec::Id).
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
bail!("no {codec_id:?} decoder");
}
// NOT `avcodec_find_decoder`: the ID lookup returns the registry's FIRST
// decoder, and for AV1 that is libdav1d (upstream orders the hwaccel-only
// native decoder last) — a software decoder that silently ignores
// `hw_device_ctx` and fails every frame's VAAPI-format guard mid-stream.
// Select by capability instead: the first decoder that can drive
// AV_PIX_FMT_VAAPI via hw_device_ctx, or fail here at open.
let codec =
crate::video::find_hw_decoder(codec_id, ffi::AVPixelFormat::AV_PIX_FMT_VAAPI)?;
let name = crate::video::codec_name(codec);
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device.as_ptr());
(*ctx).get_format = Some(pick_vaapi);
@@ -109,10 +117,16 @@ impl VaapiDecoder {
hw_device,
packet: ffi::av_packet_alloc(),
frame: ffi::av_frame_alloc(),
name,
})
}
}
/// The selected decoder's registry name (e.g. `"av1"`) — see the field doc.
pub(crate) fn name(&self) -> &str {
&self.name
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<DmabufFrame>> {
use ffmpeg::ffi;
// SAFETY: `packet`/`frame`/`ctx` are this decoder's own allocations, live for its whole
@@ -207,7 +221,7 @@ impl VaapiDecoder {
// a single modifier for the texture.
let modifier = d.objects[0].format_modifier;
log_descriptor_once(d, sw_format, fourcc, modifier);
log_descriptor_once(d, sw_format, fourcc, modifier, &self.name);
Ok(DmabufFrame {
width: (*self.frame).width as u32,
@@ -233,6 +247,7 @@ fn log_descriptor_once(
sw: ffmpeg_next::ffi::AVPixelFormat,
fourcc: u32,
modifier: u64,
decoder: &str,
) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
@@ -250,6 +265,7 @@ fn log_descriptor_once(
nb_layers = d.nb_layers,
?layers,
modifier = format_args!("{:#018x}", modifier),
decoder,
"VAAPI dmabuf descriptor layout (first frame)"
);
}
+22 -4
View File
@@ -33,6 +33,10 @@ pub(crate) struct VulkanDecoder {
/// (resolved through the same get_proc_addr chain FFmpeg uses).
wait_semaphores: pf_ffvk::PFN_vkWaitSemaphores,
vk_device: pf_ffvk::VkDevice,
/// The selected decoder's registry name (`(*codec).name`) — `"av1"` vs `"libdav1d"`
/// is the difference between hardware decode and a silent CPU fallback, so every
/// log a field report leans on carries it.
name: String,
/// Storage `AVVulkanDeviceContext` points into (extension string arrays + the
/// feature chain) — FFmpeg reads the extension lists past init (frames-context
/// setup keys code paths off them), so this lives exactly as long as `hw_device`.
@@ -245,10 +249,15 @@ impl VulkanDecoder {
}
let vk_device = (*hwctx).act_dev;
let codec = ffi::avcodec_find_decoder(codec_id.into());
if codec.is_null() {
bail!("no {codec_id:?} decoder");
}
// NOT `avcodec_find_decoder`: the ID lookup returns the registry's FIRST
// decoder, and for AV1 that is libdav1d (upstream orders the hwaccel-only
// native decoder last) — a software decoder that silently ignores
// `hw_device_ctx` and fails every frame's Vulkan-format guard mid-stream.
// Select by capability instead: the first decoder that can drive
// AV_PIX_FMT_VULKAN via hw_device_ctx, or fail here at open.
let codec =
crate::video::find_hw_decoder(codec_id, ffi::AVPixelFormat::AV_PIX_FMT_VULKAN)?;
let name = crate::video::codec_name(codec);
let ctx = ffi::avcodec_alloc_context3(codec);
(*ctx).hw_device_ctx = ffi::av_buffer_ref(hw_device.as_ptr());
(*ctx).get_format = Some(pick_vulkan);
@@ -270,11 +279,17 @@ impl VulkanDecoder {
frame: ffi::av_frame_alloc(),
wait_semaphores,
vk_device,
name,
_ctx_storage: store,
})
}
}
/// The selected decoder's registry name (e.g. `"av1"`) — see the field doc.
pub(crate) fn name(&self) -> &str {
&self.name
}
pub(crate) fn decode(&mut self, au: &[u8]) -> Result<Option<VkVideoFrame>> {
use ffmpeg::ffi;
// SAFETY: `packet`/`frame`/`ctx` are this decoder's own allocations, live for its whole
@@ -388,6 +403,7 @@ impl VulkanDecoder {
(*fc).width,
(*fc).height,
sw,
&self.name,
);
Ok(VkVideoFrame {
vkframe: vkf as usize,
@@ -423,6 +439,7 @@ fn log_layout_once(
pool_w: i32,
pool_h: i32,
sw: ffmpeg::ffi::AVPixelFormat,
decoder: &str,
) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
@@ -433,6 +450,7 @@ fn log_layout_once(
pool_w,
pool_h,
?sw,
decoder,
"Vulkan Video first frame"
);
}
+65 -9
View File
@@ -28,6 +28,7 @@ enum RowId {
Chroma444,
Audio,
Mic,
EchoCancel,
Pad,
PadType,
Touch,
@@ -42,10 +43,10 @@ enum RowId {
// The couch-relevant subset grew 2026-07-31: this screen is the ONLY settings editor in
// Gaming Mode, so a field it omits is simply unreachable there (render scale, 4:4:4,
// scroll/shortcut behavior, fullscreen-on-stream, auto-wake, the library toggle all
// were). Still deliberately smaller than the desktop dialogs — device pickers
// (GPU/speaker/mic) and the profile catalog stay desktop-only.
const ROWS: [RowId; 21] = [
// scroll/shortcut behavior, fullscreen-on-stream, auto-wake, the library toggle and echo
// cancellation all were). Still deliberately smaller than the desktop dialogs — device
// pickers (GPU/speaker/mic) and the profile catalog stay desktop-only.
const ROWS: [RowId; 22] = [
RowId::Resolution,
RowId::Refresh,
RowId::RenderScale,
@@ -57,6 +58,7 @@ const ROWS: [RowId; 21] = [
RowId::Chroma444,
RowId::Audio,
RowId::Mic,
RowId::EchoCancel,
RowId::Pad,
RowId::PadType,
RowId::Touch,
@@ -220,6 +222,9 @@ impl SettingsScreen {
fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
let s = &ctx.settings;
// Echo cancellation only means anything while the mic streams — dimmed and inert while it
// doesn't, the same relationship the desktop shells draw with a greyed-out row.
let enabled = !matches!(id, RowId::EchoCancel) || s.mic_enabled;
let (header, label, value): (Option<&'static str>, &str, String) = match id {
RowId::Resolution => (
Some("Stream"),
@@ -284,6 +289,7 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
.into(),
),
RowId::Mic => (None, "Microphone", on_off(s.mic_enabled).into()),
RowId::EchoCancel => (None, "Echo cancellation", on_off(s.echo_cancel).into()),
RowId::Pad => (
Some("Controller"),
"Use controller",
@@ -330,10 +336,10 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
header,
label: label.into(),
value: Some(value),
value_dim: false,
value_dim: !enabled,
caret: false,
adjustable: true,
enabled: true,
adjustable: enabled,
enabled,
}
}
@@ -359,10 +365,18 @@ fn detail(id: RowId) -> &'static str {
}
RowId::Chroma444 => {
"Full-colour video: crisp small text and thin lines, at more bandwidth. \
HEVC only, and only where the host can encode it."
Needs an NVIDIA host (NVENC) or the PyroWave codec other encoders \
stream 4:2:0 and the session falls back silently."
}
RowId::Audio => "The speaker layout requested from the host.",
RowId::Mic => "Send this device's microphone to the host's virtual mic.",
RowId::Mic => {
"Send this device's microphone to the host's virtual mic. \
Ctrl+Alt+Shift+V mutes and unmutes it while streaming."
}
RowId::EchoCancel => {
"Stops the host's audio, playing from this device's speakers, being picked up \
and sent back. Turn it off if your microphone already runs its own processing."
}
RowId::Pad => "Which pad is forwarded to the host, as player 1.",
RowId::PadType => "The virtual pad the host creates — Automatic matches this controller.",
RowId::Touch => {
@@ -454,6 +468,14 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
step_option(cur, AUDIO.len(), delta, wrap).map(|i| s.audio_channels = AUDIO[i].0)
}
RowId::Mic => toggle(&mut s.mic_enabled, delta, wrap),
// Inert while the mic is off — a boundary thud, matching what the dimmed row shows.
RowId::EchoCancel => {
if s.mic_enabled {
toggle(&mut s.echo_cancel, delta, wrap)
} else {
None
}
}
RowId::Pad => {
// Automatic first, then every connected pad by stable key.
let keys: Vec<String> = std::iter::once(String::new())
@@ -592,6 +614,40 @@ mod tests {
assert!(!ctx.settings.mic_enabled);
}
/// Echo cancellation follows the microphone: inert and dimmed while the mic is off, live
/// the moment it goes on. A row that silently accepted a change nobody could act on would
/// be the same lie as an enabled-looking control.
#[test]
fn echo_cancellation_follows_the_microphone() {
let (mut settings, pads) = ctx_parts();
settings.mic_enabled = false;
assert!(settings.echo_cancel, "it ships on");
let library = crate::library::LibraryShared::default();
let mut ctx = Ctx {
hosts: &[],
library: &library,
settings: &mut settings,
pads: &pads,
deck: false,
device_name: "t",
t: 0.0,
};
assert!(!row_spec(RowId::EchoCancel, &ctx).enabled);
assert!(
!adjust(RowId::EchoCancel, -1, false, &mut ctx),
"mic off = thud"
);
assert!(!adjust(RowId::EchoCancel, 1, true, &mut ctx), "A too");
assert!(ctx.settings.echo_cancel, "and nothing was written");
ctx.settings.mic_enabled = true;
assert!(row_spec(RowId::EchoCancel, &ctx).enabled);
assert!(adjust(RowId::EchoCancel, -1, false, &mut ctx));
assert!(!ctx.settings.echo_cancel);
assert!(adjust(RowId::EchoCancel, 1, true, &mut ctx));
assert!(ctx.settings.echo_cancel);
}
#[test]
fn touch_mode_steps_and_wraps() {
let (mut settings, pads) = ctx_parts();
+58 -1
View File
@@ -48,6 +48,9 @@ struct Drawn {
height: u32,
stats: Option<String>,
hint: Option<String>,
/// The mic-mute badge is up. Part of the damage key like everything else here — the badge
/// is static once drawn, so a muted stream still re-renders nothing per frame.
mic_muted: bool,
/// The UI scale this was drawn at, in percent — part of the damage key so dragging the window
/// to a differently-scaled monitor re-renders the chrome at the new size instead of keeping
/// the stale one (the text is identical, so nothing else here would notice).
@@ -390,7 +393,12 @@ impl Overlay for SkiaOverlay {
// spinning through the damage gate; `+ 1` keeps an active resize's step nonzero
// even on its first frame (phase 0) so the guard below doesn't skip it.
let resize_step = resize_phase.map_or(0, |p| (p * 120.0) as u16 + 1);
if ctx.stats.is_none() && ctx.hint.is_none() && banner_step == 0 && resize_step == 0 {
if ctx.stats.is_none()
&& ctx.hint.is_none()
&& !ctx.mic_muted
&& banner_step == 0
&& resize_step == 0
{
self.drawn = Drawn::default(); // forget content so re-show re-renders
return Ok(None);
}
@@ -402,6 +410,7 @@ impl Overlay for SkiaOverlay {
height: ctx.height,
stats: ctx.stats.map(str::to_owned),
hint: ctx.hint.map(str::to_owned),
mic_muted: ctx.mic_muted,
scale_pct: (scale * 100.0).round() as u16,
banner_step,
resize_step,
@@ -437,6 +446,11 @@ impl Overlay for SkiaOverlay {
if let Some(stats) = &want.stats {
draw_osd_panel(canvas, font, stats, ctx.width, scale);
}
// Top-RIGHT, so it never collides with the stats panel or the bottom pill: the badge
// has to stay readable at every stats tier, including Off.
if want.mic_muted {
draw_mic_muted_badge(canvas, font, ctx.width, scale);
}
if let Some(hint) = &want.hint {
draw_hint_pill(canvas, font, hint, ctx.width, ctx.height, 1.0, scale);
} else if banner_step > 0 {
@@ -660,6 +674,49 @@ fn draw_osd_panel(canvas: &Canvas, base_font: &Font, text: &str, width: u32, sca
}
}
/// The mic-mute badge: a dot in the error colour plus the words, on the same translucent pill
/// as the rest of the chrome, pinned to the TOP-RIGHT corner.
///
/// It is drawn from `FrameCtx::mic_muted` alone — not from the stats text — because it must
/// survive the stats overlay being Off, which is where most people leave it. Words, not a
/// glyph: the chrome font is a system monospace resolved at runtime and cannot be relied on to
/// carry a crossed-out microphone. Persistent by design (no fade): a fading indicator answers
/// "did the chord register?" but not "am I muted right now?", and the second question is the
/// one that matters ten minutes later.
fn draw_mic_muted_badge(canvas: &Canvas, base_font: &Font, width: u32, scale: f32) {
const LABEL: &str = "Microphone muted";
// Short line — it fits any window the stream runs in, so it takes the display scale as-is.
let font = &chrome_font(base_font, scale);
let (_, metrics) = font.metrics();
let line_h = metrics.descent - metrics.ascent;
let (pad_x, pad_y) = (base::PILL_PAD_X * scale, base::PILL_PAD_Y * scale);
let dot_r = 4.0 * scale;
let dot_gap = 8.0 * scale;
let text_w = font.measure_str(LABEL, None).0;
let w = text_w + 2.0 * dot_r + dot_gap + 2.0 * pad_x;
let h = line_h + 2.0 * pad_y;
let margin = base::OSD_MARGIN * scale;
let (x, y) = (width as f32 - w - margin, margin);
canvas.draw_rrect(
RRect::new_rect_xy(Rect::from_xywh(x, y, w, h), h / 2.0, h / 2.0),
&Paint::new(Color4f::new(0.0, 0.0, 0.0, 0.62), None),
);
canvas.draw_circle(
Point::new(x + pad_x + dot_r, y + h / 2.0),
dot_r,
&Paint::new(crate::theme::ERROR, None),
);
canvas.draw_str(
LABEL,
Point::new(
x + pad_x + 2.0 * dot_r + dot_gap,
y + pad_y - metrics.ascent,
),
font,
&Paint::new(Color4f::new(1.0, 1.0, 1.0, 0.92), None),
);
}
/// The mid-stream-resize cover: a full-screen dark scrim, the shared rotating spinner, and
/// a "Resizing…" label centered over it — so the host's 0.32 s virtual-display + encoder
/// rebuild reads as a deliberate pause rather than the stream stretching to the changed
+5 -3
View File
@@ -6,7 +6,7 @@
use crate::anim::{approach, Spring, TRAY_C, TRAY_K};
use crate::library::{BUMP_C, BUMP_K};
use crate::theme::{brand, white, Fonts, PanelStroke, BRAND, FAINT, W, WHITE};
use crate::theme::{brand, white, Fonts, PanelStroke, BRAND, DIM, FAINT, W, WHITE};
use pf_client_core::gamepad::{MenuDir, MenuEvent, MenuPulse};
use skia_safe::{Canvas, Paint, Path, RRect, Rect};
@@ -35,7 +35,9 @@ pub(crate) struct RowSpec {
pub caret: bool,
/// Show chevrons while focused (left/right steps the value).
pub adjustable: bool,
/// Action rows render dimmed when not yet actionable.
/// Rows render dimmed when they aren't actionable: an action row's centered label loses
/// its brand tint, a value row's label greys — the look for a setting that depends on
/// another one being on (Echo cancellation under Microphone).
pub enabled: bool,
}
@@ -229,7 +231,7 @@ impl MenuList {
baseline,
W::SemiBold,
16.0 * k,
WHITE,
if row.enabled { WHITE } else { DIM },
);
let value = row.value.as_deref().unwrap_or_default();
let vcolor = if row.value_dim {
+5 -4
View File
@@ -117,10 +117,11 @@ pub(super) fn resolve_split_mode(bit_depth: u8, pixel_rate: u64) -> u32 {
/// deserves a `warn`, a default being tuned an `info`. Callers LATCH this once next to their
/// resolved subframe state (an env re-read at reconfigure would violate the "open and
/// reconfigure present identical init params" invariant).
/// Linux-cfg'd like its ONLY caller (the `nvenc_cuda` query_caps latch) — Windows sessions have
/// `subframe == forced` by construction (env opt-in only) and never consult this; without the
/// cfg it is dead code on every Windows leg (item-level dead_code, the recurring trap).
#[cfg(target_os = "linux")]
/// Both direct-SDK backends latch it now: Linux at the `nvenc_cuda` query_caps latch, Windows at
/// session init since sub-frame defaults on there too (it used to be env opt-in only, so
/// `subframe == forced` held by construction and the item was Linux-cfg'd to avoid being dead
/// code on the Windows leg — the recurring item-level `dead_code` trap).
#[cfg(any(target_os = "linux", windows))]
pub(super) fn subframe_env_forced() -> bool {
matches!(
std::env::var("PUNKTFUNK_NVENC_SUBFRAME").as_deref(),
+22 -9
View File
@@ -45,7 +45,7 @@
use super::nvenc_core::{
apply_low_latency_config, build_init_params, cached_ceiling, codec_guid, plan_range_recovery,
resolve_slices, resolve_split_mode, resolve_split_subframe, resolve_subframe, store_ceiling,
CeilingKey, LowLatencyConfig, NvStatusExt, RangePlan,
subframe_env_forced, CeilingKey, LowLatencyConfig, NvStatusExt, RangePlan,
};
use super::nvenc_status;
use super::{AuChunk, ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
@@ -588,6 +588,10 @@ pub struct NvencD3d11Encoder {
input_ring_depth: Option<usize>,
/// `NV_ENC_CAPS_ASYNC_ENCODE_SUPPORT` from the caps probe — gates the async retrieve mode.
async_supported: bool,
/// `NV_ENC_CAPS_SUPPORT_SUBFRAME_READBACK` from the caps probe — gates the DEFAULT-on
/// sub-frame readback (the Linux backend's rule since its Phase 3; Windows joined after the
/// 2026-07-31 on-glass A/B), so a GPU without it never has sub-frame forced by default.
subframe_cap: bool,
/// (bitstream, mapped input resource to unmap after retrieval, pts_ns, recovery-anchor) per
/// in-flight encode. The fourth field tags the first frame encoded after a successful
/// [`invalidate_ref_frames`](Encoder::invalidate_ref_frames) — the clean re-anchor P-frame the
@@ -748,6 +752,7 @@ impl NvencD3d11Encoder {
async_rt: None,
input_ring_depth: None,
async_supported: false,
subframe_cap: false,
pending: VecDeque::new(),
frame_idx: 0,
force_kf: false,
@@ -922,6 +927,7 @@ impl NvencD3d11Encoder {
nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_CUSTOM_VBV_BUF_SIZE,
);
let async_enc = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_ASYNC_ENCODE_SUPPORT);
let subframe = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_SUBFRAME_READBACK);
let _ = (api().destroy_encoder)(enc);
// Reject an over-range mode with a clear message instead of an opaque InvalidParam.
@@ -955,10 +961,12 @@ impl NvencD3d11Encoder {
self.rfi_supported = rfi != 0;
self.custom_vbv = custom_vbv != 0;
self.async_supported = async_enc != 0;
self.subframe_cap = subframe != 0;
tracing::info!(
rfi = self.rfi_supported,
custom_vbv = self.custom_vbv,
async_encode = self.async_supported,
subframe_readback = self.subframe_cap,
max = %format!("{wmax}x{hmax}"),
ten_bit = ten_bit != 0,
"NVENC capabilities probed"
@@ -1152,11 +1160,14 @@ impl NvencD3d11Encoder {
// VIDEO_CAP_MULTI_SLICE / Moonlight slices-per-frame client gets real slices.
// `PUNKTFUNK_NVENC_SLICES` stays the operator override in both directions.
self.slices = resolve_slices(self.codec, 4.min(self.max_slices));
// Split × sub-frame arbitration (Phase 8) before the ladder/ceiling key. On Windows
// sub-frame is env-opt-in only, so resolved == forced by construction.
let subframe_req = resolve_subframe(false);
// Split × sub-frame arbitration (Phase 8) before the ladder/ceiling key. Sub-frame
// defaults ON where the GPU advertises SUBFRAME_READBACK (Linux parity; validated by
// the 2026-07-31 .173 on-glass A/B — no regression, and slice-progressive clients
// gain the encode/wire overlap); `PUNKTFUNK_NVENC_SUBFRAME` stays the tri-state
// operator escape in both directions.
let subframe_req = resolve_subframe(self.subframe_cap);
let (split_mode, subframe_req) =
resolve_split_subframe(self.codec, split_mode, subframe_req, subframe_req);
resolve_split_subframe(self.codec, split_mode, subframe_req, subframe_env_forced());
// Find the highest bitrate the GPU's codec LEVEL accepts and CLAMP to it. NVENC rejects
// `initialize_encoder` (InvalidParam) when the bitrate exceeds the level ceiling (e.g. a
// 1 Gbps request on HEVC). Strategy: try the requested rate; if the only problem is a forced
@@ -1318,8 +1329,7 @@ impl NvencD3d11Encoder {
self.session_async = use_async;
// Sub-frame chunked poll (P2f, the Windows leg of the slice pipeline): sync
// retrieve only — chunked poll is a depth-1 sync feature; the async retrieve's
// thread owns the bitstream lock. Sub-frame write itself stays env-gated
// (`PUNKTFUNK_NVENC_SUBFRAME=1`) until the Windows on-glass A/B validates it.
// thread owns the bitstream lock.
self.subframe_chunks = self.slices >= 2 && subframe_req && !use_async;
if self.subframe_chunks {
tracing::info!(
@@ -1659,8 +1669,11 @@ impl Encoder for NvencD3d11Encoder {
let anchor = std::mem::take(&mut self.pending_anchor) && flags == 0;
// Submit-time IDR intent: chunked poll must flag an AU's EARLY chunks before the
// driver reports `pictureType` (only the finishing lock sees it). Exact under
// P-only + infinite GOP: IDRs happen only when forced.
let idr_hint = flags != 0;
// P-only + infinite GOP: IDRs happen only when forced — or on the session-opening
// frame, which NVENC emits as an IDR regardless of pic flags (the Linux twin's
// `is_idr`; without the `opening` term frame 1's early chunks went out unflagged
// and the divergence WARN fired at every session start).
let idr_hint = flags != 0 || opening;
let mut pic = nv::NV_ENC_PIC_PARAMS {
version: nv::NV_ENC_PIC_PARAMS_VER,
inputWidth: self.width,
@@ -35,10 +35,14 @@ pub const DS_FEATURE_PAIRING: &[u8] = &[ // report 0x09 (pairing info: MAC at by
0x00, 0x00, 0x00, 0x00,
];
#[rustfmt::skip]
pub const DS_FEATURE_FIRMWARE: &[u8] = &[ // report 0x20 (firmware info / build date)
pub const DS_FEATURE_FIRMWARE: &[u8] = &[ // report 0x20 (firmware info / build date); bytes 44..46
// = update version, kept ABOVE Sony's real releases (0x0630 as of 2026-08) — an older value
// makes PlayStation Accessories and libScePad titles demand a firmware update the virtual pad
// cannot take ("can't complete the update"), and ≥ 0x0224 is what puts writers on the
// COMPATIBLE_VIBRATION2 convention parse_ds_output accepts alongside flag0.
0x20, 0x4A, 0x75, 0x6E, 0x20, 0x31, 0x39, 0x20, 0x32, 0x30, 0x32, 0x33, 0x31, 0x34, 0x3A, 0x34,
0x37, 0x3A, 0x33, 0x34, 0x03, 0x00, 0x44, 0x00, 0x08, 0x02, 0x00, 0x01, 0x36, 0x00, 0x00, 0x01,
0xC1, 0xC8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x54, 0x01, 0x00, 0x00,
0xC1, 0xC8, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x99, 0x09, 0x00, 0x00,
0x14, 0x00, 0x00, 0x00, 0x0B, 0x00, 0x01, 0x00, 0x06, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
];
@@ -494,10 +498,13 @@ pub fn parse_ds_output(pad: u8, data: &[u8], fb: &mut DsFeedback) {
// data[4]. Scale 0..255 → 0..0xFFFF, same (low, high) convention as the uinput pad's mixer,
// and route to the universal rumble plane (0xCA).
// Writers on firmware ≥ 2.24 signal rumble via COMPATIBLE_VIBRATION2 in valid_flag2
// (data[39] BIT2) instead of flag0 BIT0. Our feature report advertises 0x0154 so the
// kernel and SDL stay on the flag0 convention, but a writer that hardcodes v2 would
// otherwise have its rumble — including stops — silently ignored, and a missed stop
// buzzes for the rest of the session (the 500 ms refresh re-sends stale state forever).
// (data[39] BIT2) instead of flag0 BIT0. Our feature report advertises a version
// above 2.24 (DS_FEATURE_FIRMWARE bytes 44..46, chosen to keep Sony's updater
// quiet), so the kernel and SDL write the v2 flag — while older writers, and any
// that never read the version, stay on flag0. Both conventions must land here: a
// rumble dropped on either — including stops — is silently ignored, and a missed
// stop buzzes for the rest of the session (the 500 ms refresh re-sends stale state
// forever).
if flag0 & 0x03 != 0 || data[39] & 0x04 != 0 {
let high = (data[3] as u16) << 8;
let low = (data[4] as u16) << 8;
+5
View File
@@ -43,6 +43,11 @@ pub struct FrameCtx<'a> {
pub stats: Option<&'a str>,
/// The capture hint (bottom-center pill, "click to capture…"); `None` = hidden.
pub hint: Option<&'a str>,
/// The user muted their microphone mid-stream (Ctrl+Alt+Shift+V). Draws a persistent
/// badge, deliberately independent of the stats tier: a muted mic is a fact about what
/// the host is hearing, and "did my mute take?" must be answerable with the overlay off.
/// False whenever this session has no mic uplink at all — the badge never invents one.
pub mic_muted: bool,
/// A mid-stream Match-window resize is in flight (design/midstream-resolution-resize.md,
/// client UX): draw a full-screen scrim + spinner so the host's 0.32 s virtual-display
/// and encoder rebuild reads as an intentional pause rather than the stream stretching to
+140 -7
View File
@@ -12,6 +12,9 @@
//! overlay tier isn't Off (Ctrl+Alt+Shift+S cycles Off → Compact → Normal → Detailed;
//! the stdout line always carries the full Detailed text so parsers see a stable
//! shape). Logs go to stderr (the binary configures tracing so).
//!
//! In-stream chords all share the Ctrl+Alt+Shift prefix: Q release/engage, M mouse model,
//! D disconnect, S stats tier, V microphone mute.
use crate::input::{Capture, FingerPhase};
use crate::overlay::{FrameCtx, Overlay, OverlayAction, OverlayFrame, SessionPhase};
@@ -201,6 +204,11 @@ struct StreamState {
/// mid-stream re-syncs keep the end-to-end number honest after an NTP step / drift.
clock_offset: Option<Arc<std::sync::atomic::AtomicI64>>,
hdr: bool,
/// The presented lane was the CPU/software one, where a PQ stream is shown RAW — the
/// software path has no tone-map pass at all (the presenter uploads swscale RGBA
/// as-is; the CSC mode-1 tonemap is hardware-lane only) — so the OSD badge reads
/// `HDR→SDR (raw)` there instead of claiming a tone-map that never ran.
hdr_untonemapped: bool,
// Presenter-side 1 s window (design/stats-unification.md): end-to-end
// capture→displayed (host-clock corrected) p50+p95, display = decoded→displayed p50.
win_e2e_us: Vec<u64>,
@@ -305,6 +313,7 @@ impl StreamState {
latch_grid,
clock_offset: None,
hdr: false,
hdr_untonemapped: false,
win_e2e_us: Vec::with_capacity(256),
win_disp_us: Vec::with_capacity(256),
win_start: Instant::now(),
@@ -681,6 +690,23 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
tracing::info!(tier = ?stats_verbosity, "chord: stats verbosity");
continue;
}
// Mic mute (B4) — "V" for voice; M and S are taken. Per session and never
// persisted: this is the doorbell/cough key, not a settings change. The
// uplink keeps running while muted (see `MicStreamer::spawn`); only the
// sending stops. A session streaming no mic says so instead of silently
// swallowing the chord — the overlay would have nothing to show either.
if chord && sc == Scancode::V {
if let Some(st) = &stream {
match st.handle.mic.toggle() {
Some(muted) => tracing::info!(muted, "chord: microphone mute"),
None => tracing::info!(
"chord: microphone mute — this session streams no \
microphone (turn it on in Settings)"
),
}
}
continue;
}
// F11 or Alt+Enter (some keyboards' Fn layer sends a media key for
// plain F11 — the Moonlight-standard alias always exists).
let alt_enter =
@@ -1083,6 +1109,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
&st.presented,
st.hdr,
presenter.hdr_active(),
st.hdr_untonemapped,
st.profile.as_deref(),
);
if stats_verbosity != StatsVerbosity::Off {
@@ -1095,6 +1122,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
&st.presented,
st.hdr,
presenter.hdr_active(),
st.hdr_untonemapped,
st.profile.as_deref(),
);
println!("stats: {}", full.replace('\n', " | "));
@@ -1208,6 +1236,10 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
let resizing = stream
.as_ref()
.is_some_and(|st| st.connector.is_some() && st.resize_overlay.active());
// Read live from the session's control rather than mirrored into StreamState: the
// pump is what knows whether an uplink exists (it may have failed to open), and a
// mirrored copy would be the thing that goes stale at session end.
let mic_muted = stream.as_ref().is_some_and(|st| st.handle.mic.muted());
let ctx = FrameCtx {
width: pw,
height: ph,
@@ -1217,6 +1249,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
scale: overlay_scale(window.display_scale(), osd_scale_pref),
stats,
hint,
mic_muted,
resizing,
pad: pad.as_ref().map(|p| p.name.as_str()),
pad_pref: pad.as_ref().map(|p| p.pref),
@@ -1271,6 +1304,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// HDR (PQ) pyrowave session presents through the HDR10 path exactly
// like the H.26x codecs (design/pyrowave-444-hdr.md Phase 3).
st.hdr = f.color.is_pq();
st.hdr_untonemapped = false;
match presenter.present(
&window,
FrameInput::PyroWave(f),
@@ -1298,6 +1332,9 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
}
DecodedImage::Cpu(c) => {
st.hdr = c.color.is_pq();
// The software lane shows PQ raw (no tone-map pass exists there)
// — the OSD badge must not claim `HDR→SDR` for it.
st.hdr_untonemapped = true;
presenter.present(&window, FrameInput::Cpu(&c), overlay_frame.as_ref())?
}
#[cfg(target_os = "linux")]
@@ -1305,6 +1342,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if presenter.supports_dmabuf() && !st.dmabuf_demoted =>
{
st.hdr = d.color.is_pq();
st.hdr_untonemapped = false;
match presenter.present(
&window,
FrameInput::Dmabuf(d),
@@ -1355,6 +1393,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
#[cfg(windows)]
DecodedImage::D3d11(d) if presenter.supports_d3d11() && !st.dmabuf_demoted => {
st.hdr = d.color.is_pq();
st.hdr_untonemapped = false;
match presenter.present(
&window,
FrameInput::D3d11(d),
@@ -1401,6 +1440,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// demotion contract as the dmabuf path.
DecodedImage::VkFrame(v) if !st.dmabuf_demoted => {
st.hdr = v.color.is_pq();
st.hdr_untonemapped = false;
match presenter.present(
&window,
FrameInput::VkFrame(v),
@@ -1885,6 +1925,7 @@ fn bump_stats_tier(
&st.presented,
st.hdr,
presenter.hdr_active(),
st.hdr_untonemapped,
st.profile.as_deref(),
),
None => String::new(),
@@ -1982,11 +2023,15 @@ const HINT_WITH_PAD: &str = "Click the stream to capture input · Ctrl+Alt+Shift
///
/// The HDR tag is honest about the display path: `HDR` only when the swapchain actually
/// runs HDR10 (`hdr_display`); a PQ stream tone-mapped onto an SDR surface (no HDR10
/// format offered, HDR off in the compositor) shows `HDR→SDR` instead.
/// format offered, HDR off in the compositor) shows `HDR→SDR`; and a PQ stream on the
/// software-decode lane (`hdr_untonemapped`) shows `HDR→SDR (raw)` — that lane has no
/// tone-map pass at all, so the washed-out picture is named for what it is rather than
/// passed off as a tone-map.
///
/// `profile` (the session's settings profile, `None` for the global defaults) closes the
/// first line at every tier — the cheapest possible answer to "which profile am I on?"
/// (design/client-settings-profiles.md §5.2).
#[allow(clippy::too_many_arguments)]
fn stats_text(
verbosity: StatsVerbosity,
mode_line: &str,
@@ -1994,6 +2039,7 @@ fn stats_text(
p: &PresentedWindow,
hdr_stream: bool,
hdr_display: bool,
hdr_untonemapped: bool,
profile: Option<&str>,
) -> String {
let profile_tag = profile.map(|n| format!(" · {n}")).unwrap_or_default();
@@ -2043,6 +2089,7 @@ fn stats_text(
if s.decoder.is_empty() { "-" } else { s.decoder },
match (hdr_stream, hdr_display) {
(true, true) => " · HDR",
(true, false) if hdr_untonemapped => " · HDR→SDR (raw)",
(true, false) => " · HDR→SDR",
_ => "",
},
@@ -2077,6 +2124,16 @@ fn stats_text(
if s.lost > 0 {
text.push_str(&format!("\nlost {} ({:.1}%)", s.lost, s.lost_pct));
}
// The mic uplink line renders only while voice is actually going out (a healthy 10 ms-frame
// uplink reads ~100 f/s) and only in Detailed — drops here are the client shedding backlog.
// A muted mic reads 0 and drops the line; the mute has its own always-on badge instead, so
// this stays a throughput readout rather than doubling as a mute indicator.
if detailed && (s.mic_sent > 0 || s.mic_dropped > 0) {
text.push_str(&format!("\nmic {} f/s", s.mic_sent));
if s.mic_dropped > 0 {
text.push_str(&format!(" · dropped {}", s.mic_dropped));
}
}
text
}
@@ -2322,6 +2379,8 @@ mod tests {
decode_ms: 1.8,
lost: 3,
lost_pct: 0.4,
mic_sent: 0,
mic_dropped: 0,
decoder: "vulkan",
// Old-host baseline (no reported target, 4:2:0 never asked): the tier
// texts stay exactly what they were before the target/chroma elements.
@@ -2343,7 +2402,7 @@ mod tests {
#[test]
fn stats_text_tiers() {
let (s, p) = sample();
let text = |v| stats_text(v, "1920×1080@120", &s, &p, true, false, None);
let text = |v| stats_text(v, "1920×1080@120", &s, &p, true, false, false, None);
assert_eq!(text(StatsVerbosity::Off), "");
@@ -2360,6 +2419,10 @@ mod tests {
let detailed = text(StatsVerbosity::Detailed);
assert!(detailed.contains("vulkan · HDR→SDR"));
assert!(
!detailed.contains("(raw)"),
"the hardware lane tone-maps — no raw tag"
);
assert!(detailed.contains("host 1.2 · net 0.9 · decode 1.8 · display 1.1 ms"));
assert!(detailed.contains("host: queue 0.3 · encode 0.5 · xfer 0.1 · pace 0.3 ms"));
assert!(detailed.contains("lost 3 (0.4%)"));
@@ -2369,6 +2432,32 @@ mod tests {
);
}
/// The honest HDR badges: a PQ stream on the software-decode lane is shown WITHOUT
/// tone-mapping (that lane has no PQ→sRGB pass), so its badge must not read as the
/// hardware lane's `HDR→SDR` tone-map — and an HDR10 swapchain shows plain `HDR`
/// whatever the lane claims (a CPU frame forces the swapchain to SDR anyway).
#[test]
fn hdr_badge_names_the_untonemapped_cpu_lane() {
let (s, p) = sample();
let badge = |hdr_display, raw| {
stats_text(
StatsVerbosity::Detailed,
"m",
&s,
&p,
true,
hdr_display,
raw,
None,
)
};
assert!(badge(false, true).contains(" · HDR→SDR (raw)"));
assert!(!badge(false, false).contains("(raw)"));
assert!(badge(false, false).contains(" · HDR→SDR"));
assert!(badge(true, false).contains(" · HDR"));
assert!(!badge(true, false).contains("HDR→SDR"));
}
/// Detailed shows the negotiated encoder target next to the measured rate — the
/// figure whose absence let the settings-drop bug ship four releases — tagged
/// `(auto)` when the ABR owns it, plus the honest chroma tag when 4:4:4 was asked.
@@ -2376,7 +2465,7 @@ mod tests {
fn detailed_shows_target_and_chroma_resolution() {
let (mut s, p) = sample();
let line1 = |s: &Stats, v| {
stats_text(v, "m", s, &p, false, false, None)
stats_text(v, "m", s, &p, false, false, false, None)
.lines()
.next()
.unwrap()
@@ -2405,6 +2494,30 @@ mod tests {
assert!(!line1(&s, StatsVerbosity::Detailed).contains("4:4:4"));
}
/// The mic uplink line: Detailed-only, and only while the uplink is live.
#[test]
fn stats_text_mic_line() {
let (mut s, p) = sample();
let text = |s: &Stats, v| stats_text(v, "m", s, &p, false, false, false, None);
assert!(
!text(&s, StatsVerbosity::Detailed).contains("mic"),
"no mic line while the mic is off"
);
s.mic_sent = 100;
let detailed = text(&s, StatsVerbosity::Detailed);
assert!(detailed.contains("\nmic 100 f/s"));
assert!(
!detailed.contains("dropped"),
"a healthy uplink shows no drop term"
);
assert!(
!text(&s, StatsVerbosity::Normal).contains("mic"),
"mic line is Detailed-only"
);
s.mic_dropped = 7;
assert!(text(&s, StatsVerbosity::Detailed).contains("mic 100 f/s · dropped 7"));
}
/// Compact omits the latency term until the presenter's first e2e window lands.
#[test]
fn compact_waits_for_e2e() {
@@ -2412,7 +2525,16 @@ mod tests {
s.lost = 0;
let p = PresentedWindow::default();
assert_eq!(
stats_text(StatsVerbosity::Compact, "m", &s, &p, false, false, None),
stats_text(
StatsVerbosity::Compact,
"m",
&s,
&p,
false,
false,
false,
None
),
"120 fps · 24 Mb/s"
);
}
@@ -2430,6 +2552,7 @@ mod tests {
&p,
false,
false,
false,
Some("Game")
),
"120 fps · 6.4 ms · 24 Mb/s · lost 3 · Game"
@@ -2441,6 +2564,7 @@ mod tests {
&p,
false,
false,
false,
Some("Work"),
);
assert_eq!(
@@ -2454,13 +2578,22 @@ mod tests {
&p,
true,
true,
false,
Some("Work"),
);
assert!(detailed.lines().next().unwrap().ends_with("· HDR · Work"));
// No profile → the line is exactly what it always was.
assert!(
!stats_text(StatsVerbosity::Normal, "m", &s, &p, false, false, None).contains(" · ")
);
assert!(!stats_text(
StatsVerbosity::Normal,
"m",
&s,
&p,
false,
false,
false,
None
)
.contains(" · "));
}
#[test]
+21 -1
View File
@@ -40,7 +40,27 @@ impl Presenter {
// PQ→sRGB pass.
let frame_pq = match &input {
FrameInput::Redraw => None,
FrameInput::Cpu(_) => Some(false),
FrameInput::Cpu(f) => {
// The swapchain answer stays `false` (above) — but a PQ stream on this
// lane is then shown RAW: no PQ→sRGB pass exists here (the CSC mode-1
// tonemap is hardware-lane only; CPU frames are a straight RGBA upload),
// so the picture is washed out and the pq-downgrade warn below never
// fires. Say so once, or the only trace is an OSD badge. (A process-once
// latch, same idiom as the decoders' first-frame layout dumps — the
// condition is a property of the lane, not of one Presenter.)
if f.color.is_pq() {
use std::sync::atomic::{AtomicBool, Ordering};
static WARNED: AtomicBool = AtomicBool::new(false);
if !WARNED.swap(true, Ordering::Relaxed) {
tracing::warn!(
"HDR10 (PQ) stream on the software-decode lane — it has no \
PQsRGB pass, so the picture is shown untonemapped (washed \
out). Hardware decode restores correct colour."
);
}
}
Some(false)
}
#[cfg(target_os = "linux")]
FrameInput::Dmabuf(d) => Some(d.color.is_pq()),
FrameInput::VkFrame(v) => Some(v.color.is_pq()),
+19 -6
View File
@@ -731,12 +731,17 @@ pub(super) fn pick_formats(
}
/// MAILBOX when the surface offers it, FIFO otherwise (`PUNKTFUNK_PRESENT_MODE=
/// fifo|mailbox|immediate` overrides). Both are tear-free, but an arrival-paced
/// presenter must not block in FIFO's present queue: when the compositor holds images
/// for a vblank pass (gamescope's composite path) or arrival cadence drifts against
/// refresh, `acquire_next_image` stalls most of a refresh — a standing 11-13 ms added
/// to every frame at 60 Hz. MAILBOX never queues more than the newest frame, so the
/// fifo|mailbox|immediate|fifo_relaxed` overrides). Both defaults are tear-free, but an
/// arrival-paced presenter must not block in FIFO's present queue: when the compositor
/// holds images for a vblank pass (gamescope's composite path) or arrival cadence drifts
/// against refresh, `acquire_next_image` stalls most of a refresh — a standing 11-13 ms
/// added to every frame at 60 Hz. MAILBOX never queues more than the newest frame, so the
/// pipeline stays at decode latency and a late frame is replaced, not waited for.
///
/// AMD's Windows driver offers no MAILBOX (NVIDIA does), so those clients land on FIFO —
/// expected, not a client misconfiguration. FIFO_RELAXED is opt-in only: it tears exactly
/// when a stream frame misses the vblank it was pacing for, which on a drifting arrival
/// cadence is often — a trade the user must choose, never a silent fallback.
fn pick_present_mode(
surface_i: &ash::khr::surface::Instance,
pdev: vk::PhysicalDevice,
@@ -748,7 +753,15 @@ fn pick_present_mode(
let want = match std::env::var("PUNKTFUNK_PRESENT_MODE").ok().as_deref() {
Some("fifo") => vk::PresentModeKHR::FIFO,
Some("immediate") => vk::PresentModeKHR::IMMEDIATE,
_ => vk::PresentModeKHR::MAILBOX,
Some("fifo_relaxed") => vk::PresentModeKHR::FIFO_RELAXED,
Some("mailbox") | None => vk::PresentModeKHR::MAILBOX,
Some(other) => {
tracing::warn!(
value = other,
"unknown PUNKTFUNK_PRESENT_MODE (expected fifo|mailbox|immediate|fifo_relaxed) — using mailbox"
);
vk::PresentModeKHR::MAILBOX
}
};
Ok(if modes.contains(&want) {
want
+99 -12
View File
@@ -19,6 +19,43 @@ pub const DEFAULT_FEED_BASE: &str =
/// One fetch's wall-clock budget.
const FETCH_TIMEOUT: Duration = Duration::from_secs(15);
/// Why a fetch didn't produce a manifest.
///
/// Exactly one failure mode is an *expected* steady state: a channel nobody has published to
/// answers `manifest.json` with a 404. Collapsing that into the same string as a transport or
/// signature failure is what made an empty stable feed render as "last check failed: feed
/// returned HTTP 404" — telling operators their box is broken when the feed is merely empty.
/// Everything else stays a real failure, loudly.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum FeedError {
/// This channel has no manifest at all. Note the deliberate narrowness: only a 404 on
/// `manifest.json` itself counts. A 404 on the *signature* means the manifest exists
/// without its proof — a half-published pair (the publisher's manifest-then-signature
/// window, or a botched upload), which must stay fail-closed and noisy.
NotPublished,
/// A real failure: transport, an HTTP status that isn't the empty-channel 404, the size
/// cap, or signature/schema rejection.
Failed(String),
}
impl FeedError {
/// Is this the benign "nothing published on this channel yet" state?
pub fn is_not_published(&self) -> bool {
matches!(self, Self::NotPublished)
}
}
impl std::fmt::Display for FeedError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::NotPublished => f.write_str("no release has been published on this channel yet"),
Self::Failed(msg) => f.write_str(msg),
}
}
}
impl std::error::Error for FeedError {}
/// The feed base, with a `PUNKTFUNK_UPDATE_FEED` override for tests and dev feeds. This is
/// operator config (an env var on the process), never request-time input; the `https://` (or
/// loopback) requirement keeps a stray value from silently downgrading the transport.
@@ -36,9 +73,11 @@ pub fn fetch_manifest_blocking(
channel: &str,
keys: &[PublicKey],
user_agent: &str,
) -> Result<Manifest, String> {
) -> Result<Manifest, FeedError> {
if keys.is_empty() {
return Err("no update key is pinned in this build".into());
return Err(FeedError::Failed(
"no update key is pinned in this build".into(),
));
}
let agent = ureq::AgentBuilder::new()
.timeout(FETCH_TIMEOUT)
@@ -48,29 +87,42 @@ pub fn fetch_manifest_blocking(
let url = format!("{base}/{channel}/manifest.json");
let sig_url = format!("{url}.sig");
let body = read_capped(agent.get(&url).call().map_err(fetch_err)?)?;
// Only the MANIFEST leg can report an empty channel; see [`FeedError::NotPublished`].
let body = read_capped(agent.get(&url).call().map_err(manifest_err)?)?;
let sig = read_capped(agent.get(&sig_url).call().map_err(fetch_err)?)?;
let sig_text = String::from_utf8(sig).map_err(|_| "signature file is not text".to_string())?;
let sig_text = String::from_utf8(sig)
.map_err(|_| FeedError::Failed("signature file is not text".into()))?;
manifest::verify_and_parse(&body, &sig_text, keys, channel).map_err(|e| format!("{e:#}"))
manifest::verify_and_parse(&body, &sig_text, keys, channel)
.map_err(|e| FeedError::Failed(format!("{e:#}")))
}
fn fetch_err(e: ureq::Error) -> String {
/// The manifest leg: a 404 here means the channel is empty, not broken.
fn manifest_err(e: ureq::Error) -> FeedError {
match e {
ureq::Error::Status(code, _) => format!("feed returned HTTP {code}"),
other => format!("feed fetch failed: {other}"),
ureq::Error::Status(404, _) => FeedError::NotPublished,
other => fetch_err(other),
}
}
fn read_capped(resp: ureq::Response) -> Result<Vec<u8>, String> {
fn fetch_err(e: ureq::Error) -> FeedError {
FeedError::Failed(match e {
ureq::Error::Status(code, _) => format!("feed returned HTTP {code}"),
other => format!("feed fetch failed: {other}"),
})
}
fn read_capped(resp: ureq::Response) -> Result<Vec<u8>, FeedError> {
use std::io::Read as _;
let mut buf = Vec::new();
let mut reader = resp.into_reader().take(MAX_MANIFEST_BYTES as u64 + 1);
reader
.read_to_end(&mut buf)
.map_err(|e| format!("read failed: {e}"))?;
.map_err(|e| FeedError::Failed(format!("read failed: {e}")))?;
if buf.len() > MAX_MANIFEST_BYTES {
return Err("response exceeds the manifest size cap".into());
return Err(FeedError::Failed(
"response exceeds the manifest size cap".into(),
));
}
Ok(buf)
}
@@ -85,7 +137,42 @@ mod tests {
// licence to trust whatever the feed serves.
let err =
fetch_manifest_blocking("https://127.0.0.1:1", "stable", &[], "test").unwrap_err();
assert!(err.contains("no update key"), "{err}");
assert!(err.to_string().contains("no update key"), "{err}");
// And it is a real failure — never the benign empty-channel state.
assert!(!err.is_not_published());
}
fn status(code: u16) -> ureq::Error {
ureq::Error::Status(code, ureq::Response::new(code, "status", "").unwrap())
}
/// The whole point of the split: an empty channel is not a broken feed.
#[test]
fn manifest_404_is_not_published_but_other_statuses_are_failures() {
assert_eq!(manifest_err(status(404)), FeedError::NotPublished);
for code in [403, 500, 502] {
let e = manifest_err(status(code));
assert!(!e.is_not_published(), "HTTP {code} must stay a failure");
assert!(e.to_string().contains(&code.to_string()), "{e}");
}
}
/// A missing SIGNATURE is a half-published pair, not an empty channel — the manifest leg
/// already answered 200. Treating it as "nothing published yet" would quietly excuse the
/// one window where a manifest exists without its proof.
#[test]
fn signature_404_stays_a_failure() {
let e = fetch_err(status(404));
assert!(!e.is_not_published());
assert_eq!(e.to_string(), "feed returned HTTP 404");
}
#[test]
fn not_published_reads_as_plain_english() {
assert_eq!(
FeedError::NotPublished.to_string(),
"no release has been published on this channel yet"
);
}
#[test]
+1
View File
@@ -35,6 +35,7 @@ pub mod sig;
pub mod version;
pub use detect::{InstallKind, Product};
pub use feed::FeedError;
pub use manifest::{Manifest, MAX_MANIFEST_BYTES, SCHEMA};
pub use sig::{verify_signature, PublicKey};
pub use version::{canary_run, is_newer, triple, Channel};
@@ -138,7 +138,14 @@ impl KwinDisplay {
let kind = match topology {
Topology::Exclusive => TopologyKind::Exclusive,
Topology::Primary => TopologyKind::Primary,
Topology::Extend | Topology::Auto => return Vec::new(),
Topology::Extend | Topology::Auto => {
// No topology to apply — but the output must still be its OWN desktop rather than a
// mirror of someone's panel, and KWin restores a stored `replicationSource` onto our
// (stable) output name for whatever monitor set it was saved under. Applies only if
// it really is mirroring; nothing else about the user's arrangement is touched.
crate::kwin_output_mgmt::clear_replication_source(our_prefix, dims.0, dims.1);
return Vec::new();
}
};
// In-process over Wayland — immune to whatever wedges the standalone kscreen-doctor.
let outcome = crate::kwin_output_mgmt::apply_topology(our_prefix, dims.0, dims.1, kind);
@@ -112,6 +112,34 @@ const POLL_MS: i32 = 100;
/// asked — matches `kwin::CVT_H_GRANULARITY`. Used when matching the generated mode back.
const CVT_H_GRANULARITY: u32 = 8;
/// `kde_output_management_v2.set_replication_source` (and the device's `replication_source` event)
/// arrived in v13. wayland-rs does not range-check requests, so sending one to a lower-version bind
/// would be a protocol error that kills the connection — every call site gates on this.
const REPLICATION_SOURCE_SINCE: u32 = 13;
/// The `source` value that means "this output mirrors nothing" — KWin's `applyMirroring` looks the
/// source UUID up among the enabled outputs and treats an EMPTY string as no replication at all.
const NO_REPLICATION_SOURCE: &str = "";
/// Is this output currently a MIRROR of another one?
///
/// KWin persists output config per *setup* — the exact set of connected outputs, matched by
/// EDID/connector — in `kwinoutputconfig.json`, and `replicationSource` is one of the fields it
/// stores and restores (`OutputConfigurationStore::storeConfig` / `setupToConfig`). Our virtual
/// output carries a STABLE name across sessions (that is deliberate — KWin keys per-output scale by
/// it), so a stored `replicationSource` for that name is re-applied to OUR output on every session
/// that reproduces the same monitor set. The output then shows the source's viewport instead of
/// being its own desktop, which is the whole point of creating it — and per the protocol's own note
/// on `priority`, "an output may not be in the output order if it's disabled **or mirroring another
/// screen**", so the primary assertion silently stops meaning anything too.
///
/// The event carries an empty string for the ordinary case, so `Some("")` must read as "not
/// mirroring" — treating the mere presence of the event as a mirror would de-mirror every output on
/// every apply.
fn is_mirroring(replication_source: Option<&str>) -> bool {
replication_source.is_some_and(|s| !s.is_empty())
}
/// Which topology to apply once our output is resolved.
#[derive(Clone, Copy, PartialEq, Eq)]
pub(crate) enum TopologyKind {
@@ -154,6 +182,9 @@ struct DeviceState {
scale: Option<f64>,
/// KWin's output priority; 1 is the primary. `None` until the `priority` event (device ≥ v18).
priority: Option<u32>,
/// UUID of the output this one MIRRORS, from the `replication_source` event (device ≥ v13).
/// Empty / `None` ⇒ it is its own desktop. See [`is_mirroring`] for why this matters to us.
replication_source: Option<String>,
/// The `current_mode` object id; its size is looked up in [`State::mode_dims`].
current_mode: Option<ObjectId>,
/// Every mode this output advertised, in announce order — `(mode object id, proxy)` — so restore
@@ -307,6 +338,7 @@ impl Dispatch<OutputDevice, u32> for State {
DeviceEvent::Scale { factor } => entry.scale = Some(factor),
DeviceEvent::Enabled { enabled } => entry.enabled = enabled != 0,
DeviceEvent::Priority { priority } => entry.priority = Some(priority),
DeviceEvent::ReplicationSource { source } => entry.replication_source = Some(source),
DeviceEvent::CurrentMode { mode } => entry.current_mode = Some(mode.id()),
DeviceEvent::Mode { mode } => entry.modes.push((mode.id(), mode)),
DeviceEvent::Done => entry.seen_done = true,
@@ -626,6 +658,17 @@ pub(crate) fn apply_topology(
};
let our_uuid = ours.uuid.clone();
let our_id = ours.proxy.as_ref().map(|p| p.id());
if is_mirroring(ours.replication_source.as_deref()) {
// Worth a line of its own: this is the state a user experiences as "the stream just shows my
// monitor", and it comes from KWin's stored config for THIS monitor set, so it reproduces
// every session until something clears it. The config below does.
tracing::warn!(
source_uuid = ?ours.replication_source,
our_prefix,
"KWin had our streamed output MIRRORING another screen (a stored kwinoutputconfig.json \
replicationSource for this monitor set) clearing it so the output is its own desktop"
);
}
// First-slot-wins (§6.1): don't steal primary if another managed sibling already holds it
// (priority 1) — a 2nd exclusive session joins as a secondary of the shared desktop. A
@@ -681,6 +724,17 @@ pub(crate) fn apply_topology(
let config = sess.new_config();
if let Some(proxy) = ours.proxy.as_ref() {
config.enable(proxy, 1);
// State that ours is its OWN desktop, not a replica of somebody's panel. A stored
// `replicationSource` for our (stable) output name is re-applied by KWin on every session
// that reproduces the same monitor set, and it survives everything else this config says:
// enabling and prioritising a mirror still leaves it showing the source's viewport, scaled
// to the source's size (`OutputConfigurationStore::applyMirroring`). See [`is_mirroring`].
// Unconditional rather than conditional on what we enumerated: KWin may apply the stored
// setup config between our enumerate and this apply, and clearing a source that is already
// empty is exactly what KWin does for a non-mirroring output anyway.
if mgmt_version >= REPLICATION_SOURCE_SINCE {
config.set_replication_source(proxy, NO_REPLICATION_SOURCE.to_string());
}
if !sibling_is_primary {
config.set_primary_output(proxy);
if mgmt_version >= 3 {
@@ -781,6 +835,68 @@ pub(crate) fn apply_topology(
}
}
/// De-mirror the just-created virtual output (name starts with `our_prefix`, current size
/// `our_w`×`our_h`) **without touching the rest of the topology** — the `Extend`/`Auto` counterpart
/// to the clear [`apply_topology`] folds into its own config.
///
/// Those topologies deliberately issue no output-management calls: the streamed output is meant to
/// join the desk as one more head, and re-arranging the user's screens would be the rudeness the
/// setting exists to avoid. But a stored `replicationSource` (see [`is_mirroring`]) is not an
/// arrangement — it makes our output show a *physical panel's* viewport instead of its own desktop,
/// which is broken under every topology equally. So this reads the state and applies **only** when
/// our output really is mirroring; the ordinary session pays one bounded enumerate and no apply.
pub(crate) fn clear_replication_source(our_prefix: &str, our_w: u32, our_h: u32) {
let Some(mut sess) = Session::open() else {
return;
};
let deadline = Instant::now() + OP_BUDGET;
let mgmt_version = sess
.state
.mgmt_name_version
.map(|(_, v)| v)
.unwrap_or_default();
if mgmt_version < REPLICATION_SOURCE_SINCE {
return;
}
// Same resolve as `apply_topology`: managed-prefix name AND the birth size, newest global wins.
let Some(ours) = sess
.state
.devices
.values()
.filter(|d| {
d.name.as_deref().is_some_and(|n| n.starts_with(our_prefix))
&& sess.current_dims(d).map(|(w, h, _)| (w, h)) == Some((our_w, our_h))
})
.max_by_key(|d| d.global)
.cloned()
else {
return;
};
if !is_mirroring(ours.replication_source.as_deref()) {
return;
}
let Some(proxy) = ours.proxy.as_ref() else {
return;
};
tracing::warn!(
source_uuid = ?ours.replication_source,
our_prefix,
"KWin had our streamed output MIRRORING another screen (a stored kwinoutputconfig.json \
replicationSource for this monitor set) clearing it so the output is its own desktop"
);
let config = sess.new_config();
config.set_replication_source(proxy, NO_REPLICATION_SOURCE.to_string());
let ok = sess.apply(&config, deadline);
config.destroy();
if !ok {
tracing::warn!(
reason = ?sess.state.failure_reason,
"KWin output management: could not clear the streamed output's replication source — \
the stream will show the mirrored screen's content"
);
}
}
/// Install + select a `want_w`×`want_h`@`want_hz` custom mode on the just-created virtual output
/// (name starts with `our_prefix`, currently at its sacrificial birth size `birth_w`×`birth_h`) —
/// entirely over `kde_output_management_v2`, the in-process replacement for the `kscreen-doctor`
@@ -1010,6 +1126,37 @@ fn find_mode(sess: &Session, dev: &DeviceState, spec: &str) -> Option<DeviceMode
mod tests {
use super::*;
/// KWin sends `replication_source` with an EMPTY string for the ordinary, non-mirroring output.
/// Reading the event's mere presence as "mirroring" would make every apply issue a pointless
/// de-mirror — and, worse, would make the warn fire on every healthy session.
#[test]
fn an_empty_replication_source_is_not_mirroring() {
assert!(!is_mirroring(None));
assert!(!is_mirroring(Some("")));
}
/// A real source UUID is the state the field report describes: the streamed output shows a
/// physical panel's viewport instead of its own desktop.
#[test]
fn a_uuid_replication_source_is_mirroring() {
assert!(is_mirroring(Some("f7a3c1e2-0b44-4c19-9a1d-6f2b8e0c5d31")));
}
/// The clear we send must be the value KWin reads as "mirrors nothing" — an empty source, which
/// its `applyMirroring` fails to resolve to any enabled output and so treats as no replication.
#[test]
fn the_clear_value_is_the_empty_source() {
assert!(!is_mirroring(Some(NO_REPLICATION_SOURCE)));
}
/// The request/event pair is `since 13`; wayland-rs does not range-check requests, so a bind
/// below this must never reach `set_replication_source` (it would be a fatal protocol error).
#[test]
fn replication_source_version_gate_matches_the_protocol() {
assert_eq!(REPLICATION_SOURCE_SINCE, 13);
const { assert!(MGMT_MAX >= REPLICATION_SOURCE_SINCE) };
}
/// The `WxH@Hz` capture rounds mHz to whole Hz — the shape teardown parses back.
#[test]
fn mode_spec_rounds_millihertz() {
+6 -2
View File
@@ -4116,7 +4116,11 @@ pub struct PunktfunkProbeResult {
/// Application goodput bytes / access units the host offered.
pub host_bytes: u64,
pub host_packets: u32,
/// The host's measured burst duration, milliseconds (the throughput denominator).
/// The throughput denominator, milliseconds: the client-measured burst receive interval
/// (first → last probe-packet arrival) once `done`; the host's measured send-window
/// duration when fewer than two probe packets arrived (no interval to measure from). The
/// host duration alone overstates throughput — its window closes while the bottleneck
/// queue is still draining toward the client.
pub elapsed_ms: u32,
/// Delivered wire throughput = `recv_bytes * 8 / elapsed_ms` (kilobits/second).
pub throughput_kbps: u32,
@@ -4130,7 +4134,7 @@ pub struct PunktfunkProbeResult {
}
/// Start a bandwidth speed test: ask the host to burst filler over the data plane at
/// `target_kbps` of goodput for `duration_ms` (each clamped host-side to ≤ 3 Gbps / ≤ 5 s),
/// `target_kbps` of goodput for `duration_ms` (each clamped host-side to ≤ 10 Gbps / ≤ 5 s),
/// *briefly pausing video*. Non-blocking — poll [`punktfunk_connection_probe_result`] until its
/// `done` field is 1. Starting a probe resets any prior measurement.
///
+364 -5
View File
@@ -31,6 +31,10 @@
//! after ~4.5 s clean, ceilinged). Changes are rate-limited (each one costs the IDR the host's
//! rebuilt encoder opens with) and the whole controller disables itself against a host that never
//! answers [`crate::quic::BitrateChanged`] (an older build that ignores unknown control messages).
//! Standing limits are LEARNED rather than re-poked: two identical short host acks latch the
//! encoder's ceiling (`host_cap_kbps`), two consecutive decode-severe backoffs at a similar rate
//! latch the client decoder's knee (`decode_cap_kbps`) — and both re-probe slowly
//! ([`CAP_REPROBE_WINDOWS`]) so neither latch outlives the condition that taught it.
//!
//! Climbs are additionally **evidence-gated**. The target is only a *promise* to the encoder —
//! how many bits it actually emits depends on the content — so on calm content (a menu, an idle
@@ -129,7 +133,16 @@ const ENCODE_SEVERE_US: i64 = 12_000;
/// evidence, not a spec limit — without a re-probe, one heavy scene would cap the whole
/// session. A still-standing limit just re-teaches itself in two short acks, which the host
/// pre-clamps without touching the encoder — the re-probe costs no rebuild, no IDR.
/// The [`decode cap`](BitrateController::decode_cap_kbps) re-probes on the same clock for the
/// same reason: the decoder's knee moves with content and thermals, so its latch must not be
/// permanent either.
const CAP_REPROBE_WINDOWS: u32 = 80;
/// Two consecutive decode-driven backoffs latch the
/// [`decode cap`](BitrateController::decode_cap_kbps) only when their pre-backoff rates agree
/// within ±1/8: the decoder's knee is a RATE, so repeated chokes at the same rate are its
/// signature — two unrelated events (a Wi-Fi flush at 300 Mbps, a decode spike at 500) share
/// no knee and must not teach one.
const DECODE_CAP_SIMILAR_DIV: u32 = 8;
/// Rolling window (in 750 ms report windows, ~30 s) whose minimum mean is the OWD baseline.
/// Long enough to remember the uncongested floor, short enough to follow genuine path changes.
const BASELINE_WINDOWS: usize = 40;
@@ -137,6 +150,23 @@ const BASELINE_WINDOWS: usize = 40;
/// predates bitrate renegotiation and going quiet for the rest of the session.
const MAX_UNACKED: u32 = 3;
/// Operator escape hatch: `PUNKTFUNK_ABR_MAX_MBPS` (megabits/second, the
/// `PUNKTFUNK_PYROWAVE_MAX_MBPS` convention) caps the climb ceiling however it is learned.
/// The startup link-capacity probe MEASURES the ceiling, and
/// [`set_ceiling`](BitrateController::set_ceiling)'s deliberate monotonicity makes an inflated
/// measurement permanent for the session — a link that mis-measures (a bursty middlebox, a
/// queue-flattered interval) needs a knob that binds regardless of what any probe claims.
/// `PUNKTFUNK_ABR_PROBE_KBPS` is NOT that knob: it only shrinks the burst target, not what the
/// measurement may conclude. Unset/0/garbage → no cap. Read once per controller, at
/// construction.
fn ceiling_cap_from_env() -> Option<u32> {
std::env::var("PUNKTFUNK_ABR_MAX_MBPS")
.ok()
.and_then(|v| v.trim().parse::<u32>().ok())
.filter(|&m| m > 0)
.map(|m| m.saturating_mul(1_000))
}
/// One decision per report window; `Some(kbps)` = send a [`crate::quic::SetBitrate`].
pub(crate) struct BitrateController {
/// `false` = permanently off (explicit user bitrate, an old host, or ack silence).
@@ -147,6 +177,10 @@ pub(crate) struct BitrateController {
/// raises it via [`set_ceiling`](Self::set_ceiling) — that measurement is what lets an
/// Automatic session scale past its conservative start.
ceiling_kbps: u32,
/// The `PUNKTFUNK_ABR_MAX_MBPS` cap in kbps (see [`ceiling_cap_from_env`]), injected at
/// construction so tests exercise the clamp without touching the process environment.
/// `None` = no cap.
ceiling_cap_kbps: Option<u32>,
floor_kbps: u32,
/// Slow start: true until the first congestion signal — clean windows DOUBLE the rate
/// (cooldown-paced) instead of the +6 % additive step.
@@ -178,6 +212,24 @@ pub(crate) struct BitrateController {
short_acks: u32,
/// Clean windows spent parked at the learned cap (the re-probe clock).
cap_probe_windows: u32,
/// The client-decoder rate cap, mirroring [`host_cap_kbps`](Self::host_cap_kbps) for the
/// OTHER end of the pipe: latched when two CONSECUTIVE backoffs carried decode-severe
/// evidence (a deep decode-latency excursion, or a jump-to-live flush — in the
/// decoder-saturation regime the flushed backlog formed BEHIND a decoder that stopped
/// keeping up) at a similar pre-backoff rate. Without it a decoder knee below the link
/// ceiling is a permanent 3060 s sawtooth: every ×0.7 backoff re-climbs toward a ceiling
/// the decoder can't hold, and each cycle costs a flush plus a dropped-frame burst (the
/// 1440p120 HEVC field case: knee ~490 Mbps under a ~658 Mbps ceiling). Slowly re-probed
/// on the [`CAP_REPROBE_WINDOWS`] clock, exactly like the host cap, so a decoder that
/// recovers (lighter content, thermal headroom) climbs again — the latch is never
/// permanent.
decode_cap_kbps: Option<u32>,
/// The previous decode-driven backoff's pre-backoff rate (0 = the last backoff wasn't
/// decode-driven): the reference the next one must land near ([`DECODE_CAP_SIMILAR_DIV`])
/// to latch the cap — one spurious flush teaches nothing.
decode_backoff_kbps: u32,
/// Clean windows spent parked at the learned decode cap (its re-probe clock).
decode_cap_probe_windows: u32,
/// Proven throughput: the session's highest windowed ACTUAL delivered rate seen with flat
/// decode latency — the known-good high-water mark climbs are bounded against. Never decays;
/// shrinking capacity (thermals, a heavier scene) is the reactive decode signal's job. On
@@ -196,10 +248,17 @@ impl BitrateController {
/// to build a permanently-disabled controller (explicit bitrate / an old host that didn't
/// echo one — no known ceiling to work against).
pub(crate) fn new(start_kbps: u32) -> Self {
Self::with_ceiling_cap(start_kbps, ceiling_cap_from_env())
}
/// [`new`](Self::new) with the `PUNKTFUNK_ABR_MAX_MBPS` cap injected — the seam the unit
/// tests use so the clamp's behavior never depends on the test process's environment.
fn with_ceiling_cap(start_kbps: u32, ceiling_cap_kbps: Option<u32>) -> Self {
BitrateController {
enabled: start_kbps > 0,
current_kbps: start_kbps,
ceiling_kbps: start_kbps,
ceiling_cap_kbps,
floor_kbps: FLOOR_KBPS.min(start_kbps.max(1)),
probing: true,
owd_means: VecDeque::with_capacity(BASELINE_WINDOWS),
@@ -210,6 +269,9 @@ impl BitrateController {
short_ack_kbps: 0,
short_acks: 0,
cap_probe_windows: 0,
decode_cap_kbps: None,
decode_backoff_kbps: 0,
decode_cap_probe_windows: 0,
proven_kbps: 0,
bad_windows: 0,
clean_windows: 0,
@@ -222,8 +284,12 @@ impl BitrateController {
/// delivered throughput with headroom already subtracted by the caller). Without this call
/// the ceiling stays the negotiated start rate — exactly the old behavior. Never lowers:
/// a congested-moment measurement must not shrink authority below what was negotiated
/// (descent is the congestion signals' job).
/// (descent is the congestion signals' job). The `PUNKTFUNK_ABR_MAX_MBPS` cap clamps HERE
/// — the one funnel every learned ceiling passes through — so it binds no matter how the
/// ceiling was learned; monotonicity is precisely why the user needs it (one inflated
/// measurement is otherwise permanent for the session).
pub(crate) fn set_ceiling(&mut self, kbps: u32) {
let kbps = kbps.min(self.ceiling_cap_kbps.unwrap_or(u32::MAX));
if self.enabled && kbps > self.ceiling_kbps {
self.ceiling_kbps = kbps;
}
@@ -274,11 +340,16 @@ impl BitrateController {
/// An accepted mode switch: the encoder's ceiling and compute knee are properties of the
/// MODE (4K120 caps where 1080p60 never would) — drop the mode-scoped learned state. The
/// probe-measured `ceiling_kbps` (a LINK property) survives.
/// decoder's knee is just as mode-scoped (pixel rate drives both ends of the codec), so
/// the decode cap goes with it. The probe-measured `ceiling_kbps` (a LINK property)
/// survives.
pub(crate) fn on_mode_switch(&mut self) {
self.host_cap_kbps = None;
self.short_acks = 0;
self.cap_probe_windows = 0;
self.decode_cap_kbps = None;
self.decode_backoff_kbps = 0;
self.decode_cap_probe_windows = 0;
self.encode_means.clear();
}
@@ -427,6 +498,30 @@ impl BitrateController {
}
}
}
// The decode cap re-probes on the same clock and for the same reason: the knee is
// content- and thermals-dependent evidence, not a spec limit — a decoder that recovers
// must get its headroom back, so the latch clears UPWARD through here rather than ever
// being permanent. A still-standing knee re-latches from the next pair of
// decode-driven backoffs.
if let Some(cap) = self.decode_cap_kbps {
if bad {
self.decode_cap_probe_windows = 0;
} else if self.current_kbps >= cap.saturating_sub(cap / 16) {
self.decode_cap_probe_windows += 1;
if self.decode_cap_probe_windows >= CAP_REPROBE_WINDOWS {
self.decode_cap_probe_windows = 0;
let lifted = cap.saturating_add(cap / 8).min(self.ceiling_kbps);
if lifted > cap {
tracing::debug!(
from_kbps = cap,
to_kbps = lifted,
"adaptive bitrate: re-probing above the learned decode cap"
);
self.decode_cap_kbps = Some(lifted);
}
}
}
}
let cooled = self
.last_change
.is_none_or(|t| now.duration_since(t) >= CHANGE_COOLDOWN);
@@ -436,6 +531,31 @@ impl BitrateController {
if (self.bad_windows >= BAD_WINDOWS_TO_DECREASE || (severe && self.bad_windows >= 1))
&& self.current_kbps > self.floor_kbps
{
// Decode-cap learning (see [`decode_cap_kbps`](Self::decode_cap_kbps)): a backoff
// with decode-severe evidence — the deep decode excursion, or the flush that
// drained the queue behind a stalled decoder — remembers its pre-backoff rate; the
// SECOND consecutive one at a similar rate latches that rate as the decoder's
// knee. One event never latches (a spurious flush must stay a one-off), and a
// backoff without decode evidence in between breaks the streak — whatever it saw,
// it wasn't the same knee.
if decode_severe || flushed {
let rate = self.current_kbps;
let similar = self.decode_backoff_kbps > 0
&& rate.abs_diff(self.decode_backoff_kbps)
<= self.decode_backoff_kbps / DECODE_CAP_SIMILAR_DIV;
if similar && self.decode_cap_kbps.is_none_or(|c| rate < c) {
tracing::info!(
cap_kbps = rate,
"adaptive bitrate: decode cap learned (decoder knee) — climbs stop \
here until it lifts"
);
self.decode_cap_kbps = Some(rate.max(self.floor_kbps));
self.decode_cap_probe_windows = 0;
}
self.decode_backoff_kbps = rate;
} else {
self.decode_backoff_kbps = 0;
}
let next = ((self.current_kbps as u64 * 7 / 10) as u32).max(self.floor_kbps);
self.bad_windows = 0;
return self.request(next, now);
@@ -447,11 +567,13 @@ impl BitrateController {
// utilized window after a long-enough clean run climbs immediately.
let utilized =
actual_kbps as u64 * UTILIZATION_DEN >= self.current_kbps as u64 * UTILIZATION_NUM;
// The effective ceiling folds in the host-taught cap: the probe measured the LINK, but
// the host's short acks measured the ENCODER — whichever binds first is the limit.
// The effective ceiling folds in both learned caps: the probe measured the LINK, the
// host's short acks measured the ENCODER, and the decode cap measured the CLIENT
// DECODER — whichever binds first is the limit.
let eff_ceiling = self
.ceiling_kbps
.min(self.host_cap_kbps.unwrap_or(u32::MAX));
.min(self.host_cap_kbps.unwrap_or(u32::MAX))
.min(self.decode_cap_kbps.unwrap_or(u32::MAX));
let cap = eff_ceiling
.min(self.proven_kbps.saturating_mul(PROVEN_HEADROOM_NUM) / PROVEN_HEADROOM_DEN);
if self.current_kbps < eff_ceiling && utilized && cap > self.current_kbps {
@@ -1447,6 +1569,243 @@ mod tests {
}
}
#[test]
fn env_max_mbps_caps_every_learned_ceiling() {
// PUNKTFUNK_ABR_MAX_MBPS=50 (injected — `new` reads the env exactly once, at
// construction): a probe "measuring" 886 Mbps (the divisor bug's field figure) must
// not out-rank the user's cap…
let mut c = BitrateController::with_ceiling_cap(20_000, Some(50_000));
c.set_ceiling(886_312);
assert_eq!(c.ceiling_kbps, 50_000);
// …while a measurement under the cap stands untouched.
let mut c = BitrateController::with_ceiling_cap(20_000, Some(50_000));
c.set_ceiling(40_000);
assert_eq!(c.ceiling_kbps, 40_000);
// And the climb honors it: slow start doubles 20→40, the capped ceiling truncates the
// next step to 50, then quiet — never a request past the user's limit.
let mut c = BitrateController::with_ceiling_cap(20_000, Some(50_000));
c.set_ceiling(886_312);
let start = Instant::now();
assert_eq!(run_clean(&mut c, start, 0, 1), Some(40_000));
c.on_ack(40_000);
assert_eq!(run_clean(&mut c, start, 2, 1), Some(50_000));
c.on_ack(50_000);
assert_eq!(run_clean(&mut c, start, 4, 20), None);
}
#[test]
fn decode_cap_latches_after_two_consecutive_decode_severe_backoffs() {
// The 1440p120 field sawtooth: a decoder knee (~500 Mbps) well under the (inflated)
// link ceiling — nothing ever LEARNED the knee, so every re-climb ended in a flush +
// dropped-frame burst. Establish a decode baseline on calm windows, choke twice at the
// same rate, and the second decode-severe backoff must latch the knee.
let mut c = BitrateController::new(500_000);
c.set_ceiling(900_000);
let start = Instant::now();
// Calm baseline windows (2 Mb/s actual: unutilized, so no climb interferes).
for i in 0..4 {
assert_eq!(
c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
Some(8_000),
None,
2_000,
false,
0
),
None
);
}
// First deep decode excursion → immediate ×0.7, but ONE event must not latch.
assert_eq!(
c.on_window(
ticks(start, 4),
0,
0,
Some(10_000),
Some(60_000),
None,
490_000,
false,
0
),
Some(350_000)
);
assert!(c.decode_cap_kbps.is_none());
// Second consecutive decode-severe backoff at the same pre-backoff rate: latch.
assert_eq!(
c.on_window(
ticks(start, 6),
0,
0,
Some(10_000),
Some(60_000),
None,
490_000,
false,
0
),
Some(350_000)
);
assert_eq!(c.decode_cap_kbps, Some(500_000));
// The backoff applies; from here every climb must stop AT the knee — not the 900 Mbps
// link ceiling the old sawtooth kept re-poking.
c.on_ack(350_000);
let mut max_req = 0;
for i in 8..70 {
if let Some(k) = c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
Some(8_000),
None,
1_000_000,
false,
0,
) {
assert!(k <= 500_000, "climb past the decode cap: {k}");
max_req = max_req.max(k);
c.on_ack(k);
}
}
assert_eq!(max_req, 500_000);
assert_eq!(c.current_kbps, 500_000);
assert_eq!(c.decode_cap_kbps, Some(500_000));
}
#[test]
fn a_single_flush_or_dissimilar_backoffs_never_latch_a_decode_cap() {
// The latch's false-positive guards. A lone jump-to-live flush (a Wi-Fi clump can
// flush once at ANY rate) backs off but teaches nothing…
let mut c = BitrateController::new(500_000);
c.set_ceiling(900_000);
let start = Instant::now();
assert_eq!(
c.on_window(ticks(start, 0), 0, 0, None, None, None, 490_000, true, 0),
Some(350_000)
);
assert!(c.decode_cap_kbps.is_none());
c.on_ack(350_000);
// …a LOSS-driven backoff in between breaks the streak…
assert_eq!(
c.on_window(ticks(start, 2), 1, 0, None, None, None, 340_000, false, 0),
Some(245_000)
);
assert!(c.decode_cap_kbps.is_none());
c.on_ack(245_000);
// …so the next flush counts as a FIRST decode event again — still no latch…
assert_eq!(
c.on_window(ticks(start, 4), 0, 0, None, None, None, 240_000, true, 0),
Some(171_500)
);
assert!(c.decode_cap_kbps.is_none());
c.on_ack(171_500);
// …and two consecutive decode events at DISSIMILAR rates (245 vs 171.5 Mbps — no
// common knee) must not latch either.
assert_eq!(
c.on_window(ticks(start, 6), 0, 0, None, None, None, 170_000, true, 0),
Some(120_050)
);
assert!(c.decode_cap_kbps.is_none());
}
#[test]
fn decode_cap_reprobes_after_a_sustained_clean_run() {
// The knee is content/thermals evidence, not a spec limit: after ~60 s parked clean at
// the latched cap, it lifts one step (+12.5 %, ceiling-bounded) — the re-probe path is
// how the latch clears (never permanent), and a still-standing knee just re-latches
// from the next pair of decode-driven backoffs.
let mut c = BitrateController::new(500_000);
c.set_ceiling(900_000);
let start = Instant::now();
for i in 0..4 {
let _ = c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
Some(8_000),
None,
2_000,
false,
0,
);
}
for i in [4, 6] {
let _ = c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
Some(60_000),
None,
490_000,
false,
0,
);
}
assert_eq!(c.decode_cap_kbps, Some(500_000));
// The host's ack parks the session at the knee (its clamp is authoritative).
c.on_ack(500_000);
for i in 0..CAP_REPROBE_WINDOWS {
let _ = c.on_window(
ticks(start, 8 + i),
0,
0,
Some(10_000),
Some(8_000),
None,
490_000,
false,
0,
);
}
assert_eq!(c.decode_cap_kbps, Some(500_000 + 500_000 / 8));
}
#[test]
fn mode_switch_clears_the_decode_cap() {
// A 1440p120 knee means nothing at the new mode's pixel rate — the decode cap must
// not survive the switch (the probe-measured link ceiling does).
let mut c = BitrateController::new(500_000);
c.set_ceiling(900_000);
let start = Instant::now();
for i in 0..4 {
let _ = c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
Some(8_000),
None,
2_000,
false,
0,
);
}
for i in [4, 6] {
let _ = c.on_window(
ticks(start, i),
0,
0,
Some(10_000),
Some(60_000),
None,
490_000,
false,
0,
);
}
assert_eq!(c.decode_cap_kbps, Some(500_000));
c.on_mode_switch();
assert!(c.decode_cap_kbps.is_none());
assert_eq!(c.ceiling_kbps, 900_000);
}
#[test]
fn ack_silence_disables_the_controller() {
let mut c = BitrateController::new(20_000);
+71 -12
View File
@@ -63,11 +63,45 @@ fn join_host_port(host: &str, port: u16) -> String {
}
}
/// Outbound mic uplink queue depth: 5 ms Opus frames, so 64 is ~320 ms of audio — far beyond
/// any worker stall a live mic session survives anyway. On overflow the FRESH frame is dropped
/// (a tokio mpsc can't shed from the head; by the time 320 ms are queued the stream is broken
/// either way, and the bound is about memory, not audio quality) and logged at debug.
const MIC_QUEUE: usize = 64;
/// Outbound mic uplink queue depth: desktop clients send 10 ms Opus frames (mobile still 20 ms),
/// so 12 is ~120240 ms of audio — the hard memory cap, not the working depth. (The old 64 was
/// mislabeled "~320 ms of 5 ms frames"; the frames were 20 ms, so it really allowed 1.28 s, and
/// since a filled tokio mpsc can only drop the FRESH frame, every queued frame became permanent
/// standing latency once a stall filled it.) The pump's mic task now sheds the OLDEST frames
/// whenever more than [`MIC_BACKLOG_MAX`] are still waiting, so a stall costs a short dropout
/// and heals the moment the worker catches up. The producer stays non-blocking: on overflow it
/// still drops the fresh frame (logged at debug) — the shed loop keeps that a rare event.
const MIC_QUEUE: usize = 12;
/// The mic backlog the pump tolerates before shedding oldest-first: ~60 ms of 10 ms frames —
/// enough slack to ride out an encode/send hiccup, small enough that voice stays conversational
/// and never accrues a session-long standing delay.
pub(crate) const MIC_BACKLOG_MAX: usize = 6;
/// Mic uplink counters, shared between the producer side ([`NativeClient::send_mic`]) and the
/// pump's mic task. All monotonic for the session; a stats HUD windows them by diffing
/// successive [`NativeClient::mic_stats`] snapshots (the `frames_dropped` pattern).
#[derive(Debug, Default)]
pub(crate) struct MicUplinkCounters {
/// Frames handed to the QUIC datagram send (past every client-side queue).
pub(crate) sent: AtomicU64,
/// Frames shed at enqueue — the worker queue was full ([`MIC_QUEUE`]).
pub(crate) dropped_full: AtomicU64,
/// Frames shed by the pump's backlog governor (stale-oldest past [`MIC_BACKLOG_MAX`]).
pub(crate) dropped_stale: AtomicU64,
}
/// A [`NativeClient::mic_stats`] snapshot: cumulative mic uplink frame counts per stage.
#[derive(Clone, Copy, Debug, Default)]
pub struct MicUplinkStats {
/// Frames handed to the QUIC datagram send.
pub sent: u64,
/// Frames shed at enqueue (worker queue full).
pub dropped_full: u64,
/// Frames shed by the pump's backlog governor (stale-oldest — see [`MIC_QUEUE`]'s
/// self-healing note).
pub dropped_stale: u64,
}
/// Outbound control-request queue depth. The requests are sparse (mode switches, keyframe
/// requests, ~1.3 loss reports/s, clock re-syncs) — 32 is hours of headroom; a full queue means
@@ -101,9 +135,12 @@ pub struct NativeClient {
cursor_state: Mutex<Receiver<crate::quic::CursorState>>,
input_tx: tokio::sync::mpsc::UnboundedSender<InputEvent>,
/// Outbound mic frames `(seq, pts_ns, opus)` → encoded as 0xCB datagrams by the worker.
/// Bounded ([`MIC_QUEUE`]): a wedged worker drops fresh frames (logged) instead of queueing
/// audio-latency (and memory) without limit — mic is best-effort end to end.
/// Bounded ([`MIC_QUEUE`]): the pump sheds stale frames oldest-first and a full queue drops
/// the fresh one (logged) instead of queueing audio-latency (and memory) without limit —
/// mic is best-effort end to end, and a standing backlog is worse than a dropout.
mic_tx: tokio::sync::mpsc::Sender<(u32, u64, Vec<u8>)>,
/// Mic uplink counters (sent / dropped per stage) — see [`NativeClient::mic_stats`].
mic_stats: Arc<MicUplinkCounters>,
/// Outbound 0xCC rich-input plane, PRE-ENCODED datagrams: [`RichInput`] touchpad/motion
/// (encoded in [`NativeClient::send_rich_input`]) and stylus [`crate::quic::PenBatch`]es
/// (encoded in [`NativeClient::send_pen`]) share the channel — the worker's task just
@@ -402,6 +439,7 @@ impl NativeClient {
let probe = Arc::new(Mutex::new(ProbeState::default()));
let frames_dropped = Arc::new(AtomicU64::new(0));
let fec_recovered = Arc::new(AtomicU64::new(0));
let mic_stats = Arc::new(MicUplinkCounters::default());
let hot_tids = Arc::new(Mutex::new(Vec::new()));
let clock_offset = Arc::new(AtomicI64::new(0));
let decode_lat = Arc::new(Mutex::new(DecodeLatAcc::default()));
@@ -416,6 +454,7 @@ impl NativeClient {
let probe_w = probe.clone();
let frames_dropped_w = frames_dropped.clone();
let fec_recovered_w = fec_recovered.clone();
let mic_stats_w = mic_stats.clone();
let hot_tids_w = hot_tids.clone();
let clock_offset_w = clock_offset.clone();
let decode_lat_w = decode_lat.clone();
@@ -481,6 +520,7 @@ impl NativeClient {
probe: probe_w,
frames_dropped: frames_dropped_w,
fec_recovered: fec_recovered_w,
mic_stats: mic_stats_w,
hot_tids: hot_tids_w,
clock_offset: clock_offset_w,
decode_lat: decode_lat_w,
@@ -516,6 +556,7 @@ impl NativeClient {
cursor_state: Mutex::new(cursor_state_rx),
input_tx,
mic_tx,
mic_stats,
rich_input_tx,
ctrl_tx,
clip: Mutex::new(clip_event_rx),
@@ -720,6 +761,18 @@ impl NativeClient {
self.fec_recovered.load(Ordering::Relaxed)
}
/// Cumulative mic uplink frame counts per stage: handed to the QUIC datagram send, shed at
/// enqueue (worker queue full), and shed by the pump's backlog governor (stale-oldest — see
/// [`MIC_QUEUE`]'s self-healing note). All monotonic for the session; a stats HUD windows
/// them by diffing successive reads, like [`frames_dropped`](Self::frames_dropped).
pub fn mic_stats(&self) -> MicUplinkStats {
MicUplinkStats {
sent: self.mic_stats.sent.load(Ordering::Relaxed),
dropped_full: self.mic_stats.dropped_full.load(Ordering::Relaxed),
dropped_stale: self.mic_stats.dropped_stale.load(Ordering::Relaxed),
}
}
/// Whether the underlying QUIC session has ended — the worker's connection-close watcher set the
/// shutdown flag (`conn.closed()` fired: a host suspend / crash / network drop idle-timed the
/// connection out, or the host closed it), or a deliberate [`disconnect_quit`](Self::disconnect_quit)
@@ -830,7 +883,7 @@ impl NativeClient {
/// `target_kbps` of goodput for `duration_ms`, *briefly pausing video*. Non-blocking — the
/// measurement accumulates in the background; poll [`NativeClient::probe_result`] until its
/// `done` flag is set. Starting a probe resets any prior measurement. The host clamps both
/// fields (≤ 3 Gbps, ≤ 5 s).
/// fields (≤ 10 Gbps, ≤ 5 s).
pub fn request_probe(&self, target_kbps: u32, duration_ms: u32) -> Result<()> {
// Reset the accumulator so a fresh run doesn't blend into the previous one.
*self.probe.lock().unwrap() = ProbeState {
@@ -869,8 +922,12 @@ impl NativeClient {
p.rx_bytes_now.saturating_sub(base_b),
)
};
// The host's burst duration is the throughput denominator. bytes × 8 / ms = kilobits/second.
let window_ms = p.host_duration_ms;
// The throughput denominator: the client-measured receive interval once the report
// froze one, the host's send-window duration as the fallback (see
// `ProbeState::measured_interval_ms` for why the host window alone overstates the
// link). Both are 0 until the report lands, so a partial read reports 0 throughput —
// unchanged. bytes × 8 / ms = kilobits/second.
let window_ms = p.throughput_window_ms();
let throughput_kbps = if window_ms > 0 {
(delivered_bytes.saturating_mul(8) / window_ms as u64) as u32
} else {
@@ -1144,8 +1201,10 @@ impl NativeClient {
match self.mic_tx.try_send((seq, pts_ns, opus)) {
Ok(()) => Ok(()),
Err(TrySendError::Full(_)) => {
// Bounded queue full = the worker stalled for ~MIC_QUEUE x 5 ms. Shed this
// frame (mic is best-effort end to end) instead of queueing latency/memory.
// Bounded queue full = the worker stalled long enough to outrun even the
// pump's oldest-first shed. Drop this frame (mic is best-effort end to end)
// instead of queueing latency/memory; the counter keeps the loss visible.
self.mic_stats.dropped_full.fetch_add(1, Ordering::Relaxed);
tracing::debug!("mic uplink queue full — dropping frame");
Ok(())
}
+119 -5
View File
@@ -1,34 +1,50 @@
//! Speed-test probe state (`ProbeState`, pump-mirrored) and the public `ProbeOutcome`.
/// Accumulated state of an in-flight / finished speed test. The data-plane pump mirrors the
/// session's packet-level receive counters here; the control task finalizes the delivered figure
/// session's probe-scoped receive counters here; the control task finalizes the delivered figure
/// and folds in the host's [`ProbeResult`] when it lands. Read by [`NativeClient::probe_result`].
///
/// Counting at the *packet* level (every delivered wire packet) — not whole reassembled probe AUs —
/// is what makes the measurement degrade gracefully: once loss exceeds the FEC budget no AU
/// completes, so the old AU-based count cliffed to zero even though most bytes still arrived.
/// Counting *probe* packets only (the reassembler stamps dedicated counters at its FLAG_PROBE
/// routing) keeps video out of the numerator: the burst pauses video, but frames already in
/// flight land during its head, and resumed video lands between the last probe packet and the
/// host's report — both used to inflate the all-datagram byte delta this mirrored before.
#[derive(Default)]
pub(crate) struct ProbeState {
/// A probe is in progress: set by `request_probe`, cleared when the host's [`ProbeResult`]
/// lands (a re-probe just overwrites the whole state — the latest one wins).
pub(crate) active: bool,
/// `session.stats()` receive counters at the burst's start (snapshotted by the pump on its first
/// tick while active) and latest, mirrored every pump iteration.
/// Probe-scoped receive counters (`Stats::probe_*`) at the burst's start (snapshotted by the
/// pump on its first tick while active) and latest, mirrored every pump iteration.
pub(crate) base_packets: Option<u64>,
pub(crate) base_bytes: Option<u64>,
pub(crate) rx_packets_now: u64,
pub(crate) rx_bytes_now: u64,
/// First / last probe-packet arrival stamps (monotonic ns, 0 = none yet), mirrored from the
/// probe-scoped session counters. Their difference is the interval the delivered bytes
/// actually arrived in — the honest throughput denominator (see
/// [`measured_interval_ms`](Self::measured_interval_ms)).
pub(crate) first_arrival_ns: u64,
pub(crate) last_arrival_ns: u64,
/// Delivered wire packets / plaintext bytes (header + shard), frozen when the host's report lands
/// (so resumed video after the burst can't inflate them).
pub(crate) delivered_packets: u64,
pub(crate) delivered_bytes: u64,
/// The client-measured receive interval (ms), frozen alongside the delivered figures; 0 = no
/// usable interval (the burst delivered fewer than two probe packets) — consumers fall back
/// to [`host_duration_ms`](Self::host_duration_ms) via
/// [`throughput_window_ms`](Self::throughput_window_ms).
pub(crate) client_interval_ms: u32,
/// The host's end-of-burst report.
pub(crate) host_goodput_bytes: u64,
pub(crate) host_au: u32,
/// Wire packets the host actually put on the link, and the ones its send buffer dropped.
pub(crate) host_wire_packets: u32,
pub(crate) host_send_dropped: u32,
/// The host's measured burst duration (the throughput denominator).
/// The host's measured burst duration (the throughput denominator's FALLBACK — see
/// [`throughput_window_ms`](Self::throughput_window_ms)).
pub(crate) host_duration_ms: u32,
/// The host's `ProbeResult` arrived → the measurement is final.
pub(crate) done: bool,
@@ -39,6 +55,40 @@ pub(crate) struct ProbeState {
pub(crate) duration_ms: u32,
}
impl ProbeState {
/// The client-measured receive interval of a finished burst, in ms: first → last
/// probe-packet arrival, floored at 1 (a sub-ms burst divided by 0 ms would read as
/// infinite throughput). `None` — the caller falls back to the host's duration — when
/// fewer than two probe packets arrived or the stamps are degenerate (unset / identical /
/// reversed): a single arrival spans no interval.
///
/// Why not the host's `duration_ms`: it measures the SEND window, which closes while the
/// bottleneck (switch/kernel) queue is still draining toward the client — the tail of the
/// bytes lands *after* it. Dividing client-side bytes by the host-side window therefore
/// overstates the link: a 1 GbE link under a 2 Gbps burst target "measured" 1266 Mbps and
/// handed the ABR an 886 Mbps ceiling it could never deliver — and
/// [`set_ceiling`](crate::abr::BitrateController::set_ceiling) never lowers, so the lie
/// was permanent for the session.
pub(crate) fn measured_interval_ms(first_ns: u64, last_ns: u64, packets: u64) -> Option<u32> {
if packets < 2 || first_ns == 0 || last_ns <= first_ns {
return None;
}
let ms = ((last_ns - first_ns) / 1_000_000).max(1);
Some(u32::try_from(ms).unwrap_or(u32::MAX))
}
/// The throughput denominator, in ms: the client-measured receive interval when the burst
/// produced one, else the host's send-window duration (an old measurement is better than
/// none — and strictly conservative territory only when packets were too few to matter).
pub(crate) fn throughput_window_ms(&self) -> u32 {
if self.client_interval_ms > 0 {
self.client_interval_ms
} else {
self.host_duration_ms
}
}
}
/// A finished/partial speed-test measurement, returned by [`NativeClient::probe_result`].
#[derive(Clone, Copy, Debug, Default)]
pub struct ProbeOutcome {
@@ -50,7 +100,11 @@ pub struct ProbeOutcome {
/// Application goodput bytes / access units the host offered.
pub host_bytes: u64,
pub host_packets: u32,
/// The burst duration the host measured, in milliseconds (the throughput denominator).
/// The throughput denominator, in milliseconds: the client-measured receive interval
/// (first → last probe-packet arrival) once `done`; the host's measured send-window
/// duration when the burst delivered fewer than two probe packets (no interval to measure
/// from). The host duration alone overstates throughput — its window closes while the
/// bottleneck queue is still draining toward the client.
pub elapsed_ms: u32,
/// Delivered wire throughput = `recv_bytes * 8 / elapsed_ms` (kilobits/second). The figure to
/// drive a [`Hello::bitrate_kbps`] choice from (allow headroom for the FEC overhead + loss).
@@ -66,3 +120,63 @@ pub struct ProbeOutcome {
pub wire_packets_sent: u32,
pub send_dropped: u32,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn interval_needs_two_packets_and_a_nonzero_span() {
// <2 packets: no interval exists — the caller must fall back to the host duration.
assert_eq!(ProbeState::measured_interval_ms(0, 0, 0), None);
assert_eq!(
ProbeState::measured_interval_ms(5_000_000, 5_000_000, 1),
None
);
// Two packets in the same ns / a reversed pair / an unset first stamp: same fallback.
assert_eq!(
ProbeState::measured_interval_ms(5_000_000, 5_000_000, 2),
None
);
assert_eq!(
ProbeState::measured_interval_ms(9_000_000, 5_000_000, 2),
None
);
assert_eq!(ProbeState::measured_interval_ms(0, 5_000_000, 2), None);
}
#[test]
fn interval_is_floored_at_one_ms() {
// Two packets 0.4 ms apart truncate to 0 ms — the floor keeps the division honest
// instead of infinite.
assert_eq!(ProbeState::measured_interval_ms(1_000, 401_000, 2), Some(1));
}
#[test]
fn interval_measures_first_to_last_arrival() {
assert_eq!(
ProbeState::measured_interval_ms(1_000_000, 801_000_000, 1_000),
Some(800)
);
}
#[test]
fn throughput_window_falls_back_to_the_host_duration() {
// No client interval frozen (a <2-packet burst) → the host's send window is the
// denominator, exactly the old behavior.
let p = ProbeState {
host_duration_ms: 800,
..Default::default()
};
assert_eq!(p.throughput_window_ms(), 800);
// With an interval, the client measurement wins — the 1 GbE field case: the same
// bytes over 1010 ms instead of the host's 800 ms is the difference between an
// honest ~940 Mbps and an impossible 1266 Mbps.
let p = ProbeState {
client_interval_ms: 1_010,
host_duration_ms: 800,
..Default::default()
};
assert_eq!(p.throughput_window_ms(), 1_010);
}
}
+10
View File
@@ -68,6 +68,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
probe,
frames_dropped,
fec_recovered,
mic_stats,
hot_tids,
clock_offset,
decode_lat,
@@ -96,11 +97,20 @@ pub(super) async fn run_pump(args: WorkerArgs) {
tokio::spawn(input_task::run(conn.clone(), input_rx, gamepad_snapshots));
// Mic task: embedder Opus mic frames → 0xCB uplink datagrams (best-effort, dropped on loss).
// Self-healing latency bound: every frame still queued once this task catches up is standing
// mic delay from then on, so a frame with more than [`MIC_BACKLOG_MAX`] successors already
// waiting is shed as stale instead of sent — a stall costs a short dropout, not a
// session-long lag (see [`MIC_QUEUE`]). The counters feed [`NativeClient::mic_stats`].
let mic_conn = conn.clone();
tokio::spawn(async move {
while let Some((seq, pts_ns, opus)) = mic_rx.recv().await {
if mic_rx.len() > MIC_BACKLOG_MAX {
mic_stats.dropped_stale.fetch_add(1, Ordering::Relaxed);
continue;
}
let d = crate::quic::encode_mic_datagram(seq, pts_ns, &opus);
let _ = mic_conn.send_datagram(d.into());
mic_stats.sent.fetch_add(1, Ordering::Relaxed);
}
});
@@ -132,12 +132,24 @@ impl ControlTask {
}
} else if let Ok(result) = ProbeResult::decode(&msg) {
let mut p = probe.lock().unwrap();
// Freeze the delivered figures now (the burst is done), before resumed
// video can inflate the packet counters.
// Freeze the delivered figures now (the burst is done). The mirrored
// counters are probe-scoped (stamped at the reassembler's FLAG_PROBE
// routing), so video around the burst inflates nothing; the client's
// first→last arrival interval is frozen with them — the denominator
// that measures when the bytes actually ARRIVED, not when the host
// stopped sending (its window closes while the bottleneck queue is
// still draining this way, which is how a 1 GbE link once "measured"
// 1266 Mbps).
let base_p = p.base_packets.unwrap_or(p.rx_packets_now);
let base_b = p.base_bytes.unwrap_or(p.rx_bytes_now);
p.delivered_packets = p.rx_packets_now.saturating_sub(base_p);
p.delivered_bytes = p.rx_bytes_now.saturating_sub(base_b);
p.client_interval_ms = ProbeState::measured_interval_ms(
p.first_arrival_ns,
p.last_arrival_ns,
p.delivered_packets,
)
.unwrap_or(0);
p.host_goodput_bytes = result.bytes_sent;
p.host_au = result.packets_sent;
p.host_wire_packets = result.wire_packets_sent;
@@ -151,6 +163,7 @@ impl ControlTask {
send_dropped = result.send_dropped,
duration_ms = result.duration_ms,
delivered_packets = p.delivered_packets,
client_interval_ms = p.client_interval_ms,
"speed-test probe result"
);
} else if let Ok(ack) = BitrateChanged::decode(&msg) {
+28 -8
View File
@@ -200,10 +200,23 @@ impl DataPump {
let probe_active = {
let mut p = pump_probe.lock().unwrap();
if p.active && !p.done {
p.rx_packets_now = st.packets_received;
p.rx_bytes_now = st.bytes_received;
p.base_packets.get_or_insert(st.packets_received);
p.base_bytes.get_or_insert(st.bytes_received);
// Arm edge (first mirror tick): zero the arrival stamps before the burst can
// claim them — the ProbeRequest is still queued locally (the burst starts a
// round trip later), so the reset cannot race a probe packet. `st` predates
// the reset, so the stamps mirror 0 on this tick and live values after.
let arming = p.base_bytes.is_none();
if arming {
session.reset_probe_arrivals();
}
p.rx_packets_now = st.probe_packets_received;
p.rx_bytes_now = st.probe_bytes_received;
(p.first_arrival_ns, p.last_arrival_ns) = if arming {
(0, 0)
} else {
(st.probe_first_arrival_ns, st.probe_last_arrival_ns)
};
p.base_packets.get_or_insert(st.probe_packets_received);
p.base_bytes.get_or_insert(st.probe_bytes_received);
}
p.active && !p.done
};
@@ -280,16 +293,23 @@ impl DataPump {
if p.done {
capacity_probe_deadline = None;
// An all-zero reply is a decline (old host / probe-less build) — keep the
// negotiated ceiling. Otherwise: delivered wire kbps × 0.7.
// negotiated ceiling. Otherwise: delivered wire kbps × 0.7, over the
// CLIENT-measured receive interval (the host's send window closes while the
// bottleneck queue is still draining toward us, so dividing by ITS duration
// overstates the link — a 1 GbE link "measured" 1266 Mbps, and the inflated
// ceiling is permanent because set_ceiling never lowers); the host duration
// is the fallback when the burst delivered too few packets for an interval.
if p.host_duration_ms > 0 && p.delivered_bytes > 0 {
let delivered_kbps = (p.delivered_bytes.saturating_mul(8)
/ p.host_duration_ms.max(1) as u64)
as u32;
let window_ms = p.throughput_window_ms();
let delivered_kbps =
(p.delivered_bytes.saturating_mul(8) / window_ms.max(1) as u64) as u32;
let ceiling = delivered_kbps.saturating_mul(7) / 10;
abr.set_ceiling(ceiling);
tracing::info!(
delivered_kbps,
ceiling_kbps = ceiling,
client_interval_ms = p.client_interval_ms,
host_duration_ms = p.host_duration_ms,
"adaptive bitrate: link-capacity probe done — climb ceiling set"
);
} else {
@@ -66,6 +66,9 @@ pub(crate) struct WorkerArgs {
pub(crate) probe: Arc<Mutex<ProbeState>>,
pub(crate) frames_dropped: Arc<AtomicU64>,
pub(crate) fec_recovered: Arc<AtomicU64>,
/// Mic uplink counters (see [`NativeClient::mic_stats`]): the pump's mic task counts wire
/// sends and its own stale-shed drops here; the producer counts queue-full drops.
pub(crate) mic_stats: Arc<MicUplinkCounters>,
pub(crate) hot_tids: Arc<Mutex<Vec<i32>>>,
/// The live clock offset (see [`NativeClient::clock_offset`]): the worker seeds it with the
/// connect-time estimate; the control task's mid-stream re-syncs update it.
+7 -1
View File
@@ -114,7 +114,13 @@ pub use stats::Stats;
/// v13: added `punktfunk_connection_send_pen` — the stylus wire plane
/// (design/pen-tablet-input.md): a client sends `RICH_PEN` sample batches once the host
/// advertises `HOST_CAP_PEN`. Additive and capability-gated, so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 13;
/// v14: added `punktfunk_connection_report_phase` + the `PUNKTFUNK_CLIENT_CAP_PHASE_LOCK` mirror
/// — the phase-locked capture reporter (design/phase-locked-capture.md): a client that advertises
/// the cap reports its next display latch (already converted to host clock), the panel period, an
/// uncertainty and the circular arrival-lead statistic the host's controller steers on. Additive;
/// the wire grows only a new control message (`PhaseReport`, 0x32) an old host never reads and a
/// strict-prefix append on the 0xCF host-timing tail, so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 14;
/// The punktfunk/1 **wire** version — what `Hello`/`Welcome` carry and hosts equality-check.
/// Deliberately its own constant: [`ABI_VERSION`] tracks the embeddable **C surface**
@@ -409,6 +409,27 @@ impl Reassembler {
// can neither advance the video anchor nor be dropped as stale against it (and its aged-out
// frames never count as `frames_dropped`, which would fire video loss recovery).
let is_probe = hdr.user_flags & (FLAG_PROBE as u32) != 0;
if is_probe {
// Probe-scoped receive accounting (the speed-test numerator + denominator, see
// `Stats::probe_first_arrival_ns`), stamped at the routing decision so video in
// flight around the burst contaminates neither the byte count nor the arrival
// stamps. Byte unit mirrors `bytes_received` (whole plaintext packet). The first
// probe packet since the pump armed the probe claims the first-arrival slot (the
// pump zeroes it before the burst can reach the host); every probe packet
// refreshes the last-arrival stamp.
let now_ns = crate::stats::now_monotonic_ns();
StatsCounters::add(&stats.probe_packets_received, 1);
StatsCounters::add(&stats.probe_bytes_received, pkt.len() as u64);
let _ = stats.probe_first_arrival_ns.compare_exchange(
0,
now_ns,
std::sync::atomic::Ordering::Relaxed,
std::sync::atomic::Ordering::Relaxed,
);
stats
.probe_last_arrival_ns
.store(now_ns, std::sync::atomic::Ordering::Relaxed);
}
let win = if is_probe { probe } else { video };
win.advance_window(
hdr.frame_index,
+12
View File
@@ -218,6 +218,18 @@ impl Session {
self.stats.snapshot()
}
/// Re-arm the probe-scoped arrival stamps (see [`Stats::probe_first_arrival_ns`]): zero
/// them so the NEXT burst's first packet claims the first-arrival slot. Called by the
/// client pump when it arms a probe, strictly before the burst can have reached the host
/// (the `ProbeRequest` is still queued locally) — so the reset cannot race a probe packet.
/// The cumulative probe byte/packet counters are left alone: per-burst deltas come from
/// base snapshots, the same pattern the total counters use.
pub fn reset_probe_arrivals(&self) {
let l = std::sync::atomic::Ordering::Relaxed;
self.stats.probe_first_arrival_ns.store(0, l);
self.stats.probe_last_arrival_ns.store(0, l);
}
/// Wrap a packet for the wire: when encrypting, prepend the 8-byte big-endian
/// sequence (the receiver derives the GCM nonce from it) then the ciphertext.
/// Seal one plaintext packet into the reused `wire` buffer in place (no allocation): the wire is
+40 -1
View File
@@ -1,6 +1,19 @@
//! Live counters for the frame-pacing / quality logic and the web UI.
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::OnceLock;
use std::time::Instant;
/// Monotonic now, in ns since an arbitrary process-wide epoch — the basis for the probe
/// arrival stamps below. Monotonic on purpose: the stamps are only ever DIFFERENCED on this
/// machine (the burst's receive interval), and a wall-clock step mid-burst — the exact event
/// the clock re-sync machinery exists for — must not corrupt the one measurement the ABR
/// ceiling is built from, so the CLOCK_REALTIME basis `pts_ns` uses is wrong here. Floored
/// at 1 so a stamp can never collide with the 0 = "unset" sentinel.
pub(crate) fn now_monotonic_ns() -> u64 {
static EPOCH: OnceLock<Instant> = OnceLock::new();
(EPOCH.get_or_init(Instant::now).elapsed().as_nanos() as u64).max(1)
}
/// Immutable snapshot, copied across the C ABI as `PunktfunkStats`.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
@@ -26,11 +39,29 @@ pub struct Stats {
pub fec_late_shards: u64,
pub bytes_sent: u64,
pub bytes_received: u64,
/// Probe-scoped receive counters: wire packets / plaintext bytes carrying
/// [`FLAG_PROBE`](crate::packet::FLAG_PROBE) (speed-test filler), counted at the
/// reassembler's probe routing decision. `bytes_received` counts EVERY accepted datagram,
/// so a speed-test numerator built from it inherits whatever video was in flight around
/// the burst — these keep video out of the probe math. Deliberately NOT mirrored into the
/// C-ABI `PunktfunkStats` (probe measurements surface via `ProbeOutcome`).
pub probe_packets_received: u64,
pub probe_bytes_received: u64,
/// First / last probe-packet arrival (monotonic ns, see [`now_monotonic_ns`]; 0 = none
/// since the last probe arm). Their difference is the burst's client-side receive
/// interval — the honest speed-test denominator: the host's send window closes while the
/// switch/kernel queue toward the client is still draining, so dividing client bytes by
/// the HOST duration overstates the link (a 1 GbE link "measured" 1266 Mbps). The client
/// pump zeroes both when it arms a probe (`Session::reset_probe_arrivals`).
pub probe_first_arrival_ns: u64,
pub probe_last_arrival_ns: u64,
}
/// Atomic accumulators owned by a [`Session`](crate::session::Session). Snapshot to
/// [`Stats`] for readers. `Relaxed` ordering is fine: these are monotonic counters
/// read for display, never used to synchronize other memory.
/// read for display, never used to synchronize other memory. (The two probe arrival
/// stamps are the exception — slots, not counters — but they carry no synchronization
/// duty either: they are read hundreds of ms after the last write.)
#[derive(Default)]
pub struct StatsCounters {
pub frames_submitted: AtomicU64,
@@ -44,6 +75,10 @@ pub struct StatsCounters {
pub fec_late_shards: AtomicU64,
pub bytes_sent: AtomicU64,
pub bytes_received: AtomicU64,
pub probe_packets_received: AtomicU64,
pub probe_bytes_received: AtomicU64,
pub probe_first_arrival_ns: AtomicU64,
pub probe_last_arrival_ns: AtomicU64,
}
impl StatsCounters {
@@ -66,6 +101,10 @@ impl StatsCounters {
fec_late_shards: self.fec_late_shards.load(l),
bytes_sent: self.bytes_sent.load(l),
bytes_received: self.bytes_received.load(l),
probe_packets_received: self.probe_packets_received.load(l),
probe_bytes_received: self.probe_bytes_received.load(l),
probe_first_arrival_ns: self.probe_first_arrival_ns.load(l),
probe_last_arrival_ns: self.probe_last_arrival_ns.load(l),
}
}
}
+43 -1
View File
@@ -115,6 +115,47 @@ pub trait VirtualMic: Send {
fn channels(&self) -> u32 {
CHANNELS as u32
}
/// The adaptive de-jitter target (per-channel samples) the pump measured from uplink
/// arrival jitter (see `mic_jitter`). A backend with a jitter ring primes around this PLUS
/// one of its own consumer quanta: the ring must absorb arrival burstiness (the pump's
/// number) and pull granularity (the backend's own), and neither may buy the other's depth
/// — a 2048-frame recorder gets its one quantum, not three. Never called ⇒ the backend
/// keeps its legacy fixed constants, which is also how `PUNKTFUNK_MIC_LEGACY_BUFFER=1`
/// works (the pump simply never drives the target). Default: no ring, ignored.
fn set_target_depth(&self, _samples_per_ch: usize) {}
/// `(buffered, prime_target)` of the backend's jitter ring in per-channel samples — read
/// by the pump's creep trim + telemetry. `None` while unknown (consumer not yet running,
/// or a backend without a ring).
fn depth(&self) -> Option<(usize, usize)> {
None
}
/// Reset-on-read telemetry counters (see [`MicBackendStats`]). Default: all zero.
fn take_stats(&self) -> MicBackendStats {
MicBackendStats::default()
}
}
/// Reset-on-read counters a [`VirtualMic`] backend reports into the pump's periodic mic
/// telemetry line ("mic uplink health").
#[derive(Debug, Default, Clone, Copy)]
pub struct MicBackendStats {
/// Full-drain re-prime arms: the ring emptied and gates on silence until the target depth
/// rebuilds. One per talk spurt is normal; several per second mid-speech is the crackle.
pub reprimes: u64,
/// Per-channel samples dropped by the ring's overflow cap (drop-oldest).
pub overflow_dropped: u64,
}
/// One-release escape hatch (docs: configuration → Audio / microphone):
/// `PUNKTFUNK_MIC_LEGACY_BUFFER=1` keeps the pre-adaptive fixed mic buffering — the pump never
/// drives the backend target (so the rings stay on their legacy constants: 48 ms prime /
/// 120 ms cap on Windows, the 3-quanta clamp on Linux) and never creep-trims depth.
pub(crate) fn mic_legacy_buffer() -> bool {
static ON: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
*ON.get_or_init(|| std::env::var_os("PUNKTFUNK_MIC_LEGACY_BUFFER").is_some_and(|v| v != "0"))
}
/// Open a virtual microphone with `channels` interleaved channels (1 or 2). Linux: a PipeWire
@@ -152,5 +193,6 @@ mod wasapi_mic;
#[path = "audio/wiring_plan.rs"]
pub(crate) mod wiring_plan;
mod mic_jitter;
mod mic_pump;
pub use mic_pump::MicPump;
pub use mic_pump::{MicFrame, MicPump};
+77 -12
View File
@@ -29,10 +29,10 @@
mod stream_sink;
use super::{AudioCapturer, VirtualMic, SAMPLE_RATE};
use super::{AudioCapturer, MicBackendStats, VirtualMic, SAMPLE_RATE};
use anyhow::{anyhow, Context, Result};
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering};
use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError};
use std::sync::Arc;
use std::thread;
@@ -218,6 +218,26 @@ pub struct PwMicSource {
alive: Arc<AtomicBool>,
/// One-shot flush request, consumed by the process callback (clears the jitter ring).
flush: Arc<AtomicBool>,
/// Ring policy/telemetry shared with the RT process callback (see [`MicRingShared`]).
ring: Arc<MicRingShared>,
}
/// Atomics shared between [`PwMicSource`] (the pump's side) and the RT process callback: the
/// pump's adaptive de-jitter target in, ring depth/prime gauges + reset-on-read counters out.
/// All `Relaxed` — a slowly-moving target and telemetry, not synchronization.
#[derive(Default)]
struct MicRingShared {
/// Pump-set jitter target (per-channel samples). `0` = the pump never spoke (legacy mode,
/// or its first estimate hasn't landed) → the callback keeps the historical 3-quanta clamp.
target: AtomicUsize,
/// Ring depth after the last callback (per-channel samples).
depth: AtomicUsize,
/// Effective prime target of the last callback (per-channel samples).
prime: AtomicUsize,
/// Full-drain re-prime arms (see [`MicBackendStats`]).
reprimes: AtomicU64,
/// Per-channel samples dropped by the overflow cap.
overflow: AtomicU64,
}
impl PwMicSource {
@@ -234,11 +254,13 @@ impl PwMicSource {
// service started before the user session) must surface as an open ERROR — engaging the
// pump's backoff — not as an instantly-dead instance the pump would churn on.
let (ready_tx, ready_rx) = sync_channel::<Result<()>>(1);
let (alive_t, flush_t) = (alive.clone(), flush.clone());
let ring = Arc::new(MicRingShared::default());
let (alive_t, flush_t, ring_t) = (alive.clone(), flush.clone(), ring.clone());
thread::Builder::new()
.name("punktfunk-pw-mic".into())
.spawn(move || {
if let Err(e) = mic_pw_thread(pcm_rx, quit_rx, channels, flush_t, ready_tx) {
if let Err(e) = mic_pw_thread(pcm_rx, quit_rx, channels, flush_t, ring_t, ready_tx)
{
// Reaching here is always a setup/open failure (once the mainloop runs it exits
// Ok) — and it was already reported to the pump via the ready handshake, which
// owns the throttled operator-facing warn. Keep only a debug breadcrumb.
@@ -255,6 +277,7 @@ impl PwMicSource {
quit: quit_tx,
alive,
flush,
ring,
}),
Ok(Err(e)) => Err(e),
Err(_) => Err(anyhow!("pipewire virtual-mic init timed out")),
@@ -291,6 +314,20 @@ impl VirtualMic for PwMicSource {
fn channels(&self) -> u32 {
self.channels
}
fn set_target_depth(&self, samples_per_ch: usize) {
self.ring.target.store(samples_per_ch, Ordering::Relaxed);
}
fn depth(&self) -> Option<(usize, usize)> {
let prime = self.ring.prime.load(Ordering::Relaxed);
// 0 = the process callback hasn't run yet (no consumer) — nothing meaningful to report.
(prime > 0).then(|| (self.ring.depth.load(Ordering::Relaxed), prime))
}
fn take_stats(&self) -> MicBackendStats {
MicBackendStats {
reprimes: self.ring.reprimes.swap(0, Ordering::Relaxed),
overflow_dropped: self.ring.overflow.swap(0, Ordering::Relaxed),
}
}
}
/// Producer-side state for the virtual-mic loop: incoming decoded PCM and a small ring buffer
@@ -303,6 +340,8 @@ struct MicUserData {
primed: bool,
/// One-shot flush request from [`PwMicSource::discard`] (stale-audio drop after a gap).
flush: Arc<AtomicBool>,
/// Pump-driven ring policy + telemetry (see [`MicRingShared`]).
shared: Arc<MicRingShared>,
/// When the process callback last ran — a long gap means the ring content predates the
/// current consumer (the stream idles with no recorder attached) and must be dropped.
last_run: Option<std::time::Instant>,
@@ -318,6 +357,7 @@ fn mic_pw_thread(
quit_rx: pipewire::channel::Receiver<Terminate>,
channels: u32,
flush: Arc<AtomicBool>,
shared: Arc<MicRingShared>,
ready: std::sync::mpsc::SyncSender<Result<()>>,
) -> Result<()> {
use pipewire as pw;
@@ -400,6 +440,7 @@ fn mic_pw_thread(
channels: channels as usize,
primed: false,
flush,
shared,
last_run: None,
};
@@ -473,15 +514,31 @@ fn mic_pw_thread(
);
}
// Adaptive jitter buffer. The client pushes 5 ms frames; the recorder pulls a
// whole *quantum* (often 2043 ms) from an independent clock. A drain of one
// quantum must not outrun what's buffered, or every call underruns to silence
// (the original ~58% gaps). So prime to ~3 quanta before producing, hold there,
// and re-prime only after a genuine full drain (the client went quiet). The ring
// is capped at a few quanta so latency stays bounded.
let target = (3 * want).clamp(720 * ud.channels, 9600 * ud.channels);
// Jitter buffer. The client pushes frames on its own clock; the recorder
// pulls a whole *quantum* (often 2043 ms) from an independent one. A drain
// of one quantum must not outrun what's buffered, or every call underruns
// to silence (the original ~58% gaps). Prime target = one quantum (the pull
// granularity) + the pump's measured-jitter target (arrival burstiness —
// see `mic_jitter`), re-priming only after a genuine full drain (the client
// went quiet). Until the pump's first estimate lands — and forever under
// PUNKTFUNK_MIC_LEGACY_BUFFER — the historical 3-quanta clamp applies; that
// formula scaled with the RECORDING APP's quantum, so a 2048-frame recorder
// bought 128+ ms of standing mic latency for jitter it never had.
let pump_target = ud.shared.target.load(Ordering::Relaxed) * ud.channels;
let target = if pump_target == 0 {
(3 * want).clamp(720 * ud.channels, 9600 * ud.channels)
} else {
want + pump_target
};
let mut dropped = 0usize;
while ud.ring.len() > target.max(want) + want {
ud.ring.pop_front(); // bound latency: drop the oldest beyond ~1 quantum slack
dropped += 1;
}
if dropped > 0 {
ud.shared
.overflow
.fetch_add((dropped / ud.channels) as u64, Ordering::Relaxed);
}
if !ud.primed && ud.ring.len() >= target {
ud.primed = true;
@@ -501,9 +558,17 @@ fn mic_pw_thread(
} else {
0
};
if ud.ring.is_empty() {
if ud.primed && ud.ring.is_empty() {
ud.primed = false; // fully drained — re-prime before producing again
ud.shared.reprimes.fetch_add(1, Ordering::Relaxed);
}
// Publish depth/prime for the pump's creep trim + telemetry.
ud.shared
.depth
.store(ud.ring.len() / ud.channels, Ordering::Relaxed);
ud.shared
.prime
.store(target / ud.channels, Ordering::Relaxed);
let chunk = data.chunk_mut();
*chunk.offset_mut() = 0;
*chunk.stride_mut() = stride as _;
@@ -0,0 +1,471 @@
//! Backend-agnostic de-jitter for the client-mic uplink. The pump feeds every received `0xCB`
//! frame (with its datagram sequence — see [`MicFrame`]) through [`MicDejitter`], which puts a
//! two-frame reorder window in front of the Opus decoder, asks for libopus concealment on true
//! sequence gaps (so a lost datagram doesn't drain the backend ring into a silence + re-prime
//! cycle), and measures inter-arrival jitter into the adaptive target depth the backend rings
//! prime against. Mirrors the client downlink's `AudioGapTracker` (punktfunk-core `audio.rs`):
//! the same wrapping-sequence gap rules, grown the reorder hold and the jitter estimator the
//! uplink needs — the downlink's jitter rings own their depth; the uplink's live in the
//! backends and take their target from here.
use super::MicFrame;
use std::time::{Duration, Instant};
/// What the pump should do next, in playback order. `Frame` is a real frame to decode;
/// `Conceal` is one frame of libopus packet-loss concealment (`decode_float(&[], …)`, sized to
/// the last real frame) standing in for a lost one.
pub(super) enum Deliver {
Frame(MicFrame),
Conceal,
}
/// Most concealment frames one gap may synthesize (~100 ms at the common 20 ms frames) — the
/// client downlink's `AudioGapTracker` cap, scaled to the uplink's bigger frames. libopus PLC
/// fades to silence after a few frames anyway; past the cap the ring's underrun/re-prime path
/// takes over, as before.
const MAX_CONCEAL_FRAMES: u32 = 5;
/// How long one out-of-order frame may wait for its missing predecessor before the gap is
/// concealed and the held frame plays: ~one 20 ms frame interval plus scheduling slack. With
/// the incoming frame that is a two-frame window — anything needing more is treated as loss,
/// exactly like the client downlink (which holds nothing at all).
const HOLD_MAX: Duration = Duration::from_millis(30);
/// Arrival gaps above this are talk-spurt pauses (DTX, mute), not network jitter — folding
/// them into the estimate would balloon the target after every pause.
const PAUSE_GAP: Duration = Duration::from_millis(250);
/// The sliding jitter window: [`BUCKETS`] × [`BUCKET_LEN`] of per-bucket max inter-arrival gap.
/// Long enough that the old bursty Mac cadence (~two 20 ms frames every ~42 ms — the 2026-07-03
/// "crackling mic", see `wasapi_mic.rs`) is always represented; short enough that a one-off
/// spike ages out in a few seconds.
const BUCKETS: usize = 4;
const BUCKET_LEN: Duration = Duration::from_millis(1000);
/// Target before the estimator has a full [`WARMUP`] of evidence: roughly the old fixed 48 ms
/// prime minus the one consumer quantum the backends add — the crackle-safe assumption that
/// the client might be an old bursty one. Modern 10 ms-cadence clients drop to ~1525 ms once
/// measured (after their first talk pause re-primes the ring at the lower target).
const DEFAULT_TARGET_MS: u32 = 40;
/// Headroom over the measured worst burst gap.
const TARGET_PAD_MS: u32 = 5;
/// How long the estimator distrusts its own (young) window and keeps the default as a floor.
const WARMUP: Duration = Duration::from_secs(1);
/// Reset-on-read snapshot for the pump's telemetry line. Counters cover the window since the
/// last read; `cadence_ms`/`frame_ms` are gauges (EWMAs).
#[derive(Debug, Default, Clone, Copy)]
pub(super) struct DejitterStats {
pub seq_gaps: u64,
pub concealed: u64,
pub reorders: u64,
pub late_drops: u64,
/// EWMA inter-arrival gap — the uplink's burst cadence as seen by the pump.
pub cadence_ms: f32,
/// Nominal frame duration from consecutive frames' pts deltas (what the client encodes).
pub frame_ms: f32,
}
pub(super) struct MicDejitter {
/// Sequence the uplink owes next; `None` before the first frame / after
/// [`reset_stream`](Self::reset_stream).
expected: Option<u32>,
/// The reorder window: at most ONE newer frame parked ≤ [`HOLD_MAX`] for its predecessor.
held: Option<(Instant, MicFrame)>,
// --- adaptive target (inter-arrival jitter) ---
first_arrival: Option<Instant>,
last_arrival: Option<Instant>,
bucket_start: Option<Instant>,
bucket_idx: usize,
gap_max: [f32; BUCKETS],
ewma_gap_ms: f32,
// --- pts bookkeeping ---
/// `(seq, pts_ns)` of the newest ingested frame — a pts delta over exactly one seq step is
/// the client's true frame duration.
last_pts: Option<(u32, u64)>,
frame_ms: f32,
// --- counters (reset on read) ---
seq_gaps: u64,
concealed: u64,
reorders: u64,
late_drops: u64,
}
impl MicDejitter {
pub(super) fn new() -> Self {
MicDejitter {
expected: None,
held: None,
first_arrival: None,
last_arrival: None,
bucket_start: None,
bucket_idx: 0,
gap_max: [0.0; BUCKETS],
ewma_gap_ms: 0.0,
last_pts: None,
frame_ms: 0.0,
seq_gaps: 0,
concealed: 0,
reorders: 0,
late_drops: 0,
}
}
/// Feed one received frame; playback-ordered work is appended to `out`.
pub(super) fn ingest(&mut self, now: Instant, frame: MicFrame, out: &mut Vec<Deliver>) {
self.record_arrival(now, frame.seq, frame.pts_ns);
self.accept(now, frame, out);
}
fn accept(&mut self, now: Instant, frame: MicFrame, out: &mut Vec<Deliver>) {
let Some(expected) = self.expected else {
// First frame (or first after a reset): the chain starts here.
self.expected = Some(frame.seq.wrapping_add(1));
out.push(Deliver::Frame(frame));
return;
};
let delta = frame.seq.wrapping_sub(expected);
if delta == 0 {
self.expected = Some(expected.wrapping_add(1));
out.push(Deliver::Frame(frame));
self.release_held_in_order(out);
} else if delta > u32::MAX / 2 {
// Duplicate of, or older than, something already played — the window moved on.
self.late_drops += 1;
} else if let Some(held_seq) = self.held.as_ref().map(|(_, h)| h.seq) {
if held_seq == frame.seq {
self.late_drops += 1; // duplicate of the held frame
return;
}
// Two newer-than-expected frames in hand — waiting longer can't fill the gap
// within the window. Play them in order, concealing what's genuinely missing.
let (_, held) = self.held.take().expect("checked above");
let (first, second) = if frame.seq.wrapping_sub(held.seq) > u32::MAX / 2 {
(frame, held)
} else {
(held, frame)
};
self.give_up_and_play(first, out);
self.accept(now, second, out); // depth ≤ 1: `held` is None here
} else {
self.held = Some((now, frame)); // wait ≤ HOLD_MAX for the predecessor
}
}
/// If the held frame's predecessor just played, the hold is over — the reorder repaired.
fn release_held_in_order(&mut self, out: &mut Vec<Deliver>) {
if let Some((_, h)) = &self.held {
if Some(h.seq) == self.expected {
let (_, h) = self.held.take().expect("checked above");
self.expected = Some(h.seq.wrapping_add(1));
self.reorders += 1;
out.push(Deliver::Frame(h));
}
}
}
/// Stop waiting for `frame`'s predecessors: conceal the gap (capped) and play it.
fn give_up_and_play(&mut self, frame: MicFrame, out: &mut Vec<Deliver>) {
let gap = frame
.seq
.wrapping_sub(self.expected.expect("callers checked"));
if gap > 0 {
self.seq_gaps += 1;
let conceal = gap.min(MAX_CONCEAL_FRAMES);
self.concealed += u64::from(conceal);
for _ in 0..conceal {
out.push(Deliver::Conceal);
}
}
self.expected = Some(frame.seq.wrapping_add(1));
out.push(Deliver::Frame(frame));
}
/// When the current hold (if any) must be given up — the pump's next wake deadline.
pub(super) fn hold_deadline(&self) -> Option<Instant> {
self.held.as_ref().map(|(t, _)| *t + HOLD_MAX)
}
/// Give up an expired hold (the predecessor never came): conceal + play. Called on the
/// pump's timeout wakes.
pub(super) fn flush_expired_hold(&mut self, now: Instant, out: &mut Vec<Deliver>) {
if self.hold_deadline().is_some_and(|d| now >= d) {
let (_, h) = self.held.take().expect("deadline implies held");
self.give_up_and_play(h, out);
}
}
/// Uplink pause / backend reopen: whatever is parked predates the gap, and the next frame
/// restarts the chain (a pause is not loss — it must not conceal or count a gap).
pub(super) fn reset_stream(&mut self) {
self.expected = None;
self.held = None;
self.last_pts = None;
}
// ---- adaptive target -------------------------------------------------------------------
fn record_arrival(&mut self, now: Instant, seq: u32, pts_ns: u64) {
if let Some(last) = self.last_arrival {
let gap = now.duration_since(last);
if gap <= PAUSE_GAP {
let ms = gap.as_secs_f32() * 1000.0;
self.ewma_gap_ms = if self.ewma_gap_ms == 0.0 {
ms
} else {
self.ewma_gap_ms * 0.9 + ms * 0.1
};
self.rotate_buckets(now);
self.gap_max[self.bucket_idx] = self.gap_max[self.bucket_idx].max(ms);
}
}
self.first_arrival.get_or_insert(now);
self.last_arrival = Some(now);
if let Some((last_seq, last_pts)) = self.last_pts {
if seq == last_seq.wrapping_add(1) && pts_ns > last_pts {
let d = (pts_ns - last_pts) as f32 / 1_000_000.0;
if (1.0..=120.0).contains(&d) {
self.frame_ms = if self.frame_ms == 0.0 {
d
} else {
self.frame_ms * 0.9 + d * 0.1
};
}
}
}
self.last_pts = Some((seq, pts_ns));
}
fn rotate_buckets(&mut self, now: Instant) {
let Some(mut start) = self.bucket_start else {
self.bucket_start = Some(now);
return;
};
let mut advanced = 0;
while now.duration_since(start) >= BUCKET_LEN {
self.bucket_idx = (self.bucket_idx + 1) % BUCKETS;
self.gap_max[self.bucket_idx] = 0.0;
start += BUCKET_LEN;
advanced += 1;
if advanced >= BUCKETS {
// Everything aged out (long pause) — snap the window to now.
self.gap_max = [0.0; BUCKETS];
start = now;
break;
}
}
self.bucket_start = Some(start);
}
/// The jitter component of the backend ring's prime depth, in ms — the worst burst gap in
/// the window plus a pad, clamped to [10, 60]. Backends add ONE consumer quantum on top
/// (see [`VirtualMic::set_target_depth`](super::VirtualMic::set_target_depth)). Old bursty
/// clients measure ~42 ms and land near the historical 48 ms prime; modern 10 ms-cadence
/// clients settle at ~1525 ms.
pub(super) fn target_ms(&self, now: Instant) -> u32 {
let measured =
self.gap_max.iter().fold(0f32, |a, &b| a.max(b)).ceil() as u32 + TARGET_PAD_MS;
let warm = self
.first_arrival
.is_some_and(|t| now.duration_since(t) >= WARMUP);
let t = if warm {
measured
} else {
measured.max(DEFAULT_TARGET_MS)
};
t.clamp(10, 60)
}
/// Reset-on-read counters + gauges for the pump's telemetry line.
pub(super) fn take_stats(&mut self) -> DejitterStats {
let s = DejitterStats {
seq_gaps: self.seq_gaps,
concealed: self.concealed,
reorders: self.reorders,
late_drops: self.late_drops,
cadence_ms: self.ewma_gap_ms,
frame_ms: self.frame_ms,
};
self.seq_gaps = 0;
self.concealed = 0;
self.reorders = 0;
self.late_drops = 0;
s
}
}
#[cfg(test)]
mod tests {
use super::*;
fn f(seq: u32) -> MicFrame {
MicFrame {
seq,
pts_ns: u64::from(seq) * 20_000_000,
opus: vec![1],
}
}
/// Compact delivery shape: frame seqs as-is, concealment as -1.
fn seqs(out: &[Deliver]) -> Vec<i64> {
out.iter()
.map(|d| match d {
Deliver::Frame(fr) => i64::from(fr.seq),
Deliver::Conceal => -1,
})
.collect()
}
#[test]
fn in_order_passthrough() {
let mut dj = MicDejitter::new();
let t = Instant::now();
let mut out = Vec::new();
for s in 5..8 {
dj.ingest(t, f(s), &mut out);
}
assert_eq!(seqs(&out), [5, 6, 7]);
let st = dj.take_stats();
assert_eq!(
(st.seq_gaps, st.concealed, st.reorders, st.late_drops),
(0, 0, 0, 0)
);
}
#[test]
fn swapped_pair_is_repaired_not_concealed() {
let mut dj = MicDejitter::new();
let t = Instant::now();
let mut out = Vec::new();
dj.ingest(t, f(5), &mut out);
dj.ingest(t, f(7), &mut out); // held, waiting for 6
assert_eq!(seqs(&out), [5]);
dj.ingest(t, f(6), &mut out); // the missing one arrives late → both play, in order
assert_eq!(seqs(&out), [5, 6, 7]);
let st = dj.take_stats();
assert_eq!(st.reorders, 1);
assert_eq!(st.concealed, 0);
}
#[test]
fn loss_is_concealed_when_the_window_moves_on() {
let mut dj = MicDejitter::new();
let t = Instant::now();
let mut out = Vec::new();
dj.ingest(t, f(5), &mut out);
dj.ingest(t, f(7), &mut out); // held, waiting for 6
dj.ingest(t, f(8), &mut out); // 6 is lost: one PLC frame, then 7 and 8
assert_eq!(seqs(&out), [5, -1, 7, 8]);
let st = dj.take_stats();
assert_eq!(st.seq_gaps, 1);
assert_eq!(st.concealed, 1);
}
#[test]
fn expired_hold_flushes_with_concealment() {
let mut dj = MicDejitter::new();
let t = Instant::now();
let mut out = Vec::new();
dj.ingest(t, f(5), &mut out);
dj.ingest(t, f(7), &mut out);
assert!(dj.hold_deadline().is_some());
dj.flush_expired_hold(t + HOLD_MAX, &mut out);
assert_eq!(seqs(&out), [5, -1, 7]);
assert!(dj.hold_deadline().is_none());
}
#[test]
fn big_gap_conceal_is_capped() {
let mut dj = MicDejitter::new();
let t = Instant::now();
let mut out = Vec::new();
dj.ingest(t, f(5), &mut out);
dj.ingest(t, f(100), &mut out); // held
dj.flush_expired_hold(t + HOLD_MAX, &mut out);
let conceals = out.iter().filter(|d| matches!(d, Deliver::Conceal)).count();
assert_eq!(conceals, MAX_CONCEAL_FRAMES as usize);
assert_eq!(dj.take_stats().seq_gaps, 1);
}
#[test]
fn duplicates_and_stale_late_frames_drop() {
let mut dj = MicDejitter::new();
let t = Instant::now();
let mut out = Vec::new();
dj.ingest(t, f(5), &mut out);
dj.ingest(t, f(6), &mut out);
dj.ingest(t, f(6), &mut out); // duplicate
dj.ingest(t, f(3), &mut out); // ancient
dj.ingest(t, f(8), &mut out); // held
dj.ingest(t, f(8), &mut out); // duplicate of the held frame
assert_eq!(seqs(&out), [5, 6]);
assert_eq!(dj.take_stats().late_drops, 3);
}
#[test]
fn wraparound_is_a_reorder_not_an_eternity() {
let mut dj = MicDejitter::new();
let t = Instant::now();
let mut out = Vec::new();
dj.ingest(t, f(u32::MAX), &mut out);
dj.ingest(t, f(0), &mut out); // in order across the wrap
assert_eq!(seqs(&out), [i64::from(u32::MAX), 0]);
dj.ingest(t, f(u32::MAX), &mut out); // pre-wrap replay: late, not a 2^31 gap
assert_eq!(dj.take_stats().late_drops, 1);
}
#[test]
fn reset_restarts_the_chain_without_concealing() {
let mut dj = MicDejitter::new();
let t = Instant::now();
let mut out = Vec::new();
dj.ingest(t, f(5), &mut out);
dj.reset_stream();
dj.ingest(t, f(500), &mut out); // a pause is not loss
assert_eq!(seqs(&out), [5, 500]);
assert_eq!(dj.take_stats().concealed, 0);
}
#[test]
fn bursty_uplink_grows_the_target_modern_uplink_shrinks_it() {
// Old Mac cadence: two frames back-to-back every ~42 ms.
let mut dj = MicDejitter::new();
let t0 = Instant::now();
let mut out = Vec::new();
let mut now = t0;
let mut seq = 0u32;
for burst in 0u64..50 {
now = t0 + Duration::from_millis(burst * 42);
for _ in 0..2 {
dj.ingest(now, f(seq), &mut out);
seq += 1;
}
}
let target = dj.target_ms(now);
assert!((42..=60).contains(&target), "bursty target {target}");
// Modern 10 ms cadence: past warmup the target follows the small gaps down.
let mut dj = MicDejitter::new();
let mut now = t0;
for i in 0u64..200 {
now = t0 + Duration::from_millis(i * 10);
dj.ingest(now, f(i as u32), &mut out);
}
let target = dj.target_ms(now);
assert!((10..=20).contains(&target), "steady target {target}");
}
#[test]
fn warmup_holds_the_crackle_safe_default() {
let mut dj = MicDejitter::new();
let t0 = Instant::now();
let mut out = Vec::new();
dj.ingest(t0, f(0), &mut out);
dj.ingest(t0 + Duration::from_millis(10), f(1), &mut out);
// 10 ms of evidence must not shrink the target yet — the client might be bursty.
assert_eq!(
dj.target_ms(t0 + Duration::from_millis(20)),
DEFAULT_TARGET_MS
);
}
}
+268 -42
View File
@@ -5,15 +5,49 @@
//! trait, its factory ([`open_virtual_mic`](super::open_virtual_mic)) and the audio-plane sample
//! rate stay in `super`.
use super::mic_jitter::{Deliver, MicDejitter};
use super::{VirtualMic, SAMPLE_RATE};
use anyhow::Result;
/// Mic is 48 kHz stereo — matches the Opus stereo decoder and the host→client audio layout.
pub const MIC_CHANNELS: u32 = 2;
/// Bound for the shared mic frame queue (drop-newest when full): the host-lifetime queue is
/// shared across all concurrent sessions and must not grow without limit under a near-line-rate
/// flood (security-review 2026-06-28 S6). 64 × 520 ms frames ≈ 0.31.3 s of slack.
const MIC_QUEUE_CAP: usize = 64;
/// One client mic frame off the wire (`0xCB` — `punktfunk_core::quic::decode_mic_datagram`).
/// The datagram's seq + pts used to be decoded at ingest and thrown away; the pump's de-jitter
/// needs them (reorder window, loss concealment, cadence/drift), so they ride the queue with
/// the Opus payload.
pub struct MicFrame {
pub seq: u32,
pub pts_ns: u64,
pub opus: Vec<u8>,
}
/// Bound for the shared mic frame queue (drop-newest at the producer when full): the
/// host-lifetime queue is shared across all concurrent sessions and must not grow without limit
/// under a near-line-rate flood (security-review 2026-06-28 S6). 12 × 520 ms frames ≈
/// 60240 ms of slack — enough for a scheduling hiccup, and the consumer heals the rest (see
/// [`DRAIN_ABOVE`]). The old cap of 64 could park 1.28 s of standing latency here that only a
/// >600 ms silence gap ever flushed.
const MIC_QUEUE_CAP: usize = 12;
/// Consumer self-healing: a pump that wakes to a backlog deeper than this drops down to the
/// newest [`DRAIN_KEEP`] frames before decoding — the oldest frames are already late, and
/// playing them would turn a one-off stall into permanent mic delay.
const DRAIN_ABOVE: usize = 6;
const DRAIN_KEEP: usize = 4;
/// Cadence of the "mic uplink health" telemetry line (emitted only while frames flow).
const TELEMETRY_EVERY: std::time::Duration = std::time::Duration::from_secs(30);
/// Creep trim: the backend ring must sit more than this over its prime target, for
/// [`TRIM_AFTER`], before the pump starts shedding — and then only near-silent frames, at most
/// one per [`TRIM_SPACING`] (a 20 ms frame per 300 ms ≈ 7 ms per 100 ms). Depth built by a
/// burst otherwise only ever comes back down at a full drain (the device consumes exactly what
/// it plays).
const TRIM_MARGIN_MS: usize = 15;
const TRIM_AFTER: std::time::Duration = std::time::Duration::from_secs(2);
const TRIM_SPACING: std::time::Duration = std::time::Duration::from_millis(300);
/// Peak below which a decoded frame counts as a silence stretch (≈ 48 dBFS).
const TRIM_SILENCE_PEAK: f32 = 0.004;
/// Tuning for [`MicPump`]'s open/reopen/flush behaviour — parameterized so the tests can run the
/// real pump loop in milliseconds instead of seconds.
@@ -56,20 +90,24 @@ const PUMP_TUNING: PumpTuning = PumpTuning {
/// every push and on an idle heartbeat, and reopened with backoff. Sessions keep their
/// senders; nothing upstream notices.
/// - **Stale-flush**: buffered audio is discarded after an uplink gap (see [`PumpTuning`]).
/// - **De-jittered**: frames pass a sequence-aware reorder window + loss concealment, and an
/// adaptive target-depth estimator drives the backend rings' priming — see
/// [`MicDejitter`](super::mic_jitter) (`PUNKTFUNK_MIC_LEGACY_BUFFER=1` restores the fixed
/// pre-adaptive buffering).
///
/// Per-frame Opus DECODE errors stay non-fatal (dropped frame): the mic is shared across every
/// concurrent session, so one paired client's junk frames must not deny everyone's mic
/// (security-review 2026-06-28 S2). The thread exits when every sender is dropped (host
/// shutdown), tearing the backend down.
pub struct MicPump {
tx: std::sync::mpsc::SyncSender<Vec<u8>>,
tx: std::sync::mpsc::SyncSender<MicFrame>,
}
impl MicPump {
/// Start the host-lifetime pump (Linux/Windows). On platforms without a virtual-mic backend
/// the thread just drains and drops frames (sessions still count the datagrams).
pub fn start() -> MicPump {
let (tx, rx) = std::sync::mpsc::sync_channel::<Vec<u8>>(MIC_QUEUE_CAP);
let (tx, rx) = std::sync::mpsc::sync_channel::<MicFrame>(MIC_QUEUE_CAP);
let spawned = std::thread::Builder::new()
.name("punktfunk-mic-pump".into())
.spawn(move || {
@@ -90,7 +128,7 @@ impl MicPump {
/// A sender a session forwards the client's Opus mic frames to (`try_send` — never block a
/// datagram loop). Cloned per session; dropping a clone does NOT stop the pump (it holds
/// the original sender for the host life).
pub fn sender(&self) -> std::sync::mpsc::SyncSender<Vec<u8>> {
pub fn sender(&self) -> std::sync::mpsc::SyncSender<MicFrame> {
self.tx.clone()
}
}
@@ -99,7 +137,7 @@ impl MicPump {
/// never accumulates a stale backlog and senders never see a wedged queue. Returns `false` when
/// every sender is gone (host shutdown).
#[cfg_attr(not(any(target_os = "linux", target_os = "windows")), allow(dead_code))]
fn drain_sleep(rx: &std::sync::mpsc::Receiver<Vec<u8>>, dur: std::time::Duration) -> bool {
fn drain_sleep(rx: &std::sync::mpsc::Receiver<MicFrame>, dur: std::time::Duration) -> bool {
use std::sync::mpsc::RecvTimeoutError;
let deadline = std::time::Instant::now() + dur;
loop {
@@ -118,7 +156,7 @@ fn drain_sleep(rx: &std::sync::mpsc::Receiver<Vec<u8>>, dur: std::time::Duration
/// The pump loop. `opener` is injected so the tests can run the REAL loop against a mock
/// backend; production passes [`open_virtual_mic`](super::open_virtual_mic).
#[cfg_attr(not(any(target_os = "linux", target_os = "windows")), allow(dead_code))]
fn pump_thread<O>(rx: std::sync::mpsc::Receiver<Vec<u8>>, opener: O, tuning: PumpTuning)
fn pump_thread<O>(rx: std::sync::mpsc::Receiver<MicFrame>, opener: O, tuning: PumpTuning)
where
O: Fn() -> Result<Box<dyn VirtualMic>>,
{
@@ -159,40 +197,65 @@ where
let opened_at = Instant::now();
// Pump phase — runs until the backend dies (break) or the host shuts down (return).
let legacy = super::mic_legacy_buffer();
let mut decode_fails: u64 = 0;
let mut drain_drops: u64 = 0;
let mut trimmed: u64 = 0;
let mut frames_seen: u64 = 0;
let mut pcm = vec![0f32; 5760 * MIC_CHANNELS as usize]; // up to 120 ms scratch
let mut last_push = Instant::now();
loop {
match rx.recv_timeout(tuning.heartbeat) {
Ok(frame) => {
if frame.is_empty() {
continue; // DTX silence — the source underruns to silence on its own
let mut batch: Vec<MicFrame> = Vec::new();
let mut deliveries: Vec<Deliver> = Vec::new();
let mut jitter = MicDejitter::new();
// Concealment must match the last real frame's duration — libopus sizes PLC from the
// output slice. One 20 ms frame until something decodes.
let mut plc_samples: usize = 960;
let mut applied_target_ms: u32 = 0;
let mut over_since: Option<Instant> = None;
let mut last_trim = Instant::now();
let mut last_log = Instant::now();
'pump: loop {
// Wake for the next frame, the heartbeat, or a parked reorder hold aging out —
// whichever is soonest.
let timeout = jitter
.hold_deadline()
.map(|d| d.saturating_duration_since(Instant::now()))
.unwrap_or(tuning.heartbeat)
.min(tuning.heartbeat);
deliveries.clear();
match rx.recv_timeout(timeout) {
Ok(first) => {
// Take everything already queued in one gulp: normally that's just `first`,
// but after a stall the backlog IS standing mic latency — heal by jumping
// to the newest few frames instead of replaying the pile.
batch.clear();
batch.push(first);
while batch.len() <= MIC_QUEUE_CAP {
match rx.try_recv() {
Ok(f) => batch.push(f),
Err(_) => break,
}
}
if batch.len() > DRAIN_ABOVE {
let drop_n = batch.len() - DRAIN_KEEP;
drain_drops += drop_n as u64;
batch.drain(..drop_n);
}
frames_seen += batch.len() as u64;
if last_push.elapsed() > tuning.stale_gap {
mic.discard();
jitter.reset_stream();
}
match decoder.decode_float(&frame, &mut pcm, false) {
Ok(samples_per_ch) => {
let total = (samples_per_ch * MIC_CHANNELS as usize).min(pcm.len());
if !mic.push(&pcm[..total]) {
tracing::warn!("virtual mic backend died — reopening");
break;
}
last_push = Instant::now();
decode_fails = 0;
}
Err(e) => {
// Malformed/garbage frame: drop it, keep the shared mic + decoder
// (see the struct docs). Throttled log (1, 2, 4, … fails).
decode_fails += 1;
if decode_fails.is_power_of_two() {
tracing::warn!(error = %e, fails = decode_fails,
"mic opus decode failed — dropping frame");
}
}
let now = Instant::now();
for frame in batch.drain(..) {
jitter.ingest(now, frame, &mut deliveries);
}
// A hold whose window expired while traffic kept flowing (only late
// duplicates arriving) must still flush — the timeout arm never fires then.
jitter.flush_expired_hold(now, &mut deliveries);
}
Err(RecvTimeoutError::Timeout) => {
jitter.flush_expired_hold(Instant::now(), &mut deliveries);
if !mic.alive() {
tracing::warn!("virtual mic backend died while idle — reopening");
break;
@@ -203,6 +266,123 @@ where
return;
}
}
for d in deliveries.drain(..) {
let samples_per_ch = match d {
Deliver::Frame(frame) => {
if frame.opus.is_empty() {
continue; // DTX silence — the source underruns to silence on its own
}
match decoder.decode_float(&frame.opus, &mut pcm, false) {
Ok(n) => {
plc_samples = n.max(120); // ≥ 2.5 ms keeps PLC well-formed
decode_fails = 0;
n
}
Err(e) => {
// Malformed/garbage frame: drop it, keep the shared mic +
// decoder (see the struct docs). Throttled log (1, 2, 4, …).
decode_fails += 1;
if decode_fails.is_power_of_two() {
tracing::warn!(error = %e, fails = decode_fails,
"mic opus decode failed — dropping frame");
}
continue;
}
}
}
Deliver::Conceal => {
// A lost datagram: an empty-input decode synthesizes libopus PLC for
// exactly the slice's duration, so the backend ring doesn't starve
// into a silence + re-prime cycle over one missing frame.
let want = (plc_samples * MIC_CHANNELS as usize).min(pcm.len());
match decoder.decode_float(&[], &mut pcm[..want], false) {
Ok(n) => n,
Err(_) => continue, // nothing decoded yet — nothing to extend
}
}
};
let total = (samples_per_ch * MIC_CHANNELS as usize).min(pcm.len());
// Creep trim: ring depth persistently over target sheds a near-silent frame at
// a time (a few ms per 100 ms) — never speech, never a hard clear. Without it,
// depth built by a burst only ever comes back down at a full drain.
let mut shed = false;
if !legacy {
match mic.depth() {
Some((buffered, target))
if buffered > target + TRIM_MARGIN_MS * SAMPLE_RATE as usize / 1000 =>
{
let since = *over_since.get_or_insert_with(Instant::now);
if since.elapsed() >= TRIM_AFTER
&& last_trim.elapsed() >= TRIM_SPACING
&& pcm[..total].iter().all(|s| s.abs() < TRIM_SILENCE_PEAK)
{
shed = true;
last_trim = Instant::now();
trimmed += 1;
}
}
_ => over_since = None,
}
}
if shed {
last_push = Instant::now(); // the uplink is live — a trim is not a stale gap
continue;
}
if !mic.push(&pcm[..total]) {
tracing::warn!("virtual mic backend died — reopening");
break 'pump;
}
last_push = Instant::now();
}
// Drive the backend ring's prime target from the measured jitter. Legacy mode
// never calls this, which keeps the backend on its fixed constants (see
// `VirtualMic::set_target_depth`).
if !legacy {
let t = jitter.target_ms(Instant::now());
if t != applied_target_ms {
applied_target_ms = t;
mic.set_target_depth(t as usize * SAMPLE_RATE as usize / 1000);
}
}
// One structured health line per interval while the mic is live (reset-on-read).
if last_log.elapsed() >= TELEMETRY_EVERY {
if frames_seen > 0 {
let js = jitter.take_stats();
let bs = mic.take_stats();
let (depth_ms, target_ms) = mic
.depth()
.map(|(d, t)| {
(
d * 1000 / SAMPLE_RATE as usize,
t * 1000 / SAMPLE_RATE as usize,
)
})
.unwrap_or((0, 0));
tracing::info!(
depth_ms,
target_ms,
cadence_ms = (js.cadence_ms * 10.0).round() / 10.0,
frame_ms = (js.frame_ms * 10.0).round() / 10.0,
frames = frames_seen,
gaps = js.seq_gaps,
concealed = js.concealed,
reorders = js.reorders,
late = js.late_drops,
drained = drain_drops,
trimmed,
reprimes = bs.reprimes,
overflow_ms = bs.overflow_dropped * 1000 / SAMPLE_RATE as u64,
"mic uplink health"
);
}
last_log = Instant::now();
frames_seen = 0;
drain_drops = 0;
trimmed = 0;
}
}
// Death triage: an instance that lived is a one-off (PipeWire/audio-engine restart) —
@@ -252,7 +432,7 @@ mod pump_tests {
}
struct Harness {
tx: std::sync::mpsc::SyncSender<Vec<u8>>,
tx: std::sync::mpsc::SyncSender<MicFrame>,
opens: Arc<AtomicUsize>,
alive: Arc<Mutex<Option<Arc<AtomicBool>>>>, // latest instance's kill switch
pushed: Arc<AtomicUsize>,
@@ -265,7 +445,7 @@ mod pump_tests {
/// pre-killed (exercises the rapid-death churn guard). `stable_after` mirrors the tuning
/// field (ZERO = every death counts as stable → immediate reopen, keeping tests fast).
fn start_tuned(fail_first: usize, dead_on_arrival: bool, stable_after: Duration) -> Harness {
let (tx, rx) = std::sync::mpsc::sync_channel::<Vec<u8>>(MIC_QUEUE_CAP);
let (tx, rx) = std::sync::mpsc::sync_channel::<MicFrame>(MIC_QUEUE_CAP);
let opens = Arc::new(AtomicUsize::new(0));
let alive = Arc::new(Mutex::new(None::<Arc<AtomicBool>>));
let pushed = Arc::new(AtomicUsize::new(0));
@@ -317,7 +497,7 @@ mod pump_tests {
}
fn wait_until(what: &str, mut cond: impl FnMut() -> bool) {
for _ in 0..200 {
for _ in 0..600 {
if cond() {
return;
}
@@ -326,6 +506,27 @@ mod pump_tests {
panic!("timed out waiting for: {what}");
}
/// Keep a live uplink running until audio reaches the backend.
///
/// A single frame is NOT enough to prove a reopen recovered: a freshly opened backend
/// deliberately drops whatever queued while it was down (the `try_recv` drain after the
/// open — that audio predates the device), and `opens` counts from the START of the open,
/// so a frame sent the moment the counter moves lands inside that drain and is discarded
/// by design. A real uplink keeps sending, so the test does too. The sequence has to keep
/// advancing or the de-jitter reads the repeats as duplicates and drops them.
fn wait_until_pushed(what: &str, h: &Harness, from_seq: u32) {
let mut seq = from_seq;
for _ in 0..600 {
let _ = h.tx.try_send(mic_frame(seq));
seq = seq.wrapping_add(1);
if h.pushed.load(Ordering::SeqCst) > 0 {
return;
}
std::thread::sleep(Duration::from_millis(10));
}
panic!("timed out waiting for: {what}");
}
fn opus_frame() -> Vec<u8> {
let mut enc = opus::Encoder::new(48_000, opus::Channels::Stereo, opus::Application::Voip)
.expect("opus encoder");
@@ -336,6 +537,15 @@ mod pump_tests {
out
}
/// A wire-shaped frame: `seq` with the matching 20 ms pts, payload from [`opus_frame`].
fn mic_frame(seq: u32) -> MicFrame {
MicFrame {
seq,
pts_ns: seq as u64 * 20_000_000,
opus: opus_frame(),
}
}
/// Eager: the backend opens (after transient failures) with NO frame ever sent.
#[test]
fn opens_eagerly_with_backoff() {
@@ -352,7 +562,7 @@ mod pump_tests {
fn decodes_and_pushes() {
let h = start(0);
wait_until("open", || h.alive.lock().unwrap().is_some());
h.tx.send(opus_frame()).unwrap();
h.tx.send(mic_frame(0)).unwrap();
wait_until("pcm pushed", || h.pushed.load(Ordering::SeqCst) > 0);
drop(h.tx);
h.join.join().unwrap();
@@ -388,10 +598,9 @@ mod pump_tests {
.as_ref()
.unwrap()
.store(false, Ordering::Release);
h.tx.send(opus_frame()).unwrap(); // push sees death → reopen
h.tx.send(mic_frame(0)).unwrap(); // push sees death → reopen
wait_until("reopen", || h.opens.load(Ordering::SeqCst) >= 2);
h.tx.send(opus_frame()).unwrap();
wait_until("pcm after reopen", || h.pushed.load(Ordering::SeqCst) > 0);
wait_until_pushed("pcm after reopen", &h, 1);
drop(h.tx);
h.join.join().unwrap();
}
@@ -416,15 +625,32 @@ mod pump_tests {
h.join.join().unwrap();
}
/// A lost datagram is concealed: seq 0 then 2 (1 missing) must still push ~3 frames of PCM
/// (decode, PLC, decode) once the reorder window expires — the backend ring never starves
/// over a single missing frame.
#[test]
fn seq_gap_is_concealed() {
let h = start(0);
wait_until("instance", || h.alive.lock().unwrap().is_some());
h.tx.send(mic_frame(0)).unwrap();
h.tx.send(mic_frame(2)).unwrap();
// 0 plays immediately; 2 is held ≤ 30 ms for 1, then the gap is concealed and 2 plays.
wait_until("conceal + late frame pushed", || {
h.pushed.load(Ordering::SeqCst) >= 3 * 960 * 2
});
drop(h.tx);
h.join.join().unwrap();
}
/// An uplink gap discards buffered-stale audio before the next frame plays.
#[test]
fn discards_after_gap() {
let h = start(0);
wait_until("instance", || h.alive.lock().unwrap().is_some());
h.tx.send(opus_frame()).unwrap();
h.tx.send(mic_frame(0)).unwrap();
wait_until("first push", || h.pushed.load(Ordering::SeqCst) > 0);
std::thread::sleep(Duration::from_millis(150)); // > stale_gap
h.tx.send(opus_frame()).unwrap();
h.tx.send(mic_frame(1)).unwrap();
wait_until("discard on gap", || h.discards.load(Ordering::SeqCst) >= 1);
drop(h.tx);
h.join.join().unwrap();
@@ -39,7 +39,7 @@
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use super::wiring_plan::{plan, Endpoint, Wiring};
use super::wiring_plan::{self, plan, Endpoint, Wiring};
use anyhow::{anyhow, bail, Result};
use std::ffi::c_void;
use std::sync::Mutex;
@@ -75,6 +75,33 @@ pub(crate) fn host_audio_requested() -> bool {
std::env::var_os("PUNKTFUNK_HOST_AUDIO").is_some()
}
/// One wiring pass plus the inputs the desktop-audio capture loop's failure handling needs:
/// the endpoint-set fingerprint ([`wiring_plan::fingerprint`] — the wait key while a plan is
/// unsatisfiable, snapshotted from the SAME enumeration the plan consumed so a device arriving
/// mid-pass can't leave the waiter keyed to a set the plan never saw) and the render inventory
/// (the one-shot "why is there no loopback" diagnosis).
pub(crate) struct WiredPlan {
pub wiring: Wiring,
pub fingerprint: u64,
pub renders: Vec<Endpoint>,
}
/// Fingerprint of the CURRENT endpoint set (both directions) WITHOUT a wiring pass: enumeration
/// and a hash — no plan, no default-device writes, no logs. This is the cheap poll the capture
/// loop runs while waiting out a failure; the full [`wire_now`] only runs again once this moves.
/// Must run on a COM-initialized thread, like [`wire_now`].
pub(crate) fn endpoint_fingerprint() -> u64 {
wiring_plan::fingerprint(
&list_endpoints(Direction::Render),
&list_endpoints(Direction::Capture),
)
}
/// [`wire_now_full`] for callers that only need the assignment (the mic paths).
pub(crate) fn wire_now(set_playback: bool) -> Wiring {
wire_now_full(set_playback).wiring
}
/// Enumerate endpoints, compute the assignment, apply the default-device changes (unless
/// `PUNKTFUNK_KEEP_DEFAULT`), and return the plan for the caller to act on (mic target / loopback
/// echo guard). `set_playback` — true only from the desktop-audio capture open — additionally
@@ -83,14 +110,20 @@ pub(crate) fn host_audio_requested() -> bool {
/// Must run on a COM-initialized thread (the WASAPI worker threads all `initialize_mta` first).
/// Logged only when the assignment changes, so per-open recomputation stays quiet in the steady
/// state.
pub(crate) fn wire_now(set_playback: bool) -> Wiring {
pub(crate) fn wire_now_full(set_playback: bool) -> WiredPlan {
recover_orphaned_default();
let renders = list_endpoints(Direction::Render);
let captures = list_endpoints(Direction::Capture);
let fingerprint = wiring_plan::fingerprint(&renders, &captures);
let want = std::env::var("PUNKTFUNK_MIC_DEVICE")
.ok()
.map(|s| s.to_lowercase());
let wiring = plan(&renders, &captures, want.as_deref(), host_audio_requested());
let done = |wiring: Wiring| WiredPlan {
wiring,
fingerprint,
renders: renders.clone(),
};
// Log assignment changes exactly once (first plan included).
static LAST: Mutex<Option<Wiring>> = Mutex::new(None);
@@ -105,14 +138,17 @@ pub(crate) fn wire_now(set_playback: bool) -> Wiring {
mic_render = wiring.mic_render.as_ref().map(|(n, _)| n.as_str()),
mic_capture = wiring.mic_capture.as_ref().map(|(n, _)| n.as_str()),
loopback_render = wiring.loopback_render.as_ref().map(|(n, _)| n.as_str()),
loopback_last_resort = wiring.loopback_last_resort,
renders = ?renders.iter().map(|(n, _)| n.as_str()).collect::<Vec<_>>(),
"audio wiring plan"
);
if wiring.mic_render.is_some() && wiring.loopback_render.is_none() {
if wiring.mic_render.is_some() && wiring.loopback_unsatisfiable() {
// Inventory + per-endpoint reasons + ONLY the remedies not already taken — the old
// static advice here suggested installing the Steam pair to a field box that had it
// installed (its Microphone half was exactly what the mic had reserved).
tracing::warn!(
"the virtual mic reserved the only usable render endpoint — desktop audio will be \
unavailable until another output device exists (attach one, or let the host \
install the Steam Streaming pair)"
"desktop audio unavailable: {}",
wiring_plan::describe_no_loopback(&renders, &wiring)
);
}
}
@@ -123,7 +159,7 @@ pub(crate) fn wire_now(set_playback: bool) -> Wiring {
"PUNKTFUNK_KEEP_DEFAULT set — leaving the audio default devices untouched"
);
}
return wiring;
return done(wiring);
}
// Default-playback hygiene, on EVERY wire (mic pump at boot included): if the default render
// endpoint IS the mic target — VB-CABLE installs have been seen grabbing the default — every
@@ -160,18 +196,26 @@ pub(crate) fn wire_now(set_playback: bool) -> Wiring {
}
}
if let Some((name, id)) = &wiring.mic_capture {
match set_default_endpoint(id) {
Ok(()) => {
if changed {
tracing::info!(device = %name,
"audio wiring: default recording = virtual mic (apps record the client's mic)");
// `set_default_endpoint` is NOT a no-op on an unchanged default: it unconditionally
// fires SetDefaultEndpoint for all three roles (an audio-policy write plus a
// device-graph notification, each). Re-asserting on every wiring pass therefore both
// churned the policy store AND silently stomped an operator's own recording-device
// choice within one reopen cycle — write only when the plan changed or the default
// actually drifted off the target.
if changed || default_capture_id().as_deref() != Some(id.as_str()) {
match set_default_endpoint(id) {
Ok(()) => {
if changed {
tracing::info!(device = %name,
"audio wiring: default recording = virtual mic (apps record the client's mic)");
}
}
Err(e) => tracing::warn!(device = %name, error = %format!("{e:#}"),
"audio wiring: failed to set the default recording device"),
}
Err(e) => tracing::warn!(device = %name, error = %format!("{e:#}"),
"audio wiring: failed to set the default recording device"),
}
}
wiring
done(wiring)
}
/// The operator's default playback endpoint while we have it parked on the loopback sink:
@@ -194,6 +238,18 @@ fn default_render_id() -> Option<String> {
.ok()
}
/// The current default CAPTURE endpoint id, if any — the recording-side analogue of
/// [`default_render_id`], read before asserting the recording default so an already-correct
/// default costs zero IPolicyConfig writes.
fn default_capture_id() -> Option<String> {
wasapi::DeviceEnumerator::new()
.ok()?
.get_default_device(&Direction::Capture)
.ok()?
.get_id()
.ok()
}
/// Once per process: if a crash marker from a previous run exists, the host died while the
/// playback default was parked — put the operator's device back, but only if the default still
/// IS the endpoint we set (a manual change since the crash wins). Runs on the first wiring pass
@@ -18,9 +18,14 @@
//! device changing under us — the operator picked a different output mid-stream — and reacts:
//! a loopback-capturable choice is FOLLOWED (their explicit choice wins; audio then also plays
//! on the host), a known-dud choice (cable/Steam Speakers/the mic target) snaps back to the
//! plan. Device errors (endpoint invalidated, engine restart) reopen with backoff instead of
//! killing audio for the rest of the session. On thread exit (capturer dropped at stream end)
//! the parked default playback device is restored.
//! plan. Device errors (endpoint invalidated, engine restart) reopen with a capped exponential
//! backoff that an endpoint-set change cuts short. A plan with NO loopback endpoint at all is
//! never retried: `wiring_plan::plan` is pure in the endpoint set, so that verdict holds until
//! the set changes — the thread says why once, then parks on a cheap fingerprint poll and
//! re-plans the instant the set moves (the 2026-08 field case hammered a full wiring pass —
//! IPolicyConfig writes included — every 2 s for 8+ minutes without ever being able to
//! succeed). On thread exit (capturer dropped at stream end) the parked default playback
//! device is restored.
use super::{audio_control, wiring_plan, AudioCapturer, SAMPLE_RATE};
use anyhow::{anyhow, Context, Result};
@@ -131,8 +136,20 @@ enum Next {
Reopen(TargetMode),
}
/// Backoff between self-heal reopen attempts after a capture failure.
const REOPEN_BACKOFF: Duration = Duration::from_secs(2);
/// Reopen backoff after a TRANSIENT capture failure: starts here and doubles per consecutive
/// failure up to [`REOPEN_BACKOFF_CAP`], resetting on success or on an endpoint-set change
/// (mirrors the mic pump's `PUMP_TUNING` shape). The predecessor was a FLAT 2 s retry whose
/// every attempt re-ran the full wiring pass, IPolicyConfig writes included — tolerable for a
/// genuinely transient error, an 8-minute hammer in the 2026-08 field case where the failure
/// was structural.
const REOPEN_BACKOFF_START: Duration = Duration::from_secs(2);
const REOPEN_BACKOFF_CAP: Duration = Duration::from_secs(60);
/// Endpoint-set poll cadence while waiting out a failure (both the transient backoff sleep and
/// the unsatisfiable-plan wait): one enumerate-and-hash per tick, nothing else. A fingerprint
/// change ends the wait immediately — a (re)arrived endpoint (the display coming back, plugged
/// headphones) is exactly the recovery moment — so recovery stays as fast as the old 2 s hammer
/// without its side effects.
const ENDPOINT_POLL_EVERY: Duration = Duration::from_secs(2);
/// Watchdog cadence for "did the default render device change under us?" checks.
const DEFAULT_CHECK_EVERY: Duration = Duration::from_secs(1);
/// Total attempts for the FIRST open before its failure surfaces through the `ready` handshake.
@@ -168,14 +185,30 @@ fn capture_thread(
let mut mode = TargetMode::Assert;
let mut failures: u64 = 0;
let mut first_attempts: u32 = 0;
let mut backoff = REOPEN_BACKOFF_START;
// Endpoint-set fingerprint under which an unsatisfiable plan was already error-logged: an
// unchanged set means an unchanged verdict (`wiring_plan::plan` is pure), so the diagnosis
// is said once per topology — the field log drowned in 256+ copies of the same line.
let mut unsat_logged: Option<u64> = None;
while !stop.load(Ordering::Relaxed) {
match capture_once(&tx, &stop, &mut ready, channels, mode) {
Ok(Next::Stopped) => break,
Ok(Next::Reopen(m)) => {
mode = m;
failures = 0;
backoff = REOPEN_BACKOFF_START;
unsat_logged = None;
}
Err(e) if ready.is_some() => {
// An unsatisfiable PLAN cannot improve within the handshake window — the
// once-per-process Steam-pair install already ran inside `capture_once` — so
// fail the open now with the full diagnosis instead of spending the transient
// retry budget on a structural verdict. The native plane owns first-open
// retries and backs off on its own.
if e.downcast_ref::<PlanUnsatisfiable>().is_some() {
let _ = ready.take().unwrap().send(Err(anyhow!("{e:#}")));
break;
}
first_attempts += 1;
if first_attempts >= FIRST_OPEN_ATTEMPTS || stop.load(Ordering::Relaxed) {
let _ = ready.take().unwrap().send(Err(anyhow!("{e:#}")));
@@ -190,16 +223,43 @@ fn capture_thread(
}
}
Err(e) => {
failures += 1;
if failures.is_power_of_two() {
tracing::warn!(error = %format!("{e:#}"), count = failures,
"audio loopback capture failed — reopening");
}
mode = TargetMode::Assert;
// Backoff in stop-responsive slices.
let until = Instant::now() + REOPEN_BACKOFF;
while Instant::now() < until && !stop.load(Ordering::Relaxed) {
thread::sleep(Duration::from_millis(100));
if let Some(unsat) = e.downcast_ref::<PlanUnsatisfiable>() {
// Structural: retrying against the same endpoints repeats the same verdict,
// and every retry used to re-run the wiring pass — IPolicyConfig writes
// included, stomping any operator default-recording change within 2 s. Say
// why once per topology, then park on the cheap fingerprint poll; the set
// changing IS the recovery moment and re-plans immediately.
failures = 0;
backoff = REOPEN_BACKOFF_START;
if unsat_logged != Some(unsat.fingerprint) {
unsat_logged = Some(unsat.fingerprint);
tracing::error!(
"desktop audio unavailable, and retrying cannot help until the \
audio endpoint set changes waiting for that change. {unsat}"
);
}
if wait_endpoint_change(&stop, unsat.fingerprint, None) == EndpointWait::Stopped
{
break;
}
} else {
unsat_logged = None;
failures += 1;
if failures.is_power_of_two() {
tracing::warn!(error = %format!("{e:#}"), count = failures,
backoff_secs = backoff.as_secs(),
"audio loopback capture failed — reopening after backoff");
}
// Capped exponential backoff, cut short (and reset) the moment the
// endpoint set changes — a re-arrived device is the likeliest cure for
// whatever killed the capture, and it must not wait out a 60 s sleep.
let fp = audio_control::endpoint_fingerprint();
match wait_endpoint_change(&stop, fp, Some(Instant::now() + backoff)) {
EndpointWait::Stopped => break,
EndpointWait::Changed => backoff = REOPEN_BACKOFF_START,
EndpointWait::Elapsed => backoff = (backoff * 2).min(REOPEN_BACKOFF_CAP),
}
}
}
}
@@ -210,6 +270,75 @@ fn capture_thread(
Ok(())
}
/// A wiring plan with NO loopback endpoint, as a typed error: [`wiring_plan::plan`] is pure in
/// the enumerated endpoint set, so unlike every other capture error this one is PERMANENT until
/// the topology changes — retrying it is guaranteed futile (the 2026-08 field case retried it
/// flat-out for 8+ minutes, one full wiring pass per retry). Carries the set's fingerprint
/// (what the reopen loop waits on) and the full diagnosis: inventory, per-endpoint rejection
/// reasons, and only the remedies not already taken.
#[derive(Debug)]
struct PlanUnsatisfiable {
fingerprint: u64,
detail: String,
}
impl PlanUnsatisfiable {
fn from_plan(plan: &audio_control::WiredPlan) -> PlanUnsatisfiable {
debug_assert!(plan.wiring.loopback_unsatisfiable());
PlanUnsatisfiable {
fingerprint: plan.fingerprint,
detail: wiring_plan::describe_no_loopback(&plan.renders, &plan.wiring),
}
}
}
impl std::fmt::Display for PlanUnsatisfiable {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.detail)
}
}
impl std::error::Error for PlanUnsatisfiable {}
/// How a [`wait_endpoint_change`] ended.
#[derive(Clone, Copy, PartialEq, Eq)]
enum EndpointWait {
/// `stop` was set — the capturer is being dropped.
Stopped,
/// The endpoint-set fingerprint moved — re-plan NOW (this is the recovery moment).
Changed,
/// The deadline passed without a change (backoff waits only; `deadline: None` never ends
/// this way).
Elapsed,
}
/// Stop-responsive wait that polls the endpoint-set fingerprint every [`ENDPOINT_POLL_EVERY`] —
/// an enumerate-and-hash, no wiring pass, no IPolicyConfig writes, no logs — until the set
/// changes, `deadline` passes, or `stop` is set. `deadline: None` waits indefinitely: used while
/// the plan is unsatisfiable, where ONLY a topology change can alter the verdict.
fn wait_endpoint_change(
stop: &AtomicBool,
fingerprint: u64,
deadline: Option<Instant>,
) -> EndpointWait {
let mut next_poll = Instant::now() + ENDPOINT_POLL_EVERY;
loop {
if stop.load(Ordering::Relaxed) {
return EndpointWait::Stopped;
}
if deadline.is_some_and(|d| Instant::now() >= d) {
return EndpointWait::Elapsed;
}
thread::sleep(Duration::from_millis(100));
if Instant::now() >= next_poll {
next_poll = Instant::now() + ENDPOINT_POLL_EVERY;
if audio_control::endpoint_fingerprint() != fingerprint {
return EndpointWait::Changed;
}
}
}
}
/// The current default render endpoint, with its id (`None` on any enumeration failure —
/// transient failures must not kill the capture).
fn default_render(en: &DeviceEnumerator) -> Option<(Device, String)> {
@@ -220,7 +349,7 @@ fn default_render(en: &DeviceEnumerator) -> Option<(Device, String)> {
/// One endpoint open + capture loop. Returns how to continue ([`Next`]) or an error (first open:
/// retried [`FIRST_OPEN_ATTEMPTS`] times, then fatal via the `ready` handshake; later: reopen
/// with backoff).
/// with capped backoff — or, for a typed [`PlanUnsatisfiable`], an endpoint-set wait).
fn capture_once(
tx: &SyncSender<Vec<f32>>,
stop: &AtomicBool,
@@ -233,26 +362,23 @@ fn capture_once(
let keep_default = std::env::var_os("PUNKTFUNK_KEEP_DEFAULT").is_some();
// Assert-mode without KEEP_DEFAULT is the only shape that parks the playback default.
let assert_plan = mode == TargetMode::Assert && !keep_default;
let mut wiring = audio_control::wire_now(assert_plan);
let mut plan = audio_control::wire_now_full(assert_plan);
// Client-only audio needs a silent-on-host sink with a working loopback (the Steam Streaming
// Microphone's render side). If the plan had to settle for real hardware (or nothing), try —
// once per process — to install the Steam pair (present when Steam is), then re-plan.
if assert_plan && !audio_control::host_audio_requested() {
let have_silent = wiring
.loopback_render
.as_ref()
.is_some_and(|(n, _)| wiring_plan::silent_sink(&n.to_lowercase()));
static INSTALL_TRIED: AtomicBool = AtomicBool::new(false);
if !have_silent && !INSTALL_TRIED.swap(true, Ordering::SeqCst) {
if super::wasapi_mic::install_steam_audio_pair() {
wiring = audio_control::wire_now(true);
}
if !wiring
.loopback_render
let have_silent = |w: &wiring_plan::Wiring| {
w.loopback_render
.as_ref()
.is_some_and(|(n, _)| wiring_plan::silent_sink(&n.to_lowercase()))
{
};
static INSTALL_TRIED: AtomicBool = AtomicBool::new(false);
if !have_silent(&plan.wiring) && !INSTALL_TRIED.swap(true, Ordering::SeqCst) {
if super::wasapi_mic::install_steam_audio_pair() {
plan = audio_control::wire_now_full(true);
}
if !have_silent(&plan.wiring) {
tracing::info!(
"no silent virtual sink for client-only audio — desktop audio will also play \
on the host (install Steam, whose Remote Play streaming drivers provide one)"
@@ -260,6 +386,12 @@ fn capture_once(
}
}
}
let wiring = &plan.wiring;
// Only the Assert path can knowingly sit on the plan's LAST-RESORT endpoint: Follow captures
// the operator's chosen default, and `judge_default` never routes Follow onto the Steam
// Speakers (they are `excluded_from_loopback` — a Dud that snaps back to the plan).
let last_resort = assert_plan && wiring.loopback_last_resort;
let plan_fp = plan.fingerprint;
let en = DeviceEnumerator::new().context("DeviceEnumerator")?;
// Resolve the endpoint to capture. ECHO GUARD (Follow/KEEP_DEFAULT shapes): the wiring plan
@@ -268,11 +400,11 @@ fn capture_once(
// fall back to the plan's loopback endpoint, or refuse — no desktop audio beats an echo loop.
let (device, dev_name, dev_id) = if assert_plan {
let Some(ep) = wiring.loopback_render.clone() else {
anyhow::bail!(
"no loopback-capturable render endpoint (every usable endpoint is reserved for \
the virtual mic or has a silent loopback) attach an output device or install \
the Steam Streaming pair to get desktop audio"
);
// Detected BEFORE any open attempt, and typed: the plan is a pure function of the
// endpoint set, so this cannot resolve until the set changes — the reopen loop
// waits on the fingerprint instead of retrying (the old untyped bail was retried
// flat-out every 2 s, forever, in the 2026-08 field case).
return Err(PlanUnsatisfiable::from_plan(&plan).into());
};
let d = audio_control::open_endpoint(&ep)?;
(d, ep.0, ep.1)
@@ -285,10 +417,15 @@ fn capture_once(
.is_some_and(|(_, mic_id)| *mic_id == id);
if default_is_mic {
let Some(lb) = wiring.loopback_render.clone() else {
// Same inventory shape as the Assert bail, but NOT typed as unsatisfiable:
// Follow's inputs include the DEFAULT device, which the operator can change
// without a topology change (especially under PUNKTFUNK_KEEP_DEFAULT) — the
// capped backoff must keep retrying rather than a fingerprint wait sleeping
// through a default-only change.
anyhow::bail!(
"the only render endpoint is reserved for the virtual mic (capturing it would \
echo the client's voice back) attach another output device or install the \
Steam Streaming pair to get desktop audio"
"the default render endpoint is reserved for the virtual mic (capturing it \
would echo the client's voice back) {}",
wiring_plan::describe_no_loopback(&plan.renders, wiring)
);
};
tracing::warn!(mic = %wiring.mic_render.as_ref().unwrap().0, loopback = %lb.0,
@@ -338,6 +475,7 @@ fn capture_once(
}
tracing::info!(device = %dev_name,
follow = matches!(mode, TargetMode::Follow) || keep_default,
last_resort,
"audio loopback capturing");
// Watchdog seed: the default as it stands right after our open. In Assert mode the plan just
@@ -349,7 +487,7 @@ fn capture_once(
if assert_plan {
if let Some(d) = seen_default.as_deref() {
if d != dev_id {
match judge_default(&en, &wiring, d) {
match judge_default(&en, wiring, d) {
DefaultKind::Capturable(name) => {
tracing::info!(default = %name, planned = %dev_name,
"could not park the default playback on the planned endpoint — \
@@ -367,10 +505,12 @@ fn capture_once(
let mut bytes: VecDeque<u8> = VecDeque::new();
let mut last_check = Instant::now();
let mut last_fp_check = Instant::now();
// Triage breadcrumb: a broken loopback (endpoint renders but its loopback tap delivers
// nothing — the Steam Streaming Speakers failure shape) is indistinguishable from a simply
// quiet desktop, so after 30 s with zero packets say so ONCE. Info, not warn: an idle host
// is legitimately silent.
// quiet desktop, so after 30 s with zero packets say so ONCE. Info, not warn an idle host
// is legitimately silent — EXCEPT on a last-resort endpoint, where the plan already knew
// the loopback is silent and zero packets all but confirms the quality risk materialized.
let opened_at = Instant::now();
let mut saw_packets = false;
let mut silence_noted = false;
@@ -396,10 +536,18 @@ fn capture_once(
}
if !saw_packets && !silence_noted && opened_at.elapsed() >= Duration::from_secs(30) {
silence_noted = true;
tracing::info!(device = %dev_name,
"no audio captured in the first 30 s — fine if the host is quiet; if it should \
be playing audio, this endpoint's loopback may be broken (set \
PUNKTFUNK_HOST_AUDIO=1 to prefer real hardware)");
if last_resort {
tracing::warn!(device = %dev_name,
"no audio captured in the first 30 s from the LAST-RESORT loopback — the \
Steam Streaming Speakers' loopback is known-silent, so desktop audio is \
most likely not reaching the client; attach any output device to give the \
plan a working endpoint (it re-plans on the change)");
} else {
tracing::info!(device = %dev_name,
"no audio captured in the first 30 s — fine if the host is quiet; if it \
should be playing audio, this endpoint's loopback may be broken (set \
PUNKTFUNK_HOST_AUDIO=1 to prefer real hardware)");
}
}
let whole = (bytes.len() / block_align) * block_align;
if whole > 0 {
@@ -428,7 +576,7 @@ fn capture_once(
);
return Ok(Next::Reopen(TargetMode::Follow));
}
return Ok(match judge_default(&en, &wiring, &nid) {
return Ok(match judge_default(&en, wiring, &nid) {
DefaultKind::Capturable(name) => {
tracing::info!(device = %name,
"operator changed the output device mid-stream — following \
@@ -447,6 +595,24 @@ fn capture_once(
}
}
}
// A LAST-RESORT capture is a stopgap, not a steady state: the plan chose the
// known-silent Steam Speakers only because nothing better existed, so any endpoint-set
// change — the display's audio endpoint re-arriving, headphones plugged in — may unlock
// a real plan. Re-plan on the change; without this the session would ride the silent
// loopback forever AFTER the real endpoint returned (the original field defect in a
// quieter costume). Preferred endpoints don't get this watch: mid-stream re-routing
// there is the default-device watchdog's job, on the operator's terms.
if last_resort && last_fp_check.elapsed() >= ENDPOINT_POLL_EVERY {
last_fp_check = Instant::now();
if audio_control::endpoint_fingerprint() != plan_fp {
audio_client.stop_stream().ok();
tracing::info!(
"endpoint set changed while capturing the last-resort loopback — re-planning"
);
return Ok(Next::Reopen(TargetMode::Assert));
}
}
}
}
@@ -19,19 +19,21 @@
//! returns `false` and the pump reopens (re-planning, so endpoint churn re-resolves). Before this
//! existed, the first device change silently killed mic passthrough for the rest of the host's life.
//!
//! `push` enqueues decoded interleaved-f32 PCM into a bounded ring (drop-oldest beyond ~120 ms so
//! mic latency stays bounded); a dedicated COM-apartment thread renders it event-driven through an
//! adaptive jitter buffer (prime → hold → re-prime, see the render loop — clients arrive in bursts,
//! the device pulls per-period), filling silence when the client isn't talking. WASAPI objects are
//! `!Send`, so they live entirely on that thread (mirrors `WasapiLoopbackCapturer`).
//! `push` enqueues decoded interleaved-f32 PCM into a bounded ring (drop-oldest so mic latency
//! stays bounded — the bound follows the adaptive prime threshold, or the legacy ~120 ms until
//! the pump drives it); a dedicated COM-apartment thread renders it event-driven through a
//! jitter buffer (prime → hold → re-prime, see the render loop — clients arrive in bursts, the
//! device pulls per-period) whose prime depth the mic pump sets from measured uplink jitter
//! ([`VirtualMic::set_target_depth`]), filling silence when the client isn't talking. WASAPI
//! objects are `!Send`, so they live entirely on that thread (mirrors `WasapiLoopbackCapturer`).
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{audio_control, VirtualMic, SAMPLE_RATE};
use super::{audio_control, MicBackendStats, VirtualMic, SAMPLE_RATE};
use anyhow::{anyhow, Context, Result};
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering};
use std::sync::mpsc::{sync_channel, SyncSender};
use std::sync::{Arc, Mutex};
use std::thread::{self, JoinHandle};
@@ -41,26 +43,53 @@ use wasapi::{Direction, SampleType, StreamMode, WaveFormat};
const CHANNELS: u32 = 2;
/// 48 kHz stereo f32: 2 channels * 4 bytes.
const BLOCK_ALIGN: usize = 2 * 4;
/// Jitter-buffer priming depth (~48 ms): the render loop emits pure silence until this much PCM
/// is queued, then plays from the cushion. Clients deliver mic audio in BURSTS (the Mac client's
/// input tap yields ~two 20 ms Opus packets every ~42 ms) while WASAPI pulls a small block every
/// device period (~10 ms) — with no cushion the queue sits near-empty and most periods insert
/// mid-stream silence: the "crackling mic" (heard live, Mac → Windows host 2026-07-03; the Linux
/// backend's process callback primes the same way and the identical stream was clean there). The
/// depth must cover the worst inter-burst gap (~42 ms), so ~48 ms with re-prime on a full drain.
/// LEGACY jitter-buffer priming depth (~48 ms): the render loop emits pure silence until this
/// much PCM is queued, then plays from the cushion. Old clients deliver mic audio in BURSTS
/// (the Mac client's input tap yields ~two 20 ms Opus packets every ~42 ms) while WASAPI pulls
/// a small block every device period (~10 ms) — with no cushion the queue sits near-empty and
/// most periods insert mid-stream silence: the "crackling mic" (heard live, Mac → Windows host
/// 2026-07-03; the Linux backend's process callback primes the same way and the identical
/// stream was clean there). The depth had to cover the worst inter-burst gap (~42 ms), so
/// ~48 ms with re-prime on a full drain. Today the pump MEASURES that gap and drives the prime
/// threshold per client ([`VirtualMic::set_target_depth`] — bursty clients still get ~48 ms,
/// modern 10 ms-cadence ones ~2535 ms); this constant remains the fallback until the pump's
/// first estimate, and forever under `PUNKTFUNK_MIC_LEGACY_BUFFER=1`.
const PRIME_BYTES: usize = (SAMPLE_RATE as usize * 48 / 1000) * BLOCK_ALIGN;
/// Bound the inject queue at ~120 ms so the passed-through mic stays low-latency (drop oldest
/// beyond): the priming cushion (~48 ms) plus arrival-burst headroom.
/// LEGACY bound for the inject queue at ~120 ms (drop oldest beyond): the fixed priming
/// cushion plus arrival-burst headroom. Applies only while the pump isn't driving the target;
/// adaptive mode bounds the queue at prime + [`CAP_HEADROOM_BYTES`] instead (≈ 50105 ms).
const MAX_QUEUE_BYTES: usize = (SAMPLE_RATE as usize * 120 / 1000) * BLOCK_ALIGN;
/// Producer-side overflow headroom (~32 ms) over the render loop's prime threshold when the
/// adaptive target drives the ring.
const CAP_HEADROOM_BYTES: usize = (SAMPLE_RATE as usize * 32 / 1000) * BLOCK_ALIGN;
pub struct WasapiVirtualMic {
queue: Arc<Mutex<VecDeque<u8>>>,
stop: Arc<AtomicBool>,
/// False once the render thread has exited (device error or stop) — the pump's reopen signal.
alive: Arc<AtomicBool>,
/// Ring policy/telemetry shared with the render thread (see [`RingShared`]).
ring: Arc<RingShared>,
join: Option<JoinHandle<()>>,
}
/// Atomics shared between the pump-facing handle and the render thread: the pump's adaptive
/// de-jitter target in, the effective prime threshold + reset-on-read counters out. All
/// `Relaxed` — a slowly-moving target and telemetry, not synchronization.
#[derive(Default)]
struct RingShared {
/// Pump-set jitter target in bytes. `0` = the pump never spoke (legacy mode, or its first
/// estimate hasn't landed) → the render loop keeps the fixed [`PRIME_BYTES`] and `push`
/// keeps the fixed [`MAX_QUEUE_BYTES`].
target_bytes: AtomicUsize,
/// Effective prime threshold (bytes) of the last render iteration.
prime_bytes: AtomicUsize,
/// Full-drain re-prime arms (see [`MicBackendStats`]).
reprimes: AtomicU64,
/// Per-channel samples dropped by the overflow cap.
overflow: AtomicU64,
}
impl WasapiVirtualMic {
pub fn open(channels: u32) -> Result<Self> {
anyhow::ensure!(
@@ -70,14 +99,15 @@ impl WasapiVirtualMic {
let queue = Arc::new(Mutex::new(VecDeque::<u8>::new()));
let stop = Arc::new(AtomicBool::new(false));
let alive = Arc::new(AtomicBool::new(true));
let ring = Arc::new(RingShared::default());
// Bring-up handshake: report the resolved device (or the error) before returning, so a missing
// virtual-mic device surfaces as Err (the caller retries with backoff) not a silent dead thread.
let (ready_tx, ready_rx) = sync_channel::<Result<String>>(1);
let (q, st, al) = (queue.clone(), stop.clone(), alive.clone());
let (q, st, rg, al) = (queue.clone(), stop.clone(), ring.clone(), alive.clone());
let join = thread::Builder::new()
.name("punktfunk-wasapi-mic".into())
.spawn(move || {
if let Err(e) = render_thread(q, st, ready_tx) {
if let Err(e) = render_thread(q, st, rg, ready_tx) {
tracing::error!(error = %format!("{e:#}"), "wasapi virtual-mic thread failed");
}
// Normal stop or device error alike: this instance is done — the pump reopens.
@@ -92,6 +122,7 @@ impl WasapiVirtualMic {
queue,
stop,
alive,
ring,
join: Some(join),
})
}
@@ -122,10 +153,25 @@ impl VirtualMic for WasapiVirtualMic {
for &s in pcm {
q.extend(s.to_le_bytes());
}
// Drop-oldest to keep latency bounded (mic is real-time; stale audio is worse than dropped).
if q.len() > MAX_QUEUE_BYTES {
let excess = q.len() - MAX_QUEUE_BYTES;
// Drop-oldest to keep latency bounded (mic is real-time; stale audio is worse than
// dropped). With the pump driving the target, the bound follows the render loop's prime
// threshold + headroom; otherwise (legacy / no estimate yet) the fixed 120 ms applies.
let cap = if self.ring.target_bytes.load(Ordering::Relaxed) == 0 {
MAX_QUEUE_BYTES
} else {
// `max(PRIME_BYTES)` covers the one render period before the loop first publishes.
self.ring
.prime_bytes
.load(Ordering::Relaxed)
.max(PRIME_BYTES)
+ CAP_HEADROOM_BYTES
};
if q.len() > cap {
let excess = q.len() - cap;
q.drain(..excess);
self.ring
.overflow
.fetch_add((excess / BLOCK_ALIGN) as u64, Ordering::Relaxed);
}
true
}
@@ -143,6 +189,28 @@ impl VirtualMic for WasapiVirtualMic {
fn channels(&self) -> u32 {
CHANNELS
}
fn set_target_depth(&self, samples_per_ch: usize) {
self.ring
.target_bytes
.store(samples_per_ch * BLOCK_ALIGN, Ordering::Relaxed);
}
fn depth(&self) -> Option<(usize, usize)> {
let prime = self.ring.prime_bytes.load(Ordering::Relaxed);
if prime == 0 {
return None; // render loop hasn't run yet
}
let q = self.queue.lock().ok()?;
Some((q.len() / BLOCK_ALIGN, prime / BLOCK_ALIGN))
}
fn take_stats(&self) -> MicBackendStats {
MicBackendStats {
reprimes: self.ring.reprimes.swap(0, Ordering::Relaxed),
overflow_dropped: self.ring.overflow.swap(0, Ordering::Relaxed),
}
}
}
/// Resolve the mic inject target from the wiring plan, auto-installing the Steam Streaming pair
@@ -273,6 +341,7 @@ fn try_install_steam_audio(inf_name: &str) -> bool {
fn render_thread(
queue: Arc<Mutex<VecDeque<u8>>>,
stop: Arc<AtomicBool>,
shared: Arc<RingShared>,
ready: SyncSender<Result<String>>,
) -> Result<()> {
if let Err(e) = wasapi::initialize_mta()
@@ -284,7 +353,7 @@ fn render_thread(
}
// Open + start the render stream. The WASAPI objects must outlive the loop, so build them here and
// keep them (a closure that *returned* them would drop them); on any failure report Err and exit.
let setup = (|| -> Result<(wasapi::AudioClient, wasapi::AudioRenderClient, wasapi::Handle, String)> {
let setup = (|| -> Result<(wasapi::AudioClient, wasapi::AudioRenderClient, wasapi::Handle, i64, String)> {
let (device, name) = resolve_target()?;
let mut audio_client = device.get_iaudioclient().context("IAudioClient")?;
// 48 kHz stereo f32; autoconvert lets WASAPI shared-mode SRC match the device mix format.
@@ -312,9 +381,9 @@ fn render_thread(
let buf_frames = audio_client.get_buffer_size().context("buffer size")? as usize;
let _ = render_client.write_to_device(buf_frames, &vec![0u8; buf_frames * BLOCK_ALIGN], None);
audio_client.start_stream().context("start render stream")?;
Ok((audio_client, render_client, h_event, name))
Ok((audio_client, render_client, h_event, default_period, name))
})();
let (audio_client, render_client, h_event, name) = match setup {
let (audio_client, render_client, h_event, default_period, name) = match setup {
Ok(t) => t,
Err(e) => {
let _ = ready.send(Err(anyhow!("{e:#}")));
@@ -322,18 +391,26 @@ fn render_thread(
}
};
let _ = ready.send(Ok(name));
// One device period in bytes (period is in 100 ns units; floor 10 ms if it reads absurd) —
// the device-side pull granularity the adaptive prime threshold builds on.
let period_bytes = ((default_period.max(0) as usize * SAMPLE_RATE as usize / 10_000_000)
.max(SAMPLE_RATE as usize / 100))
* BLOCK_ALIGN;
// Any error below (endpoint invalidated/removed, engine restart) propagates out of the loop,
// ending the thread — the `alive` flag flips in the spawn wrapper and the pump reopens.
//
// Adaptive jitter buffer (mirrors the Linux backend's process callback): clients push mic
// audio in bursts on their own clock while the device pulls a block every period from an
// independent clock, so a greedy per-period drain leaves the queue near-empty and pads most
// periods with mid-stream silence — audible as constant crackling. Instead: emit silence
// until [`PRIME_BYTES`] is buffered, then play from the cushion (zero-filling only a
// momentary shortfall), and re-prime only after a genuine FULL drain (the client went quiet —
// between talk spurts the cushion rebuilds, and [`VirtualMic::discard`] resets it across
// session gaps).
// Jitter buffer (mirrors the Linux backend's process callback): clients push mic audio in
// bursts on their own clock while the device pulls a block every period from an independent
// clock, so a greedy per-period drain leaves the queue near-empty and pads most periods
// with mid-stream silence — audible as constant crackling. Instead: emit silence until the
// prime threshold is buffered, then play from the cushion (zero-filling only a momentary
// shortfall), and re-prime only after a genuine FULL drain (the client went quiet — between
// talk spurts the cushion rebuilds, and [`VirtualMic::discard`] resets it across session
// gaps). The threshold = one device period + the pump's measured-jitter target
// ([`VirtualMic::set_target_depth`]); the fixed [`PRIME_BYTES`] until the pump's first
// estimate, and forever under `PUNKTFUNK_MIC_LEGACY_BUFFER=1` (the pump then never drives
// the target).
let mut buf: Vec<u8> = Vec::new();
let mut primed = false;
while !stop.load(Ordering::Relaxed) {
@@ -351,11 +428,18 @@ fn render_thread(
if buf.len() < need {
buf.resize(need, 0);
}
let target = shared.target_bytes.load(Ordering::Relaxed);
let prime = if target == 0 {
PRIME_BYTES
} else {
period_bytes + target
};
shared.prime_bytes.store(prime, Ordering::Relaxed);
// Silence base; overwrite with queued mic PCM once the cushion is primed.
buf[..need].fill(0);
{
let mut q = queue.lock().unwrap();
if !primed && q.len() >= PRIME_BYTES {
if !primed && q.len() >= prime {
primed = true;
}
if primed {
@@ -365,6 +449,7 @@ fn render_thread(
}
if q.is_empty() {
primed = false; // fully drained — re-prime before producing again
shared.reprimes.fetch_add(1, Ordering::Relaxed);
}
}
}
+331 -17
View File
@@ -26,6 +26,19 @@
//! out of the host's speakers. Real hardware is the fallback (audio then plays on both ends).
//! With `host_audio` (the `PUNKTFUNK_HOST_AUDIO` opt-in) the order flips back: real hardware
//! first, so the operator hears the stream locally.
//!
//! **Last resort, and the honest failure.** When neither a silent sink nor real hardware
//! survives the mic reservation, the Steam Streaming *Speakers* are taken as a flagged LAST
//! resort ([`Wiring::loopback_last_resort`]): their loopback is known-silent (validated live) —
//! a QUALITY risk the capture side warns about and treats as a stopgap — but holding a parked
//! endpoint beats holding none (2026-08 field case: the display isolate invalidated the only
//! real render endpoint mid-session, the mic held the Streaming Microphone, and a plan with no
//! loopback left the session unrecoverable). Cables, VoiceMeeter strips and generically-
//! "virtual" endpoints are never a last resort — capturing them re-captures what the mic writes,
//! an echo/feedback CORRECTNESS risk, unlike silence — so with only those left the plan is
//! honestly unsatisfiable ([`Wiring::loopback_unsatisfiable`]): a pure verdict on the endpoint
//! set that cannot change until the set does. Callers must wait for an endpoint-set change
//! ([`fingerprint`]), not retry.
/// A `(friendly_name, endpoint_id)` pair as enumerated from WASAPI.
pub(crate) type Endpoint = (String, String);
@@ -42,6 +55,22 @@ pub(crate) struct Wiring {
pub mic_capture: Option<Endpoint>,
/// Render endpoint for the desktop-audio loopback; made the default playback device.
pub loopback_render: Option<Endpoint>,
/// `loopback_render` is the flagged LAST RESORT (the Steam Streaming Speakers, whose
/// loopback is known-silent — validated live), taken only because nothing better survived
/// the mic reservation. The capture side treats it as a stopgap: it warns when the silence
/// materializes and re-plans on any endpoint-set change instead of riding it out.
pub loopback_last_resort: bool,
}
impl Wiring {
/// This plan has NO loopback endpoint — not even the last resort. Because [`plan`] is pure,
/// this is a STRUCTURAL verdict on the endpoint set, not a transient device error:
/// reattempting a capture open without an endpoint-set change must fail identically (the
/// 2026-08 field case spent 8+ minutes of flat 2 s retries proving exactly that). Callers
/// wait for the set's [`fingerprint`] to move instead of retrying.
pub(crate) fn loopback_unsatisfiable(&self) -> bool {
self.loopback_render.is_none()
}
}
/// Render-endpoint friendly-name substrings (lowercased) usable as the virtual-mic write target,
@@ -69,11 +98,18 @@ fn capture_for(mic_render_lname: &str) -> &'static [&'static str] {
}
/// A render endpoint no loopback should capture: the VB-CABLE (reserved for the mic even when it
/// isn't the chosen target — capturing a cable someone else feeds echoes too) and the Steam
/// Streaming Speakers, whose loopback is silent (validated live). Also the capture-side
/// watchdog's test for "the operator's new default can never work — snap back to the plan".
/// isn't the chosen target — capturing a cable someone else feeds echoes too), the Steam
/// Streaming Speakers, whose loopback is silent (validated live), and any VoiceMeeter or
/// generically-"virtual" endpoint. VoiceMeeter's strips share one internal mixer: capturing ANY
/// of its render endpoints re-captures what the mic wrote into another one — a digital feedback
/// loop with no acoustic path to break it (mic=`Voicemeeter Input`, loopback=`Voicemeeter Aux
/// Input` used to pass the old name checks). Also the capture-side watchdog's test for "the
/// operator's new default can never work — snap back to the plan".
pub(crate) fn excluded_from_loopback(lname: &str) -> bool {
lname.contains("cable") || lname.contains("steam streaming speakers")
lname.contains("cable")
|| lname.contains("steam streaming speakers")
|| lname.contains("voicemeeter")
|| lname.contains("virtual")
}
/// A render endpoint that is SILENT on the host but loopback-capturable — the client-only audio
@@ -127,7 +163,8 @@ pub(crate) fn plan(
// 3. Loopback from the REMAINING renders. Client-only (default): the silent sink (Steam
// Streaming Microphone — its loopback works, unlike the Speakers') > real hardware
// (audible fallback) > any non-excluded leftover. `host_audio`: real hardware first.
// (audible fallback). `host_audio`: real hardware first. Either order can fall through
// to the flagged last resort below.
let not_mic = |id: &str| mic_render.as_ref().is_none_or(|(_, mid)| mid != id);
let real_hw = || {
renders.iter().find(|(n, id)| {
@@ -140,25 +177,131 @@ pub(crate) fn plan(
.iter()
.find(|(n, id)| not_mic(id) && silent_sink(&n.to_lowercase()))
};
let leftover = || {
// LAST RESORT — the Steam Streaming Speakers, and ONLY them. Their loopback is known-silent
// (validated live): a QUALITY risk, flagged so the capture side can warn when the silence
// materializes and re-plan when the endpoint set changes — but a parked endpoint beats none
// (2026-08: the display isolate invalidated the only real render endpoint mid-session and a
// loopback-less plan left the session unrecoverable). Never a cable, a VoiceMeeter strip, or
// a generically-"virtual" endpoint: those re-capture what the mic writes — echo/feedback
// CORRECTNESS risks — so "no loopback" stays the honest answer there. NOTE
// `excluded_from_loopback` itself stays untouched: it also powers the capture watchdog's
// judgement of a NEW operator-chosen default, where admitting the Speakers would change
// mid-stream snap-back semantics.
let last_resort = || {
renders
.iter()
.find(|(n, id)| not_mic(id) && !excluded_from_loopback(&n.to_lowercase()))
.find(|(n, id)| not_mic(id) && n.to_lowercase().contains("steam streaming speakers"))
};
let loopback_render = if host_audio {
real_hw().or_else(silent).or_else(leftover)
let preferred = if host_audio {
real_hw().or_else(silent)
} else {
silent().or_else(real_hw).or_else(leftover)
}
.cloned();
silent().or_else(real_hw)
};
let (loopback_render, loopback_last_resort) = match preferred {
Some(ep) => (Some(ep.clone()), false),
None => match last_resort() {
Some(ep) => (Some(ep.clone()), true),
None => (None, false),
},
};
Wiring {
mic_render,
mic_capture,
loopback_render,
loopback_last_resort,
}
}
/// Order-independent fingerprint of an enumerated endpoint set. [`plan`] is a pure function of
/// these inputs (the env knobs are process-stable), so an unchanged fingerprint PROVES an
/// unchanged verdict: re-planning an unsatisfiable set before the fingerprint moves only repeats
/// the same answer, with IPolicyConfig default-device writes as the side effect. The capture
/// loop polls this instead of re-planning, and treats a change as the recovery moment.
pub(crate) fn fingerprint(renders: &[Endpoint], captures: &[Endpoint]) -> u64 {
use std::collections::hash_map::DefaultHasher;
use std::hash::{Hash, Hasher};
let mut h = DefaultHasher::new();
// Each direction hashes as a length-prefixed sorted slice, so renders and captures cannot
// alias each other and endpoint order (enumeration order churns) never matters.
for eps in [renders, captures] {
let mut sorted: Vec<&Endpoint> = eps.iter().collect();
sorted.sort();
sorted.hash(&mut h);
}
h.finish()
}
/// The one-shot diagnosis for a plan with no loopback endpoint: every enumerated render with WHY
/// it was rejected, then ONLY the remedies not already taken. The static advice this replaces
/// ("attach one, or let the host install the Steam Streaming pair") was already satisfied in the
/// 2026-08 field case — the pair WAS installed, its Microphone half reserved by the mic — so the
/// message pointed at a fix the box already had. Pure, like [`plan`]: callers pass the same
/// enumeration the plan consumed.
pub(crate) fn describe_no_loopback(renders: &[Endpoint], wiring: &Wiring) -> String {
debug_assert!(wiring.loopback_unsatisfiable());
let mic_id = wiring.mic_render.as_ref().map(|(_, id)| id.as_str());
let rejected: Vec<String> = renders
.iter()
.map(|(name, id)| {
let ln = name.to_lowercase();
let why = if Some(id.as_str()) == mic_id {
"reserved for the virtual mic (its loopback would echo the client's voice back)"
} else if ln.contains("cable") {
"virtual cable (its loopback re-captures what is written into it)"
} else if ln.contains("voicemeeter") {
"VoiceMeeter strip (shares the mixer the mic writes into — a feedback loop)"
} else if ln.contains("steam streaming speakers") {
// Reachable only when the Speakers ARE the mic target (operator override) —
// the last-resort tier takes them otherwise.
"known-silent loopback (validated live)"
} else if ln.contains("virtual") {
"unrecognized virtual endpoint (assumed feedback/silence risk)"
} else {
// `plan` accepts any non-virtual render — reaching this arm means a tier
// changed without updating this diagnosis.
"rejected by the wiring plan"
};
format!("{name:?}: {why}")
})
.collect();
let inventory = if rejected.is_empty() {
"no render endpoints exist at all".to_string()
} else {
rejected.join("; ")
};
let has = |needle: &str| {
renders
.iter()
.any(|(n, _)| n.to_lowercase().contains(needle))
};
let mut remedies = vec!["attach any output device (headphones, or a monitor/TV with audio)"];
// Only useful when the mic would actually vacate a loopback-capable endpoint: with the mic
// on the Steam Streaming Microphone, a cable frees that silent sink for the loopback. A mic
// on a VoiceMeeter strip frees nothing capturable, so the advice is withheld there.
if !has("cable")
&& wiring
.mic_render
.as_ref()
.is_some_and(|(n, _)| silent_sink(&n.to_lowercase()))
{
remedies.push(
"install VB-Audio Virtual Cable — the mic then takes the cable and frees the Steam \
Streaming Microphone's render side for the loopback",
);
}
if !has("steam streaming microphone") {
remedies.push(
"install Steam — its Remote Play streaming drivers add a loopback-capable virtual \
sink",
);
}
format!(
"no loopback-capturable render endpoint: {inventory}. Remedies: {}",
remedies.join("; or ")
)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -285,6 +428,10 @@ mod tests {
w.loopback_render.unwrap().0,
"Speakers (Steam Streaming Microphone)"
);
assert!(
!w.loopback_last_resort,
"the silent sink is a PREFERRED pick"
);
assert_eq!(
w.mic_capture.unwrap().0,
"CABLE Output (VB-Audio Virtual Cable)"
@@ -319,16 +466,88 @@ mod tests {
assert!(w.loopback_render.is_none());
}
/// Steam Streaming Speakers never become the loopback (silent loopback, validated live)
/// even when they're the only non-mic endpoint.
/// Steam Streaming Speakers are never a PREFERRED loopback (their loopback is silent
/// validated live) — but when they are the only non-mic endpoint they ARE taken, flagged as
/// the last resort: a silent loopback the capture side can warn about beats a plan with no
/// endpoint at all (which is unrecoverable until the topology changes).
#[test]
fn steam_speakers_never_loopback() {
fn steam_speakers_only_as_last_resort() {
let renders = [
ep("CABLE Input (VB-Audio Virtual Cable)"),
ep("Speakers (Steam Streaming Speakers)"),
];
let w = plan(&renders, &[], None, false);
assert!(w.loopback_render.is_none());
for host_audio in [false, true] {
let w = plan(&renders, &[], None, host_audio);
assert_eq!(
w.loopback_render.as_ref().unwrap().0,
"Speakers (Steam Streaming Speakers)",
"host_audio={host_audio}"
);
assert!(w.loopback_last_resort, "host_audio={host_audio}");
}
}
/// THE 2026-08 field case: no cable, only the Steam pair left after the display isolate
/// invalidated the monitor's DP audio endpoint. The mic reserves the Streaming Microphone
/// (the only mic candidate), and the plan must then take the Speakers as the last resort —
/// the old plan yielded no loopback here and the session never recovered.
#[test]
fn field_case_steam_pair_only_takes_speakers_as_last_resort() {
let renders = [
ep("Altavoces (Steam Streaming Speakers)"),
ep("Altavoces (Steam Streaming Microphone)"),
];
let captures = [ep("Microphone (Steam Streaming Microphone)")];
let w = plan(&renders, &captures, None, false);
assert_eq!(
w.mic_render.unwrap().0,
"Altavoces (Steam Streaming Microphone)"
);
assert_eq!(
w.loopback_render.unwrap().0,
"Altavoces (Steam Streaming Speakers)"
);
assert!(w.loopback_last_resort);
}
/// The last resort never shadows a real pick: with real hardware present the Speakers stay
/// unchosen and the flag stays down, in both preference modes.
#[test]
fn last_resort_never_beats_real_hardware() {
let renders = [
ep("Speakers (Steam Streaming Microphone)"),
ep("Speakers (Steam Streaming Speakers)"),
ep("Speakers (Realtek HD Audio)"),
];
let captures = [ep("Microphone (Steam Streaming Microphone)")];
for host_audio in [false, true] {
let w = plan(&renders, &captures, None, host_audio);
assert_eq!(
w.loopback_render.as_ref().unwrap().0,
"Speakers (Realtek HD Audio)",
"host_audio={host_audio}"
);
assert!(!w.loopback_last_resort, "host_audio={host_audio}");
}
}
/// Cables and VoiceMeeter strips are CORRECTNESS risks (they re-capture what the mic
/// writes — echo/feedback), not quality risks: never the loopback, not even as a last
/// resort. The plan stays honestly unsatisfiable.
#[test]
fn cable_and_voicemeeter_never_last_resort() {
let renders = [
ep("CABLE Input (VB-Audio Virtual Cable)"),
ep("CABLE In 16ch (VB-Audio Virtual Cable)"),
ep("Voicemeeter Aux Input (VB-Audio Voicemeeter AUX VAIO)"),
];
let captures = [ep("CABLE Output (VB-Audio Virtual Cable)")];
for host_audio in [false, true] {
let w = plan(&renders, &captures, None, host_audio);
assert!(w.loopback_render.is_none(), "host_audio={host_audio}");
assert!(!w.loopback_last_resort, "host_audio={host_audio}");
assert!(w.loopback_unsatisfiable(), "host_audio={host_audio}");
}
}
/// Operator override beats the candidate order.
@@ -359,4 +578,99 @@ mod tests {
assert!(w.mic_render.is_none());
assert_eq!(w.loopback_render.unwrap().0, "Speakers (Realtek HD Audio)");
}
/// VoiceMeeter box with real hardware: no cable, so the mic takes `Voicemeeter Input` — and
/// the loopback must NEVER be `Voicemeeter Aux Input` (same internal mixer: capturing any
/// VoiceMeeter render re-captures what the mic wrote — a digital feedback loop). Real
/// hardware wins the loopback in both modes.
#[test]
fn voicemeeter_aux_never_pairs_with_voicemeeter_mic() {
let renders = [
ep("Voicemeeter Input (VB-Audio Voicemeeter VAIO)"),
ep("Voicemeeter Aux Input (VB-Audio Voicemeeter AUX VAIO)"),
ep("Speakers (Realtek HD Audio)"),
];
let captures = [ep("Voicemeeter Out B1 (VB-Audio Voicemeeter VAIO)")];
for host_audio in [false, true] {
let w = plan(&renders, &captures, None, host_audio);
assert_eq!(
w.mic_render.as_ref().unwrap().0,
"Voicemeeter Input (VB-Audio Voicemeeter VAIO)",
"host_audio={host_audio}"
);
assert_eq!(
w.loopback_render.as_ref().unwrap().0,
"Speakers (Realtek HD Audio)",
"host_audio={host_audio}"
);
}
}
/// Only VoiceMeeter endpoints (headless mixer box): mic wins one, the loopback is honestly
/// absent — like the cable-only case, never another strip of the same mixer.
#[test]
fn voicemeeter_only_no_loopback() {
let renders = [
ep("Voicemeeter Input (VB-Audio Voicemeeter VAIO)"),
ep("Voicemeeter Aux Input (VB-Audio Voicemeeter AUX VAIO)"),
];
for host_audio in [false, true] {
let w = plan(&renders, &[], None, host_audio);
assert!(w.mic_render.is_some(), "host_audio={host_audio}");
assert!(w.loopback_render.is_none(), "host_audio={host_audio}");
}
}
/// A generically-"virtual" leftover (unknown vendor cable) is refused too: the last resort
/// accepts ONLY the Steam Streaming Speakers, so a virtual endpoint that slips past
/// `excluded_from_loopback`'s name list still can't become the loopback.
#[test]
fn unknown_virtual_never_loopback() {
let renders = [
ep("CABLE Input (VB-Audio Virtual Cable)"),
ep("Speakers (Some Virtual Audio Device)"),
];
let w = plan(&renders, &[], None, false);
assert!(w.loopback_render.is_none());
}
/// The fingerprint keys the capture loop's "wait for an endpoint change" state: it must
/// ignore enumeration order (Windows churns it), react to any topology change, and never
/// alias the render and capture directions.
#[test]
fn fingerprint_order_independent_topology_sensitive() {
let a = [ep("Speakers (Realtek HD Audio)"), ep("CABLE Input")];
let a_rev = [ep("CABLE Input"), ep("Speakers (Realtek HD Audio)")];
let caps = [ep("CABLE Output")];
assert_eq!(fingerprint(&a, &caps), fingerprint(&a_rev, &caps));
assert_ne!(fingerprint(&a, &caps), fingerprint(&a[..1], &caps));
assert_ne!(fingerprint(&a, &caps), fingerprint(&caps, &a));
}
/// The unsatisfiable-plan diagnosis must name what the mic reserved and advise ONLY the
/// remedies not already taken: in the field case the Steam pair was installed (so "install
/// Steam" would point at a fix the box already had) and the cable was missing (so VB-CABLE
/// is the advice that actually frees the silent sink).
#[test]
fn describe_no_loopback_skips_satisfied_remedies() {
// Field shape minus the Speakers (mic holds the Streaming Microphone, nothing else).
let renders = [ep("Altavoces (Steam Streaming Microphone)")];
let captures = [ep("Microphone (Steam Streaming Microphone)")];
let w = plan(&renders, &captures, None, false);
assert!(w.loopback_unsatisfiable());
let msg = describe_no_loopback(&renders, &w);
assert!(msg.contains("reserved for the virtual mic"), "{msg}");
assert!(msg.contains("VB-Audio Virtual Cable"), "{msg}");
assert!(!msg.contains("install Steam"), "{msg}");
// Cable-only headless box: VB-CABLE is already installed (and freeing it wouldn't help
// anyway), while the Steam pair is the remedy that adds a capturable sink.
let renders = [ep("CABLE Input (VB-Audio Virtual Cable)")];
let captures = [ep("CABLE Output (VB-Audio Virtual Cable)")];
let w = plan(&renders, &captures, None, false);
assert!(w.loopback_unsatisfiable());
let msg = describe_no_loopback(&renders, &w);
assert!(msg.contains("install Steam"), "{msg}");
assert!(!msg.contains("install VB-Audio Virtual Cable"), "{msg}");
}
}
@@ -234,9 +234,10 @@ pub fn start(
});
}
/// Stub — the audio plane needs an audio-capture backend (PipeWire on Linux, WASAPI on Windows)
/// + libopus; this keeps the remaining targets (e.g. macOS) compiling (crate doc: "the crate
/// compiles everywhere"). Reports failure the same way the real stream thread does: clears `running`.
/// Stub — the audio plane needs an audio-capture backend (PipeWire on Linux, WASAPI on
/// Windows) + libopus; this keeps the remaining targets (e.g. macOS) compiling (crate doc:
/// "the crate compiles everywhere"). Reports failure the same way the real stream thread
/// does: clears `running`.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn start(
running: std::sync::Arc<std::sync::atomic::AtomicBool>,
+5 -4
View File
@@ -1649,10 +1649,11 @@ async fn hooks_get_shape_and_put_validation() {
// ------------------------------------------------------------------ library scanners
/// The scanner list is platform-shaped and read-only-safe; the toggle rejects unknown ids with
/// 404. (A successful toggle PUT would write the developer's real `library-scanners.json`, so the
/// write path is exercised only through the unknown-id rejection here — the settings round-trip
/// itself is unit-tested in `library::scanners` against pure shapes.)
/// The scanner list is platform-shaped and read-only-safe; the toggle rejects unknown ids
/// with 404. (A successful toggle PUT would write the developer's real
/// `library-scanners.json`, so the write path is exercised only through the unknown-id
/// rejection here — the settings round-trip itself is unit-tested in `library::scanners`
/// against pure shapes.)
#[tokio::test]
async fn library_scanner_list_and_unknown_toggle() {
let app = test_app(test_state(), None);

Some files were not shown because too many files have changed in this diff Show More