Lossless audio: the 0xD3 PCM plane, 44.1–176.4 kHz, surround, verified on glass (ABI 24) #263

Merged

enricobuehler merged 27 commits from worktree-hires-audio-pcm into main 2026-08-16 11:18:37 +00:00

Conversation 0 Commits 27 Files Changed 74

+11932 -898

27 Commits

Author	SHA1	Message	Date
enricobuehler	e6476eddcf	fix(core): a doc line starting with > read as a markdown blockquote and failed clippy ... `--all-targets` clippy treats `/// >24 kHz` as the start of a quote block and then rejects the unmarked lines that continue it. I added that test and gated it on tests and fmt but not clippy, so it would have reached CI red. Reworded rather than escaped — the sentence reads better without the symbol anyway.	2026-08-16 12:55:20 +02:00
enricobuehler	47c9162433	Merge remote-tracking branch 'origin/main' into worktree-hires-audio-pcm ... # Conflicts: # crates/pf-client-core/src/pad_audio.rs	2026-08-16 12:50:15 +02:00
enricobuehler	a9ffd3becd	feat(host): a Hello that names a format but not the capability was ignored in total silence ... Found on glass, and it cost the first run of the §13.2 check. The host resolved `plane="0xC9 Opus"` while every condition a reader could see was satisfiable — the operator policy was on, the session was stereo, the rate was supported, and the video bitrate left room. The reason was gate condition 1, which is deliberately NOT logged because "the client did not set the capability" is every ordinary session with every shipping client and would drown the log. That reasoning holds for the ordinary case and not for this one. A `Hello` carrying a rate or a depth but no `CLIENT_CAP_AUDIO_HIRES` is contradictory: something asked, and is being ignored. The two halves come from different places in a client — the capability from a settings toggle, the format from whatever that toggle resolved to — so they can drift apart, and an embedder that gets it wrong currently sees nothing at all. One warn, only in the contradictory case, naming both halves and saying they must be set together.	2026-08-16 12:39:42 +02:00
enricobuehler	362be21ebb	fix(probe): the format was on the wire but the capability bit was not, so the host was right to ignore it ... Correcting my own previous commit, which claimed the probe had been "asking for the lossless plane by accident". It had not, and the comment I deleted for being wrong was right. This `Hello` is built BY HAND and never passes through `advertised_client_caps`, the helper that derives CLIENT_CAP_AUDIO_HIRES from the requested format for the shipping clients. So the probe's explicit 48000/16 was inert: the host's gate tests the capability first, and without the bit a format on the wire is a request it correctly ignores. No session was ever mislabelled, and the old comment's "never sets CLIENT_CAP_AUDIO_HIRES" was an accurate statement about this file. What was actually broken is what I added: `--audio-format` set the rate and depth and not the bit, so it asked for nothing. Found on glass — the host resolved `plane="0xC9 Opus"` while every gate condition looked satisfiable, and nothing was logged, because condition 1 is deliberately unlogged (it is every session with every shipping client). Both are set together now, and the comment says why they must be. With that fixed the plane resolves end to end against a real Linux host: 0xD3, 96 kHz, 24-bit, 192 samples per channel — the 2 ms frame the ladder predicts for 96/24 at the default MTU.	2026-08-16 12:37:47 +02:00
enricobuehler	bca85db160	fix(probe): it asked for the lossless plane by accident and then discarded every frame of it ... The probe sent an explicit 48 000/16 as its "legacy" audio request. Under the capability rule that is a genuine hi-res request — the bit is set when either field is non-zero, because it keys on "the caller specified a format" — so against a host with the operator policy on, the probe advertised the capability, was handed the `0xD3` plane, and counted nothing, because its decode arm only ever matched `0xC9`. Its own comment asserted the opposite ("never sets CLIENT_CAP_AUDIO_HIRES"), which is what made this invisible to a reader. Same shape as the Android bug, same root: a doc that stopped being true when the rule changed. It sends the `0`/`0` sentinel now. Then it learns the plane it was accidentally asking for. `--audio-format` takes `opus` (the default, unspecified) or a rung, validated against core's own supported-rate predicate rather than a restated list; `--audio-out FILE` writes the DECODED audio as raw interleaved f32 for offline analysis. That last one is the point. Design §13.2 — "play a >24 kHz tone on the host and confirm it arrives" — has been the one check no test suite can replace, and it was written as a listening session. It is now a command: play the tone, run the probe with `--audio-format lossless96 --audio-out`, and look for energy above 24 kHz. A brick wall there indicts the host's capture specifically, because core's own test already proves the transport carries it. The decode arm reads the RESOLVED format off the Welcome, never the request. The host may decline hi-res for any of the five reasons in the gate and answer Opus, and a probe that trusted its own ask would mis-parse every datagram it was actually sent.	2026-08-16 12:19:51 +02:00
enricobuehler	b347682eb2	feat(console-ui): Gaming Mode was the one client surface with no audio-format row ... Every other client got the picker; this one load-modify-saves the whole settings struct, so a format chosen elsewhere round-tripped through it untouched and simply could not be set from the console. The row renders from `pf_client_core::session::AUDIO_FORMATS` rather than restating the values, so it picks up rungs the other clients add without being edited. An unrecognised stored value — a newer client's rung arriving through the shared profile catalog — renders as Opus rather than a dash, because that is what the wire mapping resolves it to; the row and the request agree. The stereo gate stays, but for a different reason than the one that was written down. The host's `channels != 2` decline is gone and the arithmetic supports that: at 48 kHz both 5.1 and 7.1 fit a 1 ms frame. But `pf-client-core` still filters a surround request out before it reaches the wire, so a live row under 5.1 would be a control that changes nothing — precisely the lie this screen dims rows to avoid, and it would disagree with the GTK dialog reading the same settings file on the same machine. The comment says the old reason is dead, names the real one, and says to delete the arm when the client-side filter learns the ladder rather than when the host rule changed. No override marker or profile commit: this screen has neither for any row, by design — the console pins profiles rather than editing them.	2026-08-16 12:19:49 +02:00
enricobuehler	f206feb588	test(core): nothing proved that the plane actually carries what it exists to carry ... The whole reason for a second audio plane is that Opus is 48 kHz by construction, so content above 24 kHz is gone before its encoder sees it. That claim had no test. Every existing one checks bit-exactness code by code, which says nothing about frequency — a pipeline that silently band-limited would pass all of them. A 30 kHz tone now goes through the real `0xD3` payload path at 96 kHz and 176.4 kHz and must come out with its energy intact and every sample inside one 24-bit code. The detector's discrimination is asserted too, in the same test. A single-bin DFT that reads high everywhere would "prove" survival through a pipeline that deleted the tone, so the test also measures a frequency that is NOT in the signal and requires it to read as silence. A spectral assertion without that guard is decoration. This is the SOFTWARE half of design §13.2. The other half — that the host's capture did not resample on the way in — is WASAPI autoconvert and PipeWire's resampler, and needs a host and an interface. What this settles is the inference: a brick wall at 24 kHz in an on-glass spectrum now indicts the capture path specifically, because the transport is known to carry it. Goertzel rather than an FFT, so this costs no dependency in a crate that ships into five clients.	2026-08-16 12:09:47 +02:00
enricobuehler	47d8301aeb	feat(host): the presentation clock advanced by a frame's label rather than its length ... Admits the 44.1 kHz family and hi-res surround, and fixes the drift that admitting them turns from latent into live. `next_pts_ns += frame_us * 1000` advanced the clock by the frame's NOMINAL duration. That was harmless while every rate divided every rung, and 44 100 Hz divides none of them: a "5 ms" frame carries 220 samples per channel, which is 4 988 662 ns. The clock therefore invented 2 267 574 ns every second — 2 272 ppm, 8.2 seconds an hour — and the re-anchor beside it is `max`, so it only ever corrected FORWARD. A fast clock was never pulled back, and the A/V sync loop would have chased it forever. The clock is now a running sample total rather than a sum of per-frame durations, which accumulates exactly zero where summing floored frames accumulates about a nanosecond a frame. It lives in a small `PtsClock` because the drift sat inside `audio_thread`, which no test can reach; as a struct it is provable, and the planted-error run pins every ladder rung rather than just 5 ms. `advance` folds whole seconds out of the sample count into the base — exact, since `rate x channels` samples is precisely one second — because `usize` is 32-bit on some targets and 176.4 kHz 7.1 would wrap it in under an hour. The frame pacer was changed to agree even though it is cosmetic: it is gated on real data being available, so a nominal value produced a slot the pacer waited out rather than time it invented. Two clocks describing the same frame and disagreeing by 0.23 % is the kind of thing a later reader reconciles in the wrong direction. The rate gate reads `pcm::rate_is_supported` instead of restating the set. The Windows 48 kHz floor is untouched and never needed touching — the `max` sits inside the "requested above engine" arm, so a 44 100 request never tripped it. The comment now separates the two rules that read as one line: refusing to ask for more than the engine has is the honesty rule for either plane; the 48 kHz floor is for Opus alone, because libopus accepts 8/12/16/24/48 kHz only. Surround's `channels != 2` decline is DELETED rather than relaxed. The design called it a one-line restriction; it was worse than that, because the check ran BEFORE the frame ladder and overrode the one piece of code that knows the answer. The ladder is channel-aware, so it decides and `None` declines — and the design's blanket "surround is out at the default MTU" turns out to be wrong in both directions: 48 kHz 5.1 fits at 1.5 ms and 7.1 at 1 ms, while 16-bit 5.1 fits even at 88.2 and 96 kHz. What is actually true is that nothing surround fits above 48 kHz in 24-bit, and no 7.1 fits above 48 kHz at all. Two stale figures corrected while adjacent: the budget doc claimed a 20 Mbps session affords the whole stereo ladder (176.4/24 stereo is 8.5 Mbps and wants 33.9), and the operator-facing decline log quoted 1.5–4.6 Mbps for a plane that now spans 1.4–8.5 in stereo and 33.9 in 7.1.	2026-08-16 11:52:53 +02:00
enricobuehler	6bbdafc289	feat(apple): the Swift port of the de-jitter policy had the same divide-first defect as core ... This client hand-ports `JitterPolicy`, `AvSync` and `DroughtConceal` into Swift rather than draining core's decisions, so every fix core just made had a second, silent copy here. `perMS = (rateHz / 1000) * channels` divided before it multiplied, exactly as core did, so 44 100 Hz became 44 samples per millisecond and every depth, target and reported figure was 2.3 % low. All sixteen conversion sites now multiply first. `frameSamples` delegates to the same floor-per-channel rule core uses instead of deriving from `perMS` — at 44 100 Hz a 5 ms FRAME is 440 interleaved samples where 5 ms of AUDIO is 441, and the shed and the near-miss margin both mean "exactly one packet", so a self-derived answer described a packet that does not exist. Swift traps on overflow rather than wrapping, and the samples-to-ms direction takes a caller-supplied count that `setSyncTarget` can drive to `Int.max / 2`, so that one saturates explicitly — an untrapped multiply there would abort inside the render callback. `DroughtConceal` counts frames now and derives both millisecond figures from the resolved frame, and its two thresholds are two real frames rather than a fixed 10 ms. The old shape charged 5 ms per concealed frame whatever the frame was: on the 1 ms frame that 48 kHz/24-bit 7.1 lands on, it bought a fifth of the concealment it promised and reported 24 ms of synthesis as 120. The frame count stayed right, which is why nothing played wrong and nothing caught it — only the relationship between the count and the frame length exposes it, and that is what the new test asserts. The stereo gate is gone from all three settings surfaces and from the connect path, and the format picker offers the 44.1 family. The resolved rate is rendered with its fractional kHz — integer division would have printed a 44 100 Hz session as "44 kHz", and `String(format:)` would have localised the separator.	2026-08-16 11:33:02 +02:00
enricobuehler	0196c75769	fix(android): every Opus session was advertising the lossless capability, and the ABI doc told it to ... Two things: a live opt-in bug, and the 44.1 kHz family. Android returned `(48_000, BITS_16)` for the "Standard (Opus)" setting and handed it to `connect_with_audio_format`, whose capability rule sets the bit when EITHER field is non-zero — it keys on "the caller specified a format", not "the format differs from the default", because otherwise 48/16 would be the one rung on the ladder nobody could ask for. The host's gate accepts 48/16 as a supported format. So on any host with the operator policy enabled, a user who chose Opus silently got the lossless plane at 1.5 Mbps. `punktfunk_connect_ex11`'s doc caused it: it said passing 48000/16 was "exactly `ex10`, byte-for-byte on the wire". That stopped being true when the capability rule changed and nobody updated the sentence. It now says the opposite explicitly, including that a hardcoded 48 000/16 as a stand-in for "default" opts every ordinary session in. Both ends of the Android path send the `(0, 0)` unspecified sentinel now, and two tests pin the asymmetry from either side of the JNI boundary. The 44.1 kHz half mirrors core: the client's own `per_ms` had the same divide-first defect, so its conversions are exact now, and `frame_samples` delegates to core rather than re-deriving — at 44 100 Hz a 5 ms FRAME is 440 interleaved samples where 5 ms of AUDIO is 441, and the shed and near-miss margin both mean "exactly one packet". The openable-rate probe descends the requested rate's own family before falling to the 48 kHz floor, because AAudio grants a rate or fails and never substitutes. The floor is accepted unprobed — universally granted, and the answer a failed probe falls back to — so a default session still opens no stream and pays nothing. The probe uses the requested channel count because the resolved one does not exist until the Welcome, which errs toward Opus. Surround is no longer hidden. The gates were removed rather than relaxed: core's ladder is channel-aware and the host decides, so the client asks and reports what it was given.	2026-08-16 11:27:48 +02:00
enricobuehler	f30f7e88b0	feat(client): the desktop had no way to ask for the lossless plane except an environment variable ... Apple and Android both shipped a real audio-format picker; the desktop had a stopgap env lever, because `SessionParams` had no field for it and the one struct literal that fills it lives in another crate. It has one now, and the four clients share a vocabulary. The stored values are byte-identical to Android's and Apple's — `opus`, `lossless48`, `lossless96` under the key `audio_format` — because a profile that round-trips between a phone and a TV but not to the desktop is the exact bug worth avoiding. They are read from those clients rather than invented here, and a value this build does not recognise resolves to Opus rather than refusing the connect. `PUNKTFUNK_AUDIO_HIRES` still overrides the setting, in BOTH directions, which is how this crate already treats every other `PUNKTFUNK_*` lever. A lever that loses to a stale profile is useless for the thing operators reach for it for, and one that can only switch a feature ON is half a lever. One behaviour deliberately changed: an unparseable value used to mean "off". It now warns and is ignored, so the user's setting still decides — garbage silently defeating a switch somebody set in a UI is worse than the pre-UI behaviour it replaced. Unset, off and garbage are three distinct outcomes now, and a test pins all three. The precedence itself is a pure function of (env, setting) so it can be tested without mutating the process environment, which is the idiom the neighbouring parse already used. Stats carry the RESOLVED format off the Welcome, never the requested one, and the OSD prints it — a desktop that says "lossless" while the host declined is the same class of lie as claiming a sample rate you did not get.	2026-08-16 11:27:32 +02:00
enricobuehler	ec500b0943	feat(host): monitor mode declined hi-res because nobody had written the lookup it was waiting on ... The design said hi-res is offered in stream-sink mode and declined in monitor mode "unless the monitored node's own rate can be read from the registry", and that lookup was never written — so `PUNKTFUNK_STREAM_SINK=0` could never have the feature at all, by omission rather than by decision. It exists now. Reading our own capture stream's rate would be worthless: in monitor mode we tap someone else's sink THROUGH PipeWire's resampler, which reports a clean rate whatever is upstream — the same blindness as WASAPI's autoconvert, which is the trap this whole feature is built around. So the lookup binds the graph's elected default sink and reads that NODE's format: one bounded round-trip with a 3 s timer, so a sick graph costs a fallback rather than a stalled handshake. It reads `default.audio.sink`, the elected default, NOT the neighbouring `default.configured.audio.sink` that this host's own stream-sink code writes. That one is a preference: unset on a box nobody configured, and able to name a node that no longer exists. Unknown declines. No graph default, no format, no answer inside the timer — each resolves to Opus rather than guessing, because the whole point is that a session must never claim a rate its content does not have. That asymmetry is stated at the lookup, at the call site and in the module doc, since "unknown means fine" is the opposite convention to the one the wiring plan uses a few files away. One nuance the design does not address: a monitor tap emits the GRAPH-side rate, while `Format` on an adapter node forwards to the device side. On an ordinary box those are the same number, because PipeWire opens the device at the graph rate whenever it can. They diverge only for a device that cannot run the graph's rate, where reading the device side declines something that would have worked — the safe direction. The exactly right source is the sink's monitor PORT's own format, one further registry hop, named at the call site as a follow-up rather than left implied. Also documents the one variable name both ends read: the host treats `PUNKTFUNK_AUDIO_HIRES` as a boolean and the desktop client accepts a rate or a rate/depth pair, so a box that is both sees two grammars. `1` is the spelling that means "on" to both.	2026-08-16 11:27:17 +02:00
enricobuehler	63312e0a19	feat(core): the rate divided before it multiplied, which is the whole reason 44.1 kHz was deferred ... Lifts the design's §4.1 deferral and admits 44 100 / 88 200 / 176 400 Hz. `JitterPolicy` and `AvSync` computed `per_ms = rate_hz / 1000 * channels` and then expressed every figure as `ms * per_ms`. The division came FIRST, so 44 100 became 44 samples per millisecond and every depth, target, shed threshold, hard cap, de-prime fuse and reported `buffer_ms` was 2.3 % low. 48 000 and 96 000 were exact only because they happen to divide. The design prescribed "denominate the policy in samples with a rational `per_ms`". No rational type was needed: the defect was operator ORDER. Multiplying first and dividing last is exact at every rate for one integer division per conversion, and 48/96 kHz stay bit-identical by construction — `per_sec == 1000 × per_ms` exactly there, so both conversions reduce to the old expression. Every existing policy test passes untouched. `frame_samples()` now delegates to `pcm::samples_per_frame` rather than re-deriving the count. At 44 100 Hz those differ: 5 ms of audio is 441 interleaved samples but a 5 ms FRAME is 440, because 220.5 samples per channel do not exist. The near-miss margin and the shed both mean "exactly one packet", so a self-derived answer would have described a packet that does not exist. `frame_duration_ns` is the new inverse, and it exists because a frame's label and its duration have come apart. 44 100 divides none of the seven ladder rungs, 88 200 divides only 5 ms, 176 400 only 5 ms and 2.5 ms — so a rung is a nominal length for the wire and the ring, never a duration. Anything advancing a timestamp must use the real sample count or the clock runs 0.23 % fast forever, which the A/V sync loop would fight and never win. `MAX_CONCEAL_PACKETS` was a frame count documented as "50 ms at the protocol's 5 ms frames" — 20 ms at a 2 ms lossless frame. It is now `MAX_CONCEAL_MS` with the count derived, and `AudioGapTracker` carries the frame the same way `JitterPolicy` and `DroughtConceal` already do. The ABI conceal buffer sizes its run from the derived cap while keeping the frame size at the longest rung — mismatched deliberately, and only in the direction that cannot overrun. Overflow is handled where the multiply-first order creates it: 480 000 ms at 176 400 Hz by 8 channels is 6.8e11, which wraps a 32-bit usize before the divide brings it back, so the intermediates are u64/u128 and saturate rather than wrap — a wrapped window is a tiny one, i.e. a fuse that blows instantly.	2026-08-16 10:55:55 +02:00
enricobuehler	d8f7984477	fix(core): the drought fuse was spent in 5 ms units on a plane whose frames are 2 ms ... `DroughtConceal` charges one frame per concealed frame but bounds itself in WALL-CLOCK milliseconds, and the two disagreed about how long a frame is. It assumed 5 ms, so on a 2 ms lossless frame the `max_ms` budget ran out after two fifths of the time the tuning intends, and the `plc_ms` line over-reported concealment by the same factor. The frame COUNT was always right — it charged 5 and divided by 5 — which is exactly why this went unnoticed: the load-bearing number was correct and only the two human-facing ones were wrong. It now counts frames directly and derives both millisecond figures from the negotiated frame, so `packet()` needs no division at all. The two thresholds move with it. Both were `2 * FRAME_MS`, expressing "two frames", so they are now two real frames rather than a fixed 10 ms — on a 2 ms plane the old value waited five frames before conceding there was a stall. Both clients pass their resolved frame in, and the desktop client drops the local frame tally it had to keep while core could not be told.	2026-08-16 10:05:09 +02:00
enricobuehler	bee924162a	feat(client): the desktop client decoded one plane, at one rate, and read the engine's opinion of neither ... Work package H5, client half — Linux and Windows. `AudioDec` grows a PCM arm behind the same `decode`/`conceal` pair, so the pull loop is plane-agnostic and `AudioGapTracker` still decides how many frames are missing. The lossless plane conceals with `PcmConceal` rather than libopus, because a raw frame carries no decoder model to extrapolate from. A codec the client cannot decode is now refused out loud and the session runs video-only, instead of a `0xD3` payload being fed to libopus. The conceal scratch was `5760 * channels` — 120 ms at 48 kHz, and an overrun at 96 kHz. It is now 120 ms at the resolved rate for Opus, or exactly one negotiated frame for PCM. On Windows the engine trap is closed on both sides of the connect. `can_render_at` reads the render endpoint's own mix format before the handshake and withholds the capability bit when the engine cannot carry the rate — the client-side twin of the host rule, and the only point at which declining is still free. `render_thread` reads it again before initialising and warns if the endpoint moved underneath us. `autoconvert` would otherwise have silently downsampled a 96 kHz stream on arrival and wasted the entire exercise while every log line agreed the session was hi-res. Both graphs stay 32-bit float, deliberately and with the reasoning at the site: core hands both planes over as f32, so an S24 graph would re-quantise the very samples the plane exists to deliver unrounded. §9 asks a client whose device refuses the rate to "say so and fall back". That is not implementable where the doc puts it — by the time the device opens, the wire format is fixed by the Welcome and the plane cannot switch mid-session, so falling back would mean killing audio. The decision moved to where it is still free: gate before connect, report loudly at open. PipeWire's `NODE_LATENCY` was the string literal "240/48000"; it is built at runtime now so the graph quantum stays one protocol frame at any rate, and a new `param_changed` handler logs what the graph actually granted. Verified on Linux in a container: check, clippy -D warnings, build and 210 tests, all green. `audio_wasapi.rs` compiles nowhere available here — macOS cannot build the crate at all and the MSVC cross target dies in openh264's C build — so Windows CI is its first compiler.	2026-08-16 09:56:29 +02:00
enricobuehler	45c37f1a3b	feat(apple): the ring measured milliseconds in a constant nobody had told about the negotiated rate ... Work package H6, Apple half. The Swift client is the one that PORTS the de-jitter policy rather than draining core's decoded frames, so every hardcoded 48 in `AudioRing` was a second, silent copy of an assumption core had already stopped making. `perMS`, the one-second ring capacity, and `AvSync`'s weight all follow the resolved rate now, and the frame length follows the resolved `audio_frame_us` through the accessor core grew for exactly this. That frame length matters in four places, all of which were quietly wrong on a lossless session: the target floor is a device quantum plus one frame, the smooth shed drops one frame, the seam crossfade is capped at half of one, and the near-miss margin means "less than one packet in hand". Frozen at 5 ms against a 2 ms frame the last of those means two and a half packets, which grows the target on a ring that was never close to starving. Core has been changed to agree. Frame geometry is computed in microseconds. 2 500 µs is the one rung on the ladder where routing through integer milliseconds silently loses a fifth of the frame, and every other rung is ms-exact — which is precisely what would have let that bug hide. `setPreferredSampleRate` now runs on every branch (the `.playback` ones, which are mic-off iOS and all of tvOS, asked for nothing at all before) and runs before `setActive`, since that is when the hardware is configured. What the route actually granted is read back afterwards and warned about when it differs. The graph is deliberately built at the RESOLVED rate rather than the granted one. The source node's format describes the samples, which arrive at the wire rate; declaring the device's 48 kHz while writing 96 kHz samples plays the stream at half speed. And a literal fallback is impossible anyway — the plane never switches mid-session — so the honest behaviour is to let the engine resample and SAY SO, which is report-not-refuse rather than the silent resample the design forbids. Settings gain an audio-format choice defaulting off, stereo-gated, with the same raw values the Android leg uses so a profile round-trips between the two clients. The HUD shows the RESOLVED format, because a declined lossless session that still says "lossless" is the same class of lie as claiming a rate you did not get.	2026-08-16 09:55:42 +02:00
enricobuehler	ec1e7be7dd	feat(android): AAudio grants a rate or fails, so the fallback had to happen before the Hello ... Work package H6, Android half — plus the near-miss margin in core, which the Apple leg independently proved was measuring the wrong thing. AAudio never substitutes a rate: an explicit request is granted or the open fails. So a 48 kHz rung sitting below a 96 kHz session on the open ladder could only ever produce playback at 2x speed, or a resample the design forbids in as many words. The fallback therefore runs BEFORE the handshake: a probe opens a stream at the requested rate, reads back what was granted and closes it without ever starting it, so nothing routes and no audio focus is taken. The ladder is 96 -> 48 keeping the depth -> the legacy pair, so a device that will not grant 96 kHz still gets a 48/24 lossless session rather than silence. Only 96 kHz is probed; a default session opens nothing and pays nothing. The open ladder still gains a rate dimension, because a probe is one measurement at one instant and the supervisor reopens across route changes. Rate is the outermost dimension, and the last rung is AAUDIO_UNSPECIFIED for the HAL that refuses an explicit request while already running at exactly the rate we wanted — `arm` still holds it to the session's rate, so that rung can rescue a session but never mislabel one. The conceal scratch is now sized per plane: Opus keeps its 120 ms legal maximum, PCM uses the longest ladder rung. Sizing both from the Opus figure was 24x too large at 96 kHz, and sizing both from the PCM one would have been far too small for Opus. The poll tick is one frame of the running plane rather than a fixed 5 ms, so on a 2 ms session the drought-concealment arm keeps pace with playout instead of losing 3 ms in 5. In core, the near-miss margin stops being a constant. Its own doc always said "less than one protocol frame was left in hand", but it was frozen at 5 ms — which against a 2 ms lossless frame means two and a half frames, so it grew the target on a ring that was never close to starving, inverting exactly what a near-miss exists to detect. Identical on every Opus session. The Android CI lane's test filter is an allowlist and eleven classes sat outside it, including the audio HUD ones, which had never run in CI at all. All eleven pass, so nothing was hiding — but nothing was gating either. They are listed now, explicitly rather than by glob, because the unfiltered task also drags in the screenshot scenes.	2026-08-16 09:55:21 +02:00
enricobuehler	b4255631ac	feat(host): the gate could not tell whether capture would really deliver the rate it was promising ... Closes the one place the implementation and the design did not meet, plus two constants that still measured time in the Opus frame. §8.4 says hi-res resolves only if "the capture path can genuinely deliver the rate", but the capturer is opened by the audio thread, well after `negotiate` builds the Welcome. The gate could only check that the format was well-formed; the truth arrived later and the audio thread ended the plane — a silence outcome the design calls unacceptable. The rate is now answered before the Welcome, by a probe that opens no stream and changes nothing about the box. It is three-valued on purpose. Windows reads the endpoint's own mix format via a read-only rerun of the wiring plan — deliberately NOT the real wiring pass, which parks defaults, mints endpoints and logs, none of which may happen mid-handshake for a session that is about to resolve to Opus anyway. Linux needs no query at all: in stream-sink mode the host declares the format itself, so it is honest by construction, and monitor mode cannot know, so it declines. Anything else, or any probe failure, is Unknown and declines too — unlike the wiring plan, where an unknown format means "assume it's fine", here it means "we cannot prove content matches label", which is the whole point of the feature. The audio thread's per-iteration check stays, with its role reduced to the probe-to-open race — an operator changing the endpoint format, a hotplug re-plan, a graph renegotiation. Its action is unchanged because there is still only one correct one: the plane cannot switch to Opus mid-session, and mis-clocked samples under the promised label is precisely what this feature exists to prevent. `INFILL_AFTER` and `LATE_DEPARTURE` were both denominated in the 5 ms Opus frame while documenting themselves as "one protocol frame". On a 2 ms lossless frame the first spent its 500 ms infill budget in 200 ms of real time and the second let a two-frame slip read as on-time. Both now derive from the session's resolved frame. `INFILL_MAX` deliberately stays wall-clock and says why. `SendStats` lost its `Default` in the process, deliberately: a zero frame makes every departure compare as late, and since the window is rebuilt on each 30-second flush that would not even have been a once-per-session mistake. `PUNKTFUNK_AUDIO_HIRES` is now documented for operators, with the cost stated plainly and without overselling — the design's own §12 says hi-res is likely inaudible for game content and that the real win is bit-exactness.	2026-08-16 09:41:38 +02:00
enricobuehler	a81c6ac789	feat(core): a ported de-jitter policy had no way to learn the real frame length ... Two independent workstreams hit the same wall: `PUNKTFUNK_AUDIO_FRAME_MS` is the Opus plane's 5 ms, the lossless plane runs 4 ms at 48/24 and 2 ms at 96/24, and nothing in the C surface carried the resolved value. An embedder that PORTS the de-jitter policy rather than draining `next_audio_pcm` — the Apple client does exactly this — was stuck compiling against 5 ms, which sheds 2.5 frames at a time on a 96 kHz session and puts the target floor a frame too deep. The earlier judgement that no C embedder needs this rested on `next_audio_pcm` reporting each frame's real length in `frame_count`. That is not true in the case that matters: concealed frames are PREPENDED into the same buffer, so the count answers "how many samples did I get", not "how long is one frame". The two questions coincide only when nothing was lost. Microseconds, not milliseconds — the ladder has sub-millisecond rungs and 2 500 µs must not truncate to 2 ms. `0` means the host stated nothing, in which case the old constant is right. Added, not widened, like the rate and depth accessors before it.	2026-08-16 09:38:16 +02:00
enricobuehler	88d29d3615	chore(core): regenerate the C header for the FRAME_MS note ... cbindgen picked the doc comment up on the next build; the checked-in header must not disagree with the source it is generated from.	2026-08-16 09:35:56 +02:00
enricobuehler	099904967d	docs(core): PUNKTFUNK_AUDIO_FRAME_MS is the Opus frame, and a ring sized from it is wrong on the lossless plane ... `PUNKTFUNK_AUDIO_SAMPLE_RATE_HZ` got a paragraph explaining that it is the default rate and that hi-res sessions must use the accessor. Its sibling `PUNKTFUNK_AUDIO_FRAME_MS` got nothing, and it has the identical hazard: the lossless plane negotiates 4 ms at 48/24 and 2 ms at 96/24, so an embedder sizing a playout ring as frames × PUNKTFUNK_AUDIO_FRAME_MS is wrong by up to 2.5x with nothing in the C surface saying so. Both constants are KEPT unchanged — embedders size rings from them and deleting either is a silent C break. What was missing is the note. No accessor is added because `next_audio_pcm` already reports each frame's real length in `frame_count`, which is the figure to size from; the comment now points there.	2026-08-16 09:33:12 +02:00
enricobuehler	d09b4668c6	fix(core): the cheapest lossless rung was the one format nobody could ask for ... Three follow-ups from the lossless plane. The hi-res capability bit was derived from "the caller asked for something other than 48 kHz/16-bit". That rule is wrong at exactly one point, and it is a point that matters: 48/16 is both the default AND the cheapest lossless rung (1.5 Mbps against Opus's 256 kbps), so it was the one format on the ladder no caller could request. The bit now means "the caller SPECIFIED a format", with 0 as the unspecified value. No wire byte and no ABI change — the wire already encodes an explicit 48000/16 identically to absent, and the capability bit is what carries the intent. That inverts a default, so every legacy entry point had to move with it: `connect` and the four pre-v24 `punktfunk_connect_ex*` variants passed an explicit 48000/16 and would otherwise have started advertising hi-res on behalf of embedders that never asked. They now pass 0/0, and the comment at each site says why the distinction is load-bearing. `JitterPolicy` had two decisions denominated in FRAMES rather than milliseconds — the floor under the effective target, and the smooth shed's one-frame drop — both written when 5 ms was the only frame this protocol had. The lossless plane negotiates 4 ms at 48/24 and 2 ms at 96/24, where the old constants shed 2.5 frames at a time and faded across an entire frame. `set_frame_us` fixes both and caps the seam crossfade at half a frame, because a fade as long as the material it is fading is not a crossfade. It is a setter rather than a constructor argument for two reasons: the default keeps every Opus session and all seventeen policy tests bit-identical, and `audio_frame_us` is resolved by the host and only known after the Welcome — later than the ring is built. Frame length is computed in microseconds so 2 500 µs does not truncate through integer milliseconds. `audio_kbps()` reports what the plane costs, for the stats line §4.6 asks for. It answers only for PCM, and the answer is exact rather than measured: PCM is constant-bitrate by construction, so a byte counter would add sampling noise to a number already known precisely. Opus is VBR with a host-side ladder position, so the client has no honest figure and says None instead of inventing one.	2026-08-16 09:18:08 +02:00
enricobuehler	cef4b849e7	fix(client): a Linux-only helper was gated on test, so the Windows client's clippy lane could not build ... `sink_from_props` carries `cfg(any(target_os = "linux", test))` like its neighbours in this module, but unlike them it takes a `pipewire::` type in its signature — and `pipewire` is a Linux-only dependency. The bare `test` arm therefore compiles it in test configuration on EVERY platform, where the crate does not resolve. It builds green in release, because nothing reaches the function off Linux. It only fails under `clippy --all-targets`, which builds the lib test target — so the Windows client's release build passes and its clippy lane goes red. That is how it reached main. Its only caller, `walk_graph`, is already `cfg(target_os = "linux")`, and no test calls it, so the gate simply becomes Linux-only. The neighbouring helpers keep their `test` arm: their signatures are plain data, which is the whole point of that pattern. Pre-existing on main and unrelated to the audio work in this branch; fixed here because it blocks this PR's Windows lane.	2026-08-16 02:19:00 +02:00
enricobuehler	761895ec46	feat(host): capture was pinned to 48 kHz and one oversized frame could silence audio for the whole session ... The capture facade, both backends and the encode loop take a rate; the encode loop gains a PCM branch; and the session resolves which plane it runs at handshake. `AudioCapturer` grows `sample_rate()` alongside `channels()`, and both backends report what they ACTUALLY opened rather than what was asked for. That distinction is the feature: on Windows `AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM` means asking a 48 kHz engine for 96 kHz succeeds, returns no error, and hands back interpolation — a session that would log 96 kHz, spend the bandwidth, and carry nothing above 24 kHz. The endpoint's own mix format was already being read and logged one line before the request; it is now read BEFORE the format is chosen and the request is capped to it. The design said to decline outright when the engine is slower. Taken literally that regresses the shipped Opus path: a 44.1 kHz endpoint is an ordinary Windows configuration, and declining down to 44,100 would hand libopus a rate it does not accept. The decline is floored at 48 kHz, so only a hi-res request can lose. The resolve gate is the five conditions from the design, each declining to Opus with a logged reason — a fallback is not a failure, but an unexplained one is. Its cost check compares the plane against the video bitrate because audio rides datagrams outside the ABR loop: whatever it takes is off the top and ABR can neither see it nor reclaim it. A request that does not fit declines rather than being quietly handed a cheaper rung. One condition cannot be answered where the design puts it. "The capture path can genuinely deliver the rate" is unknowable at handshake, because the capturer is opened by the audio thread well afterwards. The gate checks format validity; the audio thread checks the truth every iteration and, on the PCM plane only, ends the plane rather than shipping mislabelled samples. Ending it is a silence outcome the design calls unacceptable, and it is confined to a 96 kHz opt-in whose endpoint turns out to run at 48 kHz — the real fix is probing capture before the Welcome, which this pass does not do. Separately, a pre-existing bug this feature makes reachable: the audio and pad-audio send paths treated ANY `send_datagram` error as "connection gone" and tore the plane down for the rest of the session. Three of quinn's four error variants are not that. `TooLarge` now counts and drops one frame, `UnsupportedByPeer`/`Disabled` end the plane once with a reason, and only `ConnectionLost` breaks the session. The counts join the existing 30-second egress line. Note that the send BUFFER's drops remain invisible: quinn evicts oldest-first and returns Ok, so "buffer full" never reaches a caller. The counter here is TooLarge only. `PUNKTFUNK_AUDIO_HIRES` defaults OFF, unlike `PUNKTFUNK_AUDIO_REDUNDANCY` — this spends bandwidth the user did not previously agree to, so it is asked for on both ends. GameStream capture stays 48 kHz permanently and says why at the call site: Moonlight's protocol is Opus 48 kHz and is not ours to renegotiate.	2026-08-16 02:08:47 +02:00
enricobuehler	435962ba38	feat(core): the client had no way to be told what audio format it was about to receive ... C ABI 23 -> 24. The resolved rate, depth, codec and frame duration now follow the exact path `audio_channels` already takes — Hello carries the request, Welcome carries what the host actually resolved, and the client opens its device from the Welcome rather than from what it asked for. A host may resolve lower than requested; that is the whole point of "the client asks and the host obliges if it can". `0xD3` decodes through the same sink as `0xC9`, so nothing downstream of the demux changes. Concealment does change: a lossless format has no PLC, because there is nothing in a raw frame from which to synthesise its successor, so the PCM plane uses `PcmConceal` where the Opus plane calls into libopus. Two ABI hazards handled by adding rather than widening. `PunktfunkAudioPcm` and `PunktfunkStats` are repr(C) with no struct_size guard, so a new field in either would break every C embedder that allocates one by value; the rate and depth arrive through accessors instead, mirroring `punktfunk_connection_audio_channels`. And `punktfunk_connect_ex11` is a new entry point rather than a wider `ex10` — `ex5` through `ex10` are byte-for-byte unchanged and delegate with the 48 kHz/16-bit defaults, the `next_rumble_cmd2` precedent from ABI 18. `PUNKTFUNK_AUDIO_SAMPLE_RATE_HZ` is KEPT with its value and meaning unchanged. It is the default rate, embedders size rings from it, and deleting it would be a silent C break; hi-res sessions use the accessor. The conceal buffer is sized once and never reallocated — the embedder is already holding a pointer into it — so a PCM frame decodes into scratch and is copied in under a clamp. An oversized or malformed datagram truncates rather than growing the buffer. A test pins the pointer and length across a 200,000-sample datagram and a 38-packet loss run. The hi-res cap is advertised only when the caller asked for a non-default format, because it means "capable AND turned on" and only the embedder knows whether its device can open at that format. One consequence worth knowing: 48 kHz/16-bit is byte-identical to a legacy request, so that particular rung is not reachable through the parameter pair alone. Also records, at the line itself, why `datagram_send_buffer_size` is NOT raised for the lossless plane: it would quadruple the Opus plane's worst-case backlog, the two cannot be sized separately before the handshake resolves, and quinn evicts oldest-first while returning Ok — so those drops cannot be counted through the API at all.	2026-08-16 02:08:25 +02:00
enricobuehler	7e97e600c7	feat(wire): the audio plane could not say what format it was carrying, so lossless had no way to be negotiated ... The `0xC9` plane's header has no rate, depth or codec field, and `Hello`/`Welcome` carried only a channel count. A second, lossless plane therefore needs its own tag and its own negotiation rather than a wider header. `0xD3` carries interleaved LE PCM behind DELIBERATELY the same 13-byte header as `0xC9`, so `AudioGapTracker` and the pts/A-V-sync plumbing work unchanged and the only new logic is the payload and its concealment. One frame per datagram, never fragmented; the frame duration is chosen at session start from the raw frame size so it cannot exceed the path MTU. Redundancy is not defined for this plane and is never sent with it. `Hello` gains `audio_rate_hz`/`audio_bits` — what the client is asking for. `Welcome` gains `audio_codec`/`audio_rate_hz`/`audio_bits`/`audio_frame_us` — what the host actually resolved, which may be lower, and which the client must open its device from rather than from what it requested. Two wire traps, both now covered: `Welcome`'s tail is conditional — `cipher` at 68 and the 32-byte ChaCha key at 69..101 are emitted only for ChaCha — so an appended field lands at 79 under AES and 111 under ChaCha. Getting that wrong breaks soft-AES clients (webOS) and nothing else, which is exactly the kind of bug that ships. A test pins both offsets and both total lengths. `Hello`'s post-HDR tail is capped at 27 bytes, because with no HDR block present the decoder disambiguates by remaining length and a 28-byte tail would be misread AS an HDR block. That budget goes from 3 spent to 8; a test pins it. A default Opus session stays byte-identical on both messages: `Welcome` is still exactly 68 bytes, `Hello` still 26. `HOST_CAP_AUDIO_HIRES` takes the LAST free `host_caps` bit. The next host capability needs a second byte and an ABI bump; the constant says so where someone will read it.	2026-08-16 01:37:19 +02:00
enricobuehler	368cd3c13d	feat(audio): the jitter policy took the sample rate from a constant, so a lossless plane had nowhere to put one ... H1 of the hi-res programme, plus the PCM payload H2 needs. `JitterPolicy` and `AvSync` read `SAMPLE_RATE_HZ` directly, so every depth, target, shed threshold and reported `buffer_ms` was 48 kHz by construction. Both now take the rate explicitly via `new_at_rate`; `new` delegates at the protocol default, so all three production callers and the seventeen policy tests keep byte-identical behaviour. The rate is denominated in integer samples per millisecond and stays that way. 48 and 96 kHz are exact; 44 100 truncates to 44 samples/ms, a silent 2.3% error in every figure the policy computes. That deferral now has a `debug_assert` tripwire rather than a comment, so adding a rate the arithmetic cannot represent is loud. `audio::pcm` is the second plane's payload: interleaved LE integer samples, no codec. FLAC was the obvious choice and does not earn its place here — a datagram over the path MTU is not sent at all and this plane is never fragmented, so frame duration must be sized from the worst case, and FLAC's worst case is a VERBATIM subframe: raw plus a header. It would negotiate the same frame duration, the same packet rate and the same send-buffer sizing, buying only average bytes on a plane that is provisioned for peak because it rides outside the ABR loop. At 24 bits from an undithered float mix it saves least of all, and it would cost a spike gate plus a dependency on five packaging targets. The frame ladder is sized from raw bytes, which fixes a latent bug in the design: the doc's table sized 96/24 at 2.5 ms from a *coded* estimate of ~1000 B, but the raw frame is 1440 B and does not fit a 1472-byte-ceiling datagram. Under FLAC that would have been a verbatim frame that silently failed to send. A test pins it. Bit-exactness is proven rather than asserted: every representable code at both depths survives wire to f32 and back unchanged.	2026-08-16 01:15:32 +02:00