Three levers to lower and steady decode latency on Snapdragon (Adreno) devices:
- ADPF (Adaptive Performance Framework): a new dlsym-resolved hint session
(native/src/adpf.rs; API-33+, resolved at runtime so there's no build-time
link dependency and libpunktfunk_android.so still loads on API 31/32) tells
the CPU governor the video pipeline runs a per-frame real-time workload, so it
keeps those threads on fast cores at high clocks. It now covers all three
latency-critical threads — the pf-decode feed/drain/present loop, the core
data-plane pump (UDP receive + FEC reassembly), and the audio thread — via a
new generic hot-thread registry on NativeClient (register_hot_thread /
hot_thread_ids; the pump self-registers). The session is built lazily on the
first presented frame, since ADPF createSession rejects a set containing any
not-yet-live tid.
- operating-rate -> Short.MAX ("as fast as possible"): pushes the Qualcomm
decoder to run each frame at max clocks instead of merely sustaining the
display rate at a power-saving clock that adds per-frame decode latency.
- appCategory="game": makes the app eligible for OEM Game Mode / Game Dashboard
performance profiles.
The core registry is cross-platform (gettid on Linux/Android, a no-op
elsewhere) — no Android-specific pollution of the shared core. Host workspace +
64 core tests green; Android arm64-v8a + x86_64 (platform 31) build + clippy
clean. On-device Snapdragon validation pending.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- native: the 756-line session.rs becomes session/{mod,connect,input,planes}.rs
around a SessionHandle (connect lifecycle + trust, input plane shims, plane
start/stop + stats drain).
- Decode-stats sampling is HUD-gated (nativeSetVideoStatsEnabled): with the
overlay hidden the decode thread skips the per-AU clock read + lock; enabling
resets the measurement window.
- audio: the AAudio open path is a per-sharing-mode try_open closure — the
realtime callback state (ring, prime, free-list) is rebuilt per attempt, so a
failed exclusive-mode try can't leak state into the shared-mode retry.
- Kotlin: ConnectScreen/StreamScreen slimmed by extracting ConnectDialogs,
StatsOverlay and TouchInput.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rounds out codec negotiation across the last three clients — each advertises what it can decode,
builds its decoder from the resolved Welcome.codec, and exposes a "Video codec" preference picker.
**Windows** (Rust, mirrors Linux): `decodable_codecs()` + `ffmpeg_codec_id()`; the D3D11VA and
software FFmpeg decoders (and the mid-session D3D11VA→software demotion) open the negotiated codec
instead of hardcoding HEVC; settings gain a `codec` field + reactor ComboBox; `--codec` CLI flag.
**Apple** (Swift/C-ABI): AnnexB is now codec-aware — a `VideoCodec` enum drives H.264 vs HEVC NAL
parsing / parameter-set extraction (`CMVideoFormatDescriptionCreateFromH264ParameterSets` for H.264,
no VPS) and AVCC repacking; `PunktfunkConnection` advertises H264|HEVC via `punktfunk_connect_ex7`,
reads `resolvedCodec` (`punktfunk_connection_codec`), and threads `videoCodec` into the stage-1/2
pipelines + `VideoDecoder`; SettingsView "Video codec" Picker (auto/HEVC/H.264). AV1 is left out
(hosts don't emit it on the native path, and it's not an AnnexB codec). Test call sites updated.
**Android** (Kotlin + Rust JNI): the JNI `nativeConnect` gains `preferredCodec`; the native decode
loop picks the AMediaCodec MIME (`video/hevc`|`video/avc`) from `connector.codec` and advertises
H264|HEVC; Settings `codec` field + Compose dropdown.
Core/host/probe/Linux clippy + tests green (unchanged from 2a). Windows/Apple/Android compile on
their platform CI (this Linux box can't build them — Windows toolchain / Xcode / the Android NDK's
opus-cmake toolchain). All follow the Linux client's validated pattern.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Continues docs/hdr-pipeline-plan.md. Steps 0/1 + Step 2 (Windows/Android) already
landed in ffae2a3; this is Step 2 (Apple) + Step 3 (all clients). Client-only — no
core/host/ABI change (the 0xCE/next_hdr_meta/color_info surfaces shipped in Step 0).
Step 2 — clients APPLY the host's HDR metadata (each remaps from the wire form: ST.2086
G,B,R order, mastering luminance in 0.0001 cd/m2):
- Apple: connect via punktfunk_connect_ex5 (resurrects the previously-dead HDR pipeline);
nextHdrMeta/colorInfo wrappers + HdrMeta SEI-blob builders; the pump drains nextHdrMeta
-> VideoDecoder.setHdrMeta -> CVBufferSetAttachment of MasteringDisplayColorVolume (24B
BE) + ContentLightLevelInfo (4B BE) on each HDR pixel buffer (correct for the
itur_2100_PQ layer; CAEDRMetadata avoided as ambiguous there).
Step 3 — capability-gate: advertise HDR caps ONLY when the display can present it, so an
SDR display gets a proper BT.709 stream instead of PQ it would mis-tone-map; an HDR
display self-tone-maps from the Step-1/2 mastering metadata.
- Windows: present::display_supports_hdr() (DXGI any IDXGIOutput6 colour space == G2084),
ANDed with the user HDR setting in session.rs; logs the SDR drop.
- Apple: NSScreen.maximumExtendedDynamicRangeColorComponentValue>1 (macOS) /
UIScreen.main.potentialEDRHeadroom>1 (iOS) in SessionModel.
- Android: Settings.displaySupportsHdr (Display.getHdrCapabilities HDR10/HDR10+) passed
through a new hdr_enabled jboolean on nativeConnect; session.rs gates the caps.
Validation: Android native (incl. the jboolean gate) builds + clippy clean via cargo-ndk;
fmt clean. Windows (MSVC), Apple (Swift) and the Kotlin side are CI/on-glass validated —
not compilable on the Linux dev box. Deferred to the RTX box: mid-session Reconfigure
SDR-downgrade on monitor move, and confirming the host emits SDR for an SDR client off an
HDR desktop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two strands, entangled in punktfunk1.rs, committed together (one builds-green tree).
HDR pipeline Step 0 — glass-to-glass colour-metadata transport (docs/hdr-pipeline-plan.md):
- Protocol/ABI: ColorInfo on the Welcome + a 0xCE HdrMeta datagram carry the source colour
space + HDR10 static mastering metadata (quic.rs, abi.rs connect_ex5 fixing caps=0).
- New platform-independent, unit-tested HDR static-metadata helpers (hdr.rs): chromaticities
(1/50000), mastering luminance (0.0001 cd/m2), MaxCLL/MaxFALL in HDR10/ST.2086 units.
- Capture/encode hooks (capture.rs, encode.rs set_hdr_meta) + Linux client / probe plumbing.
Security-audit hardening — top 3 from docs/security-review.md, each adversarially verified:
- #1 [HIGH] Secret file permissions. The host key.pem/cert.pem and both trust stores are now
written owner-only: 0600 + dir 0700 on Unix (mirrors mgmt_token), best-effort
SYSTEM/Administrators/OWNER-only icacls DACL on Windows (%ProgramData% is Users-readable).
Closes a local key-disclosure -> host-impersonation gap. New gamestream::{create_private_dir,
write_secret_file} + a 0600 regression test.
- #2 [HIGH] Native SPAKE2 PIN is single-use. The PIN is consumed the moment the host sends its
key-confirmation (which lets the client test its one guess), before reading the proof, so any
completed attempt -- right OR wrong -- disarms the window. A wrong PIN isn't observable
host-side (the client aborts before sending its proof), so consuming on first attempt is what
delivers the documented "one online guess" instead of an unbounded brute-force of the static
4-digit PIN. Test verifies single-use.
- #3 [MEDIUM] RTSP packetSize is bounded ([64,2048] in stream_config) and VideoPacketizer::new
uses saturating .max(1), killing a PRE-AUTH div-by-zero/underflow panic of the video thread.
Tests for {0,15,16,17} + out-of-range rejection.
fmt + clippy -D warnings clean; full workspace test suite green (93 host tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ndk's DataSpace derives Copy/PartialEq/Eq and impls Display (no Debug), so the
{ds:?} in the HDR dataspace log statements wouldn't compile under cargo-ndk.
Host clippy can't catch it — decode.rs is android-gated. Switch to {ds}.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirrors the Apple client's HDR path so the Android client can display HDR from a
Windows HDR host:
- nativeConnect now advertises VIDEO_CAP_10BIT | VIDEO_CAP_HDR (was 0), so the
host upgrades to a Main10 / BT.2020 PQ encode.
- decode.rs detects HDR reactively from the decoder's reported output colour
(color-transfer ST2084=6 / HLG=7, color-range) -- the AMediaCodec analogue of
VideoToolbox's format description on Apple -- and signals the Surface dataspace
(Bt2020[Itu]Pq / Bt2020[Itu]Hlg) so the compositor/display switch to HDR.
AMediaCodec decodes Main10 from the in-band SPS; no profile override needed.
Also fixes the Android build: set_frame_rate (added in e9de730) is gated on the
ndk `nativewindow` + `api-level-30` features, which weren't enabled -- so that
commit could not compile under cargo-ndk. Enable
features = ["media","audio","nativewindow","api-level-31"] (minSdk 31): covers
set_frame_rate (api-30), set_buffers_data_space + the DataSpace module (api-28),
and ANativeWindow (nativewindow).
Verified host-side: fmt --all + clippy --workspace (the caps advertise + JNI
surface). The android-gated decode + NDK gating verified against the ndk 0.9
sources; android.yml (cargo-ndk) is the compile gate, and real HDR display needs
an HDR device + Windows HDR host.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stats HUD (mirrors the Apple client): the decode thread accumulates FPS, receive
throughput, and capture->client latency (p50/p95, skew-corrected) in Rust
(clients/android/native/src/stats.rs); nativeVideoStats drains a snapshot ~1 Hz
over JNI as a DoubleArray. StreamScreen renders a Compose overlay
(W*H@Hz / fps / Mb/s / latency, + dropped-under-loss), toggled by a Settings
switch (persisted, default on) or a 3-finger tap.
Performance (decode.rs):
- ANativeWindow_setFrameRate(refresh_hz): align display vsync to the stream rate
(no 60-in-120 judder); safe since minSdk 31 >= API 30.
- Raise the decode thread toward URGENT_DISPLAY (best-effort setpriority) so
background work can't preempt it under load.
- Codec low-latency hints KEY_PRIORITY=0 (realtime) + KEY_OPERATING_RATE.
Verified host-side: cargo build/clippy/fmt --workspace (the ungated stats + JNI
accessor). The android-gated decode.rs (NDK) and the Kotlin build only in CI
(android.yml: gradle + cargo-ndk) -- APIs verified against crate sources.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two bodies of work in one commit (the rename moved files the fixes also touched).
Naming/structure cleanup (pre-launch):
- Host modules m3.rs->punktfunk1.rs, m0.rs->spike.rs; CLI m3-host->punktfunk1-host,
m0->spike; bare `punktfunk-host` now prints help. Types M3Options/M3Source->
Punktfunk1Options/Punktfunk1Source.
- Clients consolidated out of crates/ into clients/: punktfunk-client-rs->
clients/probe (crate punktfunk-probe), client-linux->clients/linux,
client-windows->clients/windows, punktfunk-android->clients/android/native
(crate punktfunk-client-android; kept [lib] name=punktfunk_android so the JNI
contract is unchanged). crates/ now holds only core + host.
- Milestone codes M0-M4 purged from code/CLI/CLAUDE.md/README/docs/docs-site,
kept only in docs/implementation-plan.md. docs/m2-plan.md->
docs/gamestream-host-plan.md. CI/gradle/flatpak paths updated.
Client loss-recovery (video froze and never recovered after a brief drop):
- Export punktfunk_connection_frames_dropped through the C ABI (the core already
tracked it for the client keyframe-recovery loop; it was never reachable from
the ABI clients). Regenerated punktfunk_core.h.
- Apple (StreamPump + Stage2Pipeline) and Android (decode.rs) now poll
frames_dropped and request a keyframe when it climbs -- the same loss-driven
recovery Linux/Windows already had. Under infinite GOP the decoder silently
conceals reference-missing frames, so the decode-error trigger rarely fires.
Apple rumble robustness (worked then went spotty -- DualSense + Xbox):
- Add CHHapticEngine stopped/reset handlers (rebuild on app background / audio
interruption / server reset) and drop the permanent `broken` latch on a
transient drive failure; latch only when the controller truly has no haptics.
- Surface swallowed SDL set_rumble errors on Linux/Windows + diagnostic logging.
Verified: cargo build/clippy/fmt --workspace, C-ABI harness, header drift.
Not runnable on this box (verify in CI): Gitea workflows, gradle/Android,
flatpak, Swift/decky.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler