Clients derive Y'CbCr->RGB from the stream's SIGNALED matrix x range x depth
via shared csc rows (Rust csc_rows + Swift CscRows) instead of hardcoded
709/2020 - a BT.601-signaled stream (a Linux host's RGB-input NVENC) no longer
renders with a constant hue error. Host-side signaling made honest across
NVENC/VAAPI/openh264/GameStream and the session plan's chroma/bit-depth.
Decoded color-bar fixtures (601/709 x limited/full) pin the math in tests on
both cores.
Same presenter, tvOS HDR: tvOS has no Metal EDR API and a bare PQ colorspace
tag composites UNTONE-MAPPED (the "overblown" Apple TV report), so HDR now
splits on the display's live EDR headroom - PQ passthrough when the
per-session AVDisplayManager mode switch landed (a real HDR10 output
tone-maps itself), else an in-shader PQ->SDR tone-map (203-nit reference
white, extended-Reinhard 1000-nit knee, 2020->709) into the proven SDR layer
config. The 10-bit stream keeps its full decode depth either way.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Live-diagnosed on the RTX box during the adaptive-bitrate A/B: 2 of 3 streams died
with VK_ERROR_DEVICE_LOST at stream start or at a mid-stream encoder rebuild, then
zombied — the demote-to-software path rebuilt the decoder against the same dead
device, FFmpeg wedged inside the rebuild, and the pump flushed a never-draining
backlog every 2 s forever. No OS TDR: the client's own Vulkan misuse. Two causes:
1. The presenter creates ONE graphics-family queue and hands FFmpeg's
AVVulkanDeviceContext the same family (nb_graphics_queues=1 ⇒ queue 0) for its
transfer/compute prep — so the pump thread and the presenter thread submitted to
the SAME VkQueue with no shared lock, violating vkQueueSubmit's external-sync
rule exactly when FFmpeg puts work on the graphics queue (decoder open /
frames-context rebuild = stream start and every ABR encoder re-target). New
guard-less QueueLock (FFmpeg's lock_queue/unlock_queue callbacks are a raw pair)
shared by all four queue users: FFmpeg (callbacks installed via user_opaque),
the presenter's submit/present/wait-idle, and the Skia overlay's flushes.
2. Swapchain recreation destroyed the old swapchain + render semaphores after ONE
fence cycle — the fence proves our submit, not the presentation engine's
semaphore consumption (VUID-vkDestroySemaphore-05149 +
VUID-vkDestroySwapchainKHR-01282 on every recreate). Recreate now drains the
queue (vkQueueWaitIdle under the shared lock — safe now that FFmpeg honours it)
and destroys immediately; the deferred DisplayGarbage machinery is gone.
Resilience: VK_ERROR_DEVICE_LOST anywhere in a present error chain is now fatal —
the run loop fails the session loudly instead of demoting to software on a dead
device (the zombie path).
Verified on the RTX box (RTX 4090 → host .21) under VK_LAYER_KHRONOS_validation:
3/3 stream start/stop cycles clean, then 8 mid-stream encoder rebuilds in one
session (4 ABR down-steps under 10% induced loss + 4 clean-link recovery up-steps
— the exact scenarios that previously killed the device): 0 device losses, 0
wedges, both recreate VUIDs gone (previously fired on every path). Remaining
validation messages are FFmpeg's own video-session VUIDs, untouched by this
change. Linux: clippy -D warnings + tests green (home-worker-2).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The vendor-agnostic DXVA path for GPUs without Vulkan Video (Intel's Windows driver foremost,
which previously landed on CPU decode). Ported from the retired WinUI presenter's decoder with
its Intel-safe discipline intact (decode pool stays libavcodec-derived — a hand-built pool
broke Intel at the first SubmitDecoderBuffers), on a decode device LUID-matched to the
presenter's adapter.
Hand-off is a ring of shareable BGRA8 textures (SHARED_NTHANDLE | SHARED_KEYEDMUTEX) filled by
the fixed-function ID3D11VideoProcessor (NV12/P010 → BGRA8, colour spaces from the per-frame
CICP; PQ is tone-mapped to SDR by the processor — HDR-first boxes take Vulkan Video). BGRA is
deliberate: importing a multiplanar NV12 D3D11 texture device-losts on NVIDIA however it is
consumed (plane-view sampling and DMA copy both validation-clean, both TDR — bisected), while
single-plane RGBA D3D11↔Vulkan interop is the path Chromium/ANGLE exercise on every driver.
The presenter imports a slot's NT handle per frame (VK_KHR_external_memory_win32, gated on the
spec-required external-format probe) and blits it into the video image — no CSC pass; the DXGI
keyed mutex (key 0 both sides, drop-tolerant) is the cross-API lock and visibility barrier.
Verified live vs a real host at 5120x1440@240 HEVC on an RTX 4090: 240 fps, e2e 2.7/3.0 ms
p50/p95 under the Khronos validation layer — parity with Vulkan Video (2.6 ms); auto still
resolves vulkan on NVIDIA. PUNKTFUNK_DECODER=d3d11va forces it; import/present failures demote
to software on the existing streak contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The punktfunk-session Vulkan client (clients/linux-session, now clients/session)
builds and runs on Windows; the WinUI shell spawns it for every stream. Verified
live: 10-bit HEVC via Vulkan Video on both AMD (iGPU) and NVIDIA, 5120x1440 at
130 fps / 8 ms end-to-end on the RTX 4090.
- pf-ffvk: Windows bindgen branch (FFMPEG_DIR + PF_FFVK_VULKAN_INCLUDE, no
pkg-config); provisioning fetches Vulkan-Headers (pinned v1.4.309).
- pf-client-core: builds on Windows — WASAPI audio (audio_wasapi.rs, cfg-swapped
via #[path], same surface as the PipeWire twin), VAAPI/dmabuf gated inline
(chain = vulkan -> software), trust reads the WinUI shell's %APPDATA% stores
(parity tests pin both serialized shapes), Settings gains adapter/hdr_enabled
(serde-defaulted; Linux stores unaffected).
- pf-presenter: builds on Windows — dmabuf module Linux-gated; SDL keyboard grab
while captured (Alt+Tab/Win reach the host); pick_device ranks discrete over
integrated (device 0 was the iGPU on hybrid boxes — the silent footgun) and
honors PUNKTFUNK_VK_ADAPTER (the Settings GPU pick, exported by the session).
- run loop: block in one SDL wait woken by input AND decoded frames (a per-
session forwarder pushes a FrameWake user event) instead of a 1 ms poll —
measured 111%% -> 5%% of a core (NVIDIA), 86%% -> 3.5%% (AMD), stats unchanged.
The pump's decode-fence wait became once-per-window sampling (no per-frame
pipeline stall; the stat now shows true backlog).
- pf-console-ui: builds on Windows (skia-safe msvc prebuilts); font lookup falls
through fontconfig aliases to concrete DirectWrite families (Consolas/Segoe UI)
— browse/coverflow works, verified against a live host.
- WinUI shell: session-always via new src/spawn.rs (GTK spawn.rs port —
CREATE_NO_WINDOW, stdout contract, kill handle); the Stream screen is a status
card (chips + stage lines from the child's stats). The legacy in-process
D3D11VA path stays behind Settings "Streaming engine" / PUNKTFUNK_BUILTIN_
STREAM=1 as the A/B baseline until Phase 8 deletes it. SessionParams.video_caps
makes the HDR toggle real.
- clients/linux-session renamed to clients/session (builds for both OSes).
- CI/MSIX: both workflows build/test both bins with widened path filters; the
MSIX ships punktfunk-session.exe. ARM64 session builds --no-default-features
(rust-skia has no aarch64-pc-windows-msvc prebuilts; flip when it does).
A/B on this box (5120x1440 HEVC vs home-worker-5): NVIDIA Vulkan 130 fps / 8 ms
e2e / 1.6 ms decode — clearly better than the built-in path. The AMD iGPU VCN
saturates at ~52 fps where its own D3D11VA does ~70 — Adrenalin Vulkan decode is
slower on APU silicon; discrete RDNA validation gates Phase 8.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`cargo fmt --all --check` (ci.yml) was red on main: the client re-architecture
commits and origin's windows-shortcut commit landed with rustfmt violations
(e.g. a 104-char .with_context line in hyprland.rs, an unsorted mod block in
vdisplay.rs, the input.rs `{`-placement CI flagged). Reformat the tree so the
fmt gate passes; no functional changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The client now advertises VIDEO_CAP_10BIT|HDR and carries the result all
the way to glass:
- csc_rows is bit-depth exact (10-bit limited code points differ from
8-bit by ~half a code) and folds in the P010/X6 MSB-packing factor;
new 10-bit white/black tests.
- The CSC shader grows a params block: mode 0 passes the transfer
through (SDR as-is, or PQ onto an HDR10 swapchain); mode 1 tonemaps
PQ→SDR in-shader (ST.2084 EOTF, 203-nit reference white exposure,
BT.2020→709, soft maxRGB rolloff, sRGB encode) for desktops without
an HDR surface. PUNKTFUNK_TONEMAP_PEAK tunes the rolloff.
- The presenter probes VK_EXT_swapchain_colorspace + an HDR10/ST.2084
10-bit surface format and flips modes in-band with the stream's PQ
signaling: fence-quiesce, then CSC pass + video image (10-bit
A2B10G10R10 intermediate — PQ in 8 bits bands) + overlay pipe +
swapchain rebuild through the deferred-destroy rules.
- P010 decodes through all three paths: Vulkan Video (X6 multiplanar
pool, R10X6 plane views), VAAPI dmabuf (R16/RG1616 plane imports),
software (swscale as before).
- session pump advertises the caps; the host still gates Main10 behind
its PUNKTFUNK_10BIT policy.
Probed on glass hardware: the KDE/NVIDIA surface exposes
A2B10G10R10+HDR10_ST2084, so true PQ passthrough is available there.
Known v1 gaps: software-decode PQ shows untonemapped (8-bit RGBA
carries the transfer baked); the SDR overlay composites unscaled onto
an HDR10 surface (dim OSD); no vkSetHdrMetadataEXT yet.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Vulkan path's receive_frame returns at SUBMISSION (~0.1 ms) — the
hardware decodes asynchronously, so the decode stat was truthful but
measured the wrong boundary. The pump now ships the frame to the
presenter FIRST, then waits the frame's timeline fence (vkWaitSemaphores
resolved through the shared device's proc chain) and stamps
received→decode-COMPLETE — true NVDEC time at zero pipeline cost, since
the presenter's own GPU wait is what actually gates sampling. Software/
VAAPI keep their synchronous stamps.
Also: Alt+Enter joins F11 as the fullscreen toggle (some keyboards' Fn
layer sends a media key for plain F11 — observed on glass as
'F11 only works with shift'); the shell's shortcuts panel lists both.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The silent settings-driven software path cost a debugging round on the
first Vulkan Video glass test (stale decoder=software from the VAAPI-
broken-on-NVIDIA era) — now it says so. Module docs updated: auto is
vulkan → vaapi → software.
Validated on glass (RTX 5070 Ti, HEVC 4K@144): decode 11 ms → 0.1 ms,
e2e p50 ~115 ms → 8.6 ms (the old number was software-decode queueing,
not clock skew), 144 fps locked, 2686 frames, zero errors.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
FFmpeg's Vulkan Video decoder now runs on the PRESENTER's own VkDevice —
the decoded VkImage feeds the existing CICP CSC pass directly: zero
copy, no interop, and NVIDIA gets hardware decode for the first time
(its VAAPI is unusable by design). One decode architecture for every
vendor going forward; VAAPI-dmabuf and software remain the fallbacks
(auto: vulkan → vaapi → software; PUNKTFUNK_DECODER=vulkan pins it).
Presenter: instance 1.3; probes VK_KHR_video_queue/decode_queue + codec
extensions, a VIDEO_DECODE queue family (+ its codec caps via
QueueFamilyVideoPropertiesKHR), and the samplerYcbcrConversion/
timelineSemaphore/synchronization2 features — all enabled at device
creation when present, exported as a VulkanDecodeDevice handle bundle.
Decoder: AVVulkanDeviceContext built over those handles via pf-ffvk
(features chain, extension lists, deprecated queue indices + the qf[]
map); get_format supplies OUR frames context with MUTABLE_FORMAT so the
presenter's per-plane views are legal; output is DecodedImage::VkFrame
carrying AVVkFrame/frames-ctx pointers plus the lock fns.
Present: R8+R8G8 plane views over the multiplanar image, the live sync
state read under the AVVulkanFramesContext lock, a timeline-semaphore
wait(sem_value)/signal(sem_value+1) folded into the submit, layout/
queue-family/sem_value written back per FFmpeg's contract, and the frame
guard parked in the retire queue until the fence. CSC pass + video
framebuffer are now unconditional (NVIDIA has no dmabuf-import path).
Verified on the RTX 5070 Ti: device creates with decode_qf=3,
caps=DECODE_H264|H265|AV1|VP9; swapchain unaffected. Live stream
validation next.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Under infinite GOP the pump only re-requested an IDR when the
reassembler's drop count climbed. A lost initial IDR (or a mid-GOP join)
delivers complete-but-undecodable delta frames instead — the reassembler
never drops, so recovery never fired and the stream froze on the last
good frame while libavcodec flooded stderr with missing-reference
errors. Reproduced at 4K@144 (large IDRs, higher loss); lower modes hid
it. Now a 3-frame no-output streak (~50 ms) forces a fresh IDR,
throttled and re-armed across the request→IDR round trip. Verified on
glass: 4K@144 recovers and holds. Also quiets libavcodec's raw stderr
(it bypassed tracing) to fatal-only, PUNKTFUNK_FFMPEG_LOG restores it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Session pump, FFmpeg decode, PipeWire audio, SDL3 gamepads, keymap, trust
store, mDNS discovery, library client and Wake-on-LAN move verbatim from
clients/linux into crates/pf-client-core, shared with the upcoming Vulkan
session binary (punktfunk-planning: linux-client-rearchitecture.md).
The GTK client re-exports them at the crate root so every existing
crate::-path keeps resolving; its manifest drops the moved-only deps.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
enricobuehler