Compare commits

..

7 Commits

Author SHA1 Message Date
enricobuehler dab40ed98e feat(client/windows): PyroWave decode + surface it in the GUI codec picker
The decoder was gated to Linux because, when it landed, the Windows client
still had its own in-process WinUI/D3D11 presenter and the PyroWave present
path there was an open question. That client has since been retired: Windows
now spawns the SAME Vulkan session presenter as Linux, and the decoder is
plain Vulkan compute on the presenter's device (no fds, no dmabuf, no D3D11
interop), so the question that gated it answered itself. pyrowave-sys already
builds on Windows too -- the Windows HOST encoder ships on it.

So this is a port by un-gating: every cfg(all(target_os = "linux", feature =
"pyrowave")) becomes any(linux, windows) -- decoder module, backend variant,
Decoder::new_pyrowave, the CODEC_PYROWAVE advertisement, the session pump's
opt-in/build/label arms, and the presenter's planar CSC pass. No new code.

Then offer it in the Windows GUI, which is what prompted this. It stays
preference-only (resolve_codec never auto-picks it) and a host or device that
can't do PyroWave just falls back down the ladder to HEVC.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 16:10:42 +02:00
enricobuehler 1d587a259e fix(client/d3d11va): green bar at the bottom — clamp the blit to the frame
ci / web (push) Successful in 47s
ci / docs-site (push) Successful in 1m8s
apple / swift (push) Successful in 1m18s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 13s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m49s
ci / bench (push) Successful in 7m2s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
apple / screenshots (push) Successful in 6m12s
deb / build-publish (push) Successful in 9m49s
android / android (push) Successful in 13m29s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m53s
docker / deploy-docs (push) Successful in 27s
flatpak / build-publish (push) Successful in 6m32s
deb / build-publish-host (push) Successful in 13m26s
arch / build-publish (push) Successful in 17m13s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 8m0s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m10s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 21m2s
ci / rust (push) Successful in 28m51s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 9m0s
The decode surface is DXVA-aligned (height rounded up to the profile's
alignment, 128 for HEVC/AV1), so it is taller than the frame: a 2400-line
stream decodes into a 2432-line texture. VideoProcessorBlt was called with
no stream source rect, so it blitted the WHOLE surface — the uninitialized
padding rows (NV12 Y=0,U=V=0, which converts to vivid green) landed at the
bottom of the output and the picture was squashed to fit. Set the source
rect to the real frame.

Pre-existing on this backend, but 40030e90 made D3D11VA auto's first choice
on Intel, so it went from a corner case to what every Intel user sees — a
~32 px green bar at 3840x2400, reported on glass within the hour.

Also dedupe the Settings GPU picker by description: that string IS the
identity downstream (persisted as Settings::adapter, matched by name as
PUNKTFUNK_VK_ADAPTER), so repeated enumerations of one adapter offered the
user the same choice twice — live on an Intel Arc laptop, whose Vulkan ICD
also enumerates the single physical iGPU twice. Both paths now log their
raw enumeration (decode texture dims; DXGI name/LUID/ids) for the next
report.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 14:44:05 +02:00
enricobuehler 1436c0b9b4 docs(status): Windows HDR10 present validated on glass (Intel D3D11VA pass-through)
ci / web (push) Successful in 48s
ci / bench (push) Successful in 5m54s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
ci / docs-site (push) Successful in 57s
apple / swift (push) Successful in 1m14s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
decky / build-publish (push) Successful in 18s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 46s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
apple / screenshots (push) Successful in 6m7s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
deb / build-publish (push) Successful in 9m34s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m35s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
flatpak / build-publish (push) Failing after 8m3s
docker / deploy-docs (push) Successful in 27s
arch / build-publish (push) Successful in 11m45s
deb / build-publish-host (push) Successful in 12m2s
android / android (push) Successful in 16m10s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m50s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m39s
ci / rust (push) Successful in 23m19s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m5s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 8m13s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 9m6s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 14:31:34 +02:00
enricobuehler fb8deb31a5 feat(client): HDR pass-through on the D3D11VA path
A PQ stream on the D3D11VA backend was always tone-mapped to sRGB by the
VideoProcessor — with D3D11VA now auto's first choice on Intel (40030e90),
Intel Windows users would have lost HDR entirely. When the presenter can
import an RGB10A2 D3D11 texture AND offers an HDR10 swapchain (the new
VulkanDecodeDevice::d3d11_hdr10 probe), the hand-off ring switches to
RGB10A2 and the VideoProcessor does a pure colorspace conversion (YCbCr
G2084 -> RGB G2084, no tone mapping); the emitted frame carries PQ/BT.2020
color, so the presenter flips its HDR10 swapchain and video image exactly
as it does for Vulkan Video PQ frames, and the blit passes the PQ values
through untouched. SDR-only paths keep the tonemap-to-sRGB BGRA8 ring;
in-band PQ flips rebuild the ring like a resize (generation-bumped).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 14:19:35 +02:00
enricobuehler 40030e90c8 fix(client): vendor-aware Windows decode order — D3D11VA first on Intel
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 24s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 6m9s
ci / bench (push) Successful in 6m51s
apple / screenshots (push) Successful in 6m14s
flatpak / build-publish (push) Successful in 6m57s
deb / build-publish-host (push) Successful in 10m19s
docker / deploy-docs (push) Successful in 28s
deb / build-publish (push) Successful in 11m30s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m38s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m56s
arch / build-publish (push) Successful in 15m27s
android / android (push) Successful in 16m10s
ci / web (push) Successful in 58s
ci / rust (push) Successful in 20m46s
ci / docs-site (push) Successful in 1m2s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 8m0s
apple / swift (push) Successful in 1m19s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m15s
decky / build-publish (push) Successful in 19s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m58s
Intel's Windows driver advertises Vulkan Video (Arc drivers since 2023), so
the capability gate alone no longer keeps Intel off FFmpeg-Vulkan — and that
combination is broken in the field (B580 report: strobing + 7.2 ms p50
decodes at 4K120) and on glass (Arc Pro iGPU: 29-33 ms p50 decodes at 4K60
where D3D11VA does 2.5 ms on the SAME GPU). Mirror the Linux vendor order on
Windows: NVIDIA/AMD keep Vulkan Video first, Intel/unknown take D3D11VA
first; vulkan stays reachable by explicit preference and as auto's fallback.

Also:
- Vulkan→D3D11VA mid-session demotion rung (the Windows analog of Linux's
  Vulkan→VAAPI rung) — a failing Vulkan backend lands on hardware, not on
  software, which cannot survive 4K120.
- The demotion streak now needs 1 s of age as well as 3 consecutive errors:
  a startup loss burst produced 3 errors in 20 ms and stranded the session
  on software decode (one-way) before the IDR requested on the first error
  could possibly arrive — live-hit on the Intel iGPU leg.
- Stale "Intel's Windows driver has no Vulkan Video" comments corrected,
  docs updated to the per-vendor order, vendor-order test extended with
  discrete/iGPU Arc cases.

Verified on an Arc Pro iGPU + RTX 3500 Ada laptop against a CachyOS/NVENC
host: auto now picks d3d11va on Intel (60 fps, 2.5-2.8 ms decode, e2e 26 ms
vs 50 ms before) and still picks vulkan on NVIDIA. 27/27 pf-client-core
tests, clippy clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 14:04:32 +02:00
enricobuehler 2149673f89 fix(apple): default iOS to glass-gated present pacing with a depth-2 gate — the 23 ms display-stage fix
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 6m52s
release / apple (push) Successful in 8m46s
deb / build-publish (push) Successful in 11m40s
docker / deploy-docs (push) Successful in 24s
arch / build-publish (push) Successful in 12m42s
deb / build-publish-host (push) Successful in 12m6s
android / android (push) Successful in 13m55s
apple / screenshots (push) Successful in 6m36s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m39s
ci / rust (push) Successful in 24m30s
ci / docs-site (push) Successful in 50s
ci / web (push) Successful in 54s
apple / swift (push) Successful in 1m15s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
Field report (iPad Pro 13" M4, 2752x2064@120): display 23.1 ms on the default
arrival pacing vs 14 ms glass-gated, dominating an otherwise ~14 ms pipeline.
On iOS the layer ALWAYS vsync-latches (displaySyncEnabled is macOS-only API)
and default-on VRR steers the panel to the stream rate, so arrival pacing's
sticky-FIFO saturation (~2-3 refreshes of queue) is the common case, not the
corner — the exact regime that made tvOS default to glass.

- PresenterChoice.platformDefault -> stage3 on iOS/iPadOS (joins tvOS);
  explicit stage-2 stays the honest arrival A/B; macOS unchanged.
- PresentGate generalized to a capacity: depth 1 is bit-identical to before;
  iOS runs depth 2 (one flip scanning out + one queued for the next latch),
  so a decoded frame presents immediately and latches the very next vsync
  instead of serializing on the previous flip's on-glass callback — expected
  ~5-8 ms at 120 Hz. tvOS keeps depth 1 (proven; A/B first), macOS is pinned
  to 1 (glass there is the DCP swapID-panic mitigation — serialization is
  its point). PUNKTFUNK_GATE_DEPTH (1-3) is the on-device A/B lever.
- Settings picker derives its "(default)" marker from presenterDefault
  instead of hardcoding stage-2 as default / stage-3 as experimental.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 12:42:34 +02:00
enricobuehler c4d6e6a877 ci(arch): install ninja — skia-bindings needs it to build Skia from source
apple / screenshots (push) Successful in 6m18s
ci / bench (push) Successful in 6m19s
decky / build-publish (push) Successful in 18s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
deb / build-publish (push) Successful in 10m51s
arch / build-publish (push) Successful in 14m44s
docker / deploy-docs (push) Successful in 23s
android / android (push) Successful in 16m1s
deb / build-publish-host (push) Successful in 10m33s
ci / rust (push) Successful in 18m59s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 13m43s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m59s
apple / swift (push) Successful in 1m16s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 51s
The Arch package build died in skia-bindings:

    failed to run `ninja`, does it exist in PATH?: Os { code: 2, kind: NotFound }

pacman got cmake and nasm but never ninja, which skia-bindings' GN/Ninja
generator shells out to. The job only ever passed because a warm cargo cache
carried a prebuilt Skia, so it never reached the source build — one cache
eviction from breaking, which is exactly what happened on the v0.14.0 retag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 12:01:43 +02:00
22 changed files with 647 additions and 227 deletions
+1 -1
View File
@@ -42,7 +42,7 @@ jobs:
- name: Install build + runtime-dev deps
run: |
pacman -Syu --noconfirm --needed \
git nodejs rust clang cmake nasm pkgconf python vulkan-headers \
git nodejs rust clang cmake ninja nasm pkgconf python vulkan-headers \
gtk4 libadwaita sdl3 ffmpeg pipewire wayland libxkbcommon opus libei \
mesa libglvnd unzip libarchive
# bun builds the punktfunk-web console + the punktfunk-scripting runner AND is vendored as
@@ -50,14 +50,14 @@ enum SettingsOptions {
return options
}
/// The platform's presenter default (mirrors SessionPresenter's platformDefault tvOS runs
/// glass pacing, everything else arrival). Views seed their @AppStorage display from this so
/// an untouched picker shows what actually runs.
/// The platform's presenter default (mirrors SessionPresenter's platformDefault tvOS and
/// iOS/iPadOS run glass pacing, macOS arrival). Views seed their @AppStorage display from
/// this so an untouched picker shows what actually runs.
static var presenterDefault: String {
#if os(tvOS)
"stage3"
#else
#if os(macOS)
"stage2"
#else
"stage3"
#endif
}
@@ -412,27 +412,28 @@ extension SettingsView {
#endif
}
// Stage-2 (Metal/VTDecompressionSession, present on frame arrival) is the proven default;
// stage-3 is the same pipeline with glass-gated present pacing a user-visible A/B while the
// pacing work settles (see Stage2Pipeline's PresentPacing for the queue-saturation rationale).
// Stage-1 (compressed video straight to the system layer) stays a DEBUG-only diagnostic it
// freezes hard on a lost HEVC reference.
// Stage-2 (Metal/VTDecompressionSession, present on frame arrival) vs stage-3 (the same
// pipeline with glass-gated present pacing). The default is per-platform glass on iOS/tvOS,
// whose layers always vsync-latch, arrival on macOS (see Stage2Pipeline's PresentPacing for
// the queue-saturation rationale) so the "(default)" marker rides presenterDefault instead
// of a hardcoded row. Stage-1 (compressed video straight to the system layer) stays a
// DEBUG-only diagnostic it freezes hard on a lost HEVC reference.
@ViewBuilder var presenterSection: some View {
Section {
Picker("Presenter", selection: $presenter) {
Text("Stage 2 (default)").tag("stage2")
Text("Stage 3 (experimental)").tag("stage3")
#if DEBUG
Text("Stage 1 (debug)").tag("stage1")
#endif
ForEach(SettingsOptions.presenters, id: \.tag) { option in
Text(option.tag == SettingsOptions.presenterDefault
? "\(option.label) (default)" : option.label)
.tag(option.tag)
}
}
} header: {
Text("Video presenter")
} footer: {
Text("Stage 2: each frame is shown the moment it's decoded — proven, but on displays "
Text("Stage 2: each frame is shown the moment it's decoded — but on displays "
+ "running near the stream's frame rate, queued frames can add two to three "
+ "refreshes of display latency that never drains. Stage 3: presents are paced "
+ "to the display — at most one undisplayed frame in flight, always the freshest, "
+ "to the display — a strict cap on undisplayed frames, always the freshest, "
+ "dropping late frames instead of queueing them. Watch the statistics overlay's "
+ "display time to compare. Applies from the next session.")
.font(.geist(12, relativeTo: .caption))
@@ -1,8 +1,10 @@
// Per-session presenter stack shared by the macOS and iOS/tvOS stream views: stage-2 (explicit
// VTDecompressionSession decode CAMetalLayer, driven by the hosting view's CADisplayLink) is the
// default; stage-1 (StreamPump AVSampleBufferDisplayLayer) is the Metal-unavailable / DEBUG
// fallback. The views own the platform bits capture, window/scale tracking, and constructing the
// display link and delegate the shared presenter lifecycle here.
// Per-session presenter stack shared by the macOS and iOS/tvOS stream views: the Metal pipeline
// (explicit VTDecompressionSession decode CAMetalLayer, driven by the hosting view's
// CADisplayLink) is the default glass-paced stage-3 on tvOS + iOS, arrival-paced stage-2 on
// macOS (see PresenterChoice.platformDefault); stage-1 (StreamPump AVSampleBufferDisplayLayer)
// is the Metal-unavailable / DEBUG fallback. The views own the platform bits capture,
// window/scale tracking, and constructing the display link and delegate the shared presenter
// lifecycle here.
//
// Main-thread only: start/layout/stop and the display-link tick all run on the main runloop.
@@ -59,13 +61,18 @@ enum PresenterChoice: Equatable {
}
}
/// tvOS defaults to GLASS pacing: an Apple TV is the sticky-FIFO worst case by construction
/// a fixed 60 Hz panel fed a 60 fps stream, where arrival pacing pins the layer's image queue
/// at ~3 drawables and every frame rides ~50 ms of queue (the measured display stage there).
/// The Settings picker can still force stage-2 for an A/B. Everything else keeps stage-2 (the
/// proven default; ProMotion/desktop panels out-tick the stream often enough to drain).
/// tvOS and iOS/iPadOS default to GLASS pacing: their layers ALWAYS vsync-latch presents into
/// the FIFO image queue (`displaySyncEnabled` is macOS-only API), so whenever the panel runs
/// near the stream rate an Apple TV's fixed 60 Hz fed a 60 fps stream by construction; an
/// iPhone/iPad fed a stream at the panel rate, which VRR (default on, preferred = stream rate)
/// makes the COMMON case arrival pacing pins the queue at ~`maximumDrawableCount` and every
/// frame rides ~23 refreshes of it (measured: ~50 ms on Apple TV; 2330 ms at 120 Hz on
/// ProMotion iPads the 2026-07 iPad Pro 2752×2064@120 field report read display 23.1 ms on
/// arrival vs 14 ms glass). The Settings picker can still force stage-2 for an A/B. macOS
/// keeps stage-2: with the layer's sync off, presents are out-of-band flips that don't queue,
/// so arrival is genuinely lowest-latency there.
static var platformDefault: PresenterChoice {
#if os(tvOS)
#if os(tvOS) || os(iOS)
.stage3
#else
.stage2
@@ -99,6 +106,34 @@ final class SessionPresenter {
return .arrival
}
/// The glass gate's in-flight present budget (`PresentGate` capacity) for this platform.
///
/// - iOS/iPadOS: 2 one flip scanning out plus one queued for the next latch. A decoded
/// frame presents immediately (the gate is open in steady state) and latches the very next
/// vsync, so the present cadence never serializes on the on-glass callback's own latency
/// depth 1's extra-refresh cost (the field-measured 14 ms display stage at 120 Hz, vs a
/// ~half-refresh floor). The queue still can't build past two flips, so arrival pacing's
/// FIFO saturation (2330 ms) stays gone.
/// - tvOS: 1 the proven fixed-60-Hz config. Depth 2 should shorten its display stage the
/// same way but is unmeasured there; A/B first (`PUNKTFUNK_GATE_DEPTH=2`), then flip.
/// - macOS: pinned to 1, env ignored glass pacing exists there as the DCP swapID
/// kernel-panic mitigation (see `pacing`), and STRICT present serialization is its point.
///
/// `PUNKTFUNK_GATE_DEPTH` (13) overrides on iOS/tvOS for on-device A/B, mirroring the other
/// presenter env levers. Internal (not private) for unit tests.
static func gateDepth(env: String?) -> Int {
#if os(macOS)
return 1
#else
if let env, let depth = Int(env), (1...3).contains(depth) { return depth }
#if os(tvOS)
return 1
#else
return 2
#endif
#endif
}
private var pump: StreamPump?
private var stage2: Stage2Pipeline?
private var stage2Link: CADisplayLink?
@@ -140,12 +175,13 @@ final class SessionPresenter {
stop()
self.connection = connection
// Presenter choice stage-2 is the DEFAULT (explicit VTDecompressionSession decode + a
// CAMetalLayer/display-link present): it can detect + recover a wedged decoder where
// stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference. Stage-3 is
// the same pipeline with glass-gated present pacing (the settings picker's live A/B see
// PresentPacing). Stage-1 is reachable only via the DEBUG presenter value; release maps it
// back to stage-2 (the stage-1 pump below stays the automatic fallback if Metal is missing).
// Presenter choice the Metal pipeline is the DEFAULT (explicit VTDecompressionSession
// decode + a CAMetalLayer/display-link present): it can detect + recover a wedged decoder
// where stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference. Which
// pacing it defaults to is per-platform (glass-gated stage-3 on tvOS/iOS, arrival stage-2
// on macOS see PresenterChoice.platformDefault); the settings picker is the live A/B.
// Stage-1 is reachable only via the DEBUG presenter value; release maps it back to the
// default (the stage-1 pump below stays the automatic fallback if Metal is missing).
#if DEBUG
let allowStage1 = true
#else
@@ -161,7 +197,9 @@ final class SessionPresenter {
endToEndMeter: endToEndMeter, decodeMeter: decodeMeter,
displayMeter: displayMeter,
pacing: Self.pacing(
for: choice, explicit: explicit, codec: connection.videoCodec)) {
for: choice, explicit: explicit, codec: connection.videoCodec),
gateDepth: Self.gateDepth(
env: ProcessInfo.processInfo.environment["PUNKTFUNK_GATE_DEPTH"])) {
let metal = pipeline.layer
// The opaque metal layer composites OVER the AVSampleBufferDisplayLayer base, which
// sits idle (un-enqueued) in stage-2. contentsScale + frame are set in layout().
@@ -20,7 +20,8 @@
// schedule can never sit far in the future holding drawables hostage.
// Present PACING is the stage-2 vs stage-3 presenter split (`PresentPacing`, chosen per session
// by SessionPresenter from the presenter setting / PUNKTFUNK_PRESENTER): stage-2 presents on
// frame arrival; stage-3 additionally gates presents to ONE undisplayed drawable so the layer's
// frame arrival; stage-3 additionally gates presents to a bounded number of undisplayed
// drawables (the gate depth see PresentPacing + SessionPresenter.gateDepth) so the layer's
// FIFO image queue can never saturate see PresentPacing's doc for the full rationale.
// Rendering lives on its own thread so any `nextDrawable()` wait lands off-main (input, SwiftUI).
//
@@ -107,18 +108,19 @@ private final class VsyncClock: @unchecked Sendable {
/// When a ready frame is pushed to the layer the stage-2 vs stage-3 presenter split. Same decode
/// half, same newest-wins ring; only the present cadence differs.
///
/// - `arrival` (stage-2, the default): present the moment a frame is decoded. Lowest latency while
/// the layer's image queue is shallow but that queue is FIFO and consumed at one drawable per
/// refresh (iOS always vsync-latches; the macOS 26 compositor latch-paces our out-of-band
/// - `arrival` (stage-2, the macOS default): present the moment a frame is decoded. Lowest latency
/// while the layer's image queue is shallow but that queue is FIFO and consumed at one drawable
/// per refresh (iOS always vsync-latches; the macOS 26 compositor latch-paces our out-of-band
/// presents the same way when composited), so at stream rate refresh rate its depth is STICKY:
/// one early burst (session start, a Wi-Fi clump) fills it to `maximumDrawableCount` and with
/// arrivals and latches then running at the same rate it never drains. Every later frame rides
/// ~23 refreshes of queue (the measured 2930 ms display stage on 120 Hz ProMotion panels), and
/// ~23 refreshes of queue (the measured 2330 ms display stage on 120 Hz ProMotion panels), and
/// the full-queue regime is where hostpanel clock drift turns into periodic repeats/drops (the
/// "fixed-interval" jitter reports).
/// - `glass` (stage-3, experimental): at most ONE presented-but-undisplayed drawable in flight
/// (`PresentGate`). The render thread presents only when the previous flip reached glass (the
/// drawable's presented handler reopens the gate and re-signals); frames decoded meanwhile
/// - `glass` (stage-3, the tvOS + iOS default): at most a small BOUNDED number of presented-but-
/// undisplayed drawables in flight (`PresentGate`; the depth 1 or 2 is per-platform, see
/// `SessionPresenter.gateDepth`). The render thread presents only while a gate slot is free (a
/// drawable's presented handler reopens its slot and re-signals); frames decoded meanwhile
/// coalesce in the newest-wins ring. Freshness is preserved by DROPPING stale frames before
/// present instead of queueing them behind the display the hidden queue latency becomes
/// explicit, correct frame drops.
@@ -132,43 +134,58 @@ public enum PresentPacing: Sendable {
case glass
}
/// Stage-3's present gate: admits one in-flight (presented, not yet on glass) drawable. The render
/// thread `tryAcquire`s before taking a frame; the drawable's presented handler `release`s and
/// re-signals the render thread. `staleAfter` is insurance against a present whose handler never
/// fires (the macOS "out-of-band presents aren't damage" hazard class see MetalVideoPresenter's
/// init post-mortem): rather than freezing the stream, a stuck gate force-opens after 100 ms, a
/// visible ~10 fps degradation that PUNKTFUNK_PRESENT_DEBUG's `forced` counter exposes (it reads 0
/// on healthy systems). Internal (not private) for unit tests. Sendable; lock-guarded the
/// releaser runs on a Metal callback thread.
/// Stage-3's present gate: admits `capacity` in-flight (presented, not yet on glass) drawables.
/// The render thread `tryAcquire`s before taking a frame; the drawable's presented handler
/// `release`s and re-signals the render thread. Depth 1 fully serializes presents on the on-glass
/// callback which costs a refresh whenever the callback's own latency pushes the next present
/// past a vsync; depth 2 keeps one flip queued behind the one scanning out, so a decoded frame
/// presents immediately and latches the very next vsync while the queue still can't build (see
/// `SessionPresenter.gateDepth` for the per-platform choice). `staleAfter` is insurance against a
/// present whose handler never fires (the macOS "out-of-band presents aren't damage" hazard class
/// see MetalVideoPresenter's init post-mortem): rather than freezing the stream, a full gate
/// force-opens a slot 100 ms after its oldest present, a visible ~10 fps degradation that
/// PUNKTFUNK_PRESENT_DEBUG's `forced` counter exposes (it reads 0 on healthy systems). Internal
/// (not private) for unit tests. Sendable; lock-guarded the releaser runs on a Metal callback
/// thread.
final class PresentGate: @unchecked Sendable {
/// How long one pending present may hold the gate before it's presumed lost.
/// How long one pending present may hold its slot before it's presumed lost.
static let staleAfter: CFTimeInterval = 0.1
private let lock = NSLock()
private var pending = false
private var armedAt: CFTimeInterval = 0
private let capacity: Int
/// Arm instants of the in-flight presents, oldest first ( `capacity` entries).
private var armed: [CFTimeInterval] = []
private var forced = 0
/// Arm the gate for one present. False = a present is already in flight (and not stale)
/// leave the frame in the ring; the presented handler's release/re-signal (or the next
/// `capacity` = the in-flight present budget (clamped to 1) see the type doc.
init(capacity: Int = 1) {
self.capacity = max(1, capacity)
}
/// Arm the gate for one present. False = the gate is full of live presents (none stale)
/// leave the frame in the ring; a presented handler's release/re-signal (or the next
/// display-link tick) retries with the freshest frame then.
func tryAcquire(now: CFTimeInterval) -> Bool {
lock.lock()
defer { lock.unlock() }
if pending {
guard now - armedAt > Self.staleAfter else { return false }
forced += 1 // presumed-lost present reopen rather than stall the stream
if armed.count >= capacity {
// Full: reopen only by presuming the OLDEST in-flight present lost (its handler
// never fired) rather than stalling the stream.
guard let oldest = armed.first, now - oldest > Self.staleAfter else { return false }
armed.removeFirst()
forced += 1
}
pending = true
armedAt = now
armed.append(now)
return true
}
/// The in-flight present reached glass (or was dropped, or its render failed before a present
/// was registered) reopen. Idempotent: a late stale-path double-release is harmless.
/// One in-flight present reached glass (or was dropped, or its render failed before a present
/// was registered) free the oldest slot. A release with nothing in flight is a no-op; a
/// lost present's handler firing late after its stale force-open can transiently over-admit
/// one flip, which the next glass callback corrects.
func release() {
lock.lock()
pending = false
if !armed.isEmpty { armed.removeFirst() }
lock.unlock()
}
@@ -195,7 +212,7 @@ private final class PresentDebugStats: @unchecked Sendable {
private var glassDeltasMs: [Double] = []
/// Presented-but-not-yet-on-glass drawables right now / the window's peak the direct
/// measurement of the layer image-queue depth the stage-3 gate exists to bound (stage-2 on a
/// 120 Hz panel saturates this at ~maximumDrawableCount; stage-3 should peg it at 1).
/// 120 Hz panel saturates this at ~maximumDrawableCount; stage-3 pegs it at the gate depth).
private var inFlight = 0
private var maxInFlight = 0
@@ -285,6 +302,9 @@ public final class Stage2Pipeline {
/// Present cadence `.arrival` (stage-2) or `.glass` (stage-3, the present gate). Fixed for
/// the pipeline's lifetime; SessionPresenter resolves it per session (see PresentPacing).
private let pacing: PresentPacing
/// The glass gate's in-flight present budget (`PresentGate` capacity) meaningful only under
/// `.glass`; SessionPresenter resolves it per platform (see `SessionPresenter.gateDepth`).
private let gateDepth: Int
private let endToEndMeter: LatencyMeter?
private let decodeMeter: LatencyMeter?
private let displayMeter: LatencyMeter?
@@ -328,16 +348,19 @@ public final class Stage2Pipeline {
/// render + vsync the tail stage-2 exists to shorten). All optional: metering never gates
/// the presenter choice. Returns nil if Metal can't be set up (headless / no GPU) caller
/// falls back to the stage-1 presenter. `pacing` selects the stage-2 (arrival) vs stage-3
/// (glass-gated) present cadence see PresentPacing.
/// (glass-gated) present cadence see PresentPacing; `gateDepth` is the glass gate's
/// in-flight present budget (see `SessionPresenter.gateDepth`).
public init?(
endToEndMeter: LatencyMeter?,
decodeMeter: LatencyMeter? = nil,
displayMeter: LatencyMeter? = nil,
pacing: PresentPacing = .arrival
pacing: PresentPacing = .arrival,
gateDepth: Int = 1
) {
guard let presenter = MetalVideoPresenter.make() else { return nil }
self.presenter = presenter
self.pacing = pacing
self.gateDepth = gateDepth
self.endToEndMeter = endToEndMeter
self.decodeMeter = decodeMeter
self.displayMeter = displayMeter
@@ -534,9 +557,9 @@ public final class Stage2Pipeline {
&& UserDefaults.standard.bool(forKey: DefaultsKey.vsync))
let debugStats = presentDebug ? PresentDebugStats() : nil
let vsyncClock = vsyncClock
// Stage-3's one-in-flight present gate; nil = stage-2's present-on-arrival. A local (like
// the ring) so neither the render thread nor the presented handlers capture `self`.
let gate: PresentGate? = pacing == .glass ? PresentGate() : nil
// Stage-3's bounded in-flight present gate; nil = stage-2's present-on-arrival. A local
// (like the ring) so neither the render thread nor the presented handlers capture `self`.
let gate: PresentGate? = pacing == .glass ? PresentGate(capacity: gateDepth) : nil
let renderThread = Thread {
defer { renderStopped.signal() }
// Every iteration drains its own autorelease pool (`return` = the old `continue`):
@@ -4,13 +4,14 @@ import XCTest
import QuartzCore
@testable import PunktfunkKit
/// Stage-3 present pacing: the one-in-flight `PresentGate` and the stage-1/2/3 `PresenterChoice`
/// resolution (setting + PUNKTFUNK_PRESENTER env override + the release-build stage-1 gate).
/// Stage-3 present pacing: the bounded in-flight `PresentGate` (depth 1 + depth 2), the
/// stage-1/2/3 `PresenterChoice` resolution (setting + PUNKTFUNK_PRESENTER env override + the
/// release-build stage-1 gate), and the per-platform glass-gate depth.
final class PresentPacingTests: XCTestCase {
// MARK: - PresentGate
/// The core invariant: one present in flight. A second acquire while pending must fail (the
/// frame stays in the ring for the presented handler's re-signal); release reopens.
/// The depth-1 invariant: one present in flight. A second acquire while pending must fail
/// (the frame stays in the ring for the presented handler's re-signal); release reopens.
func testGateAdmitsOneInFlightPresent() {
let gate = PresentGate()
XCTAssertTrue(gate.tryAcquire(now: 0), "an idle gate must admit the first present")
@@ -20,6 +21,32 @@ final class PresentPacingTests: XCTestCase {
XCTAssertEqual(gate.drainForced(), 0, "no stale present was force-cleared")
}
/// Depth 2 (the iOS default): a second present may queue behind the flip scanning out the
/// bound only bites at the THIRD. One release (a glass callback) reopens exactly one slot.
func testGateDepthTwoAdmitsTwoInFlightPresents() {
let gate = PresentGate(capacity: 2)
XCTAssertTrue(gate.tryAcquire(now: 0))
XCTAssertTrue(gate.tryAcquire(now: 0.001), "depth 2 must admit a queued second flip")
XCTAssertFalse(gate.tryAcquire(now: 0.002), "the third present must wait for glass")
gate.release()
XCTAssertTrue(gate.tryAcquire(now: 0.003), "one glass callback frees one slot")
XCTAssertFalse(gate.tryAcquire(now: 0.004))
XCTAssertEqual(gate.drainForced(), 0)
}
/// Depth 2 staleness anchors to the OLDEST in-flight present: a full gate stays closed while
/// the oldest is live, force-opens once it ages out, and the younger present keeps its slot.
func testGateDepthTwoForceOpensOnTheOldestStalePresent() {
let gate = PresentGate(capacity: 2)
XCTAssertTrue(gate.tryAcquire(now: 10))
XCTAssertTrue(gate.tryAcquire(now: 10.05))
XCTAssertFalse(gate.tryAcquire(now: 10 + PresentGate.staleAfter - 0.01))
XCTAssertTrue(gate.tryAcquire(now: 10 + PresentGate.staleAfter + 0.01))
XCTAssertEqual(gate.drainForced(), 1)
// The 10.05 present is still live, so the gate is full again right after the force-open.
XCTAssertFalse(gate.tryAcquire(now: 10 + PresentGate.staleAfter + 0.02))
}
/// The lost-handler insurance: a present whose handler never fires (the macOS "presents
/// aren't damage" hazard class) must not freeze the stream past `staleAfter` the gate
/// force-opens and counts the event for the PUNKTFUNK_PRESENT_DEBUG `forced` stat.
@@ -47,11 +74,21 @@ final class PresentPacingTests: XCTestCase {
// MARK: - PresenterChoice
func testPresenterChoiceDefaultsToStage2() {
/// The platform default: glass-paced stage-3 where the layer always vsync-latches (iOS,
/// tvOS arrival pacing saturates the FIFO image queue there), arrival stage-2 on macOS
/// (sync-off presents don't queue). No selection / garbage falls back to it.
func testPresenterChoiceFallsBackToPlatformDefault() {
#if os(macOS)
XCTAssertEqual(PresenterChoice.platformDefault, .stage2)
#else
XCTAssertEqual(PresenterChoice.platformDefault, .stage3)
#endif
XCTAssertEqual(
PresenterChoice.resolve(setting: nil, env: nil, allowStage1: true), .stage2)
PresenterChoice.resolve(setting: nil, env: nil, allowStage1: true),
PresenterChoice.platformDefault)
XCTAssertEqual(
PresenterChoice.resolve(setting: "garbage", env: nil, allowStage1: true), .stage2)
PresenterChoice.resolve(setting: "garbage", env: nil, allowStage1: true),
PresenterChoice.platformDefault)
XCTAssertEqual(
PresenterChoice.resolve(setting: "stage2", env: nil, allowStage1: true), .stage2)
}
@@ -70,14 +107,16 @@ final class PresentPacingTests: XCTestCase {
}
/// Stage-1 (the freeze-prone system-layer diagnostic) resolves only where allowed (DEBUG
/// builds); a leftover "stage1" value in a release build maps back to stage-2.
/// builds); a leftover "stage1" value in a release build maps back to the platform default.
func testPresenterChoiceGatesStage1() {
XCTAssertEqual(
PresenterChoice.resolve(setting: "stage1", env: nil, allowStage1: true), .stage1)
XCTAssertEqual(
PresenterChoice.resolve(setting: "stage1", env: nil, allowStage1: false), .stage2)
PresenterChoice.resolve(setting: "stage1", env: nil, allowStage1: false),
PresenterChoice.platformDefault)
XCTAssertEqual(
PresenterChoice.resolve(setting: nil, env: "stage1", allowStage1: false), .stage2)
PresenterChoice.resolve(setting: nil, env: "stage1", allowStage1: false),
PresenterChoice.platformDefault)
}
/// `explicit` is nil exactly when `resolve` would fall back to the platform default the
@@ -118,5 +157,30 @@ final class PresentPacingTests: XCTestCase {
XCTAssertEqual(
SessionPresenter.pacing(for: .stage3, explicit: nil, codec: .pyrowave), .glass)
}
// MARK: - Glass-gate depth
/// The per-platform in-flight present budget: 2 on iOS/iPadOS (one flip scanning out + one
/// queued the display-latency fix), 1 on tvOS (proven config), 1 pinned on macOS (glass
/// there is the swapID-panic mitigation strict serialization is its point, so the env
/// lever must not widen it). PUNKTFUNK_GATE_DEPTH A/Bs iOS/tvOS; out-of-range/garbage
/// values are ignored.
func testGateDepthPlatformDefaultsAndEnvOverride() {
#if os(macOS)
XCTAssertEqual(SessionPresenter.gateDepth(env: nil), 1)
XCTAssertEqual(SessionPresenter.gateDepth(env: "2"), 1, "macOS is pinned to 1")
#elseif os(tvOS)
XCTAssertEqual(SessionPresenter.gateDepth(env: nil), 1)
XCTAssertEqual(SessionPresenter.gateDepth(env: "2"), 2, "the on-device A/B lever")
#else
XCTAssertEqual(SessionPresenter.gateDepth(env: nil), 2)
XCTAssertEqual(SessionPresenter.gateDepth(env: "1"), 1, "the on-device A/B lever")
#endif
XCTAssertEqual(
SessionPresenter.gateDepth(env: "0"), SessionPresenter.gateDepth(env: nil),
"out-of-range env values fall back to the platform depth")
XCTAssertEqual(
SessionPresenter.gateDepth(env: "garbage"), SessionPresenter.gateDepth(env: nil))
}
}
#endif
+4 -1
View File
@@ -51,6 +51,9 @@ const CODECS: &[(&str, &str)] = &[
("hevc", "HEVC (H.265)"),
("h264", "H.264 (AVC)"),
("av1", "AV1"),
// Preference-only by design: `resolve_codec` never auto-picks PyroWave, and asking for
// it on a host or device that can't do it simply falls back down the ladder to HEVC.
("pyrowave", "PyroWave (wired LAN)"),
];
/// Virtual-pad presets: `(stored value, display label)` — the pad the HOST creates. Same set the
/// GTK client offers; "Automatic" resolves from the physical controller at connect.
@@ -278,7 +281,7 @@ pub(crate) fn settings_page(
s.codec = CODECS[i].0.to_string();
})
.tooltip(
"A soft preference \u{2014} the host falls back to the best codec both sides support.",
"A soft preference \u{2014} the host falls back to the best codec both sides support. PyroWave is the low-latency wavelet codec for a WIRED link: it trades bitrate (hundreds of Mb/s) for near-zero decode time, so it needs gigabit Ethernet.",
);
// Free-form Mb/s (0 = host default) instead of presets, so a speed-test recommendation
// round-trips exactly.
+35 -10
View File
@@ -41,16 +41,41 @@ fn all_adapters() -> Vec<IDXGIAdapter> {
/// Descriptions of the real (hardware, non-WARP) GPUs — the Settings GPU picker's option list.
/// The picker only shows when this has more than one entry.
///
/// **Deduplicated by description**, because the description IS the identity everywhere
/// downstream: the pick is persisted as that string (`Settings::adapter`) and matched by
/// name in the session binary (`PUNKTFUNK_VK_ADAPTER`). So two entries with the same name
/// are one selectable choice however many times DXGI enumerates them — listing it twice
/// only offers the user a meaningless coin flip. Seen live on an Intel Arc laptop
/// (2026-07-19), whose Vulkan ICD likewise enumerates the one physical iGPU twice.
pub fn adapter_names() -> Vec<String> {
const DXGI_ADAPTER_FLAG_SOFTWARE: u32 = 2; // dxgi.h; not in this windows-rs feature set
all_adapters()
.iter()
.filter(|a| {
a.cast::<windows::Win32::Graphics::Dxgi::IDXGIAdapter1>()
.and_then(|a1| unsafe { a1.GetDesc1() })
.map(|d| d.Flags & DXGI_ADAPTER_FLAG_SOFTWARE == 0)
.unwrap_or(true)
})
.map(adapter_name)
.collect()
let mut names: Vec<String> = Vec::new();
for a in all_adapters() {
let desc1 = a
.cast::<windows::Win32::Graphics::Dxgi::IDXGIAdapter1>()
.and_then(|a1| unsafe { a1.GetDesc1() })
.ok();
let name = adapter_name(&a);
// Forensics for the next duplicate/oddity report — which adapters DXGI actually
// returned, and whether the repeats share a LUID (one adapter enumerated twice)
// or are distinct devices that merely present the same description.
if let Some(d) = &desc1 {
tracing::debug!(
name = %name,
luid = format!("{:08x}-{:08x}", d.AdapterLuid.HighPart, d.AdapterLuid.LowPart),
vendor = format_args!("{:#06x}", d.VendorId),
device = format_args!("{:#06x}", d.DeviceId),
flags = d.Flags,
"DXGI adapter"
);
}
if desc1.is_some_and(|d| d.Flags & DXGI_ADAPTER_FLAG_SOFTWARE != 0) {
continue; // WARP / software renderer — never a streaming target
}
if !names.contains(&name) {
names.push(name);
}
}
names
}
+8 -5
View File
@@ -24,6 +24,14 @@ ffmpeg-next = "8"
opus = "0.3"
mdns-sd = "0.20"
# PyroWave decode (the opt-in wired-LAN wavelet codec, design/pyrowave-codec-plan.md
# §4.5) — pure Vulkan compute on the presenter's shared device, so it builds wherever the
# spawned Vulkan session presenter runs: Linux AND Windows (pyrowave-sys covers both; it
# is an empty stub elsewhere). `ash` only wraps the presenter's existing raw handles
# (same pinned version as pf-presenter).
pyrowave-sys = { path = "../pyrowave-sys", optional = true }
ash = { version = "0.38", optional = true }
# Game-library fetch from the host's management API over mTLS + fingerprint pinning.
# `ureq` is small + sync (the host uses it too) and its rustls unifies with the
# workspace's (quinn's) 0.23; the pinning verifier mirrors core's private `PinVerify`.
@@ -40,11 +48,6 @@ tracing = "0.1"
[target.'cfg(target_os = "linux")'.dependencies]
pipewire = "0.9"
sdl3 = { version = "0.18", features = ["hidapi"] }
# PyroWave decode (the opt-in wired-LAN wavelet codec, design/pyrowave-codec-plan.md
# §4.5) — pure Vulkan compute on the presenter's shared device. `ash` only wraps the
# presenter's existing raw handles (same pinned version as pf-presenter).
pyrowave-sys = { path = "../pyrowave-sys", optional = true }
ash = { version = "0.38", optional = true }
[target.'cfg(windows)'.dependencies]
wasapi = "0.23"
+6 -3
View File
@@ -41,11 +41,14 @@ mod video_software;
mod video_vaapi;
#[cfg(any(target_os = "linux", windows))]
mod video_vulkan;
// PyroWave decode — Linux + `pyrowave` feature only (plan §4.5; the Windows client's
// present-path decision and the Apple Metal port are their own phases).
// PyroWave decode — Linux + Windows (plan §4.5; the Apple Metal port is its own phase).
// Windows joined once its client moved to the SAME spawned Vulkan session presenter as
// Linux's: the decoder is plain Vulkan compute on the presenter's device (no fds, no
// dmabuf, no D3D11 interop), so the old "Windows present-path decision" that gated it
// resolved itself — the present path is now literally the same code.
#[cfg(windows)]
pub mod video_d3d11;
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
pub mod video_pyrowave;
pub mod wol;
+8 -5
View File
@@ -227,7 +227,7 @@ fn pump(
// the plan-§3 contract: the host only ever picks PyroWave when the client names it.
#[allow(unused_mut)]
let mut preferred = params.preferred_codec;
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
if std::env::var("PUNKTFUNK_PREFER_PYROWAVE").as_deref() == Ok("1") {
if params.vulkan.as_ref().is_some_and(|v| v.pyrowave_decode) {
preferred = punktfunk_core::quic::CODEC_PYROWAVE;
@@ -296,7 +296,7 @@ fn pump(
// A negotiated PyroWave session decodes on the presenter's device, no FFmpeg —
// reachable only through the explicit preference above (resolve_codec never
// auto-picks the bit), so failing loudly here is failing an opted-in experiment.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
let built = if connector.codec == punktfunk_core::quic::CODEC_PYROWAVE {
let mode = connector.mode();
// The wavelet bitstream has no VUI: the negotiated Welcome colour signalling IS
@@ -325,7 +325,7 @@ fn pump(
} else {
Decoder::new(codec_id, &params.decoder, params.vulkan.as_ref())
};
#[cfg(not(all(target_os = "linux", feature = "pyrowave")))]
#[cfg(not(all(any(target_os = "linux", windows), feature = "pyrowave")))]
let built = Decoder::new(codec_id, &params.decoder, params.vulkan.as_ref());
let mut decoder = match built {
Ok(d) => d,
@@ -528,7 +528,7 @@ fn pump(
DecodedImage::VkFrame(_) => "vulkan",
#[cfg(windows)]
DecodedImage::D3d11(_) => "d3d11va",
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
DecodedImage::PyroWave(_) => "pyrowave",
};
if total_frames == 1 {
@@ -539,7 +539,10 @@ fn pump(
DecodedImage::VkFrame(v) => (v.width, v.height, "vulkan-video"),
#[cfg(windows)]
DecodedImage::D3d11(d) => (d.width, d.height, "d3d11va"),
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(
any(target_os = "linux", windows),
feature = "pyrowave"
))]
DecodedImage::PyroWave(f) => (f.width, f.height, "pyrowave"),
};
tracing::info!(width = w, height = h, path, "first frame decoded");
+191 -54
View File
@@ -1,9 +1,10 @@
//! Video decode: reassembled HEVC access units → frames for the presenter.
//!
//! Three backends, picked at session start (auto on Linux: vaapi → vulkan → software on
//! desktop Mesa, vulkan first on NVIDIA/VanGogh — see
//! [`VulkanDecodeDevice::prefer_vulkan_over_vaapi`];
//! override: `PUNKTFUNK_DECODER=vulkan|vaapi|software`):
//! Three backends, picked at session start (auto is vendor-ordered on BOTH desktop OSes —
//! see [`VulkanDecodeDevice::prefer_vulkan_first`]. Linux: vaapi → vulkan → software on
//! desktop Mesa, vulkan first on NVIDIA/VanGogh. Windows: d3d11va → vulkan → software on
//! Intel/unknown, vulkan first on NVIDIA/AMD.
//! Override: `PUNKTFUNK_DECODER=vulkan|vaapi|d3d11va|software`):
//!
//! * **Vulkan Video**: FFmpeg's Vulkan decoder running on the PRESENTER's own VkDevice
//! (its handles arrive via [`VulkanDecodeDevice`]) — the decoded VkImage feeds the
@@ -22,9 +23,12 @@
//! B-frames, in-band parameter sets on every IDR), so decode is strictly one-in/one-out.
//!
//! On Windows the VAAPI/dmabuf backend does not exist (DRM-PRIME is a Linux concept); the
//! chain there is Vulkan **D3D11VA** (`crate::video_d3d11` — the vendor-agnostic DXVA
//! path, which is how Intel's Windows driver gets hardware decode without Vulkan Video)
//! → software. Everything dmabuf-shaped is `cfg(target_os = "linux")`-gated inline.
//! hardware pair there is Vulkan Video and **D3D11VA** (`crate::video_d3d11` — the
//! vendor-agnostic DXVA path every Windows video player exercises), ordered per vendor:
//! Intel's driver DOES advertise Vulkan Video (Arc drivers since 2023), but FFmpeg-Vulkan
//! on it strobes and burns the frame budget (B580 field report, 2026-07) where D3D11VA
//! streams clean — so Intel/unknown take D3D11VA first and NVIDIA/AMD keep Vulkan first.
//! Everything dmabuf-shaped is `cfg(target_os = "linux")`-gated inline.
// bindgen's C-enum repr is target-dependent (u32 on Linux/clang, i32 on MSVC), so the
// pf-ffvk Vulkan flag/enum casts below are required on one platform and no-ops on the
@@ -73,7 +77,7 @@ pub enum DecodedImage {
/// PyroWave planar output: three R8 plane views on the presenter's own device,
/// decode already fence-complete, GENERAL layout — the presenter's planar CSC
/// samples them directly (BT.709 limited, the codec's fixed colour contract).
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
PyroWave(crate::video_pyrowave::PyroWavePlanarFrame),
}
@@ -151,7 +155,7 @@ impl DecodedImage {
DecodedImage::VkFrame(f) => f.keyframe,
#[cfg(windows)]
DecodedImage::D3d11(f) => f.keyframe,
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
DecodedImage::PyroWave(f) => f.keyframe,
}
}
@@ -167,7 +171,7 @@ impl DecodedImage {
DecodedImage::VkFrame(f) => (f.width, f.height),
#[cfg(windows)]
DecodedImage::D3d11(f) => (f.width, f.height),
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
DecodedImage::PyroWave(f) => (f.width, f.height),
}
}
@@ -234,9 +238,10 @@ enum Backend {
#[cfg(windows)]
D3d11va(crate::video_d3d11::D3d11vaDecoder),
/// PyroWave (wired-LAN wavelet codec): pyrowave compute on the presenter's device,
/// no FFmpeg involvement. No demotion rung — there is no other decoder for it.
/// no FFmpeg involvement (Linux + Windows — same Vulkan presenter on both). No demotion
/// rung — there is no other decoder for it.
/// Boxed: the decoder (pinned create-info hold + plane ring) dwarfs the other variants.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
PyroWave(Box<crate::video_pyrowave::PyroWaveDecoder>),
Software(SoftwareDecoder),
}
@@ -250,16 +255,42 @@ pub struct Decoder {
/// (e.g. a reference-missing frame after packet loss) shouldn't cost the whole
/// session its hardware decoder.
vaapi_fails: u32,
/// When the current error streak started. Demotion needs the streak to be OLD as well
/// as long: one startup loss burst produces 3+ consecutive failing AUs within
/// milliseconds — demoting on count alone (live-hit: Intel iGPU, 2026-07-19, three
/// errors in 20 ms → software forever) never gives the IDR requested on the FIRST
/// error (~100300 ms round trip) a chance to rescue the hardware decoder.
first_fail: Option<std::time::Instant>,
/// Set when the decoder needs a fresh IDR to resynchronize (after an error or a demotion).
/// The pump drains it and asks the host — under the infinite GOP there is no periodic
/// keyframe, so a rebuilt/erroring decoder would otherwise stay gray/frozen forever.
want_keyframe: bool,
/// The presenter has the win32 external-memory import path, so D3D11VA frames can reach
/// the screen — kept for the mid-session Vulkan→D3D11VA demotion rung (the Windows
/// analog of Linux's Vulkan→VAAPI rung).
#[cfg(windows)]
d3d11_import: bool,
/// The presenter adapter's LUID (see [`VulkanDecodeDevice::adapter_luid`]) so a demotion
/// rebuild lands on the SAME GPU.
#[cfg(windows)]
adapter_luid: Option<[u8; 8]>,
/// [`VulkanDecodeDevice::d3d11_hdr10`], for the same demotion rebuild.
#[cfg(windows)]
d3d11_hdr10: bool,
}
/// Demote VAAPI→software only after this many consecutive hardware decode errors; a lone
/// transient error just re-requests an IDR and keeps the hardware decoder.
/// Demote a hardware backend (Vulkan→VAAPI/D3D11VA, VAAPI/D3D11VA→software) only after
/// this many consecutive decode errors; a lone transient error just re-requests an IDR
/// and keeps the hardware decoder.
const VAAPI_DEMOTE_AFTER: u32 = 3;
/// ...AND only when the streak has lasted this long. Every error re-requests an IDR, and
/// one arriving + decoding resets the streak — so a genuinely broken driver (errors keep
/// flowing through multiple IDR cycles) still demotes ~a second in, while a burst of
/// consecutive bad AUs from a single loss event no longer strands the session on
/// software before the first requested IDR could even arrive.
const HW_DEMOTE_MIN_STREAK: std::time::Duration = std::time::Duration::from_millis(1000);
/// Map a negotiated `quic` codec bit to the FFmpeg decoder id the client opens.
pub fn ffmpeg_codec_id(wire: u8) -> ffmpeg::codec::Id {
match wire {
@@ -292,11 +323,11 @@ pub fn decodable_codecs() -> u8 {
/// under its explicit opt-in.
pub fn decodable_codecs_for(vk: Option<&VulkanDecodeDevice>) -> u8 {
let bits = decodable_codecs();
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
if vk.map(|v| v.pyrowave_decode).unwrap_or(false) {
return bits | punktfunk_core::quic::CODEC_PYROWAVE;
}
#[cfg(not(all(target_os = "linux", feature = "pyrowave")))]
#[cfg(not(all(any(target_os = "linux", windows), feature = "pyrowave")))]
let _ = vk;
bits
}
@@ -328,10 +359,12 @@ impl Decoder {
/// Vulkan Video decoder — decode lands as VkImages the presenter samples directly.
/// Precedence: the `PUNKTFUNK_DECODER` env override wins (support/debug escape
/// hatch, and the documented knob), then the setting; both default to auto.
/// Auto's hardware order on Linux depends on the device
/// ([`VulkanDecodeDevice::prefer_vulkan_over_vaapi`]): VAAPI → Vulkan → software on
/// Auto's hardware order depends on the device on BOTH desktop OSes
/// ([`VulkanDecodeDevice::prefer_vulkan_first`]). Linux: VAAPI → Vulkan → software on
/// desktop Mesa (AMD/Intel), Vulkan → VAAPI → software on NVIDIA and the Deck's
/// VanGogh. Windows is Vulkan → D3D11VA → software (no VAAPI there).
/// VanGogh. Windows (no VAAPI there): Vulkan → D3D11VA → software on NVIDIA/AMD,
/// D3D11VA → Vulkan → software on Intel/unknown (Intel's driver advertises Vulkan
/// Video, but FFmpeg-Vulkan on it strobes/overruns the budget — B580 field report).
pub fn new(
codec_id: ffmpeg::codec::Id,
pref: &str,
@@ -343,12 +376,25 @@ impl Decoder {
.ok()
.filter(|v| !v.is_empty())
.unwrap_or_else(|| pref.to_string());
#[cfg(windows)]
let (d3d11_import, adapter_luid, d3d11_hdr10) = (
vk.is_some_and(|v| v.d3d11_import),
vk.and_then(|v| v.adapter_luid),
vk.is_some_and(|v| v.d3d11_hdr10),
);
let done = |backend| {
Ok(Decoder {
backend,
codec_id,
vaapi_fails: 0,
first_fail: None,
want_keyframe: false,
#[cfg(windows)]
d3d11_import,
#[cfg(windows)]
adapter_luid,
#[cfg(windows)]
d3d11_hdr10,
})
};
// Linux `auto`: try VAAPI FIRST unless this device is one where Vulkan Video is
@@ -363,7 +409,7 @@ impl Decoder {
if matches!(choice.as_str(), "auto" | "" | "hardware")
&& !vk
.filter(|v| v.video_decode)
.is_some_and(|v| v.prefer_vulkan_over_vaapi())
.is_some_and(|v| v.prefer_vulkan_first())
{
vaapi_tried = true;
match VaapiDecoder::new(codec_id) {
@@ -376,6 +422,44 @@ impl Decoder {
}
}
}
// Windows `auto`: D3D11VA FIRST unless this device is one where Vulkan Video is
// the established right answer (NVIDIA/AMD). Intel's Windows driver advertises
// Vulkan Video (Arc drivers since 2023) so the capability gate alone no longer
// keeps Intel off FFmpeg-Vulkan — and that combination is field-broken (B580,
// 2026-07: strobing between clean anchors and corrupt inter frames that never
// trips the error-streak demotion, 7 ms p50 decodes blowing the 120 Hz budget)
// where D3D11VA — the DXVA path every Windows video player exercises, and what
// this backend was built for — streams clean. Vulkan stays reachable below by
// explicit preference and as auto's fallback when D3D11VA can't be built.
#[cfg(windows)]
let mut d3d11_tried = false;
#[cfg(windows)]
if matches!(choice.as_str(), "auto" | "" | "hardware")
&& !vk
.filter(|v| v.video_decode)
.is_some_and(|v| v.prefer_vulkan_first())
{
if let Some(v) = vk.filter(|v| v.d3d11_import) {
d3d11_tried = true;
match crate::video_d3d11::D3d11vaDecoder::new(
codec_id,
v.adapter_luid,
v.d3d11_hdr10,
) {
Ok(d) => {
tracing::info!(
?codec_id,
"D3D11VA hardware decode active (shared-texture hand-off)"
);
return done(Backend::D3d11va(d));
}
Err(e) => {
tracing::info!(reason = %format!("{e:#}"),
"D3D11VA unavailable — trying Vulkan Video");
}
}
}
}
if matches!(choice.as_str(), "auto" | "" | "vulkan" | "hardware") {
// `video_decode` gates the Vulkan Video attempt: the presenter now exports its
// handle bundle even when the device has no decode queue (Windows D3D11 interop
@@ -426,14 +510,20 @@ impl Decoder {
}
}
}
// Windows: D3D11VA is the vendor-agnostic DXVA fallback when Vulkan Video isn't
// available (Intel's Windows driver foremost) — gated on the presenter having the
// win32 external-memory import path, else its frames could never reach the screen.
// Windows: D3D11VA as the fallback rung for NVIDIA/AMD auto (Vulkan Video missing
// or failed to open) and the explicit `d3d11va` preference — gated on the presenter
// having the win32 external-memory import path, else its frames could never reach
// the screen. (On Intel/unknown auto it was already tried above — `d3d11_tried`
// skips the repeat.)
#[cfg(windows)]
if choice != "software" && choice != "vulkan" {
if choice != "software" && choice != "vulkan" && !d3d11_tried {
match vk.filter(|v| v.d3d11_import) {
Some(v) => {
match crate::video_d3d11::D3d11vaDecoder::new(codec_id, v.adapter_luid) {
match crate::video_d3d11::D3d11vaDecoder::new(
codec_id,
v.adapter_luid,
v.d3d11_hdr10,
) {
Ok(d) => {
tracing::info!(
?codec_id,
@@ -483,7 +573,7 @@ impl Decoder {
/// Open a PyroWave decoder for a `CODEC_PYROWAVE` session (plan §4.5): pyrowave
/// compute on the presenter's device, no FFmpeg. `codec_id` is irrelevant (kept as
/// HEVC so an — impossible — demotion path stays well-formed).
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
pub fn new_pyrowave(
vk: &VulkanDecodeDevice,
width: u32,
@@ -505,7 +595,17 @@ impl Decoder {
)?)),
codec_id: ffmpeg::codec::Id::HEVC,
vaapi_fails: 0,
first_fail: None,
want_keyframe: false,
// A PyroWave session never demotes (nothing else decodes it — a failure
// renegotiates the codec instead), so the D3D11VA rebuild facts are unused
// here; keep them well-formed rather than plumbing them in for nothing.
#[cfg(windows)]
d3d11_import: false,
#[cfg(windows)]
adapter_luid: None,
#[cfg(windows)]
d3d11_hdr10: false,
})
}
@@ -524,6 +624,7 @@ impl Decoder {
tracing::warn!("presenter can't display hardware frames — demoting to software decode");
self.backend = Backend::Software(SoftwareDecoder::new(self.codec_id)?);
self.vaapi_fails = 0;
self.first_fail = None;
self.want_keyframe = true;
Ok(())
}
@@ -548,7 +649,7 @@ impl Decoder {
au: &[u8],
// Only the PyroWave backend reads the flags; without that feature the param is unused.
#[cfg_attr(
not(all(target_os = "linux", feature = "pyrowave")),
not(all(any(target_os = "linux", windows), feature = "pyrowave")),
allow(unused_variables)
)]
user_flags: u32,
@@ -566,7 +667,7 @@ impl Decoder {
// No demote ladder below PyroWave (nothing else decodes it): propagate the
// error; the pump surfaces it and the session falls back to HEVC by
// renegotiation (plan §4.6), not by decoder swap.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
Backend::PyroWave(p) => {
let aligned = user_flags & punktfunk_core::packet::USER_FLAG_CHUNK_ALIGNED != 0;
return Ok(p
@@ -578,6 +679,7 @@ impl Decoder {
match result {
Ok(f) => {
self.vaapi_fails = 0;
self.first_fail = None;
Ok(f)
}
Err(e) => {
@@ -589,7 +691,9 @@ impl Decoder {
};
self.vaapi_fails += 1;
self.want_keyframe = true;
if self.vaapi_fails >= VAAPI_DEMOTE_AFTER {
let first = *self.first_fail.get_or_insert_with(std::time::Instant::now);
if self.vaapi_fails >= VAAPI_DEMOTE_AFTER && first.elapsed() >= HW_DEMOTE_MIN_STREAK
{
// A failing Vulkan backend still has a hardware rung below it on
// Linux — demote to VAAPI first (user-reported: FFmpeg-Vulkan-on-Mesa
// error-streaking where VAAPI streams perfectly); only when that
@@ -602,16 +706,39 @@ impl Decoder {
"Vulkan Video decode failing repeatedly — demoting to VAAPI");
self.backend = Backend::Vaapi(v);
self.vaapi_fails = 0;
self.first_fail = None;
return Ok(None);
}
Err(va) => tracing::info!(reason = %va,
"VAAPI unavailable for demotion — software decode"),
}
}
// Windows' hardware rung below Vulkan is D3D11VA (a 4K120 stream is
// not survivable on software) — same-GPU rebuild via the stashed LUID.
#[cfg(windows)]
if matches!(self.backend, Backend::Vulkan(_)) && self.d3d11_import {
match crate::video_d3d11::D3d11vaDecoder::new(
self.codec_id,
self.adapter_luid,
self.d3d11_hdr10,
) {
Ok(d) => {
tracing::warn!(error = %e, fails = self.vaapi_fails,
"Vulkan Video decode failing repeatedly — demoting to D3D11VA");
self.backend = Backend::D3d11va(d);
self.vaapi_fails = 0;
self.first_fail = None;
return Ok(None);
}
Err(dx) => tracing::info!(reason = %dx,
"D3D11VA unavailable for demotion — software decode"),
}
}
tracing::warn!(error = %e, fails = self.vaapi_fails,
"{which} decode failing repeatedly — demoting to software");
self.backend = Backend::Software(SoftwareDecoder::new(self.codec_id)?);
self.vaapi_fails = 0;
self.first_fail = None;
} else {
tracing::debug!(backend = which, error = %e,
"decode error — requesting keyframe, keeping hardware decode");
@@ -718,10 +845,10 @@ pub struct VulkanDecodeDevice {
pub physical_device: usize,
pub device: usize,
/// PCI vendor of the presenter's physical device (0x10DE NVIDIA, 0x1002 AMD,
/// 0x8086 Intel) — drives [`Self::prefer_vulkan_over_vaapi`].
/// 0x8086 Intel) — drives [`Self::prefer_vulkan_first`].
pub vendor_id: u32,
/// The driver's device-name string (e.g. "AMD RADV VANGOGH") — the VanGogh/Deck
/// detection for [`Self::prefer_vulkan_over_vaapi`].
/// detection for [`Self::prefer_vulkan_first`].
pub device_name: String,
/// The presenter's graphics+present family (FFmpeg's "required" tx/comp family too).
pub graphics_qf: u32,
@@ -764,6 +891,10 @@ pub struct VulkanDecodeDevice {
/// The presenter enabled `VK_KHR_external_memory_win32` + `VK_KHR_win32_keyed_mutex`:
/// D3D11 shared-texture frames can reach the screen. Always `false` off Windows.
pub d3d11_import: bool,
/// The presenter can also import the RGB10A2 hand-off texture AND offers an HDR10
/// swapchain — the D3D11VA backend emits its HDR (RGB10 PQ pass-through) ring flavor
/// for PQ streams instead of tone-mapping to sRGB. Always `false` off Windows.
pub d3d11_hdr10: bool,
/// `VkPhysicalDeviceIDProperties::deviceLUID` when the driver reports one — the D3D11VA
/// backend creates its decode device on the SAME adapter so shared textures never cross
/// GPUs. `None` when not reported (or off Windows, where it's unused).
@@ -775,9 +906,11 @@ pub struct VulkanDecodeDevice {
}
impl VulkanDecodeDevice {
/// Should `auto` try Vulkan Video BEFORE VAAPI on this device?
/// * **NVIDIA** — Vulkan is its only hardware path (no usable VAAPI; the
/// nvidia-vaapi-driver is broken for this, Moonlight blacklists it).
/// Should `auto` try Vulkan Video BEFORE the platform's other hardware path (VAAPI on
/// Linux, D3D11VA on Windows) on this device?
/// * **NVIDIA** — Vulkan Video is the proven path (on Linux the only one: no usable
/// VAAPI — the nvidia-vaapi-driver is broken for this, Moonlight blacklists it;
/// on Windows it's the validated zero-copy default, 4K@144 with 0.1 ms decode).
/// * **AMD (RADV, VanGogh included)** — Vulkan decode outperforms VAAPI on RADV
/// (on-glass verdict), and on VanGogh VAAPI's separate-plane dmabuf import
/// additionally shows chroma fringing; the session binary opts RADV into
@@ -785,10 +918,11 @@ impl VulkanDecodeDevice {
/// because a mid-session Vulkan failure streak demotes to VAAPI (not software),
/// so a broken Mesa Vulkan path still lands on the working driver.
///
/// Intel (ANV) and unknown vendors keep the battle-tested zero-copy VAAPI first —
/// ANV's Vulkan Video is the least-proven Mesa path and VAAPI is what every other
/// Linux client uses there.
pub fn prefer_vulkan_over_vaapi(&self) -> bool {
/// Intel and unknown vendors take the battle-tested path first: VAAPI on Linux (ANV's
/// Vulkan Video is the least-proven Mesa path), D3D11VA on Windows — Intel's Windows
/// driver advertises Vulkan Video (Arc drivers since 2023), but FFmpeg-Vulkan on it is
/// field-broken (B580, 2026-07: strobing + ~7 ms decodes) where DXVA streams clean.
pub fn prefer_vulkan_first(&self) -> bool {
const VENDOR_NVIDIA: u32 = 0x10DE;
const VENDOR_AMD: u32 = 0x1002;
self.vendor_id == VENDOR_NVIDIA || self.vendor_id == VENDOR_AMD
@@ -845,30 +979,33 @@ mod tests {
pyrowave_decode: false,
video_decode: true,
d3d11_import: false,
d3d11_hdr10: false,
adapter_luid: None,
queue_lock: std::sync::Arc::new(QueueLock::new()),
}
}
/// Auto's Linux hardware order: Vulkan-first on NVIDIA (no usable VAAPI) and ALL AMD
/// (Vulkan decode outperforms VAAPI on RADV — on-glass verdict; VanGogh additionally
/// chroma-fringes over VAAPI); Intel/unknown keep VAAPI first (ANV's Vulkan Video is
/// the least-proven Mesa path). A Vulkan failure streak still demotes to VAAPI, so
/// Vulkan-first can never strand a box on software decode.
/// Auto's hardware order (both OSes): Vulkan-first on NVIDIA (on Linux: no usable
/// VAAPI) and ALL AMD (Vulkan decode outperforms VAAPI on RADV — on-glass verdict;
/// VanGogh additionally chroma-fringes over VAAPI); Intel/unknown take the proven
/// path first — VAAPI on Linux (ANV's Vulkan Video is the least-proven Mesa path),
/// D3D11VA on Windows (Intel's driver advertises Vulkan Video since 2023, but
/// FFmpeg-Vulkan on it strobes — B580 field report). A Vulkan failure streak still
/// demotes to hardware (VAAPI/D3D11VA), so Vulkan-first can never strand a box on
/// software decode.
#[test]
fn vulkan_over_vaapi_on_nvidia_and_amd() {
assert!(decode_device(0x10DE, "NVIDIA GeForce RTX 5070 Ti").prefer_vulkan_over_vaapi());
assert!(decode_device(0x1002, "AMD RADV VANGOGH").prefer_vulkan_over_vaapi());
fn vulkan_first_on_nvidia_and_amd_only() {
assert!(decode_device(0x10DE, "NVIDIA GeForce RTX 5070 Ti").prefer_vulkan_first());
assert!(decode_device(0x1002, "AMD RADV VANGOGH").prefer_vulkan_first());
assert!(decode_device(0x1002, "AMD Custom GPU 0405 (RADV VANGOGH)").prefer_vulkan_first());
assert!(decode_device(0x1002, "AMD Radeon RX 7800 XT (RADV NAVI32)").prefer_vulkan_first());
assert!(
decode_device(0x1002, "AMD Custom GPU 0405 (RADV VANGOGH)").prefer_vulkan_over_vaapi()
);
assert!(
decode_device(0x1002, "AMD Radeon RX 7800 XT (RADV NAVI32)").prefer_vulkan_over_vaapi()
);
assert!(
!decode_device(0x8086, "Intel(R) Arc(tm) A770 Graphics (DG2)")
.prefer_vulkan_over_vaapi()
!decode_device(0x8086, "Intel(R) Arc(tm) A770 Graphics (DG2)").prefer_vulkan_first()
);
// The Windows-side motivation: discrete Arc advertises Vulkan Video and must
// still land on D3D11VA in auto.
assert!(!decode_device(0x8086, "Intel(R) Arc(TM) B580 Graphics").prefer_vulkan_first());
assert!(!decode_device(0x8086, "Intel(R) Arc(TM) Pro Graphics").prefer_vulkan_first());
}
/// Lock the DRM FourCC magic numbers against typos — these are the exact values
+119 -31
View File
@@ -1,6 +1,9 @@
//! D3D11VA hardware decode (Windows) for the Vulkan presenter — the vendor-agnostic DXVA
//! path that covers what Vulkan Video can't (Intel's Windows driver foremost, which has no
//! video-decode queue and previously landed on CPU decode).
//! path, and auto's FIRST choice on Intel/unknown vendors. Intel's Windows driver DOES
//! advertise Vulkan Video (Arc drivers since 2023 — don't trust the capability gate to
//! keep Intel off it), but FFmpeg-Vulkan on it is field-broken (B580, 2026-07: strobing +
//! ~7 ms decodes) where this path streams clean; on NVIDIA/AMD it is the fallback rung
//! below Vulkan Video, in `auto` and via mid-session demotion.
//!
//! Ported from the retired in-process WinUI presenter's decoder (`clients/windows/src/video.rs`)
//! with one structural change: that presenter sampled D3D11 textures directly, while ours draws
@@ -24,9 +27,11 @@
//! (`VK_KHR_win32_keyed_mutex`); both sides take and release it with **key 0**: a frame the
//! presenter drops (arrival-paced, newest wins) is simply never acquired, which a
//! key-ping-pong protocol would deadlock on.
//! * An HDR (PQ/BT.2020) stream is tone-mapped to SDR by the video processor (input colour
//! space `G2084_P2020`, output sRGB): correct picture, no HDR presentation on this backend —
//! its targets (Intel iGPU laptops) are SDR panels; HDR-first boxes take Vulkan Video.
//! * An HDR (PQ/BT.2020) stream passes through when the presenter can take it (RGB10A2
//! import + an HDR10 swapchain — [`crate::video::VulkanDecodeDevice::d3d11_hdr10`]): the
//! video processor converts YCbCr G2084 → RGB G2084 into an RGB10A2 ring, colorspace
//! only, no tone mapping. On an SDR-only path it tone-maps to sRGB instead (input
//! `G2084_P2020`, output sRGB) — correct picture, no HDR presentation.
//!
//! The decode device is created on the **presenter's adapter** (matched by the Vulkan device's
//! LUID) so the shared textures never cross GPUs on a multi-adapter box.
@@ -37,7 +42,7 @@ use ffmpeg_next as ffmpeg;
use std::ffi::c_void;
use std::ptr;
use windows::core::{Interface, GUID};
use windows::Win32::Foundation::HANDLE;
use windows::Win32::Foundation::{HANDLE, RECT};
use windows::Win32::Graphics::Direct3D::{D3D_FEATURE_LEVEL_11_0, D3D_FEATURE_LEVEL_11_1};
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, ID3D11Device, ID3D11DeviceContext, ID3D11Multithread, ID3D11Texture2D,
@@ -52,11 +57,12 @@ use windows::Win32::Graphics::Direct3D11::{
D3D11_VPOV_DIMENSION_TEXTURE2D,
};
use windows::Win32::Graphics::Dxgi::Common::{
DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709, DXGI_COLOR_SPACE_YCBCR_FULL_G22_LEFT_P2020,
DXGI_COLOR_SPACE_YCBCR_FULL_G22_LEFT_P601, DXGI_COLOR_SPACE_YCBCR_FULL_G22_LEFT_P709,
DXGI_COLOR_SPACE_YCBCR_STUDIO_G2084_LEFT_P2020, DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P2020,
DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P601, DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P709,
DXGI_FORMAT, DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_FORMAT_NV12, DXGI_FORMAT_P010, DXGI_RATIONAL,
DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020, DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709,
DXGI_COLOR_SPACE_YCBCR_FULL_G22_LEFT_P2020, DXGI_COLOR_SPACE_YCBCR_FULL_G22_LEFT_P601,
DXGI_COLOR_SPACE_YCBCR_FULL_G22_LEFT_P709, DXGI_COLOR_SPACE_YCBCR_STUDIO_G2084_LEFT_P2020,
DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P2020, DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P601,
DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P709, DXGI_FORMAT, DXGI_FORMAT_B8G8R8A8_UNORM,
DXGI_FORMAT_NV12, DXGI_FORMAT_P010, DXGI_FORMAT_R10G10B10A2_UNORM, DXGI_RATIONAL,
DXGI_SAMPLE_DESC,
};
use windows::Win32::Graphics::Dxgi::{
@@ -95,10 +101,14 @@ const PROFILE_AV1_VLD_PROFILE0: GUID = GUID::from_u128(0xb8be4ccb_cf53_46ba_8d59
pub struct D3d11Frame {
pub width: u32,
pub height: u32,
/// What the ring slot actually CONTAINS after the video processor's conversion: sRGB
/// BT.709 full-range RGB — regardless of the stream's own CICP (a PQ stream was
/// tone-mapped). The presenter keys SDR/HDR handling off this, so it always reads SDR.
/// What the ring slot actually CONTAINS after the video processor's conversion:
/// sRGB BT.709 full-range RGB normally (a PQ stream was tone-mapped), or PQ BT.2020
/// full-range RGB when the HDR pass-through ring is active (`rgb10`) — the presenter
/// keys its SDR/HDR handling off this.
pub color: ColorDesc,
/// The ring slot's texture format: `false` = BGRA8, `true` = RGB10A2 (the HDR PQ
/// pass-through flavor) — the presenter's Vulkan import must match it exactly.
pub rgb10: bool,
/// Intra keyframe (IDR/I) — the pump's post-loss re-anchor signal. See
/// `crate::video::VkVideoFrame`.
pub keyframe: bool,
@@ -331,6 +341,9 @@ struct SharedRing {
height: u32,
next: usize,
generation: u32,
/// HDR flavor: RGB10A2 slots the processor fills with PQ BT.2020 RGB (colorspace
/// conversion only — both sides G2084, no tone mapping). `false` = BGRA8 sRGB.
pq_out: bool,
}
impl SharedRing {
@@ -340,6 +353,7 @@ impl SharedRing {
width: u32,
height: u32,
generation: u32,
pq_out: bool,
) -> Result<SharedRing> {
// The video processor: NV12/P010 in, BGRA8 out, 1:1 (no scaling — the Vulkan side
// scales at composite time like every other path). Frame rates are advisory.
@@ -369,10 +383,15 @@ impl SharedRing {
Height: height,
MipLevels: 1,
ArraySize: 1,
// Single-plane BGRA8: the ONLY hand-off format whose Vulkan import is a
// Single-plane RGB: the ONLY hand-off family whose Vulkan import is a
// universally exercised driver path (see the module docs — NV12 import TDRs
// on NVIDIA despite being advertised).
Format: DXGI_FORMAT_B8G8R8A8_UNORM,
// on NVIDIA despite being advertised). RGB10A2 for the HDR pass-through
// flavor (gated on the presenter's probe), BGRA8 otherwise.
Format: if pq_out {
DXGI_FORMAT_R10G10B10A2_UNORM
} else {
DXGI_FORMAT_B8G8R8A8_UNORM
},
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
@@ -430,7 +449,8 @@ impl SharedRing {
height,
slots = RING_SLOTS,
generation,
"D3D11 shared hand-off ring built (VideoProcessor → BGRA8)"
hdr = pq_out,
"D3D11 shared hand-off ring built (VideoProcessor → RGB)"
);
Ok(SharedRing {
slots,
@@ -440,6 +460,7 @@ impl SharedRing {
height,
next: 0,
generation,
pq_out,
})
}
}
@@ -457,6 +478,10 @@ pub(crate) struct D3d11vaDecoder {
/// setters (Win10 1703+, universally present — init fails to software without it).
video_context1: ID3D11VideoContext1,
ring: Option<SharedRing>,
/// The presenter can import RGB10A2 AND offers an HDR10 swapchain
/// ([`crate::video::VulkanDecodeDevice::d3d11_hdr10`]) — PQ streams get the HDR
/// pass-through ring; without it they keep the tonemap-to-sRGB ring.
hdr10_out: bool,
}
// Single-owner pointers + COM interfaces, only touched from the session pump thread (the
@@ -467,6 +492,7 @@ impl D3d11vaDecoder {
pub(crate) fn new(
codec_id: ffmpeg::codec::Id,
luid: Option<[u8; 8]>,
hdr10_out: bool,
) -> Result<D3d11vaDecoder> {
use ffmpeg::ffi;
let (device, context) = create_device(luid)?;
@@ -537,6 +563,7 @@ impl D3d11vaDecoder {
video_device,
video_context1,
ring: None,
hdr10_out,
})
}
}
@@ -591,12 +618,15 @@ impl D3d11vaDecoder {
let video_device = self.video_device.clone();
let video_context1 = self.video_context1.clone();
let context = self.context.clone();
// (Re)build the ring + video processor on first use or a stream size change (the
// hand-off is BGRA8 regardless of the stream's bit depth, so depth never rebuilds).
// (Re)build the ring + video processor on first use, a stream size change, or a
// flavor change (the host flips PQ in-band; SDR↔HDR swaps the slot format, so
// it rebuilds like a resize — bit DEPTH alone still never rebuilds: an SDR
// 10-bit stream and an 8-bit one share the same output flavor).
let pq_out = self.hdr10_out && color.is_pq();
let rebuild = self
.ring
.as_ref()
.is_none_or(|r| r.width != width || r.height != height);
.is_none_or(|r| r.width != width || r.height != height || r.pq_out != pq_out);
if rebuild {
let generation = self.ring.as_ref().map_or(0, |r| r.generation + 1);
self.ring = Some(SharedRing::build(
@@ -605,6 +635,7 @@ impl D3d11vaDecoder {
width,
height,
generation,
pq_out,
)?);
}
let ring = self.ring.as_mut().expect("ring built above");
@@ -647,10 +678,36 @@ impl D3d11vaDecoder {
(_, _, true) => DXGI_COLOR_SPACE_YCBCR_FULL_G22_LEFT_P709,
_ => DXGI_COLOR_SPACE_YCBCR_STUDIO_G22_LEFT_P709,
};
// The DECODE surface is DXVA-aligned (height rounded up to the profile's
// macroblock/tile alignment — 128 for HEVC/AV1), so it is TALLER than the
// frame: a 2400-line stream decodes into a 2432-line texture. Without an
// explicit source rect the processor blits the WHOLE surface — the padding
// rows (uninitialized NV12: Y=0,U=V=0, which converts to vivid green) land at
// the bottom of the output and the picture is squashed to fit. Clamp the
// source to the real frame; the dest stays the whole (frame-sized) slot.
// Live-hit on Intel 3840x2400 as a ~32 px green bar (2026-07-19).
video_context1.VideoProcessorSetStreamSourceRect(
&ring.vp,
0,
true,
Some(&RECT {
left: 0,
top: 0,
right: width as i32,
bottom: height as i32,
}),
);
video_context1.VideoProcessorSetStreamColorSpace1(&ring.vp, 0, in_cs);
video_context1.VideoProcessorSetOutputColorSpace1(
&ring.vp,
DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709,
// HDR ring: PQ in, PQ out — a pure colorspace conversion (YCbCr→RGB),
// no tone mapping; the presenter passes the values through to its HDR10
// swapchain. SDR ring: sRGB out (a PQ stream is tone-mapped here).
if ring.pq_out {
DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020
} else {
DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
},
);
let stream = D3D11_VIDEO_PROCESSOR_STREAM {
@@ -684,17 +741,38 @@ impl D3d11vaDecoder {
// completion, and an unflushed deferred batch would add a driver-decided delay.
context.Flush();
log_layout_once(width, height, index, color.is_pq());
let mut src_desc = D3D11_TEXTURE2D_DESC::default();
src.GetDesc(&mut src_desc);
log_layout_once(
width,
height,
src_desc.Width,
src_desc.Height,
index,
color.is_pq(),
);
Ok(D3d11Frame {
width,
height,
// What the slot now CONTAINS: sRGB BT.709 full-range RGB (PQ was tone-mapped).
color: ColorDesc {
primaries: 1,
transfer: 13, // sRGB (H.273)
matrix: 0, // identity — RGB
full_range: true,
// What the slot now CONTAINS. HDR ring: PQ BT.2020 full-range RGB (the
// presenter reads is_pq() and flips its HDR10 swapchain). SDR ring: sRGB
// BT.709 full-range RGB (PQ was tone-mapped above).
color: if ring.pq_out {
ColorDesc {
primaries: 9,
transfer: 16, // PQ / SMPTE ST.2084
matrix: 0, // identity — RGB
full_range: true,
}
} else {
ColorDesc {
primaries: 1,
transfer: 13, // sRGB (H.273)
matrix: 0, // identity — RGB
full_range: true,
}
},
rgb10: ring.pq_out,
// SAFETY: `self.frame` is the live decoded AVFrame for this call.
keyframe: crate::video::frame_is_keyframe(self.frame),
handle,
@@ -720,10 +798,20 @@ impl Drop for D3d11vaDecoder {
}
/// One-time dump of the first decoded surface's layout — the forensics for a new GPU/driver.
fn log_layout_once(width: u32, height: u32, index: u32, pq: bool) {
/// `tex_*` is the DXVA-aligned decode surface (>= the frame); the gap is the padding the
/// stream source rect excludes.
fn log_layout_once(width: u32, height: u32, tex_w: u32, tex_h: u32, index: u32, pq: bool) {
use std::sync::atomic::{AtomicBool, Ordering};
static ONCE: AtomicBool = AtomicBool::new(true);
if ONCE.swap(false, Ordering::Relaxed) {
tracing::info!(width, height, slice = index, pq, "D3D11VA first frame");
tracing::info!(
width,
height,
tex_w,
tex_h,
slice = index,
pq,
"D3D11VA first frame"
);
}
}
+29 -12
View File
@@ -1,6 +1,7 @@
//! D3D11 shared-texture → Vulkan import (Windows): the presenter half of the D3D11VA
//! decode path (`pf_client_core::video_d3d11`). Each decoded frame arrives as the NT
//! handle of a shareable **BGRA8** texture (the decoder's VideoProcessor already did
//! handle of a shareable single-plane RGB texture — **BGRA8** sRGB normally, **RGB10A2**
//! PQ for the HDR pass-through flavor (the decoder's VideoProcessor already did
//! YUV→RGB); we import it as a single-plane VkImage (`VK_KHR_external_memory_win32`,
//! dedicated allocation) and the presenter blits it straight into its video image — no
//! CSC pass. Single-plane RGBA is deliberate: importing the earlier multiplanar NV12
@@ -28,31 +29,41 @@ pub const DEVICE_EXTENSIONS: [&std::ffi::CStr; 2] = [
ash::khr::win32_keyed_mutex::NAME,
];
/// Can this device import a D3D11 BGRA8 texture as a blit source? The spec-required
/// Can this device import a D3D11 texture of `format` as a blit source? The spec-required
/// capability probe for the exact image the import path creates — creating an external
/// image the driver doesn't support is undefined behavior (observed as
/// `VK_ERROR_DEVICE_LOST` at the first submits with the old NV12 hand-off).
pub fn import_supported(instance: &ash::Instance, pdev: vk::PhysicalDevice) -> bool {
fn format_importable(
instance: &ash::Instance,
pdev: vk::PhysicalDevice,
format: vk::Format,
) -> bool {
let mut ext_info = vk::PhysicalDeviceExternalImageFormatInfo::default()
.handle_type(vk::ExternalMemoryHandleTypeFlags::D3D11_TEXTURE);
let fmt_info = vk::PhysicalDeviceImageFormatInfo2::default()
.format(vk::Format::B8G8R8A8_UNORM)
.format(format)
.ty(vk::ImageType::TYPE_2D)
.tiling(vk::ImageTiling::OPTIMAL)
.usage(vk::ImageUsageFlags::TRANSFER_SRC)
.push_next(&mut ext_info);
let mut ext_props = vk::ExternalImageFormatProperties::default();
let mut props = vk::ImageFormatProperties2::default().push_next(&mut ext_props);
let ok = unsafe {
instance.get_physical_device_image_format_properties2(pdev, &fmt_info, &mut props)
}
.is_ok()
unsafe { instance.get_physical_device_image_format_properties2(pdev, &fmt_info, &mut props) }
.is_ok()
&& ext_props
.external_memory_properties
.external_memory_features
.contains(vk::ExternalMemoryFeatureFlags::IMPORTABLE);
tracing::info!(bgra8 = ok, "D3D11 texture → Vulkan import support");
ok
.contains(vk::ExternalMemoryFeatureFlags::IMPORTABLE)
}
/// The two hand-off flavors' import support: `.0` = BGRA8 (the SDR ring — gates the whole
/// D3D11VA path), `.1` = RGB10A2 (the HDR PQ ring — gates only the pass-through flavor;
/// without it a PQ stream keeps the decoder-side tonemap to BGRA8).
pub fn import_supported(instance: &ash::Instance, pdev: vk::PhysicalDevice) -> (bool, bool) {
let bgra8 = format_importable(instance, pdev, vk::Format::B8G8R8A8_UNORM);
let rgb10 = format_importable(instance, pdev, vk::Format::A2B10G10R10_UNORM_PACK32);
tracing::info!(bgra8, rgb10, "D3D11 texture → Vulkan import support");
(bgra8, rgb10)
}
/// One imported frame: the BGRA8 image over the shared texture and its imported
@@ -98,7 +109,13 @@ pub fn import(
if std::env::var_os("PUNKTFUNK_HW_FAULT").is_some_and(|v| v == "import") {
bail!("injected import failure (PUNKTFUNK_HW_FAULT=import)");
}
let mp_format = vk::Format::B8G8R8A8_UNORM;
// DXGI R10G10B10A2 and Vulkan A2B10G10R10_PACK32 are the same bit layout (R in the
// low bits) — the standard interop pairing, same as BGRA8 ↔ B8G8R8A8.
let mp_format = if frame.rgb10 {
vk::Format::A2B10G10R10_UNORM_PACK32
} else {
vk::Format::B8G8R8A8_UNORM
};
let handle_type = vk::ExternalMemoryHandleTypeFlags::D3D11_TEXTURE;
// One single-plane image over the whole texture, transfer-source only — the blit is
+3 -3
View File
@@ -194,7 +194,7 @@ struct StreamState {
/// PyroWave present has no demote rung (nothing else decodes the codec), so a
/// persistent non-device-lost present failure would warn on every frame. Latch it:
/// warn on the first failure of a streak, then stay quiet until a present succeeds.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
pyro_present_warned: bool,
hw_fails: u32,
/// The OSD's text (multi-line; rebuilt each Stats window and on a live tier cycle).
@@ -267,7 +267,7 @@ impl StreamState {
win_start: Instant::now(),
presented: PresentedWindow::default(),
dmabuf_demoted: false,
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
pyro_present_warned: false,
hw_fails: 0,
osd_text: String::new(),
@@ -1007,7 +1007,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// PyroWave planar frames: already on the presenter's device and
// fence-complete — a present failure has no demote rung (nothing
// else decodes the codec); only device loss ends the session.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
DecodedImage::PyroWave(f) => {
// The wavelet stream carries the negotiated ColorInfo (no VUI): an
// HDR (PQ) pyrowave session presents through the HDR10 path exactly
+3 -3
View File
@@ -47,7 +47,7 @@ pub enum FrameInput<'a> {
D3d11(pf_client_core::video::D3d11Frame),
/// PyroWave planar output — three R8 plane views already on THIS device, decode
/// fence-complete, GENERAL layout (`pf_client_core::video_pyrowave`).
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
PyroWave(pf_client_core::video_pyrowave::PyroWavePlanarFrame),
}
@@ -136,7 +136,7 @@ pub struct Presenter {
csc: CscPass,
/// The planar (3-plane) CSC variant for PyroWave frames; built only when the device
/// passed the pyrowave probe.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
csc_planar: Option<CscPass>,
/// FFmpeg Vulkan Video decode handles — `None` when the stack can't do it.
video_export: Option<pf_client_core::video::VulkanDecodeDevice>,
@@ -304,7 +304,7 @@ impl Drop for Presenter {
#[cfg(target_os = "linux")]
self.hw.take();
self.csc.destroy(&self.device);
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
if let Some(p) = &self.csc_planar {
p.destroy(&self.device);
}
+11 -10
View File
@@ -38,7 +38,7 @@ impl Presenter {
FrameInput::VkFrame(v) => Some(v.color.is_pq()),
#[cfg(windows)]
FrameInput::D3d11(d) => Some(d.color.is_pq()),
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
FrameInput::PyroWave(f) => Some(f.color.is_pq()),
};
if let Some(pq) = frame_pq {
@@ -68,7 +68,7 @@ impl Presenter {
#[cfg(windows)]
let mut win_frame: Option<crate::d3d11::HwFrame> = None;
let mut vk_frame: Option<(VkVideoFrame, [vk::ImageView; 2])> = None;
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
let mut pyro_frame: Option<pf_client_core::video_pyrowave::PyroWavePlanarFrame> = None;
let cpu_frame = match input {
FrameInput::Redraw => None,
@@ -96,7 +96,7 @@ impl Presenter {
vk_frame = Some((v, views));
None
}
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
FrameInput::PyroWave(f) => {
pyro_frame = Some(f);
None
@@ -155,7 +155,7 @@ impl Presenter {
}
self.csc.bind_planes(&self.device, views[0], views[1]);
}
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
if let Some(f) = &pyro_frame {
if self
.video
@@ -239,11 +239,12 @@ impl Presenter {
);
}
// D3D11 frame: acquire the imported BGRA texture from the external "queue
// D3D11 frame: acquire the imported RGB texture from the external "queue
// family" (the keyed mutex on the submit is the actual cross-API sync) and
// blit it into the video image — the frame arrives as ready sRGB from the
// decoder's VideoProcessor, so there is no CSC pass; the blit converts the
// BGRA→RGBA component order. Same layout dance as the CPU staging path.
// blit it into the video image — the frame arrives as ready RGB from the
// decoder's VideoProcessor (sRGB BGRA8, or PQ RGB10A2 on the HDR ring —
// matching the HDR-mode video image), so there is no CSC pass; the blit
// converts component order. Same layout dance as the CPU staging path.
#[cfg(windows)]
if let (Some(f), Some(v)) = (&win_frame, &self.video) {
external_acquire_barrier(&self.device, self.cmd_buf, f.image(), self.qfi);
@@ -316,7 +317,7 @@ impl Presenter {
// PyroWave frame: the planes are already on THIS device, decode
// fence-complete and barriered to fragment sampling (GENERAL) by the
// decoder — no acquire needed, just the planar CSC pass.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
if let (Some(f), Some(v)) = (&pyro_frame, &self.video) {
let extent = vk::Extent2D {
width: v.width,
@@ -694,7 +695,7 @@ impl Presenter {
}
/// [`record_csc`] over the planar (PyroWave) pass — always 8-bit, no MSB packing.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
unsafe fn record_csc_planar(
&self,
framebuffer: vk::Framebuffer,
+1 -1
View File
@@ -207,7 +207,7 @@ impl Presenter {
// The planar (PyroWave) pass renders to the same intermediate — rebuild it at the
// new format too (an HDR pyrowave session needs the 10-bit intermediate exactly
// like the H.26x path; 8-bit PQ bands visibly).
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
if let Some(p) = self.csc_planar.take() {
p.destroy(&self.device);
self.csc_planar = Some(CscPass::new_planar(&self.device, self.video_format)?);
+16 -4
View File
@@ -94,8 +94,9 @@ impl Presenter {
// (vkGetPhysicalDeviceImageFormatProperties2 — creating an unsupported external
// image is UB, observed as VK_ERROR_DEVICE_LOST at the first submits on NVIDIA).
#[cfg(windows)]
let win_capable = crate::d3d11::DEVICE_EXTENSIONS.iter().all(|n| has(n))
&& crate::d3d11::import_supported(&instance, pdev);
let (import_bgra8, import_rgb10) = crate::d3d11::import_supported(&instance, pdev);
#[cfg(windows)]
let win_capable = crate::d3d11::DEVICE_EXTENSIONS.iter().all(|n| has(n)) && import_bgra8;
#[cfg(windows)]
if win_capable {
dev_exts.extend(crate::d3d11::DEVICE_EXTENSIONS.iter().map(|n| n.as_ptr()));
@@ -317,7 +318,7 @@ impl Presenter {
let csc = CscPass::new(&device, vk::Format::R8G8B8A8_UNORM)?;
// Starts SDR like `csc`; an HDR (PQ) pyrowave session rebuilds it at the 10-bit
// intermediate via `set_hdr_mode`, exactly like the H.26x pass.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
let csc_planar = if pyrowave_ok {
Some(CscPass::new_planar(&device, vk::Format::R8G8B8A8_UNORM)?)
} else {
@@ -392,14 +393,25 @@ impl Presenter {
d3d11_import: win_capable,
#[cfg(not(windows))]
d3d11_import: false,
// Filled in below — the HDR10 surface facts arrive with pick_formats.
d3d11_hdr10: false,
adapter_luid,
queue_lock: queue_lock.clone(),
})
} else {
None
};
#[cfg(windows)]
let mut video_export = video_export;
let (format, hdr10_format) = pick_formats(&surface_i, pdev, surface, has_colorspace_ext)?;
// The D3D11VA backend may emit its HDR (RGB10 PQ) ring only when this device can
// import the 10-bit texture AND the surface offers an HDR10 swapchain to pass it
// through to; otherwise a PQ stream keeps the decoder-side tonemap to sRGB.
#[cfg(windows)]
if let Some(v) = video_export.as_mut() {
v.d3d11_hdr10 = win_capable && import_rgb10 && hdr10_format.is_some();
}
let present_mode = pick_present_mode(&surface_i, pdev, surface)?;
tracing::info!(
?format,
@@ -451,7 +463,7 @@ impl Presenter {
#[cfg(windows)]
hw_win,
csc,
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
#[cfg(all(any(target_os = "linux", windows), feature = "pyrowave"))]
csc_planar,
video_export,
overlay_pipe,
+3 -1
View File
@@ -75,7 +75,9 @@ The app is in **Google Play Internal Testing** — request a tester invite on ou
`punktfunk-client` for Windows (`clients/windows`) is the native graphical client for Windows — pure
Rust, the same `punktfunk/1` core as the Apple, Linux, and Android apps, with a **WinUI 3** UI (host
list, settings, PIN pairing); the stream itself runs in punktfunk's Vulkan presenter. Its decoder
tries **Vulkan Video, then D3D11VA, then software**, with 10-bit/HDR present, WASAPI audio + mic,
order is per-vendor: **Vulkan Video, then D3D11VA, then software** on NVIDIA and AMD, and
**D3D11VA first** on Intel and other GPUs (Intel's driver advertises Vulkan Video, but DXVA is the
proven path there), with 10-bit/HDR present, WASAPI audio + mic,
SDL3 controllers (rumble, lightbar, DualSense), network discovery, and the full PIN-pairing trust
surface. It builds for both `x86_64` and `aarch64` and ships as a **signed MSIX**. Launch it and
pick a host from the list, just like the other native apps.
+1 -1
View File
@@ -161,7 +161,7 @@ A few knobs are read by the native **clients**, not the host:
| Setting | Values | Meaning |
|---|---|---|
| `PUNKTFUNK_DECODER` | `software` · `vaapi` · `vulkan` (Linux) · `d3d11va` (Windows) | Force the decode path. Default auto-selects hardware (VAAPI on Intel/AMD, Vulkan Video on NVIDIA and the Steam Deck, D3D11VA/Vulkan on Windows) with a software fallback. |
| `PUNKTFUNK_DECODER` | `software` · `vaapi` · `vulkan` (Linux) · `d3d11va` (Windows) | Force the decode path. Default auto-selects hardware per vendor (Linux: VAAPI on Intel/AMD, Vulkan Video on NVIDIA and the Steam Deck; Windows: Vulkan Video on NVIDIA/AMD, D3D11VA on Intel and others) with a software fallback. |
## Bitrate
+1 -1
View File
@@ -60,7 +60,7 @@ host is newer than the Linux host.)
|---|---|
| **macOS / iOS / iPadOS / tvOS** | VideoToolbox HEVC + AV1 decode (AV1 on hardware that decodes it — M3-class Macs, A17 Pro-class iPhones), GameController capture, full DualSense feedback, mDNS discovery, PIN pairing + TOFU, network speed test, latency HUD. Stage-2 presenter (`VTDecompressionSession``CAMetalLayer`, ~11 ms p50 capture→present) is validated on glass and is the default (stage 1 remains the fallback when Metal is unavailable). Ships as one universal TestFlight build / App Store listing. |
| **Linux** (`punktfunk-client`) | relm4/GTK4 shell; streams run in the spawned Vulkan session presenter. Hardware decode via Vulkan Video (all vendors, incl. NVIDIA) with VAAPI dmabuf and software fallbacks, PipeWire audio + mic, SDL3 gamepads incl. DualSense, mDNS discovery, PIN pairing + TOFU, speed test, Skia console UI + tiered stats overlay. Ships as Flatpak, apt, rpm, and Arch packages. |
| **Windows** (`punktfunk-client`) | WinUI 3 shell; streams run in the Vulkan session presenter with a Vulkan Video → D3D11VA → software decode chain, HDR10, WASAPI audio + mic, SDL3 gamepads incl. DualSense, mDNS discovery, and the full PIN/TOFU trust surface. Ships as a signed MSIX (x86_64 + ARM64). **Hardware decode validated on NVIDIA and Intel; HDR present pending on-glass validation.** |
| **Windows** (`punktfunk-client`) | WinUI 3 shell; streams run in the Vulkan session presenter with a per-vendor decode chain — Vulkan Video → D3D11VA → software on NVIDIA/AMD, D3D11VA → Vulkan Video → software on Intel and others — plus HDR10, WASAPI audio + mic, SDL3 gamepads incl. DualSense, mDNS discovery, and the full PIN/TOFU trust surface. Ships as a signed MSIX (x86_64 + ARM64). **Hardware decode and HDR10 present validated on glass on NVIDIA and Intel (incl. HDR pass-through on the Intel D3D11VA path).** |
| **Android** (phone + Android TV) | Kotlin app with a Rust core over JNI. NDK `AMediaCodec` hardware HEVC decode + HDR10 (Main10/BT.2020 PQ), Opus/Oboe audio + mic, gamepad input with rumble/HID feedback, mDNS discovery, PIN pairing + TOFU (Keystore identity), tiered stats overlay (three-finger tap), custom resolutions, and D-pad/controller focus navigation for TV. Ships to the Google Play Internal Testing track. |
`punktfunk-probe` is a headless reference and measurement client (for testing and