feat(recovery): clean mid-stream loss recovery — freeze-until-reanchor + AMD LTR-RFI
ci / rust (push) Failing after 53s
ci / docs-site (push) Successful in 54s
ci / web (push) Successful in 58s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
decky / build-publish (push) Successful in 18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 7s
apple / swift (push) Successful in 4m57s
ci / bench (push) Successful in 5m43s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m4s
docker / deploy-docs (push) Successful in 21s
windows-host / package (push) Failing after 8m42s
flatpak / build-publish (push) Failing after 8m6s
android / android (push) Successful in 11m59s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m59s
arch / build-publish (push) Successful in 13m20s
deb / build-publish (push) Successful in 14m36s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m45s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m44s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m21s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m1s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 5m28s
release / apple (push) Successful in 25m54s
apple / screenshots (push) Successful in 19m35s

Removes the "gray frames with motion" artifact on Vulkan-Video clients and lets
AMD/NVENC hosts re-anchor after loss WITHOUT a 20-40x IDR spike.

Client (pf-client-core): after a reference loss the hardware decoder conceals the
missing-reference deltas (on RADV, a gray plate with new motion painted over) and
returns Ok. The pump now freezes on the last good picture until a clean re-anchor
instead of showing the concealment — lifting on a real IDR, an intra-refresh
recovery mark (2nd wave boundary), or an LTR-RFI recovery anchor (1st). The
frame_index gap is the early, precise loss signal and drives an RFI request.

Host recovery signals (inert unless the backend supports them):
- USER_FLAG_RECOVERY_POINT — intra-refresh wave boundary (NVENC constrained GDR).
- USER_FLAG_RECOVERY_ANCHOR — AMD LTR reference-frame-invalidation recovery frame.

AMD LTR-RFI (encode/windows/amf.rs) — the AMD twin of NVENC RFI. AMF's AVC/HEVC API
has no constrained-intra property (intra-refresh cannot heal; PSNR-proven), so the
only clean-recovery lever is user LTR: mark frames as long-term references, and on
loss force the next frame to re-reference the newest known-good one — a clean
P-frame, not an IDR. Two rotating LTR slots, ~0.5s mark cadence, on by default for
AVC/HEVC (PUNKTFUNK_NO_AMF_LTR disables). invalidate_ref_frames picks the newest LTR
before the loss; a range older than the live slots falls back to a keyframe.

Protocol (punktfunk-core): RfiRequest control message + NativeClient::request_rfi().
Host: RfiRequest dispatch -> invalidate_ref_frames (IDR fallback); an RFI success
anchors the keyframe cooldown so the client's frames_dropped echo of the same loss
is coalesced away rather than emitting a redundant IDR.

Spike: synthetic NV12 GPU source for headless AMF encoder testing.

Validated: core rfi_request_roundtrip; pf-client-core 31 unit tests
(incl. an_rfi_anchor_lifts_immediately); punktfunk-host builds + 271 tests on Linux;
punktfunk-host builds clean on Windows; real AMD iGPU spike (invalidate at frame 90
forced re-reference to LTR frame 60 — 180 frames, keyframes=1, no recovery IDR).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-11 17:31:17 +02:00
parent 890c7531d8
commit e55ff1bb28
21 changed files with 1228 additions and 33 deletions
@@ -0,0 +1,181 @@
//! A headless synthetic **NV12 D3D11** capture source for exercising the GPU encoders on Windows
//! without a real capture session.
//!
//! The native AMF path (and the D3D11 zero-copy NVENC/QSV paths) require an NV12 texture that lives
//! on the GPU — the CPU-Bgrx [`SyntheticCapturer`](crate::capture::SyntheticCapturer) can't provide
//! one, and DXGI Desktop Duplication can't create one under an ssh session-0 (E_ACCESSDENIED). This
//! source builds an NV12 texture on the selected render adapter and fills it with a **moving** luma
//! ramp each frame, so the encoder sees genuine motion (P-frame residuals + the intra-refresh wave
//! under content change) — exactly what an intra-refresh recovery validation needs. Driven by
//! `spike --source synthetic-nv12`.
use crate::capture::dxgi::{make_device, D3d11Frame};
use crate::capture::{CapturedFrame, Capturer, FramePayload, PixelFormat};
use anyhow::{Context, Result};
use windows::Win32::Graphics::Direct3D11::{
ID3D11Device, ID3D11DeviceContext, ID3D11Texture2D, D3D11_BIND_SHADER_RESOURCE,
D3D11_CPU_ACCESS_WRITE, D3D11_MAPPED_SUBRESOURCE, D3D11_MAP_WRITE, D3D11_TEXTURE2D_DESC,
D3D11_USAGE, D3D11_USAGE_DEFAULT, D3D11_USAGE_STAGING,
};
use windows::Win32::Graphics::Dxgi::Common::{DXGI_FORMAT_NV12, DXGI_SAMPLE_DESC};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
/// Synthetic NV12 frames on the GPU. Owns its own D3D11 device + immediate context and two NV12
/// textures: a CPU-writable STAGING scratch it fills each frame, and a DEFAULT texture it copies
/// into and hands to the encoder. The encoder copies out of the DEFAULT texture synchronously
/// (spike drives capture→submit→poll on one thread), so reusing one DEFAULT texture is sound.
pub struct SyntheticNv12Capturer {
device: ID3D11Device,
context: ID3D11DeviceContext,
default_tex: ID3D11Texture2D,
staging: ID3D11Texture2D,
width: u32,
height: u32,
fps: u32,
frame_idx: u64,
}
// SAFETY: mirrors `D3d11Frame`'s reasoning — the device is created free-threaded (`make_device`
// passes no `SINGLETHREADED` flag) and D3D11 uses interlocked COM refcounting, so moving the whole
// capturer (device + immediate context + textures) to its owning thread and using it only there is
// sound. The value is moved, never aliased (no `Sync`), so the single-threaded immediate context is
// never touched concurrently.
unsafe impl Send for SyntheticNv12Capturer {}
impl SyntheticNv12Capturer {
pub fn new(width: u32, height: u32, fps: u32) -> Result<Self> {
// NV12 is 4:2:0 — both dimensions must be even (the chroma plane is width/2 × height/2).
let width = (width & !1).max(2);
let height = (height & !1).max(2);
// SAFETY: a self-contained builder owning every handle it creates; each COM call is checked
// and the returned owners drop with their wrappers.
unsafe {
let adapter = resolve_render_adapter().context("resolve render adapter for NV12 source")?;
let (device, context) = make_device(&adapter).context("create D3D11 device")?;
let default_tex = create_nv12(
&device,
width,
height,
D3D11_USAGE_DEFAULT,
0,
D3D11_BIND_SHADER_RESOURCE.0 as u32,
)
.context("create NV12 default texture")?;
let staging = create_nv12(
&device,
width,
height,
D3D11_USAGE_STAGING,
D3D11_CPU_ACCESS_WRITE.0 as u32,
0,
)
.context("create NV12 staging texture")?;
Ok(SyntheticNv12Capturer {
device,
context,
default_tex,
staging,
width,
height,
fps,
frame_idx: 0,
})
}
}
}
impl Capturer for SyntheticNv12Capturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
let pts_ns = self.frame_idx * 1_000_000_000 / self.fps.max(1) as u64;
// SAFETY: Map/Unmap/CopyResource on this capturer's own single-threaded immediate context;
// all writes stay within the mapped NV12 surface (Y: H rows of RowPitch; UV: H/2 rows of
// RowPitch beginning at RowPitch*H — the standard NV12 plane layout).
unsafe {
let mut map = D3D11_MAPPED_SUBRESOURCE::default();
self.context
.Map(&self.staging, 0, D3D11_MAP_WRITE, 0, Some(&mut map))
.context("Map(NV12 staging)")?;
let pitch = map.RowPitch as usize;
let base = map.pData as *mut u8;
// A diagonal luma ramp that shifts 4 codes/frame — strong, deterministic motion.
let shift = (self.frame_idx as u32).wrapping_mul(4);
for y in 0..self.height {
let row = base.add(y as usize * pitch);
for x in 0..self.width {
*row.add(x as usize) = x.wrapping_add(y).wrapping_add(shift) as u8;
}
}
// UV plane (neutral gray = 128) at offset RowPitch*H: H/2 rows, `width` bytes each
// (width/2 interleaved Cb,Cr pairs).
let uv = base.add(pitch * self.height as usize);
for r in 0..(self.height / 2) {
let row = uv.add(r as usize * pitch);
for c in 0..self.width {
*row.add(c as usize) = 128;
}
}
self.context.Unmap(&self.staging, 0);
self.context.CopyResource(&self.default_tex, &self.staging);
}
self.frame_idx += 1;
Ok(CapturedFrame {
width: self.width,
height: self.height,
pts_ns,
format: PixelFormat::Nv12,
payload: FramePayload::D3d11(D3d11Frame {
texture: self.default_tex.clone(),
device: self.device.clone(),
}),
})
}
}
/// Resolve the same render adapter the encoder will pick (`PUNKTFUNK_RENDER_ADAPTER` / preference /
/// max-VRAM LUID), falling back to adapter 0.
///
/// # Safety
/// Calls DXGI factory/adapter enumeration; returns owned COM objects or an error.
unsafe fn resolve_render_adapter() -> Result<IDXGIAdapter1> {
let factory: IDXGIFactory4 = CreateDXGIFactory1().context("CreateDXGIFactory1")?;
if let Some(luid) = crate::win_adapter::resolve_render_adapter_luid() {
if let Ok(a) = factory.EnumAdapterByLuid::<IDXGIAdapter1>(luid) {
return Ok(a);
}
}
factory.EnumAdapters1(0).context("EnumAdapters1(0)")
}
/// Create an NV12 `Texture2D` with the given usage/CPU-access/bind flags.
///
/// # Safety
/// `device` must be a live D3D11 device; the returned texture is owned by the caller.
unsafe fn create_nv12(
device: &ID3D11Device,
width: u32,
height: u32,
usage: D3D11_USAGE,
cpu_access: u32,
bind: u32,
) -> Result<ID3D11Texture2D> {
let desc = D3D11_TEXTURE2D_DESC {
Width: width,
Height: height,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_NV12,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: usage,
BindFlags: bind,
CPUAccessFlags: cpu_access,
..Default::default()
};
let mut tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&desc, None, Some(&mut tex))
.context("CreateTexture2D(NV12)")?;
tex.context("CreateTexture2D returned a null NV12 texture")
}