diff --git a/design/native-amf-encoder.md b/design/native-amf-encoder.md new file mode 100644 index 0000000..2682828 --- /dev/null +++ b/design/native-amf-encoder.md @@ -0,0 +1,228 @@ +# Native AMF encoder — handoff design + +> **Status: PROPOSED (handoff, 2026-07-06).** Design + decision record for replacing the +> libavcodec `*_amf` path with a direct AMF SDK encoder on Windows AMD +> (`encode/windows/amf.rs`, the AMD analogue of the direct-SDK `encode/windows/nvenc.rs`). +> Written for an implementer without prior context; every in-repo seam is named. Companion +> context: the encode-stall watchdog + `Encoder::reset()` (punktfunk1.rs / encode.rs, shipped +> 2026-07-06) and the AMF/QSV backend module docs in `encode/windows/ffmpeg_win.rs`. + +## 1. Why (measured, not speculative) + +Three independent reasons, in order of weight: + +1. **The libavcodec AMF wrapper's structural ~2-frame output hold.** `hevc_amf`/`av1_amf` + need frame N+2 submitted before they release frame N's AU. Measured on the Ryzen 7000 + iGPU (VCN, 720p60): encode→retrieve **36 ms p50, dead-stable**, invariant across pipeline + depth 1/2, every `usage` preset, and any spin budget (a 150 ms poll spin provably never + produced the owed AU — it pegged at exactly 150 ms). See the `poll` doc comment in + `ffmpeg_win.rs`. The direct-SDK NVENC path retrieves in ~1–2 ms. At 60 Hz this is ~33 ms + of pure pipeline latency no FFmpeg-side knob can remove; at 120 Hz it is two whole frame + budgets. +2. **Silent driver wedges surface as forever-EAGAIN, not errors.** The field failure + (AMD/Intel streams freezing after ~3–5 min) was invisible because the wrapper's only + "not ready" signal is EAGAIN, indistinguishable from a healthy pipeline warming up. The + 2026-07-06 watchdog converts that into a bounded rebuild + IDR, but it is a safety net + with a ~2 s detection floor. The AMF runtime itself returns typed `AMF_RESULT` codes + (`AMF_INPUT_FULL`, device-lost, etc.) — a native path sees the wedge on the frame it + happens. +3. **Feature gaps libavcodec cannot express.** No intra-refresh wave (every + FEC-unrecoverable loss is answered with a full IDR — the 20–40× frame-size spike the + Linux NVENC intra-refresh mode exists to avoid), no in-band HDR mastering SEI + (`EncoderCaps::supports_hdr_metadata` is NVENC-only today), coarse per-frame control. + +## 2. The decision: drop FFmpeg for AMF, keep it for QSV + +**Native AMF replaces the libavcodec AMF path (phased, §7). FFmpeg stays for QSV.** + +- QSV via libavcodec with `async_depth=1` + `low_power` VDEnc is already near the hardware + latency floor; a direct libvpl port would buy little for its cost. Revisit only if Intel + field data shows a QSV-specific gap (separate doc if so). +- Because QSV stays on FFmpeg, the FFmpeg DLLs keep shipping and the `amf-qsv` build + feature keeps existing. Dropping FFmpeg *entirely* is therefore not on the table here — + "drop" means: the AMF dispatch stops going through it once the native path is validated. +- During bring-up the ffmpeg-AMF path remains as an automatic open-failure fallback and an + explicit escape hatch (§7), then its AMF dispatch is deleted in Phase 3. Two permanently + maintained AMF paths would double the driver-matrix burden, and the one we'd keep "for + safety" is precisely the one with the wedge/latency pathology. + +## 3. Architecture + +New module `crates/punktfunk-host/src/encode/windows/amf.rs` implementing +`crate::encode::Encoder`, compiled unconditionally on Windows (**no new build-time +dependency and no new cargo feature**): the AMF runtime is loaded at runtime from the +driver-installed `amfrt64.dll`, exactly as `nvenc.rs` loads `nvEncodeAPI64.dll` +(`load_api`). A box without an AMD driver simply fails the open and the dispatch falls +through. In-tree FFI decl module (`amf_sys` submodule or `#[repr(C)]` blocks in-file, +mirroring the small interface subset we use) — model it on how `ffmpeg_win.rs` mirrors +`AVD3D11VADeviceContext` rather than pulling a binding crate (none is maintained). + +### 3.1 FFI strategy (the load-bearing detail) + +The AMF public headers (GPUOpen `AMF/amf/public/include`) define **C-compatible vtable +structs** for every interface (`AMFFactoryVtbl`, `AMFContextVtbl`, `AMFComponentVtbl`, +`AMFSurfaceVtbl`, `AMFDataVtbl`, `AMFBufferVtbl`, `AMFVariantStruct`, …) — this is not a +guess: FFmpeg's `amfenc.c` is plain C and drives AMF exclusively through those vtables, so +the C ABI is the stable, supported surface. Mirror only what we call: + +- Entry points: `GetProcAddress("AMFQueryVersion")` (gate: runtime ≥ the pinned + `AMF_FULL_VERSION` we mirror headers from) and `GetProcAddress("AMFInit")` → + `AMFFactory*`. +- `factory->CreateContext` → `context->InitDX11(capturer_device, AMF_DX11_1)` — **the + capturer's own `ID3D11Device`**, same-device requirement as every other backend (the + capture textures are not shared-handle; see the `ensure_inner_d3d11` rebind logic in + `ffmpeg_win.rs` for the device-change lifecycle to replicate). +- `factory->CreateComponent(context, name)` with `AMFVideoEncoderVCE_AVC` / + `AMFVideoEncoder_HEVC` / `AMFVideoEncoder_AV1` → `encoder->Init(AMF_SURFACE_NV12|P010, + w, h)`. +- Per-frame: `context->CreateSurfaceFromDX11Native(texture, &surface, observer)` → + per-surface properties (pts via `SetPts`, forced-IDR picture type) → + `encoder->SubmitInput(surface)`; retrieve via `encoder->QueryOutput(&data)` + (`AMF_REPEAT` = not ready), `AMFBuffer::GetNative/GetSize` → `EncodedFrame`. +- Every mirrored struct/call carries a `// SAFETY:` proof — the whole encode module tree is + under `#![deny(clippy::undocumented_unsafe_blocks)]` (unsafe-proof program). + +Reference implementations to crib from (read both before writing FFI): FFmpeg `amfenc.c` +(the C vtbl usage, property plumbing, result-code handling) and OBS +`plugins/obs-ffmpeg/texture-amf.cpp` (D3D11 texture submission + low-latency streaming +config, C++ but the call sequence is what matters). **Verify every property name against +the pinned SDK headers** — names below are from those references and must not be trusted +blind. + +### 3.2 Input path (zero-copy by construction) + +Own a small D3D11 texture ring (NV12 or P010, `D3D11_BIND_RENDER_TARGET | +SHADER_RESOURCE`, size = `pipeline_depth + 2`), `CopySubresourceRegion` the captured +texture into the next slot (GPU-local, same pattern as `ZeroCopyInner::submit`), wrap the +slot with `CreateSurfaceFromDX11Native`, submit. The copy decouples the encoder from the +capturer's rotating IDD ring; do NOT wrap the capturer's texture directly. This makes +`PUNKTFUNK_ZEROCOPY` irrelevant for native AMF — there is no readback path to fall back +to. Handle the capturer's video-processor format fallback (`Bgra`/`Rgb10a2` instead of +NV12/P010 — see `pool_mismatch` in `ffmpeg_win.rs`) by returning an open/submit error in +Phase 1 so dispatch falls back to the ffmpeg path; an AMFVideoConverter front-end is a +Phase 2 option if that fallback ever fires in the field. `FramePayload::Cpu` (DDA without +video processor): same treatment — ffmpeg fallback in Phase 1. + +### 3.3 Retrieval model + +Bounded-blocking poll, the `vaapi.rs::poll` model: after `SubmitInput`, spin +`QueryOutput` with ~250 µs sleeps up to a budget of `min(3/4 frame interval, 12 ms)`; on +expiry return `Ok(None)` (the session loop keeps the frame in flight and the watchdog +arbitrates wedges). VCN encode at streaming settings is ~1–5 ms, so the AU ships the same +tick — this is where the ~2-frame hold dies. Expected observable — measure **`encode_us` +(submit→AU, in `FrameMsg`/the web-console stats), not `wait_us`**: on the ffmpeg path the +hold hides in `encode_us` (~2 frame periods) because its non-blocking poll returns EAGAIN +in ~2 µs; on the native bounded poll the ASIC wait becomes visible as a few ms of +`wait_us` while `encode_us` collapses to ~1 frame period or less. + +### 3.4 Encoder configuration (initial property set) + +Mirror the intent of the ffmpeg opts block in `open_win_encoder` (`ffmpeg_win.rs:216-247`). +AVC names given; HEVC/AV1 have `_HEVC_`/`_AV1_` twins — check headers: + +| Intent | AMF property (verify!) | +| --- | --- | +| usage preset (keep `PUNKTFUNK_AMF_USAGE` mapping) | `AMF_VIDEO_ENCODER_USAGE` = `ULTRA_LOW_LATENCY` (default) | +| CBR, target==peak | `RATE_CONTROL_METHOD=CBR`, `TARGET_BITRATE`, `PEAK_BITRATE` | +| 1-frame VBV (keep `PUNKTFUNK_VBV_FRAMES`) | `VBV_BUFFER_SIZE` | +| HRD + no filler | `ENFORCE_HRD=true`, `FILLER_DATA_ENABLE=false` | +| latency-first quality | `QUALITY_PRESET=SPEED` | +| no B-frames (AVC) | `B_PIC_PATTERN=0` | +| infinite GOP | `IDR_PERIOD=0` (HEVC: `GOP_SIZE`/`NUM_GOPS_PER_IDR` — check) | +| low-latency submission | `LOWLATENCY_MODE=true` (newer SDKs) | +| in-band VPS/SPS/PPS on IDR (wire contract: `EncodedFrame` doc) | HEVC `HEADER_INSERTION_MODE=IDR_ALIGNED`; AVC `HEADER_INSERTION_SPACING` — check | +| SDR/HDR VUI | `FULL_RANGE_COLOR=false` + color primaries/transfer/matrix props (BT.709 vs BT.2020-PQ, mirroring `open_win_encoder`) | +| 10-bit | `COLOR_BIT_DEPTH=10` + P010 surfaces | +| per-frame forced IDR | on the input surface: `AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE=IDR` | +| intra-refresh wave (Phase 2) | AVC `INTRA_REFRESH_NUM_MBS_PER_SLOT`; HEVC CTB twin — check | +| HDR mastering SEI (Phase 2) | HEVC `INPUT_HDR_METADATA` (`AMFHDRMetadata` buffer) | + +`SetProperty` failures on *optional* properties (LOWLATENCY_MODE, intra-refresh) must be +log-and-continue, not fatal — availability varies by VCN generation/driver. + +### 3.5 Error + stall semantics (interplay with the 2026-07-06 watchdog) + +- `SubmitInput` → `AMF_INPUT_FULL`: return `Err` from `submit` — the session loop's + submit-failure path runs the bounded in-place reset. Any other non-OK result: `Err`. +- `QueryOutput` → `AMF_REPEAT`: keep spinning within the poll budget, then `Ok(None)`. + `AMF_EOF`: `Ok(None)` after flush. Anything else: `Err` (the loop's poll-error path + resets). +- Implement `Encoder::reset()` natively: `encoder->Drain/Flush`, `Terminate()`, re-`Init` + on the same context (fall back to full context teardown if re-Init fails). Cheaper and + more targeted than the ffmpeg path's drop-and-lazily-reopen. +- `caps()`: `supports_rfi: false` (AMF has no NVENC-style reference invalidation — + intra-refresh is the substitute), `intra_refresh: true` once Phase 2 lands (this flag is + what makes the session glue rate-limit client keyframe requests — see the `IDR_WINDOW` + logic in punktfunk1.rs), `supports_hdr_metadata: true` once the SEI lands, + `chroma_444: false` **permanently** (VCN hardware does not encode 4:4:4; + `probe_can_encode_444` stays false — this is not an FFmpeg limitation). + +### 3.6 Encoder trait contract (do not break) + +From `encode.rs` + the punktfunk1 loop: AUs must come out FIFO in submit order +(`inflight.pop_front()` pairs with poll order); `data` is Annex-B with in-band headers on +IDRs (both a playable ES and self-contained wire AUs); `poll` returning `Ok(None)` is +legal and watchdog-arbitrated; `submit` must never block indefinitely; after `flush()`, +`poll` drains remaining AUs then returns `Ok(None)`; single encode thread owns the +encoder (manual `unsafe impl Send` with the same proof shape as `FfmpegWinEncoder`). + +## 4. Integration seams (exact) + +- `encode.rs::open_video_backend`, `WindowsBackend::Amf` arm: try `amf::AmfEncoder::open` + first; on `Err`, `tracing::warn!` + fall back to `ffmpeg_win` (when the `amf-qsv` + feature is built) — the same graceful-degrade shape as zero-copy→system today. Escape + hatch: `PUNKTFUNK_AMF_FFMPEG=1` skips the native path (field triage). Phase 3 deletes + both the fallback arm and the hatch. +- `resolved_backend_label` / `crate::gpu` session record: new label `"amf"` stays (the + mgmt API shows the same name; add `"amf-ffmpeg"` only if the fallback fires, so field + logs distinguish the paths). +- `probe_can_encode` (GameStream codec advertisement) and `windows_codec_support`: replace + the ffmpeg open-probe with a native factory probe (`CreateComponent` per codec on the + selected adapter) once Phase 2 lands; cache shape stays. +- `can_encode_444`: unchanged (`false`). +- The encode-stall watchdog and `Encoder::reset` (punktfunk1.rs): unchanged — it remains + the backstop for in-FFI hangs the native path can't self-detect. + +## 5. Validation plan (this box has an AMD iGPU — use it) + +Baseline first, on the ffmpeg path (already deployed 2026-07-06 with the watchdog): a +long session on the iGPU with `PUNKTFUNK_PERF=1`, record `wait_us_p50/p99`, +`encode_us`, client-measured latency, and whether the watchdog ever fires. Then per phase: + +1. Open/probe smoke per codec (AVC, HEVC, HEVC-10) on the iGPU. +2. A/B the encode latency: expect `encode_us` p50 ~2 frame periods → ≤ 1 frame period + (see §3.3 for why `wait_us` is the wrong metric on the ffmpeg side); client + capture→glass p50 drops by ~30 ms at 60 Hz / ~17 ms at 120 Hz. Zero-copy baseline for + the input side already measured 2026-07-06 on the lab iGPU (1080p120 HDR P010): + `submit_us` p50 2.7–2.9 ms (system readback) → **0.26 ms** (zero-copy D3D11), p99 + 6.6 ms → 0.5 ms. +3. Behavior: IDR on connect; mode switch mid-stream; HDR session (PQ VUI + 0xCE + convergence); client keyframe-request recovery; encoder `reset()` under an injected + failure; ≥30 min soak for the freeze class (watchdog log line + `encode stall detected` must NOT appear). +4. Driver matrix beyond the lab box is field data: VCN1 (Raven) through VCN4/5 differ in + preset support — the optional-property tolerance in §3.4 is what absorbs this. + +## 6. Risks / open questions + +- **Vtable mirroring correctness** is the concentrated risk: pin one AMF header version in + a comment, mirror minimally, and unit-test `AMFQueryVersion`/`AMFInit` + a headless + `CreateComponent` probe (skips cleanly on non-AMD boxes, like the NVENC live-gated + tests). +- Per-frame `CreateSurfaceFromDX11Native` allocation churn — if it shows up in profiles, + AMF supports pre-created surface pools; start simple. +- AV1 is RDNA3+; probe, never assume (same rule as everywhere in this codebase). +- Hybrid boxes: context must init on the *selected* adapter's device (the capture + device) — inherited for free by taking the capturer's device, but test with the + web-console GPU preference pointed at each GPU. +- The AMF runtime ships with the AMD driver, not with us — a missing/ancient `amfrt64.dll` + must produce a clean "install/update the AMD driver" error at open, then fall back + (Phase 1) or fail the session with that message (Phase 3). + +## 7. Phasing + +| Phase | Scope | Exit criterion | +| --- | --- | --- | +| 1 | FFI layer + AVC/HEVC (SDR + 10-bit HDR), bounded poll, native `reset()`, dispatch with ffmpeg fallback + `PUNKTFUNK_AMF_FFMPEG` hatch | §5.2–5.3 pass on the lab iGPU | +| 2 | Intra-refresh (`caps().intra_refresh`), in-band HDR SEI (`supports_hdr_metadata`), AV1, native codec probe | field-validated on ≥2 VCN generations | +| 3 | Delete the ffmpeg-AMF dispatch arm + hatch; FFmpeg remains QSV-only | one release of field silence on the fallback label |