punktfunk/design/native-amf-encoder.md

# Native AMF encoder — handoff design

> **Status: PHASES 1 + 2 + 3 IMPLEMENTED (2026-07-06).** `encode/windows/amf.rs` ships the
> direct-SDK encoder per §3 — FFI pinned to AMF headers v1.4.36, bounded poll, native `reset()`.
> Phase 2: **AV1** (open-time probe gate; per-codec enum divergences honored — AV1 swaps the
> ULL/LL usage values and uses GOP=0 + FORCE_FRAME_TYPE_KEY=1), **intra-refresh**
> (`PUNKTFUNK_INTRA_REFRESH` opt-in mirroring Linux NVENC; `caps().intra_refresh` reflects the
> driver's actual acceptance), **in-band HDR mastering/CLL metadata** (`*InHDRMetadata` host
> buffer; HEVC + AV1), and the **native codec probe**. Phase 3: **the ffmpeg-AMF dispatch
> fallback + `PUNKTFUNK_AMF_FFMPEG` hatch are deleted** — AMD dispatch / codec advertisement /
> 4:4:4 answer are native-only; FFmpeg serves QSV only (`ffmpeg_win.rs`'s AMF machinery is kept
> solely as the A/B comparator). `windows_backend_is_ffmpeg` → `windows_backend_is_probed`. **The
> §7 field-silence gate on Phase 3 was pre-empted on explicit direction** — see the §7 gate note
> for what that costs (VP-format-fallback now fails the session; AMFVideoConverter is the owed
> native fix).
> Live-validated on the lab Ryzen iGPU (VCN 3): AVC + HEVC batches across a native in-place reset
> (Annex-B IDR contract, FIFO pairing); HEVC Main10 P010 with the mastering + CLL prefix SEIs
> **confirmed present in the encoded IDR**; intra-refresh property accepted on both codecs; probe
> honestly answers h264/h265=true, av1=false on this RDNA2 part. The **§5.2 latency A/B is
> measured** (`amf_latency_ab_bench`, 1080p60 HEVC): native `encode_us` p50 **5.18 ms (0.31 frame
> periods)** vs libavcodec-AMF **16.9 ms (1.01)** — 3.3× lower, the frame-hold gone. Owed: §5.3
> on-glass session behaviors + soak (macOS-client on-glass test in progress), and field
> validation on ≥2 VCN generations (AV1/RDNA3 has no lab hardware).
> Companion context: the encode-stall watchdog + `Encoder::reset()` (punktfunk1.rs / encode.rs,
> shipped 2026-07-06) and the QSV backend module docs in `encode/windows/ffmpeg_win.rs`.

## 1. Why (measured, not speculative)

Three independent reasons, in order of weight:

1. **The libavcodec AMF wrapper's structural ~2-frame output hold.** `hevc_amf`/`av1_amf`
   need frame N+2 submitted before they release frame N's AU. Measured on the Ryzen 7000
   iGPU (VCN, 720p60): encode→retrieve **36 ms p50, dead-stable**, invariant across pipeline
   depth 1/2, every `usage` preset, and any spin budget (a 150 ms poll spin provably never
   produced the owed AU — it pegged at exactly 150 ms). See the `poll` doc comment in
   `ffmpeg_win.rs`. The direct-SDK NVENC path retrieves in ~1–2 ms. At 60 Hz this is ~33 ms
   of pure pipeline latency no FFmpeg-side knob can remove; at 120 Hz it is two whole frame
   budgets.
2. **Silent driver wedges surface as forever-EAGAIN, not errors.** The field failure
   (AMD/Intel streams freezing after ~3–5 min) was invisible because the wrapper's only
   "not ready" signal is EAGAIN, indistinguishable from a healthy pipeline warming up. The
   2026-07-06 watchdog converts that into a bounded rebuild + IDR, but it is a safety net
   with a ~2 s detection floor. The AMF runtime itself returns typed `AMF_RESULT` codes
   (`AMF_INPUT_FULL`, device-lost, etc.) — a native path sees the wedge on the frame it
   happens.
3. **Feature gaps libavcodec cannot express.** No intra-refresh wave (every
   FEC-unrecoverable loss is answered with a full IDR — the 20–40× frame-size spike the
   Linux NVENC intra-refresh mode exists to avoid), no in-band HDR mastering SEI
   (`EncoderCaps::supports_hdr_metadata` is NVENC-only today), coarse per-frame control.

## 2. The decision: drop FFmpeg for AMF, keep it for QSV

**Native AMF replaces the libavcodec AMF path (phased, §7). FFmpeg stays for QSV.**

- QSV via libavcodec with `async_depth=1` + `low_power` VDEnc is already near the hardware
  latency floor; a direct libvpl port would buy little for its cost. Revisit only if Intel
  field data shows a QSV-specific gap (separate doc if so).
- Because QSV stays on FFmpeg, the FFmpeg DLLs keep shipping and the `amf-qsv` build
  feature keeps existing. Dropping FFmpeg *entirely* is therefore not on the table here —
  "drop" means: the AMF dispatch stops going through it once the native path is validated.
- During bring-up the ffmpeg-AMF path remains as an automatic open-failure fallback and an
  explicit escape hatch (§7), then its AMF dispatch is deleted in Phase 3. Two permanently
  maintained AMF paths would double the driver-matrix burden, and the one we'd keep "for
  safety" is precisely the one with the wedge/latency pathology.

## 3. Architecture

New module `crates/punktfunk-host/src/encode/windows/amf.rs` implementing
`crate::encode::Encoder`, compiled unconditionally on Windows (**no new build-time
dependency and no new cargo feature**): the AMF runtime is loaded at runtime from the
driver-installed `amfrt64.dll`, exactly as `nvenc.rs` loads `nvEncodeAPI64.dll`
(`load_api`). A box without an AMD driver simply fails the open and the dispatch falls
through. In-tree FFI decl module (`amf_sys` submodule or `#[repr(C)]` blocks in-file,
mirroring the small interface subset we use) — model it on how `ffmpeg_win.rs` mirrors
`AVD3D11VADeviceContext` rather than pulling a binding crate (none is maintained).

### 3.1 FFI strategy (the load-bearing detail)

The AMF public headers (GPUOpen `AMF/amf/public/include`) define **C-compatible vtable
structs** for every interface (`AMFFactoryVtbl`, `AMFContextVtbl`, `AMFComponentVtbl`,
`AMFSurfaceVtbl`, `AMFDataVtbl`, `AMFBufferVtbl`, `AMFVariantStruct`, …) — this is not a
guess: FFmpeg's `amfenc.c` is plain C and drives AMF exclusively through those vtables, so
the C ABI is the stable, supported surface. Mirror only what we call:

- Entry points: `GetProcAddress("AMFQueryVersion")` (gate: runtime ≥ the pinned
  `AMF_FULL_VERSION` we mirror headers from) and `GetProcAddress("AMFInit")` →
  `AMFFactory*`.
- `factory->CreateContext` → `context->InitDX11(capturer_device, AMF_DX11_1)` — **the
  capturer's own `ID3D11Device`**, same-device requirement as every other backend (the
  capture textures are not shared-handle; see the `ensure_inner_d3d11` rebind logic in
  `ffmpeg_win.rs` for the device-change lifecycle to replicate).
- `factory->CreateComponent(context, name)` with `AMFVideoEncoderVCE_AVC` /
  `AMFVideoEncoder_HEVC` / `AMFVideoEncoder_AV1` → `encoder->Init(AMF_SURFACE_NV12|P010,
  w, h)`.
- Per-frame: `context->CreateSurfaceFromDX11Native(texture, &surface, observer)` →
  per-surface properties (pts via `SetPts`, forced-IDR picture type) →
  `encoder->SubmitInput(surface)`; retrieve via `encoder->QueryOutput(&data)`
  (`AMF_REPEAT` = not ready), `AMFBuffer::GetNative/GetSize` → `EncodedFrame`.
- Every mirrored struct/call carries a `// SAFETY:` proof — the whole encode module tree is
  under `#![deny(clippy::undocumented_unsafe_blocks)]` (unsafe-proof program).

Reference implementations to crib from (read both before writing FFI): FFmpeg `amfenc.c`
(the C vtbl usage, property plumbing, result-code handling) and OBS
`plugins/obs-ffmpeg/texture-amf.cpp` (D3D11 texture submission + low-latency streaming
config, C++ but the call sequence is what matters). **Verify every property name against
the pinned SDK headers** — names below are from those references and must not be trusted
blind.

### 3.2 Input path (zero-copy by construction)

Own a small D3D11 texture ring (NV12 or P010, `D3D11_BIND_RENDER_TARGET |
SHADER_RESOURCE`, size = `pipeline_depth + 2`), `CopySubresourceRegion` the captured
texture into the next slot (GPU-local, same pattern as `ZeroCopyInner::submit`), wrap the
slot with `CreateSurfaceFromDX11Native`, submit. The copy decouples the encoder from the
capturer's rotating IDD ring; do NOT wrap the capturer's texture directly. This makes
`PUNKTFUNK_ZEROCOPY` irrelevant for native AMF — there is no readback path to fall back
to. Handle the capturer's video-processor format fallback (`Bgra`/`Rgb10a2` instead of
NV12/P010 — see `pool_mismatch` in `ffmpeg_win.rs`) by returning an open/submit error in
Phase 1 so dispatch falls back to the ffmpeg path; an AMFVideoConverter front-end is a
Phase 2 option if that fallback ever fires in the field. `FramePayload::Cpu` (DDA without
video processor): same treatment — ffmpeg fallback in Phase 1.

### 3.3 Retrieval model

Bounded-blocking poll, the `vaapi.rs::poll` model: after `SubmitInput`, spin
`QueryOutput` with ~250 µs sleeps up to a budget of `min(3/4 frame interval, 12 ms)`; on
expiry return `Ok(None)` (the session loop keeps the frame in flight and the watchdog
arbitrates wedges). VCN encode at streaming settings is ~1–5 ms, so the AU ships the same
tick — this is where the ~2-frame hold dies. Expected observable — measure **`encode_us`
(submit→AU, in `FrameMsg`/the web-console stats), not `wait_us`**: on the ffmpeg path the
hold hides in `encode_us` (~2 frame periods) because its non-blocking poll returns EAGAIN
in ~2 µs; on the native bounded poll the ASIC wait becomes visible as a few ms of
`wait_us` while `encode_us` collapses to ~1 frame period or less.

### 3.4 Encoder configuration (initial property set)

Mirror the intent of the ffmpeg opts block in `open_win_encoder` (`ffmpeg_win.rs:216-247`).
AVC names given; HEVC/AV1 have `_HEVC_`/`_AV1_` twins — check headers:

| Intent | AMF property (verify!) |
| --- | --- |
| usage preset (keep `PUNKTFUNK_AMF_USAGE` mapping) | `AMF_VIDEO_ENCODER_USAGE` = `ULTRA_LOW_LATENCY` (default) |
| CBR, target==peak | `RATE_CONTROL_METHOD=CBR`, `TARGET_BITRATE`, `PEAK_BITRATE` |
| 1-frame VBV (keep `PUNKTFUNK_VBV_FRAMES`) | `VBV_BUFFER_SIZE` |
| HRD + no filler | `ENFORCE_HRD=true`, `FILLER_DATA_ENABLE=false` |
| latency-first quality | `QUALITY_PRESET=SPEED` |
| no B-frames (AVC) | `B_PIC_PATTERN=0` |
| infinite GOP | `IDR_PERIOD=0` (HEVC: `GOP_SIZE`/`NUM_GOPS_PER_IDR` — check) |
| low-latency submission | `LOWLATENCY_MODE=true` (newer SDKs) |
| in-band VPS/SPS/PPS on IDR (wire contract: `EncodedFrame` doc) | HEVC `HEADER_INSERTION_MODE=IDR_ALIGNED`; AVC `HEADER_INSERTION_SPACING` — check |
| SDR/HDR VUI | `FULL_RANGE_COLOR=false` + color primaries/transfer/matrix props (BT.709 vs BT.2020-PQ, mirroring `open_win_encoder`) |
| 10-bit | `COLOR_BIT_DEPTH=10` + P010 surfaces |
| per-frame forced IDR | on the input surface: `AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE=IDR` |
| intra-refresh wave (Phase 2) | AVC `INTRA_REFRESH_NUM_MBS_PER_SLOT`; HEVC CTB twin — check |
| HDR mastering SEI (Phase 2) | HEVC `INPUT_HDR_METADATA` (`AMFHDRMetadata` buffer) |

`SetProperty` failures on *optional* properties (LOWLATENCY_MODE, intra-refresh) must be
log-and-continue, not fatal — availability varies by VCN generation/driver.

### 3.5 Error + stall semantics (interplay with the 2026-07-06 watchdog)

- `SubmitInput` → `AMF_INPUT_FULL`: **back-pressure, NOT a wedge — drain and retry, do not
  reset.** (Original prescription "return `Err` → in-place reset" was **wrong**, disproven
  on-glass 2026-07-06: at throughput-ceiling loads — 5120x1440@240 P010 on the lab iGPU —
  `INPUT_FULL` → reset → forced IDR → a bigger keyframe → worse overload → a ~320 ms
  reset/IDR cascade, strictly worse than the libavcodec path's 16-deep input queue riding it
  out as latency. The log showed dozens of `submit failed … AMF_INPUT_FULL … rebuilt in place`
  and **zero** watchdog stalls.) The shipped handling: `submit` bounds in-flight surfaces below
  the input ring depth (`pending.len() < RING`) by draining finished AUs (buffered in a `ready`
  deque for `poll`, FIFO-preserved) to free a slot *before* reusing it, and treats a stray
  `INPUT_FULL` from `SubmitInput` the same way (drain + retry the surface). Only a drain that
  makes NO progress for a bounded budget (`INPUT_DRAIN_BUDGET`, 200 ms — well under the
  session watchdog's ~2 s) is a genuine wedge that escalates to `Err` → the in-place reset. This
  also closed a **latent corruption**: the old path let in-flight grow to AMF's internal input
  queue limit (16) against a ring of 4, so surfaces referenced ring slots already overwritten —
  the reset masked it. Any other non-OK `SubmitInput` result: `Err`.
- `QueryOutput` → `AMF_REPEAT`: keep spinning within the poll budget, then `Ok(None)`.
  `AMF_EOF`: `Ok(None)` after flush. Anything else: `Err` (the loop's poll-error path
  resets).
- Implement `Encoder::reset()` natively: `encoder->Drain/Flush`, `Terminate()`, re-`Init`
  on the same context (fall back to full context teardown if re-Init fails). Cheaper and
  more targeted than the ffmpeg path's drop-and-lazily-reopen.
- `caps()`: `supports_rfi: false` (AMF has no NVENC-style reference invalidation —
  intra-refresh is the substitute), `intra_refresh: true` once Phase 2 lands (this flag is
  what makes the session glue rate-limit client keyframe requests — see the `IDR_WINDOW`
  logic in punktfunk1.rs), `supports_hdr_metadata: true` once the SEI lands,
  `chroma_444: false` **permanently** (VCN hardware does not encode 4:4:4;
  `probe_can_encode_444` stays false — this is not an FFmpeg limitation).

### 3.6 Encoder trait contract (do not break)

From `encode.rs` + the punktfunk1 loop: AUs must come out FIFO in submit order
(`inflight.pop_front()` pairs with poll order); `data` is Annex-B with in-band headers on
IDRs (both a playable ES and self-contained wire AUs); `poll` returning `Ok(None)` is
legal and watchdog-arbitrated; `submit` must never block indefinitely; after `flush()`,
`poll` drains remaining AUs then returns `Ok(None)`; single encode thread owns the
encoder (manual `unsafe impl Send` with the same proof shape as `FfmpegWinEncoder`).

## 4. Integration seams (exact)

- `encode.rs::open_video_backend`, `WindowsBackend::Amf` arm: try `amf::AmfEncoder::open`
  first; on `Err`, `tracing::warn!` + fall back to `ffmpeg_win` (when the `amf-qsv`
  feature is built) — the same graceful-degrade shape as zero-copy→system today. Escape
  hatch: `PUNKTFUNK_AMF_FFMPEG=1` skips the native path (field triage). Phase 3 deletes
  both the fallback arm and the hatch.
- `resolved_backend_label` / `crate::gpu` session record: new label `"amf"` stays (the
  mgmt API shows the same name; add `"amf-ffmpeg"` only if the fallback fires, so field
  logs distinguish the paths).
- `probe_can_encode` (GameStream codec advertisement) and `windows_codec_support`: replace
  the ffmpeg open-probe with a native factory probe (`CreateComponent` per codec on the
  selected adapter) once Phase 2 lands; cache shape stays.
- `can_encode_444`: unchanged (`false`).
- The encode-stall watchdog and `Encoder::reset` (punktfunk1.rs): unchanged — it remains
  the backstop for in-FFI hangs the native path can't self-detect.

## 5. Validation plan (this box has an AMD iGPU — use it)

Baseline first, on the ffmpeg path (already deployed 2026-07-06 with the watchdog): a
long session on the iGPU with `PUNKTFUNK_PERF=1`, record `wait_us_p50/p99`,
`encode_us`, client-measured latency, and whether the watchdog ever fires. Then per phase:

1. Open/probe smoke per codec (AVC, HEVC, HEVC-10) on the iGPU. **DONE** — the gated live
   tests in `amf.rs` (`amf_encode_live_smoke` AVC+HEVC+AV1-probe, `amf_hdr_encode_live_smoke`,
   `amf_native_probe_live`, `amf_intra_refresh_property_live`) pass on the lab Ryzen iGPU
   (VCN3/RDNA2): both codecs across a native `reset()`, HEVC Main10 IDR carrying the
   mastering(137)+CLL(144) SEIs byte-verified, intra-refresh property accepted, probe honestly
   `h264/h265=true, av1=false`.
2. A/B the encode latency: expect `encode_us` p50 ~2 frame periods → ≤ 1 frame period
   (see §3.3 for why `wait_us` is the wrong metric on the ffmpeg side). **MEASURED**
   2026-07-06 by the gated `amf_latency_ab_bench` (`PUNKTFUNK_AMF_BENCH=1`, 1080p60 HEVC,
   180 paced frames, same D3D11 NV12 input to both encoders, lab iGPU, debug build):
   native `encode_us` p50 **5.18 ms (0.31 frame periods)** / p99 5.81 ms vs libavcodec-AMF
   p50 **16.9 ms (1.01 frame periods)** / p99 17.5 ms — **3.3× lower, ~11.7 ms/frame saved**,
   and the native path is decisively sub-frame (the ~2-frame hold that used to live in
   `encode_us` is gone). Note the ffmpeg baseline came in at ~1 frame period, not the ~2 this
   plan projected: the shipping ffmpeg config already sets AMF `latency=true` (a ~1-frame
   hold), so the realized win is 3.3× / ~12 ms rather than the ~30 ms projected against an
   un-tuned 2-frame baseline; direction and sub-frame collapse are exactly as §3.3 described.
   Release builds should show a lower native number still (debug charges host-side
   surface-create + copy-submit into the 5.18 ms). Zero-copy baseline for the input side
   already measured 2026-07-06 on the lab iGPU (1080p120 HDR P010): `submit_us` p50 2.7–2.9 ms
   (system readback) → **0.26 ms** (zero-copy D3D11), p99 6.6 ms → 0.5 ms.
3. Behavior: IDR on connect; mode switch mid-stream; HDR session (PQ VUI + 0xCE
   convergence); client keyframe-request recovery; encoder `reset()` under an injected
   failure; ≥30 min soak for the freeze class (watchdog log line
   `encode stall detected` must NOT appear).
4. Driver matrix beyond the lab box is field data: VCN1 (Raven) through VCN4/5 differ in
   preset support — the optional-property tolerance in §3.4 is what absorbs this.

## 6. Risks / open questions

- **Vtable mirroring correctness** is the concentrated risk: pin one AMF header version in
  a comment, mirror minimally, and unit-test `AMFQueryVersion`/`AMFInit` + a headless
  `CreateComponent` probe (skips cleanly on non-AMD boxes, like the NVENC live-gated
  tests).
- Per-frame `CreateSurfaceFromDX11Native` allocation churn — if it shows up in profiles,
  AMF supports pre-created surface pools; start simple.
- AV1 is RDNA3+; probe, never assume (same rule as everywhere in this codebase).
- Hybrid boxes: context must init on the *selected* adapter's device (the capture
  device) — inherited for free by taking the capturer's device, but test with the
  web-console GPU preference pointed at each GPU.
- The AMF runtime ships with the AMD driver, not with us — a missing/ancient `amfrt64.dll`
  must produce a clean "install/update the AMD driver" error at open, then fall back
  (Phase 1) or fail the session with that message (Phase 3).

## 7. Phasing

| Phase | Scope | Exit criterion | Status |
| --- | --- | --- | --- |
| 1 | FFI layer + AVC/HEVC (SDR + 10-bit HDR), bounded poll, native `reset()`, dispatch with ffmpeg fallback + `PUNKTFUNK_AMF_FFMPEG` hatch | §5.2–5.3 pass on the lab iGPU | **DONE** 2026-07-06 (§5.2 measured; §5.3 on-glass in progress) |
| 2 | Intra-refresh (`caps().intra_refresh`), in-band HDR SEI (`supports_hdr_metadata`), AV1, native codec probe | field-validated on ≥2 VCN generations | **CODE DONE** 2026-07-06 (lab VCN3 only; AV1/RDNA3 + 2nd VCN gen still owed) |
| 3 | Delete the ffmpeg-AMF dispatch arm + hatch; FFmpeg remains QSV-only | one release of field silence on the fallback label | **DONE** 2026-07-06 — see the gate note below |

**Phase 3 gate note (honesty):** the stated exit criterion (one release of field silence on the
fallback label) was **NOT met** — Phase 3 was cut the same day the native path was written, on
explicit direction, alongside a live macOS-client on-glass test. What Phase 3 removed: the
`WindowsBackend::Amf` libavcodec fallback arm, the `PUNKTFUNK_AMF_FFMPEG` hatch, and the
AMF→ffmpeg routes in `windows_codec_support` / `can_encode_444`. AMD dispatch, codec
advertisement, and the 4:4:4 answer are all native-only now; FFmpeg (`ffmpeg_win.rs`) is reached
only for QSV in production (its `WinVendor::Amf` machinery is retained solely as the
`amf_latency_ab_bench` comparator, not deleted — excising it would churn the Intel-unvalidated
QSV code for no gain). **Residual risk this pre-emption carries:** with the ffmpeg readback path
gone, an AMD box whose capturer can't produce video-processor NV12/P010 (falls back to
Bgra/Rgb10a2, or hands DDA CPU frames) now **fails the session** instead of degrading — the
design's answer is the native AMFVideoConverter front-end (§3.2), owed if that fallback is ever
seen in the field. Not observed on lab hardware (the VP yields NV12/P010). Reverting Phase 3 is a
small, localized diff if field data turns up trouble.