The six built-in library scanners become plugins — M6/WP6.4 #199

Merged

enricobuehler merged 3 commits from worktree-remove-builtin-scanners into main 2026-08-13 15:19:44 +00:00

Conversation 0 Commits 3 Files Changed 24

+411 -2429

enricobuehler commented 2026-08-13 14:53:00 +00:00

Owner

Copy Link

Copy Source

Closes out the library-scanner migration: the host stops scanning launchers itself, and every game source becomes a plugin. This is WP6.4 in punktfunk-planning/design/library-scanner-plugins-implementation-plan.md, whose bridge half shipped in v0.26.0.

Intended for 0.28.0 — see "Release choreography" at the bottom.

What changes

library/{steam,epic,gog,heroic,lutris,xbox}.rs and the scanner_defs() table are deleted. GET /library/scanners now lists exactly what the operator has installed, and every row reports origin: "plugin".

A host with no library plugins installed has an empty grid. That is the upgrade note. The console's one-click install per source (the D9 nudge) is unchanged and still never auto-installs.

Why this could be a deletion rather than a rewrite

A plugin claims its store (D2), and a claimed entry surfaces under the deterministic <store>:<external_id> id the scanner used to produce. Entry ids, GameStream FNV-1a app ids, client-side art caches, Moonlight pins, the per-source toggles and the per-entry hides are all keyed on that id, and none of them move. library-scanners.json keeps its name, shape and contents — an operator who had steam switched off still has it switched off, with no migration step. That forward seam was designed in from the start; this PR is it paying off.

Kept deliberately

• launch.rs in full. Launch is host-owned by design D1 — a plugin publishes a validated value, the host builds the command — so every typed kind (steam_appid, steam_ui, launcher_ui, epic, gog, aumid, xbox, lutris_id, playnite) survives untouched.
• xbox_pfn() moved out of the deleted xbox.rs into launch.rs. Resolving a package Identity to its PackageFamilyName needs AppRepository enumeration, which is readable by the host (LocalSystem) and denied to the plugin runner (LocalService) — that measured asymmetry (2026-08-06) is the entire reason the xbox launch kind exists, so the resolver is launch vocabulary, not scanner vocabulary. Its test moved with it.
• SourceOrigin::Builtin. No host build emits it any more, but the web console ships as its own package and is expected to drive an N-1 host that still does, so the variant stays in the OpenAPI enum and the console keeps its builtin handling (and its migration banner, which simply never triggers against a v0.28.0 host).
• A store-label table, so a source row doesn't rename itself from "Steam" to steam the day its plugin takes over.

Removed with them

The background cover-art warmer and its on-disk cache — they existed only for the GOG and Xbox scanners, the two sources that had to ask a network catalog what a cover was; a plugin resolves art while it scans. Also the legacy steam: branch of the art proxy, and GameMeta::pc().

The host now makes no outbound HTTP request to build a library at all.

⚠ Dependency drop (packager-visible)

rusqlite (with its bundled, cc-compiled SQLite) and roxmltree leave the host's graph — audited, no other users. winreg stays: launch.rs, procscan/windows.rs and the two audio/windows/ modules still need it. base64/ureq stay, exactly as the M6 plan predicted. Cargo.lock loses that subtree; the one added line is cargo dropping a now-unneeded version qualifier because only one hashbrown remains.

Note on Xbox

The xbox plugin is published to the registry (0.1.1) but was never pinned into punktfunk-plugin-index, so it isn't installable from the console's catalog. Raised before starting; the built-in Xbox scanner never worked in practice, so nothing is lost by removing it. An index pin is still owed if Xbox/Game Pass should be reachable — that PR is the user's to merge.

Second commit

fix(web): retry the console image's dependency install once — unrelated to the above, requested alongside it. CI hit error: Fail extracting tarball for "@rolldown/binding-linux-x64-musl" in the console image build.

The lockfile is not stale, which is worth recording because it's the obvious suspect: bun install --frozen-lockfile accepts it, regenerating it with bun 1.3.14 (the version in the failing log) is byte-identical, the tarball downloads and extracts cleanly, and that exact layer builds green for --platform linux/amd64 with --no-cache. So the failure was a truncated download, and the commit adds one bounded retry — which recovers a truncated download and deliberately does not paper over a runner that is out of disk, since that fails identically on every attempt. Drop this commit if you'd rather just re-run the leg.

Gates

Both usual Linux boxes were down (VM 103 stopped, .70 unreachable), so verification ran through the documented Docker route under emulated amd64: cargo check/clippy --all-targets -D warnings/cargo test -p punktfunk-host. cargo fmt --all --check clean, biome clean on the console file. CI is the authoritative gate here — in particular the Windows legs, since the #[cfg(windows)] deletions (Epic/GOG/Xbox) can't be compiled from the Mac.

Not done, and owed: api/openapi.json + docs-site/public/openapi.json regen (description-only changes — no schema shape moved, since Builtin was kept), and the WP6.4 full client-matrix smoke against a plugin-only host.

Release choreography

The 0.28.0 release commit already exists on the local, unpushed worktree-release-0280 and renames the CHANGELOG's in-development heading to ## v0.28.0. This branch adds its entry under that existing heading on main, so it lands in the 0.28.0 section when the release is re-cut. The user-facing half (docs/releases/v0.28.0.md) lives only on that release branch, so it isn't touched here — the paragraph to fold in is the "Where your games come from" rewrite in docs-site/content/docs/game-library.md.

⚠ 0.28.0 was already prepared and on-glass verified before this landed; folding a library change of this size in re-opens that verification.

Closes out the library-scanner migration: the host stops scanning launchers itself, and every game source becomes a plugin. This is **WP6.4** in `punktfunk-planning/design/library-scanner-plugins-implementation-plan.md`, whose bridge half shipped in v0.26.0. Intended for **0.28.0** — see "Release choreography" at the bottom. ## What changes `library/{steam,epic,gog,heroic,lutris,xbox}.rs` and the `scanner_defs()` table are deleted. `GET /library/scanners` now lists exactly what the operator has installed, and every row reports `origin: "plugin"`. **A host with no library plugins installed has an empty grid.** That is the upgrade note. The console's one-click install per source (the D9 nudge) is unchanged and still never auto-installs. ## Why this could be a deletion rather than a rewrite A plugin **claims** its store (D2), and a claimed entry surfaces under the deterministic `<store>:<external_id>` id the scanner used to produce. Entry ids, GameStream FNV-1a app ids, client-side art caches, Moonlight pins, the per-source toggles and the per-entry hides are all keyed on that id, and none of them move. `library-scanners.json` keeps its name, shape and contents — an operator who had `steam` switched off still has it switched off, with no migration step. That forward seam was designed in from the start; this PR is it paying off. ## Kept deliberately - **`launch.rs` in full.** Launch is host-owned by design D1 — a plugin publishes a validated *value*, the host builds the command — so every typed kind (`steam_appid`, `steam_ui`, `launcher_ui`, `epic`, `gog`, `aumid`, `xbox`, `lutris_id`, `playnite`) survives untouched. - **`xbox_pfn()` moved** out of the deleted `xbox.rs` into `launch.rs`. Resolving a package Identity to its PackageFamilyName needs `AppRepository` enumeration, which is readable by the host (LocalSystem) and denied to the plugin runner (LocalService) — that measured asymmetry (2026-08-06) is the entire reason the `xbox` launch kind exists, so the resolver is launch vocabulary, not scanner vocabulary. Its test moved with it. - **`SourceOrigin::Builtin`.** No host build emits it any more, but the web console ships as its own package and is expected to drive an N-1 host that still does, so the variant stays in the OpenAPI enum and the console keeps its `builtin` handling (and its migration banner, which simply never triggers against a v0.28.0 host). - **A store-label table**, so a source row doesn't rename itself from "Steam" to `steam` the day its plugin takes over. ## Removed with them The background cover-art warmer and its on-disk cache — they existed only for the GOG and Xbox scanners, the two sources that had to ask a network catalog what a cover was; a plugin resolves art while it scans. Also the legacy `steam:` branch of the art proxy, and `GameMeta::pc()`. **The host now makes no outbound HTTP request to build a library at all.** ## ⚠ Dependency drop (packager-visible) `rusqlite` (with its bundled, `cc`-compiled SQLite) and `roxmltree` leave the host's graph — audited, no other users. `winreg` **stays**: `launch.rs`, `procscan/windows.rs` and the two `audio/windows/` modules still need it. `base64`/`ureq` stay, exactly as the M6 plan predicted. `Cargo.lock` loses that subtree; the one added line is cargo dropping a now-unneeded version qualifier because only one `hashbrown` remains. ## Note on Xbox The `xbox` plugin is published to the registry (0.1.1) but was never pinned into `punktfunk-plugin-index`, so it isn't installable from the console's catalog. Raised before starting; the built-in Xbox scanner never worked in practice, so nothing is lost by removing it. **An index pin is still owed** if Xbox/Game Pass should be reachable — that PR is the user's to merge. ## Second commit `fix(web): retry the console image's dependency install once` — unrelated to the above, requested alongside it. CI hit `error: Fail extracting tarball for "@rolldown/binding-linux-x64-musl"` in the console image build. **The lockfile is not stale**, which is worth recording because it's the obvious suspect: `bun install --frozen-lockfile` accepts it, regenerating it with bun 1.3.14 (the version in the failing log) is byte-identical, the tarball downloads and extracts cleanly, and that exact layer builds green for `--platform linux/amd64` with `--no-cache`. So the failure was a truncated download, and the commit adds one bounded retry — which recovers a truncated download and deliberately does *not* paper over a runner that is out of disk, since that fails identically on every attempt. Drop this commit if you'd rather just re-run the leg. ## Gates Both usual Linux boxes were down (VM 103 stopped, .70 unreachable), so verification ran through the documented Docker route under emulated amd64: `cargo check`/`clippy --all-targets -D warnings`/`cargo test -p punktfunk-host`. `cargo fmt --all --check` clean, biome clean on the console file. **CI is the authoritative gate here** — in particular the Windows legs, since the `#[cfg(windows)]` deletions (Epic/GOG/Xbox) can't be compiled from the Mac. Not done, and owed: **`api/openapi.json` + `docs-site/public/openapi.json` regen** (description-only changes — no schema shape moved, since `Builtin` was kept), and the WP6.4 full client-matrix smoke against a plugin-only host. ## Release choreography The 0.28.0 release commit already exists on the local, unpushed `worktree-release-0280` and renames the CHANGELOG's in-development heading to `## v0.28.0`. This branch adds its entry under that existing heading on `main`, so it lands in the 0.28.0 section when the release is re-cut. The user-facing half (`docs/releases/v0.28.0.md`) lives only on that release branch, so it isn't touched here — the paragraph to fold in is the "Where your games come from" rewrite in `docs-site/content/docs/game-library.md`. ⚠ 0.28.0 was already prepared and on-glass verified before this landed; folding a library change of this size in re-opens that verification.

enricobuehler added 2 commits 2026-08-13 14:53:03 +00:00

feat(host/library)!: the six built-in scanners become plugins — M6/WP6.4 ... 7403450a8d

The host no longer scans any launcher itself. `library/{steam,epic,gog,heroic,
lutris,xbox}.rs` and the `scanner_defs()` table are gone; `GET /library/scanners`
now lists exactly what the operator installed, every row `origin: "plugin"`.

This is the end of the migration whose bridge half shipped in v0.26.0. The
plugins have been published and index-pinned since 2026-08-08, so the
replacement has been in the field for the whole bridge window.

A host with no library plugins installed has an empty grid — that is the upgrade
note. The console's one-click install per source (the D9 nudge) is unchanged and
still never auto-installs.

Nothing about a title changes when its plugin takes over, and that is why this
could be a deletion rather than a rewrite: a plugin CLAIMS its store (D2), and a
claimed entry surfaces under the deterministic `<store>:<external_id>` id the
scanner used to produce. Entry ids, GameStream FNV-1a app ids, client art
caches, Moonlight pins, the per-source toggles and the per-entry hides all key on
that id and none of them move. `library-scanners.json` keeps its name, shape and
contents: an operator who had `steam` off still has it off, with no migration.

Kept deliberately:

* `launch.rs` in full. Launch is host-owned by design D1 — a plugin publishes a
  validated value, the host builds the command — so every typed kind survives.
  `xbox_pfn()` MOVED here out of the deleted `xbox.rs`: resolving a package
  Identity to its PackageFamilyName needs `AppRepository` enumeration, readable
  by the host (LocalSystem) and denied to the plugin runner (LocalService). That
  measured asymmetry is the whole reason the `xbox` launch kind exists, so the
  resolver is launch vocabulary, not scanner vocabulary.
* `SourceOrigin::Builtin`. No host build emits it, but the console ships as its
  own package and drives an N-1 host that still does, so the variant stays in the
  schema and the console keeps its `builtin` handling.
* A store-label table, so a source row does not rename itself from "Steam" to
  `steam` the day its plugin takes over.

Removed with the scanners: the background cover-art warmer and its on-disk cache
(they existed only for GOG and Xbox, the two sources that had to ask a network
catalog what a cover was — a plugin resolves art while it scans), the legacy
`steam:` branch of the art proxy, and `GameMeta::pc()`. The host now makes no
outbound HTTP request to build a library at all.

Dependency audit, as WP6.4 required: `rusqlite` (with its bundled, cc-compiled
SQLite) and `roxmltree` leave the graph — verified no other users. `winreg`
stays: `launch.rs`, `procscan/windows.rs` and two `audio/windows/` modules need
it. `base64`/`ureq` stay, exactly as the plan predicted.

A stale `library-art-cache.json` from an older host is ignored, not migrated.

fix(web): retry the console image's dependency install once ... e568513f74

A single failed tarball kills `bun install` and takes the whole image build with
it. Seen in CI as:

    error: Fail extracting tarball for "@rolldown/binding-linux-x64-musl"

— a 7.7 MB optional binding that bun fetches on any linux-x64 host (the lockfile
records `os`/`cpu` but no libc, so the musl and glibc bindings are equally
eligible) and that had arrived truncated.

The lockfile is NOT at fault, which is worth recording because it is the obvious
suspect: `bun install --frozen-lockfile` accepts it, regenerating it with bun
1.3.14 — the version in the failing log — is byte-identical, the tarball
downloads and extracts cleanly, and this exact layer builds green for
`--platform linux/amd64` with `--no-cache`.

So this is a transient-download guard, not a lockfile fix: two attempts with a
pause, then fail for real. It recovers a truncated download and deliberately does
NOT paper over a runner that is out of disk, which fails identically every time.

enricobuehler added 1 commit 2026-08-13 15:06:25 +00:00

chore(api): regenerate openapi.json for the scanner-removal doc changes ... 362d532d25

`mgmt::tests::openapi_document_is_complete_and_checked_in` compares the served
document against the checked-in snapshot, so the endpoint doc edits in the
scanner removal made it stale and failed `ci / rust`.

Regenerated with `cargo run -p punktfunk-host -- openapi > api/openapi.json`.
The diff is 11 lines, all descriptions — no path, operationId or schema shape
moved. In particular `SourceOrigin` still enumerates ["builtin", "plugin"]:
the variant was kept deliberately so the console, which ships as its own
package and drives an N-1 host that still reports builtin sources, does not
have its generated union narrowed out from under that pairing.

enricobuehler merged commit 3d5d8e2d76 into main 2026-08-13 15:19:44 +00:00

enricobuehler deleted branch worktree-remove-builtin-scanners 2026-08-13 15:19:45 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

FernandoMX enricobuehler (Enrico Bühler) kszlim nannku neoney sassycrown

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unom/punktfunk#199