Files
punktfunk/design/game-library-stores.md
T
enricobuehler 7b99b41ede docs(design): trim shipped plans, consolidate cluster, add index
Much of design/ described work that has since shipped. Trim each doc to
its durable rationale + still-open items (the code is the source of truth
for shipped detail; git history holds the full originals).

- Shipped plans -> status stubs: stats-capture, gamestream-host-plan,
  apple-stage2-presenter, windows-service.
- Trimmed completed-out / open-kept: implementation-plan, hdr-pipeline,
  host-latency, gpu-contention (fixed stale status table), game-library,
  linux-setup (fixed m0->spike + stale zero-copy claim),
  session-aware-host-followups, windows-client-bootstrap,
  windows-dualsense-{scoping,game-detection}, windows-virtual-display,
  security-review (per-finding status table; #12 still open),
  apollo-comparison (shipped backlog collapsed to one-liners).
- Windows-host cluster consolidated: windows-host.md -> redirect into
  windows-host-rewrite.md (whose stale scorecard is corrected -- goal1 is
  merged, M4 done); windows-secure-desktop.md archived (now a fallback
  behind IDD-push primary).
- Kept evergreen: ci.md, gamescope-multiuser.md, windows-build-and-packaging.md.
- New design/README.md: per-doc status table + consolidated open-items
  roll-up so nothing is tracked in only one buried doc.
- Repoint 5 code comments to the archived secure-desktop doc path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:39:06 +00:00

301 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Game library: more game stores
> **Status:** Phases 14 SHIPPED — 6 `LibraryProvider` impls (Steam, Lutris, Heroic, Epic, GOG,
> Xbox) in [`crates/punktfunk-host/src/library.rs`](../crates/punktfunk-host/src/library.rs)
> (1869 lines; commits `5f8c6b6` Lutris+Heroic, `b657452` Epic+GOG, `aed0bf0` Xbox, `5acc12d` shared
> art cache, `7e9023f` GameStream/Windows+non-gamescope launch wiring, `203ad80` web store badges).
> Phases 56 (the remaining 6 providers + the `/library/art` endpoint) are **pending**. This doc is
> trimmed to design rationale + open items; the shipped code is the source of truth.
Goal: extend the unified game library so it enumerates and launches titles from more stores —
on **Windows** Xbox / Game Pass, Epic, EA app (and GOG / Ubisoft / Battle.net / Amazon);
on **Linux** Heroic (Epic+GOG+Amazon), Lutris, and a `.desktop`/Flatpak catch-all.
---
## 1. The extension point
The library lives in [`library.rs`](../crates/punktfunk-host/src/library.rs) and is a plug-in system:
adding a store is a new `LibraryProvider`, not a rewrite.
```rust
pub trait LibraryProvider {
fn store(&self) -> &'static str; // "steam", ...
fn list(&self) -> Vec<GameEntry>; // best-effort: empty (not Err) if the store is absent
}
pub struct GameEntry { id: String /* "<store>:<localid>" */, store, title, art: Artwork, launch: Option<LaunchSpec> }
pub struct Artwork { portrait, hero, logo, header: Option<String> } // URLs the CLIENT fetches
pub struct LaunchSpec{ kind: String, value: String }
```
**The "read the launcher's own on-disk files, no auth" approach is the gold standard we replicate per
store.** Launcher-need-not-be-running unless noted.
> The legacy GameStream `apps.json` ([`gamestream/apps.rs`](../crates/punktfunk-host/src/gamestream/apps.rs))
> is a **separate** Moonlight surface (session recipes: compositor + nested command) and stays as-is.
---
## 2. The two cross-cutting pieces (this is the real work)
Per-store enumeration is mostly easy. Two shared problems gate everything — especially Windows.
### 2a. Launch abstraction + the Windows launch gap
- **Linux** runs the chosen title as a shell command **nested in the per-session gamescope**
(`set_launch_command` / `PUNKTFUNK_GAMESCOPE_APP`). Works today.
- **Windows** captures the whole desktop (DXGI/WGC); there is no nesting, and
`VirtualDisplay::set_launch_command` is a **no-op** ([`vdisplay.rs:57`](../crates/punktfunk-host/src/vdisplay.rs)).
So on Windows **nothing is auto-started** — the user just sees the desktop.
**Plan.** Stop returning a single Linux shell string from `command_for`; introduce an internal enum and
an OS-aware resolver:
```rust
enum LaunchAction { Shell(String), Spawn { exe: PathBuf, args: Vec<String>, workdir: Option<PathBuf> } }
fn resolve_launch(&LaunchSpec) -> Option<LaunchAction> // cfg-aware
fn launch_command(id) -> Option<String> // Linux: thin Shell wrapper (back-compat)
#[cfg(windows)] fn launch_title(id) -> Result<()> // resolve Spawn + run in interactive session
```
**The Windows launcher already exists in the codebase — reuse it.**
[`capture/windows/wgc_relay.rs:196-204`](../crates/punktfunk-host/src/capture/windows/wgc_relay.rs)
does exactly the needed sequence:
`WTSGetActiveConsoleSessionId → WTSQueryUserToken → DuplicateTokenEx(TokenPrimary) →
CreateEnvironmentBlock → CreateProcessAsUserW(lpDesktop="winsta0\\default")`.
- Factor that into `windows/interactive.rs::spawn_in_active_session(exe, args, workdir) -> u32`.
- **Critical:** use the **logged-in user token** (`WTSQueryUserToken`, as `wgc_relay` does) — **not**
`windows/service.rs:449-510`'s variant, which duplicates the **SYSTEM** token and only retargets its
session id. UWP/appx activation, the user-hive protocol handlers (`HKCU\Software\Classes`), and each
launcher's auth/entitlement context all require the *real user's* token. The host process stays SYSTEM.
- For URI-handoff kinds (Epic/Steam/EA/Amazon/GOG-Galaxy) build a **concrete EXE + the URI as a separate
argv element**. `CreateProcessAsUserW` does **no** shell/protocol resolution — never `cmd /c`, never a
bare URI. For schemes with no exe-argv form (`amazon-games://`, `origin2://`), add an impersonate-token
`ShellExecuteEx` fallback (`ImpersonateLoggedOnUser` on a worker thread + `CoInitialize`).
- **Order:** launch the title **after** the interactive capture pipeline is live, so the game renders onto
the already-captured desktop and grabs foreground.
- **Caveats:** `WTSQueryUserToken` fails when no interactive user is logged on (a pre-login box can stream
the login/secure desktop but can't auto-launch a title); on the lock/secure desktop a launch may queue
until unlock. **Needs on-glass validation** (RTX box) that each launcher EXE accepts its URI on argv and
that post-capture launch grabs foreground.
### 2b. Artwork: a layered, no-auth-first `ArtResolver`
Steam gets free CDN art keyed by appid. Most stores don't. Layered ladder, degrade to a title-only card:
1. **Steam** → public Steam CDN by appid (unchanged, client fetches directly).
2. **Stores that already hold public CDN URLs** → emit verbatim, **no host endpoint**: Heroic
`store_cache` `art_*` (Epic/GOG/Amazon CDN), itch `cover_url`, GOG via public `api.gog.com/products/<id>?expand=images`
(one cached lookup), Epic via local `catcache.bin` keyImages.
3. **Xbox** → one **unofficial** no-auth `displaycatalog.mp.microsoft.com` lookup by StoreId, cached,
degrade to no-art offline. (Not a stable contract — tolerate drift.)
4. **Genuinely-local art** (Lutris `coverart`/`banners` JPEGs, Flatpak/.desktop icons, Bottles) → a
**new host-served endpoint is required**, because `Artwork` carries URLs the client fetches and a file
on the host has no public URL.
5. **Opt-in SteamGridDB** enrichment (v2 API `https://www.steamgriddb.com/api/v2`, `Authorization: Bearer
<operator key>`, **off by default**) to fill gaps. Not no-auth; never blocks listing.
6. **None** → existing title-only card.
**New endpoint (still pending):** `GET /library/art/<entryId>/<slot>` (slot ∈ `portrait|hero|logo|header`)
on `mgmt.rs`. It resolves `entryId` in the host library to a **known on-disk absolute path** (never
interpolates raw client input into a filesystem path), sanitizes the slot, rejects `..`, streams the bytes
with the right content-type. Reserve `data:` URLs for tiny logos only (don't bloat the catalog JSON that
crosses the control plane). See open question on whether this GET bypasses the mgmt bearer (images are
non-sensitive and the streaming client connects over punktfunk/1, not the bearer-gated REST).
---
## 3. Security model (preserved and extended)
The invariant is unchanged: **the client sends only a store-qualified `GameEntry.id`** (e.g. `lutris:42`,
`xbox:9NBLGGH4R315`, `epic:fn:4fe…:Fortnite`) in `Hello.launch`. The host looks it up in its **own**
enumerated library, reads the **host-derived** `LaunchSpec`, and resolves it. The client never sends a
`LaunchSpec`, command, URI, or path.
Per-kind charset validators are belt-and-suspenders before any interpolation (values are already
host-derived from local files the host owns):
| kind | guard |
|---|---|
| `steam_appid`, `lutris_id`, `uplay` | digits only |
| `battlenet` | `^[A-Za-z0-9]+$` (case-sensitive) |
| `amazon` | `^[A-Za-z0-9-]+$` |
| `aumid` | `^[A-Za-z0-9._-]+![A-Za-z0-9._-]+$` (the `!` separator) |
| `epic` | ≤3 `:`-split parts, each `^[A-Za-z0-9._-]+$`, then URL-encode colons |
| `heroic` | runner ∈ {legendary,gog,nile} + appName `^[A-Za-z0-9._-]+$` |
| `ea_offer_ids` | `^[A-Za-z0-9._,-]+$` (allow comma) |
On **Windows never route a client-influenced string through `cmd /c start`.** `resolve_launch` yields
`Spawn{exe,args,workdir}`; `CreateProcessAsUserW` launches a concrete EXE with the URI/flags as separate
argv elements. The operator-only `command` kind (custom store + provider-generated Linux shell lines for
`desktop`/`itch`) is host-derived/operator-typed, never client-set.
The one net-new surface is `GET /library/art` — covered in §2b (id-resolved path, no traversal).
---
## 4. `LaunchSpec` kinds
| kind | value holds | maps to |
|---|---|---|
| `lutris_id` | `pga.db` `games.id` (digits) | Linux Shell `lutris lutris:rungameid/<id>` (nests in gamescope) |
| `heroic` | `<runner>:<appName>` | Linux argv `heroic --no-gui "heroic://launch?appName=<app>&runner=<runner>"` |
| `aumid` | `<PFN>!<AppId>` | Windows Spawn `explorer.exe "shell:AppsFolder\<aumid>"` (interactive session) |
| `epic` | `<namespace>:<catalogItemId>:<appName>` | Windows Spawn `EpicGamesLauncher.exe` + `com.epicgames.launcher://apps/<ns>%3A<cat>%3A<app>?action=launch&silent=true` |
| `gog` | host-resolved `exe \t args \t workdir` | Windows Spawn `CreateProcessAsUserW(exe,args,workdir)` (direct exe, no Galaxy) |
| `uplay` | Ubisoft gameId (digits) | Windows `uplay://launch/<gameId>/0` |
| `battlenet` | product code (e.g. `WTCG`, `Fen`, `OSI`) | Windows Spawn `Battle.net.exe --exec="launch <code>"` |
| `amazon` | Amazon Games `DbSet.Id` | Windows `amazon-games://play/<Id>` (impersonate ShellExecute) |
| `ea_offer_ids` | comma-joined contentID list | Windows `origin2://game/launch/?offerIds=<list>&autoDownload=1` |
| `command` (existing) | host-derived shell line | Linux gamescope-nested (desktop/flatpak/itch reuse this) |
---
## 5. Per-store provider catalog
All enumeration is no-auth, local. Confidence is **after** adversarial web-verification.
### Shipped (phases 14)
| store | OS | enumerate | launch kind | art |
|---|---|---|---|---|
| **Steam** | both | local `.acf`/`.vdf` | `steam_appid` | Steam CDN (client-direct) |
| **Lutris** | Linux | read-only `pga.db` (`installed=1`) | `lutris_id` | local JPEGs (needs `/library/art`, still pending) |
| **Heroic** | Linux | `store_cache/{legendary,gog,nile}_library.json` | `heroic` | free public CDN (`art_*`) |
| **Epic** | Windows | `…\Manifests\*.item` | `epic` | local `catcache.bin` keyImages |
| **GOG** | Windows | registry + `goggame-<id>.info` | `gog` (direct-exe) | `api.gog.com/products/<id>?expand=images` |
| **Xbox / Game Pass** | Windows | `XboxGames\*\Content\MicrosoftGame.config` + AppRepository PFN | `aumid` | unofficial `displaycatalog` lookup |
The hard-won corrections folded into these (keep when revisiting): Epic uses Playnite's **exclusion**
filter (skip `UE_`, DLC `addons` w/o `addons/launchable`), builds the namespace:catalog:app **triple** when
ids exist else **falls back to the bare `appName` URI** (don't set launch=None); GOG launches the
**direct exe** (dodges Galaxy cold-start/anti-cheat); Xbox **reads** the PackageFamilyName from the
`AppRepository\Packages\<PackageFullName>` dir name (**never** hash the publisher), scans `XboxGames` rather
than parse the undocumented `.GamingRoot`, and UWP `aumid` activation is load-bearing on the interactive
user token; Heroic `gui=false` is inert (`--no-gui` does it) and single-instance Electron forwards-and-exits
(launch was gated). Misses pure-UWP (non-GDK) Store games under ACL-locked `WindowsApps` — accepted for v1.
### Remaining providers (phases 56)
#### Desktop (`.desktop` + Flatpak) — Linux, P1, effort M, confidence medium (universal catch-all)
- **Enumerate:** scan `{/var/lib/flatpak/exports/share/applications,
~/.local/share/flatpak/.../applications, /usr/share/applications, /usr/local/share/applications,
~/.local/share/applications}/*.desktop`. Require `Type=Application` + `Categories` contains `Game`; skip
`NoDisplay`/`Hidden`/`Terminal=true` and known launcher app-ids (Steam/Heroic/Lutris/Bottles/RetroArch)
to avoid recursion/dupes.
- **Launch:** reuse `command` (host-derived shell line, nested in gamescope): cleaned `Exec` (strip
`%U/%F/%f/%u/%i/%c/%k`) else `flatpak run <app-id>`.
- **Artwork:** local — resolve `Icon=` via the hicolor theme / flatpak exported icons → `/library/art`.
App icons are low-res, not box art (acceptable header fallback).
- **Notes:** run **last** and dedup by install path / drop ids already surfaced by Steam/Heroic/Lutris.
#### itch.io — Linux + Windows, P3, effort S, confidence medium
- **Enumerate:** read-only `rusqlite` of `butler.db` (`~/.config/itch/db/butler.db`; Flatpak
`io.itch.itch`; Windows `%AppData%\itch\db`, per-user). JOIN `caves`→`games`. **Key on `cave.ID`** (a
game can have multiple caves; install location + verdict are per-cave). Read game title / `cover_url`;
resolve install dir from `InstallLocationID`+`InstallFolderName`||`CustomInstallFolder` + the Verdict
candidate. Confirm exact column names on-box.
- **Launch:** `command` → direct binary `basePath`+`candidate.path`, **only** for Verdict candidates with
`flavor==native` (html/jar/love need itch's runtime — fall back to custom).
- **Artwork:** **free** — `games.cover_url` is a public itch CDN URL.
#### Ubisoft Connect — Windows, P2, effort S, confidence medium
- **Enumerate:** registry `HKLM\SOFTWARE\WOW6432Node\Ubisoft\Launcher\Installs\<gameId>` (both reg views),
read `InstallDir`; title = install-dir leaf folder (primary) else the `Uplay Install <gameId>` Uninstall
`DisplayName`.
- **Launch:** `uplay` → `uplay://launch/<gameId>/0`. **Artwork:** none → title-only.
- **Notes:** smallest effort once the Windows URI-launch wiring exists; hive+scheme unchanged across the
Origin→EA migration.
#### Amazon Games — Windows, P2, effort S, confidence medium
- **Enumerate:** read-only `rusqlite` of
`%LocalAppData%\Amazon Games\Data\Games\Sql\GameInstallInfo.sqlite`:
`SELECT Id,ProductTitle,InstallDirectory FROM DbSet WHERE Installed=1`. **Per-user path** — the SYSTEM
service must resolve the **active session user's** profile (not the SYSTEM profile).
- **Launch:** `amazon` → `amazon-games://play/<Id>` (impersonate-token ShellExecute; no clean exe-argv form).
- **Artwork:** `ProductIconUrl`/`ProductLogoUrl` columns when present, else none.
#### Battle.net — Windows, P2, effort **L**, confidence medium (high catalog value: WoW/Diablo IV/Overwatch 2/CoD)
- **Enumerate:** hand-roll a ~4-field protobuf decode of `C:\ProgramData\Battle.net\Agent\product.db`
(`product_install{ uid, product_code, settings.install_path, cached_product_state.base_product_state.installed }`).
Registry fallback: Uninstall keys whose `UninstallString` matches `Battle.net.exe --uid=<uid>`.
`product.db` has **no titles** → maintain a ~30-entry `product_code`→name map (source from
bnetlauncher/Lutris/Heroic; codes are **case-sensitive**).
- **Launch:** `battlenet` → `Battle.net.exe --exec="launch <code>"` (more reliable than the
`battlenet://<code>` URI, which only hands off). **Artwork:** none → title-only.
- **Notes:** the protobuf + name map + no-art make it L; pin the `.proto` and decode defensively.
#### EA app — Windows, P2, effort M, confidence medium (most closed/fragile — ship last)
- **Enumerate:** registry `HKLM\SOFTWARE\WOW6432Node\{EA Games,Origin Games}\<id>` (Install Dir /
DisplayName), parse `<dir>\__Installer\installerdata.xml` for the **full** `<contentIDs>` list +
`<gameTitle locale='en_US'>`. Registry under-reports for EA-app (vs legacy Origin) installs — known
completeness gap. Keep the AES-256 encrypted `IS`-file decrypt **out** of the default path (optional
feature flag for completeness).
- **Launch:** `ea_offer_ids` → `origin2://game/launch/?offerIds=<full,comma,list>&autoDownload=1`. **Emit
the full contentID list** — a single offerId generally no longer launches under the EA app.
- **Artwork:** none no-auth → title-only.
#### Rockstar — P3, fold into custom
- Registry `HKLM\SOFTWARE\WOW6432Node\Rockstar Games\<Title>\InstallFolder`; direct-exe Spawn; no art.
Tiny catalog, most titles now bought on Steam/Epic.
---
## 6. Structure & phasing
**Structure (still pending refactor).** Split the 1869-line `library.rs` → a `library/` dir before it
balloons further: `mod.rs` (trait, wire types, `LaunchAction`, custom CRUD, `all_games`, `resolve_launch`,
`launch_command`/`launch_title`), `steam.rs`, one file per provider, `art.rs` (ArtResolver +
displaycatalog/gog-api/steamgriddb helpers), `win_util.rs` (HKLM subkey enumerator, read-only SQLite
opener, tiny read-only XML reader). Deps in play: `rusqlite` (bundled, read-only) for lutris/itch/amazon
DBs; `roxmltree`/`quick-xml` for the Windows manifests; registry via the `windows` crate's
`Win32_System_Registry` feature (no new crate). Avoid `prost` — hand-roll the ~4 Battle.net fields.
- **Phases 14 — DONE:** launch abstraction + Windows interactive-session spawn; Steam/Lutris/Heroic
providers + Linux art; Epic/GOG providers; Xbox / Game Pass provider; shared art warmer + cache; web
store badge generalized per `game.store`.
- **Phase 5 — future:** Linux Desktop catch-all (last + dedup, icons via `/library/art`), Ubisoft
(`UplayProvider`), Amazon (`AmazonProvider` + per-user-profile-under-SYSTEM helper); land the
`GET /library/art/<id>/<slot>` endpoint that Lutris/Desktop local art still needs.
- **Phase 6 — future:** Battle.net (hand-rolled protobuf + code→name map), EA app, itch.io; Rockstar/
Bottles → custom; optional SteamGridDB v2 enrichment behind an operator key.
---
## 7. Open questions
- **Art delivery auth:** the streaming client connects over punktfunk/1 (QUIC), not the bearer-gated mgmt
REST, yet already fetches Steam CDN URLs over plain HTTP. Should `GET /library/art/*` be an
unauthenticated read-only image GET on the mgmt listener (bearer bypass for that path only), a separate
tiny image server, or should local-art bytes ride the punktfunk/1 control plane?
- **Windows launch ordering** needs on-glass RTX-box validation: confirm launching *after* capture is live
grabs foreground+capture, and that `CreateProcessAsUserW(EpicGamesLauncher.exe/steam.exe, URI-as-argv)`
actually starts the game per launcher (vs needing the impersonate-ShellExecute fallback).
- **Per-user-profile resolution under SYSTEM** for Amazon (`%LocalAppData%`) and itch (`%AppData%`): add
`WTSQueryUserToken` + `GetUserProfileDirectoryW` (or read `USERPROFILE` from `CreateEnvironmentBlock`)?
- **`rusqlite` bundled SQLite** — acceptable for deb/rpm/flatpak and no link conflict? Otherwise fall back
to `lutris -l -j` (fragile: single-instance D-Bus forwarding).
- **Battle.net** product-code→name map source/maintenance, and `product.db` `.proto` drift across Agent versions.
- **Unofficial art sources** (Xbox displaycatalog): best-effort with aggressive caching + no-art degrade,
or Xbox-art local-tile-only for v1?
- **Heroic launch:** ship enumeration+art only at first, or invest in direct legendary/gogdl/nile CLI
launch (needs the user's on-disk auth tokens) to dodge the single-instance-Electron / gamescope-escape problem?
- **`config_dir()` consistency:** `library.rs` uses an XDG/HOME-based dir; confirm the Windows SYSTEM host
lands its art cache + custom store under `%ProgramData%\punktfunk` (there's a separate
`gamestream::config_dir()` that already does this).
- Should provider-generated Linux shell lines (`desktop`/`itch`) reuse the `command` kind (documented
"operator-only") or get a distinct internal kind to keep the mgmt-UI `command` semantics clean?
---
## Open items (what's left)
- **6 remaining providers:** Desktop/Flatpak (Linux), itch.io (Linux+Windows), Ubisoft Connect, Amazon
Games, Battle.net, EA app (recipes in §5).
- **`GET /library/art/<entryId>/<slot>` mgmt endpoint** — still missing; Lutris local JPEGs (and the future
Desktop icons) have no public URL without it.
- **Refactor `library.rs` (1869 lines) into a `library/` directory** (structure in §6).
- The **8 open questions** in §7.
- **Optional SteamGridDB v2 enrichment** behind an operator key (off by default; never blocks listing).