{
"summary": "Extract the exact GameStream/Moonlight P1 host protocol from Sunshine + moonlight-common-c",
"agentCount": 6,
"logs": [
"[research:control-input] failed: API Error: The socket connection was closed unexpectedly. For more information, pass `verbose: true` in the second argument to fetch()"
],
"result": [
{
"area": "GameStream HTTP serverinfo + pairing handshake (host side, what stock Moonlight expects)",
"summary": "A GameStream host runs two HTTP servers from the same NvHTTP code: plain HTTP on port 47989 (insecure, unauthenticated) and HTTPS with mutual TLS on 47984. Ports are derived from a base port (config default 47989) plus a signed offset: PORT_HTTP=0 -> 47989, PORT_HTTPS=-5 -> 47984. Moonlight first GETs /serverinfo over HTTP (before pairing) to read an XML document of capabilities and the host's pairing/running state; key fields it parses are hostname, appversion (its major version selects the pairing hash: >=7 -> SHA-256, else SHA-1; Sunshine advertises \"7.1.431.-1\"), GfeVersion, uniqueid, HttpsPort, ExternalPort, mac, MaxLumaPixelsHEVC, ServerCodecModeSupport (a bitmask: 3=H264-only, 259=+HEVC, 3843=+AV1), PairStatus, currentgame and state. Pairing is a 4-phase challenge/response over /pair driven entirely by repeated HTTP GETs with a `phrase` query param: getservercert, clientchallenge, serverchallengeresp, clientpairingsecret, followed by a final pairchallenge over HTTPS. The shared secret is an AES-128 key = SHA(salt(16) || PIN-as-utf8) truncated to 16 bytes; salt is client-generated random 16 bytes sent hex-encoded in phase 1. All pairing AES uses AES-128 in ECB mode with NO padding (inputs zero-extended to a 16-byte multiple, so a 32-byte SHA-256 hash is exactly two blocks). Each side proves it knows the PIN by exchanging encrypted random challenges and verifying SHA hashes that bind both X.509 cert signatures and a per-side 16-byte secret; the secrets are additionally RSA-SHA256-signed by each side's cert key and verified. On success the host stores the client's self-signed X.509 cert in an allow-list; thereafter every HTTPS request requires that exact client cert (cert pinning via a custom OpenSSL verify callback comparing the presented cert's PEM against stored authorized clients). All XML responses are an `...` tree; pairing replies carry `paired` (1/0) plus the phase-specific element (plaincert, challengeresponse, pairingsecret).",
"ports": [
"47989/tcp = HTTP (insecure NvHTTP), offset PORT_HTTP=0 from base port",
"47984/tcp = HTTPS (mutual-TLS NvHTTP), offset PORT_HTTPS=-5 from base port",
"Base/config port default = 47989; map_port(p) = config.port + p (uint16, warns if <1024 or >65535)",
"Related stream ports (offsets, not part of this area but advertised/used post-launch): 48010/tcp RTSP (+21), 47998/udp video (+9), 47999/udp control (+10), 48000/udp audio (+11), 48002/udp mic (+13)",
"Moonlight resolves stream ports via LiGetPortFromPortFlagIndex; TCP set {47984,47989,48010}, UDP set {47998,47999,48000,48010}"
],
"wire_formats": [
{
"name": "/serverinfo XML response",
"layout": "\n ...\n 7.1.431.-1 (VERSION; major>=7 => client uses SHA-256)\n 3.23.0.74 (GFE_VERSION)\n ... (host unique id)\n 47984 (net::map_port(PORT_HTTPS))\n 47989 (net::map_port(PORT_HTTP))\n 1869449984 (or \"0\" if HEVC disabled)\n aa:bb:cc:dd:ee:ff (real MAC on HTTPS; \"00:00:00:00:00:00\" on plain HTTP)\n ...\n 3843 (bitmask; 3=H264 only,259=+HEVC,3843=+AV1)\n ... (conditional)\n 1 (1 if request carried a known uniqueid over HTTPS, else 0)\n 0 (0 if idle, else running app id)\n SUNSHINE_SERVER_FREE (or SUNSHINE_SERVER_BUSY; GFE uses MJOLNIR_IDLE/_SERVER_BUSY)\n",
"notes": "Served on BOTH HTTP(47989) and HTTPS(47984). Plain-HTTP version hides mac and forces PairStatus=0. Element NAMES are case-sensitive and Moonlight requires hostname, appversion, PairStatus, currentgame, state, and at least one port. appversion MAJOR number is the SHA-1-vs-SHA-256 switch."
},
{
"name": "/pair phase 1 request+response (getservercert)",
"layout": "GET /pair?uniqueid=&uuid=&devicename=&updateState=1&phrase=getservercert&salt=<32 hex chars = 16 bytes>&clientcert=\nResponse: 1hex(server X.509 PEM)",
"notes": "salt is client-random 16 bytes, sent as 32 hex chars. Server validates salt length >=32 hex chars, takes first 16 bytes, computes AES key = SHA256(salt||PIN)[..16]. clientcert is the client's self-signed cert (kept for later signature checks and TLS pinning). If PIN not yet entered, server may stall here until the user enters it."
},
{
"name": "/pair phase 2 request+response (clientchallenge)",
"layout": "GET /pair?uniqueid=&clientchallenge=\nResponse: 1hex(ECB-encrypt(hash[H] || serverChallenge[16]))",
"notes": "Client sends AES-ECB(16-byte random). Server decrypts -> clientChallenge; computes hash = SHA( clientChallenge || serverCertSignature || serversecret[16 random] ); generates serverChallenge[16 random]; returns ECB(hash || serverChallenge). H = 32 (SHA-256) or 20 (SHA-1). With no padding, the ECB input is hash(32)+serverChallenge(16)=48 bytes = 3 blocks for SHA-256."
},
{
"name": "/pair phase 3 request+response (serverchallengeresp)",
"layout": "GET /pair?uniqueid=&serverchallengeresp=\nResponse: hex( serversecret[16] || RSA-SHA256-sign(serversecret) )1",
"notes": "Client decrypts phase-2 challengeresponse into [serverResponseHash(H) || serverChallenge(16)], generates clientSecret[16 random], computes challengeRespHash = SHA( serverChallenge || clientCert.signature || clientSecret ), sends ECB(challengeRespHash). Server stores decrypted value as clienthash for the phase-4 check, then returns its serversecret plus that secret signed by the server cert's private key (sign256)."
},
{
"name": "/pair phase 4 request+response (clientpairingsecret)",
"layout": "GET /pair?uniqueid=&clientpairingsecret=\nResponse: 1 or 0",
"notes": "Server splits into secret(16)+signature(rest). Builds data = serverChallenge || clientCert.signature || clientSecret, hashes it, compares to the clienthash stored in phase 3 (proves client knew PIN). Also verify256(clientCert, clientSecret, signature) (proves client owns its cert key). Both must pass; on success the client cert is added to the authorized allow-list. paired=0 means PIN/cert mismatch."
},
{
"name": "/pair final pairchallenge (HTTPS)",
"layout": "GET /pair?uniqueid=&phrase=pairchallenge (over HTTPS:47984, presenting the now-trusted client cert)\nResponse: 1",
"notes": "Moonlight calls executePairingChallenge() after phase 4; it must succeed over the mutual-TLS connection using the freshly-paired client cert, confirming the cert pinning round-trips. If this fails Moonlight calls /unpair and reports FAILED."
},
{
"name": "Other HTTPS endpoints (params only, post-pair)",
"layout": "/applist -> XML list of ; /appasset?appid=&assetidx=&assettype= -> PNG; /launch?uniqueid=&appid=&mode=WxHxFPS&additionalStates=&sops=&rikey=&rikeyid=&localAudioPlayMode=&surroundAudioInfo=&hdrMode=&corever=; /resume?rikey=&rikeyid=&surroundAudioInfo=; /cancel (no params)",
"notes": "Out of this area's depth but listed for completeness. rikey/rikeyid carry the AES-128 RTSP/stream key (16-byte key as hex, plus a 4-byte key id) used to seal the control/RTSP plane; mode is 'WIDTHxHEIGHTxFPS'. /launch and /resume require an already-paired (pinned) HTTPS connection."
}
],
"flow": [
"0. Moonlight GET http://host:47989/serverinfo -> parses appversion major (>=7 => SHA-256 hash, else SHA-1), HttpsPort, PairStatus, state.",
"1. Client generates salt=random16 and a self-signed RSA-2048 X.509 cert. GET /pair?...&phrase=getservercert&salt=hex(salt)&clientcert=hex(certPEM). User enters PIN on host. Both sides compute aesKey = SHA(salt || pinUTF8)[0..16].",
"2. Server replies plaincert=hex(serverCertPEM), paired=1. Client stores server cert.",
"3. Client: randomChallenge=random16; GET ...&clientchallenge=hex(AES_ECB_enc(randomChallenge, aesKey)).",
"4. Server: decrypt -> clientChallenge; serversecret=random16; respHash=SHA(clientChallenge || serverCert.signature || serversecret); serverChallenge=random16; reply challengeresponse=hex(AES_ECB_enc(respHash || serverChallenge)).",
"5. Client: decrypt challengeresponse -> [serverRespHash(H) || serverChallenge(16)]; clientSecret=random16; challengeRespHash=SHA(serverChallenge || clientCert.signature || clientSecret); GET ...&serverchallengeresp=hex(AES_ECB_enc(challengeRespHash)).",
"6. Server: decrypt -> store as clienthash; reply pairingsecret=hex(serversecret || RSA_SHA256_sign(serversecret, serverPrivKey)), paired=1.",
"7. Client: split pairingsecret into [serversecret(16) || serverSig]; verify expected = SHA(randomChallenge || serverCert.signature || serversecret) (sanity) and RSA-verify serverSig over serversecret with server cert. GET ...&clientpairingsecret=hex(clientSecret || RSA_SHA256_sign(clientSecret, clientPrivKey)).",
"8. Server: split into [clientSecret(16) || clientSig]; recompute SHA(serverChallenge || clientCert.signature || clientSecret) and compare to stored clienthash; RSA-verify clientSig over clientSecret with clientCert. Both pass => add clientCert to authorized list; reply paired=1 (else paired=0).",
"9. Client: GET https://host:47984/pair?...&phrase=pairchallenge over mutual-TLS presenting its now-trusted cert; expects paired=1. Pairing complete -> PairState.PAIRED.",
"10. On any mismatch the client GET /unpair and returns FAILED / PIN_WRONG."
],
"crypto": "PIN-derived key: aesKey = HASH(salt[16] || PIN_utf8)[0..16], where HASH = SHA-256 if server appversion major >=7 (Sunshine: 7.1.431.-1) else SHA-1. Salt = client random 16 bytes. PIN is the 4-digit code shown/entered by user; concatenation is salt FIRST then pin. Pairing cipher: AES-128 in ECB mode, NO padding / padding DISABLED (Sunshine ecb_t(key,false); Moonlight AESLightEngine block loop). Inputs are zero-extended to a 16-byte multiple before encryption (a 32-byte SHA-256 hash = 2 blocks; respHash(32)+serverChallenge(16)=48=3 blocks). Per-side proofs: serverHash = SHA(clientChallenge || serverCert.signature || serversecret16); clientHash = SHA(serverChallenge || clientCert.signature || clientSecret16) (cert.signature = the DER signature bytes of the self-signed X.509). Identity binding: each side RSA-signs its own 16-byte secret with its cert's private key using RSA-PKCS1 over SHA-256 (sign256/verify256), other side verifies. Certs: self-signed RSA-2048, SHA-256 signed, ~20-year validity. Result of pairing is NOT a streaming key — it only establishes mutual TLS trust (pinned certs). The actual AES-128 STREAM key is delivered separately at /launch as rikey (16-byte hex) + rikeyid; that is where punktfunk-core's existing AES-128-GCM session crypto plugs in. IMPORTANT: pairing AES-ECB-no-pad is distinct from and unrelated to punktfunk-core's AES-128-GCM session sealing.",
"rust_options": "HTTP/HTTPS control plane belongs in crates/punktfunk-host/src/web.rs (the existing stub explicitly permits tokio/axum here, off the hot path). Use axum or hyper for the two servers. TLS with mutual auth + custom cert pinning: use rustls via axum-server/tokio-rustls with a custom ClientCertVerifier (rustls::server::danger::ClientCertVerifier) that accepts any well-formed cert at handshake time and then matches the presented leaf DER/PEM against the paired allow-list (mirror Sunshine's verify callback), OR use openssl/openssl crate to match Sunshine 1:1. XML: build/parse with quick-xml or xml-rs (or just format! the small fixed templates and a tiny extractor). Crypto: aes crate (already a dep transitively) in ECB mode via the `ecb` crate with NoPadding (Aes128 + ecb::Decryptor/Encryptor, manual block handling) — note RustCrypto deprecates ECB so call the block cipher directly (aes::Aes128 + cipher::BlockEncrypt/BlockDecrypt over 16-byte chunks). Hashing: sha2 (SHA-256) and sha1 crates. X.509 self-signed cert generation + RSA-SHA256 sign/verify: rcgen for cert gen, rsa + sha2 for PKCS1v15 sign/verify, x509-parser or x509-cert to extract the cert's signature bytes (cert.getSignature() equivalent) and to do TLS-trust comparison. Persist authorized client certs (PEM) in a small JSON/sled store. Run all of this on tokio in web.rs; keep it fully separate from the native-thread per-frame pipeline.",
"reuse_from_punktfunk": "REUSE little of punktfunk-core's crypto here — its crypto.rs is AES-128-GCM session sealing (nonce = salt||seq, seq as AAD) for the VIDEO/INPUT plane, which corresponds to the post-/launch rikey stream key, NOT the pairing handshake. Pairing needs AES-128-ECB-no-pad + SHA-256/SHA-1 + RSA, none of which exist in punktfunk yet and must be newly built (best placed in punktfunk-host, not punktfunk-core, since it is control-plane only). The natural seam is the existing web.rs stub (WebConfig::run) whose TODO already says 'GameStream serverinfo, pairing handshake, RTSP SETUP' — implement the two HTTP servers and the 4-phase /pair state machine there. punktfunk-core's AES-128-GCM SessionCrypto IS reusable downstream: once paired and /launch hands over rikey (16-byte AES key) + rikeyid, feed that key into punktfunk-core's Session/SessionCrypto for the encrypted video/control planes. The internal 40-byte packet format is unrelated to this HTTP/pairing area. So: new pairing crypto + axum servers in punktfunk-host/web.rs (control), reuse punktfunk-core GCM for the data plane post-launch.",
"gotchas": [
"appversion MAJOR number is load-bearing: it silently switches the client's pairing hash. Advertise major >=7 (e.g. \"7.1.431.-1\") to get SHA-256; advertise <7 and the client uses SHA-1 with 20-byte hashes (changes all the ECB block counts). Mismatch => silent pairing failure.",
"AES-ECB has NO padding and NO IV. Do not use a library that auto-pads (PKCS7) — Sunshine passes ecb_t(key,false). Inputs are zero-extended to 16-byte multiples; a 32-byte SHA-256 hash is exactly 2 blocks but respHash(32)+serverChallenge(16)=48 must be encrypted as one 48-byte buffer.",
"The hash inputs use cert.getSignature() = the X.509 DER SIGNATURE bytes (the signatureValue), NOT the cert body, NOT a hash of the cert. Getting this field wrong is the most common pairing bug.",
"Salt+PIN order is salt FIRST then PIN (UTF-8 ascii digits). PIN is the literal 4-char string, not parsed as an integer.",
"Pairing does NOT yield the stream key. The streaming AES key arrives later as /launch?rikey=&rikeyid=. Don't conflate the PIN-derived ECB key with the GCM stream key.",
"Two distinct servers: /serverinfo and /pair answer on BOTH 47989 (HTTP) and 47984 (HTTPS); /applist,/appasset,/launch,/resume,/cancel are HTTPS-ONLY and require the pinned client cert. The plain-HTTP /serverinfo must zero out mac and force PairStatus=0.",
"TLS pinning is custom: standard CA validation is bypassed; the verify callback accepts the handshake then checks the presented leaf cert's PEM against the stored authorized-client list. Client certs are self-signed, so a normal rustls verifier would reject them — you must supply a permissive ClientCertVerifier + post-hoc allow-list match.",
"All responses wrap in ... ; the status_code attribute (and HTTP 200) both matter. paired=1/0 appears in every pairing reply and the client checks it as a string \"1\".",
"RSA signatures are PKCS#1 v1.5 over SHA-256 (sign256/verify256). Use the cert's RSA key; ECDSA certs would change this.",
"ServerCodecModeSupport is a bitmask, advertise the decimal: 3 (H264 only), 259 (H264+HEVC), 3843 (H264+HEVC+AV1); flags SCM_H264=0x1, SCM_HEVC=0x100, SCM_HEVC_MAIN10=0x200, SCM_AV1_MAIN8=0x10000, SCM_AV1_MAIN10=0x20000.",
"If the user hasn't entered the PIN yet, the host stalls the phase-1 response until it's entered (or until clientchallenge); design the state machine to await PIN input keyed by uniqueid.",
"Each pairing phase validates the previous phase happened for that uniqueid; the host keeps per-client pairing session state (last_phase, cipher_key, serversecret, serverChallenge, clienthash) across the 4 separate HTTP GETs."
],
"sources": [
"Sunshine src/nvhttp.cpp: serverinfo() (root.hostname/appversion/GfeVersion/uniqueid/HttpsPort/ExternalPort/MaxLumaPixelsHEVC/mac/LocalIP/ServerCodecModeSupport/PairStatus/currentgame/state; codec_mode_flags ORing SCM_*; MaxLumaPixelsHEVC=\"1869449984\"; state SUNSHINE_SERVER_FREE/BUSY) and pair() 4-phase state machine (getservercert/clientchallenge/serverchallengeresp/clientpairingsecret; ecb_t(*cipher_key,false); crypto::hash; sign256; verify256; add_authorized_client) and the https_server.verify pinning callback",
"Sunshine src/nvhttp.h: VERSION=\"7.1.431.-1\", GFE_VERSION=\"3.23.0.74\", PORT_HTTP=0, PORT_HTTPS=-5",
"Sunshine src/crypto.cpp: gen_aes_key (salt||pin -> SHA-256 -> first 16 bytes), hash() = EVP_sha256, sign256/verify256 = EVP_sha256, ecb_t/gcm_t/cbc_t (EVP_aes_128_ecb/gcm/cbc, EVP_CIPHER_CTX_set_padding), AES-128 throughout",
"Sunshine src/network.cpp: map_port(p) = config::sunshine.port + p",
"moonlight-android PairingManager.java: serverMajorVersion>=7 -> Sha256PairingHash else Sha1PairingHash; saltPin (salt then pin utf-8); generateAesKey = copyOf(hash,16); encryptAes/decryptAes AESLightEngine ECB; performBlockCipher 16-byte block loop with zero-pad blockRoundedSize; phase byte orders (clientchallenge=enc(random16); challengeRespHash=hash(serverChallenge||cert.signature||clientSecret); clientPairingSecret=clientSecret||signData(clientSecret)); Sha256PairingHash.getHashLength=32 / Sha1=20; signData=SHA256withRSA; verifySignature; PairState{NOT_PAIRED,PAIRED,PIN_WRONG,FAILED,ALREADY_IN_PROGRESS}; executePairingChallenge final pairchallenge",
"moonlight-common-c src/Limelight.h: SCM_H264=0x1, SCM_HEVC=0x100, SCM_HEVC_MAIN10=0x200, SCM_AV1_MAIN8=0x10000, SCM_AV1_MAIN10=0x20000, SCM_*_444 flags; VIDEO_FORMAT_* constants; default ports 47984/47989/48010 tcp, 47998/47999/48000/48010 udp",
"DeepWiki LizardByte/Sunshine NVHTTP page: ServerCodecModeSupport decimal values 3 / 259 / 3843",
"punktfunk repo (local): crates/punktfunk-host/src/web.rs (WebConfig stub, control-plane seam) and crates/punktfunk-core/src/crypto.rs (AES-128-GCM SessionCrypto = data-plane, not pairing)"
]
},
{
"area": "RTSP handshake + SDP + stream config negotiation (GameStream / Sunshine ↔ Moonlight)",
"summary": "GameStream negotiation is two phases. Phase 1 is an HTTPS GET to /launch (or /resume) on port 47984/47989 where the client passes the session parameters as URL query args — most importantly rikey (a 32-hex-char = 16-byte AES-128 key) and rikeyid (a signed 32-bit int). The host derives the per-stream AES-GCM key directly from rikey and a 16-byte IV from rikeyid as a big-endian uint32 left-padded into a 16-byte buffer (Sunshine make_launch_session). Phase 2 is a GameStream-flavored RTSP/1.0 exchange over TCP on port 48010 (RTSP_SETUP_PORT = base 47989 + 21). The sequence is OPTIONS → DESCRIBE → SETUP(audio) → SETUP(video) → SETUP(control) → ANNOUNCE → PLAY, each carrying a CSeq header. DESCRIBE returns an SDP-ish body of a= attributes advertising host capabilities (x-ss-general.featureFlags, encryptionSupported/Requested, refPicInvalidation, AV1 rtpmap, Opus surround-params). SETUP returns Session: DEADBEEFCAFE;timeout = 90 plus Transport: server_port= and an X-SS-Connect-Data (control) or X-SS-Ping-Payload (A/V) header. ANNOUNCE is where the client sends the actual negotiated stream config as an SDP body of x-nv-video[0].*, x-nv-vqos[0].*, x-nv-general.*, x-nv-audio.*, x-nv-aqos.*, x-ss-video[0].*, x-ml-* attributes (resolution, fps, bitrate, packetSize, fecPercentage, codec/bitStreamFormat, HDR, surround, encryptionEnabled). PLAY simply ACKs and the host begins sending RTP. When encryption is negotiated the RTSP messages themselves are wrapped in an encrypted framing (typeAndLength MSB-flagged + sequenceNumber + 16-byte GCM tag) keyed by the same gcm_key with IV bytes 10/11 = direction+'R'.",
"ports": [
"47984 = HTTPS control (PORT_HTTPS, base+5) — serves /launch, /resume, /serverinfo, /pair over TLS",
"47989 = HTTP control (base port; PORT_HTTP) — unauthenticated /serverinfo etc.",
"48010 TCP = RTSP_SETUP_PORT (base 47989 + 21) — the RTSP handshake; this is the task's stated port",
"47998 UDP = VIDEO_STREAM_PORT (base + 9) — RTP video, returned in SETUP video Transport server_port",
"47999 UDP = CONTROL_PORT (base + 10) — ENet/control + remote input, returned in SETUP control",
"48000 UDP = AUDIO_STREAM_PORT (base + 11) — RTP audio, returned in SETUP audio",
"All ports are base+offset via net::map_port(); base is configurable (default 47989). Moonlight overrides VideoPortNumber/AudioPortNumber/ControlPortNumber from the SETUP Transport server_port= field, with fallbacks video=47998, audio=48000, control=47999"
],
"wire_formats": [
{
"name": "RTSP request line + headers (plaintext)",
"layout": "ASCII text, CRLF line endings. Request line: RTSP/1.0\\r\\n. Methods: OPTIONS, DESCRIBE, SETUP, ANNOUNCE, PLAY. Common headers: CSeq: \\r\\n (monotonic, set by client currentSeqNumber), X-GS-ClientVersion: (AppVersionQuad[0] map 3→10,4→11,5→12,7→14), Host: (TCP only). Header block terminated by \\r\\n\\r\\n. Body (if any) preceded by Content-type: application/sdp and Content-length: .",
"notes": "Parsed by moonlight-common-c parseRtspMessage(), which Sunshine vendors in. NOT 100% standard RTSP — targets use streamid= scheme and there is an alternate ENet 'rtspru://' transport for ancient GFE (\\r\\nSession: DEADBEEFCAFE;timeout = 90\\r\\nTransport: server_port=\\r\\n then ONE of: X-SS-Connect-Data: (for control stream) OR X-SS-Ping-Payload: (for audio/video). 200 OK.",
"notes": "Session string is the literal constant DEADBEEFCAFE;timeout = 90 (note the spaces around '='). server_port is the UDP port the client must send/recv on. X-SS-Ping-Payload is the per-session magic the client must send as the first UDP datagram to A/V ports so the host learns the client's source port; X-SS-Connect-Data likewise for control."
},
{
"name": "DESCRIBE response body (host capabilities SDP)",
"layout": "Newline-joined a= lines (Sunshine cmd_describe builds via stringstream, each << std::endl):\\n a=x-ss-general.featureFlags:\\n a=x-ss-general.encryptionSupported:\\n a=x-ss-general.encryptionRequested:\\n a=x-nv-video[0].refPicInvalidation:1 (only if encoder supports RFI)\\n sprop-parameter-sets=AAAAAU (emitted unless HEVC-only forced; HEVC indicator)\\n a=rtpmap:98 AV1/90000 (emitted unless AV1 forced off)\\n a=fmtp:97 surround-params= (one per audio::stream_configs entry; 5.1/7.1 rotate mapping)\\n a=rtpmap audio is implied via fmtp:97 / fmtp:96.",
"notes": "encryptionSupported default = SS_ENC_CONTROL_V2|SS_ENC_AUDIO (=0x05); adds SS_ENC_VIDEO(0x02) unless encryption mode NEVER. encryptionRequested default = SS_ENC_CONTROL_V2 (=0x01); adds VIDEO|AUDIO if mode MANDATORY. Moonlight scans this body for 'AV1/90000', 'sprop-parameter-sets=AAAAAU' (HEVC), refPicInvalidation, the x-ss-general.* flags, and the fmtp:97 surround-params (channelCount, streamCount, coupledStreams, channel mapping for Opus multistream)."
},
{
"name": "ANNOUNCE request body (client→host negotiated config SDP)",
"layout": "Body: v=0\\r\\no=android 0 IN \\r\\ns=NVIDIA Streaming Client\\r\\n then a=: lines, then t=0 0\\r\\nm=video \\r\\n. Sunshine cmd_announce splits on lines, takes s= as client name and a=name:value into a map. Keys parsed into stream::config_t: x-nv-video[0].clientViewportWd (width), x-nv-video[0].clientViewportHt (height), x-nv-video[0].maxFPS (fps int), x-nv-video[0].clientRefreshRateX100 (fps*100), x-nv-video[0].packetSize (MTU payload), x-nv-video[0].videoEncoderSlicesPerFrame (slices), x-nv-video[0].maxNumReferenceFrames, x-nv-video[0].encoderCscMode, x-nv-video[0].dynamicRangeMode (HDR 0/1), x-nv-vqos[0].bitStreamFormat (codec: 0=H264,1=HEVC,2=AV1), x-nv-vqos[0].bw.maximumBitrateKbps (bitrate), x-nv-vqos[0].fec.minRequiredFecPackets, x-nv-vqos[0].qosTrafficType, x-nv-audio.surround.numChannels, x-nv-audio.surround.channelMask, x-nv-audio.surround.AudioQuality, x-nv-aqos.packetDuration, x-nv-aqos.qosTrafficType, x-nv-general.useReliableUdp (controlProtocolType, 13 or 1), x-nv-general.featureFlags (bit 0x20 ⇒ enable SS_ENC_AUDIO), x-ml-general.featureFlags, x-ml-video.configuredBitrateKbps, x-ss-general.encryptionEnabled (the actual negotiated enc bitmask), x-ss-video[0].chromaSamplingType (0=4:2:0,1=4:4:4), x-ss-video[0].intraRefresh.",
"notes": "Sunshine fills defaults via try_emplace before reading: encoderCscMode=0, bitStreamFormat=0, dynamicRangeMode=0, packetDuration=5, useReliableUdp=1, fec.minRequiredFecPackets=0, x-nv-general.featureFlags=135, x-ml-general.featureFlags=0, vqos qosTrafficType=5, aqos qosTrafficType=4, configuredBitrateKbps=0, encryptionEnabled=0, chromaSamplingType=0, intraRefresh=0, clientRefreshRateX100=0. Missing REQUIRED keys (width/height/fps/packetSize/bitrate/channels) throw std::out_of_range → 400 BAD REQUEST. Moonlight's SdpGenerator.c additionally emits but Sunshine ignores: rateControlMode=4, timeoutLengthMs=7000, framesWithInvalidRefThreshold=0, fec.enable=1, fec.repairPercent (5 or 20), fec.minRequiredFecPackets=2, bllFec.enable=0, videoQualityScoreUpdateTime=5000, bw.minimumBitrateKbps, initialBitrateKbps, x-nv-clientSupportHevc, surround.enable, enableRecoveryMode=0, x-nv-ri.useControlChannel=1."
},
{
"name": "Encrypted RTSP framing (when SS_ENC_CONTROL_V2 negotiated)",
"layout": "struct encrypted_rtsp_header_t { uint32_t typeAndLength; /*big-endian; MSB ENCRYPTED_MESSAGE_TYPE_BIT=0x80000000 set, low bits = payload length*/ uint32_t sequenceNumber; /*big-endian, monotonic*/ uint8_t tag[16]; /*AES-128-GCM auth tag*/ }; followed by ciphertext payload (the plaintext RTSP message).",
"notes": "AES-GCM IV (12 bytes, NIST SP800-38D 8.2.1): bytes[0..4]=sequenceNumber big-endian, byte[10]='C'(client-originated) or 'H'(host-originated), byte[11]='R' (RTSP). Decrypt input is tag||ciphertext. Plaintext RTSP (no enc) is delimited by \\r\\n\\r\\n."
},
{
"name": "/launch HTTPS query (where RIKEY/RIKEYID arrive)",
"layout": "GET https://:47984/launch?uniqueid=...&appid=&mode=xx&additionalStates=...&sops=<0|1>&rikey=<32 hex chars>&rikeyid=&localAudioPlayMode=<0|1>&surroundAudioInfo=&remoteControllersBitmap=...&gcmap=...&hdrMode=<0|1>&clientHdrCapabilities=...&corever=. /resume is the same minus appid/mode.",
"notes": "Sunshine requires rikey, rikeyid, localAudioPlayMode, appid present or 400. rikey = util::from_hex_vec(rikey,true) → gcm_key (16 bytes). iv = 16-byte buffer with big-endian uint32(rikeyid) in the FIRST 4 bytes, rest zero. mode split on 'x' → width/height/fps. surroundAudioInfo default 196610 (=0x30002 ⇒ 2-channel mask 0x3 high16=channels). corever decides whether encrypted RTSP is used. The launch response gives root.sessionUrl0 = :48010 telling Moonlight where to open RTSP."
}
],
"flow": [
"0. (HTTPS, prior to RTSP) Client GETs /serverinfo, then /launch?...&rikey=&rikeyid=&mode=WxHxF&... on port 47984. Host stores gcm_key=hex(rikey), iv=BE32(rikeyid)||zeros, parses mode/sops/surroundAudioInfo/hdrMode, allocates the launch_session, and responds with root.sessionUrl0=rtsp[enc]://:48010.",
"1. Client opens TCP to 48010 and sends OPTIONS RTSP/1.0 with CSeq:1, X-GS-ClientVersion. Host replies 200 OK echoing CSeq (cmd_option).",
"2. Client sends DESCRIBE RTSP/1.0 (CSeq:2, Accept/If-Modified-Since). Host replies 200 OK with the a= capability body (featureFlags, encryptionSupported/Requested, refPicInvalidation, AV1 rtpmap, surround-params).",
"3. Client sends SETUP streamid=audio/0/0 (CSeq:3). Host replies Session: DEADBEEFCAFE;timeout = 90, Transport: server_port=48000, X-SS-Ping-Payload:<...>.",
"4. Client sends SETUP streamid=video/0/0 (CSeq:4). Host replies Transport: server_port=47998, X-SS-Ping-Payload:<...>.",
"5. Client sends SETUP streamid=control/13/0 (CSeq:5). Host replies Transport: server_port=47999, X-SS-Connect-Data:<...>. (Moonlight latches these server_port values into Video/Audio/ControlPortNumber.)",
"6. Client sends ANNOUNCE (CSeq:6) with Content-type: application/sdp and the full x-nv-*/x-ss-*/x-ml-* config body (resolution, fps, bitrate, packetSize, fecPercent, bitStreamFormat codec, dynamicRangeMode HDR, surround, encryptionEnabled). Host parses into config_t; missing required key → 400. 200 OK on success.",
"7. Client sends PLAY (CSeq:7; single PLAY '/' for GFE≥7.1.431, else per-stream). Host replies 200 OK and the RTP video/audio + control/input flows begin on the UDP ports. First UDP datagram from client on each A/V port carries the X-SS-Ping-Payload so the host learns the client source port (NAT punch / port-learn)."
],
"crypto": "RIKEY/RIKEYID origin: the Moonlight client app generates remoteInputAesKey[16] and remoteInputAesIv[16] (STREAM_CONFIGURATION in Limelight.h) before connecting; rikey = hex(remoteInputAesKey), rikeyid = a 32-bit int. They are delivered to the host NOT in RTSP but in the HTTPS /launch query string. Sunshine make_launch_session: gcm_key = from_hex_vec(rikey) (16 bytes, the AES-128-GCM key shared by video/audio/control/input/RTSP ciphers); iv (16 bytes) = big-endian uint32(rikeyid) in bytes[0..4], zero-padded. AES-128-GCM is used everywhere. Per-stream 12-byte GCM nonce construction (Sunshine stream.cpp / rtsp.cpp): VIDEO (host→client): bytes[0..]=gcm_iv_counter (LE incrementing), byte[11]='V'. CONTROL host→client: 12-byte iv = seq(LE) || byte[10]='H', byte[11]='C'. CONTROL client→host: seq(LE) || byte[10]='C', byte[11]='C'. RTSP (encrypted handshake): seq(BE in bytes[0..4]) || byte[10]='C'(client)|'H'(host), byte[11]='R'. Encryption is negotiated, not key-exchanged: DESCRIBE advertises x-ss-general.encryptionSupported (default SS_ENC_CONTROL_V2|SS_ENC_AUDIO=0x05, +SS_ENC_VIDEO=0x02 if allowed) and encryptionRequested (default SS_ENC_CONTROL_V2=0x01, +VIDEO|AUDIO if MANDATORY); the client echoes the chosen bitmask in ANNOUNCE x-ss-general.encryptionEnabled, and x-nv-general.featureFlags bit 0x20 forces SS_ENC_AUDIO on. Flags: SS_ENC_CONTROL_V2=0x01, SS_ENC_VIDEO=0x02, SS_ENC_AUDIO=0x04. Codec tag bitStreamFormat: 0=H264,1=HEVC,2=AV1 (client capability bits VIDEO_FORMAT_H264=0x0001,H265=0x0100,H265_MAIN10=0x0200,AV1_MAIN8=0x1000,AV1_MAIN10=0x2000).",
"rust_options": "For RTSP on 48010: a tiny synchronous TCP server using std::net::TcpListener on a native thread (NO tokio — keep off the hot path, consistent with punktfunk's no-async-on-hot-path invariant). Parse RTSP/1.0 manually: read until \\\\r\\\\n\\\\r\\\\n, split request line + headers, read Content-length body for ANNOUNCE. There is no need for a crate; a hand-rolled parser mirroring Sunshine's parseRtspMessage is simplest and avoids pulling RTSP libs that assume standard semantics (the streamid= targets and DEADBEEFCAFE session break them). For SDP build/parse, just format!/split on lines — it is line-oriented a=key:value. For the /launch HTTPS endpoint, reuse the existing crates/punktfunk-host/src/web.rs seam; a small hyper or tiny_http + rustls TLS server (control plane only, async OK here since it is not the hot path — matches the 'quic feature gated' precedent). For encrypted-RTSP framing use the aes-gcm crate already in punktfunk-core. Suggested new types: an RtspServer in punktfunk-host that produces a punktfunk_core::Config from the ANNOUNCE map. Hex decode rikey with the `hex` crate (or from_str_radix). Big-endian rikeyid → IV with u32::to_be_bytes.",
"reuse_from_punktfunk": "REUSE: punktfunk-core/src/crypto.rs SessionCrypto already implements AES-128-GCM with per-direction salting and seq-as-AAD — but NOTE it is NOT byte-compatible with GameStream's nonce layout. punktfunk uses nonce = salt(4) || seq(8, BE) with a direction bit folded into salt[0], whereas GameStream uses iv = seq(LE or BE per stream) with literal direction/stream marker bytes at [10]/[11] ('V', 'H'/'C'+'C', 'C'/'H'+'R'). To talk to stock Moonlight you must add a GameStream-exact nonce mode (new constructor or a feature) rather than reuse the existing salt scheme verbatim. The Aes128Gcm cipher init and seal/open plumbing are reusable. REUSE: punktfunk-core Config/FecConfig — fec_percent maps to GameStream's repairPercent and the recovery_for() ceil(k*pct/100) already matches GameStream's FEC math; map ANNOUNCE packetSize→shard_payload, maximumBitrateKbps→bitrate, fec.minRequiredFecPackets→minRequiredFecPackets. FecScheme::Gf8 is the GameStream-compatible field. BUILD NEW: the entire RTSP/SDP/launch negotiation layer (punktfunk's internal 40-byte packet format and Config are not wire-exact to RTP/RTSP); the RTSP server, SDP describe/announce codec, the /launch query parser that produces gcm_key+iv from rikey/rikeyid, and the GameStream RTP video/audio packetization + RTPFEC are all new (separate areas). The Session in punktfunk-core can consume the negotiated Config but its on-wire packet header must be swapped for GameStream RTP for Moonlight compat.",
"gotchas": [
"punktfunk-core's AES-GCM nonce layout is NOT GameStream-compatible (salt+BE-seq vs literal 'V'/'C'/'R' marker bytes + LE/BE seq). A stock Moonlight will fail auth unless you implement the exact per-stream IV construction. This is the single biggest bridging hazard.",
"rikeyid is parsed as a SIGNED int then cast to big-endian uint32 in Sunshine (util::from_view → int → endian::big). Negative rikeyid values wrap; match the signed-int→BE-u32 path exactly.",
"The IV from /launch (BE32(rikeyid)||zeros, 16 bytes) is the *base*; the actual per-packet 12-byte GCM nonce is rebuilt per stream with seq + marker bytes — do not just use the 16-byte launch IV directly.",
"Session header value is the literal 'DEADBEEFCAFE;timeout = 90' WITH spaces around '='. Moonlight is lenient but match it.",
"SETUP Transport must be exactly 'server_port=' (Moonlight greps for 'server_port='); it also expects X-SS-Ping-Payload (A/V) / X-SS-Connect-Data (control) headers and will send that payload as the first UDP datagram for port-learning — the host must accept it.",
"ANNOUNCE keys are case-sensitive and bracketed exactly 'x-nv-video[0].' — the [0] index is literal. Missing a REQUIRED key (width/height/fps/packetSize/bitrate/channels) yields 400; supply the same try_emplace defaults Sunshine uses or Moonlight builds may omit them.",
"Codec is bitStreamFormat (0/1/2), but capability advertisement in DESCRIBE uses sprop-parameter-sets=AAAAAU (HEVC marker) and a=rtpmap:98 AV1/90000 — Moonlight infers codec support from those strings, so emit/omit them to steer codec.",
"fec.repairPercent (GFE<7.1.431) vs fec.minRequiredFecPackets (GFE≥7.1.431) — newer clients send the latter; Sunshine reads minRequiredFecPackets and defaults repairPercent handling. Handle both.",
"Stream targets differ by GFE version: modern 'streamid=video/0/0' and 'streamid=control/13/0', legacy 'streamid=video'. Parse by splitting on '=' and '/', taking the type token, like Sunshine.",
"There is an alternate ENet 'rtspru://' RTSP transport for very old GFE (>= 8; &= 0xFFFFFF (24-bit stream index). Flags/extraFlags/multiFecFlags/multiFecBlocks are single bytes (no swap)."
},
{
"name": "fecInfo bit packing (uint32, LE on wire)",
"layout": "fecInfo = (fecIndex << 12) | (dataShards << 22) | (fecPercentage << 4). Decode (RtpVideoQueue.c): fecIndex = (fecInfo & 0x3FF000) >> 12 (10 bits, the shard's RS index within its block); dataShards = (fecInfo & 0xFFC00000) >> 22 (10 bits); fecPercentage = (fecInfo & 0xFF0) >> 4 (8 bits). Bits 0..3 unused. parityShards is NOT transmitted — client recomputes: parityShards = (dataShards * fecPercentage + 99) / 100.",
"notes": "stream.cpp ~1485-1488 set side; RtpVideoQueue.c lines 583, 703-705 decode side. fecIndex tops out at 1023 (>=1024 packets/block = unrecoverable, logged as error). dataShards <= 255 (DATA_SHARDS_MAX)."
},
{
"name": "multiFecBlocks / multiFecFlags bit packing (uint8 each)",
"layout": "multiFecFlags = 0x10 (constant; marks multi-FEC protocol). multiFecBlocks = (currentBlockIndex << 4) | ((totalBlocks-1) << 6). Decode: currentBlock = (multiFecBlocks >> 4) & 0x3; lastBlock = (multiFecBlocks >> 6) & 0x3; totalBlocks = lastBlock+1 (1..4). Legacy/non-multiFec servers: client forces multiFecFlags=0x10, multiFecBlocks=0x00.",
"notes": "stream.cpp 1438-1439; RtpVideoQueue.c 584, 709. Only 2 bits each for current and last block → max 4 FEC blocks per frame (MAX_FEC_BLOCKS=4)."
},
{
"name": "video_short_frame_header_t (8 bytes, prepended to the frame bitstream, LITTLE-endian)",
"layout": "offset 0: uint8 headerType (always 0x01); offset 1: uint16 frame_processing_latency (LE, 1/10 ms, Sunshine ext, 0 if N/A); offset 3: uint8 frameType (1=normal P, 2=IDR, 4=P w/ intra-refresh, 5=P after ref-frame-invalidation); offset 4: uint16 lastPayloadLen (LE, length of final packet's real payload, for codecs like AV1 that can't tolerate zero padding); offset 6: uint8 unknown[2]. This 8-byte header precedes the actual H.264/HEVC/AV1 access unit, and the concatenation is what gets striped into shards.",
"notes": "stream.cpp video_short_frame_header_t (static_assert ==8). lastPayloadLen = (payloadSize + 8) % (packetSize - sizeof(NV_VIDEO_PACKET)); if 0 → set to (packetSize - sizeof(NV_VIDEO_PACKET)). frameType 2 == IDR is how the client detects keyframes."
},
{
"name": "ENC_VIDEO_HEADER / video_packet_enc_prefix_t (32 bytes) — present only when SS_ENC_VIDEO",
"layout": "offset 0: uint8 iv[12]; offset 12: uint32 frameNumber (LE); offset 16: uint8 tag[16]. This is a WIRE PREFIX that sits in front of the (encrypted) shard, NOT inside the FEC blocksize. On-wire encrypted packet = ENC_VIDEO_HEADER (32B) || ciphertext(blocksize bytes). ciphertext is the AES-128-GCM encryption of the entire plaintext shard (RTP+reserved+NV_VIDEO_PACKET+payload = blocksize). tag is the 16-byte GCM tag. iv is the literal 12-byte nonce used (sent so the client doesn't have to reconstruct it). frameNumber = packet->frame_index().",
"notes": "Video.h _ENC_VIDEO_HEADER and stream.cpp video_packet_enc_prefix_t + encrypt call ~1498-1515. Header must be a multiple of 16 bytes so the FEC blocksize stays a multiple of 16 (comment in Video.h). When encryption is on, the SDP negotiation subtracts sizeof(ENC_VIDEO_HEADER) from packetSize (SdpGenerator.c line 325)."
},
{
"name": "Full on-wire datagram (one shard)",
"layout": "UNENCRYPTED: [RTP_PACKET 12B][reserved 4B][NV_VIDEO_PACKET 16B][payload up to (packetSize - sizeof(NV_VIDEO_PACKET))]. Total header before payload = 32B = sizeof(video_packet_raw_t). ENCRYPTED: [ENC_VIDEO_HEADER 32B] || AES-GCM-ciphertext( the entire 32B+payload plaintext = blocksize ). The plaintext that gets encrypted INCLUDES the RTP and NV headers; the only cleartext fields on an encrypted packet are the 32-byte ENC_VIDEO_HEADER prefix.",
"notes": "Client recv buffer sizing (VideoStream.c 96-99): decryptedSize = packetSize + MAX_RTP_HEADER_SIZE(16) = blocksize; receiveSize = decryptedSize + (encrypted ? 32 : 0). So blocksize == packetSize + 16. After decrypt, client byteswaps RTP seq/timestamp/ssrc to host order, then calls RtpvAddPacket."
}
],
"flow": [
"Host has an encoded access unit (HEVC/H264/AV1) for frame N from NVENC. It computes frame_index N (monotonic) and builds the 8-byte video_short_frame_header_t (headerType=0x01, frameType per IDR/P, lastPayloadLen, latency).",
"Host prepends sizeof(video_packet_raw_t)=32 bytes of header space per shard via concat_and_insert with payload_blocksize = blocksize - 32, where blocksize = packetSize + 16. This produces the striped payload buffer with room for per-shard headers.",
"Host decides FEC block count: max_data_shards_per_fec_block = (255*100)/(100+fecPercentage); fec_blocks_needed = ceil(payload / (max_data_shards*blocksize)), capped at 4. If >4 needed, FEC is disabled for that frame (fecPercentage=0). Each block aligned to blocksize: aligned_size = roundup(payload/blocks, blocksize).",
"For each FEC block: host fills each data shard's RTP+NV headers (frameIndex, streamPacketIndex=(lowseq+x)<<8, multiFecFlags=0x10, multiFecBlocks, flags incl SOF on x==0 and EOF on last data packet).",
"Host calls fec::encode(block, blocksize, fecPercentage, minRequiredFecPackets, prefixsize): data_shards = ceil(blockBytes/blocksize) (last data shard zero-padded), parity_shards = ceil(data_shards*fecPercentage/100) (raised to minRequiredFecPackets if below), then nanors reed_solomon_new(data_shards, parity_shards) + reed_solomon_encode over all shards at blocksize.",
"Host stamps every shard (data AND parity) with fecInfo = (x<<12 | data_shards<<22 | percentage<<4), rtp.header=0x80|0x10, rtp.sequenceNumber=BE16(lowseq+x), rtp.timestamp=BE32(timestamp), multiFecBlocks, frameIndex.",
"If video encryption: for each shard, build iv = (gcm_iv_counter as 8 LE bytes)||0,0,0 with iv[11]='V'; increment counter; AES-128-GCM encrypt the whole blocksize buffer in place (no AAD), writing tag into the prefix; set prefix.frameNumber=frame_index, prefix.iv=iv. Wire packet = prefix(32B)||ciphertext(blocksize).",
"Host sends all data then parity shards as UDP datagrams (paced ~80% of 1Gbps, batched up to 64 packets) to the client's video endpoint; lowseq advances by total shards across all blocks of the frame.",
"Client recv: if encrypted, read prefix, drop early if prefix.frameNumber < currentFrameNumber, else AES-128-GCM decrypt ciphertext into buffer using iv+tag from prefix (auth-fail → drop). Then byteswap RTP seq/timestamp/ssrc to host order.",
"Client RtpvAddPacket: parse NV header (LE32 swaps), derive fecIndex, currentBlock, dataShards, fecPercentage, recompute parityShards; set bufferLowestSequenceNumber = seq - fecIndex; place shard at RS index = seq - bufferLowestSequenceNumber.",
"When a block has >= dataShards shards (data+parity), client runs nanors reed_solomon_decode(rs, packets, marks, totalPackets, blocksize) to recover any missing data shards (missing slots zero-filled, marks[]=1 for missing). Recovered data shards get synthetic RTP headers (seq/header/timestamp/ssrc copied from a present packet).",
"Client advances through blocks (currentBlock 0..lastBlock); once all blocks' data shards are present/recovered it strips the 32-byte video_packet_raw_t header off each data shard, concatenates payloads in sequence order, parses the 8-byte short frame header, and hands the reassembled access unit to the depacketizer/decoder."
],
"crypto": "\"Cipher: AES-128-GCM (EVP_aes_128_gcm). Key: the 16-byte RIKEY — Sunshine `launch_session.gcm_key`, Moonlight `StreamConfig.remoteInputAesKey[16]` — established during RTSP/pairing; the SAME key is used for the control stream and (if enabled) audio. There is no separate video key and no key derivation: the raw 16-byte RIKEY is used directly. IV/nonce: 12 bytes, constructed deterministically (NIST SP 800-38D 8.2.1): iv[0..8] = a 64-bit per-session counter (session->video.gcm_iv_counter, starts at 0) copied in NATIVE byte order (little-endian on x86), iv[8..11] = 0 except iv[11] = 'V' (0x56, the video-stream fixed field). The counter increments once per shard. The full 12-byte IV is transmitted in the ENC_VIDEO_HEADER so the client uses it verbatim (it does not reconstruct it). Tag: 16 bytes, standard GCM tag, transmitted in ENC_VIDEO_HEADER. AAD / associated data: NONE — Sunshine's gcm_t::encrypt for video calls EVP_EncryptUpdate only with plaintext (no AAD update), and Moonlight's PltDecryptMessage passes no AAD argument. (NOTE: this differs from punktfunk-core/crypto.rs which uses seq-as-AAD and a per-direction salt; GameStream video does neither.) Order: FEC FIRST, THEN ENCRYPT — encryption is applied per-shard after RS parity is computed, over the entire blocksize shard buffer (RTP+NV+payload), so the client must DECRYPT each shard before it can run FEC reconstruction. Encrypted plaintext length == blocksize; ciphertext length == blocksize (GCM is a stream cipher, no expansion); on the wire the packet grows only by the 32-byte prefix.\"",
"rust_options": "\"FEC math: punktfunk already has Gf8Coder over `reed-solomon-erasure` (galois_8). CRITICAL RISK: this is NOT guaranteed byte-compatible with nanors. Both Sunshine and Moonlight use nanors (Sunshine's rswrapper.h is 'a drop-in replacement for nanors rs.h', DATA_SHARDS_MAX=255), which uses a specific GF(2^8) field (primitive poly 0x11d) and a Vandermonde-derived generator matrix with a particular systematic encoding. `reed-solomon-erasure` uses Cauchy matrices by default and may produce DIFFERENT parity bytes — meaning Moonlight would FAIL to recover frames where any data shard is lost. RECOMMENDED: vendor/FFI the actual nanors C library (it is tiny, MIT, header+rs.c+oblas) and call reed_solomon_new/encode through a thin Rust FFI, OR port nanors' matrix construction exactly into a new gf8 backend. Do NOT assume reed-solomon-erasure interop without a byte-for-byte test against nanors output. (For punktfunk-to-punktfunk P2 traffic, keep the existing coders; for GameStream-client compat, use nanors.) Crypto: use `aes-gcm` (already a dep) but build a NEW path that (a) takes the raw RIKEY as the key, (b) builds the 12-byte IV as counter_le[8]||0||0||0||'V', (c) uses NO AAD, (d) does NOT use the per-direction salt logic in SessionCrypto. The cleanest approach is a small standalone `Aes128Gcm` call rather than reusing SessionCrypto (whose nonce/AAD scheme is incompatible). Byte layout: define `#[repr(C, packed)]` structs RtpPacket, NvVideoPacket, EncVideoHeader and use explicit `to_be_bytes`/`to_le_bytes` per field (RTP=BE, NV=LE) — do not rely on struct memory layout for endianness.\"",
"reuse_from_punktfunk": "\"REUSE the GF(2^8) concept/structure but NOT necessarily the implementation: punktfunk's `ErasureCoder` trait and `Gf8Coder` (reed-solomon-erasure) give the right data||parity systematic layout and the 255-shard ceiling, but parity bytes likely won't match nanors — so for client-facing GameStream compat add a `nanors`-backed coder (FFI or exact port) behind the same trait. The trait's reconstruct(data_count, recovery_count, received: indices 0..K originals, K..K+M recovery) maps cleanly onto Moonlight's layout (data shards at RS index 0..dataShards, parity after) so the adapter just needs to map RTP-seq→shard-index. REUSE aes-gcm crate but NOT SessionCrypto (its salt+seq-AAD scheme is wire-incompatible with GameStream video which uses no AAD and a counter||'V' IV). REUSE punktfunk's UDP transport for sending datagrams. Do NOT reuse punktfunk's internal 40-byte packet format — GameStream needs the exact 12+4+16 header + optional 32-byte enc prefix. NEW work: a `gamestream` wire module in punktfunk-host (NOT punktfunk-core, to keep punktfunk-core's clean internal protocol) that (1) builds RTP_PACKET/NV_VIDEO_PACKET/ENC_VIDEO_HEADER bytes, (2) implements the frame→FEC-block split (max 4 blocks, the 255/(1+F) shard math), (3) drives a nanors coder, (4) does the per-shard counter-IV AES-128-GCM-no-AAD encrypt, (5) paces/batches sends. Best location: a new file like crates/punktfunk-host/src/gamestream/video.rs (host side) with the nanors FFI either in punktfunk-host or a small new crate; punktfunk-core stays the punktfunk-native protocol and only its aes-gcm + the gf8 *math* are conceptually shared. The 'adapter' lives at the punktfunk-host pipeline seam: take the NVENC access unit + frame metadata from encode.rs and emit GameStream datagrams instead of (or alongside) punktfunk-native packets.\"",
"gotchas": [
"ENDIANNESS SPLIT: RTP_PACKET fields are BIG-endian, NV_VIDEO_PACKET fields are LITTLE-endian, within the SAME packet. Easy to get wrong. RTP: BE16/BE32; NV streamPacketIndex/frameIndex/fecInfo: LE32.",
"ENCRYPT-AFTER-FEC, not before: the GCM-encrypted region is the WHOLE shard (RTP+NV+payload) and the client must decrypt each shard before FEC. The 32-byte ENC_VIDEO_HEADER is a wire PREFIX outside the FEC blocksize, not part of the protected data. If you FEC after encrypt or include the prefix in the FEC math, recovery breaks.",
"NO AAD on video GCM — unlike punktfunk-core's SessionCrypto which authenticates the sequence number as AAD. Using SessionCrypto verbatim will fail Moonlight's tag check.",
"IV counter byte order: Sunshine copies the 64-bit counter with std::copy_n in NATIVE order (little-endian on the x86 build), so iv[0..8] is the counter LE; iv[11]='V'(0x56), iv[8..11]=0. The client uses the transmitted iv verbatim, so as long as you SEND the iv you used, internal byte order is self-consistent — but match LE to mirror Sunshine exactly and to keep nonces unique.",
"FEC parity matrix must match nanors EXACTLY. reed-solomon-erasure (punktfunk's current backend) is likely NOT byte-compatible (Cauchy vs nanors Vandermonde). Without a byte-for-byte match, Moonlight silently fails to recover any frame with a lost data shard. Validate against real nanors output or FFI nanors.",
"streamPacketIndex is (lowseq+x)<<8 with low byte zero; client does >>=8 then &0xFFFFFF → a 24-bit stream-wide packet index, distinct from the 16-bit RTP sequenceNumber. Both must be set consistently or the depacketizer's continuity check (FLAG_SOF / streamPacketIndex == lastPacketInStream+1) rejects the frame.",
"Max 4 FEC blocks per frame (2-bit fields). Max 1024 packets per block (10-bit fecIndex). Max 255 shards/block (GF(2^8)). data_shards per block = 255*100/(100+fecPercentage). Exceeding these → FEC disabled or unrecoverable frame.",
"Last data shard is zero-padded to blocksize before RS encode; lastPayloadLen in the short frame header tells the client the real length of the final packet's payload (needed for AV1). Padding must be zeros so RS math and the client's memset-padding agree.",
"fecPercentage and parityShards: the host transmits dataShards and fecPercentage in fecInfo but NOT parityShards; the client recomputes parityShards = (dataShards*fecPercentage+99)/100. Use the IDENTICAL rounding (ceil) or shard indices misalign. Sunshine may also bump fecPercentage up to satisfy minRequiredFecPackets — recompute percentage = 100*parity/data in that case and stamp the bumped value into fecInfo.",
"blocksize = packetSize + MAX_RTP_HEADER_SIZE(16). When encryption is enabled the SDP negotiation REDUCES packetSize by sizeof(ENC_VIDEO_HEADER)=32 first, so the encrypted-shard plaintext stays the original size. Get this off-by-32 right or buffers mismatch.",
"packetType in RTP_PACKET is effectively unused for video (Sunshine doesn't set it; client ignores it for video, keying on FLAG_EXTENSION instead). Do not rely on a video packetType constant like 97 (that's the AUDIO packetType; audio FEC is 127)."
],
"sources": [
"Sunshine src/stream.cpp — packetTypes[] (audio 97 / audio-fec 127, control types); struct video_short_frame_header_t (8B), video_packet_raw_t (RTP + reserved[4] + NV_VIDEO_PACKET), video_packet_enc_prefix_t (iv[12]/frameNumber/tag[16]); fec::encode() (data/parity shard math, nanors reed_solomon_new/encode, zero-pad last data shard); videoBroadcastThread() (frame->FEC-block split, fecInfo/multiFecBlocks/streamPacketIndex/flags packing, per-shard AES-GCM encrypt with counter||'V' IV, lines ~1434-1515); session cipher init from launch_session.gcm_key with SS_ENC_VIDEO (lines ~2032-2040).",
"Sunshine src/crypto.cpp — gcm_t::encrypt (EVP_aes_128_gcm, no AAD update, EVP_CTRL_GCM_GET_TAG tag_size=16), init_encrypt_gcm (EVP_CTRL_GCM_SET_IVLEN to iv size 12).",
"Sunshine src/rswrapper.h — 'drop-in replacement for nanors rs.h', #define DATA_SHARDS_MAX 255, reed_solomon_new/encode/decode signatures.",
"Sunshine src/rtsp.cpp — RIKEY/gcm IV context (12-byte IV, deterministic construction comment) confirming RIKEY is the GCM key.",
"moonlight-common-c src/Video.h — _RTP_PACKET (12B), _NV_VIDEO_PACKET (16B: streamPacketIndex/frameIndex/flags/extraFlags/multiFecFlags/multiFecBlocks/fecInfo), _ENC_VIDEO_HEADER (iv[12]/frameNumber/tag[16]), FLAG_CONTAINS_PIC_DATA=0x1/FLAG_EOF=0x2/FLAG_SOF=0x4, FLAG_EXTENSION=0x10, FIXED_RTP_HEADER_SIZE=12, MAX_RTP_HEADER_SIZE=16, SS_FRAME_FEC_STATUS.",
"moonlight-common-c src/RtpVideoQueue.c — RtpvAddPacket(): dataOffset=12(+4 for extension)=16; LE32 swaps of streamPacketIndex/frameIndex/fecInfo; fecIndex=(fecInfo&0x3FF000)>>12, dataShards=(fecInfo&0xFFC00000)>>22, fecPercentage=(fecInfo&0xFF0)>>4; currentBlock=(multiFecBlocks>>4)&0x3, lastBlock=(multiFecBlocks>>6)&0x3; parityShards=(dataShards*pct+99)/100; bufferLowestSequenceNumber=seq-fecIndex; reconstructFecBlock() nanors reed_solomon_new/decode over totalPackets at blocksize=packetSize+16, RS index = seq-bufferLowestSequenceNumber, recovered shards get synthetic RTP headers.",
"moonlight-common-c src/VideoStream.c — VideoReceiveThreadProc(): receiveSize=packetSize+16(+32 if SS_ENC_VIDEO); per-packet AES-GCM decrypt via PltDecryptMessage(ALGORITHM_AES_GCM, key=remoteInputAesKey[16], iv=encHeader->iv[12], tag=encHeader->tag[16], ciphertext after 32B header, NO AAD); early-drop if encHeader->frameNumber < currentFrameNumber; then BE16/BE32 swap of RTP seq/timestamp/ssrc before RtpvAddPacket.",
"moonlight-common-c src/VideoDepacketizer.c — processRtpPayload/reassembleFrame: streamPacketIndex >>= 8 & 0xFFFFFF (24-bit), SOF/EOF continuity, frameType for IDR detection.",
"moonlight-common-c src/PlatformCrypto.h — ALGORITHM_AES_GCM=2, PltDecryptMessage/PltEncryptMessage signatures (key,iv,tag,input,output — no AAD parameter).",
"moonlight-common-c src/Limelight.h — ENCFLG_VIDEO=0x2, ENCFLG_AUDIO=0x1, remoteInputAesKey[16] (the RIKEY).",
"moonlight-common-c src/SdpGenerator.c — SS_ENC_VIDEO negotiation; when enabled StreamConfig.packetSize -= sizeof(ENC_VIDEO_HEADER) (32).",
"moonlight-common-c .gitmodules + repo tree — nanors at /nanors (rs.c, rs.h, deps/obl GF(2^8) tables), confirming both host(Sunshine via rswrapper) and client use nanors.",
"punktfunk local: crates/punktfunk-core/src/crypto.rs (SessionCrypto: salt+seq-AAD scheme — incompatible with GameStream video), crates/punktfunk-core/src/fec/{mod.rs,gf8.rs} (ErasureCoder trait + reed-solomon-erasure galois_8 — needs nanors compat verification)."
]
},
{
"area": "GameStream audio stream — UDP RTP transport, Opus multistream config, Reed-Solomon FEC (4+2 over GF(2^8)), AES-CBC encryption",
"summary": "The audio stream is a one-way UDP RTP flow from host to client on the \"audio\" port (base+10; with the default GameStream base port 47989 that is 47999 — the well-known \"port 48000\" in the task refers to the default-numbered GameStream audio slot, but the offset is +10 / control is +11; both Sunshine and moonlight derive ports as base+offset). The host sends Opus-encoded 48 kHz audio in RTP packets with a fixed 12-byte RTP header where packetType=97 for data and 127 for FEC. Audio is grouped into fixed FEC blocks of 4 data shards + 2 parity shards (RTPA_DATA_SHARDS=4, RTPA_FEC_SHARDS=2) using Reed-Solomon over GF(2^8); critically Nvidia/Sunshine use a HARDCODED parity matrix that differs from a generic RS implementation (moonlight-common-c overrides its matrix with the bytes {0x77,0x40,0x38,0x0e,0xc7,0xa7,0x0d,0x6c} from OpenFEC to match the wire). Each Opus frame is one data shard; after every 4th data packet (sequenceNumber % 4 == 0 marks block start, (seq+1)%4==0 triggers encode) two FEC packets are emitted carrying an 12-byte AUDIO_FEC_HEADER after the RTP header. Opus is configured by the host: stereo = 2ch/1 stream/1 coupled, 5.1 = 6ch/4 streams/2 coupled (or 6/0 high quality), 7.1 = 8ch/5 streams/3 coupled (or 8/0 high quality), with channel mappings matching SMPTE/Vorbis order (FL,FR,FC,LFE,RL,RR,SL,SR). samplesPerFrame = 48 * AudioPacketDuration where AudioPacketDuration defaults to 5 ms (240 samples/channel) for lowest latency, optionally 10 ms (480 samples). Audio payload (when SS_ENC_AUDIO is enabled) is encrypted with AES-128-CBC (NOT GCM like video/control), keyed by the same session GCM/RI key, with a per-packet 16-byte IV whose first 4 bytes are big-endian (avRiKeyId + sequenceNumber) and the rest zero; payload is PKCS7-padded. The FEC parity is computed over the encrypted+padded shard bytes, all shards padded to the same block size. This differs from video FEC which uses larger dynamic block sizes, multi-FEC blocks, and a different packet header; audio FEC is a tiny fixed 4+2 layout.",
"ports": [
"Audio RTP (UDP, host→client): base_port + 10. With default GameStream base 47989 → 47999. The task's 'port 48000' corresponds to the canonical GameStream audio slot; verify against the actual base. Client binds and sends pings to this port (moonlight: SET_PORT(&saddr, AudioPortNumber), AudioPortNumber parsed from RTSP SETUP).",
"Video RTP: base_port + 9 (47998 default)",
"Control (ENet): base_port + 11 (48000 default)",
"RTSP setup: base_port + 21 (48010 default)",
"Audio port is dynamically negotiated via RTSP SETUP (moonlight reads AudioPortNumber); do not hardcode — but offset is +10 in Sunshine net::map_port(AUDIO_STREAM_PORT)"
],
"wire_formats": [
{
"name": "RTP_PACKET (12-byte audio RTP header)",
"layout": "Offset 0: header (uint8, set to 0x80 = RTP version 2, no padding/ext/CSRC). Offset 1: packetType (uint8): 97 (RTP_PAYLOAD_TYPE_AUDIO) for data, 127 (RTP_PAYLOAD_TYPE_FEC) for FEC. Offset 2: sequenceNumber (uint16, big-endian on wire, host-order after parse). Offset 4: timestamp (uint32, big-endian). Offset 8: ssrc (uint32, big-endian; Sunshine sets ssrc=0). Total 12 bytes, then payload.",
"notes": "Sunshine: audio_packet.rtp.header=0x80; .packetType=97; .ssrc=0; .sequenceNumber=big(seq); .timestamp=big(ts). seq increments by 1 per packet; timestamp += packetDuration (ms units, i.e. 5 or 10) each packet. Moonlight checks rtp->packetType==97 for data."
},
{
"name": "AUDIO_FEC_HEADER (12 bytes, follows RTP header in FEC packets)",
"layout": "Offset 0 (after the 12-byte RTP header, i.e. file offset 12): fecShardIndex (uint8) = 0 or 1 (which of the 2 parity shards). Offset 1: payloadType (uint8) = 97 (the data payload type being protected). Offset 2: baseSequenceNumber (uint16, big-endian) = seq of first data packet in the block (block start). Offset 4: baseTimestamp (uint32, big-endian) = timestamp of first data packet in block. Offset 8: ssrc (uint32, big-endian). Total 12 bytes, then the parity shard bytes (length = blockSize).",
"notes": "moonlight struct order: {uint8 fecShardIndex; uint8 payloadType; uint16 baseSequenceNumber; uint32 baseTimestamp; uint32 ssrc;}. Sunshine audio_fec_packet_t = {RTP_PACKET rtp; AUDIO_FEC_HEADER fecHeader;}. FEC packet rtp.packetType=127, rtp.sequenceNumber=big(baseSeq + x + 1) for shard x, fecHeader.payloadType=97, fecHeader.fecShardIndex=x."
},
{
"name": "FEC block (logical grouping)",
"layout": "4 data packets (RTPA_DATA_SHARDS) + 2 FEC packets (RTPA_FEC_SHARDS) = 6 total shards (RTPA_TOTAL_SHARDS). Block starts when sequenceNumber % 4 == 0. All shards (data payload + parity) padded to common blockSize for RS. Block can be recovered if any 4 of the 6 shards arrive.",
"notes": "Fixed small block: 4+2. moonlight RTPA_FEC_BLOCK holds dataPackets[4], fecPackets[2], marks[6]. Boundary: 'FEC blocks must start on a RTPA_DATA_SHARDS boundary.' This is unlike video which uses variable block sizes and multi-FEC blocks."
},
{
"name": "Encrypted audio payload (AES-128-CBC)",
"layout": "When SS_ENC_AUDIO flag set: payload = AES-128-CBC(opus_frame_PKCS7_padded). IV = 16 bytes: bytes[0..4] = big-endian uint32 (avRiKeyId + sequenceNumber), bytes[4..16] = 0. Key = session RI/GCM key (16 bytes, from remoteInputAesKey). Block size rounded via round_to_pkcs7_padded; max_block_size = round_to_pkcs7_padded(2048).",
"notes": "CBC, not GCM. No auth tag appended (unlike video/control GCM). moonlight: ivSeq = BE32(avRiKeyId + rtp->sequenceNumber); memcpy(iv,&ivSeq,4); decrypts then strips PKCS7. FEC parity computed over the ENCRYPTED+padded shard."
}
],
"flow": [
"RTSP SETUP negotiates audio: client/host agree on audio config (channels via AUDIO_CONFIGURATION mask), AudioPacketDuration (5 ms default, 10 ms fallback), and the audio UDP port (AudioPortNumber). avRiKeyId and the AES key/IV come from the launch/resume request (remoteInputAesKey/Iv).",
"Client opens UDP socket to host audio port and begins sending periodic ping packets every 500 ms to punch NAT and tell host where to send (legacy ping = ASCII 'PING' {0x50,0x49,0x4E,0x47}; modern = AudioPingPayload/SS_PING with sequence). Host learns client addr from first ping.",
"Host captures PCM, encodes with opus_multistream_encode_float at 48 kHz into samplesPerFrame-sized frames (240 samples/ch at 5 ms).",
"Host builds RTP data packet: header=0x80, packetType=97, seq++ (big-endian), timestamp += packetDuration (big-endian), ssrc=0. If SS_ENC_AUDIO: AES-128-CBC encrypt the PKCS7-padded Opus frame with IV=BE32(avRiKeyId+seq)||0. Send to client.",
"FEC accumulation: when seq % 4 == 0, record block baseSequenceNumber/baseTimestamp. The 4 (possibly encrypted) data shard payloads are placed in shards_p[seq % 4].",
"When (seq+1) % 4 == 0 (i.e. after the 4th data packet): reed_solomon_encode(rs, shards, RTPA_TOTAL_SHARDS=6, blockSize) generates 2 parity shards. Host sends 2 FEC packets: rtp.packetType=127, rtp.sequenceNumber=big(baseSeq + x + 1), fecHeader.fecShardIndex=x (0,1), fecHeader.payloadType=97, baseSequenceNumber/baseTimestamp/ssrc set.",
"Client (RtpAudioQueue) groups incoming packets by block (base seq aligned to 4). If ≥4 of 6 shards present and any data missing, reed_solomon_decode(rs, shards, marks, 6, blockSize) recovers them — using the hardcoded Nvidia parity matrix {0x77,0x40,0x38,0x0e,0xc7,0xa7,0x0d,0x6c}.",
"Client decrypts recovered/received data shards (AES-128-CBC, strip PKCS7), reorders by sequence, and feeds frames to opus_multistream_decoder_create/decode_float."
],
"crypto": "Cipher: AES-128-CBC (NOT GCM — video & control use GCM, audio uses CBC). Key: 16 bytes = the session AES key (Sunshine launch_session.gcm_key / moonlight StreamConfig.remoteInputAesKey), same key family used for the input/RI channel. IV: 16 bytes, per-packet — first 4 bytes = big-endian uint32 of (avRiKeyId + sequenceNumber), remaining 12 bytes = 0x00. avRiKeyId is a per-session 32-bit value from the RTSP/launch negotiation (session->audio.avRiKeyId). Padding: PKCS7 to AES block (16-byte) multiple; round_to_pkcs7_padded, max_block_size = round_to_pkcs7_padded(2048). No GMAC/auth tag is appended to audio packets. Encryption is gated by the SS_ENC_AUDIO encryption flag (config.encryptionFlagsEnabled & SS_ENC_AUDIO) — if disabled, raw Opus frame is sent. FEC parity is computed over the post-encryption, post-padding shard bytes so recovery yields ciphertext that is then decrypted.",
"rust_options": "Opus encode: use the `audiopus` crate or `opus` (libopus bindings) — both expose multistream via opus_multistream_encoder; if missing, FFI to libopus opus_multistream_encoder_create/opus_multistream_encode_float directly. Configure sampleRate=48000, the streams/coupledStreams and mapping per the negotiated AUDIO_CONFIGURATION; frame size = 48*packetDuration samples/ch. AES-128-CBC: use the `aes` + `cbc` crates (cbc::Encryptor) with manual PKCS7 (`block-padding`/`Pkcs7`) — build the 16-byte IV as BE32(avRiKeyId+seq) || [0u8;12]. Reed-Solomon: do NOT use a generic RS matrix; the wire requires Nvidia's specific parity matrix. The `reed-solomon-erasure` crate computes its own (Vandermonde/Cauchy) matrix that will NOT match — either (a) port moonlight's approach: take the rs lib's encode path but inject the OpenFEC parity matrix bytes {0x77,0x40,0x38,0x0e,0xc7,0xa7,0x0d,0x6c} for the 4+2 case, or (b) hand-roll a tiny GF(2^8) 4-data/2-parity encoder/decoder using that exact 2x4 parity matrix (8 bytes = 2 parity rows × 4 data cols). punktfunk's existing `reed-solomon` GF(2^8) code can be reused ONLY if it lets you supply a custom generator/parity matrix; otherwise add a dedicated audio-FEC path. Big-endian field writes: use `byteorder`/`to_be_bytes`. UDP: std::net::UdpSocket on a native thread (no async, matching punktfunk's hot-path rule).",
"reuse_from_punktfunk": "REUSE: punktfunk-core's AES-128 primitive (crypto.rs) underlies CBC, but the MODE differs — punktfunk uses AES-128-GCM with per-direction nonce salts + seq-as-AAD; GameStream audio needs AES-128-CBC with the BE32(avRiKeyId+seq) IV and PKCS7, no AAD/tag. So add a CBC path; do not reuse the GCM nonce/AAD scheme for audio. punktfunk's GF(2^8) Reed-Solomon (reed-solomon crate) is the right field but the MATRIX is wrong for the wire — must supply Nvidia's hardcoded {0x77,0x40,0x38,0x0e,0xc7,0xa7,0x0d,0x6c} parity matrix or hand-roll the 4+2 encoder; punktfunk's internal 40-byte packet format and its FEC block sizing are NOT wire-compatible and cannot be reused for the on-wire audio packets. punktfunk's UDP transport + native-thread pacing model is reusable as plumbing. NEW: 12-byte RTP header serializer, 12-byte AUDIO_FEC_HEADER, the fixed 4+2 audio FEC block state machine (block starts at seq%4==0, encode at (seq+1)%4==0), the Opus multistream encoder integration, the 500 ms ping listener, and the AES-CBC+PKCS7 audio path. These are GameStream-specific and don't exist in punktfunk-core.",
"gotchas": [
"AES-CBC, not GCM. Audio is the one stream using CBC; reusing punktfunk's GCM code verbatim will break interop. No auth tag is on the wire for audio.",
"IV is only 4 meaningful bytes: BE32(avRiKeyId + sequenceNumber) then 12 zero bytes. The addition wraps as uint32. avRiKeyId is per-session from RTSP launch.",
"The Reed-Solomon parity matrix MUST be Nvidia's hardcoded one. moonlight explicitly notes 'the RS parity matrix computed by our RS implementation doesn't match the one Nvidia uses' and overrides it with the 8 OpenFEC bytes {0x77,0x40,0x38,0x0e,0xc7,0xa7,0x0d,0x6c}. A stock reed-solomon-erasure encoder will produce parity the client cannot decode.",
"FEC is a FIXED 4+2 block (RTPA_DATA_SHARDS=4, RTPA_FEC_SHARDS=2), unlike video which uses dynamic/large blocks and multi-FEC. Block boundaries must align to seq%4==0 or the client's queue logic rejects them.",
"FEC packets carry their OWN incrementing rtp.sequenceNumber = baseSeq + shardIndex + 1, distinct from data packet seq space conceptually but in the same 16-bit counter — get the +x+1 right.",
"FEC parity is computed AFTER encryption+padding (over ciphertext shards). All shards must be padded to the same blockSize before encode, and parity packets carry blockSize bytes.",
"Port offset is +10 (Sunshine), but the actual audio port is negotiated in RTSP SETUP (AudioPortNumber). Don't hardcode 48000 — 48000 is the CONTROL port (+11) under the default base; audio is +10 (47999 default). Confirm against your base port.",
"timestamp increments by packetDuration in ms units (5 or 10), not by sample count — Sunshine: timestamp += packetDuration.",
"samplesPerFrame = 48 * AudioPacketDuration → 240 samples/ch at 5 ms default, 480 at 10 ms. Opus must be configured to encode exactly this frame size.",
"Surround Opus uses multistream with specific stream/coupled counts (5.1: 4 streams/2 coupled normal or 6/0 high; 7.1: 5/3 normal or 8/0 high) — wrong stream layout makes the client's multistream decoder produce garbage. Channel mapping is FL,FR,FC,LFE,RL,RR,SL,SR (indices 0..7).",
"rtp.header byte is 0x80 (RTP v2); ssrc=0 in Sunshine. Match these or some clients may drop packets.",
"Client sends 500 ms pings to the audio port; host must read pings to discover the client's UDP source address before sending audio (one-way send relies on the learned addr)."
],
"sources": [
"moonlight-common-c/src/AudioStream.c — packetType==97 check, AES-CBC IV ivSeq=BE32(avRiKeyId+rtp->sequenceNumber), samplesPerFrame=48*AudioPacketDuration, chosenConfig=High/NormalQualityOpusConfig, SET_PORT(&saddr, AudioPortNumber), 500ms ping, MAX_PACKET_SIZE 1400, QUEUED_AUDIO_PACKET",
"moonlight-common-c/src/RtpAudioQueue.c — reed_solomon_new(RTPA_DATA_SHARDS, RTPA_FEC_SHARDS), hardcoded OpenFEC parity matrix {0x77,0x40,0x38,0x0e,0xc7,0xa7,0x0d,0x6c} ('doesn't match the one Nvidia uses'), reed_solomon_decode(rs, shards, marks, RTPA_TOTAL_SHARDS, blockSize), FEC block boundary on RTPA_DATA_SHARDS",
"moonlight-common-c/src/RtpAudioQueue.h — RTPA_DATA_SHARDS=4, RTPA_FEC_SHARDS=2, RTPA_TOTAL_SHARDS=6, AUDIO_FEC_HEADER {uint8 fecShardIndex; uint8 payloadType; uint16 baseSequenceNumber; uint32 baseTimestamp; uint32 ssrc;}, RTPA_FEC_BLOCK, RTP_AUDIO_QUEUE",
"moonlight-common-c/src/Limelight.h — MAKE_AUDIO_CONFIGURATION, AUDIO_CONFIGURATION_STEREO/51/71, CHANNEL_COUNT_FROM_AUDIO_CONFIGURATION, OPUS_MULTISTREAM_CONFIGURATION {sampleRate,channelCount,streams,coupledStreams,samplesPerFrame,mapping[8]}, remoteInputAesKey[16]/remoteInputAesIv[16]",
"moonlight-common-c/src/Limelight-internal.h — extern NormalQualityOpusConfig/HighQualityOpusConfig, extern int AudioPacketDuration, extern SS_PING AudioPingPayload",
"moonlight-common-c/src/SdpGenerator.c (via search) — AudioPacketDuration default 5 ms ('Use 5 ms packets by default for lowest latency'), 10 ms fallback",
"Sunshine/src/audio.cpp — SAMPLE_RATE=48000, opus_stream_config_t stream_configs[]: STEREO 2ch/1str/1coupled/96000bps, HIGH_STEREO 2/1/1/512000, SURROUND51 6/4/2/256000, HIGH_SURROUND51 6/6/0/1536000, SURROUND71 8/5/3/450000, HIGH_SURROUND71 8/8/0/2048000; opus_multistream_encode_float; buffer 1400",
"Sunshine/src/audio.h — opus_stream_config_t {int32 sampleRate;int channelCount;int streams;int coupledStreams;const uint8* mapping;int bitrate;}, stream_params_t, config_t {packetDuration,channels,mask,...}, enum stream_config_e {STEREO,HIGH_STEREO,SURROUND51,HIGH_SURROUND51,SURROUND71,HIGH_SURROUND71,MAX_STREAM_CONFIG}",
"Sunshine/src/stream.cpp — audioBroadcastThread: audio_packet.rtp.header=0x80/.packetType=97/.ssrc=0/.sequenceNumber=big(seq)/.timestamp=big(ts); seq++; timestamp+=packetDuration; IV=BE32(avRiKeyId+sequenceNumber); cbc cipher; if seq%4==0 set fecHeader.baseSequenceNumber/baseTimestamp; if (seq+1)%4==0 reed_solomon_encode(rs,shards,RTPA_TOTAL_SHARDS,bytes); fec_packet.rtp.packetType=127/.sequenceNumber=big(seq+x+1); fecHeader.fecShardIndex=x/.payloadType=97; net::map_port(AUDIO_STREAM_PORT); cbc_t{gcm_key,true}; round_to_pkcs7_padded(2048)",
"Sunshine/src/platform/common.h — speaker enum {FRONT_LEFT,FRONT_RIGHT,FRONT_CENTER,LOW_FREQUENCY,BACK_LEFT,BACK_RIGHT,SIDE_LEFT,SIDE_RIGHT}; map_stereo={FL,FR}; map_surround51={FL,FR,FC,LFE,BL,BR}; map_surround71={FL,FR,FC,LFE,BL,BR,SL,SR}",
"DeepWiki LizardByte/Sunshine Network Configuration — base port 47989, offsets: HTTP+0, HTTPS+1, Video+9 (47998), Audio+10 (47999), Control+11 (48000), RTSP+21 (48010); net::map_port validates 1024-65535",
"Sunshine/src/network.h — uint16_t map_port(int port) (maps offset onto base port)"
]
},
{
"area": "GameStream wire-format gap analysis + architecture recommendation for punktfunk-host (P1 / M2)",
"summary": "punktfunk-core today speaks an INTERNAL protocol that is structurally similar to GameStream but byte-incompatible on every wire surface, so a stock Moonlight client cannot connect to it as-is. Differences: (1) punktfunk prefixes each shard with a 40-byte little-endian `PacketHeader` and no RTP layer; GameStream uses a 12-byte big-endian RTP header + 4 reserved bytes + a 16-byte `NV_VIDEO_PACKET` (28 bytes total) carrying frameIndex/streamPacketIndex/flags and the FEC params bit-packed into a single `fecInfo` u32 and `multiFecBlocks` u8. (2) punktfunk's RS-FEC interleaves data+recovery shards within one block keyed by `shard_index`; GameStream packs ALL data shards first then ALL parity shards across a contiguous RTP sequence range, derives (data,parity,fecIndex,pct) from `fecInfo`, splits a frame into up to 4 FEC blocks via `multiFecBlocks`, and the data shards must be the literal RTP-framed bytes of the H.264/HEVC NAL slices (the depacketizer concatenates payloads to rebuild Annex-B). (3) punktfunk seals the whole 40-byte+payload packet under AES-128-GCM with an 8-byte seq prefix and seq-as-AAD; GameStream encrypts only the post-RTP payload, prefixing a `video_packet_enc_prefix_t {iv[12]; u32 frameNumber; u8 tag[16]}` where the IV is an 8-byte little-endian per-stream counter with iv[11]='V'. The RS math itself is identical (ceil(k*pct/100), GF(2^8), <=255 shards) so punktfunk's `reed-solomon` GF(2^8) coder CAN produce Moonlight-recoverable parity, but ONLY if punktfunk abandons its own shard layout and emits shards in GameStream's data-then-parity contiguous order with GameStream's exact shard size (packetSize + 4 reserved + RTP). Beyond video, GameStream needs an entire control plane punktfunk has not started: HTTPS:47984/HTTP:47989 nvhttp pairing (PIN->AES-128 via SHA-256(salt||pin)[..16], ECB challenge exchange, RSA-signed client cert), an RTSP:48010 handshake (OPTIONS/DESCRIBE/SETUP/ANNOUNCE/PLAY) carrying SDP `x-nv-*` params, an ENet control stream (UDP 48000) with its own AES-128-GCM framing and opcodes (request-IDR, loss-stats, ping, HDR, termination, rumble), an AES-CBC audio stream (UDP 47999), and mDNS `_nvstream._tcp` advertisement. Recommendation: put the GameStream video/FEC/crypto wire codec as a P1 \"wire mode\" INSIDE punktfunk-core (the invariant says protocol logic lives in the core), but keep the stateful control plane (nvhttp/RTSP/ENet/pairing/mDNS) in punktfunk-host as a tokio control-plane adapter that calls into core codec functions, because that machinery is I/O-bound, async, and not part of the hot path.",
"ports": [
"TCP 47984 — HTTPS nvhttp (paired control: /serverinfo, /pair, /applist, /launch, /resume, /cancel). Client-cert pinned to the paired client.",
"TCP 47989 — HTTP nvhttp (unpaired: /serverinfo unauthenticated, /pair PIN flow).",
"TCP 48010 — RTSP setup (OPTIONS/DESCRIBE/SETUP/ANNOUNCE/PLAY). Plaintext over TCP, or encrypted_rtsp_header_t {u32 typeAndLength MSB=0x80000000; u32 seq; u8 tag[16]} when encryption negotiated.",
"UDP 47998 — Video RTP stream (NV_VIDEO_PACKET + RS-FEC). ML_PORT_INDEX_UDP_47998=8.",
"UDP 47999 — Audio RTP stream (Opus, AES-CBC, RS-FEC). ML_PORT_INDEX_UDP_47999=9.",
"UDP 48000 — ENet control stream (reliable, AES-128-GCM, opcodes). ML_PORT_INDEX_UDP_48000=10.",
"UDP/mDNS 5353 — _nvstream._tcp.local advertisement so Moonlight auto-discovers the host.",
"Note: Moonlight derives all of these by offset from the HTTP base port (default 47989); changing the base shifts the whole set. punktfunk-host must advertise the actual HttpsPort/ExternalPort in serverinfo XML."
],
"wire_formats": [
{
"name": "DELTA: video packet header (punktfunk vs GameStream)",
"layout": "punktfunk PacketHeader = 40 bytes, little-endian, repr(C): pts_ns u64, frame_index u32, stream_seq u32, frame_bytes u32, user_flags u32, block_index u16, block_count u16, data_shards u16, recovery_shards u16, shard_index u16, shard_bytes u16, magic u8(0xC9), version u8, fec_scheme u8, flags u8. || GameStream on-wire = RTP_PACKET(12, big-endian: u8 header, u8 packetType, u16 sequenceNumber, u32 timestamp, u32 ssrc) + char reserved[4] + NV_VIDEO_PACKET(16, little-endian: u32 streamPacketIndex@0, u32 frameIndex@4, u8 flags@8, u8 extraFlags@9 (NV_VIDEO_PACKET_EXTRA_FLAG_LTR_FRAME=0x1), u8 multiFecFlags@10, u8 multiFecBlocks@11, u32 fecInfo@12) = 28 bytes before payload.",
"notes": "DELTA: drop pts_ns/frame_bytes/shard_bytes from the wire (GameStream carries none of these per-packet); add the RTP header + reserved[4]; replace explicit u16 FEC fields with the bit-packed fecInfo+multiFecBlocks. flags map 1:1 (FLAG_CONTAINS_PIC_DATA=0x1==punktfunk FLAG_PIC, FLAG_EOF=0x2, FLAG_SOF=0x4). frameIndex == punktfunk frame_index; streamPacketIndex == per-stream packet counter (NOT punktfunk stream_seq which is per-AU)."
},
{
"name": "fecInfo bit-packing (GameStream, exact)",
"layout": "fecInfo (u32, little-endian field) = (dataShards << 22) | (fecIndex << 12) | (fecPercentage << 4). Decode masks (moonlight-common-c): dataShards=(fecInfo & 0xFFC00000)>>22 (bits 22-31, 10 bits, <=1023 but RS caps at 255); fecIndex=(fecInfo & 0x3FF000)>>12 (bits 12-21, the shard's index within its block); fecPercentage=(fecInfo & 0xFF0)>>4 (bits 4-11). parityShards = (dataShards*fecPercentage + 99)/100 (ceiling). bits 0-3 unused.",
"notes": "This is IDENTICAL math to punktfunk's FecConfig::recovery_for (ceil(k*pct/100)). punktfunk already computes data_shards/recovery_shards as explicit u16; the only delta is packing them into this bitfield and emitting fecIndex as the contiguous index across [0..data) then [data..data+parity)."
},
{
"name": "multiFecBlocks bit-packing (GameStream, exact)",
"layout": "multiFecBlocks (u8) = (blockIndex << 4) | ((fec_blocks_needed - 1) << 6). Decode: fecCurrentBlockNumber=(multiFecBlocks>>4)&0x3; lastBlockNumber=(multiFecBlocks>>6)&0x3. Max 4 FEC blocks per frame (2 bits each).",
"notes": "DELTA vs punktfunk: punktfunk uses u16 block_index/block_count with no 4-block ceiling. For P1 wire mode, max_data_per_block must be chosen so a frame needs <=4 blocks AND each block <=255 total shards. punktfunk's p1_defaults (max_data_per_block=200, 15% FEC -> 230 total) already respects 255; just cap blocks at 4 for GameStream mode."
},
{
"name": "RS-FEC shard arrangement (the recoverability question)",
"layout": "GameStream: within one FEC block, RTP sequence numbers are contiguous: data shards occupy [bufferLowestSequenceNumber .. bufferFirstParitySequenceNumber-1], parity shards immediately follow. totalPackets = highest-lowest+1 = dataShards+parityShards. Each shard is exactly receiveSize = packetSize + MAX_RTP_HEADER_SIZE bytes, the last data shard zero-padded to receiveSize. Decode: rs=reed_solomon_new(dataShards, parityShards); reed_solomon_decode(rs, packets[], marks[], totalPackets, receiveSize) with marks[i]=1 for missing. Data shards = the RTP-framed bytes of the video payload concatenated; depacketizer strips RTP+NV header and concatenates payloads to rebuild the Annex-B AU.",
"notes": "punktfunk TODAY: shard_index addresses data [0..K) then recovery [K..K+M) within a block, reconstruct() takes received[] of length K+M with None=lost — STRUCTURALLY THE SAME ordering as GameStream's data-then-parity. VERDICT: punktfunk's reed-solomon GF(2^8) coder CAN produce moonlight-recoverable shards, because both use the same Vandermonde/Cauchy RS over GF(2^8) with data-first layout. BUT the byte CONTENT of each data shard must be GameStream's RTP-framed packet bytes (not punktfunk's 40-byte-header packets), and the shard size must be packetSize+RTP, and the parity must be computed over those exact bytes. CAVEAT (unverified at byte level): the specific RS library Moonlight uses (reed-solomon-new / Fec.c, a CM256/Plank-style Cauchy matrix) may use a different generator matrix than the Rust `reed-solomon` crate; parity bytes are only interoperable if the matrices match. This MUST be validated against real Moonlight before trusting it — if matrices differ, punktfunk must port/match Moonlight's Fec.c matrix exactly (this is the single highest-risk interop item)."
},
{
"name": "video AES-GCM crypto (DELTA)",
"layout": "GameStream video_packet_enc_prefix_t = { u8 iv[12]; u32 frameNumber; u8 tag[16] } prepended to the encrypted payload. IV = 8-byte little-endian per-stream gcm_iv_counter in iv[0..8], iv[11]='V' (0x56), iv[8..11]=0; counter increments per packet. Cipher = AES-128-GCM, key = the GCM key from /launch (riKey). Only the post-RTP/post-NV payload is encrypted; RTP+NV header stay in clear. video_short_frame_header_t (8 bytes, inside the encrypted payload, first packet of frame) = { u8 headerType=0x01; le_u16 frame_processing_latency; u8 frameType (1=P,2=IDR,4=intra-refresh,5=after-ref-invalidation); le_u16 lastPayloadLen; u8 unknown[2] }.",
"notes": "DELTA vs punktfunk crypto.rs: punktfunk seals the ENTIRE packet (header+payload) and uses a 4-byte salt + 8-byte big-endian seq nonce with seq as AAD, prefixing an 8-byte seq. GameStream encrypts only payload, uses 8-byte LE counter + 'V' marker (NO AAD), and the prefix carries iv+frameNumber+tag explicitly. punktfunk's per-direction salt-bit trick is a punktfunk invention not on the GameStream wire. For P1 wire mode the core needs a SEPARATE gcm path matching this prefix exactly."
},
{
"name": "ENet control crypto + opcodes (new in host)",
"layout": "Encrypted control: NVCTL_ENCRYPTED_PACKET_HEADER { le_u16 encryptedHeaderType=0x0001; le_u16 length; u32 seq } then [16-byte AES-GCM tag][encrypted V2 header + payload]. Cipher AES-128-GCM (Sunshine SS_ENC_CONTROL_V2): 12-byte LE IV = seq in bytes 0-3, bytes 10-11='CC'. Plain header V2 = { u16 type; u16 payloadLength }. Opcodes (Gen7 plain): 0x0305 Start A, 0x0307 Start B, 0x0301 invalidate-ref-frames, 0x0201 loss-stats, 0x0206 input, 0x010b rumble, 0x0100 termination, 0x010e HDR, 0x0302 request-IDR (encrypted gen). Periodic ping {le_u16 len=4; le_u32 ts}.",
"notes": "Entirely absent from punktfunk. Belongs in punktfunk-host (ENet via a Rust ENet crate); the AES-128-GCM seal/open of the control payload can reuse a core crypto primitive but the framing is host-side."
},
{
"name": "audio packet (new in host)",
"layout": "audio_packet_t = RTP_PACKET (12) + Opus payload. audio_fec_packet_t = RTP_PACKET + AUDIO_FEC_HEADER. Encryption = AES-128-CBC (NOT GCM); IV = big-endian u32(avRiKeyId + sequenceNumber) where avRiKeyId = first 4 bytes of the launch IV. Fixed RTPA_DATA_SHARDS / RTPA_FEC_SHARDS RS-FEC.",
"notes": "Audio is AES-CBC, different from video GCM — a separate codec path. Lower priority for M2 (can stream video-only first; Moonlight tolerates audio coming up after video)."
}
],
"flow": [
"PHASE A (core, low risk): Add a P1 'gamestream wire mode' to punktfunk-core alongside the internal format. New module crates/punktfunk-core/src/protocol/gamestream.rs implementing (a) RTP+reserved+NV_VIDEO_PACKET serialize/parse with exact bit-packing, (b) a GameStream-layout FEC packetizer/reassembler that emits data-then-parity contiguous RTP shards at packetSize+RTP shard size, (c) the video_packet_enc_prefix_t AES-128-GCM path. Gate behind ProtocolPhase::P1GameStream (already exists). Keep punktfunk's internal 40-byte format for P2.",
"PHASE B (validate the FEC matrix — HIGHEST RISK, do early): Before building any host networking, prove byte-for-byte that punktfunk's reed-solomon GF(2^8) parity matches Moonlight's expectation. Capture real Sunshine video packets (or vendor moonlight-common-c's Fec.c into a test) and assert punktfunk-encoded parity is decodable by Moonlight's RS and vice versa. If the generator matrices differ, port Moonlight's Cauchy matrix into punktfunk's gf8 coder. This gates everything: if shards aren't interoperable, P1 is dead.",
"PHASE C (host control plane, in punktfunk-host): Implement nvhttp on TCP 47989 (HTTP) + 47984 (HTTPS): /serverinfo XML (appversion, GfeVersion, uniqueid, HttpsPort, ExternalPort, mac, MaxLumaPixelsHEVC, ServerCodecModeSupport, currentgame, PairStatus, sessionUrl0), the /pair PIN state machine (getservercert -> clientchallenge -> serverchallengeresp -> clientpairingsecret) with PIN-AES = SHA-256(salt||pin)[..16], AES-128-ECB challenge, SHA-256, X.509 + RSA sign/verify. Persist the paired client cert; pin it for HTTPS client-cert auth.",
"PHASE D (RTSP on TCP 48010): OPTIONS/DESCRIBE/SETUP/ANNOUNCE/PLAY. DESCRIBE returns SDP with x-nv-video[0].* , x-nv-vqos[0].fec.* , x-ss-general.* attributes. SETUP returns server_port= per stream. ANNOUNCE parses client's packetSize, fec.minRequiredFecPackets, maximumBitrateKbps, videoEncoderSlicesPerFrame — feed these into the punktfunk-core Config (shard_payload=packetSize, fec_percent, etc).",
"PHASE E (data plane wiring): On PLAY, bind UDP 47998 (video), spawn the M0 capture->NVENC pipeline, and drive punktfunk-core's P1 packetizer to that socket. Bind UDP 48000 ENet control (request-IDR -> force NVENC keyframe; loss-stats -> adjust; termination). Audio (UDP 47999, AES-CBC) and full input can follow.",
"PHASE F (discovery + display): mDNS-advertise _nvstream._tcp. On RTSP SETUP/PLAY, create the wlroots virtual output sized to the negotiated WxH@fps, point M0 capture at it, tear down on RTSP TEARDOWN / ENet termination."
],
"crypto": "VIDEO (P1 wire): AES-128-GCM, key=riKey from /launch (16 bytes). Per-packet prefix video_packet_enc_prefix_t{iv[12],u32 frameNumber,u8 tag[16]}; IV = 8-byte LE per-stream counter in iv[0..8], iv[11]='V'(0x56), no AAD. Only payload encrypted. ||| CONTROL: AES-128-GCM (Sunshine SS_ENC_CONTROL_V2), 12-byte LE IV = seq[0..4], iv[10..12]='CC', 16-byte tag, NVCTL_ENCRYPTED_PACKET_HEADER prefix. ||| AUDIO: AES-128-CBC, IV = BE u32(avRiKeyId + seq), avRiKeyId = first 4 bytes of launch IV. ||| PAIRING (nvhttp): PIN-derived key = first 16 bytes of SHA-256(salt(16) || ascii-pin(4)); AES-128-ECB for the challenge/response blocks; SHA-256 for the rolling hashes; RSA (server key + client cert) for signing/verifying the pairing secret; X.509 certs exchanged (server cert returned in getservercert, client cert pinned for HTTPS). ||| DELTA vs punktfunk crypto.rs: punktfunk uses AES-128-GCM but with a 4-byte random salt + 8-byte BE seq nonce and seq-as-AAD, sealing the WHOLE packet and prefixing 8-byte seq — none of these match GameStream's iv/marker/prefix/no-AAD scheme. punktfunk has NO ECB/CBC, NO RSA/X.509, NO PIN-KDF. So: keep punktfunk's GCM for P2; add a distinct gamestream-gcm path for P1; add ECB+CBC+RSA+X.509+SHA-256-KDF in the host pairing layer (rustls/aws-lc-rs/rsa/x509 crates).",
"rust_options": "FEC: KEEP the existing `reed-solomon` GF(2^8) coder in punktfunk-core for math, but it MUST be validated byte-compatible with Moonlight's Fec.c (CM256/Plank Cauchy matrix) — if not, port that matrix. (reed-solomon-simd is GF(2^16), P2 only, NOT moonlight-compatible.) ||| ENet control: `rusty_enet` (pure-Rust ENet 1.3.x, no_std-friendly, actively maintained) — speaks the exact ENet wire protocol Moonlight expects; alternative is FFI to libenet via `enet-sys`. ||| RTSP: NO good off-the-shelf server crate handles GameStream's non-standard interleaved/encrypted RTSP — hand-roll a minimal parser over a tokio TcpListener (it's ~6 verbs); `httparse`-style manual parsing. Do NOT pull a full RTSP stack. ||| HTTPS with pinned client-cert: `axum`/`hyper` + `rustls` (ServerConfig with a custom `ClientCertVerifier` that checks the cert against the paired set) + `tokio-rustls`; or `actix-web` with rustls. The plan already commits to axum+tokio for the control plane. ||| X.509 gen: `rcgen` (generate the self-signed server cert + key on first run); parse/verify client certs with `x509-parser` + `rsa` + `sha2`. PIN-KDF and ECB/CBC/GCM via `aes`, `aes-gcm`, `cbc`, `ecb` (RustCrypto) or `aws-lc-rs`/`openssl`. ||| mDNS: `mdns-sd` (pure-Rust, registers `_nvstream._tcp.local` with TXT records) or `zeroconf` (FFI to Avahi). `mdns-sd` preferred (no daemon dependency). ||| Opus audio: `audiopus`/`opus` crate if/when audio is implemented.",
"reuse_from_punktfunk": "REUSE: (1) punktfunk-core's GF(2^8) `reed-solomon` coder and its data-then-parity reconstruct() contract — same ordering as GameStream; (2) FecConfig::recovery_for ceil(k*pct/100) — IDENTICAL to Moonlight's parity math; (3) the ReassemblerLimits bounds-before-allocate hardening pattern — reuse the same discipline when parsing attacker-controlled NV_VIDEO_PACKET fields; (4) aes-gcm dependency and crypto.rs structure (the GCM primitive itself, even though nonce/prefix scheme differs); (5) ProtocolPhase::P1GameStream / FecScheme::Gf8 enums already exist as the negotiation hook; (6) punktfunk-host M0's capture->NVENC pipeline produces exactly the HEVC/H264 Annex-B AUs that become GameStream video payload; (7) the Packetizer/Reassembler split is the right shape — add a parallel GameStream packetizer/reassembler beside them. ||| MUST BUILD NEW: the RTP+NV_VIDEO_PACKET (de)serialization with bit-packed fecInfo/multiFecBlocks; the GameStream-layout shard emitter (contiguous data-then-parity, packetSize+RTP shard size, no 40-byte punktfunk header); the video_packet_enc_prefix_t GCM path (iv counter + 'V', payload-only, no AAD); the ENTIRE control plane (nvhttp pairing, RTSP, ENet control, mDNS, X.509/RSA, ECB/CBC); audio AES-CBC path. ||| CANNOT REUSE on the wire: punktfunk's 40-byte PacketHeader, its 8-byte-seq GCM framing, its per-direction salt bit — all are punktfunk-internal inventions absent from GameStream.",
"gotchas": [
"RS GENERATOR MATRIX is the #1 interop risk: same GF(2^8) RS and same data-first ordering does NOT guarantee byte-compatible parity. Moonlight's Fec.c uses a specific Cauchy/Vandermonde matrix; the Rust `reed-solomon` crate may differ. Validate against real Moonlight FIRST (Phase B) or all of P1 fails silently as 'unrecoverable loss'.",
"GameStream RTP header is BIG-endian; punktfunk's PacketHeader is little-endian. NV_VIDEO_PACKET itself is little-endian. Don't conflate them.",
"streamPacketIndex is a per-STREAM monotonic packet counter (the RTP-ish sequence), NOT punktfunk's per-AU stream_seq. frameIndex is the per-frame counter. Two different counters.",
"multiFecBlocks caps a frame at 4 FEC blocks (2 bits). Combined with the 255-shard GF(2^8) cap, large frames at high res can overflow — this IS the 1 Gbps wall the plan describes. P1 must keep frames within 4 blocks x 255 shards; reduce via slicesPerFrame / bitrate from ANNOUNCE.",
"Video GCM uses NO AAD and an 8-byte LE counter + 'V' marker; punktfunk's GCM uses seq-as-AAD + per-direction salt. A naive reuse of punktfunk's seal() will produce undecryptable-by-Moonlight packets. Build the gamestream-gcm path separately.",
"Encryption is NEGOTIATED (ENCFLG_VIDEO=0x2, ENCFLG_AUDIO=0x1 in serverinfo/SETUP). Many Moonlight setups stream video in the CLEAR on LAN — implement plaintext video first, add GCM second; serverinfo's encryptionSupported/Requested controls this.",
"Audio is AES-CBC not GCM, with a BE counter IV — a third distinct crypto scheme. Easy to get wrong if you assume GCM everywhere.",
"Pairing PIN key = SHA-256(salt||pin)[..16] where pin is the 4 ASCII digits and salt is 16 raw bytes from the client — order and encoding matter exactly. ECB (not CBC) for the challenge blocks.",
"The shard size is packetSize + RTP_HEADER (not just packetSize). punktfunk's shard_payload must be set to the negotiated packetSize and the shard the core FEC-protects must include the RTP/NV framing bytes, else the depacketizer mis-aligns.",
"HTTPS endpoint pins the CLIENT cert obtained during pairing; a stock TLS server that accepts any cert will let unpaired clients in. Use a custom rustls ClientCertVerifier.",
"mDNS service name must be exactly _nvstream._tcp with the right TXT records or Moonlight won't auto-discover (manual IP add still works as a fallback for testing)."
],
"sources": [
"/home/enricobuehler/punktfunk/crates/punktfunk-core/src/packet.rs (PacketHeader 40-byte layout, Packetizer/Reassembler, ReassemblerLimits hardening, FLAG_* constants)",
"/home/enricobuehler/punktfunk/crates/punktfunk-core/src/crypto.rs (SessionCrypto AES-128-GCM, 4-byte salt + 8-byte BE seq nonce, seq-as-AAD, per-direction salt bit)",
"/home/enricobuehler/punktfunk/crates/punktfunk-core/src/config.rs (FecConfig::recovery_for ceil(k*pct/100), FecScheme::max_total_shards Gf8=255, ProtocolPhase::P1GameStream, p1_defaults)",
"/home/enricobuehler/punktfunk/crates/punktfunk-core/src/session.rs (seal_for_wire 8-byte seq prefix, submit_frame/poll_frame hot path)",
"/home/enricobuehler/punktfunk/crates/punktfunk-core/src/fec/mod.rs (ErasureCoder trait, data-then-parity reconstruct contract, GF(2^8) Gf8Coder)",
"/home/enricobuehler/punktfunk/crates/punktfunk-host/src/{web.rs,vdisplay.rs,inject.rs,pipeline.rs,m0.rs} (control-plane stub, VirtualDisplay trait + wlroots/kwin/mutter stubs, M0 capture->NVENC->AU pipeline + punktfunk-core loopback)",
"/home/enricobuehler/punktfunk/docs/implementation-plan.md sections 3,5,6,8 (P1/P2/P3 strategy, C ABI, virtual-display orchestration, milestones M0/M2)",
"moonlight-common-c/src/Video.h (NV_VIDEO_PACKET 16-byte struct, RTP_PACKET 12-byte struct, FLAG_CONTAINS_PIC_DATA/EOF/SOF, FIXED_RTP_HEADER_SIZE)",
"moonlight-common-c/src/RtpVideoQueue.c (fecInfo masks 0xFFC00000>>22 / 0x3FF000>>12 / 0xFF0>>4, parity=(data*pct+99)/100, reed_solomon_new/reed_solomon_decode, receiveSize=packetSize+MAX_RTP_HEADER_SIZE, contiguous data-then-parity sequence range, multiFecBlocks>>4&0x3 / >>6&0x3)",
"moonlight-common-c/src/ControlStream.c (ENet channels, opcodes 0x0305/0x0307/0x0301/0x0201/0x010b/0x0100/0x010e/0x0302, NVCTL_ENCRYPTED_PACKET_HEADER, AES-128-GCM control IV seq+'CC', ping/loss-stats/HDR/rumble/termination layouts)",
"moonlight-common-c/src/Limelight.h (ML_PORT_INDEX/FLAG constants 47984/47989/48010/47998/47999/48000, ENCFLG_AUDIO=0x1/VIDEO=0x2, VIDEO_FORMAT_* codec masks, STREAM_CONFIGURATION fields incl remoteInputAesKey/Iv, packetSize)",
"Sunshine/src/stream.cpp (video_packet_raw_t = RTP+reserved[4]+NV_VIDEO_PACKET, fecInfo send packing x<<12|data<<22|pct<<4, multiFecBlocks (block<<4)|((n-1)<<6), video_short_frame_header_t, video_packet_enc_prefix_t iv[12]+frameNumber+tag[16], IV 8-byte counter + iv[11]='V', audio AES-CBC IV=BE(avRiKeyId+seq), CONTROL/VIDEO/AUDIO_STREAM_PORT via map_port)",
"Sunshine/src/rtsp.cpp (OPTIONS/DESCRIBE/SETUP/ANNOUNCE/PLAY flow, server_port= response, SDP x-nv-video/x-nv-vqos/x-ss-general attributes, encrypted_rtsp_header_t MSB 0x80000000, RTSP_SETUP_PORT default TCP 48010, ANNOUNCE carries packetSize/fec/bitrate/slicesPerFrame)",
"Sunshine/src/nvhttp.cpp (endpoints /serverinfo /pair /applist /launch /resume /cancel, HTTPS 47984 / HTTP 47989, pairing state machine getservercert/clientchallenge/serverchallengeresp/clientpairingsecret, serverinfo XML fields)",
"Sunshine/src/crypto.cpp (gen_aes_key = SHA-256(salt||pin) truncated to 16 bytes, AES-128 ECB/GCM/CBC modes)",
"Moonlight/Sunshine port documentation (TCP 47984/47989/48010, UDP 47998-48000/48010) — moonlight-stream wiki and portforward.com (port roles confirmation)"
]
}
]
}