Skip to content

Read Unreal pak version 9 (Monster Jam Steel Titans 2 shaders) - #1618

Open
528z4b2kgc-ux wants to merge 1 commit into
seedds:mainfrom
528z4b2kgc-ux:fix/pak-version-9
Open

528z4b2kgc-ux wants to merge 1 commit into
seedds:mainfrom
528z4b2kgc-ux:fix/pak-version-9

Conversation

@528z4b2kgc-ux

@528z4b2kgc-ux 528z4b2kgc-ux commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

What and why

Unreal Engine 4.25 writes version 9 paks (PakFile_Version_FrozenIndex). Their footer has a frozen-index flag byte between the SHA-1 and the compression method names, so the magic is 0xcd bytes from the end, and uepak.py never found it. unreal_paks therefore skipped the paks, so neither the archive decoder (uepak.job) nor the shader split saw them. Past the footer the pak is version 8's.

uepak.py reads version 9's footer at 0xcd and its index as version 8's (PER_FILE = LEGACY + (9,)), and refuses a frozen index with a reason. Unlike 4 and 8, a version 9 pak is not taken as inline outright: inline_shaders reads its file names. Monster Jam's pakchunk0 holds the two .ushaderbytecode archives and goes to uepak.job; pakchunk1 holds only cooked packages and goes to ue_legacy_jobs. Its indexes are encrypted; since 050580d they are opened with main's key through uepak.Keys like versions 4 and 8.

Rebased onto current main (Nov 12): this PR's earlier key handling (pak_key, main= parameters in inline_shaders, unreal_paks, ue_legacy_jobs and profile_guess.py) is gone, since main now does that for every legacy pak. What's left is the version 9 footer and index.

Reference entry: Unreal Engine 4.25 paks (version 9).

Checked

  • Tests: new dm/tools/tests/test_uepak_v9.py (synthetic paks and a synthetic main, no game files or keys): 6 tests (footer, frozen index refused, Zlib and stored files read, a pak with archives not inline, encrypted index opened with the key from main, key carried in the jobs), all of which fail on main. The test_uepak_* tests (60), test_docs, test_profile_guess and test_nsp_* (426) pass. ruff 0.17.2 is clean on the changed files; pyright not run locally.
  • Game files: Monster Jam Steel Titans 2 (010051B0131F0000, 1.0.2), its two paks (4.7 GB) with this branch: main finds neither; with the change pakchunk0 is an archive pak (44,512 + 1,722 records, the 44,148 + 1,717 unique shaders of the game's build) and pakchunk1 an inline one. The game played on the Mac and an iPhone with those shaders, with the earlier version of this change.
  • Devices: M5 Pro, macOS 27.2; iPhone (iPhone19,4), iOS 27.2.
  • Not checked: a full NSP build on this branch (the archives' Oodle decode); regress.py on other Unreal games (no builds on hand). Version 4, 8 and 11 paks take the same code paths as before.

Docs

  • Reference entry added and docs_symptoms.py run
  • No game files, keys or extracted game data in the change

🤖 Generated with Claude Code

@github-actions github-actions Bot added packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers tools Tools and scripts (dm/tools, scripts) docs Documentation labels Nov 7, 2026
@528z4b2kgc-ux
528z4b2kgc-ux force-pushed the fix/pak-version-9 branch 2 times, most recently from 11a8df3 to 0dc7522 Compare November 12, 2026 21:37
@528z4b2kgc-ux 528z4b2kgc-ux changed the title Read Unreal pak version 9, with encrypted indexes opened by main's key (Monster Jam Steel Titans 2 shaders) Read Unreal pak version 9 (Monster Jam Steel Titans 2 shaders) Nov 12, 2026
@528z4b2kgc-ux

Copy link
Copy Markdown
Contributor Author

The two red jobs (host tests and sanitizers) are the same single check in t_save_sync.c: "both changed: the report says conflict, with the cloud's device". That run started before dd0f2ef landed, so it still had the old test, which expects a "Mac" prefix where the runner reports VirtualMac2,1. This PR only touches uepak.py, its test and two docs. A re-run against current main should come back green.

…rs are found

Unreal Engine 4.25 writes version 9 paks (PakFile_Version_FrozenIndex). Their footer has a u8 frozen index flag between the SHA-1 and the five compression method names, so
the magic is 0xcd bytes from the end; uepak.py looked only at 0x2c, 0xc8 and 0xcc, so unreal_paks skipped them and neither the archive decoder nor the shader split saw them.
Past the footer the pak is version 8's (per-file index, methods named in the footer).

uepak.py reads version 9's footer at 0xcd and its index as version 8's (PER_FILE = LEGACY + (9,)), and refuses a frozen index with a reason. Unlike 4 and 8, a version 9 pak
is not taken as inline outright: inline_shaders reads its file names, so Monster Jam's pakchunk0 (two .ushaderbytecode archives) goes to uepak.job and pakchunk1 (cooked
packages only) to ue_legacy_jobs. An encrypted index (Monster Jam's are) is opened with main's key through uepak.Keys, as for versions 4 and 8.

Checked: dm/tools/tests/test_uepak_v9.py (new; synthetic paks and main, no game files or keys; 6 tests, all fail on main); the uepak tests (60), test_docs, test_profile_guess
and test_nsp_* (426) pass; ruff 0.17.2 clean on the changed files (pyright not run locally). Monster Jam Steel Titans 2 1.0.2's two paks: main finds neither; with this change
pakchunk0 is an archive pak (44,512 + 1,722 records, the 44,148 + 1,717 unique shaders of the game's build) and pakchunk1 an inline one. The game played on the Mac and an
iPhone with these shaders (with the earlier version of this change).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
YoungTreeTree added a commit to YoungTreeTree/nxnative that referenced this pull request Nov 14, 2026
Pak v9 moves the footer magic by one byte and can store its index as a
frozen UE4 memory image. The reader previously skipped these archives,
leaving Demon Slayer's shader split empty.

Read the FrozenIndex flag and bounded relative pointers, UTF-16 strings,
entry arrays and sparse directory maps. Preserve the serialized v9 index
path and recognize encrypted frozen mounts during key discovery. Classify
audio-only paks separately from shader archives after reading the index.

Synthetic tests cover both layouts, sparse holes, backward pointers,
multi-block compression, encrypted indices and malformed references.
Both real game indices validate, all 57,982 shaders build, and an M2 Pro
run reaches the Simplified Chinese title menu. Combat is not tested.

Related to seedds#2134 and seedds#1618.

Co-authored-by: YoungTreeTree <24680817+YoungTreeTree@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers tools Tools and scripts (dm/tools, scripts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant