Repository navigation
packing/uepak: read version 9 frozen pak indices - #2476
Open
YoungTreeTree wants to merge 2 commits into
Open
YoungTreeTree wants to merge 2 commits into
YoungTreeTree wants to merge 2 commits into
Conversation
Pak v9 moves the footer magic by one byte and can store its index as a frozen UE4 memory image. The reader previously skipped these archives, leaving Demon Slayer's shader split empty. Read the FrozenIndex flag and bounded relative pointers, UTF-16 strings, entry arrays and sparse directory maps. Preserve the serialized v9 index path and recognize encrypted frozen mounts during key discovery. Classify audio-only paks separately from shader archives after reading the index. Synthetic tests cover both layouts, sparse holes, backward pointers, multi-block compression, encrypted indices and malformed references. Both real game indices validate, all 57,982 shaders build, and an M2 Pro run reaches the Simplified Chinese title menu. Combat is not tested. Related to seedds#2134 and seedds#1618. Co-authored-by: YoungTreeTree <24680817+YoungTreeTree@users.noreply.github.com>
The CI clang-tidy run reports unchecked nullable Metal resources in graphics tests. Check their buffers, views and shader functions before use. Initialize the swap chain with valid enum values and check its creation result before reading the returned images. Check the wide-line fixture's file paths and invoke its Python translator with arguments through NSTask. Retain the sparse-copy test's linear command records until submission, then free them, matching the production queue's ownership. Regenerate the shrinking lint baseline with LLVM 23.1.4 so it no longer lists resolved findings. All seven affected GPU tests pass on an M2 Pro with Xcode 27.2. The four additional tests also pass with Metal API validation enabled. The 63 pak tests, documentation tests and pinned Python lint ratchet pass as well.
YoungTreeTree
force-pushed
the
fix/uepak-v9-frozen-index
branch
from
November 14, 2026 15:48
0646558 to
f694d4d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Pak v9 archives were skipped because their footer adds a FrozenIndex byte and moves the magic to 0xcd bytes from EOF. When that flag is set, accepting the version alone is insufficient: the index is a frozen UE4 memory image.
This reads bounded relative pointers, UTF-16 strings, FPakEntry arrays and nested sparse directory maps, retaining entry-relative compression blocks and the existing serialized-index path when FrozenIndex is clear. Encrypted-index key discovery checks the pointed-to mount. Shader discovery reads the directory before distinguishing inline/global-cache paks, shared shader archives and audio-only paks.
This addresses the Pak parsing/extraction failure reported in #2134. Related: #1618 handles ordinary v9 indices but explicitly rejects frozen indices; the Demon Slayer archives tested here use FrozenIndex=1.
Finding: Unreal Engine frozen pak indices (version 9).
Checked
ce370e1base): all 63test_uepak*.pytests (including eight new v9 tests), all 6 documentation tests, and the pinned Python lint ratchet pass (zero new findings).make -C dm/gfx/tests -j2 GAME=0100309016E7A000 runpassed.0100309016E7A000, 1.53 / v589824. Confirmed the Simplified Chinese autosave/new-save prompts and reached the title menu with the character-pack notification. Combat, full levels, long-term stability and iOS were not tested. The game remains on a provisional build profile; this does not add it to the supported-games table.The real-game build/run, GPU tests and regression report below were collected on
547c5dbplus this parser change before rebasing onto034e91a. Upstream did not change the pak reader in between; parser/documentation/lint checks were rerun after rebasing. Runtime was not rerun against the newer host.Regression report
scripts/regress.py check --games 0100309016E7A000: PASS, full mode, 324.7 seconds.Baseline
547c5db647a2fc3880661d1357041d4dd1aae371; candidate the same revision plus this patch.0100309016E7A000This was the only local game build available and directly exercises frozen v9 paks. Other games are not runtime-validated; existing v4/v8/v11 synthetic tests cover their parser paths. There is no registered smoke route for this provisional profile, so the report's boot/performance fields are empty; the manual menu observation above is separate. The harness compares the prepared shader package, while the real-archive and unit checks validate extraction.
CI checks
The branch is based on
ce370e1. The save-sync runner-model check now comes from upstream; this PR no longer changes that test.A separate test-only commit addresses the clang-tidy failures in the graphics tests: check nullable Metal resources before use; initialize and check the Vulkan swap chain; check the wide-line fixture paths and run its translator through
NSTask; and free the sparse-copy test's linear command records after submission. The full LLVM run also regenerates the shrinking baseline to remove resolved findings. Runtime and shader-chain code are unchanged by this CI commit.t_res_commit(310 checks),t_upload_src(41),t_zero_depth(56),t_sparse_copy(43),t_vk_swapchain_life(141),t_wline(22), andt_pfv(zero failed). The latter four also ran with Metal API validation enabled.lint_c.py tidy --update; no checks were disabled or findings added. Final check:python3 dm/tools/lint_c.py tidy -j 6 --against ce370e1.The real-game and shader regression evidence above predates this rebase. No new game-runtime or iOS validation is claimed for the updated base.
Docs
python3 dm/tools/docs_symptoms.pyrun.