loader: reject corrupt NSO modules that fail to decompress by Fernandess07 · Pull Request #863 · Eglantine-Community/Eglantine

Fernandess07 · 2026-11-14T00:21:13Z

I have read and followed the Contribution Guidelines.
I have read and followed the AI Policy.
I have read and followed the Coding Guidelines to the best of my ability.
I have put [HUMANMADE] in the title if the pr was human-mande

Summary

Previously, when an NSO module failed to decompress, the emulator got
stuck in an endless loading loop, leaving the user unable to tell
whether the problem was the emulator or the dump.

The two ASSERTs were replaced by a check that returns a new error,
ErrorCorruptNSO, which is shown to the user and written to the log.

The NSO header stores the SHA-256 of each segment, but the emulator
never compared it. It now does, which helps identify the problem. A
mismatch only logs a warning, so games are not prevented from running
when the hash does not match.

Assisted-by: Claude Opus 5.5 (diagnosis of the corrupt dump, implementation and testing)

Type of change

Related issues

Related to #70

Testing

Games tested:

Corrupt Pokopia dump (0400A3A01EE4C000): previously stuck in an endless loading loop, now fails with an error (checked in both the Qt frontend and eden-cli).

Checklist

I built it locally and it compiles without new warnings
I did not add any copyrighted game data, keys or firmware
Docs updated if needed
I added AI Disclosure if one was used

Screenshots / logs

Before (UI stuck on "Launching...", only this in the log):
Debug <Critical> core/loader/nso.cpp:125: assert r == int(nso_header.segments[i].size)

After:
Loader <Error> core/loader/nso.cpp:111: NSO module 'main' is corrupt: .text failed LZ4 decompression
Core <Critical> core/core.cpp:406:Load: Failed to load ROM (Error 70)!

Bad segment hash only (loading continues):
Loader <Warning> core/loader/nso.cpp:152:LoadModule: NSO module 'main': .text does not match its SHA-256

Summary by CodeRabbit

Bug Fixes
- Improved error reporting when a game module is corrupt or cannot be decompressed, replacing a generic loading error with a more specific message.
- Improved validation of decompressed module data to help prevent invalid or incomplete content from being loaded.

github-actions · 2026-11-14T00:21:26Z

Retargeted to nx2-experimental for pre-build testing. It will be promoted to nx2 after validation.

coderabbitai · 2026-11-14T00:21:33Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: defaults
Review profile: CHILL
Plan: Advanced
Run ID: 4708e6ae-60d0-464a-9fe2-482fa5103042

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

Trigger review

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

aidankinzett · 2026-11-14T05:16:58Z

                        std::span<const u8>{compressed_data}.first(nso_header.segments_compressed_size[i])
                    );
-                    ASSERT(r > 0);
+                    if (r <= 0) {


This only catches outright zstd errors. ZSTD_decompress can succeed and write fewer bytes than the segment size, and the rest of decompressed_size (left over from the previous segment) then gets copied in. Could this match the LZ4 branch?

if (r != int(nso_header.segments[i].size)) {

Also, DecompressDataZBIC returns -1 when dst is empty, so a compressed 0-byte segment (e.g. an empty .data) would now fail the load, where before the assert only logged. Probably worth skipping decompression when segments[i].size == 0.

Changed the check to match the LZ4 branch and skipped size-0 segments

aidankinzett · 2026-11-14T05:16:58Z

+            return std::nullopt;
+        };
        for (std::size_t i = 0; i < nso_header.segments.size(); ++i) {
            nso_file.Read(compressed_data.data(), nso_header.segments_compressed_size[i], nso_header.segments[i].offset);


The return value of Read is ignored, so a truncated dump (probably the most common kind of corruption) shows up as a decompression failure or as stale bytes from the previous segment. Since this PR is about corrupt dumps, could this be checked as well?

if (nso_file.Read(compressed_data.data(), nso_header.segments_compressed_size[i], nso_header.segments[i].offset) != nso_header.segments_compressed_size[i]) { return corrupt(i, "is truncated"); }

Did what you suggested, tested with an rtld cut by 200 bytes and the log shows .data is truncated.

aidankinzett · 2026-11-14T05:16:59Z

+            if (nso_header.IsSegmentHashChecked(i)) {
+                NSOHeader::SHA256Hash hash{};
+                unsigned int hash_size = 0;
+                EVP_Digest(segment_data, nso_header.segments[i].size, hash.data(), &hash_size,


For an uncompressed segment segment_data points into compressed_data, which is sized to the largest compressed segment, but this hashes segments[i].size bytes. A corrupt header with size > segments_compressed_size[i] reads past the heap buffer here. The uncompressed memcpy below already has this bug, but this adds a second over-read. Could uncompressed segments where segments_compressed_size[i] != segments[i].size be rejected via corrupt()?

Minor: EVP_Digest's return value is ignored. That's harmless, since a failure only gives a false warning, but it's easy to check.

Rejected uncompressed segments with mismatched sizes and now check EVP_Digest

aidankinzett · 2026-11-14T05:16:59Z

    "Integrity verification failed.",
    "The NUP container is malformed or its TOC hash does not match.",
    "The NUP container does not contain a loadable base package.",
+    "An ExeFS module is corrupt and could not be decompressed. The game needs to be redumped.",


Nit: this is also shown when a standalone .nso is loaded through AppLoader_NSO::Load, which isn't ExeFS. Something like "An executable module (NSO) is corrupt…" would cover both.

Changed the message to the one you suggested, and "decompressed" to "loaded" since it now also covers truncated segments.

aidankinzett · 2026-11-14T05:16:59Z

                                           std::vector<Core::NCE::Patcher>* patches = nullptr,
-                                           s32 patch_index = -1);
+                                           s32 patch_index = -1,
+                                           ResultStatus* out_status = nullptr);


Nit, not a blocker: this is a tenth parameter after several defaulted ones. Returning the status (e.g. std::expected<VAddr, ResultStatus>) would avoid the out-pointer and the ResultStatus module_status = ErrorLoadingNSO; setup at each call site.

Didn't do it in this PR to keep the diff small, but I can do it in a separate one.

…ck (Ocarina of Time foliage shadows) (Eglantine-Community#863) * vulkan, shader: draw points and lines through the mesh compute fallback The compute fallback (Eglantine-Community#569) only took mesh programs that output triangles, so programs with point or line topology were dropped on devices without VK_EXT_mesh_shader. Ocarina of Time draws its foliage shadow map (1600x1600 D32) with a point mesh program, so on those devices the map stayed empty and grass and bushes cast no shadow. The fallback now keeps PrimitiveVertices() indices per primitive (map bytes 4 + k * p, as the native epilogue reads them), draws a point, line or triangle list, and gives points the fixed state point size unless the program writes its own. A repeated index is not degenerate for points and lines, so slots past the primitive count get MESH_EMULATION_UNUSED_VERTEX and the vertex shader culls them with a negative clip distance. Moving them outside the view volume was not enough on RADV (RX 5700 XT): some of them still reached the shadow map, and the far field beyond it came out fully shadowed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SDzM9hQaYmCiLmPrj79xs8 * vulkan, shader: document the functions the point and line fallback touches Doc comments for ComputeMeshEpilogue, EmitMeshEmulationVertex, GraphicsPipeline::ConfigureDraw, GraphicsPipeline::MakePipeline, RasterizerVulkan::ReadMeshEmulationProbe and PrepareDraw. No code change. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SDzM9hQaYmCiLmPrj79xs8 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Previously, when an NSO module failed to decompress, the emulator got stuck in an endless loading loop, leaving the user unable to tell whether the problem was the emulator or the dump. The two ASSERTs were replaced by a check that returns a new error, ErrorCorruptNSO, which is shown to the user and written to the log. The NSO header stores the SHA-256 of each segment, but the emulator never compared it. It now does, which helps identify the problem. A mismatch only logs a warning, so games are not prevented from running when the hash does not match. Assisted-by: Claude Opus 5.5 (diagnosis of the corrupt dump, implementation and testing) Signed-off-by: Pedro Fernandes <pedrocufe05@gmail.com>

github-actions Bot changed the base branch from nx2 to nx2-experimental November 14, 2026 00:21

Fernandess07 force-pushed the loader/nso-integrity-check branch from 997d1c9 to 09e8ca1 Compare November 14, 2026 00:29

aidankinzett requested changes Nov 14, 2026

View reviewed changes

Fernandess07 force-pushed the loader/nso-integrity-check branch from 09e8ca1 to 555dbc2 Compare November 14, 2026 13:52

Fernandess07 requested a review from aidankinzett November 14, 2026 14:09

PastaLaPate added this to Release Tracker Nov 14, 2026

github-project-automation Bot moved this to Backlog in Release Tracker Nov 14, 2026

aidankinzett self-assigned this Nov 14, 2026

Navigation Menu

Repository navigation

loader: reject corrupt NSO modules that fail to decompress - #863

loader: reject corrupt NSO modules that fail to decompress#863
Fernandess07 wants to merge 1 commit into
Eglantine-Community:nx2-experimentalfrom
Fernandess07:loader/nso-integrity-check

Fernandess07 commented Nov 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 14, 2026

Uh oh!

coderabbitai Bot commented Nov 14, 2026 •

edited

Loading

Review skipped

Uh oh!

aidankinzett Nov 14, 2026

Uh oh!

Fernandess07 Nov 14, 2026

Uh oh!

aidankinzett Nov 14, 2026

Uh oh!

Fernandess07 Nov 14, 2026

Uh oh!

aidankinzett Nov 14, 2026

Uh oh!

Fernandess07 Nov 14, 2026

Uh oh!

aidankinzett Nov 14, 2026

Uh oh!

Fernandess07 Nov 14, 2026

Uh oh!

aidankinzett Nov 14, 2026

Uh oh!

Fernandess07 Nov 14, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Fernandess07 commented Nov 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of change

Related issues

Testing

Checklist

Screenshots / logs

Summary by CodeRabbit

Uh oh!

github-actions Bot commented Nov 14, 2026

Uh oh!

coderabbitai Bot commented Nov 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Fernandess07 Nov 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Fernandess07 commented Nov 14, 2026 •

edited

Loading

coderabbitai Bot commented Nov 14, 2026 •

edited

Loading

Fernandess07 Nov 14, 2026 •

edited

Loading