Skip to content

Latest commit

 

History

History
587 lines (491 loc) · 41 KB

UNITY_ASR_FOLLOWUP_PLAN.md

File metadata and controls

587 lines (491 loc) · 41 KB

ASR follow-up: October 2026 porting plan

Status: steps 0–6 completed on 2026-10-04. The planned ASR follow-up is finished. This is a finite follow-up to the completed Unity migration, not a reopening of its optimization work.

Comparison and findings

  • Previous ASR cutoff, exclusive: cf732d3, the merge of #160.
  • New ASR cutoff, inclusive: ce7a966, the merge of #181, fetched from upstream master on 2026-10-04.
  • SplitScript inspected: c4bcede6863ddfa31b92c7d3ef77fa751aa31d4c.
  • The ASR range contains 89 commits and 10 merged PRs, all authored by ero-qt, with maintainer contributions during review. It changes 37 files. #178 is squash-merged; counting only merge commits would miss it.
  • Research used the final source and individual PR diffs in the isolated, ignored target/asr-followup-review checkout, plus GitHub PR descriptions and reviews. The dirty C:\Projekte\asr branch was not modified.

The work falls into four substantive areas: current IL2CPP layouts and discovery, Mono profiles replacing version families, native scene-manager layouts, and scanner correctness/capabilities. Nested managed values and snapshots already exist and must keep working. PR #172 is largely already covered by SplitScript; PR #181's central scanner-sharing improvement is also already present here.

No builds, games, or runtime test suites were run for this planning task. Statements about existing support below come from source and maintained fixtures, not a fresh execution result. ASR's reported player coverage and Rust Wasm size reductions are upstream evidence, not measurements of SplitScript.

PR inventory

PR / merged date Upstream change SplitScript disposition
#163, Sep 19 Adds six IL2CPP profiles at 2019.1, 2020.2 and 6000.6 layout boundaries; makes fixtures independent of the table being tested. Missing boundary coverage. Import the final #167 catalog directly; do not introduce #163's intermediate catalog first. Keep independently laid-out test memory.
#165, Sep 27 Measured scene-manager builds; engine-in-executable fallback; correct loaded-scene array; inline paths; root traversal order; indirect managed references; accurate component terminology. Substantial port required. SplitScript has the high-level scene API but still uses fixed offsets and old representations. Retain its typed snapshots and component<T>().
#166, Oct 2 Replaces Mono Version and fallback offset tables with complete public profiles, exact identity then nearest-version selection, and executable version fallback. Renames field record alignment to stride. Replace MonoVersion and family-based selectors/layouts. Existing binary identity readers and assembly discovery can be reused. SplitScript already calls the field distance fieldStride; update the importer from upstream field.alignment to field.stride.
#178, Oct 3 Fixes previous-page tail copying, short final-page panic, missed unaligned-boundary matches, and matches fabricated before the requested range. Different scanner implementation: there is no carried-tail copy to patch. Transfer the regression cases and check page-failure isolation in SplitScript's overlapping reads.
#167, Oct 3 Reduces IL2CPP to eight actual layout starts at each width, adds early 2018 and 2022.2 coverage, selects by patch, rejects versions below 2018.1. Replace the existing 22-entry catalog with the final 16 entries and replace exact-four-part/major-minor fallback selection. Its temporary 2018.1 scan signatures were subsequently removed by #179.
#179, Oct 3 Finds IL2CPP globals from exports using bounded x86/x64 instruction walks and x64 exception-directory function bounds. Removes global-metadata.dat discovery. Major missing attachment improvement. Reuse Module.peExport; replace the old scan route completely once the new route passes.
#172, Oct 3 Resolves old IL2CPP plain classes/value types through the type table, fixing parallel-array dictionaries; index zero is valid. ASR still does not resolve newer plain definition handles here. Already covered, including newer handles, by UnityIl2CppTypeTable and collection adapters. Add/retain an explicit 2018.4 dictionary regression when changing catalogs; do not replace the stronger implementation.
#173, Oct 3 Scene catalog keyed to actual patch starts; patch-aware selection across release series; x86 signature/layout corrections; persistent scene absent in 5.6.0–5.6.1. Import final scene data directly. Add typed persistent-scene absence and preserve read errors separately. Do not copy #165's superseded same-series fallback.
#175, Oct 3 Correct Windows Mono patch starts, including the 2018.1 layout reversion and old 2017.1 BDWGC class layout. Windows rejects versions before its first compatible sample; Unix retains oldest-sample fallback. Port together with #166. Library, platform and pointer-width isolation are essential; a version alone cannot select a profile.
#181, Oct 4 One scanner core across signature lengths; 64 KiB reads with page fallback; rare-byte anchors and word/SIMD comparisons; arbitrary bit masks; scanning already-read slices. Core sharing already exists. Port relevant correctness and missing capabilities; evaluate its performance techniques separately against SplitScript's cooperative scanning and scratch model. Do not promise ASR's size savings here.

Current gaps and intended design

1. Profiles: use the final catalogs and remove superseded selection

The importers still pin cf732d3. scripts/import-il2cpp-profiles.py assumes 22 entries and generates four-component exact matches followed by major/minor fallback. scripts/import-mono-profiles.py parses the now-deleted version tables and old record shape. Merely changing the revision string cannot work.

The final ASR catalogs contain:

Catalog Entries Selection facts
Windows IL2CPP 16 Eight starts at both widths: 2018.1.0, 2018.2.0, 2018.3.0, 2019.1.0, 2020.2.0, 2022.2.0, 6000.5.0, 6000.6.0.
Windows Mono 14 mono.dll: 5.0.0, 2017.4.6, 2018.1.0, 2018.1.7. BDWGC: 2017.1.0, 2017.2.0, 2021.2.0. Both widths.
Linux Mono 4 5.6.7 and 2017.4.40 for Mono; 2017.3.0 and 2021.2.20 for BDWGC, all x64.
macOS Mono 2 identities 6000.5.10 x64/ARM64 identities sharing one BDWGC layout.
Windows scenes 29 Width-specific layout starts from 5.6.0 onward, including patch-level changes and repeated layouts.

Selection compares (major, minor, patch) and ignores the fourth file-version component. Windows IL2CPP, scenes and Mono return unsupported below their first compatible entry. Linux/macOS Mono still use their oldest compatible sample when no sample precedes the game. Mono exact identity matching remains first, including the existing Linux runtime/player build-ID precedence. Non-Windows samples are not promises of measured transition boundaries.

Recommended public shape: a complete MonoProfile, analogous to Il2CppProfile, with pointer width, runtime library and metadata layout facts. Keep the existing separate provider roots Unity.mono(profile), Unity.monoLinux(profile) and Unity.monoMac(profile) so explicit platform selection excludes other discovery paths. Provide platform-specific measured constructors and custom descriptors. No grammar change is needed. Replace family arguments everywhere, including fixtures, generated catalog contracts, tooling examples and docs. Remove MonoVersion, old family factories, old detection probes and stored version tags; old-vtable behavior follows runtime-library/profile facts instead.

The IL2CPP constructor names also change. Update all callers to the corresponding measured boundary profile; Lunistice's 2022.3 layout maps to 2022.2.0. This should be a profile-name change only, with unchanged game fields, aliases, edition selection and timer behavior. Preserve an original-source comparison artifact through the change so source-fingerprint churn cannot hide a size regression. No compatibility aliases, migration diagnostics or migration guide are needed.

Supplemental storage facts (resolved in step 1): SplitScript needs class storage facts beyond ASR's catalog. The sixteen boundary players were checked against hash-verified PDBs, including the relative byval_arg and definition members used for plain-class resolution. The fixture collection supplies 2018.3.0f2; its class/type headers match f1 byte-for-byte after LF normalization.

Those measurements exposed an unsafe assumption in the proposed re-keying: classValueTypeBit changes inside ASR profile ranges. For example, the measured 2018.3 x64 bit is 2345, while the previously audited 2018.4 bit is 2353; the 2020.2 x64 bit is 2449, while 2021.3 stores it at 351. Do not extrapolate these bits across a boundary profile's range. Step 1 removes the runtime bit field and classifies inflated generic storage through its immediate System.ValueType parent. Other parents, including a null parent for generic interfaces, represent reference storage. Value storage still uses the measured instance size. Parent metadata is read only by demanded collection operations, with work charges and checked reads. The bit measurements remain evidence, not runtime inputs.

The reproducible imports live in scripts/import-il2cpp-storage.py, scripts/il2cpp_profile_data.py, scripts/unity-pdb/, and the two IL2CPP fixture catalogs. Explicit/custom profiles remain independent of automatic tables.

2. IL2CPP discovery: replace the old route, without another framework

Current Unity.discoverIl2cpp32/64, il2cppAssemblies, il2cppNameReference32 and il2cppTable32 still scan raw signatures and global-metadata.dat. The final upstream route starts at:

  • il2cpp_domain_get_assemblies, following either its getter or inline load.
  • il2cpp_image_get_class, falling back to il2cpp_type_get_class_or_element_class when the first export is absent.

Port the small instruction-length/control-flow decoder and global matchers from il2cpp/instruction.rs and globals.rs as private library operations. Use existing PE export lookup and add a bounded private exception-directory reader. Preserve x86/x64 specialization. A synchronous bounded step with a small async cursor should carry traversal state; do not generate one giant async body or expose a general-purpose disassembler API.

Preserve upstream behavior for calls, conditional branches, tail jumps, returns, operand bytes that resemble opcodes, external conditional targets and unsupported instruction forms. Upstream uses call depths 1/4, finite function/branch/step limits, and a 256-byte code cache. Translate those into demand-created cursor storage and cooperative work slices; do not copy large Rust stack arrays into every module's scratch space. Failed reads must remain retryable where metadata is still initializing, and process closure must cancel attachment.

Once the replacement works, remove the old global-discovery helpers and their fixtures' fake scan anchors in the same slice. Do not retain both implementations as an unrequested fallback. Update shared IL2CPP fixture builders and Lunistice's mock export/function bytes so existing collection tests exercise the real new attachment route. ASR reports attachment on 80 Windows fixture players versus 29 before #179; this motivates the port but is not a SplitScript coverage claim.

3. Scenes: keep the typed API, replace its layout assumptions

Unity.sceneManager, UnityScenes, UnityScene and UnityGameObject in stdlib/standard.split use hard-coded offsets selected mostly by pointer width. Import the final scene profiles into a private descriptor containing signatures, displacement positions, offsets, path representation and managed-reference shape. Add a reproducible scripts/import-unity-scene-profiles.py with a checked-in fixture catalog and revision check. Generate only reachable constructors/tables.

Required behavior changes:

  1. On Windows, select by player patch and width, falling back to the main executable for old embedded-engine games. Check all signature matches: they must resolve to the same scene-manager global. Keep ASR's existing ELF/Mach-O x64 fallback behavior; these PRs do not add measured Unix scenes or native ARM64 scene discovery.
  2. Read the real m_Scenes vector and its count, rather than the waiting-for-load pointer. Maintain SplitScript's all-or-nothing [UnityScene] snapshots.
  3. Decode pointer, inline-NUL and inline-spare scene paths, including long paths spilled behind a pointer. Return owned String values and preserve name/index snapshots. Native path reads remain bounded; this does not reintroduce managed maxLength or maxCount keywords.
  4. Traverse the circular root list from its embedded sentinel in forward order; cover empty lists and retain bounded lookup. Use profile offsets for child transforms, GameObject names and components as well.
  5. Follow direct cached objects or indirect root slots according to the profile. Keep component<T>() -> T.Ref! and the already-resolved runtime header check; do not add ASR's separate string-based Mono/IL2CPP component APIs.
  6. Change persistent() to UnityScene?!: successful None for Unity 5.6.0 and 5.6.1, Some(snapshot) where the scene exists, and an error on failed reads. This distinction matters for state snapshots: absence updates state, whereas failure retains the last accepted state. Update callers/docs directly.

Scene support must remain reachable only through use of unity.scenes. An explicit managed IL2CPP script that never reads native scenes must not retain the new scene catalog, path readers or component-reference code.

4. Existing managed readers: verify rather than rewrite

UnityCollectionTypes.classFromType and arrayElementClass already route plain types to UnityIl2CppTypeTable. Existing il2cpp_plain_storage_runtime.mjs tests index zero, negative indices, sparse tables, other images, plain references and values, and handle-based layouts. Keyed fixtures cover parallel arrays and the recursively owned Map/Set surface. Preserve this stronger support.

After the profile update, ensure a 2018.4 parallel-array dictionary actually resolves its non-generic link type through the old type table at both widths. Also run the newer handle, generic-storage and deep-snapshot fixtures. Several existing fixtures infer handles from version[0] >= 2021; that becomes wrong when the new catalog explicitly includes 2020.2. Use independently documented layout facts and concrete transition fixtures instead of carrying that shortcut forward. Do not derive all test-memory offsets from the table under test.

5. Scanning: separate parity from optional speed work

SplitScript already emits one ScanProcessRange helper taking needle, mask and length (src/codegen/runtime_helpers/process.rs). It does not monomorphize a scanner per signature length. Its reads overlap by length - 1; there is no ASR-style previous-page tail buffer. src/signature.rs only parses literals. The current ARM64 Mono discovery already performs exact sub-byte word masks.

First transfer #178/#181 boundary tests: unaligned starts, short first/last pages, matches crossing readable boundaries, unreadable gaps, all-wildcard/nibble masks, and signatures of length 1 and 255. Compare results to a small independent brute-force oracle. The current all-or-nothing overlapping read skips a whole candidate block on failure; test whether that loses readable candidates beside an unreadable page, then fix failure isolation if reproduced. Preserve scanOnce exhaustion, waiting-scan retries, scanAny ordering and cancellation.

For missing capabilities, add ordinary methods on the existing Signature type, not new literal syntax: a checked masked constructor and finite matching against already-read bytes. Recommended contracts are Signature.masked(needle: [u8], mask: [u8]) -> Signature! and signature.findIn(bytes: [u8], start: u32) -> u32?; repeating the latter from match + 1 covers overlapping slice matches without introducing a new iterator model. Enforce equal lengths, 1–255 bytes, cleared wildcard bits, and owned pattern storage; dynamic signatures must inform scratch planning without retaining dynamic construction for literal-only scripts. Confirm these contracts against existing type/representation conventions during implementation. The existing ARM64 decoder need not change merely to use them.

Only then consider rare-byte anchors, a second-byte filter, wider comparisons, and larger reads. Compute literal anchor choices in the compiler so the frequency table is absent from Wasm. A 64 KiB buffer is not a free default: check initial pages, scratch ownership across suspended scans and per-poll work. SIMD, a full scanner rewrite and general optimizer work are optional follow-ups, not required to close the Unity profile/discovery/scene port. Keep an optimization only for a demonstrated useful speed gain with acceptable size, or a substantial size saving (roughly 500 bytes or more); do not resume a search for tiny savings.

Implementation sequence and acceptance

Each row is a bounded slice with its own reviewable commit. Run focused checks while working, then one full relevant verification at the end.

Order Deliverable Acceptance before moving on
0 Freeze the current sources/toolchain and measure explicit/automatic Lunistice, minimal non-Unity, Mono, scenes, nested collections and multiple signature lengths. Record ordinary Release output, scratch/pages and retained helpers. No compiler changes or external optimization pipeline.
1 Update IL2CPP importer/catalog/constructors/selection to final #163+#167; verify supplemental storage facts; retain #172 coverage. All 16 imported descriptors match upstream. Every boundary and just-below version selects correctly at both widths; fourth-part differences do not matter; pre-2018.1 fails. Explicit/custom profile checks, plain/generic collections and existing Lunistice behavior pass.
2 Replace Mono families with complete profiles using final #166+#175. Identity precedence; library/width isolation; Windows lower bound; Unix oldest fallback; all 2018.1 reversion/restoration boundaries; old main-executable versions; Linux/macOS version-string parsing; inherited statics and 2017.1 BDWGC walks pass. No active MonoVersion code or stale examples remain.
3 Port #179 export-based IL2CPP discovery and delete its predecessor. x86/x64 getter/inline exports, old export fallback, operand decoys, return/tail-jump boundaries, external conditional targets, x64 function bounds, cycles/work limits, unsupported decoder forms and cancellation pass. Existing IL2CPP runtime and Lunistice fixtures use the new route.
4 Port final #165+#173 scene layouts/readers into the existing typed context. All 29 imported entries match upstream. Boundary tests include 5.6.0/5.6.2, 2023.1.0/2023.1.4, 6000.0.58/.59, and x86 6000.1/6000.2.0/.2 reversions. Loaded-vs-waiting decoys, path shapes, forward roots, empty lists, direct/slot components, optional persistence and snapshot failures pass. Managed-only scripts retain no scene code.
5 Transfer scanner regressions and add missing mask/slice capabilities from #178+#181. Oracle agreement, page-failure isolation, cooperative cancellation, ordering, multiple lengths, dynamic ownership and demand-driven emission pass. Existing scan APIs retain their semantics. Performance experiments are separately measured and may be skipped.
6 Final verification, docs, baseline review and completion commit. Profile importer checks, focused/private adapters, compiler suite, registered runtime scenarios, base/DLC Lunistice, generated docs/tooling contracts, formatting and Clippy pass. All size changes are explained; no missing required row remains.

Useful existing test locations: tests/compiler/profiles_codegen.rs, tests/compiler/async_runtime.rs, tests/compiler/catalogs_types.rs, tests/il2cpp_profiles_runtime.mjs, tests/mono_profiles_runtime.mjs, tests/mono_linux_runtime.mjs, tests/mono_mac_runtime.mjs, tests/mono_explicit_unix_runtime.mjs, tests/managed_*_runtime.mjs, src/managed_collection_layout_tests.rs, and tests/lunistice_runtime.mjs. Extend the existing scene test in async_runtime.rs with maintained catalog-wide runtime fixtures; add all new runtime scenarios to src/bin/xtask.rs.

Size, resources and stopping rules

At the start, tests/baselines/unity.json recorded 26,151 bytes explicit / 109,217 bytes automatic for Lunistice with release-size-optimized-wasm. Its compiler stamp was 0a0686da94e1. Step 0 reproduced those measurements exactly at bff00fe; step 1 records 26,142 / 108,592 bytes, and step 2 records 26,142 / 101,781 bytes. Step 3 records 33,022 / 104,790 bytes; the export reader and instruction walk have a measured cost, detailed in BASELINES.md. The old 32,121-byte result used an earlier output pipeline and is historical. Do not reset the new budget upward to it or compare different pipelines.

For every implementation slice, compare module/section sizes, retained feature helpers, scratch capacity and initial pages using the existing Unity baseline gate. Keep non-Unity and unused-feature controls unchanged. Profile selection must not keep other profiles; collection-free scripts must not keep collection layout discovery; scene-free scripts must not keep scene readers. Required discovery changes may have a measured cost, but investigate and explain that cost before accepting a baseline update. Do not weaken functionality or alter the measurement pipeline to make a number smaller.

All compiler/build/runtime commands remain serialized through scripts/run_limited.py: at most 3,072 MiB for native builds and 768 MiB for compiler/runtime runs, with existing smaller limits for lightweight checks. Do not launch a broad build farm, raise caps, or clean caches for this plan. Use synthetic fixtures for the width/platform matrix. A short Lunistice demo probe can validate attachment when needed; stop the game immediately afterward. Lunistice cannot validate Mono or every scene layout. Native Linux/macOS results must be identified separately from fixture coverage.

Completion means the required six implementation/verification slices above are closed, with tested behavior and reviewed demand-driven size results. Optional scanner tuning does not extend the finish line. General compiler optimizations, field/property syntax, raw UTF-16 APIs, new container semantics, native time managers and extra platform support remain separate work.

Completed checkpoint: steps 0–1

Implemented on 2026-10-04:

  • Imported ASR's final sixteen IL2CPP profiles and removed superseded constructors. Automatic selection uses the three-part patch floor, ignores the fourth part, rejects pre-2018.1 players, and can reattach after rejection. Explicit/custom selection still omits version detection and the automatic catalog.
  • Verified all sixteen profiles against PDBs (176 core facts and 32 class-member relationships). Replaced the unstable supplemental bit as described above. Updated independent fixtures for early-2018 field strides, 2020.2 indirect image starts, and 6000.6 cached-class offsets. Existing plain-class resolution already covers #172 and remains in place.
  • Updated callers and documentation; Lunistice changes only its profile name. No source aliases, migration layer, or new syntax were introduced.

Verification: 87 automatic profile/boundary/rejection/reattachment cases; explicit x86/x64/custom/invalid-profile cases; 26 selected registered runtime scenarios covering nested collections and snapshots; all seven private metadata adapter suites in Debug and Release (including 336 ordinary and 256 generic storage cases per build profile); eight focused compiler demand/width tests; PDB and profile importer checks; Rust formatting; base-game and DLC Lunistice fixtures. The user reported that the delivered Lunistice checkpoint works. The final checkpoint is byte-identical to that tested artifact.

The reviewed 38-fixture size report is committed in tests/baselines/unity.json. Lunistice is 26,142 bytes explicit (-9) / 108,592 automatic (-625). Demanded IL2CPP collection fixtures grow 19–28 bytes for parent-based classification; non-Unity and explicit Mono sizes are unchanged. All scratch/read capacities and initial page counts are unchanged. See BASELINES.md for the full rationale.

Local testable artifacts, frozen source, and reports: target/asr-followup/00-baseline/ and target/asr-followup/01-il2cpp-profiles/. The latter contains checkpoint.json with the exact source/artifact hashes. Build/test logs are in target/asr-followup/01-*.log.

This checkpoint validates layouts and preserves current discovery. Real early 2018 attachment coverage still depends on step 3's export-based discovery; synthetic profile fixtures must not be presented as that live-game coverage. The next commit-sized slice is step 2 (Mono profiles), followed by discovery, scenes, scanner changes, and final verification. No optimization project was added to the remaining scope.

Completed checkpoint: step 2 — Mono profiles

Replaced MonoVersion and all family factories/detection probes with complete public MonoProfile descriptors. The importer pins the same final ASR revision and reproduces 14 Windows builds, four Linux builds, and two macOS identities sharing one profile. Callers select measured constructors or custom descriptors; there are no compatibility aliases or migration diagnostics.

Automatic selection validates exact identity against library/architecture first, then uses platform/library/width-specific three-part version selection. Windows rejects players below its first compatible sample; Unix retains oldest-sample fallback. Version discovery supports UnityPlayer and embedded main executables, including bounded Unix version-string parsing. Static storage follows the runtime library rather than a stored version tag. Explicit selectors omit automatic catalogs, identity readers, player version lookup, and other platforms.

Verification covers all measured identities, 2017.1 BDWGC, Windows patch boundaries and the 2018.1 reversion/restoration, width/library rejection, identity precedence, Unix oldest fallback, malformed strings, embedded executables, cancellation and reattachment. Explicit x86/x64, Linux/macOS and custom-profile runtime fixtures pass. The independent fixture memory shapes retain test-only labels; no runtime family API or fallback offset table remains. Obsolete tests for unmeasured macOS family factories were removed; custom descriptors support separately measured layouts.

The checkpoint is target/asr-followup/02-mono-profiles/. See BASELINES.md for size measurements. Coverage is synthetic; no native Linux/macOS game was run. Nested managed values and snapshots are unchanged. The next bounded slice is step 3: replace IL2CPP signature/global-metadata discovery with the new export walk and remove its predecessor.

Completed checkpoint: step 3 — export-based IL2CPP discovery

Replaced both old signature/global-metadata discovery routes with the final ASR #179 export walk. Assemblies start at il2cpp_domain_get_assemblies; the type table starts at il2cpp_image_get_class or its older fallback export. The decoder follows instruction boundaries, conditional branches, calls and tail jumps, using x64 exception-directory bounds when present. Getter/master patterns cover both pointer widths. Unsupported encodings terminate that branch.

The private cursor uses mutable arrays because SplitScript struct fields are immutable. Limits match upstream, with an additional 64-work-unit cooperative slice and a 15-byte instruction limit. Accepted globals and complete patterns must fit their module/function ranges. Transient code reads restart on a later tick; an exhausted readable walk reports failure and waits for process closure. All state belongs to the attachment, so cancellation and reattachment start fresh.

The new independent export fixtures replace old signature anchors in every IL2CPP attachment fixture. The 54-case walk matrix covers both widths, fallback exports, getter variants, call-depth/queue/instruction limits, operand decoys, returns/unsupported encodings, cycles, function boundaries, out-of-module globals, cache refills, transient reads and cancellation. A long walk must span at least 32 ticks. The private decoder probe separately checks instruction lengths and control-flow operands in Debug and Release.

Lunistice's nine registered scenarios, base/DLC Release fixtures, profile matrix, instance enumeration, 20 selected nested-managed/snapshot scenarios and metadata bounds pass. The metadata-bounds fixture was corrected to reflect step 2: Windows Mono profiles use image names for version selection as well as identity selection, rather than the removed assembly-name fallback.

The frozen checkpoint is target/asr-followup/03-il2cpp-discovery/, with exact source/artifact hashes. Lunistice source is unchanged. Its explicit build is 33,022 bytes (+6,880) and automatic build 104,790 (+3,009). The required export reader/decoder/walk explains this increase; no old discovery remains. Non-Unity and explicit Mono sizes are unchanged. See BASELINES.md for section, helper and resource accounting. The user reported that this exact checkpoint worked in-game.

All eight private runtime suites pass in Debug and Release, including 89 direct decoder cases per build profile. Seven focused compiler/catalog checks pass, including explicit profile/width demand and catalog documentation. The final 54-case walk matrix passes in both profiles, including cooperative timing. Rust formatting and diff whitespace checks pass. Builds/tests were serialized under the existing memory guard; the agent did not launch the game.

The next bounded slice is step 4 (scenes), followed by step 5 (scanner parity) and step 6 (final verification). General optimizer work remains outside this plan.

Checkpoint: step 4 — native scene layouts and readers

Imported all 29 measured Windows layouts from the pinned final ASR scene catalog, plus its existing Unix x64 fallback. The reproducible importer is scripts/import-unity-scene-profiles.py; --check verifies the generated private standard-library descriptor and tests/fixtures/unity-scene-profiles.json. Selection compares player patch and pointer width and ignores the fourth version part, preserving the 2018.2, 2023.1 and Unity 6 layout reversions. Unsupported Windows versions remain pending. Discovery checks every anchor match for a consistent global, retries while the manager is uninitialized, and reconsiders module selection when UnityPlayer loads after the executable. Engines embedded in the Windows executable are supported. Unix fallback remains x64-only and unmeasured; no native Linux/macOS scene-game validation is claimed.

Loaded scenes now use the actual dynamic array, rechecking its header after snapshotting all entries. Paths support pointers, inline NULs, inline spare-byte lengths and spilled strings. Roots traverse forward from the embedded sentinel, including empty rings. GameObject names use the corrected two-pointer path; the obsolete extra dereference was removed. Children and components use profile offsets, and components follow either cached objects or indirect root slots. The typed component<T>() interface is unchanged. Native traversal retains finite limits; no managed length/count keywords or new syntax were introduced.

persistent() now returns UnityScene?!: successful None on 5.6.0–5.6.1, Some(snapshot) where the scene exists, and errors for failed reads. Callers, examples, tests and the standard-library/porting documentation were updated. State retains its preceding value after failed scene/array reads, while an absent persistent scene is an accepted value.

Independent fixtures specify memory shapes separately from the imported offsets. They cover all catalog versions, just-below patch boundaries, fourth-part variation, both widths, pointer/inline/spilled paths, empty/forward/broken roots, loaded-vs-waiting decoys, changed array headers, persistent absence, failed reads, conflicting anchors, engine-in-executable discovery, cancellation and reattachment. The component probe passes 159 cases in each build profile; the focused compiler checks cover snapshot typing, native-scene demand, typed components and the standard-library catalog. The final public scene matrix passes 150 cases in each build profile, including missing-version retries and late UnityPlayer loads. Header/profile reads use one synchronous helper and one async retry edge, avoiding continuation duplication. The checkpoint is frozen at target/asr-followup/04-scenes/.

All 38 pre-existing size fixtures remain unchanged, including Lunistice at 33,022 / 104,790 bytes with unchanged scratch, read capacity and initial pages. Its source and executable bytes are unchanged; only the embedded compiler revision differs from the step-3 artifact confirmed by the user. Native scenes are demanded only by scripts that use them. A dedicated unity-scenes fixture now extends the size gate to this subsystem; its same-source pre-change Release measurement was 12,419 bytes. Final measurements are recorded in BASELINES.md.

After this step, only scanner parity (step 5) and final verification (step 6) remain. No optimizer work was added.

Checkpoint: step 5a — scanner page-failure isolation

The first scanner slice reproduces and fixes a missed match beside an unreadable page. SplitScript previously discarded all 4 KiB of candidate starts when an overlapping host read failed. It now retries the current page plus its overlap, then the page alone if needed, and advances to the next page boundary. The first retry preserves a readable boundary crossing when an unaligned bulk read touches a third, unreadable page. Failed reads never feed the matcher.

The normal successful-read path keeps its existing window size and scratch. The async callers retain the 512 KiB candidate budget, mandatory yield, finite exhaustion, waiting retries and cancellation. This is a correctness change, not optional scanner tuning; no SIMD, larger buffer or anchor table was added.

tests/signature_boundaries_runtime.mjs compares results with an independent byte-by-byte oracle across short and unaligned ranges, gaps, 1–255-byte patterns, nibble/full wildcards and range-exterior decoys. Failed host reads poison the scratch buffer. Long-range, retry and cancellation cases check cooperative work. All 786 oracle/runtime cases pass in Debug and Release, as do the existing process scan and scan-any scenarios and 150 scene cases per profile. Lunistice base/DLC fixtures pass. Release Lunistice remains 33,022 bytes; automatic selection adds 86 bytes (104,876 total). Scanner users add 85–86 code bytes with no scratch/page/helper growth; scanner-free fixtures remain unchanged. The reviewed report now has 40 fixtures, including the scanner control. Artifacts and hashes are frozen in target/asr-followup/05-scanner-pages/; this new checkpoint has not yet been confirmed by the user in-game.

Step 5 is not complete: owned arbitrary-mask construction and findIn over already-read arrays still need implementation and demand/ownership tests. The remaining upstream boundary review includes the end of the address space. Step 6 will then run final verification; optional speed work does not extend this list.

Checkpoint: step 5b — owned masks, array search and address bounds

Added Signature.masked(needle: [u8], mask: [u8]) -> Signature! with equal, nonempty 1–255-byte bounds and normalization of wildcard bits. Its storage is copied and GC-owned, so input mutation, suspension and concurrent scans cannot change the pattern. Added signature.findIn(bytes: [u8], start: u32) -> u32?, including overlapping matches by resuming at the preceding match plus one. The public methods are ordinary library code over three private representation primitives; no new literal syntax was introduced.

Programs with no reachable constructor keep the prior packed static representation. Programs constructing signatures use private GC byte storage for all signature values, including literals and arrays of signatures. Process scans restage the selected pattern into a 510-byte companion buffer each poll; their 255-byte maximum informs scratch planning even with no signature literals. Only dynamic process scans retain this helper/buffer. Array-only searches need neither it nor the process scanner. No unbounded linear-memory allocation or permanent table of signature handles was introduced.

All scan entry points now truncate ranges at the end of the 64-bit address space before advancing their cooperative cursors. This preserves valid matches in the final bytes and prevents wrapping into low memory on subsequent polls.

Runtime validation passes in Debug and Release: 47 dynamic scan scenarios cover all seven APIs, arbitrary masks, lengths 1/3/17/255, absence, retry, cancellation, long scans and address-space-end behavior. A 100-case independent slice oracle, 13 value/literal assertions, concurrent scans and the existing 786-case boundary matrix pass too. Input mutations and GC ownership across suspension are exercised. Twelve focused compiler, catalog, helper-registry and scratch-layout checks pass, including byte-identical output when dynamic construction is unreachable. Existing process-scan/scan-any and all 150 scene cases pass in both profiles. Formatting and Lunistice base/DLC fixtures pass. The reviewed 43-fixture report retains all previous resources; address bounds add 22 bytes per scan poll, with no dynamic-signature cost in existing programs. Lunistice remains 33,022 bytes explicit and 104,942 automatic (+66). Details are in BASELINES.md; artifacts and hashes are frozen in target/asr-followup/05-signatures/.

There is no required scanner speed experiment left. Only step 6's final cross-feature verification remains.

Completion: step 6 — final verification

The implementation is complete through ASR ce7a966. No required feature or verification row remains. Optional scanner speed experiments and general compiler optimization remain outside this work.

Final checks passed:

  • All 1,450 native tests: 589 library, 700 compiler, 108 syntax, 29 standard-library loader, 19 CLI, one language-server and four verification-tool tests. Three opt-in measurement tests remain ignored by the normal suite.
  • All 226 registered runtime scenarios, covering 185 unique Wasm artifacts, plus the Debug/Release diagnostic contrast. Each artifact was validated. Private managed-layout and instruction-decoder probes also pass in the library suite.
  • IL2CPP, Mono and scene importer checks against the pinned checkout. The supplemental storage check verified 16 PDBs, 176 ASR layout facts, 32 class member relationships and all 16 value-type bits.
  • All 43 size fixtures and Lunistice base/DLC behavior. Every fixture is unchanged from the reviewed step-5b baseline.
  • Generated documentation validation, the reference graph/snapshot, editor type checks and 45 editor contract tests, formatting and release Clippy with warnings denied.

Verification exposed stale expectations rather than missing runtime features: the Mono parser fixture still expected an enum path; IL2CPP scratch assertions still included the removed 24-byte signature; reference documentation needed updated links and its new snapshot. Unused-declaration tests now compare opcode and constant histograms, data, retained helpers and resources, and reject added types/locals instead of requiring equal encoded file lengths. Renumbered GC indices can cross LEB width boundaries and change size-based cleanup decisions. In Lunistice, the variant containing an unused declaration actually removes one unused local and type, with identical opcode/constant histograms and data. No size allowance or production optimizer change was introduced.

Final artifacts, source, size report, base/DLC results and hashes are frozen in target/asr-followup/06-final/. Lunistice is 33,022 bytes with its explicit profile and 104,942 bytes with automatic selection. Both have identical executable sections to step 5b; only compiler revision metadata differs. The source is unchanged. This checkpoint has not been tested in-game by the user; no game was launched during final verification. Platform/profile coverage is fixture-based, not a claim of native game testing on every supported platform.

All jobs were serialized through scripts/run_limited.py. Native builds stayed under 3,072 MiB; test/compiler/runtime jobs stayed under 768 MiB. The full runtime matrix peaked at 671.8 MiB, and the final native rerun at 636.5 MiB. No resource limit was raised and no cleanup or optimization project was added.