Skip to content

fix(build): 🐛 Sort every walked folder and date archives by the last config commit so builds match across machines - #59

Merged
Stoupy51 merged 1 commit into
mainfrom
fix/deterministic-build
Sep 30, 2026
Merged

Stoupy51 merged 1 commit into
mainfrom
fix/deterministic-build

Conversation

@Stoupy51

@Stoupy51 Stoupy51 commented Sep 30, 2026 •

Copy link
Copy Markdown
Owner

A Windows build and a Linux build of the same project gave different outputs: NTFS lists a folder in alphabetical order, ext4 in hash order.

Sorted file walks

Every os.walk, os.listdir, glob and rglob whose order reaches the output is now sorted:

  • beet_patches.fast_list_files: the order a pack loads its files (and so the zip entry order)
  • sounds: sounds.json entries
  • records, simplenergy gui and cables, ingame_manual font textures: definitions and generated files
  • libs, merge_smithed_weld glob: merge order
  • compute_sha1: sha1_hashes.json key order
  • the texture dicts keyed by basename (materials, item_models, custom_blocks, ingame_manual), custom_paintings and project_images: which file wins when two textures share a name

Sorted order equals the NTFS one for lowercase names, so Windows builds do not reorder.
Walks used only for lookups, the CLI, dump and migrate are left as they are.

Archive date

get_consistent_timestamp first uses the date of the last commit of the beet config found by locate_config (any of DETECT_CONFIG_FILES), in UTC, which is the same in every clone. Git does not keep file modification times, so the beet cache mtime is only the fallback outside a git repository, then 2025-01-01.
In a shallow clone that does not reach the last commit of the config, the date is the one of the shallow boundary.

Not covered

PNG bytes still differ between Windows and Linux with identical pixels: Pillow does not ship the same zlib on both.

Checks

  • Survisland built on Linux with this branch: sounds.json, _give_all, creative_loot_table.json, set_items_storage and definitions_debug.json are byte-identical to the Windows build, and two builds with different PYTHONHASHSEED are identical.
  • scripts/all_doctests.py and scripts/run_integration_tests.py (33/33) pass.
  • ruff and stouputils check report nothing new, apart from one line added to the already long fast_list_files docstring.

@Stoupy51
Stoupy51 merged commit 86886f5 into main Sep 30, 2026
4 of 6 checks passed
@Stoupy51
Stoupy51 deleted the fix/deterministic-build branch September 30, 2026 09:19
Stoupy51 added a commit that referenced this pull request Oct 6, 2026
…config commit so builds match across machines (#59)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant