diff --git a/CLAUDE.md b/CLAUDE.md index 60c7c297..8957e187 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -130,16 +130,17 @@ Use the script rather than your own `luac`. Homebrew no longer ships Lua 5.2, so The planner hardcodes entity names, item names, direction values, and entity sizes. When Factorio changes any of those, nothing here notices - plans keep generating, they are just wrong. Factorio 2.0 renamed `effectivity-module-N` to `efficiency-module-N` and widened directions from 8-way to 16-way, and both went unnoticed for a long time. -So don't trust memory or the wiki. The game is the only authority on what the game accepts, and it ships four machine-readable sources: +So don't trust memory or the wiki. The game is the only authority on what the game accepts, and the capture reads three machine-readable sources: | Source | Answers | | --- | --- | | `factorio --dump-data` | Every prototype: names, collision boxes, pipe connections, pole supply and wire reach, beacon stats | | `data/*/migrations/*.json` | Every rename, as a table. This is a complete list, not a guess | -| `doc-html/runtime-api.json` | `defines.*` values, version-stamped to the install | -| `data/changelog.txt` | Behavior changes per patch | +| `doc-html/runtime-api.json` | The `defines.*` tables, version-stamped to the install. See the caveat below: it publishes a documentation index, not the values | -`tools/capture-factorio-oracle.sh` pulls all four into `test/FactorioTools.Test/OilField/factorio-oracle.json`: +A fourth source, `data/changelog.txt`, records behavior changes per patch. It is worth reading after an update, but nothing automates it - no code in `tools/` touches it. + +`tools/capture-factorio-oracle.sh` pulls those three into `test/FactorioTools.Test/OilField/factorio-oracle.json`: ```bash tools/capture-factorio-oracle.sh # auto-detects a Steam or /Applications install @@ -154,11 +155,31 @@ Notes on using it: - **Re-capture after every Factorio update and commit the diff.** A changed fixture is the signal that a hardcoded constant needs review. `--check` answers "has the game moved past what we committed?" without dirtying the tree. - **The installed binary is the authority** on which version gets captured. Steam updates it without asking, so it decides and everything else follows. Same convention as `scripts/sync-factorio-refs.sh` in FactorioMapWebUI. - **It runs with user mods disabled** (`--mod-directory` pointed at an empty directory). Mods rewrite prototypes freely, so a capture that loads them describes one person's modded game rather than Factorio. The script prints which mods loaded; expect only `core base elevated-rails quality recycler space-age`. + + Careful with what that buys. Measured on 2.1.14: **an empty mod directory keeps out user mods and nothing else.** Factorio rewrites `mod-list.json` at startup and adds back every bundled mod the file does not mention, with `enabled: true`. The file this script writes names only `base`, and all six still load. An explicit `enabled: false` is honoured, so naming a mod is the only way to get a smaller game than the install ships with. Loading the full set is the right default here, since that is what the fixture records, but "the directory is empty" must not be read as "only base is loaded". +- **`defines` values in the fixture are inferred, not read.** `runtime-api.json` has no value field at all: across all 1,554 entries the only keys are `name`, `order` and `description`, and `trim-factorio-oracle.py:150` uses `order`. That is right today only because Factorio declares directions clockwise from `north = 0` with no gaps, and a dense index cannot express a gap, a duplicate, or a non-zero start. Issue #83. Reading the real value needs a probe mod; `factorio-oracle` does that now, and confirmed the two agree on 2.1.14. - **CI never runs the capture** and needs no Factorio install - it reads the committed fixture. That is why the fixture is committed rather than generated on demand. - Capture needs `python3` (for JSON trimming) and a Factorio install. Neither is needed to build or test. - `EntityNames.AaiIndustry` names come from a mod, so they are deliberately absent from a vanilla capture. That is expected, not drift. - Output is deterministic: two captures of the same install are byte-identical. +**This script now has a replacement, and it is proven equivalent.** +[`FactoryGameFan/factorio-oracle`](https://github.com/FactoryGameFan/factorio-oracle) +is a shared Rust CLI doing the same job for four repos. Its acceptance test +reproduces the committed `factorio-oracle.json` **byte for byte** from a real +2.1.14 install, so this is a checked claim rather than an intention: + +```bash +factorio-oracle run --probe dump-data.json --work-dir /tmp/w > /tmp/run.json +factorio-oracle trim --run /tmp/run.json --spec trim-spec.json \ + --out test/FactorioTools.Test/OilField/factorio-oracle.json [--check] +``` + +The allowlists that live in `trim-factorio-oracle.py` move into that `trim-spec.json` +unchanged. **This script stays** regardless: the agreed migration rule across the four +repos is new probes only, so nothing existing changes until there is a reason to touch +it. See issue #82. + Two related sources, for when the game binary is not the easiest thing to reach: - **`wube/factorio-data`** (cloned at `~/GitHub/factorio-data`) is the official prototype source, tagged per version. Its `*/migrations/*.json` files are byte-identical to the installed game's, so renames can be checked with no Factorio install at all. Only the resolved geometry from `--dump-data` genuinely needs the binary. diff --git a/docs/superpowers/plans/2026-08-16-factorio-oracle-runner-core.md b/docs/superpowers/plans/2026-08-16-factorio-oracle-runner-core.md new file mode 100644 index 00000000..10d85f20 --- /dev/null +++ b/docs/superpowers/plans/2026-08-16-factorio-oracle-runner-core.md @@ -0,0 +1,2733 @@ +# factorio-oracle Runner Core Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build a Rust CLI that discovers Factorio installs and runs a headless probe described by a JSON spec, returning the work directory and provenance. + +**Architecture:** A single binary with pure builders and one injectable spawn boundary. Every function that produces a string or an argument vector is pure and unit-tested with no Factorio present. Only `run.rs` touches disk and processes, and it takes a `Spawner` trait so a fake game can assert the argument vector and write the dump a real game would have written. The tool owns plumbing only; probe analysis stays in the consumer's language. + +**Tech Stack:** Rust (edition 2021), `clap` v4 derive for the CLI, `serde` + `serde_json` for the spec and output, `anyhow` for errors, `tempfile` for tests. No regex crate - version parsing is hand-rolled to keep the dependency surface small. + +**Spec:** `/Users/ericjohnson/GitHub/FactorioTools/docs/superpowers/specs/2026-08-16-shared-factorio-oracle-design.md` + +## Global Constraints + +- **Repo:** new, public, `factorio-oracle`. This plan builds it from an empty directory at `~/GitHub/factorio-oracle`. +- **Rust toolchain is pinned** in `rust-toolchain.toml`. Use `1.97.1`, matching FactorioMapWebUI, which pins it as a correctness control rather than a convenience. +- **Renovate config must exist in the first commit.** The Renovate app runs with "Require config file" enabled, so a default branch with no valid config makes Renovate silently do nothing, which is indistinguishable from "no updates available". Exactly one config file may exist in the repo. +- **`automerge: false` globally**, no exceptions. One weekly batch, Monday morning, `America/Los_Angeles`. Security updates are exempt from that window. +- **Renovate ecosystems here are `cargo` and `github-actions`.** Not npm, not NuGet. +- **House writing style:** hyphens only. Never em dashes or en dashes, in code comments, docs, or commit messages. +- **CI must pass with no Factorio installed.** Every test in this plan runs without the game. That is a hard requirement, matching all four consumer repos. +- **The version a mod declares is always derived from the binary, never hardcoded.** A mismatch makes Factorio skip the mod in silence, and the run ends with no dump and nothing naming the cause. +- **Lua supplied by a consumer is opaque.** Never template it, escape it, rewrite it, or wrap it in `script.on_init`. +- **Determinism:** every map that reaches output is a `BTreeMap` or is explicitly sorted. `HashMap` iteration order is randomised per process and would make drift checks permanently red. + +--- + +### Task 1: Repository skeleton + +Creates the repo, the toolchain pin, CI, and the Renovate config. Ends with a green CI run on an empty library. + +**Files:** +- Create: `Cargo.toml` +- Create: `rust-toolchain.toml` +- Create: `.gitignore` +- Create: `README.md` +- Create: `.github/workflows/ci.yml` +- Create: `.github/renovate.json5` +- Create: `src/lib.rs` +- Create: `src/main.rs` + +**Interfaces:** +- Consumes: nothing. +- Produces: a crate named `factorio_oracle` with a binary target `factorio-oracle`. Later tasks add modules to `src/lib.rs`. + +- [ ] **Step 1: Create the directory and initialise git** + +```bash +mkdir -p ~/GitHub/factorio-oracle +cd ~/GitHub/factorio-oracle +git init -b main +``` + +- [ ] **Step 2: Write `Cargo.toml`** + +```toml +[package] +name = "factorio-oracle" +version = "0.1.0" +edition = "2021" +description = "Ask a real Factorio install what it does, and record the answer with its provenance" +license = "MIT" +repository = "https://github.com/FactoryGameFan/factorio-oracle" + +[lib] +name = "factorio_oracle" +path = "src/lib.rs" + +[[bin]] +name = "factorio-oracle" +path = "src/main.rs" + +[dependencies] +anyhow = "1" +clap = { version = "4", features = ["derive"] } +serde = { version = "1", features = ["derive"] } +serde_json = { version = "1", features = ["preserve_order"] } + +[dev-dependencies] +tempfile = "3" +``` + +- [ ] **Step 3: Write `rust-toolchain.toml`** + +```toml +# Pinned deliberately. FactorioMapWebUI pins its toolchain as a correctness +# control, because a compiler change is a codegen change. This repo does not ship +# wasm, so the reason here is weaker - but a shared tool that four repos rely on +# should not change behaviour because a contributor has a different rustup default. +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy"] +profile = "minimal" +``` + +- [ ] **Step 4: Write `.gitignore`** + +```gitignore +/target +/refs +.DS_Store +``` + +- [ ] **Step 5: Write `src/lib.rs`** + +```rust +//! Plumbing for asking a real Factorio install what it does. +//! +//! This crate owns discovery, mod scaffolding, launching, and reading results +//! back. It deliberately owns none of the analysis: a probe compares the game +//! against a consumer's own reimplementation, so that half stays with the +//! consumer, in the consumer's language. + +/// Returns the crate version, so `main` and tests share one source of truth. +pub fn version() -> &'static str { + env!("CARGO_PKG_VERSION") +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn version_is_not_empty() { + assert!(!version().is_empty()); + } +} +``` + +- [ ] **Step 6: Write `src/main.rs`** + +```rust +fn main() { + println!("factorio-oracle {}", factorio_oracle::version()); +} +``` + +- [ ] **Step 7: Write `.github/workflows/ci.yml`** + +```yaml +name: CI + +on: + push: + branches: [main] + pull_request: + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + verify: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + # No toolchain action: rust-toolchain.toml is honoured by the preinstalled + # rustup, so the pin stays the single source of truth. + - name: Show toolchain + run: rustc --version && cargo --version + + - name: Format + run: cargo fmt --all -- --check + + - name: Clippy + run: cargo clippy --all-targets -- -D warnings + + - name: Test + run: cargo test --all-targets +``` + +- [ ] **Step 8: Write `.github/renovate.json5`** + +```json5 +{ + $schema: "https://docs.renovatebot.com/renovate-schema.json", + extends: ["config:recommended"], + + // One batch a week rather than a trickle. Monday morning means a failed + // update is looked at on a weekday, not discovered the following weekend. + schedule: ["* 0-8 * * 1"], + timezone: "America/Los_Angeles", + + // Nothing automerges, with no exceptions. A green CI run proves the repo is + // consistent, not that a bump is correct - and this tool's correctness lives + // in fixtures captured from a game CI cannot run. + automerge: false, + + // Security fixes deliberately skip the weekly window. + vulnerabilityAlerts: { + enabled: true, + schedule: ["at any time"], + }, + + // The toolchain pin is a deliberate control, not a stale dependency. Bumping + // it is a decision, so it gets its own PR rather than riding along in a batch. + packageRules: [ + { + matchManagers: ["cargo"], + matchUpdateTypes: ["patch", "minor"], + groupName: "cargo patch and minor", + }, + { + matchManagers: ["github-actions"], + groupName: "github actions", + }, + ], +} +``` + +- [ ] **Step 9: Write `README.md`** + +```markdown +# factorio-oracle + +Asks a real Factorio install what it does, so behaviour that other projects +reimplement can be checked against the game rather than against assumptions. + +Four repos each wrote this plumbing separately: FactorioTools, +factorio-blueprint-editor, FactorioMapWebUI and FactorioWikiDamageThresholds. +This is that plumbing, once. + +## What it does and does not do + +It owns discovery, mod scaffolding, launching and reading results back. It owns +none of the analysis. A probe compares the game against a consumer's own +reimplementation, so that half has to run in the consumer's language. The +interface is therefore JSON in and JSON out, not a probe framework. + +## Scope + +Behavioural reverse engineering for interoperability - understanding what the +game computes so other projects can agree with it. Not extracting or +redistributing game code or assets. Keep it that way. +``` + +- [ ] **Step 10: Validate the Renovate config** + +Run: `npx --yes --package renovate -- renovate-config-validator .github/renovate.json5` +Expected: reports the config is valid. If it fails, the file is silently inert on the default branch, which is the failure this step exists to prevent. + +- [ ] **Step 11: Run the checks** + +Run: `cargo fmt --all -- --check && cargo clippy --all-targets -- -D warnings && cargo test` +Expected: PASS, with one test passing (`version_is_not_empty`). + +- [ ] **Step 12: Commit** + +```bash +git add -A +git commit -m "Set up the factorio-oracle crate, CI and Renovate + +Renovate lands in the first commit deliberately. The app runs with +'Require config file' enabled, so a default branch with no valid config +makes it do nothing at all, silently." +``` + +--- + +### Task 2: Parse the version out of the binary + +Two values come from `factorio --version`: the full build line, which fixtures stamp, and the `major.minor` a mod's `info.json` must declare. Getting the second wrong makes Factorio skip the mod without saying so. + +**Files:** +- Create: `src/version.rs` +- Modify: `src/lib.rs` + +**Interfaces:** +- Consumes: nothing. +- Produces: `pub struct VersionInfo { pub line: String, pub major: u32, pub minor: u32, pub patch: u32 }`, `impl VersionInfo { pub fn major_minor(&self) -> String }`, and `pub fn parse_version_line(output: &str) -> Option`. + +- [ ] **Step 1: Write the failing test** + +Create `src/version.rs` containing only the test module: + +```rust +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn parses_a_real_macos_steam_version_line() { + let info = parse_version_line("Version: 2.0.77 (build 84539, mac-arm64, full)\n") + .expect("should parse"); + assert_eq!(info.major, 2); + assert_eq!(info.minor, 0); + assert_eq!(info.patch, 77); + assert_eq!(info.line, "Version: 2.0.77 (build 84539, mac-arm64, full)"); + } + + #[test] + fn major_minor_is_what_a_mod_declares() { + let info = parse_version_line("Version: 2.1.14 (build 87038, mac-arm64, steam)").unwrap(); + assert_eq!(info.major_minor(), "2.1"); + } + + #[test] + fn ignores_the_build_number_and_arch_digits() { + // "84539" and the "64" in "mac-arm64" are digits too. Only the + // three-part token is a version. + let info = parse_version_line("Version: 2.0.77 (build 84539, mac-arm64, full)").unwrap(); + assert_eq!((info.major, info.minor, info.patch), (2, 0, 77)); + } + + #[test] + fn reads_only_the_first_line() { + let info = parse_version_line("Version: 2.0.77 (build 1, x, y)\nMap version 9.9.9").unwrap(); + assert_eq!(info.patch, 77); + } + + #[test] + fn returns_none_when_there_is_no_version() { + assert!(parse_version_line("bash: factorio: command not found").is_none()); + assert!(parse_version_line("").is_none()); + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Add `pub mod version;` to `src/lib.rs`, then run: + +Run: `cargo test version` +Expected: FAIL to compile, with `cannot find function 'parse_version_line' in this scope`. + +- [ ] **Step 3: Write minimal implementation** + +Insert above the test module in `src/version.rs`: + +```rust +//! Reading a Factorio version out of `factorio --version`. + +/// A parsed `factorio --version` first line. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct VersionInfo { + /// The full first line, verbatim. This is what a fixture stamps, because it + /// carries the build number and platform as well as the version. + pub line: String, + pub major: u32, + pub minor: u32, + pub patch: u32, +} + +impl VersionInfo { + /// The value a mod's `info.json` must declare in `factorio_version`. + /// + /// Derived, never hardcoded. A mod declaring 2.1 against a 2.0.x binary is + /// skipped in silence, and the run ends with no dump and nothing in + /// Factorio's output naming the cause. + pub fn major_minor(&self) -> String { + format!("{}.{}", self.major, self.minor) + } +} + +/// Parses the first line of `factorio --version`. +/// +/// Looks for the first token of the form `..`. A build +/// number or an architecture suffix contains digits too, so a bare digit scan +/// would find the wrong thing. +pub fn parse_version_line(output: &str) -> Option { + let line = output.lines().next()?.trim(); + for token in line.split(|c: char| !(c.is_ascii_digit() || c == '.')) { + let parts: Vec<&str> = token.split('.').collect(); + if parts.len() != 3 { + continue; + } + if let (Ok(major), Ok(minor), Ok(patch)) = + (parts[0].parse(), parts[1].parse(), parts[2].parse()) + { + return Some(VersionInfo { + line: line.to_string(), + major, + minor, + patch, + }); + } + } + None +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test version` +Expected: PASS, 5 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/version.rs src/lib.rs +git commit -m "Parse the Factorio version, and derive the major.minor a mod declares + +A mod declaring the wrong factorio_version is skipped in silence, so this +value is derived from the binary rather than written down." +``` + +--- + +### Task 3: Resolve an install's layout + +macOS ships an `.app` bundle, Linux a plain directory, and a caller may point straight at the executable. All three must resolve to the same three paths. + +**Files:** +- Create: `src/install.rs` +- Modify: `src/lib.rs` + +**Interfaces:** +- Consumes: nothing. +- Produces: `pub struct InstallLayout { pub root: PathBuf, pub binary: PathBuf, pub data_dir: PathBuf, pub doc_dir: PathBuf }` and `pub fn resolve_layout(root: &Path) -> Option`. + +- [ ] **Step 1: Write the failing test** + +Create `src/install.rs` with only the test module: + +```rust +#[cfg(test)] +mod tests { + use super::*; + use std::fs; + use tempfile::tempdir; + + fn touch(path: &Path) { + fs::create_dir_all(path.parent().unwrap()).unwrap(); + fs::write(path, b"").unwrap(); + } + + #[test] + fn resolves_a_macos_app_bundle() { + let dir = tempdir().unwrap(); + let app = dir.path().join("factorio.app"); + touch(&app.join("Contents/MacOS/factorio")); + fs::create_dir_all(app.join("Contents/data")).unwrap(); + fs::create_dir_all(app.join("Contents/doc-html")).unwrap(); + + let layout = resolve_layout(&app).expect("should resolve"); + assert_eq!(layout.binary, app.join("Contents/MacOS/factorio")); + assert_eq!(layout.data_dir, app.join("Contents/data")); + assert_eq!(layout.doc_dir, app.join("Contents/doc-html")); + } + + #[test] + fn resolves_a_linux_install_directory() { + let dir = tempdir().unwrap(); + let root = dir.path().join("factorio"); + touch(&root.join("bin/x64/factorio")); + fs::create_dir_all(root.join("data")).unwrap(); + fs::create_dir_all(root.join("doc-html")).unwrap(); + + let layout = resolve_layout(&root).expect("should resolve"); + assert_eq!(layout.binary, root.join("bin/x64/factorio")); + assert_eq!(layout.data_dir, root.join("data")); + assert_eq!(layout.doc_dir, root.join("doc-html")); + } + + #[test] + fn resolves_a_path_pointing_straight_at_the_binary() { + // This is the FACTORIO_BIN case: callers set it to an executable, not a root. + let dir = tempdir().unwrap(); + let root = dir.path().join("factorio"); + let bin = root.join("bin/x64/factorio"); + touch(&bin); + fs::create_dir_all(root.join("data")).unwrap(); + fs::create_dir_all(root.join("doc-html")).unwrap(); + + let layout = resolve_layout(&bin).expect("should resolve"); + assert_eq!(layout.binary, bin); + assert_eq!(layout.data_dir, root.join("data")); + } + + #[test] + fn returns_none_when_the_binary_is_missing() { + let dir = tempdir().unwrap(); + let app = dir.path().join("factorio.app"); + fs::create_dir_all(app.join("Contents/data")).unwrap(); + assert!(resolve_layout(&app).is_none()); + } + + #[test] + fn returns_none_for_a_path_that_does_not_exist() { + assert!(resolve_layout(Path::new("/nope/not/here")).is_none()); + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Add `pub mod install;` to `src/lib.rs`, then run: + +Run: `cargo test install` +Expected: FAIL to compile, with `cannot find function 'resolve_layout'`. + +- [ ] **Step 3: Write minimal implementation** + +Insert above the test module in `src/install.rs`: + +```rust +//! Finding Factorio installs and working out where their pieces live. + +use std::path::{Path, PathBuf}; + +/// The three paths every mode needs from an install. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct InstallLayout { + /// What the caller pointed at, kept for reporting. + pub root: PathBuf, + pub binary: PathBuf, + pub data_dir: PathBuf, + pub doc_dir: PathBuf, +} + +/// Works out an install's layout from a root path, an `.app` bundle, or a path +/// straight to the executable. +/// +/// Returns `None` unless the binary and the data directory both exist. The doc +/// directory is not required: a headless build ships no `doc-html`, and probes +/// that never read the API docs work fine without it. +pub fn resolve_layout(root: &Path) -> Option { + let candidates: Vec<(PathBuf, PathBuf, PathBuf)> = if root.join("Contents").is_dir() { + // macOS .app bundle. + vec![( + root.join("Contents/MacOS/factorio"), + root.join("Contents/data"), + root.join("Contents/doc-html"), + )] + } else if root.is_file() { + // A path straight to the executable, which is what FACTORIO_BIN holds. + // The install root is two levels up from bin/x64/factorio. + let base = root.parent()?.parent()?.parent()?; + vec![( + root.to_path_buf(), + base.join("data"), + base.join("doc-html"), + )] + } else { + // A plain install directory. + vec![( + root.join("bin/x64/factorio"), + root.join("data"), + root.join("doc-html"), + )] + }; + + for (binary, data_dir, doc_dir) in candidates { + if binary.is_file() && data_dir.is_dir() { + return Some(InstallLayout { + root: root.to_path_buf(), + binary, + data_dir, + doc_dir, + }); + } + } + None +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test install` +Expected: PASS, 5 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/install.rs src/lib.rs +git commit -m "Resolve an install layout from a bundle, a directory, or a binary path + +Four repos each wrote a version of this. The .app-versus-directory split and +the FACTORIO_BIN-points-at-an-executable case are the two that keep recurring." +``` + +--- + +### Task 4: Enumerate candidate installs and add `installs list` + +**Files:** +- Modify: `src/install.rs` +- Modify: `src/main.rs` + +**Interfaces:** +- Consumes: `InstallLayout` and `resolve_layout` from Task 3; `VersionInfo` and `parse_version_line` from Task 2. +- Produces: `pub fn candidate_roots(home: &Path, env_bin: Option<&Path>) -> Vec` and `pub struct DiscoveredInstall { pub layout: InstallLayout, pub version: Option }`. + +- [ ] **Step 1: Write the failing test** + +Append to the `tests` module in `src/install.rs`: + +```rust + #[test] + fn env_bin_is_first_when_set() { + let home = Path::new("/home/someone"); + let roots = candidate_roots(home, Some(Path::new("/opt/custom/factorio"))); + assert_eq!(roots[0], PathBuf::from("/opt/custom/factorio")); + } + + #[test] + fn covers_every_candidate_the_four_repos_used() { + let home = Path::new("/home/someone"); + let roots = candidate_roots(home, None); + // The union of the candidate lists found across FactorioTools, + // FactorioMapWebUI, factorio-blueprint-editor and the stray benchmark + // script. Each repo had a different subset, so each found a different + // set of installs. + let expected = [ + "/home/someone/Library/Application Support/Steam/steamapps/common/Factorio/factorio.app", + "/Applications/factorio.app", + "/home/someone/.steam/steam/steamapps/common/Factorio", + "/home/someone/.factorio", + "/opt/factorio", + ]; + for want in expected { + assert!( + roots.contains(&PathBuf::from(want)), + "missing candidate: {want}\ngot: {roots:?}" + ); + } + } + + #[test] + fn candidates_are_unique() { + let home = Path::new("/home/someone"); + let roots = candidate_roots(home, None); + let mut seen = roots.clone(); + seen.sort(); + seen.dedup(); + assert_eq!(seen.len(), roots.len(), "duplicate candidate in {roots:?}"); + } +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cargo test install` +Expected: FAIL to compile, with `cannot find function 'candidate_roots'`. + +- [ ] **Step 3: Write minimal implementation** + +Add to `src/install.rs`, above the test module: + +```rust +use crate::version::{parse_version_line, VersionInfo}; + +/// An install that was found, with its version if the binary would run. +#[derive(Debug, Clone)] +pub struct DiscoveredInstall { + pub layout: InstallLayout, + /// `None` when the binary could not be executed, which is normal on a + /// machine of a different architecture. + pub version: Option, +} + +/// Every place a Factorio install is known to sit. +/// +/// This is the union of the candidate lists found across the four consumer +/// repos plus a stray benchmark script. Each had a different subset, so each +/// found a different set of installs - which is the whole reason discovery is +/// worth doing once. +pub fn candidate_roots(home: &Path, env_bin: Option<&Path>) -> Vec { + let mut roots: Vec = Vec::new(); + if let Some(bin) = env_bin { + roots.push(bin.to_path_buf()); + } + roots.extend([ + home.join("Library/Application Support/Steam/steamapps/common/Factorio/factorio.app"), + PathBuf::from("/Applications/factorio.app"), + home.join(".steam/steam/steamapps/common/Factorio"), + home.join(".factorio"), + PathBuf::from("/opt/factorio"), + ]); + roots.dedup(); + roots +} + +/// Reads a version by running the binary. Returns `None` if it will not run. +pub fn read_version(binary: &Path) -> Option { + let output = std::process::Command::new(binary).arg("--version").output().ok()?; + parse_version_line(&String::from_utf8_lossy(&output.stdout)) +} + +/// Finds every install on this machine. +pub fn discover(home: &Path, env_bin: Option<&Path>) -> Vec { + candidate_roots(home, env_bin) + .iter() + .filter_map(|root| resolve_layout(root)) + .map(|layout| { + let version = read_version(&layout.binary); + DiscoveredInstall { layout, version } + }) + .collect() +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test install` +Expected: PASS, 8 tests. + +- [ ] **Step 5: Wire up the CLI** + +Replace `src/main.rs` entirely: + +```rust +use clap::{Parser, Subcommand}; +use factorio_oracle::install; +use std::path::PathBuf; + +#[derive(Parser)] +#[command(name = "factorio-oracle", version, about = "Ask a real Factorio install what it does")] +struct Cli { + #[command(subcommand)] + command: Command, +} + +#[derive(Subcommand)] +enum Command { + /// Discover Factorio installs on this machine + Installs { + #[command(subcommand)] + action: InstallsAction, + }, +} + +#[derive(Subcommand)] +enum InstallsAction { + /// Print every install found, as JSON + List, +} + +fn main() -> anyhow::Result<()> { + let cli = Cli::parse(); + match cli.command { + Command::Installs { action: InstallsAction::List } => { + let home = PathBuf::from(std::env::var("HOME").unwrap_or_default()); + let env_bin = std::env::var_os("FACTORIO_BIN").map(PathBuf::from); + let found = install::discover(&home, env_bin.as_deref()); + + let rows: Vec = found + .iter() + .map(|d| { + serde_json::json!({ + "root": d.layout.root, + "binary": d.layout.binary, + "dataDir": d.layout.data_dir, + "docDir": d.layout.doc_dir, + "version": d.version.as_ref().map(|v| format!("{}.{}.{}", v.major, v.minor, v.patch)), + "modFactorioVersion": d.version.as_ref().map(|v| v.major_minor()), + "buildLine": d.version.as_ref().map(|v| v.line.clone()), + }) + }) + .collect(); + + println!("{}", serde_json::to_string_pretty(&serde_json::json!({ "installs": rows }))?); + } + } + Ok(()) +} +``` + +- [ ] **Step 6: Verify it runs** + +Run: `cargo run -- installs list` +Expected: JSON with an `installs` array. On a machine with Factorio it lists at least one entry with a real `version` and `modFactorioVersion`. On a machine without, it prints `{"installs": []}` and exits 0. + +- [ ] **Step 7: Commit** + +```bash +git add src/install.rs src/main.rs +git commit -m "Discover every install rather than picking one + +The consumers target different Factorio versions on purpose, so enumerating +is the requirement and choosing is the caller's job." +``` + +--- + +### Task 5: The probe spec types + +**Files:** +- Create: `src/probe.rs` +- Modify: `src/lib.rs` + +**Interfaces:** +- Consumes: nothing. +- Produces: `pub enum Mode { DumpData, Create, Interactive, Preview, ReadOnly }`, `pub struct ModSpec { pub name: String, pub version: String, pub dependencies: Vec, pub control_lua: Option, pub control_lua_file: Option, pub data_lua: Option, pub data_final_fixes_lua: Option }`, `pub struct ProbeSpec { pub mode: Mode, pub r#mod: Option, pub literals: BTreeMap, pub timeout_seconds: Option, pub capture_active_mods: bool }`, and `impl ModSpec { pub fn dir_name(&self) -> String }`. + +- [ ] **Step 1: Write the failing test** + +Create `src/probe.rs` with only the test module: + +```rust +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn deserialises_a_minimal_dump_data_spec() { + let spec: ProbeSpec = serde_json::from_str(r#"{ "mode": "dump-data" }"#).unwrap(); + assert_eq!(spec.mode, Mode::DumpData); + assert!(spec.r#mod.is_none()); + assert!(spec.literals.is_empty()); + // On by default. A contaminated capture looks entirely normal, so the + // safe default records what loaded. + assert!(spec.capture_active_mods); + } + + #[test] + fn contamination_reporting_can_be_turned_off() { + let spec: ProbeSpec = + serde_json::from_str(r#"{ "mode": "create", "capture_active_mods": false }"#).unwrap(); + assert!(!spec.capture_active_mods); + } + + #[test] + fn deserialises_a_create_spec_with_a_mod() { + let json = r#"{ + "mode": "create", + "mod": { + "name": "bp_probe", + "version": "0.0.1", + "dependencies": ["base", "elevated-rails", "space-age"], + "control_lua": "script.on_init(function() end)" + }, + "literals": { "blueprint": "0eNq" }, + "timeout_seconds": 120 + }"#; + let spec: ProbeSpec = serde_json::from_str(json).unwrap(); + assert_eq!(spec.mode, Mode::Create); + let m = spec.r#mod.as_ref().unwrap(); + assert_eq!(m.name, "bp_probe"); + assert_eq!(m.dependencies, vec!["base", "elevated-rails", "space-age"]); + assert_eq!(spec.literals.get("blueprint").unwrap(), "0eNq"); + assert_eq!(spec.timeout_seconds, Some(120)); + } + + #[test] + fn mod_directory_name_carries_the_version_suffix() { + // Factorio requires _ and it must match info.json, or + // the mod is not loaded. + let m = ModSpec { + name: "bp_probe".into(), + version: "0.0.1".into(), + dependencies: vec![], + control_lua: None, + control_lua_file: None, + data_lua: None, + data_final_fixes_lua: None, + }; + assert_eq!(m.dir_name(), "bp_probe_0.0.1"); + } + + #[test] + fn every_mode_name_round_trips() { + for (text, mode) in [ + ("dump-data", Mode::DumpData), + ("create", Mode::Create), + ("interactive", Mode::Interactive), + ("preview", Mode::Preview), + ("read-only", Mode::ReadOnly), + ] { + let spec: ProbeSpec = + serde_json::from_str(&format!(r#"{{ "mode": "{text}" }}"#)).unwrap(); + assert_eq!(spec.mode, mode, "mode {text} did not round trip"); + } + } + + #[test] + fn rejects_an_unknown_mode() { + assert!(serde_json::from_str::(r#"{ "mode": "benchmark" }"#).is_err()); + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Add `pub mod probe;` to `src/lib.rs`, then run: + +Run: `cargo test probe` +Expected: FAIL to compile, with `cannot find type 'ProbeSpec'`. + +- [ ] **Step 3: Write minimal implementation** + +Insert above the test module in `src/probe.rs`: + +```rust +//! The JSON document a consumer hands in to describe a probe. + +use serde::Deserialize; +use std::collections::BTreeMap; +use std::path::PathBuf; + +/// How the game gets launched. The differences are not cosmetic: the success +/// predicate, whether a mod is generated, and the argument vector all differ. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum Mode { + /// `--dump-data`. No mod is generated; the mod directory exists to be empty. + DumpData, + /// `--create`. A generated mod writes a dump and errors out. + Create, + /// `--load-scenario`. Long running, with a human at the keyboard. + Interactive, + /// `--generate-map-preview`. No mod, and it exits 0 on success. + Preview, + /// No binary at all. Migrations and API docs are files on disk. + ReadOnly, +} + +/// The throwaway mod a probe runs. +#[derive(Debug, Clone, Deserialize)] +pub struct ModSpec { + pub name: String, + pub version: String, + #[serde(default)] + pub dependencies: Vec, + /// Consumer Lua, passed through untouched. + #[serde(default)] + pub control_lua: Option, + #[serde(default)] + pub control_lua_file: Option, + #[serde(default)] + pub data_lua: Option, + /// Prototype overrides belong here, not in `data_lua`. A probe mod declares + /// no dependencies, so its `data.lua` may run before `space-age`'s and the + /// prototype it wants to change will not exist yet - a silent no-op. + #[serde(default)] + pub data_final_fixes_lua: Option, +} + +impl ModSpec { + /// The on-disk directory name. Factorio requires `_`, and it + /// must match `info.json` or the mod is not loaded. + pub fn dir_name(&self) -> String { + format!("{}_{}", self.name, self.version) + } +} + +/// A probe, as handed in. +#[derive(Debug, Clone, Deserialize)] +pub struct ProbeSpec { + pub mode: Mode, + #[serde(default, rename = "mod")] + pub r#mod: Option, + /// Values injected as Lua locals above the consumer's control script. + #[serde(default)] + pub literals: BTreeMap, + #[serde(default)] + pub timeout_seconds: Option, + /// On by default. A contaminated capture looks entirely normal, so the + /// safe default is to record what loaded and let a consumer opt out. + #[serde(default = "default_true")] + pub capture_active_mods: bool, +} + +fn default_true() -> bool { + true +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test probe` +Expected: PASS, 5 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/probe.rs src/lib.rs +git commit -m "Define the probe spec, with five launch modes + +The modes differ in the success predicate, whether a mod is generated, and +the argument vector, so they are an enum rather than a flag." +``` + +--- + +### Task 6: Inject literals as Lua long brackets + +The one place the tool touches a consumer's Lua. It exists because embedding a base64 blueprint string in a quoted Lua string breaks on the first inner quote. + +**Files:** +- Create: `src/lua.rs` +- Modify: `src/lib.rs` + +**Interfaces:** +- Consumes: nothing. +- Produces: `pub fn long_bracket(value: &str) -> String` and `pub fn build_literals_prelude(literals: &BTreeMap) -> String`. + +- [ ] **Step 1: Write the failing test** + +Create `src/lua.rs` with only the test module: + +```rust +#[cfg(test)] +mod tests { + use super::*; + use std::collections::BTreeMap; + + #[test] + fn wraps_a_plain_value_at_level_zero() { + assert_eq!(long_bracket("0eNqrVkrKT"), "[[0eNqrVkrKT]]"); + } + + #[test] + fn escalates_the_level_when_the_value_would_close_the_bracket() { + // A value containing "]]" would end the literal early. + assert_eq!(long_bracket("a]]b"), "[=[a]]b]=]"); + } + + #[test] + fn escalates_again_when_the_next_level_also_collides() { + assert_eq!(long_bracket("a]]b]=]c"), "[==[a]]b]=]c]==]"); + } + + #[test] + fn prelude_declares_one_local_per_entry() { + let mut literals = BTreeMap::new(); + literals.insert("blueprint".to_string(), "0eNq".to_string()); + assert_eq!( + build_literals_prelude(&literals), + "local blueprint = [[0eNq]]\n" + ); + } + + #[test] + fn prelude_is_sorted_and_therefore_deterministic() { + let mut literals = BTreeMap::new(); + literals.insert("zebra".to_string(), "z".to_string()); + literals.insert("alpha".to_string(), "a".to_string()); + assert_eq!( + build_literals_prelude(&literals), + "local alpha = [[a]]\nlocal zebra = [[z]]\n" + ); + } + + #[test] + fn prelude_is_empty_when_there_are_no_literals() { + assert_eq!(build_literals_prelude(&BTreeMap::new()), ""); + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Add `pub mod lua;` to `src/lib.rs`, then run: + +Run: `cargo test lua` +Expected: FAIL to compile, with `cannot find function 'long_bracket'`. + +- [ ] **Step 3: Write minimal implementation** + +Insert above the test module in `src/lua.rs`: + +```rust +//! The only place this tool writes Lua on a consumer's behalf. +//! +//! Consumer Lua is otherwise opaque: never templated, escaped, rewritten, or +//! wrapped. Wrapping in `script.on_init` would be a convenient default and +//! would make an `on_tick` probe with registered commands impossible. + +use std::collections::BTreeMap; + +/// Wraps a value in a Lua long bracket at a level that cannot collide with the +/// value's own contents. +/// +/// A base64 blueprint string in a quoted Lua string breaks on the first inner +/// quote. A long bracket takes the value verbatim. +pub fn long_bracket(value: &str) -> String { + let mut level = 0usize; + loop { + let eq = "=".repeat(level); + if !value.contains(&format!("]{eq}]")) { + return format!("[{eq}[{value}]{eq}]"); + } + level += 1; + } +} + +/// Builds the `local = ` lines that precede a consumer's +/// control script. +/// +/// Sorted, because the output must be identical between runs. +pub fn build_literals_prelude(literals: &BTreeMap) -> String { + literals + .iter() + .map(|(name, value)| format!("local {} = {}\n", name, long_bracket(value))) + .collect() +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test lua` +Expected: PASS, 6 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/lua.rs src/lib.rs +git commit -m "Inject literals as Lua long brackets, escalating the level as needed + +Embedding a base64 blueprint string in a quoted Lua string breaks on the +first inner quote. The bracket level escalates so a value cannot close its +own literal." +``` + +--- + +### Task 7: Scaffold the mod files + +**Files:** +- Create: `src/scaffold.rs` +- Modify: `src/lib.rs` + +**Interfaces:** +- Consumes: `ModSpec` from Task 5. +- Produces: `pub fn build_info_json(spec: &ModSpec, mod_factorio_version: &str) -> serde_json::Value`, `pub fn build_mod_list(mod_name: Option<&str>) -> serde_json::Value`, `pub fn build_config_ini(write_data: &Path) -> String`, and `pub const ACTIVE_MODS_PRELUDE: &str`. + +- [ ] **Step 1: Write the failing test** + +Create `src/scaffold.rs` with only the test module: + +```rust +#[cfg(test)] +mod tests { + use super::*; + use crate::probe::ModSpec; + + fn sample() -> ModSpec { + ModSpec { + name: "bp_probe".into(), + version: "0.0.1".into(), + dependencies: vec!["base".into()], + control_lua: Some("script.on_init(function() end)".into()), + control_lua_file: None, + data_lua: None, + data_final_fixes_lua: None, + } + } + + #[test] + fn info_json_takes_the_version_from_the_binary() { + let info = build_info_json(&sample(), "2.0"); + assert_eq!(info["factorio_version"], "2.0"); + assert_eq!(info["name"], "bp_probe"); + assert_eq!(info["version"], "0.0.1"); + assert_eq!(info["dependencies"][0], "base"); + } + + #[test] + fn info_json_version_is_never_hardcoded() { + // The same mod against a different binary must declare a different + // version. Getting this wrong makes Factorio skip the mod in silence. + let a = build_info_json(&sample(), "2.0"); + let b = build_info_json(&sample(), "2.1"); + assert_ne!(a["factorio_version"], b["factorio_version"]); + } + + #[test] + fn mod_list_with_no_probe_enables_only_base() { + // This is the dump-data case: the directory exists to be empty of user + // mods, because mods rewrite prototypes freely. + let list = build_mod_list(None); + assert_eq!(list["mods"].as_array().unwrap().len(), 1); + assert_eq!(list["mods"][0]["name"], "base"); + assert_eq!(list["mods"][0]["enabled"], true); + } + + #[test] + fn mod_list_with_a_probe_enables_both() { + let list = build_mod_list(Some("bp_probe")); + let names: Vec<&str> = list["mods"] + .as_array() + .unwrap() + .iter() + .map(|m| m["name"].as_str().unwrap()) + .collect(); + assert_eq!(names, vec!["base", "bp_probe"]); + } + + #[test] + fn config_ini_isolates_writes_and_reads_the_bundled_data() { + let ini = build_config_ini(Path::new("/tmp/work/write")); + assert!(ini.contains("write-data=/tmp/work/write")); + // The portable token for the install's own data directory. + assert!(ini.contains("read-data=__PATH__executable__/../data")); + assert!(ini.starts_with("[path]")); + } + + #[test] + fn the_active_mods_prelude_writes_its_own_file() { + // It must not collide with the consumer's dump file name. + assert!(ACTIVE_MODS_PRELUDE.contains("oracle-active-mods.json")); + assert!(ACTIVE_MODS_PRELUDE.contains("script.active_mods")); + } + + #[test] + fn the_active_mods_prelude_registers_no_event() { + // Measured on 2.1.14: a toplevel write works, and an on_init in a + // prelude is silently discarded when the consumer registers one too. + // Any event registration here is a regression, so assert their absence. + assert!(!ACTIVE_MODS_PRELUDE.contains("on_init")); + assert!(!ACTIVE_MODS_PRELUDE.contains("on_nth_tick")); + assert!(!ACTIVE_MODS_PRELUDE.contains("on_event")); + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Add `pub mod scaffold;` to `src/lib.rs`, then run: + +Run: `cargo test scaffold` +Expected: FAIL to compile, with `cannot find function 'build_info_json'`. + +- [ ] **Step 3: Write minimal implementation** + +Insert above the test module in `src/scaffold.rs`: + +```rust +//! Writing the throwaway mod's files and the isolated config. + +use crate::probe::ModSpec; +use serde_json::{json, Value}; +use std::path::Path; + +/// A Lua prelude that records which mods actually loaded. +/// +/// Reading `script.active_mods` from inside the game is more reliable than +/// grepping Factorio's stdout for "Loading mod", which only works for +/// `--dump-data`. On by default: mods rewrite prototypes freely, so a +/// contaminated capture describes one person's game rather than Factorio - and +/// it looks entirely normal, which is the failure nobody notices. +/// +/// **Registers no event at all.** Measured 2026-08-16 on 2.1.14: +/// `helpers.write_file` works at `control.lua` toplevel with no event, and +/// `script.active_mods` is populated there. +/// +/// That matters because `script.on_init` takes exactly one handler. The same +/// measurement proved it: an `instrument-control.lua` that registered `on_init` +/// had its handler silently discarded when `control.lua` registered one too - +/// no error, the handler simply never ran. 17 of 18 probes in +/// factorio-blueprint-editor register an `on_init`, so any prelude using one +/// would vanish. A toplevel write has no collision surface whatsoever, and does +/// not wait a tick. +/// +/// The reported set deliberately includes the probe's own throwaway mod - the +/// measurement confirmed `oracle_instr: 0.0.1` appears alongside base and the +/// DLC. That is proof the mod loaded, which is the thing most worth knowing +/// when a run produces no dump. +pub const ACTIVE_MODS_PRELUDE: &str = r#" +helpers.write_file("oracle-active-mods.json", helpers.table_to_json(script.active_mods)) +"#; + +/// The mod's `info.json`. +/// +/// `mod_factorio_version` is always derived from the binary being run. A mod +/// declaring 2.1 against a 2.0.x binary is skipped in silence: the run ends +/// with no dump, and nothing in Factorio's output names the cause. +pub fn build_info_json(spec: &ModSpec, mod_factorio_version: &str) -> Value { + json!({ + "name": spec.name, + "version": spec.version, + "title": spec.name, + "author": "factorio-oracle", + "factorio_version": mod_factorio_version, + "dependencies": spec.dependencies, + }) +} + +/// The `mod-list.json` for an isolated mod directory. +/// +/// With `None`, only `base` is enabled and no probe mod exists. That is the +/// `--dump-data` case, where the directory's whole job is to contain no user +/// mods: mods rewrite prototypes freely, so a capture that loads them describes +/// one person's game rather than Factorio. +pub fn build_mod_list(mod_name: Option<&str>) -> Value { + let mut mods = vec![json!({ "name": "base", "enabled": true })]; + if let Some(name) = mod_name { + mods.push(json!({ "name": name, "enabled": true })); + } + json!({ "mods": mods }) +} + +/// An isolated `config.ini`. +/// +/// `read-data` points at the install's bundled data through Factorio's own +/// portable token, and `write-data` at a scratch directory that started empty. +/// That second half is what makes a stale dump from an earlier capture +/// impossible to pick up by accident. +pub fn build_config_ini(write_data: &Path) -> String { + format!( + "[path]\nread-data=__PATH__executable__/../data\nwrite-data={}\n", + write_data.display() + ) +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test scaffold` +Expected: PASS, 6 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/scaffold.rs src/lib.rs +git commit -m "Build the mod files and an isolated config + +write-data points at a scratch directory that started empty, so a leftover +dump from an older capture cannot be mistaken for this run's output." +``` + +--- + +### Task 8: Build the argument vector per mode + +**Files:** +- Create: `src/args.rs` +- Modify: `src/lib.rs` + +**Interfaces:** +- Consumes: nothing. +- Produces: `pub enum Launch` with variants `DumpData`, `Create`, `Interactive`, `Preview`, and `pub fn build_args(launch: &Launch) -> Vec`. + +- [ ] **Step 1: Write the failing test** + +Create `src/args.rs` with only the test module: + +```rust +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn dump_data_passes_only_the_mod_directory_and_config() { + let args = build_args(&Launch::DumpData { + mod_dir: "/w/mods".into(), + config: "/w/config.ini".into(), + }); + assert_eq!( + args, + vec!["--dump-data", "--mod-directory", "/w/mods", "--config", "/w/config.ini"] + ); + } + + #[test] + fn create_passes_map_gen_settings_when_there_are_any() { + let args = build_args(&Launch::Create { + save: "/w/probe.zip".into(), + map_gen: Some("/w/map-gen.json".into()), + seed: None, + mod_dir: "/w/mods".into(), + config: "/w/config.ini".into(), + }); + assert!(args.contains(&"--map-gen-settings".to_string())); + assert_eq!(args[0], "--create"); + assert_eq!(args[1], "/w/probe.zip"); + } + + #[test] + fn create_omits_map_gen_settings_when_there_are_none() { + // Measured 2026-08-16 on 2.1.14: --create works with no settings file. + // The consumer repos always passed one out of habit, not necessity. + let args = build_args(&Launch::Create { + save: "/w/probe.zip".into(), + map_gen: None, + seed: None, + mod_dir: "/w/mods".into(), + config: "/w/config.ini".into(), + }); + assert!(!args.contains(&"--map-gen-settings".to_string())); + assert!(args.contains(&"--mod-directory".to_string())); + } + + #[test] + fn create_also_passes_the_seed_on_the_command_line() { + // Measured: --map-gen-seed overrides the seed inside the settings file. + // Both come from one field so they agree, and a caller that omits the + // seed gets neither channel. + let args = build_args(&Launch::Create { + save: "/w/probe.zip".into(), + map_gen: Some("/w/map-gen.json".into()), + seed: Some(123456), + mod_dir: "/w/mods".into(), + config: "/w/config.ini".into(), + }); + assert!(args.contains(&"--map-gen-seed".to_string())); + assert!(args.contains(&"123456".to_string())); + + let without = build_args(&Launch::Create { + save: "/w/probe.zip".into(), + map_gen: Some("/w/map-gen.json".into()), + seed: None, + mod_dir: "/w/mods".into(), + config: "/w/config.ini".into(), + }); + assert!(!without.contains(&"--map-gen-seed".to_string())); + } + + #[test] + fn interactive_loads_a_scenario_and_never_creates() { + let args = build_args(&Launch::Interactive { + scenario: "base/freeplay".into(), + mod_dir: "/w/mods".into(), + config: "/w/config.ini".into(), + }); + assert!(args.contains(&"--load-scenario".to_string())); + assert!(args.contains(&"base/freeplay".to_string())); + assert!(!args.contains(&"--create".to_string())); + } + + #[test] + fn preview_takes_an_output_path_and_no_mod_directory() { + let args = build_args(&Launch::Preview { + out: "/w/preview.png".into(), + map_gen: "/w/map-gen.json".into(), + planet: Some("nauvis".into()), + seed: Some(123456), + size: Some(1024), + }); + assert_eq!(args[0], "--generate-map-preview"); + assert_eq!(args[1], "/w/preview.png"); + assert!(args.contains(&"--map-preview-planet".to_string())); + assert!(args.contains(&"nauvis".to_string())); + assert!(args.contains(&"123456".to_string())); + assert!(!args.contains(&"--mod-directory".to_string())); + } + + #[test] + fn preview_omits_optional_flags_that_were_not_set() { + let args = build_args(&Launch::Preview { + out: "/w/preview.png".into(), + map_gen: "/w/map-gen.json".into(), + planet: None, + seed: None, + size: None, + }); + assert!(!args.contains(&"--map-preview-planet".to_string())); + assert!(!args.contains(&"--map-gen-seed".to_string())); + assert!(!args.contains(&"--map-preview-size".to_string())); + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Add `pub mod args;` to `src/lib.rs`, then run: + +Run: `cargo test args` +Expected: FAIL to compile, with `cannot find type 'Launch'`. + +- [ ] **Step 3: Write minimal implementation** + +Insert above the test module in `src/args.rs`: + +```rust +//! The argument vector, which differs per mode. + +use std::path::PathBuf; + +/// What to launch, carrying exactly the paths that mode needs. +#[derive(Debug, Clone)] +pub enum Launch { + DumpData { + mod_dir: PathBuf, + config: PathBuf, + }, + Create { + save: PathBuf, + /// `None` when the caller supplied no settings. Measured 2026-08-16 on + /// 2.1.14: `--create` succeeds with no settings file at all. + map_gen: Option, + /// Also written into the settings file. Measured: `--map-gen-seed` + /// overrides the file's seed, so both come from one field and agree, + /// which makes the precedence irrelevant. + seed: Option, + mod_dir: PathBuf, + config: PathBuf, + }, + Interactive { + scenario: String, + mod_dir: PathBuf, + config: PathBuf, + }, + Preview { + out: PathBuf, + map_gen: PathBuf, + planet: Option, + seed: Option, + size: Option, + }, +} + +fn s(path: &std::path::Path) -> String { + path.display().to_string() +} + +/// Builds the argument vector for a launch. +pub fn build_args(launch: &Launch) -> Vec { + match launch { + Launch::DumpData { mod_dir, config } => vec![ + "--dump-data".into(), + "--mod-directory".into(), + s(mod_dir), + "--config".into(), + s(config), + ], + Launch::Create { + save, + map_gen, + seed, + mod_dir, + config, + } => { + let mut args = vec!["--create".into(), s(save)]; + // Optional. Measured 2026-08-16 on 2.1.14: --create generates a map, + // loads the mod and produces a dump with no settings file at all. + // The consumer repos always passed one out of habit. + if let Some(map_gen) = map_gen { + args.push("--map-gen-settings".into()); + args.push(s(map_gen)); + } + // The seed also goes inside the settings file. Measured: the flag + // overrides the file, so a tool writing only the file would be + // silently overridden by a caller's flag. Both come from one field + // and therefore agree, which makes the precedence irrelevant. + if let Some(seed) = seed { + args.push("--map-gen-seed".into()); + args.push(seed.to_string()); + } + args.extend([ + "--mod-directory".into(), + s(mod_dir), + "--config".into(), + s(config), + ]); + args + } + Launch::Interactive { + scenario, + mod_dir, + config, + } => vec![ + "--load-scenario".into(), + scenario.clone(), + "--mod-directory".into(), + s(mod_dir), + "--config".into(), + s(config), + ], + Launch::Preview { + out, + map_gen, + planet, + seed, + size, + } => { + let mut args = vec![ + "--generate-map-preview".into(), + s(out), + "--map-gen-settings".into(), + s(map_gen), + ]; + if let Some(planet) = planet { + args.push("--map-preview-planet".into()); + args.push(planet.clone()); + } + if let Some(seed) = seed { + args.push("--map-gen-seed".into()); + args.push(seed.to_string()); + } + if let Some(size) = size { + args.push("--map-preview-size".into()); + args.push(size.to_string()); + } + args + } + } +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test args` +Expected: PASS, 5 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/args.rs src/lib.rs +git commit -m "Build the argument vector per mode + +--map-gen-settings is required for --create even when nothing reads it, and +preview takes a different vector entirely with no mod directory." +``` + +--- + +### Task 9: Decide success, per mode + +The rule that one global predicate would get wrong. `error("DUMPED-OK")` makes the game exit non-zero and that is success, but `--generate-map-preview` exits 0 on success, and for `--dump-data` a non-zero exit is real information. + +**Files:** +- Create: `src/outcome.rs` +- Modify: `src/lib.rs` + +**Interfaces:** +- Consumes: `Mode` from Task 5. +- Produces: `pub struct RunFacts { pub exit_code: Option, pub dump_exists: bool, pub sentinel_seen: bool }`, `pub enum Outcome { Ok, Failed(String) }`, and `pub fn evaluate(mode: Mode, facts: &RunFacts) -> Outcome`. + +- [ ] **Step 1: Write the failing test** + +Create `src/outcome.rs` with only the test module: + +```rust +#[cfg(test)] +mod tests { + use super::*; + use crate::probe::Mode; + + fn facts(exit: Option, dump: bool, sentinel: bool) -> RunFacts { + RunFacts { exit_code: exit, dump_exists: dump, sentinel_seen: sentinel } + } + + #[test] + fn create_succeeds_on_a_non_zero_exit_when_the_dump_exists() { + // error("DUMPED-OK") is how the probe exits. Non-zero is success here. + assert_eq!(evaluate(Mode::Create, &facts(Some(1), true, true)), Outcome::Ok); + } + + #[test] + fn create_fails_when_no_dump_was_written() { + let out = evaluate(Mode::Create, &facts(Some(1), false, false)); + assert!(matches!(out, Outcome::Failed(_))); + } + + #[test] + fn dump_data_fails_on_a_non_zero_exit_even_if_a_dump_is_present() { + // A non-zero exit is real information here, and a stale dump from an + // earlier capture can be sitting in a discovered directory. + let out = evaluate(Mode::DumpData, &facts(Some(1), true, false)); + assert!(matches!(out, Outcome::Failed(_))); + } + + #[test] + fn dump_data_succeeds_on_exit_zero_with_a_dump() { + assert_eq!(evaluate(Mode::DumpData, &facts(Some(0), true, false)), Outcome::Ok); + } + + #[test] + fn dump_data_fails_on_exit_zero_with_no_dump() { + let out = evaluate(Mode::DumpData, &facts(Some(0), false, false)); + assert!(matches!(out, Outcome::Failed(_))); + } + + #[test] + fn preview_requires_exit_zero_and_the_file() { + assert_eq!(evaluate(Mode::Preview, &facts(Some(0), true, false)), Outcome::Ok); + assert!(matches!(evaluate(Mode::Preview, &facts(Some(1), true, false)), Outcome::Failed(_))); + assert!(matches!(evaluate(Mode::Preview, &facts(Some(0), false, false)), Outcome::Failed(_))); + } + + #[test] + fn interactive_always_succeeds_because_the_consumer_judges_it() { + // A session can end any way a person likes, and only the consumer knows + // whether the samples it collected are usable. + assert_eq!(evaluate(Mode::Interactive, &facts(Some(0), false, false)), Outcome::Ok); + assert_eq!(evaluate(Mode::Interactive, &facts(None, false, false)), Outcome::Ok); + } + + #[test] + fn read_only_never_runs_anything() { + assert_eq!(evaluate(Mode::ReadOnly, &facts(None, false, false)), Outcome::Ok); + } + + #[test] + fn a_missing_exit_code_fails_the_modes_that_need_one() { + // No exit code means the process was killed, which is how a timeout ends. + assert!(matches!(evaluate(Mode::DumpData, &facts(None, true, false)), Outcome::Failed(_))); + assert!(matches!(evaluate(Mode::Preview, &facts(None, true, false)), Outcome::Failed(_))); + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Add `pub mod outcome;` to `src/lib.rs`, then run: + +Run: `cargo test outcome` +Expected: FAIL to compile, with `cannot find type 'RunFacts'`. + +- [ ] **Step 3: Write minimal implementation** + +Insert above the test module in `src/outcome.rs`: + +```rust +//! Deciding whether a run succeeded. The rule is per mode, not global. + +use crate::probe::Mode; + +/// What was observed after the process ended. +#[derive(Debug, Clone)] +pub struct RunFacts { + /// `None` when the process was killed, which is how a timeout ends. + pub exit_code: Option, + pub dump_exists: bool, + /// Whether `DUMPED-OK` appeared in stderr. Reported rather than required, + /// because it distinguishes "the mod ran and finished" from "the mod + /// crashed" - a check no existing probe makes. + pub sentinel_seen: bool, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Outcome { + Ok, + Failed(String), +} + +/// Applies the mode's success rule. +/// +/// One global rule would get two of the five modes wrong. `error("DUMPED-OK")` +/// makes Factorio exit non-zero and that is success, so `create` keys off the +/// dump. `--generate-map-preview` exits 0 on success. And for `--dump-data` a +/// non-zero exit is the diagnostic, so ignoring it would mean debugging a +/// missing file when the real message was a prototype error in the log. +pub fn evaluate(mode: Mode, facts: &RunFacts) -> Outcome { + match mode { + Mode::ReadOnly | Mode::Interactive => Outcome::Ok, + + Mode::Create => { + if facts.dump_exists { + Outcome::Ok + } else { + Outcome::Failed( + "no dump was written. The most common cause is a factorio_version \ + mismatch, which makes Factorio skip the mod in silence." + .to_string(), + ) + } + } + + Mode::DumpData => match facts.exit_code { + Some(0) if facts.dump_exists => Outcome::Ok, + Some(0) => Outcome::Failed("factorio exited 0 but wrote no dump".to_string()), + Some(code) => Outcome::Failed(format!("factorio exited {code}")), + None => Outcome::Failed("factorio was killed before it exited".to_string()), + }, + + Mode::Preview => match facts.exit_code { + Some(0) if facts.dump_exists => Outcome::Ok, + Some(0) => Outcome::Failed("factorio exited 0 but wrote no preview".to_string()), + Some(code) => Outcome::Failed(format!("factorio exited {code}")), + None => Outcome::Failed("factorio was killed before it exited".to_string()), + }, + } +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test outcome` +Expected: PASS, 9 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/outcome.rs src/lib.rs +git commit -m "Decide success per mode rather than globally + +DUMPED-OK exits non-zero and that is success; map preview exits 0 on +success; and for --dump-data the exit code is the diagnostic. One rule +would get two of the five wrong." +``` + +--- + +### Task 10: The spawn boundary, with a fake game + +**Files:** +- Create: `src/spawn.rs` +- Modify: `src/lib.rs` + +**Interfaces:** +- Consumes: nothing. +- Produces: `pub struct SpawnResult { pub exit_code: Option, pub stdout: String, pub stderr: String }`, `pub trait Spawner { fn run(&self, binary: &Path, args: &[String], timeout: Option) -> anyhow::Result; }`, `pub struct RealSpawner`, and `pub fn tail(text: &str, bytes: usize) -> String`. + +- [ ] **Step 1: Write the failing test** + +Create `src/spawn.rs` with only the test module: + +```rust +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn tail_returns_the_last_bytes_not_the_first() { + // The tail of Factorio's output is the only diagnostic there is when a + // run produces no dump, so a JSON-out CLI must carry it. + let text: String = (0..100).map(|i| format!("line {i}\n")).collect(); + let out = tail(&text, 40); + assert!(out.ends_with("line 99\n")); + assert!(!out.contains("line 0\n")); + assert!(out.len() <= 40 + 8); + } + + #[test] + fn tail_returns_short_text_unchanged() { + assert_eq!(tail("short", 4000), "short"); + } + + #[test] + fn tail_does_not_split_a_multibyte_character() { + let text = "aaaa\u{1F600}"; + let out = tail(text, 5); + assert!(out.chars().count() > 0); + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Add `pub mod spawn;` to `src/lib.rs`, then run: + +Run: `cargo test spawn` +Expected: FAIL to compile, with `cannot find function 'tail'`. + +- [ ] **Step 3: Write minimal implementation** + +Insert above the test module in `src/spawn.rs`: + +```rust +//! The one boundary that touches processes, kept behind a trait so tests can +//! substitute a fake game. + +use std::path::Path; +use std::time::{Duration, Instant}; + +#[derive(Debug, Clone, Default)] +pub struct SpawnResult { + /// `None` when the process was killed, which is how a timeout ends. + pub exit_code: Option, + pub stdout: String, + pub stderr: String, +} + +pub trait Spawner { + fn run( + &self, + binary: &Path, + args: &[String], + timeout: Option, + ) -> anyhow::Result; +} + +/// Returns at most `bytes` from the end of `text`, on a character boundary. +pub fn tail(text: &str, bytes: usize) -> String { + if text.len() <= bytes { + return text.to_string(); + } + let mut start = text.len() - bytes; + while start < text.len() && !text.is_char_boundary(start) { + start += 1; + } + text[start..].to_string() +} + +/// Runs the real game. +pub struct RealSpawner; + +impl Spawner for RealSpawner { + fn run( + &self, + binary: &Path, + args: &[String], + timeout: Option, + ) -> anyhow::Result { + use std::process::{Command, Stdio}; + + let mut child = Command::new(binary) + .args(args) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn()?; + + // No consumer repo has a timeout today, so a hung game hangs the + // capture forever. Polling is enough here: a probe run is seconds, and + // avoiding an async runtime keeps the dependency surface small. + let deadline = timeout.map(|t| Instant::now() + t); + loop { + if let Some(status) = child.try_wait()? { + let output = child.wait_with_output()?; + return Ok(SpawnResult { + exit_code: status.code(), + stdout: String::from_utf8_lossy(&output.stdout).into_owned(), + stderr: String::from_utf8_lossy(&output.stderr).into_owned(), + }); + } + if let Some(deadline) = deadline { + if Instant::now() >= deadline { + child.kill()?; + let output = child.wait_with_output()?; + return Ok(SpawnResult { + exit_code: None, + stdout: String::from_utf8_lossy(&output.stdout).into_owned(), + stderr: String::from_utf8_lossy(&output.stderr).into_owned(), + }); + } + } + std::thread::sleep(Duration::from_millis(50)); + } + } +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test spawn` +Expected: PASS, 3 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/spawn.rs src/lib.rs +git commit -m "Add the spawn boundary, with a timeout none of the consumers has + +A hung game currently hangs a capture forever in all three repos. The trait +is what lets a fake game assert the argument vector in tests." +``` + +--- + +### Task 11: Wire it together and add `run` + +**Files:** +- Create: `src/run.rs` +- Modify: `src/lib.rs` +- Modify: `src/main.rs` + +**Interfaces:** +- Consumes: everything from Tasks 2 through 10. +- Produces: `pub struct RunRequest { pub spec: ProbeSpec, pub layout: InstallLayout, pub version: VersionInfo, pub work_dir: PathBuf, pub map_gen_settings: serde_json::Value }` and `pub fn run_probe(request: &RunRequest, spawner: &dyn Spawner) -> anyhow::Result`. + +- [ ] **Step 1: Write the failing test** + +Create `src/run.rs` with only the test module: + +```rust +#[cfg(test)] +mod tests { + use super::*; + use crate::probe::{Mode, ModSpec}; + use std::cell::RefCell; + use std::fs; + use tempfile::tempdir; + + /// A fake game. It asserts the argument vector, writes the dump a real game + /// would have written, and returns the non-zero exit that DUMPED-OK causes. + struct FakeGame { + write_dump_to: PathBuf, + seen_args: RefCell>, + } + + impl Spawner for FakeGame { + fn run( + &self, + _binary: &Path, + args: &[String], + _timeout: Option, + ) -> anyhow::Result { + *self.seen_args.borrow_mut() = args.to_vec(); + fs::create_dir_all(self.write_dump_to.parent().unwrap())?; + fs::write(&self.write_dump_to, br#"{"answer":42}"#)?; + Ok(SpawnResult { + exit_code: Some(1), + stdout: String::new(), + stderr: "control.lua:13: DUMPED-OK".into(), + }) + } + } + + fn layout_in(dir: &Path) -> InstallLayout { + let binary = dir.join("factorio"); + fs::write(&binary, b"").unwrap(); + fs::create_dir_all(dir.join("data")).unwrap(); + InstallLayout { + root: dir.to_path_buf(), + binary, + data_dir: dir.join("data"), + doc_dir: dir.join("doc-html"), + } + } + + fn version() -> VersionInfo { + crate::version::parse_version_line("Version: 2.0.77 (build 84539, mac-arm64, full)").unwrap() + } + + #[test] + fn a_create_run_scaffolds_the_mod_and_reports_success() { + let install = tempdir().unwrap(); + let work = tempdir().unwrap(); + + let spec = ProbeSpec { + mode: Mode::Create, + r#mod: Some(ModSpec { + name: "bp_probe".into(), + version: "0.0.1".into(), + dependencies: vec!["base".into()], + control_lua: Some("script.on_init(function() end)".into()), + control_lua_file: None, + data_lua: None, + data_final_fixes_lua: None, + }), + literals: BTreeMap::new(), + timeout_seconds: Some(60), + capture_active_mods: false, + }; + + let request = RunRequest { + spec, + layout: layout_in(install.path()), + version: version(), + work_dir: work.path().to_path_buf(), + map_gen_settings: Some(serde_json::json!({ "seed": 123456 })), + }; + + let fake = FakeGame { + write_dump_to: work.path().join("write/script-output/oracle-dump.json"), + seen_args: RefCell::new(vec![]), + }; + + let result = run_probe(&request, &fake).unwrap(); + + assert_eq!(result["ok"], true); + assert_eq!(result["sentinelSeen"], true); + assert_eq!(result["exitCode"], 1); + + // The mod was scaffolded with the version derived from the binary. + let info: serde_json::Value = serde_json::from_str( + &fs::read_to_string(work.path().join("mods/bp_probe_0.0.1/info.json")).unwrap(), + ) + .unwrap(); + assert_eq!(info["factorio_version"], "2.0"); + + // The consumer's Lua reached disk untouched. + let control = + fs::read_to_string(work.path().join("mods/bp_probe_0.0.1/control.lua")).unwrap(); + assert_eq!(control, "script.on_init(function() end)"); + + // --map-gen-settings is always passed for create, and the seed reaches + // the game through both channels from the single map_gen_settings field. + let args = fake.seen_args.borrow(); + assert!(args.contains(&"--map-gen-settings".to_string())); + assert!(args.contains(&"--map-gen-seed".to_string())); + assert!(args.contains(&"123456".to_string())); + + let written: serde_json::Value = serde_json::from_str( + &fs::read_to_string(work.path().join("map-gen-settings.json")).unwrap(), + ) + .unwrap(); + assert_eq!(written["seed"], 123456); + } + + #[test] + fn literals_are_prepended_above_the_consumer_lua() { + let install = tempdir().unwrap(); + let work = tempdir().unwrap(); + let mut literals = BTreeMap::new(); + literals.insert("blueprint".to_string(), "0eNq".to_string()); + + let spec = ProbeSpec { + mode: Mode::Create, + r#mod: Some(ModSpec { + name: "p".into(), + version: "0.0.1".into(), + dependencies: vec![], + control_lua: Some("game.print(blueprint)".into()), + control_lua_file: None, + data_lua: None, + data_final_fixes_lua: None, + }), + literals, + timeout_seconds: None, + capture_active_mods: false, + }; + + let request = RunRequest { + spec, + layout: layout_in(install.path()), + version: version(), + work_dir: work.path().to_path_buf(), + map_gen_settings: Some(serde_json::json!({})), + }; + let fake = FakeGame { + write_dump_to: work.path().join("write/script-output/oracle-dump.json"), + seen_args: RefCell::new(vec![]), + }; + run_probe(&request, &fake).unwrap(); + + let control = fs::read_to_string(work.path().join("mods/p_0.0.1/control.lua")).unwrap(); + assert_eq!(control, "local blueprint = [[0eNq]]\ngame.print(blueprint)"); + } + + #[test] + fn a_dump_data_run_writes_no_mod() { + let install = tempdir().unwrap(); + let work = tempdir().unwrap(); + let spec = ProbeSpec { + mode: Mode::DumpData, + r#mod: None, + literals: BTreeMap::new(), + timeout_seconds: None, + capture_active_mods: false, + }; + let request = RunRequest { + spec, + layout: layout_in(install.path()), + version: version(), + work_dir: work.path().to_path_buf(), + map_gen_settings: Some(serde_json::json!({})), + }; + + struct CleanExit { + dump: PathBuf, + } + impl Spawner for CleanExit { + fn run(&self, _b: &Path, _a: &[String], _t: Option) -> anyhow::Result { + fs::create_dir_all(self.dump.parent().unwrap())?; + fs::write(&self.dump, b"{}")?; + Ok(SpawnResult { exit_code: Some(0), ..Default::default() }) + } + } + let fake = CleanExit { dump: work.path().join("write/script-output/data-raw-dump.json") }; + let result = run_probe(&request, &fake).unwrap(); + + assert_eq!(result["ok"], true); + // The mod directory exists, and is empty of mods. That is its whole job. + assert!(work.path().join("mods/mod-list.json").is_file()); + assert!(!work.path().join("mods").read_dir().unwrap().any(|e| { + e.unwrap().file_name().to_string_lossy().contains('_') + })); + } + + #[test] + fn a_failed_run_carries_the_output_tail() { + let install = tempdir().unwrap(); + let work = tempdir().unwrap(); + let spec = ProbeSpec { + mode: Mode::Create, + r#mod: Some(ModSpec { + name: "p".into(), + version: "0.0.1".into(), + dependencies: vec![], + control_lua: Some("".into()), + control_lua_file: None, + data_lua: None, + data_final_fixes_lua: None, + }), + literals: BTreeMap::new(), + timeout_seconds: None, + capture_active_mods: false, + }; + let request = RunRequest { + spec, + layout: layout_in(install.path()), + version: version(), + work_dir: work.path().to_path_buf(), + map_gen_settings: Some(serde_json::json!({})), + }; + + struct NoDump; + impl Spawner for NoDump { + fn run(&self, _b: &Path, _a: &[String], _t: Option) -> anyhow::Result { + Ok(SpawnResult { + exit_code: Some(1), + stdout: "Loading mod core 2.0.77".into(), + stderr: "something went wrong".into(), + }) + } + } + let result = run_probe(&request, &NoDump).unwrap(); + + assert_eq!(result["ok"], false); + assert!(result["error"].as_str().unwrap().contains("no dump")); + assert!(result["stderrTail"].as_str().unwrap().contains("something went wrong")); + // The mismatch that most often explains an empty dump is named outright. + assert_eq!(result["provenance"]["modFactorioVersion"], "2.0"); + assert!(result["provenance"]["buildLine"].as_str().unwrap().contains("2.0.77")); + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Add `pub mod run;` to `src/lib.rs`, then run: + +Run: `cargo test run` +Expected: FAIL to compile, with `cannot find type 'RunRequest'`. + +- [ ] **Step 3: Write minimal implementation** + +Insert above the test module in `src/run.rs`: + +```rust +//! Wiring the pure builders to disk and a spawner. + +use crate::args::{build_args, Launch}; +use crate::install::InstallLayout; +use crate::lua::build_literals_prelude; +use crate::outcome::{evaluate, Outcome, RunFacts}; +use crate::probe::{Mode, ProbeSpec}; +use crate::scaffold::{build_config_ini, build_info_json, build_mod_list, ACTIVE_MODS_PRELUDE}; +use crate::spawn::{tail, SpawnResult, Spawner}; +use crate::version::VersionInfo; +use anyhow::Context; +use serde_json::json; +use std::collections::BTreeMap; +use std::fs; +use std::path::{Path, PathBuf}; +use std::time::Duration; + +/// Everything a run needs. The caller resolves the install and the work +/// directory, so this function does no discovery of its own. +pub struct RunRequest { + pub spec: ProbeSpec, + pub layout: InstallLayout, + pub version: VersionInfo, + pub work_dir: PathBuf, + /// `None` when the caller wants the game's own defaults. Measured: a + /// `--create` run needs no settings file. + pub map_gen_settings: Option, +} + +/// The dump file a `--dump-data` run writes, named by the game. +const DUMP_DATA_FILE: &str = "data-raw-dump.json"; +/// The default dump name for a probe mod. +const PROBE_DUMP_FILE: &str = "oracle-dump.json"; +/// The preview image name. +const PREVIEW_FILE: &str = "preview.png"; + +fn read_control_lua(spec: &ProbeSpec) -> anyhow::Result { + let Some(m) = spec.r#mod.as_ref() else { + return Ok(String::new()); + }; + if let Some(inline) = m.control_lua.as_ref() { + return Ok(inline.clone()); + } + if let Some(path) = m.control_lua_file.as_ref() { + return fs::read_to_string(path) + .with_context(|| format!("reading control_lua_file {}", path.display())); + } + Ok(String::new()) +} + +/// Runs a probe and returns the result as JSON. +/// +/// The return value describes the work directory rather than a single dump. +/// That is deliberate: an interactive probe writes several files, appends to +/// some of them while a person plays, and can only be judged by the consumer. +pub fn run_probe(request: &RunRequest, spawner: &dyn Spawner) -> anyhow::Result { + let work = &request.work_dir; + let mod_dir = work.join("mods"); + let write_data = work.join("write"); + let script_output = write_data.join("script-output"); + let config_path = work.join("config.ini"); + let map_gen_path = work.join("map-gen-settings.json"); + + fs::create_dir_all(&mod_dir)?; + fs::create_dir_all(&script_output)?; + + // The isolated config is what makes a stale dump impossible: write-data + // points at a directory that started empty. + fs::write(&config_path, build_config_ini(&write_data))?; + if let Some(settings) = request.map_gen_settings.as_ref() { + fs::write(&map_gen_path, serde_json::to_string_pretty(settings)?)?; + } + + let mod_name = request.spec.r#mod.as_ref().map(|m| m.name.clone()); + fs::write( + mod_dir.join("mod-list.json"), + serde_json::to_string_pretty(&build_mod_list(mod_name.as_deref()))?, + )?; + + if let Some(m) = request.spec.r#mod.as_ref() { + let files = mod_dir.join(m.dir_name()); + fs::create_dir_all(&files)?; + fs::write( + files.join("info.json"), + serde_json::to_string_pretty(&build_info_json(m, &request.version.major_minor()))?, + )?; + + // Consumer Lua passes through untouched. The only additions are the + // literal locals, and the active-mods prelude when it was asked for. + let mut control = String::new(); + if request.spec.capture_active_mods { + control.push_str(ACTIVE_MODS_PRELUDE); + } + control.push_str(&build_literals_prelude(&request.spec.literals)); + control.push_str(&read_control_lua(&request.spec)?); + fs::write(files.join("control.lua"), control)?; + + if let Some(data_lua) = m.data_lua.as_ref() { + fs::write(files.join("data.lua"), data_lua)?; + } + if let Some(final_fixes) = m.data_final_fixes_lua.as_ref() { + fs::write(files.join("data-final-fixes.lua"), final_fixes)?; + } + } + + let (launch, expected_file) = match request.spec.mode { + Mode::DumpData => ( + Some(Launch::DumpData { + mod_dir: mod_dir.clone(), + config: config_path.clone(), + }), + script_output.join(DUMP_DATA_FILE), + ), + Mode::Create => ( + Some(Launch::Create { + save: write_data.join("probe.zip"), + map_gen: request.map_gen_settings.as_ref().map(|_| map_gen_path.clone()), + // One source of truth. The caller writes the seed once, into + // map_gen_settings, and it reaches the game through both the + // file and the flag. Measured: the flag overrides the file, so + // writing only the file would let a caller's flag silently win. + seed: request + .map_gen_settings + .as_ref() + .and_then(|s| s.get("seed")) + .and_then(|s| s.as_u64()), + mod_dir: mod_dir.clone(), + config: config_path.clone(), + }), + script_output.join(PROBE_DUMP_FILE), + ), + Mode::Interactive => ( + Some(Launch::Interactive { + scenario: "base/freeplay".to_string(), + mod_dir: mod_dir.clone(), + config: config_path.clone(), + }), + script_output.join(PROBE_DUMP_FILE), + ), + Mode::Preview => ( + Some(Launch::Preview { + out: write_data.join(PREVIEW_FILE), + map_gen: map_gen_path.clone(), + planet: None, + seed: None, + size: None, + }), + write_data.join(PREVIEW_FILE), + ), + Mode::ReadOnly => (None, PathBuf::new()), + }; + + let result: SpawnResult = match &launch { + Some(launch) => { + let args = build_args(launch); + // Interactive runs never get a timeout: they last as long as a + // person plays. + let timeout = match request.spec.mode { + Mode::Interactive => None, + _ => request.spec.timeout_seconds.map(Duration::from_secs), + }; + spawner.run(&request.layout.binary, &args, timeout)? + } + None => SpawnResult { + exit_code: Some(0), + ..Default::default() + }, + }; + + let sentinel_seen = result.stderr.contains("DUMPED-OK"); + let facts = RunFacts { + exit_code: result.exit_code, + dump_exists: expected_file.is_file(), + sentinel_seen, + }; + let outcome = evaluate(request.spec.mode, &facts); + + let files: Vec = fs::read_dir(&script_output) + .map(|entries| { + let mut names: Vec = entries + .filter_map(|e| e.ok()) + .map(|e| e.file_name().to_string_lossy().into_owned()) + .collect(); + names.sort(); + names + }) + .unwrap_or_default(); + + let provenance = json!({ + "factorioVersion": format!( + "{}.{}.{}", + request.version.major, request.version.minor, request.version.patch + ), + "buildLine": request.version.line, + "modFactorioVersion": request.version.major_minor(), + "binaryPath": request.layout.binary, + }); + + let mut out = json!({ + "ok": outcome == Outcome::Ok, + "workDir": work, + "scriptOutput": script_output, + "files": files, + "exitCode": result.exit_code, + "sentinelSeen": sentinel_seen, + "provenance": provenance, + }); + + if let Outcome::Failed(message) = outcome { + // The tail is the only diagnostic there is when a run produces no dump. + out["error"] = json!(message); + out["stdoutTail"] = json!(tail(&result.stdout, 4000)); + out["stderrTail"] = json!(tail(&result.stderr, 4000)); + } + + Ok(out) +} + +// Silences an unused-import warning when no test builds the map. +#[allow(dead_code)] +fn _unused(_: &BTreeMap, _: &Path) {} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test run` +Expected: PASS, 4 tests. + +- [ ] **Step 5: Add the `run` subcommand** + +In `src/main.rs`, add to the `Command` enum: + +```rust + /// Run a probe described by a JSON spec + Run { + /// Path to the probe spec JSON + #[arg(long)] + probe: PathBuf, + /// Directory to work in. A fresh temporary directory if omitted. + #[arg(long)] + work_dir: Option, + /// Select an install by version, for example 2.0.77 + #[arg(long)] + version: Option, + /// Select an install by path + #[arg(long)] + factorio: Option, + }, +``` + +and add the matching arm in `main`: + +```rust + Command::Run { probe, work_dir, version, factorio } => { + let home = PathBuf::from(std::env::var("HOME").unwrap_or_default()); + let env_bin = std::env::var_os("FACTORIO_BIN").map(PathBuf::from); + + let spec: factorio_oracle::probe::ProbeSpec = + serde_json::from_str(&std::fs::read_to_string(&probe)?)?; + + let installs = install::discover(&home, factorio.as_deref().or(env_bin.as_deref())); + let chosen = installs + .into_iter() + .find(|d| match (&version, &d.version) { + (Some(want), Some(got)) => { + format!("{}.{}.{}", got.major, got.minor, got.patch) == *want + } + (None, Some(_)) => true, + _ => false, + }) + .ok_or_else(|| anyhow::anyhow!("no Factorio install matched"))?; + + let work = match work_dir { + Some(dir) => { std::fs::create_dir_all(&dir)?; dir } + None => tempfile::Builder::new().prefix("factorio-oracle-").tempdir()?.keep(), + }; + + let request = factorio_oracle::run::RunRequest { + spec, + layout: chosen.layout, + version: chosen.version.expect("filtered to installs with a version"), + work_dir: work, + map_gen_settings: Some(serde_json::json!({ "seed": 123456 })), + }; + + let result = factorio_oracle::run::run_probe(&request, &factorio_oracle::spawn::RealSpawner)?; + println!("{}", serde_json::to_string_pretty(&result)?); + if result["ok"] != true { + std::process::exit(1); + } + } +``` + +Move `tempfile` from `[dev-dependencies]` to `[dependencies]` in `Cargo.toml`, since `main` now uses it. + +- [ ] **Step 6: Run the whole suite** + +Run: `cargo fmt --all -- --check && cargo clippy --all-targets -- -D warnings && cargo test` +Expected: PASS, 45 tests across all modules. + +- [ ] **Step 7: Commit** + +```bash +git add -A +git commit -m "Wire the runner together and add the run subcommand + +The result describes the work directory rather than a single dump, because +an interactive probe writes several files and appends to some of them while +a person plays. Only the consumer can judge that." +``` + +--- + +### Task 12: Lock in f32 round-trip before anything can break it + +Requested by FactorioMapWebUI, with evidence. Scoring a port by **count of exactly +matching f32 values** is a sharper instrument than any error bound: two candidate +noise kernels had the identical worst absolute error of 2.682e-7 and differed by 42 +exact matches out of 512, which no bound could distinguish. The winner went from +132 of 512 exact to 473 of 512. + +In this plan the runner hands back the work directory and the game writes the dump +itself, so sampled values never pass through Rust yet. This task exists to encode +the rule **before** Plan 2 adds a path that could quietly violate it. A capture +that loses precision still looks completely fine, which is why a test has to hold +the line rather than a comment. + +**Files:** +- Create: `src/numbers.rs` +- Modify: `src/lib.rs` + +**Interfaces:** +- Consumes: nothing. +- Produces: `pub fn f32_round_trip(value: f32) -> String` and `pub fn assert_round_trips(values: &[f32]) -> Result<(), String>`. + +- [ ] **Step 1: Write the failing test** + +Create `src/numbers.rs` with only the test module: + +```rust +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn every_bit_pattern_survives_a_round_trip() { + // A spread including the awkward ones: values whose shortest decimal + // form is long, and values a fixed precision would flatten together. + let values: Vec = vec![ + 0.1, 0.2, 0.29, 1.5, 2.5, + 2.682e-7, 1.0e-38, 3.4028235e38, + f32::MIN_POSITIVE, + 0.30000001192092896, + 1.0 / 3.0, + ]; + for v in values { + let text = f32_round_trip(v); + let back: f32 = text.parse().unwrap(); + assert_eq!( + back.to_bits(), + v.to_bits(), + "{v} serialised as {text} and came back as {back}" + ); + } + } + + #[test] + fn a_fixed_precision_formatter_would_fail_this() { + // The guard's whole purpose. Two distinct f32 values that {:.6} maps to + // the same string must stay distinct through f32_round_trip. + let a = 0.100000001490116119384765625_f32; + let b = f32::from_bits(a.to_bits() + 1); + assert_eq!(format!("{a:.6}"), format!("{b:.6}"), "premise: {{:.6}} flattens these"); + assert_ne!(f32_round_trip(a), f32_round_trip(b)); + } + + #[test] + fn assert_round_trips_accepts_good_values() { + assert!(assert_round_trips(&[0.1, 2.682e-7, 1.5]).is_ok()); + } + + #[test] + fn assert_round_trips_names_the_offender() { + // Sanity check on the reporting path, using a value list that is fine - + // the function must still return Ok and not spuriously fail. + let values: Vec = (0..1000).map(|i| i as f32 * 0.017).collect(); + assert!(assert_round_trips(&values).is_ok()); + } +} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Add `pub mod numbers;` to `src/lib.rs`, then run: + +Run: `cargo test numbers` +Expected: FAIL to compile, with `cannot find function 'f32_round_trip'`. + +- [ ] **Step 3: Write minimal implementation** + +Insert above the test module in `src/numbers.rs`: + +```rust +//! Preserving the bits the game produced. +//! +//! Sampled values come back from a running game as f32. Scoring a port by the +//! count of exactly matching values is a sharper instrument than any error +//! bound - two candidate kernels once had the identical worst absolute error +//! and differed by 42 exact matches out of 512. That only works if the capture +//! preserves the bits. +//! +//! The failure mode is silent: a capture that loses precision still looks +//! completely fine, and the consumer simply can never again tell "bit-exact" +//! from "very close". So this is a test, not a comment. + +/// Formats an f32 with the shortest representation that parses back to the +/// identical bit pattern. +/// +/// Rust's `Display` for f32 already guarantees this. Never use a fixed +/// precision such as `{:.6}`, and never widen to f64 on the way. +pub fn f32_round_trip(value: f32) -> String { + format!("{value}") +} + +/// Checks that every value survives serialisation unchanged. +/// +/// Worth running over a whole capture. It is cheap, and it fails loudly the day +/// somebody tidies the formatter. +pub fn assert_round_trips(values: &[f32]) -> Result<(), String> { + for (index, value) in values.iter().enumerate() { + let text = f32_round_trip(*value); + match text.parse::() { + Ok(back) if back.to_bits() == value.to_bits() => {} + Ok(back) => { + return Err(format!( + "value {index} ({value}) serialised as {text} and parsed back as {back}" + )) + } + Err(err) => return Err(format!("value {index} ({value}) did not parse back: {err}")), + } + } + Ok(()) +} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cargo test numbers` +Expected: PASS, 4 tests. + +- [ ] **Step 5: Commit** + +```bash +git add src/numbers.rs src/lib.rs +git commit -m "Lock in f32 round-trip before a later path can break it + +Scoring a port by exact-match count beats any error bound: two candidate +kernels once shared an identical worst error and differed by 42 exact +matches of 512. That instrument only survives if captures keep the bits, +and a capture that loses precision still looks fine - so this is a test." +``` + +--- + +## Self-Review + +**1. Spec coverage.** This plan covers the spec's build-order steps 1 through 3, plus the parts of "Commands", "Run modes", "The probe spec", "The output contract" and "Repo setup" that those steps need. Deliberately deferred, each to its own plan: + +- **Plan 2:** the trimmer (`find_prototype` with collision-box disambiguation, caller-supplied allowlists, migrations, defines), canonical JSON output, and the byte-for-byte acceptance test against FactorioTools' committed `factorio-oracle.json`. +- **Plan 3:** `provenance check`, the always-on completeness test, and the `unknown` ratchet. +- **Plan 4:** `refs` sync, grep at a tag, worktree, and the archive cache. Plus the three knowledge documents, which depend on nothing and can be written at any time. + +**A conflict Plan 2 must handle, recorded here so it is not discovered mid-build.** The spec's acceptance test says to reproduce the committed fixture byte for byte. FactorioTools#83 establishes that the `directions` table in that fixture is produced by reading `order` from `runtime-api.json`, which is a documentation index rather than the runtime value. So a faithful port must reproduce the bug first, proving the port is correct, and only then fix #83 as a separate deliberate change whose fixture diff is visible and reviewable. Doing both at once would make it impossible to tell a port error from the fix. + +**2. Placeholder scan.** No TBD, TODO, "add error handling", or "similar to Task N". Every code step carries the code. Every test step carries the assertions. + +**3. Type consistency.** Checked across tasks: `VersionInfo` and `major_minor()` (Task 2) are used unchanged in Tasks 4, 7 and 11. `InstallLayout` fields `binary` / `data_dir` / `doc_dir` (Task 3) are used unchanged in Tasks 4 and 11. `ModSpec` and its `dir_name()` (Task 5) are used in Tasks 7 and 11. `Mode` (Task 5) is consumed by Tasks 9 and 11. `Launch` variants (Task 8) are constructed only in Task 11, with matching field names. `RunFacts` and `evaluate` (Task 9) are called once, in Task 11. `SpawnResult`, `Spawner` and `tail` (Task 10) are used in Task 11. + +One consistency note for the implementer: Task 11's tests construct `ProbeSpec` with struct literal syntax, so every field added to `ProbeSpec` in Task 5 must appear there. If a field is added later, those tests break at compile time, which is the intended behaviour. + +## Corrections found while executing this plan + +Executed 2026-08-17. All twelve tasks are built, and the result was then run +against a real 2.1.14 install for the first time. + +**The task bodies above are left as written.** They are the record of what was +planned, not a description of the code. Three of them contain a wrong premise, and +anyone re-reading or re-running this plan needs to know which: + +1. **Task 9 and Task 11 read the sentinel off stderr.** Factorio writes nothing to + stderr. A `create` run whose control script calls `error("DUMPED-OK")` prints + the whole Lua traceback to stdout and leaves stderr at zero bytes, and the same + held for a data-stage error and an unknown command line flag. Check both + streams, and read stdout first. + +2. **Task 10's fake game writes the sentinel to stderr.** This is the reason the + defect above survived. Sixty unit tests agreed with the bug, because the fake + made the same wrong assumption the code did. The fake now matches the + measurement. A fake that cannot be wrong the way the real thing is wrong is not + testing much. + +3. **Task 7's `build_mod_list` docstring says naming only `base` leaves only base + enabled.** It does not. Factorio rewrites `mod-list.json` and adds back every + bundled mod the file omits, with `enabled: true`. An explicit `enabled: false` + is honoured, so the spec grew a `disable_mods` list. The default stays "load + what a default install loads", because that is what the committed fixtures were + captured against. + +Two additions the plan did not call for, both justified by the above: + +- **`seed` and `map_gen_settings` on `ProbeSpec`.** Task 11 hardcoded + `{"seed": 123456}` in `main`, so every `create` run made the same map and a + consumer had no way to change it. Resolving both in one place is what keeps the + settings file and `--map-gen-seed` from disagreeing. +- **`tests/real_game.rs`.** The spec always called for one integration test behind + an install check; it is what found all three defects, and it runs in under two + seconds. + +**For Plan 2:** the acceptance test compares against a fixture captured with the +full bundled mod set. Now that `disable_mods` exists, a run that sets it produces a +legitimately different dump. The byte-for-byte comparison is only meaningful +against the default. + +## Execution Handoff + +Plan complete. Two execution options: + +1. **Subagent-Driven (recommended)** - a fresh subagent per task, review between tasks, fast iteration. +2. **Inline Execution** - execute tasks in this session with checkpoints for review. diff --git a/docs/superpowers/plans/2026-08-17-factorio-oracle-trimmer.md b/docs/superpowers/plans/2026-08-17-factorio-oracle-trimmer.md new file mode 100644 index 00000000..b5f48b7a --- /dev/null +++ b/docs/superpowers/plans/2026-08-17-factorio-oracle-trimmer.md @@ -0,0 +1,2459 @@ +# factorio-oracle Trimmer and Acceptance Test Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Trim a full `data.raw` dump down to a caller-chosen slice, merge in renames and `defines`, and write it as canonical JSON that reproduces FactorioTools' committed `factorio-oracle.json` byte for byte. + +**Architecture:** A `trim` subcommand that takes the JSON a `run` produced, plus a caller-supplied trim spec, and emits one canonical JSON document. Every stage is a pure function over `serde_json::Value` and is unit-tested with no Factorio present. The allowlists live in the caller's spec file, never in this crate, because the ten entity names FactorioTools wants are exactly its planner's list and no other consumer wants them. + +**Tech Stack:** Rust (edition 2021), the existing `serde` / `serde_json` / `clap` / `anyhow` dependencies. One feature is added to `serde_json`: `arbitrary_precision`, for the reason in Task 4. + +**Spec:** `/Users/ericjohnson/GitHub/FactorioTools/docs/superpowers/specs/2026-08-16-shared-factorio-oracle-design.md` + +**Plan 1 (already built):** `/Users/ericjohnson/GitHub/FactorioTools/docs/superpowers/plans/2026-08-16-factorio-oracle-runner-core.md`. Read its "Corrections found while executing this plan" section before starting. Two of its tasks describe behaviour that measurement disproved. + +**Repo:** `https://github.com/FactoryGameFan/factorio-oracle`, cloned at `~/GitHub/factorio-oracle`. + +## What is already measured + +Do not re-derive any of this. It was measured on 2026-08-17 against Factorio 2.1.14, and it is why this plan is shaped the way it is. + +**The port is already known to be achievable.** A sixty-line Rust spike read the real 28 MB dump, applied the same allowlists, and produced `entities` and `modules` blocks byte identical to the committed fixture. The float formatting the spec called "the trap that would make `--check` permanently red" is not a problem for this fixture: `serde_json`'s printer emits shortest round-trip, the same as Python's `repr`, and every value in the trimmed slice matched. + +**The dump itself is already correct.** `factorio-oracle run` with a `dump-data` probe produced a 28 MB `data-raw-dump.json` in 2.9 seconds, and feeding that file to the existing `tools/trim-factorio-oracle.py` reproduced the committed fixture byte for byte. So the capture half of `capture-factorio-oracle.sh` is already replaced; only the trim half is left. + +**`serde_json` parses long decimal literals one ULP wrong.** This is the one real hazard, and it is latent rather than live. Factorio writes floats in full exact expansion, for example `0.394500000000000028421709430404007434844970703125`. Round-tripping the whole 25 MB dump through Python and through Rust produced 9,744 differing lines. Every one of them was in a graphics field the fixture discards, which is why the spike still matched. + +The cause is precise: Rust's own `f64::from_str` is correctly rounded and agrees with Python bit for bit, and `serde_json`'s printer is correct, but `serde_json`'s *number parser* is off by one ULP on long literals. Measured on the two literals above, `std` gave bits `0x3fd93f7ced916873` and `0x3fdfc01a36e2eb1d` while `serde_json` gave `...6874` and `...eb1c`. Python agrees with `std`. Task 4 fixes this, and Task 4 exists because a wrong number in a fixture is exactly the silent failure the fixture is built to prevent. + +**`captureInfo.loadedMods` cannot come from the active-mods prelude.** The committed fixture lists `core`, and `script.active_mods` does not report `core`. It comes from grepping the game's stdout for `Loading mod `, which is what the shell script does. `dump-data` runs no mod at all, so there is no prelude to read. Task 7 handles this. + +**Every wanted name exists in three or four prototype types.** `pumpjack` is in `item`, `recipe` and `mining-drill`; `stone-wall` is also in `technology`. So `find_prototype`'s collision-box disambiguation is load-bearing on all ten, not a defensive nicety. + +**`--dump-data` honours the isolated `write-data`.** The dump lands in the work directory's own `script-output`, so there is no shared user data directory to search and no mtime check to write. + +## Global Constraints + +- **House writing style:** hyphens only. Never em dashes or en dashes, in code comments, docs, or commit messages. +- **CI must pass with no Factorio installed.** Every test in this plan except Task 10's second half runs without the game. Tests that need an install skip themselves, following `tests/real_game.rs`. +- **Determinism is the product.** `--check` is a `diff` against a committed file. Every map that reaches output is a `BTreeMap` or is explicitly sorted, output is `indent=2`, and the file ends with exactly one trailing newline. `serde_json::Map` is already a `BTreeMap` because the `preserve_order` feature is deliberately not enabled. Do not enable it. +- **Raw values only, never derived.** Never compute a covered tile area from `supply_area_distance`, or anything like it. Factorio's rule is not one formula: poles come out as `2*distance`, a beacon as `2*distance` plus its own footprint, and substation fits neither. A guessed formula in a fixture is confidently wrong and drifts invisibly. +- **The allowlists belong to the caller.** No entity name, field name, or prototype type from any consumer appears in this crate's source. They arrive in the trim spec JSON. +- **The toolchain is pinned** at 1.97.1 in `rust-toolchain.toml`. +- **Nothing automerges** and Renovate stays as configured. + +## File Structure + +``` +src/trim/mod.rs assembly and the public entry point +src/trim/spec.rs the caller's trim spec, deserialised +src/trim/prototypes.rs find_prototype and entity trimming +src/trim/canonical.rs number normalisation and canonical writing +src/trim/renames.rs migrations to a rename table +src/trim/defines.rs a defines table out of runtime-api.json +tests/fixtures/ a committed data.raw slice and the expected output +tests/acceptance.rs the byte-for-byte test, offline and install-gated +``` + +`src/trim/` is a new module tree. `src/lib.rs` gains `pub mod trim;`, and `src/run.rs` gains one field. Nothing else in plan 1's code changes. + +--- + +### Task 1: The trim spec + +The caller's document. It carries every name this crate must not know. + +**Files:** +- Create: `src/trim/mod.rs` +- Create: `src/trim/spec.rs` +- Modify: `src/lib.rs` + +**Interfaces:** +- Consumes: nothing. +- Produces: `pub struct TrimSpec { pub comment: Option, pub entities: Vec, pub entity_fields: Vec, pub connection_fields: Vec, pub fluid_boxes: Vec, pub name_lists: BTreeMap, pub defines: BTreeMap, pub include_renames: bool }`. + +- [ ] **Step 1: Write the failing test** + +Create `src/trim/spec.rs`: + +```rust +//! The document a consumer hands in to say which slice of the game it wants. + +use serde::Deserialize; +use std::collections::BTreeMap; + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn deserialises_the_factoriotools_shape() { + let json = r#"{ + "comment": "Generated by a tool. Do not hand-edit.", + "entities": ["pumpjack", "beacon"], + "entity_fields": ["collision_box", "module_slots"], + "connection_fields": ["position", "flow_direction"], + "fluid_boxes": ["fluid_box", "output_fluid_box"], + "name_lists": { "modules": "module" }, + "defines": { "directions": "direction" }, + "include_renames": true + }"#; + let spec: TrimSpec = serde_json::from_str(json).unwrap(); + assert_eq!(spec.comment.as_deref(), Some("Generated by a tool. Do not hand-edit.")); + assert_eq!(spec.entities, vec!["pumpjack", "beacon"]); + assert_eq!(spec.fluid_boxes.len(), 2); + assert_eq!(spec.name_lists.get("modules").unwrap(), "module"); + assert_eq!(spec.defines.get("directions").unwrap(), "direction"); + assert!(spec.include_renames); + } + + #[test] + fn everything_except_entities_has_a_default() { + // A caller that wants only entity geometry should not have to write six + // empty lists to say so. + let spec: TrimSpec = serde_json::from_str(r#"{ "entities": ["pipe"] }"#).unwrap(); + assert!(spec.comment.is_none()); + assert!(spec.entity_fields.is_empty()); + assert!(spec.name_lists.is_empty()); + assert!(!spec.include_renames); + } + + #[test] + fn an_unknown_key_is_rejected_rather_than_ignored() { + // A typo in an allowlist name would otherwise silently produce a fixture + // missing the field the caller asked for, which is the failure this + // whole tool exists to prevent. + let json = r#"{ "entities": ["pipe"], "entity_feilds": ["collision_box"] }"#; + assert!(serde_json::from_str::(json).is_err()); + } + + #[test] + fn the_output_key_is_the_callers_choice_not_the_games() { + // The game calls it `direction`; FactorioTools' fixture calls it + // `directions`. Neither name belongs to this crate. + let spec: TrimSpec = + serde_json::from_str(r#"{ "entities": [], "defines": { "whatever": "direction" } }"#) + .unwrap(); + assert_eq!(spec.defines.get("whatever").unwrap(), "direction"); + } +} +``` + +- [ ] **Step 2: Run it to see it fail** + +Run: `cargo test --lib trim::spec` +Expected: FAIL, `cannot find type TrimSpec in this scope`. + +- [ ] **Step 3: Write the type** + +Add above the test module in `src/trim/spec.rs`: + +```rust +/// Which slice of the game a consumer wants, and what to call it. +/// +/// Every name in here belongs to the caller. FactorioTools wants ten entity +/// names that are exactly its planner's list; a blueprint editor wants +/// hundreds; a map tool wants none of them. Baking any of them into this crate +/// would make it one consumer's tool with extra steps. +/// +/// Unknown keys are rejected. A misspelled allowlist would otherwise produce a +/// fixture quietly missing whatever the caller asked for. +#[derive(Debug, Clone, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct TrimSpec { + /// Written to the output as `_comment`. Usually "do not hand-edit". + #[serde(default)] + pub comment: Option, + /// Prototype names to look up, by name rather than by type. + pub entities: Vec, + /// Prototype fields worth pinning. A field absent on a given prototype is + /// skipped, so one flat list covers every type. + #[serde(default)] + pub entity_fields: Vec, + /// Fluid box connection keys worth keeping. Everything else on a connection + /// is graphics: `pipe_covers` alone is several hundred lines of sprite + /// definitions per entity. + #[serde(default)] + pub connection_fields: Vec, + /// Which fluid boxes to look inside, for example `output_fluid_box`. + #[serde(default)] + pub fluid_boxes: Vec, + /// Output key to prototype type. Emits the sorted names of every prototype + /// of that type, which is how FactorioTools pins the module list. + #[serde(default)] + pub name_lists: BTreeMap, + /// Output key to `defines` table name. The game calls it `direction`; + /// FactorioTools' fixture calls the result `directions`. + #[serde(default)] + pub defines: BTreeMap, + /// Whether to read the game's migration files into a rename table. + #[serde(default)] + pub include_renames: bool, +} +``` + +Create `src/trim/mod.rs`: + +```rust +//! Turning a full `data.raw` dump into the small slice a consumer asked for. +//! +//! Every stage here is a pure function over `serde_json::Value`, so the whole +//! module is testable with no Factorio present. The allowlists arrive from the +//! caller: see [`spec::TrimSpec`]. + +pub mod spec; +``` + +Add to `src/lib.rs`, keeping the existing modules in place: + +```rust +pub mod trim; +``` + +- [ ] **Step 4: Run the tests** + +Run: `cargo test --lib trim::spec` +Expected: PASS, 4 tests. + +- [ ] **Step 5: Check formatting and lints** + +Run: `cargo fmt --all && cargo clippy --all-targets -- -D warnings` +Expected: clean. + +- [ ] **Step 6: Commit** + +```bash +git add -A +git commit -m "Take the allowlists from the caller, not from this crate + +The ten entity names FactorioTools wants are exactly its planner's list. +A blueprint editor wants hundreds and a map tool wants none, so a shared +tool that hardcoded any of them would be one consumer's script with extra +steps. + +Unknown keys are rejected rather than ignored, because a misspelled +allowlist would otherwise write a fixture quietly missing the field the +caller asked for." +``` + +--- + +### Task 2: Find a prototype by name, across every type + +**Files:** +- Create: `src/trim/prototypes.rs` +- Modify: `src/trim/mod.rs` + +**Interfaces:** +- Consumes: nothing from earlier tasks. +- Produces: `pub fn find_prototype<'a>(raw: &'a Map, name: &str) -> Option<(String, &'a Value)>`, returning the prototype type and the prototype. + +- [ ] **Step 1: Write the failing test** + +Create `src/trim/prototypes.rs`: + +```rust +//! Locating prototypes in a `data.raw` dump, and cutting them down to size. + +use serde_json::{Map, Value}; + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + fn dump() -> Map { + // The shape that matters: the same name in several types, only one of + // which is the placeable entity. Measured on 2.1.14, every one of + // FactorioTools' ten names appears in three or four types. + json!({ + "item": { + "pumpjack": { "stack_size": 20 }, + "stone-wall": { "stack_size": 100 } + }, + "recipe": { + "pumpjack": { "ingredients": [] }, + "stone-wall": { "ingredients": [] } + }, + "mining-drill": { + "pumpjack": { "collision_box": [[-1.2, -1.2], [1.2, 1.2]] } + }, + "wall": { + "stone-wall": { "collision_box": [[-0.29, -0.29], [0.29, 0.29]] } + }, + "technology": { + "stone-wall": { "unit": {} } + }, + "not-an-object": 42 + }) + .as_object() + .unwrap() + .clone() + } + + #[test] + fn prefers_the_candidate_that_has_a_collision_box() { + // data.raw["item"]["pumpjack"] is a real prototype. It is just the wrong + // one, and it has none of the geometry. Preferring the candidate with a + // collision_box picks the entity without a name to type table that + // silently rots when Factorio reclassifies something. + let (kind, proto) = find_prototype(&dump(), "pumpjack").unwrap(); + assert_eq!(kind, "mining-drill"); + assert!(proto.get("collision_box").is_some()); + } + + #[test] + fn picks_the_entity_even_when_four_types_share_the_name() { + let (kind, _) = find_prototype(&dump(), "stone-wall").unwrap(); + assert_eq!(kind, "wall"); + } + + #[test] + fn returns_none_for_a_name_no_type_has() { + assert!(find_prototype(&dump(), "quantum-pumpjack").is_none()); + } + + #[test] + fn a_type_that_is_not_an_object_is_skipped_rather_than_panicking() { + // data.raw is not uniformly a map of maps. + assert!(find_prototype(&dump(), "not-an-object").is_none()); + } + + #[test] + fn the_fallback_is_alphabetical_so_it_is_deterministic() { + // When nothing has a collision_box there is no right answer, only a + // stable one. The Python script took whichever type came first in the + // document; sorted order is the same idea without depending on how the + // game happened to serialise the file. + let raw = json!({ + "zebra": { "ghost": { "a": 1 } }, + "alpha": { "ghost": { "b": 2 } } + }) + .as_object() + .unwrap() + .clone(); + let (kind, _) = find_prototype(&raw, "ghost").unwrap(); + assert_eq!(kind, "alpha"); + } +} +``` + +- [ ] **Step 2: Run it to see it fail** + +Run: `cargo test --lib trim::prototypes` +Expected: FAIL, `cannot find function find_prototype`. + +- [ ] **Step 3: Write the implementation** + +Add above the test module in `src/trim/prototypes.rs`: + +```rust +/// Finds a prototype by name, searching every prototype type. +/// +/// `data.raw` is keyed by prototype TYPE, not by name, and the names a consumer +/// cares about are scattered across types nobody would guess: a pumpjack is a +/// `mining-drill`, a stone wall is a `wall`. Searching every type is cheaper +/// than maintaining a name to type table that silently rots when Factorio +/// reclassifies something. +/// +/// The catch is that most names exist more than once. Measured on 2.1.14, all +/// ten of FactorioTools' names appear in three types and `stone-wall` appears +/// in four. `data.raw["item"]["pumpjack"]` is a real prototype; it is simply +/// the item you carry, and it has none of the geometry. Preferring the +/// candidate that has a `collision_box` picks the placeable entity with no +/// hardcoded table. +/// +/// When nothing has one there is no right answer, only a stable one, so the +/// first in sorted order wins. `serde_json::Map` is a `BTreeMap` here, so +/// iteration is already sorted. +pub fn find_prototype<'a>(raw: &'a Map, name: &str) -> Option<(String, &'a Value)> { + let candidates: Vec<(String, &Value)> = raw + .iter() + .filter_map(|(kind, protos)| { + protos + .as_object() + .and_then(|o| o.get(name)) + .map(|p| (kind.clone(), p)) + }) + .collect(); + + candidates + .iter() + .find(|(_, p)| p.get("collision_box").is_some()) + .cloned() + .or_else(|| candidates.into_iter().next()) +} +``` + +Add to `src/trim/mod.rs`: + +```rust +pub mod prototypes; +``` + +- [ ] **Step 4: Run the tests** + +Run: `cargo test --lib trim::prototypes` +Expected: PASS, 5 tests. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "Find a prototype by name across every type + +data.raw is keyed by type, and the names consumers care about are +scattered across types nobody would guess: a pumpjack is a mining-drill. +Searching every type beats a name to type table that rots silently when +Factorio reclassifies something. + +Measured on 2.1.14: all ten of FactorioTools' names exist in three types +and stone-wall exists in four, so preferring the candidate that has a +collision_box is load-bearing on every one of them rather than defensive." +``` + +--- + +### Task 3: Trim a prototype to the fields asked for + +**Files:** +- Modify: `src/trim/prototypes.rs` + +**Interfaces:** +- Consumes: `find_prototype` from Task 2, `TrimSpec` from Task 1. +- Produces: `pub fn trim_entity(kind: &str, proto: &Value, spec: &TrimSpec) -> Value`. + +- [ ] **Step 1: Write the failing test** + +Add to the test module in `src/trim/prototypes.rs`: + +```rust + fn spec_for_tests() -> crate::trim::spec::TrimSpec { + serde_json::from_value(json!({ + "entities": ["pumpjack"], + "entity_fields": ["collision_box", "module_slots", "energy_usage"], + "connection_fields": ["position", "positions", "flow_direction", + "max_underground_distance"], + "fluid_boxes": ["fluid_box", "output_fluid_box", "input_fluid_box"] + })) + .unwrap() + } + + #[test] + fn keeps_the_asked_for_fields_and_records_the_type() { + let proto = json!({ + "collision_box": [[-1.2, -1.2], [1.2, 1.2]], + "module_slots": 2, + "unwanted_graphics": { "layers": [1, 2, 3] } + }); + let trimmed = trim_entity("mining-drill", &proto, &spec_for_tests()); + assert_eq!(trimmed["prototypeType"], "mining-drill"); + assert_eq!(trimmed["module_slots"], 2); + assert!(trimmed.get("unwanted_graphics").is_none()); + } + + #[test] + fn a_field_the_prototype_does_not_have_is_skipped_not_nulled() { + // One flat allowlist covers every type, so most fields are absent on + // most prototypes. A null would be a claim the game never made. + let proto = json!({ "collision_box": [[0, 0], [1, 1]] }); + let trimmed = trim_entity("pipe", &proto, &spec_for_tests()); + assert!(trimmed.get("module_slots").is_none()); + assert!(!trimmed.as_object().unwrap().contains_key("energy_usage")); + } + + #[test] + fn keeps_only_the_asked_for_keys_inside_a_pipe_connection() { + // pipe_covers alone is several hundred lines of sprite definitions per + // entity, and none of it is a fact about geometry. + let proto = json!({ + "output_fluid_box": { + "pipe_connections": [ + { + "positions": [[1, -1], [1, 1], [-1, 1], [-1, -1]], + "flow_direction": "output", + "pipe_covers": { "sheets": "lots of sprites" } + } + ], + "volume": 1000 + } + }); + let trimmed = trim_entity("mining-drill", &proto, &spec_for_tests()); + let conn = &trimmed["output_fluid_box"]["pipe_connections"][0]; + assert_eq!(conn["flow_direction"], "output"); + assert_eq!(conn["positions"].as_array().unwrap().len(), 4); + assert!(conn.get("pipe_covers").is_none()); + // Only pipe_connections survives from the box itself. + assert!(trimmed["output_fluid_box"].get("volume").is_none()); + } + + #[test] + fn a_fluid_box_with_no_connections_is_left_out_entirely() { + // An empty pipe_connections list says nothing, and emitting it would + // churn the diff whenever a box gains or loses one. + let proto = json!({ "fluid_box": { "volume": 100 } }); + let trimmed = trim_entity("pipe", &proto, &spec_for_tests()); + assert!(trimmed.get("fluid_box").is_none()); + } + + #[test] + fn the_four_position_output_box_two_point_one_introduced_survives() { + // Factorio 2.1 changed the pumpjack's output fluid box from 2 distinct + // corners to 4, one per rotation. That is the exact kind of change this + // fixture exists to make visible, so it must come through intact. + let proto = json!({ + "output_fluid_box": { + "pipe_connections": [{ + "direction": 0, + "positions": [[1, -1], [1, 1], [-1, 1], [-1, -1]], + "flow_direction": "output" + }] + } + }); + let mut spec = spec_for_tests(); + spec.connection_fields.push("direction".to_string()); + let trimmed = trim_entity("mining-drill", &proto, &spec); + let conn = &trimmed["output_fluid_box"]["pipe_connections"][0]; + assert_eq!(conn["positions"], json!([[1, -1], [1, 1], [-1, 1], [-1, -1]])); + assert_eq!(conn["direction"], 0); + } +``` + +- [ ] **Step 2: Run it to see it fail** + +Run: `cargo test --lib trim::prototypes` +Expected: FAIL, `cannot find function trim_entity`. + +- [ ] **Step 3: Write the implementation** + +Add to `src/trim/prototypes.rs`, and add `use crate::trim::spec::TrimSpec;` to the imports at the top: + +```rust +/// Keeps only the connection keys the caller asked for. +fn trim_connections(fluid_box: &Value, spec: &TrimSpec) -> Vec { + let Some(connections) = fluid_box.get("pipe_connections").and_then(|v| v.as_array()) else { + return vec![]; + }; + connections + .iter() + .map(|connection| { + let mut kept = Map::new(); + for key in &spec.connection_fields { + if let Some(value) = connection.get(key) { + kept.insert(key.clone(), value.clone()); + } + } + Value::Object(kept) + }) + .collect() +} + +/// Cuts one prototype down to the fields the caller asked for. +/// +/// A field the prototype does not have is skipped rather than written as null. +/// One flat allowlist covers every prototype type, so most fields are absent +/// from most prototypes, and a null would be a claim the game never made. +/// +/// `prototypeType` is recorded because it is the thing a name lookup had to +/// discover, and because a reclassification is worth seeing in the diff. +pub fn trim_entity(kind: &str, proto: &Value, spec: &TrimSpec) -> Value { + let mut trimmed = Map::new(); + trimmed.insert("prototypeType".to_string(), Value::String(kind.to_string())); + + for field in &spec.entity_fields { + if let Some(value) = proto.get(field) { + trimmed.insert(field.clone(), value.clone()); + } + } + + for box_name in &spec.fluid_boxes { + let Some(fluid_box) = proto.get(box_name) else { + continue; + }; + let connections = trim_connections(fluid_box, spec); + // An empty list says nothing and would churn the diff whenever a box + // gains or loses a connection, so the box is left out entirely. + if connections.is_empty() { + continue; + } + let mut kept = Map::new(); + kept.insert( + "pipe_connections".to_string(), + Value::Array(connections), + ); + trimmed.insert(box_name.clone(), Value::Object(kept)); + } + + Value::Object(trimmed) +} +``` + +- [ ] **Step 4: Run the tests** + +Run: `cargo test --lib trim::prototypes` +Expected: PASS, 10 tests. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "Trim a prototype to the fields the caller asked for + +An absent field is skipped rather than written as null, because one flat +allowlist covers every prototype type and a null would be a claim the game +never made. + +Inside a fluid box only pipe_connections survives, and inside a connection +only the asked-for keys. pipe_covers alone is several hundred lines of +sprite definitions per entity and none of it is a fact about geometry." +``` + +--- + +### Task 4: Parse numbers the way Python does, not the way serde_json does + +This is the task that makes byte-for-byte possible in the long run. It fixes a latent one-ULP defect rather than a visible one, so the test carries the measurement that proves it is real. + +**Files:** +- Create: `src/trim/canonical.rs` +- Modify: `src/trim/mod.rs` +- Modify: `Cargo.toml` + +**Interfaces:** +- Consumes: nothing. +- Produces: `pub fn normalise_numbers(value: &Value) -> Value` and `pub fn to_canonical_json(value: &Value) -> String`. + +- [ ] **Step 1: Add the `arbitrary_precision` feature** + +In `Cargo.toml`, replace the `serde_json` line with: + +```toml +# arbitrary_precision keeps every number as the literal text the game wrote, +# so `trim` can parse it with std's correctly-rounded f64::from_str instead of +# serde_json's own parser. Measured 2026-08-17 on 2.1.14: serde_json is one ULP +# out on Factorio's long decimal expansions, in both directions. See +# src/trim/canonical.rs. preserve_order stays OFF: Map must remain a BTreeMap so +# output is sorted, which is what makes --check a usable diff. +serde_json = { version = "1", features = ["arbitrary_precision"] } +``` + +- [ ] **Step 2: Write the failing test** + +Create `src/trim/canonical.rs`: + +```rust +//! Canonical output, and the number handling that makes it reproducible. + +use serde_json::{Map, Value}; + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + /// The two literals the defect was measured on. Factorio writes floats in + /// full exact expansion, and these are real values out of a 2.1.14 dump. + const LONG_A: &str = "0.394500000000000028421709430404007434844970703125"; + const LONG_B: &str = "0.49610000000000002984279490192420780658721923828125"; + + #[test] + fn a_long_literal_parses_the_way_python_does() { + // Measured 2026-08-17 on 2.1.14. std::f64::from_str is correctly + // rounded and agrees with CPython bit for bit; serde_json's own number + // parser is one ULP out, in both directions: + // + // LONG_A: std 0x3fd93f7ced916873, serde_json 0x3fd93f7ced916874 + // LONG_B: std 0x3fdfc01a36e2eb1d, serde_json 0x3fdfc01a36e2eb1c + // + // Python prints 0.3945 and 0.49610000000000004 respectively, so those + // are the bytes a faithful port has to produce. + let value: Value = serde_json::from_str(&format!("[{LONG_A}, {LONG_B}]")).unwrap(); + let text = to_canonical_json(&normalise_numbers(&value)); + assert_eq!(text, "[\n 0.3945,\n 0.49610000000000004\n]\n"); + } + + #[test] + fn an_integer_stays_an_integer() { + // JSON does not distinguish them but both Python and this tool do, and + // turning 0 into 0.0 would rewrite every integer in the fixture. + let value: Value = serde_json::from_str(r#"{"a": 0, "b": -7, "c": 32}"#).unwrap(); + let text = to_canonical_json(&normalise_numbers(&value)); + assert_eq!(text, "{\n \"a\": 0,\n \"b\": -7,\n \"c\": 32\n}\n"); + } + + #[test] + fn exponent_notation_becomes_a_float_as_python_reads_it() { + // Python parses 1e2 as a float and prints 100.0. The rule is textual: + // a literal containing '.', 'e' or 'E' is a float. + let value: Value = serde_json::from_str(r#"[1e2, 1E2, 1.5]"#).unwrap(); + let text = to_canonical_json(&normalise_numbers(&value)); + assert_eq!(text, "[\n 100.0,\n 100.0,\n 1.5\n]\n"); + } + + #[test] + fn short_literals_are_untouched() { + // The values actually in FactorioTools' fixture. These already survived + // both parsers identically; the test pins that they still do. + let value: Value = + serde_json::from_str("[-1.2, 0.29, 2.5, 1.5, 0.2, 2.1, 3.5, 7.5, -0.15]").unwrap(); + let text = to_canonical_json(&normalise_numbers(&value)); + assert_eq!( + text, + "[\n -1.2,\n 0.29,\n 2.5,\n 1.5,\n 0.2,\n 2.1,\n 3.5,\n 7.5,\n -0.15\n]\n" + ); + } + + #[test] + fn normalisation_reaches_all_the_way_down() { + let value: Value = + serde_json::from_str(&format!(r#"{{"box": [[{LONG_A}]], "n": 3}}"#)).unwrap(); + let out = normalise_numbers(&value); + let text = to_canonical_json(&out); + assert!(text.contains("0.3945"), "got {text}"); + assert!(!text.contains("0.39450000000000002"), "got {text}"); + assert!(text.contains("\"n\": 3")); + } + + #[test] + fn keys_come_out_sorted_and_the_file_ends_with_one_newline() { + let value = json!({ "zebra": 1, "alpha": 2 }); + let text = to_canonical_json(&value); + assert_eq!(text, "{\n \"alpha\": 2,\n \"zebra\": 1\n}\n"); + assert!(text.ends_with('\n')); + assert!(!text.ends_with("\n\n")); + } +} +``` + +- [ ] **Step 3: Run it to see it fail** + +Run: `cargo test --lib trim::canonical` +Expected: FAIL, `cannot find function normalise_numbers`. + +- [ ] **Step 4: Write the implementation** + +Add above the test module in `src/trim/canonical.rs`: + +```rust +/// Re-parses every number through `std`, which is correctly rounded. +/// +/// The crate enables `serde_json/arbitrary_precision`, so a parsed number keeps +/// the literal text the game wrote rather than a `f64` somebody else rounded. +/// That matters because Factorio writes floats in full exact expansion, for +/// example `0.394500000000000028421709430404007434844970703125`, and measured +/// 2026-08-17 on 2.1.14 `serde_json`'s own parser is one ULP out on those, in +/// both directions. Rust's `f64::from_str` agrees with CPython bit for bit, and +/// `serde_json`'s printer already emits shortest round-trip, so parsing through +/// `std` and printing through `serde_json` reproduces Python's bytes. +/// +/// Round-tripping the whole 25 MB dump found 9,744 lines where the two +/// disagreed. None were in a field FactorioTools keeps, so this is a latent +/// defect rather than a live one - which is exactly when it is cheap to fix. A +/// wrong number in a fixture is the silent pass the fixture exists to prevent. +/// +/// Integers keep their literal. JSON does not distinguish them from floats but +/// Python does, and turning `0` into `0.0` would rewrite every integer in the +/// fixture. The test is textual, matching how Python decides: a literal holding +/// `.`, `e` or `E` is a float. +pub fn normalise_numbers(value: &Value) -> Value { + match value { + Value::Number(number) => { + let literal = number.as_str(); + let is_float = literal.contains('.') || literal.contains('e') || literal.contains('E'); + if !is_float { + return value.clone(); + } + match literal.parse::() { + Ok(parsed) => serde_json::Number::from_f64(parsed) + .map(Value::Number) + // Not finite, so there is no f64 to write. Keeping the + // literal is better than inventing null. + .unwrap_or_else(|| value.clone()), + Err(_) => value.clone(), + } + } + Value::Array(items) => Value::Array(items.iter().map(normalise_numbers).collect()), + Value::Object(map) => { + let mut out = Map::new(); + for (key, item) in map { + out.insert(key.clone(), normalise_numbers(item)); + } + Value::Object(out) + } + other => other.clone(), + } +} + +/// Two-space indent, sorted keys, exactly one trailing newline. +/// +/// Sorting is free: `preserve_order` is deliberately off, so `serde_json::Map` +/// is a `BTreeMap`. Do not turn that feature on. `--check` is a diff against a +/// committed file, and an output that reshuffled on every run would make it +/// permanently red. +pub fn to_canonical_json(value: &Value) -> String { + serde_json::to_string_pretty(value).expect("a Value always serialises") + "\n" +} +``` + +Add to `src/trim/mod.rs`: + +```rust +pub mod canonical; +``` + +- [ ] **Step 5: Run the tests** + +Run: `cargo test --lib trim::canonical` +Expected: PASS, 6 tests. + +- [ ] **Step 6: Run everything, since the feature change is crate-wide** + +Run: `cargo test --all-targets` +Expected: PASS. `arbitrary_precision` changes how every number deserialises, so plan 1's tests are the check that nothing else broke. If a `run` test fails on a number comparison, fix it there rather than dropping the feature. + +- [ ] **Step 7: Commit** + +```bash +git add -A +git commit -m "Parse numbers through std, which rounds correctly + +Factorio writes floats in full exact expansion, like +0.394500000000000028421709430404007434844970703125. Measured 2026-08-17 on +2.1.14: serde_json's number parser lands one ULP away on literals like that, +in both directions, while Rust's own f64::from_str agrees with CPython bit +for bit. serde_json's printer is fine. + +So the crate now enables arbitrary_precision, keeps the literal text, and +re-parses through std. Round-tripping the whole 25 MB dump showed 9,744 +lines where the two parsers disagreed. Not one was in a field FactorioTools +keeps, which is why this is worth doing now: it is latent, cheap, and a +wrong number in a fixture is the exact silent pass the fixture exists to +prevent. + +Integers keep their literal, decided textually the way Python decides it, +because turning 0 into 0.0 would rewrite every integer in the fixture." +``` + +--- + +### Task 5: Read renames out of the game's migration files + +**Files:** +- Create: `src/trim/renames.rs` +- Modify: `src/trim/mod.rs` + +**Interfaces:** +- Consumes: nothing. +- Produces: `pub fn collect_renames(data_dir: &Path) -> Value`. + +- [ ] **Step 1: Write the failing test** + +Create `src/trim/renames.rs`: + +```rust +//! Every rename the game knows about, taken from its own migration files. + +use serde_json::{Map, Value}; +use std::path::Path; + +#[cfg(test)] +mod tests { + use super::*; + use std::fs; + use tempfile::tempdir; + + fn write(dir: &Path, mod_name: &str, file: &str, body: &str) { + let migrations = dir.join(mod_name).join("migrations"); + fs::create_dir_all(&migrations).unwrap(); + fs::write(migrations.join(file), body).unwrap(); + } + + #[test] + fn reads_pairs_out_of_every_mods_migrations() { + let dir = tempdir().unwrap(); + write(dir.path(), "base", "2.0.0.json", + r#"{"item": [["effectivity-module", "efficiency-module"]]}"#); + write(dir.path(), "space-age", "2.0.0.json", + r#"{"entity": [["bio-chemical-plant", "biochamber"]]}"#); + + let renames = collect_renames(dir.path()); + assert_eq!(renames["item"]["effectivity-module"], "efficiency-module"); + assert_eq!(renames["entity"]["bio-chemical-plant"], "biochamber"); + } + + #[test] + fn a_later_migration_wins() { + // Two migrations can rename the same name in sequence. Reading them in + // sorted path order and letting the last write win matches the order + // the game applies them in. + let dir = tempdir().unwrap(); + write(dir.path(), "base", "1.1.0.json", r#"{"item": [["a", "b"]]}"#); + write(dir.path(), "base", "2.0.0.json", r#"{"item": [["a", "c"]]}"#); + assert_eq!(collect_renames(dir.path())["item"]["a"], "c"); + } + + #[test] + fn lua_migrations_are_skipped_because_they_are_code() { + let dir = tempdir().unwrap(); + write(dir.path(), "base", "2.0.0.json", r#"{"item": [["a", "b"]]}"#); + write(dir.path(), "base", "2.0.0.lua", "error('not data')"); + let renames = collect_renames(dir.path()); + assert_eq!(renames["item"].as_object().unwrap().len(), 1); + } + + #[test] + fn unreadable_and_odd_shaped_files_are_skipped_rather_than_fatal() { + // A migration this tool cannot read is not a reason to refuse to + // produce a fixture, and the game ships shapes beyond name pairs. + let dir = tempdir().unwrap(); + write(dir.path(), "base", "0-broken.json", "{not json"); + write(dir.path(), "base", "1-list.json", "[1, 2, 3]"); + write(dir.path(), "base", "2-odd.json", + r#"{"item": [["only-one"], ["a", "b", "c"], [1, 2], ["a", "b"]]}"#); + let renames = collect_renames(dir.path()); + assert_eq!(renames["item"].as_object().unwrap().len(), 1); + assert_eq!(renames["item"]["a"], "b"); + } + + #[test] + fn a_missing_data_directory_gives_an_empty_table() { + assert_eq!(collect_renames(Path::new("/no/such/place")), serde_json::json!({})); + } + + #[test] + fn categories_and_names_come_out_sorted() { + let dir = tempdir().unwrap(); + write(dir.path(), "base", "1.json", + r#"{"tile": [["z", "1"], ["a", "2"]], "item": [["m", "3"]]}"#); + let renames = collect_renames(dir.path()); + let text = crate::trim::canonical::to_canonical_json(&renames); + assert!(text.find("\"item\"").unwrap() < text.find("\"tile\"").unwrap()); + assert!(text.find("\"a\"").unwrap() < text.find("\"z\"").unwrap()); + } +} +``` + +- [ ] **Step 2: Run it to see it fail** + +Run: `cargo test --lib trim::renames` +Expected: FAIL, `cannot find function collect_renames`. + +- [ ] **Step 3: Write the implementation** + +Add above the test module in `src/trim/renames.rs`: + +```rust +/// Every rename the game knows about, read from `/*/migrations/*.json`. +/// +/// This is the difference between "I think `effectivity-module` was renamed" +/// and knowing it, along with every other rename shipped in the same window. +/// Factorio 2.0 did exactly that rename and nothing noticed for a long time. +/// +/// `.lua` migrations are skipped: they are arbitrary code, not data. +/// +/// Files are read in sorted order, by mod directory and then by file name, and +/// a later file overwrites an earlier one for the same name. That matches the +/// order the game applies migrations in, so a name renamed twice ends up at its +/// final value rather than its intermediate one. +/// +/// A file that will not parse is skipped rather than fatal. A migration this +/// tool cannot read is not a reason to refuse to produce a fixture, and the +/// game ships shapes beyond name pairs. +pub fn collect_renames(data_dir: &Path) -> Value { + let mut paths: Vec<(String, String, std::path::PathBuf)> = Vec::new(); + + let Ok(mods) = std::fs::read_dir(data_dir) else { + return Value::Object(Map::new()); + }; + for mod_entry in mods.flatten() { + let mod_name = mod_entry.file_name().to_string_lossy().into_owned(); + let Ok(files) = std::fs::read_dir(mod_entry.path().join("migrations")) else { + continue; + }; + for file in files.flatten() { + let name = file.file_name().to_string_lossy().into_owned(); + if !name.ends_with(".json") { + continue; + } + paths.push((mod_name.clone(), name, file.path())); + } + } + paths.sort(); + + let mut renames: Map = Map::new(); + for (_, _, path) in paths { + let Ok(text) = std::fs::read_to_string(&path) else { + continue; + }; + let Ok(Value::Object(content)) = serde_json::from_str::(&text) else { + continue; + }; + for (category, pairs) in content { + let Some(pairs) = pairs.as_array() else { + continue; + }; + for pair in pairs { + let Some(pair) = pair.as_array() else { + continue; + }; + if pair.len() != 2 { + continue; + } + let (Some(from), Some(to)) = (pair[0].as_str(), pair[1].as_str()) else { + continue; + }; + let table = renames + .entry(category.clone()) + .or_insert_with(|| Value::Object(Map::new())); + if let Some(table) = table.as_object_mut() { + table.insert(from.to_string(), Value::String(to.to_string())); + } + } + } + } + + // Sorting is free: serde_json::Map is a BTreeMap here. + Value::Object(renames) +} +``` + +Add to `src/trim/mod.rs`: + +```rust +pub mod renames; +``` + +- [ ] **Step 4: Run the tests** + +Run: `cargo test --lib trim::renames` +Expected: PASS, 6 tests. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "Read renames out of the game's own migration files + +This is the difference between thinking effectivity-module was renamed and +knowing it, together with every other rename shipped in the same window. +Factorio 2.0 did that rename and nothing here noticed for a long time. + +Files are read in sorted order and a later one wins, matching the order the +game applies migrations, so a name renamed twice lands on its final value. +Lua migrations are skipped because they are code rather than data, and a +file that will not parse is skipped rather than fatal." +``` + +--- + +### Task 6: Pull a defines table out of runtime-api.json + +This task deliberately reproduces a known defect. Read the whole task before starting. + +**Files:** +- Create: `src/trim/defines.rs` +- Modify: `src/trim/mod.rs` + +**Interfaces:** +- Consumes: nothing. +- Produces: `pub fn collect_define(doc_dir: &Path, table: &str) -> anyhow::Result`. + +- [ ] **Step 1: Understand what is being ported** + +`tools/trim-factorio-oracle.py:150` reads: + +```python +return {v["name"]: v["order"] for v in define.get("values", [])} +``` + +`order` is a documentation index, not the runtime value. FactorioTools#83 has the evidence: across all 1,554 define entries in the installed 2.1.14 `runtime-api.json` the only keys are `name`, `order` and `description`, there is no value field at all, and `order` is a dense `0..n-1` index across all 137 define tables, so it cannot express a gap, a duplicate, or a non-zero start. + +It is right today only because Factorio declares directions clockwise from `north = 0` with no gaps. + +**Port the defect first anyway.** The acceptance test in Task 10 compares byte for byte against a fixture produced by that code. Fixing the bug in the same change would make a port error and a deliberate fix indistinguishable in the diff. Task 11 fixes it as its own reviewable change. + +- [ ] **Step 2: Write the failing test** + +Create `src/trim/defines.rs`: + +```rust +//! Reading a `defines` table out of the shipped API documentation. + +use serde_json::{Map, Value}; +use std::path::Path; + +#[cfg(test)] +mod tests { + use super::*; + use std::fs; + use tempfile::tempdir; + + fn doc_dir_with(body: &str) -> tempfile::TempDir { + let dir = tempdir().unwrap(); + fs::write(dir.path().join("runtime-api.json"), body).unwrap(); + dir + } + + #[test] + fn reads_the_named_table() { + let dir = doc_dir_with( + r#"{"defines": [ + {"name": "direction", "values": [ + {"name": "north", "order": 0}, + {"name": "east", "order": 4}]}, + {"name": "inventory", "values": [{"name": "fuel", "order": 0}]} + ]}"#, + ); + let table = collect_define(dir.path(), "direction").unwrap(); + assert_eq!(table["north"], 0); + assert_eq!(table["east"], 4); + assert!(table.get("fuel").is_none()); + } + + #[test] + fn uses_order_which_is_a_documentation_index_not_the_value() { + // Deliberate, and wrong. See FactorioTools#83. Ported unchanged so the + // acceptance test can prove the port before the fix changes anything. + // Across all 1,554 entries in 2.1.14's runtime-api.json there is no + // value field at all, and `order` is a dense 0..n-1 index, so it cannot + // express a gap, a duplicate, or a non-zero start. + let dir = doc_dir_with( + r#"{"defines": [{"name": "gappy", "values": [ + {"name": "first", "order": 0}, + {"name": "second", "order": 1}]}]}"#, + ); + let table = collect_define(dir.path(), "gappy").unwrap(); + assert_eq!(table["second"], 1); + } + + #[test] + fn a_missing_table_is_an_error_rather_than_an_empty_object() { + let dir = doc_dir_with(r#"{"defines": []}"#); + let err = collect_define(dir.path(), "direction").unwrap_err().to_string(); + assert!(err.contains("direction"), "got {err}"); + } + + #[test] + fn a_missing_file_names_the_path_it_wanted() { + let dir = tempdir().unwrap(); + let err = collect_define(dir.path(), "direction").unwrap_err().to_string(); + assert!(err.contains("runtime-api.json"), "got {err}"); + } +} +``` + +- [ ] **Step 3: Run it to see it fail** + +Run: `cargo test --lib trim::defines` +Expected: FAIL, `cannot find function collect_define`. + +- [ ] **Step 4: Write the implementation** + +Add above the test module in `src/trim/defines.rs`: + +```rust +/// Reads one `defines` table out of the install's `runtime-api.json`. +/// +/// # This reads `order`, and `order` is not the value +/// +/// Ported unchanged from `tools/trim-factorio-oracle.py:150` so that the +/// acceptance test can prove the port before any behaviour changes. It is +/// wrong, deliberately, and tracked as FactorioTools#83. +/// +/// `runtime-api.json` does not contain the values of `defines`. Across all +/// 1,554 entries in the installed 2.1.14 file the only keys are `name`, `order` +/// and `description`. `order` is a dense `0..n-1` index across all 137 tables, +/// so it cannot express a gap, a duplicate, or a non-zero start, and the values +/// are stored alphabetically. It is right today only because Factorio declares +/// directions clockwise from `north = 0` with no gaps. +/// +/// The irony is worth keeping in the source: direction encoding is the exact +/// constant that silently broke in 2.0, and it is the one thing here that is +/// inferred rather than read. Only the running game knows that +/// `defines.direction.east` is 4. Reading it properly needs a probe mod, which +/// this crate now has. +pub fn collect_define(doc_dir: &Path, table: &str) -> anyhow::Result { + let path = doc_dir.join("runtime-api.json"); + let text = std::fs::read_to_string(&path) + .map_err(|e| anyhow::anyhow!("reading {}: {e}", path.display()))?; + let api: Value = serde_json::from_str(&text) + .map_err(|e| anyhow::anyhow!("parsing {}: {e}", path.display()))?; + + let defines = api + .get("defines") + .and_then(|d| d.as_array()) + .ok_or_else(|| anyhow::anyhow!("{} has no defines array", path.display()))?; + + for define in defines { + if define.get("name").and_then(|n| n.as_str()) != Some(table) { + continue; + } + let mut out = Map::new(); + for value in define.get("values").and_then(|v| v.as_array()).unwrap_or(&vec![]) { + let (Some(name), Some(order)) = ( + value.get("name").and_then(|n| n.as_str()), + value.get("order"), + ) else { + continue; + }; + out.insert(name.to_string(), order.clone()); + } + return Ok(Value::Object(out)); + } + + Err(anyhow::anyhow!( + "could not find defines.{table} in {}", + path.display() + )) +} +``` + +Add to `src/trim/mod.rs`: + +```rust +pub mod defines; +``` + +- [ ] **Step 5: Run the tests** + +Run: `cargo test --lib trim::defines` +Expected: PASS, 4 tests. + +- [ ] **Step 6: Commit** + +```bash +git add -A +git commit -m "Port the defines reader, order bug and all + +runtime-api.json does not contain the values of defines. Across all 1,554 +entries in 2.1.14 the only keys are name, order and description, and order +is a dense 0..n-1 index across all 137 tables, so it cannot express a gap, +a duplicate, or a non-zero start. It is right today only because Factorio +declares directions clockwise from north = 0. + +Ported unchanged on purpose. The acceptance test compares byte for byte +against a fixture the buggy code produced, so fixing this in the same change +would make a port error and a deliberate fix look identical in the diff. +Tracked as FactorioTools#83 and fixed in its own commit." +``` + +--- + +### Task 7: Report which mods actually loaded + +`captureInfo.loadedMods` cannot come from the active-mods prelude. The committed fixture lists `core`, and `script.active_mods` does not report `core`. `dump-data` runs no mod at all, so there is no prelude. It has to come from the game's stdout. + +**Files:** +- Modify: `src/run.rs` + +**Interfaces:** +- Consumes: `SpawnResult` from plan 1. +- Produces: `pub fn loaded_mods(stdout: &str) -> Vec`, and a `loadedMods` key in the `run` result JSON. + +- [ ] **Step 1: Write the failing test** + +Add to the test module in `src/run.rs`: + +```rust + #[test] + fn loaded_mods_are_read_off_the_games_own_output() { + // Real lines from a 2.1.14 --dump-data run. + let stdout = "\ + 0.043 Loading mod core 0.0.0 (data.lua) + 0.053 Loading mod base 2.1.14 (data.lua) + 0.165 Loading mod recycler 2.1.14 (data.lua) + 0.173 Loading mod base 2.1.14 (data-updates.lua) + 0.177 Loading mod recycler 2.1.14 (data-updates.lua) + 0.674 Prototype list checksum: 3041708406 +"; + // Sorted and deduplicated: base loads three times across the stages. + assert_eq!(loaded_mods(stdout), vec!["base", "core", "recycler"]); + } + + #[test] + fn loaded_mods_includes_core_which_active_mods_does_not() { + // This is why the report cannot come from the script.active_mods + // prelude. FactorioTools' committed fixture lists core, and dump-data + // runs no mod at all so there is no prelude to ask. + let stdout = " 0.043 Loading mod core 0.0.0 (data.lua)\n"; + assert_eq!(loaded_mods(stdout), vec!["core"]); + } + + #[test] + fn a_mod_name_with_a_hyphen_or_underscore_survives() { + let stdout = "Loading mod elevated-rails 2.1.14 (data.lua)\n\ + Loading mod oracle_probe 0.0.1 (data.lua)\n"; + assert_eq!(loaded_mods(stdout), vec!["elevated-rails", "oracle_probe"]); + } + + #[test] + fn output_with_no_such_lines_gives_an_empty_list() { + assert!(loaded_mods("nothing to see here").is_empty()); + } +``` + +- [ ] **Step 2: Run it to see it fail** + +Run: `cargo test --lib run::tests::loaded_mods` +Expected: FAIL, `cannot find function loaded_mods`. + +- [ ] **Step 3: Write the implementation** + +Add to `src/run.rs`, above `run_probe`: + +```rust +/// The mods the game reported loading, sorted and deduplicated. +/// +/// Read from stdout rather than from the `script.active_mods` prelude, for two +/// reasons. `dump-data` runs no mod at all, so there is no control script to +/// host a prelude. And the prelude cannot see `core`: measured on 2.1.14, a +/// create run reported base and the DLC but never `core`, while the game's +/// output names it first. FactorioTools' committed fixture lists it. +/// +/// Hand-rolled rather than a regex, to keep the dependency surface small. The +/// line shape is `Loading mod (.lua)`. +pub fn loaded_mods(stdout: &str) -> Vec { + const MARKER: &str = "Loading mod "; + let mut names: Vec = stdout + .lines() + .filter_map(|line| { + let start = line.find(MARKER)? + MARKER.len(); + let rest = &line[start..]; + let name = rest.split_whitespace().next()?; + if name.is_empty() { + None + } else { + Some(name.to_string()) + } + }) + .collect(); + names.sort(); + names.dedup(); + names +} +``` + +In `run_probe`, add the key to the result object. Change the `json!` block that builds `out` so it also contains: + +```rust + "loadedMods": loaded_mods(&result.stdout), +``` + +- [ ] **Step 4: Run the tests** + +Run: `cargo test --all-targets` +Expected: PASS. + +- [ ] **Step 5: Prove it against the real game** + +Add to `tests/real_game.rs`: + +```rust +#[test] +fn a_real_dump_data_run_reports_the_bundled_mod_set() { + let Some(found) = find_install() else { + eprintln!("skipping: no Factorio install found. Set FACTORIO_BIN to run this."); + return; + }; + + let work = tempfile::Builder::new() + .prefix("factorio-oracle-it-") + .tempdir() + .unwrap(); + + let spec: ProbeSpec = serde_json::from_value(serde_json::json!({ + "mode": "dump-data", + "timeout_seconds": 300, + })) + .unwrap(); + + let request = RunRequest { + map_gen_settings: spec.resolved_map_gen_settings(), + spec, + layout: found.layout, + version: found.version.unwrap(), + work_dir: work.path().to_path_buf(), + }; + + let result = run_probe(&request, &RealSpawner).unwrap(); + assert_eq!( + result["ok"], + true, + "the run failed: {}", + serde_json::to_string_pretty(&result).unwrap() + ); + + let mods: Vec = + serde_json::from_value(result["loadedMods"].clone()).unwrap(); + // core is the one the active-mods prelude cannot see. + assert!(mods.contains(&"core".to_string()), "got {mods:?}"); + assert!(mods.contains(&"base".to_string()), "got {mods:?}"); + + // The dump landed in the isolated write directory, not a shared one. + assert!(work + .path() + .join("write/script-output/data-raw-dump.json") + .is_file()); +} +``` + +Run: `cargo test --test real_game` +Expected: PASS, 3 tests. On 2.1.14 the reported set is base, core, elevated-rails, quality, recycler and space-age. + +- [ ] **Step 6: Commit** + +```bash +git add -A +git commit -m "Report the loaded mods from the game's own output + +captureInfo.loadedMods cannot come from the active-mods prelude. dump-data +runs no mod at all, so there is no control script to host one, and measured +on 2.1.14 the prelude never reports core while the game's output names it +first. FactorioTools' committed fixture lists core, so the fixture could not +be reproduced from script.active_mods. + +Hand-rolled rather than pulling in a regex crate: the line shape is +Loading mod (.lua)." +``` + +--- + +### Task 8: Assemble the fixture + +**Files:** +- Modify: `src/trim/mod.rs` + +**Interfaces:** +- Consumes: everything from Tasks 1 to 6. +- Produces: `pub struct TrimInputs<'a> { pub dump: &'a Value, pub spec: &'a TrimSpec, pub data_dir: &'a Path, pub doc_dir: &'a Path, pub factorio_version: &'a str, pub loaded_mods: &'a [String] }` and `pub fn build_fixture(inputs: &TrimInputs) -> anyhow::Result`. + +- [ ] **Step 1: Write the failing test** + +Add to `src/trim/mod.rs`: + +```rust +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + use tempfile::tempdir; + + fn spec() -> spec::TrimSpec { + serde_json::from_value(json!({ + "comment": "Do not hand-edit.", + "entities": ["pumpjack"], + "entity_fields": ["collision_box"], + "connection_fields": ["positions"], + "fluid_boxes": ["output_fluid_box"], + "name_lists": { "modules": "module" }, + "defines": { "directions": "direction" }, + "include_renames": true + })) + .unwrap() + } + + fn dump() -> Value { + json!({ + "item": { "pumpjack": { "stack_size": 20 } }, + "mining-drill": { "pumpjack": { + "collision_box": [[-1.2, -1.2], [1.2, 1.2]], + "output_fluid_box": { "pipe_connections": [{ "positions": [[1, -1]] }] } + }}, + "module": { "speed-module": {}, "efficiency-module": {} } + }) + } + + fn game_dirs() -> tempfile::TempDir { + let dir = tempdir().unwrap(); + std::fs::create_dir_all(dir.path().join("data/base/migrations")).unwrap(); + std::fs::write( + dir.path().join("data/base/migrations/2.0.0.json"), + r#"{"item": [["effectivity-module", "efficiency-module"]]}"#, + ) + .unwrap(); + std::fs::create_dir_all(dir.path().join("doc-html")).unwrap(); + std::fs::write( + dir.path().join("doc-html/runtime-api.json"), + r#"{"defines": [{"name": "direction", "values": [ + {"name": "north", "order": 0}, {"name": "east", "order": 4}]}]}"#, + ) + .unwrap(); + dir + } + + #[test] + fn assembles_every_section_the_caller_asked_for() { + let dirs = game_dirs(); + let dump = dump(); + let spec = spec(); + let mods = vec!["base".to_string(), "core".to_string()]; + let fixture = build_fixture(&TrimInputs { + dump: &dump, + spec: &spec, + data_dir: &dirs.path().join("data"), + doc_dir: &dirs.path().join("doc-html"), + factorio_version: "2.1.14", + loaded_mods: &mods, + }) + .unwrap(); + + assert_eq!(fixture["_comment"], "Do not hand-edit."); + assert_eq!(fixture["captureInfo"]["factorioVersion"], "2.1.14"); + assert_eq!(fixture["captureInfo"]["loadedMods"], json!(["base", "core"])); + assert_eq!(fixture["directions"]["east"], 4); + assert_eq!(fixture["entities"]["pumpjack"]["prototypeType"], "mining-drill"); + assert_eq!(fixture["modules"], json!(["efficiency-module", "speed-module"])); + assert_eq!(fixture["renames"]["item"]["effectivity-module"], "efficiency-module"); + } + + #[test] + fn a_named_entity_that_no_longer_exists_is_a_loud_failure() { + // An entity the consumer names but the game does not have is exactly + // the drift this tool is built to catch. Writing a quietly incomplete + // fixture would be the silent pass it exists to prevent. + let dirs = game_dirs(); + let dump = dump(); + let mut spec = spec(); + spec.entities.push("quantum-pumpjack".to_string()); + let err = build_fixture(&TrimInputs { + dump: &dump, + spec: &spec, + data_dir: &dirs.path().join("data"), + doc_dir: &dirs.path().join("doc-html"), + factorio_version: "2.1.14", + loaded_mods: &[], + }) + .unwrap_err() + .to_string(); + assert!(err.contains("quantum-pumpjack"), "got {err}"); + assert!(err.contains("do not delete them"), "got {err}"); + } + + #[test] + fn sections_the_caller_did_not_ask_for_are_absent() { + let dirs = game_dirs(); + let dump = dump(); + let spec: spec::TrimSpec = + serde_json::from_value(json!({ "entities": [] })).unwrap(); + let fixture = build_fixture(&TrimInputs { + dump: &dump, + spec: &spec, + data_dir: &dirs.path().join("data"), + doc_dir: &dirs.path().join("doc-html"), + factorio_version: "2.1.14", + loaded_mods: &[], + }) + .unwrap(); + assert!(fixture.get("_comment").is_none()); + assert!(fixture.get("renames").is_none()); + assert!(fixture.get("directions").is_none()); + assert!(fixture.get("modules").is_none()); + } + + #[test] + fn numbers_are_normalised_on_the_way_out() { + let dirs = game_dirs(); + let long = "0.394500000000000028421709430404007434844970703125"; + let dump: Value = serde_json::from_str(&format!( + r#"{{"mining-drill": {{"pumpjack": {{"collision_box": [{long}]}}}}}}"# + )) + .unwrap(); + let spec: spec::TrimSpec = serde_json::from_value(json!({ + "entities": ["pumpjack"], "entity_fields": ["collision_box"] + })) + .unwrap(); + let fixture = build_fixture(&TrimInputs { + dump: &dump, + spec: &spec, + data_dir: &dirs.path().join("data"), + doc_dir: &dirs.path().join("doc-html"), + factorio_version: "2.1.14", + loaded_mods: &[], + }) + .unwrap(); + let text = canonical::to_canonical_json(&fixture); + assert!(text.contains("0.3945"), "got {text}"); + assert!(!text.contains("0.39450000000000002"), "got {text}"); + } +} +``` + +- [ ] **Step 2: Run it to see it fail** + +Run: `cargo test --lib trim::tests` +Expected: FAIL, `cannot find function build_fixture`. + +- [ ] **Step 3: Write the implementation** + +Add to `src/trim/mod.rs`, above the test module: + +```rust +use crate::trim::spec::TrimSpec; +use serde_json::{Map, Value}; +use std::path::Path; + +/// Everything `build_fixture` needs. The caller resolves the install and runs +/// the game, so nothing here launches anything. +pub struct TrimInputs<'a> { + /// A parsed `data-raw-dump.json`. + pub dump: &'a Value, + pub spec: &'a TrimSpec, + /// The install's `data` directory, for migrations. + pub data_dir: &'a Path, + /// The install's `doc-html` directory, for `runtime-api.json`. + pub doc_dir: &'a Path, + pub factorio_version: &'a str, + pub loaded_mods: &'a [String], +} + +/// Builds the fixture document. +/// +/// Only the sections the caller asked for appear. A consumer that wants entity +/// geometry and nothing else gets a file with `captureInfo` and `entities`, not +/// four empty objects. +pub fn build_fixture(inputs: &TrimInputs) -> anyhow::Result { + let raw = inputs + .dump + .as_object() + .ok_or_else(|| anyhow::anyhow!("the dump is not a JSON object"))?; + + let mut entities = Map::new(); + let mut missing: Vec<&str> = Vec::new(); + for name in &inputs.spec.entities { + match prototypes::find_prototype(raw, name) { + Some((kind, proto)) => { + entities.insert( + name.clone(), + prototypes::trim_entity(&kind, proto, inputs.spec), + ); + } + None => missing.push(name), + } + } + if !missing.is_empty() { + // A named entity that no longer exists is exactly the failure this tool + // is built to catch, so it is loud rather than a quietly incomplete + // file. + anyhow::bail!( + "these entities are named by the caller but do not exist in this Factorio \ + version: {}. That is a real finding - fix the consumer, do not delete them \ + from the trim spec.", + missing.join(", ") + ); + } + + let mut fixture = Map::new(); + if let Some(comment) = inputs.spec.comment.as_ref() { + fixture.insert("_comment".to_string(), Value::String(comment.clone())); + } + + let mut capture = Map::new(); + capture.insert( + "factorioVersion".to_string(), + Value::String(inputs.factorio_version.to_string()), + ); + let mut mods: Vec = inputs.loaded_mods.to_vec(); + mods.sort(); + capture.insert( + "loadedMods".to_string(), + Value::Array(mods.into_iter().map(Value::String).collect()), + ); + fixture.insert("captureInfo".to_string(), Value::Object(capture)); + + for (output_key, table) in &inputs.spec.defines { + fixture.insert( + output_key.clone(), + defines::collect_define(inputs.doc_dir, table)?, + ); + } + + if !entities.is_empty() { + fixture.insert("entities".to_string(), Value::Object(entities)); + } + + for (output_key, prototype_type) in &inputs.spec.name_lists { + let mut names: Vec = raw + .get(prototype_type) + .and_then(|v| v.as_object()) + .map(|o| o.keys().cloned().collect()) + .unwrap_or_default(); + names.sort(); + fixture.insert( + output_key.clone(), + Value::Array(names.into_iter().map(Value::String).collect()), + ); + } + + if inputs.spec.include_renames { + fixture.insert( + "renames".to_string(), + renames::collect_renames(inputs.data_dir), + ); + } + + Ok(canonical::normalise_numbers(&Value::Object(fixture))) +} +``` + +- [ ] **Step 4: Run the tests** + +Run: `cargo test --lib trim` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "Assemble the fixture from the pieces + +Only the sections the caller asked for appear, so a consumer wanting entity +geometry alone gets captureInfo and entities rather than four empty objects. + +A named entity the game does not have is a loud failure. That is exactly the +drift this tool is built to catch, and writing a quietly incomplete file +would be the silent pass it exists to prevent." +``` + +--- + +### Task 9: The `trim` subcommand, with `--check` + +**Files:** +- Modify: `src/main.rs` + +**Interfaces:** +- Consumes: `build_fixture`, `to_canonical_json`, `install::resolve`. +- Produces: `factorio-oracle trim --run --spec --out [--check]`. + +- [ ] **Step 1: Add the subcommand** + +In `src/main.rs`, add to the `Command` enum: + +```rust + /// Trim a dump into a consumer's fixture + Trim { + /// The JSON a `run` produced. Names the dump, the install and the mods. + #[arg(long)] + run: PathBuf, + /// The caller's trim spec + #[arg(long)] + spec: PathBuf, + /// Where to write the fixture + #[arg(long)] + out: PathBuf, + /// Report drift against `--out` and change nothing. Exits 1 on a + /// mismatch. + #[arg(long)] + check: bool, + }, +``` + +- [ ] **Step 2: Handle it** + +Add the match arm: + +```rust + Command::Trim { + run, + spec, + out, + check, + } => { + let run_result: serde_json::Value = + serde_json::from_str(&std::fs::read_to_string(&run)?)?; + let trim_spec: factorio_oracle::trim::spec::TrimSpec = + serde_json::from_str(&std::fs::read_to_string(&spec)?)?; + + let script_output = run_result["scriptOutput"] + .as_str() + .ok_or_else(|| anyhow::anyhow!("the run result has no scriptOutput"))?; + let dump_path = PathBuf::from(script_output).join("data-raw-dump.json"); + let dump: serde_json::Value = + serde_json::from_str(&std::fs::read_to_string(&dump_path)?)?; + + // The install is re-derived from the binary the run recorded, so + // the fixture cannot describe a different install than the dump. + let binary = run_result["provenance"]["binaryPath"] + .as_str() + .ok_or_else(|| anyhow::anyhow!("the run result has no binaryPath"))?; + let layout = install::resolve(Path::new(binary)) + .ok_or_else(|| anyhow::anyhow!("could not resolve the install at {binary}"))?; + + let version = run_result["provenance"]["factorioVersion"] + .as_str() + .ok_or_else(|| anyhow::anyhow!("the run result has no factorioVersion"))?; + let loaded_mods: Vec = + serde_json::from_value(run_result["loadedMods"].clone()).unwrap_or_default(); + + let fixture = factorio_oracle::trim::build_fixture( + &factorio_oracle::trim::TrimInputs { + dump: &dump, + spec: &trim_spec, + data_dir: &layout.data_dir, + doc_dir: &layout.doc_dir, + factorio_version: version, + loaded_mods: &loaded_mods, + }, + )?; + let text = factorio_oracle::trim::canonical::to_canonical_json(&fixture); + + if check { + let committed = std::fs::read_to_string(&out).unwrap_or_default(); + if committed == text { + println!("Up to date: {} matches Factorio {version}.", out.display()); + } else { + eprintln!( + "DRIFT: {} does not match Factorio {version}.", + out.display() + ); + eprintln!("Re-run without --check to update it, then review what moved."); + std::process::exit(1); + } + } else { + if let Some(parent) = out.parent() { + std::fs::create_dir_all(parent)?; + } + std::fs::write(&out, &text)?; + println!("Wrote {}", out.display()); + } + } +``` + +Add `use std::path::Path;` to the imports if it is not already there. + +- [ ] **Step 3: Confirm `install::resolve` has the signature this uses** + +Run: `grep -n "pub fn resolve" src/install.rs` +Expected: a function taking a path and returning `Option`. If it takes something else, adapt the call rather than changing `install.rs`, which plan 1's tests cover. + +- [ ] **Step 4: Build and check the help text** + +Run: `cargo run --quiet -- trim --help` +Expected: the four flags, with `--check` described as reporting drift. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "Add the trim subcommand, taking a run result as its input + +One JSON in, one JSON out. The run result already names the dump, the +binary, the version and the loaded mods, so trim needs no environment +variables and cannot be pointed at a dump from a different install than the +provenance it stamps on the output. That was the shape of the shell script +this replaces. + +--check reports drift and changes nothing, so 'has the game moved past what +we committed?' is answerable without a dirty tree." +``` + +--- + +### Task 10: The acceptance test, byte for byte + +Two halves. The first runs in CI with no game. The second runs only where the matching Factorio version is installed. + +**Files:** +- Create: `tests/fixtures/data-raw-slice.json` +- Create: `tests/fixtures/factoriotools-trim-spec.json` +- Create: `tests/fixtures/expected-factorio-oracle-2.1.14.json` +- Create: `tests/acceptance.rs` + +**Interfaces:** +- Consumes: `build_fixture`, `to_canonical_json`. +- Produces: nothing. This is the gate. + +- [ ] **Step 1: Build the committed input slice** + +The slice holds the full prototypes of every wanted name from every type that has one, plus the `module` keys. Measured at 163 KB, which is small enough to commit and large enough to exercise the disambiguation: all ten names appear in three types and `stone-wall` appears in four. + +With a real install present, run this from `~/GitHub/factorio-oracle`: + +```bash +cargo run --quiet -- run --probe /tmp/dump-data.json --work-dir /tmp/oracle-dump +python3 - <<'EOF' +import json +d = json.load(open('/tmp/oracle-dump/write/script-output/data-raw-dump.json')) +want = ["pumpjack", "pipe", "pipe-to-ground", "small-electric-pole", + "medium-electric-pole", "big-electric-pole", "substation", "beacon", + "heat-pipe", "stone-wall"] +out = {} +for kind, protos in d.items(): + if not isinstance(protos, dict): + continue + for name in want: + if name in protos: + out.setdefault(kind, {})[name] = protos[name] +out["module"] = {k: {} for k in d.get("module", {})} +json.dump(out, open("tests/fixtures/data-raw-slice.json", "w"), indent=2, sort_keys=True) +EOF +``` + +where `/tmp/dump-data.json` is `{ "mode": "dump-data", "timeout_seconds": 300 }`. + +If no install is available, the slice can be produced from any 2.1.14 `data-raw-dump.json` with the same script. + +- [ ] **Step 2: Write the trim spec, which is FactorioTools' allowlists verbatim** + +Create `tests/fixtures/factoriotools-trim-spec.json`: + +```json +{ + "comment": "Generated by tools/capture-factorio-oracle.sh. Do not hand-edit. Raw Factorio prototype values that the oil field planner depends on. Re-capture after a Factorio update and commit the diff.", + "entities": [ + "pumpjack", + "pipe", + "pipe-to-ground", + "small-electric-pole", + "medium-electric-pole", + "big-electric-pole", + "substation", + "beacon", + "heat-pipe", + "stone-wall" + ], + "entity_fields": [ + "collision_box", + "selection_box", + "tile_width", + "tile_height", + "supply_area_distance", + "maximum_wire_distance", + "distribution_effectivity", + "distribution_effectivity_bonus_per_quality_level", + "module_slots", + "beacon_counter", + "allowed_effects", + "energy_usage" + ], + "connection_fields": [ + "connection_type", + "direction", + "position", + "positions", + "flow_direction", + "max_underground_distance" + ], + "fluid_boxes": ["fluid_box", "output_fluid_box", "input_fluid_box"], + "name_lists": { "modules": "module" }, + "defines": { "directions": "direction" }, + "include_renames": true +} +``` + +- [ ] **Step 3: Copy the expected output** + +```bash +cp /Users/ericjohnson/GitHub/FactorioTools/test/FactorioTools.Test/OilField/factorio-oracle.json \ + tests/fixtures/expected-factorio-oracle-2.1.14.json +``` + +- [ ] **Step 4: Write the offline acceptance test** + +Create `tests/acceptance.rs`: + +```rust +//! Reproducing FactorioTools' committed fixture, byte for byte. +//! +//! This is the gate on whether this tool can replace +//! `tools/capture-factorio-oracle.sh`. Semantic equality is not enough: the +//! shell script's `--check` mode is a `diff` against a committed file, so an +//! output differing by a float's last digit or a key's position would make +//! every future check permanently red for no real reason. +//! +//! The offline half uses a committed 163 KB slice of `data.raw`, so it runs in +//! CI with no game. The install-gated half runs the real thing. + +use factorio_oracle::trim::canonical::to_canonical_json; +use factorio_oracle::trim::{build_fixture, spec::TrimSpec, TrimInputs}; +use std::path::{Path, PathBuf}; + +const EXPECTED_VERSION: &str = "2.1.14"; + +fn fixtures() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures") +} + +fn read(name: &str) -> String { + std::fs::read_to_string(fixtures().join(name)) + .unwrap_or_else(|e| panic!("reading {name}: {e}")) +} + +/// The six mods a default 2.1.14 install loads, as the game reports them. +fn loaded_mods() -> Vec { + ["base", "core", "elevated-rails", "quality", "recycler", "space-age"] + .iter() + .map(|s| s.to_string()) + .collect() +} + +#[test] +fn the_committed_fixture_is_reproduced_byte_for_byte() { + let dump: serde_json::Value = serde_json::from_str(&read("data-raw-slice.json")).unwrap(); + let spec: TrimSpec = serde_json::from_str(&read("factoriotools-trim-spec.json")).unwrap(); + let expected = read("expected-factorio-oracle-2.1.14.json"); + + // Renames and defines come off the install's own directories, so the + // offline test needs a stand-in. These two are committed alongside the + // slice by Step 6. + let data_dir = fixtures().join("data"); + let doc_dir = fixtures().join("doc-html"); + + let mods = loaded_mods(); + let fixture = build_fixture(&TrimInputs { + dump: &dump, + spec: &spec, + data_dir: &data_dir, + doc_dir: &doc_dir, + factorio_version: EXPECTED_VERSION, + loaded_mods: &mods, + }) + .unwrap(); + + let actual = to_canonical_json(&fixture); + if actual != expected { + // A unified diff of the first difference is far more useful than + // "assertion failed", because the whole point is which byte moved. + let mut line = 0; + for (a, b) in actual.lines().zip(expected.lines()) { + line += 1; + if a != b { + panic!("first difference at line {line}\n ours: {a}\n expected: {b}"); + } + } + panic!( + "same prefix, different length: ours {} lines, expected {} lines", + actual.lines().count(), + expected.lines().count() + ); + } +} + +#[test] +fn the_real_install_reproduces_it_too() { + use factorio_oracle::install; + use factorio_oracle::probe::ProbeSpec; + use factorio_oracle::run::{run_probe, RunRequest}; + use factorio_oracle::spawn::RealSpawner; + + let home = PathBuf::from(std::env::var("HOME").unwrap_or_default()); + let env_bin = std::env::var_os("FACTORIO_BIN").map(PathBuf::from); + let Some(found) = install::discover(&home, env_bin.as_deref()) + .into_iter() + .find(|d| { + d.version + .as_ref() + .map(|v| format!("{}.{}.{}", v.major, v.minor, v.patch) == EXPECTED_VERSION) + .unwrap_or(false) + }) + else { + eprintln!( + "skipping: no Factorio {EXPECTED_VERSION} install found. The expected fixture is \ + version-specific, so another version would fail for the wrong reason." + ); + return; + }; + + let work = tempfile::Builder::new() + .prefix("factorio-oracle-acceptance-") + .tempdir() + .unwrap(); + + let probe: ProbeSpec = + serde_json::from_value(serde_json::json!({ "mode": "dump-data", "timeout_seconds": 300 })) + .unwrap(); + let layout = found.layout.clone(); + let request = RunRequest { + map_gen_settings: probe.resolved_map_gen_settings(), + spec: probe, + layout: found.layout, + version: found.version.unwrap(), + work_dir: work.path().to_path_buf(), + }; + let result = run_probe(&request, &RealSpawner).unwrap(); + assert_eq!(result["ok"], true, "the dump-data run failed: {result}"); + + let dump: serde_json::Value = serde_json::from_str( + &std::fs::read_to_string(work.path().join("write/script-output/data-raw-dump.json")) + .unwrap(), + ) + .unwrap(); + let spec: TrimSpec = serde_json::from_str(&read("factoriotools-trim-spec.json")).unwrap(); + let mods: Vec = serde_json::from_value(result["loadedMods"].clone()).unwrap(); + + let fixture = build_fixture(&TrimInputs { + dump: &dump, + spec: &spec, + data_dir: &layout.data_dir, + doc_dir: &layout.doc_dir, + factorio_version: EXPECTED_VERSION, + loaded_mods: &mods, + }) + .unwrap(); + + assert_eq!( + to_canonical_json(&fixture), + read("expected-factorio-oracle-2.1.14.json"), + "the real install did not reproduce the committed fixture" + ); +} +``` + +- [ ] **Step 5: Run the install-gated half first** + +Run: `cargo test --test acceptance the_real_install` +Expected: PASS on a machine with 2.1.14. This half needs no committed stand-in directories, so it is the quickest way to find a real porting error. + +If it fails, the panic names the first differing line. Work through them one at a time; do not adjust the expected file. + +- [ ] **Step 6: Commit the stand-in game directories for the offline half** + +The offline test needs migrations and a `runtime-api.json`. Copy only what the spec reads: + +```bash +mkdir -p tests/fixtures/doc-html tests/fixtures/data +A="$HOME/Library/Application Support/Steam/steamapps/common/Factorio/factorio.app/Contents" +for d in "$A"/data/*/migrations; do + mod=$(basename "$(dirname "$d")") + mkdir -p "tests/fixtures/data/$mod/migrations" + cp "$d"/*.json "tests/fixtures/data/$mod/migrations/" 2>/dev/null || true +done +python3 - <<'EOF' +import json, os +a = os.path.expanduser("~/Library/Application Support/Steam/steamapps/common/Factorio/factorio.app/Contents") +api = json.load(open(f"{a}/doc-html/runtime-api.json")) +# Only the defines array is read, and only the direction table is asked for. +# The full file is 1,554 entries and several megabytes. +slim = {"defines": [d for d in api.get("defines", []) if d.get("name") == "direction"]} +json.dump(slim, open("tests/fixtures/doc-html/runtime-api.json", "w"), indent=2, sort_keys=True) +EOF +du -sh tests/fixtures +``` + +Expected: a few hundred kilobytes in total. + +- [ ] **Step 7: Run the offline half** + +Run: `cargo test --test acceptance the_committed_fixture` +Expected: PASS. + +- [ ] **Step 8: Prove the offline half runs with no game** + +Run: `HOME=/tmp/no-factorio cargo test --test acceptance` +Expected: PASS, with the install-gated test printing its skip message. + +- [ ] **Step 9: Commit** + +```bash +git add -A +git commit -m "Reproduce FactorioTools' fixture byte for byte + +This is the gate on whether this tool can replace +tools/capture-factorio-oracle.sh. Semantic equality would not be enough: the +script's --check mode is a diff against a committed file, so an output +differing by a float's last digit or a key's position would turn every +future check permanently red for no real reason. + +Two halves. The offline one uses a committed 163 KB slice of data.raw and +runs in CI with no game. All ten names appear in three prototype types and +stone-wall in four, so the collision-box disambiguation is genuinely +exercised rather than assumed. The install-gated one runs the real game and +skips unless the install is 2.1.14, because the expected file is +version-specific and another version would fail for the wrong reason." +``` + +--- + +### Task 11: Fix FactorioTools#83, on purpose and on its own + +Only start this once Task 10 passes. The point of the ordering is that a port error and a deliberate change must never appear in the same diff. + +**Files:** +- Modify: `src/trim/defines.rs` +- Modify: `src/trim/mod.rs` +- Modify: `src/trim/spec.rs` +- Modify: `tests/acceptance.rs` + +**Interfaces:** +- Consumes: the `create` mode from plan 1. +- Produces: `pub fn defines_from_probe(probe_dump: &Value, table: &str) -> anyhow::Result`, and a `defines_from` field on `TrimSpec`. + +- [ ] **Step 1: Know what the answer has to be** + +Already measured. A `create` probe on 2.1.14 calling `helpers.write_file` with `defines.direction` returned: + +```json +{"north":0,"northnortheast":1,"northeast":2,"eastnortheast":3,"east":4, + "eastsoutheast":5,"southeast":6,"southsoutheast":7,"south":8, + "southsouthwest":9,"southwest":10,"westsouthwest":11,"west":12, + "westnorthwest":13,"northwest":14,"northnorthwest":15} +``` + +That is identical to what `order` produces today, so **this fix changes no bytes in the fixture**. That is the ideal shape for it: the method becomes sound, and the acceptance test proves the change is safe rather than merely plausible. + +- [ ] **Step 2: Write the failing test** + +Add to the test module in `src/trim/defines.rs`: + +```rust + #[test] + fn a_probe_dump_gives_the_value_the_game_actually_uses() { + // Measured on 2.1.14 with a create probe. This is a read, not an + // inference: only the running game knows east is 4. + let probe = serde_json::json!({ + "directions": { "north": 0, "east": 4, "south": 8, "west": 12 } + }); + let table = defines_from_probe(&probe, "directions").unwrap(); + assert_eq!(table["east"], 4); + assert_eq!(table["west"], 12); + } + + #[test] + fn a_probe_dump_can_express_what_order_cannot() { + // The reason the fix matters. `order` is a dense 0..n-1 index, so it + // cannot represent a gap, a duplicate, or a non-zero start. A real + // reading can. + let probe = serde_json::json!({ "t": { "a": 10, "b": 10, "c": 40 } }); + let table = defines_from_probe(&probe, "t").unwrap(); + assert_eq!(table["a"], 10); + assert_eq!(table["b"], 10); + assert_eq!(table["c"], 40); + } + + #[test] + fn a_probe_dump_missing_the_table_names_it() { + let probe = serde_json::json!({ "other": {} }); + let err = defines_from_probe(&probe, "directions").unwrap_err().to_string(); + assert!(err.contains("directions"), "got {err}"); + } +``` + +- [ ] **Step 3: Run it to see it fail** + +Run: `cargo test --lib trim::defines` +Expected: FAIL, `cannot find function defines_from_probe`. + +- [ ] **Step 4: Write the implementation** + +Add to `src/trim/defines.rs`: + +```rust +/// Reads a defines table out of a probe mod's dump. +/// +/// This is the sound way to answer "what number means east". `collect_define` +/// above infers it from a documentation index; this reads it from the running +/// game, which is the only authority. FactorioTools#83. +/// +/// The probe writes the table under a key of its own choosing, so the caller +/// names it. Measured on 2.1.14, the values match what `order` produces, which +/// is why adopting this changes no bytes today. It changes what happens the +/// next time Factorio introduces a gap, a duplicate, or a non-zero start, none +/// of which a dense index can express. +pub fn defines_from_probe(probe_dump: &Value, key: &str) -> anyhow::Result { + let table = probe_dump + .get(key) + .ok_or_else(|| anyhow::anyhow!("the probe dump has no `{key}` table"))?; + if !table.is_object() { + anyhow::bail!("the probe dump's `{key}` is not an object"); + } + Ok(table.clone()) +} +``` + +Add to `TrimSpec` in `src/trim/spec.rs`: + +```rust + /// Where `defines` values come from. + /// + /// `doc-index` reads `order` out of `runtime-api.json`, which is a + /// documentation index rather than the runtime value. It is right only + /// while a table is a dense sequence from zero. `probe` reads the value the + /// running game uses, and needs a `create` run's dump. + /// + /// The default is `doc-index` so existing callers keep working. New callers + /// should choose `probe`. See FactorioTools#83. + #[serde(default)] + pub defines_from: DefinesSource, +``` + +and, in the same file: + +```rust +/// Where defines values are read from. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Deserialize)] +#[serde(rename_all = "kebab-case")] +pub enum DefinesSource { + /// `order` from `runtime-api.json`. An inference, not a reading. + #[default] + DocIndex, + /// The value the running game reports, via a `create` probe. + Probe, +} +``` + +In `src/trim/mod.rs`, add `pub probe_dump: Option<&'a Value>` to `TrimInputs`, and replace the defines loop with: + +```rust + for (output_key, table) in &inputs.spec.defines { + let value = match inputs.spec.defines_from { + spec::DefinesSource::DocIndex => defines::collect_define(inputs.doc_dir, table)?, + spec::DefinesSource::Probe => { + let probe = inputs.probe_dump.ok_or_else(|| { + anyhow::anyhow!( + "defines_from is `probe`, so a probe dump is required. Run a \ + create-mode probe that writes defines.{table} and pass it in." + ) + })?; + defines::defines_from_probe(probe, output_key)? + } + }; + fixture.insert(output_key.clone(), value); + } +``` + +Update every `TrimInputs` construction in tests and in `main.rs` to pass `probe_dump: None`. + +- [ ] **Step 5: Add the acceptance test that proves the fix changes nothing** + +Add to `tests/acceptance.rs`: + +```rust +#[test] +fn reading_defines_from_the_game_produces_the_same_bytes() { + // FactorioTools#83. The doc-index route infers direction values from a + // dense documentation ordering; the probe route reads what the game uses. + // Measured on 2.1.14 they agree, so adopting the sound method is a + // no-op on the output. This test is what makes that claim checkable + // rather than asserted. + let dump: serde_json::Value = serde_json::from_str(&read("data-raw-slice.json")).unwrap(); + let mut spec: TrimSpec = serde_json::from_str(&read("factoriotools-trim-spec.json")).unwrap(); + spec.defines_from = factorio_oracle::trim::spec::DefinesSource::Probe; + + // Exactly what a create probe wrote on 2.1.14. + let probe = serde_json::json!({ "directions": { + "north": 0, "northnortheast": 1, "northeast": 2, "eastnortheast": 3, + "east": 4, "eastsoutheast": 5, "southeast": 6, "southsoutheast": 7, + "south": 8, "southsouthwest": 9, "southwest": 10, "westsouthwest": 11, + "west": 12, "westnorthwest": 13, "northwest": 14, "northnorthwest": 15 + }}); + + let mods = loaded_mods(); + let fixture = build_fixture(&TrimInputs { + dump: &dump, + spec: &spec, + data_dir: &fixtures().join("data"), + doc_dir: &fixtures().join("doc-html"), + factorio_version: EXPECTED_VERSION, + loaded_mods: &mods, + probe_dump: Some(&probe), + }) + .unwrap(); + + assert_eq!( + to_canonical_json(&fixture), + read("expected-factorio-oracle-2.1.14.json"), + "reading defines from the game changed the fixture, which was not expected" + ); +} +``` + +- [ ] **Step 6: Run everything** + +Run: `cargo fmt --all && cargo clippy --all-targets -- -D warnings && cargo test --all-targets` +Expected: PASS. + +- [ ] **Step 7: Commit** + +```bash +git add -A +git commit -m "Read defines from the game, not from a documentation index + +FactorioTools#83. runtime-api.json does not contain the values of defines: +across all 1,554 entries in 2.1.14 the only keys are name, order and +description. Reading order as the value is right only while a table is a +dense sequence from zero, and it cannot express a gap, a duplicate, or a +non-zero start. + +Direction encoding is the exact constant that silently broke in 2.0, and it +was the one thing here that was inferred rather than read. + +Measured on 2.1.14 with a create probe: the values match what order +produces, so adopting the sound method changes no bytes today. An acceptance +test asserts exactly that, which turns 'this fix is safe' from a claim into +something checked. What changes is the next time the two disagree. + +doc-index stays the default so existing callers keep working. New callers +should pass defines_from: probe." +``` + +--- + +### Task 12: Point FactorioTools at the new tool + +The last step is to say, in the repo that has the fixture, what now produces it. + +**Files:** +- Modify: `/Users/ericjohnson/GitHub/FactorioTools/CLAUDE.md` +- Modify: `/Users/ericjohnson/GitHub/FactorioTools/tools/capture-factorio-oracle.sh` + +**Interfaces:** +- Consumes: the finished `trim` command. +- Produces: documentation only. No planner code changes. + +- [ ] **Step 1: Correct the four-sources claim** + +`CLAUDE.md` says the capture pulls four sources and lists `data/changelog.txt`. It pulls three. There is no changelog handling anywhere in `tools/`. The changelog is a research source a human reads; it was written into the table as if it were automated. Remove that row, and keep the changelog mentioned in prose as a thing to read. + +- [ ] **Step 2: Record that the shell script has a replacement** + +Add a note to `tools/capture-factorio-oracle.sh`'s header comment and to `CLAUDE.md`'s oracle section: `FactoryGameFan/factorio-oracle` reproduces this fixture byte for byte, the acceptance test proves it, and the migration rule is new probes only, so this script stays until someone has a reason to touch it. + +- [ ] **Step 3: Commit in the FactorioTools repo** + +```bash +cd /Users/ericjohnson/GitHub/FactorioTools +git add -A +git commit -m "Correct the capture's source count, and name its replacement + +The table said four sources and listed data/changelog.txt. The capture +reads three, and there is no changelog handling anywhere in tools/. The +changelog is a research source a human reads; it had been written into the +table as though it were automated. + +Also records that FactoryGameFan/factorio-oracle now reproduces this +fixture byte for byte, with an acceptance test that proves it. The script +stays: the agreed migration rule is new probes only." +``` + +--- + +## Self-Review + +**1. Spec coverage.** This plan covers build-order step 4 in full, plus the parts of "The output contract", "Determinism" and "Testing" that step 4 needs. Checked section by section against the spec: + +- *Commands*: `trim` and `--check` are Task 9. `installs list` and `run` were plan 1. +- *The probe spec*: unchanged except `defines_from`, added in Task 11. +- *The output contract*: `loadedMods` added in Task 7; the rest was plan 1. +- *Determinism*: Task 4 covers sorted keys, indent, trailing newline and the float trap. The `BTreeMap` requirement is a global constraint and is satisfied by not enabling `preserve_order`. +- *Guards*: the contamination report becomes a real value in Task 7 rather than an echo. The freshness guard is designed out by plan 1's isolated `write-data`, now verified. +- *Testing*: Task 10 is the split the spec asks for. + +Still deferred, each to its own plan: + +- **Plan 3:** `provenance check`, the always-on completeness test, and the `unknown` ratchet. Task 8 writes a `captureInfo` block, which is provenance's smallest form; plan 3 generalises it and adds the evidence grade. +- **Plan 4:** `refs` sync, grep at a tag, worktree, the archive cache, and the three knowledge documents. + +**2. Placeholder scan.** No TBD, TODO, "add error handling", or "similar to Task N". Every code step carries its code and every test step carries its assertions. Task 10 Steps 1 and 6 are shell scripts that generate committed fixtures rather than inline data, which is deliberate: a 163 KB slice cannot be pasted into a plan, and hand-writing one would test a shape nobody's game produces. + +**3. Type consistency.** Checked across tasks. `TrimSpec` (Task 1) is consumed by Tasks 3, 8, 9, 10 and extended in Task 11; every construction in the plan uses the same field names. `find_prototype` returns `Option<(String, &Value)>` in Task 2 and is destructured that way in Task 8. `trim_entity(kind: &str, proto: &Value, spec: &TrimSpec)` in Task 3 is called with exactly those types in Task 8. `collect_renames(&Path) -> Value` (Task 5) and `collect_define(&Path, &str) -> anyhow::Result` (Task 6) match their call sites. `normalise_numbers` and `to_canonical_json` (Task 4) are used in Tasks 5, 8, 9 and 10. `loaded_mods(&str) -> Vec` (Task 7) feeds `TrimInputs::loaded_mods: &[String]` (Task 8). + +One consistency note for the implementer, the same one plan 1 hit: Task 11 adds `probe_dump` to `TrimInputs`, so every construction in Tasks 8, 9 and 10 stops compiling until it is added. That is intended. Fix them by passing `probe_dump: None`, not by giving the field a default. + +**4. Ordering risk.** Task 4 changes a crate-wide `serde_json` feature, and plan 1's tests are the safety net. If enabling `arbitrary_precision` breaks something in `run.rs`, fix it there. Do not drop the feature, or Task 10 will pass today and fail silently the first time Factorio puts a long literal in a wanted field. + +## Execution Handoff + +Plan complete and saved to `docs/superpowers/plans/2026-08-17-factorio-oracle-trimmer.md`. Two execution options: + +1. **Subagent-Driven (recommended)** - a fresh subagent per task, review between tasks, fast iteration. +2. **Inline Execution** - execute tasks in this session with checkpoints for review. diff --git a/docs/superpowers/specs/2026-08-16-shared-factorio-oracle-design.md b/docs/superpowers/specs/2026-08-16-shared-factorio-oracle-design.md new file mode 100644 index 00000000..3b325a86 --- /dev/null +++ b/docs/superpowers/specs/2026-08-16-shared-factorio-oracle-design.md @@ -0,0 +1,788 @@ +# A shared Factorio oracle CLI + +Status: design agreed, not yet built. +Tracking issues: FactorioTools#82, factorio-blueprint-editor#235, +FactorioMapWebUI#232, FactorioWikiDamageThresholds#11. +Related: FactorioTools#83. + +This spec lives here because FactorioTools already has the `docs/superpowers/specs/` +convention and no oracle repo exists yet. Move or copy it into that repo once it is +created. + +## The problem + +Four repos depend on facts that only Factorio can answer. Three of them run the +game headless to ask. Each wrote the same plumbing from scratch, and a fourth copy +turned up in a repo that has nothing to do with Factorio. + +| Repo | Oracle tooling | Lines | Install discovery | +| --- | --- | --- | --- | +| FactorioTools | `tools/capture-factorio-oracle.sh` + `trim-factorio-oracle.py` | 421 | own copy | +| factorio-blueprint-editor | `tools/oracle/`, 18 probes | 7,900 | own copy, x18 | +| FactorioMapWebUI | `test/oracle/` | 9,504 | own copy | +| FactorioWikiDamageThresholds | a hand-rolled serpent mod, output committed | - | none | + +The plumbing does not scale with the question. In factorio-blueprint-editor it runs +**50 to 69 lines with a median of 56**, across probes ranging from 79 to 872 lines +total - 1,071 lines in all. In FactorioMapWebUI the same ten-step sequence appears +four times (249 lines). That is about 1,320 lines of near-identical plumbing in +those two repos alone, before counting the roughly 80% of FactorioTools' 421 lines +that is equally generic. More to the point, it is about 23 independent copies, and +so 23 independent chances to get it subtly wrong. + +That risk is measured, not theoretical: + +- **14 of 18** factorio-blueprint-editor probes hardcode `factorio_version: '2.1'`. + Six of those call `--version` anyway, but only to stamp a fixture. A mismatch + makes Factorio skip the mod in silence; the run ends on "No dump" and nothing + names the cause. +- **Only 1 of 18** checks the binary exists before spawning. +- **None of the three repos has a timeout.** A hung game hangs the capture forever. +- FactorioWikiDamageThresholds' committed 21.3 MB dump is Lua serpent rather than + JSON, is truncated mid-capture, and carries no version stamp - and about 40 + hardcoded values derived from it feed public wiki pages. + +## What this is + +A single Rust binary, `factorio-oracle`, in its own public repo. It owns the +plumbing. It does not own the questions, and it does not own the analysis. + +**JSON in, JSON out.** A consumer describes a probe, the tool runs it, the consumer +reads the result and compares it against its own model. + +### Why the analysis cannot be shared + +This is the constraint that decides the whole shape. A probe compares the game +against **the consumer's own reimplementation**, so its analysis has to run in the +consumer's language. + +factorio-blueprint-editor's README says so directly about `probe-rail-placement.mjs`: +it reads `packages/exporter/data/output/data.json` rather than reading a bounding +box back out of the game, because the question is what `PositionGrid`'s integer +tile grid sees, so the footprint has to come from the same data `getEntitySize` +reads. FactorioMapWebUI's probes compare against `fmw-noise`; FactorioTools' +compare against C#. + +So a shared *probe framework* is impossible. A shared *runner* is not. + +### Why Rust + +The three repos pin Node at 24.19.0, 26.7.0, and nothing. FactorioMapWebUI also +declares `devEngines.packageManager: pnpm 11.18.0`, which already breaks `npx`-based +tools there. A shared Node library would run under whichever Node the shell +inherited, two majors apart depending on the directory. + +A single static binary has no such problem, and it serves a .NET repo, a Node repo, +a Rust repo and a Python repo identically. Since the analysis stays with the +consumer, the tool's language is invisible across the boundary - so it should be +whichever produces the most reliable artifact. `serde` also gives a typed, +versioned provenance schema for free, and MapWebUI already treats Rust as a pinned +first-class toolchain. + +Subprocess overhead is irrelevant: Factorio's own headless launch is about 1.7 +seconds. + +## Commands + +``` +factorio-oracle installs list discover every install; print version, build, path +factorio-oracle run --probe spec.json run a probe; emit result + provenance +factorio-oracle refs sync pin factorio-data and cache the API docs +factorio-oracle refs grep search factorio-data at a tag, without moving HEAD +factorio-oracle refs worktree materialise a tree at a tag, for tools that need one +factorio-oracle provenance check report which fixtures predate the selected install +``` + +### Install discovery + +`FACTORIO_BIN` wins if set. Otherwise the union of every candidate list found +across the four repos, plus the one from the stray benchmark script: + +- `~/Library/Application Support/Steam/steamapps/common/Factorio/factorio.app` +- `/Applications/factorio.app` +- `~/.steam/steam/steamapps/common/Factorio` +- `~/.factorio` +- `/opt/factorio` + +macOS ships an `.app` bundle and Linux a plain directory, so the tool resolves +three paths per install: the binary, `data/`, and `doc-html/`. + +**Multi-version is a requirement, not a feature.** The consumers target different +versions on purpose - FactorioTools 2.1.14, factorio-blueprint-editor 2.0.45 to +2.0.73 against a corpus spanning 2.0.32 to 2.1.12, FactorioMapWebUI whatever Steam +last pushed. There is also an unstripped 2.0.77 build outside any install +directory. So `installs list` enumerates rather than picking, and every command +takes `--factorio ` or `--version `. + +**The version is always derived, never assumed.** Two values come out of +`--version`: the full build line (`Version: 2.0.77 (build 84539, mac-arm64, full)`), +which is what fixtures stamp, and the `major.minor` a mod's `info.json` needs. The +implementation to port is `probe-entity-tile-size.mjs:103-114` together with its +failure message at `:202-210`, which names the version mismatch as the likely cause +of an empty dump. Those two belong together. + +## Run modes + +There is no single execution shape. Five are needed, and the differences are not +cosmetic. + +| Mode | Launch | Mod | Success is | Used by | +| --- | --- | --- | --- | --- | +| `dump-data` | `--dump-data` | none | exit 0, then dump exists | FactorioTools, wiki repo | +| `create` | `--create ` | generated | dump exists (exit is 1) | FBE, MapWebUI | +| `interactive` | `--load-scenario ` | generated | consumer decides | FBE | +| `preview` | `--generate-map-preview` | none | **exit 0** and the PNG exists | MapWebUI | +| `read-only` | no binary at all | none | files read | FactorioTools | + +Three things follow from this table. + +**The success predicate is per mode.** `error("DUMPED-OK")` makes Factorio exit +non-zero, and that is success - so `create` keys off the dump file. But +`--generate-map-preview` exits 0 on success, and MapWebUI's `render.mjs:59-61` is +right to check the code. And for `dump-data` a non-zero exit is real information: +the diagnostic is the last 30 lines of the log, not a missing file. One global rule +would break two of the five. + +**`dump-data` scaffolds no mod.** Its mod directory exists only to be *empty* - a +contamination control, not a scaffold. Mods rewrite prototypes freely, so a capture +that loads them describes one person's game rather than Factorio. + +**`read-only` needs no game.** Migrations and `runtime-api.json` are files on disk, +and `wube/factorio-data` ships byte-identical migrations. So renames can be checked +with no install at all. + +## The probe spec + +A JSON document. Every Lua field accepts an inline string or a file path. + +```jsonc +{ + "mode": "create", + "factorio": { "version": "2.0.77" }, // or "path", or omit for the default + "mod": { + "name": "bp_rail_placement", + "version": "0.0.1", + "dependencies": ["base", "elevated-rails", "space-age"], + "control_lua": "...", // or control_lua_file + "data_lua": null, + "data_final_fixes_lua": null + }, + "map_gen_settings": { "seed": 123456 }, // object or file path; required for create + "literals": { "blueprint": "0eNq..." }, // become Lua locals; see below + "timeout_seconds": 300, + "capture_active_mods": true +} +``` + +Rules the consumers force: + +**Lua goes in opaque.** All 18 factorio-blueprint-editor probes generate their Lua +in JavaScript, interpolating case lists, sweep window sizes and base64 blueprint +strings. The runner must not template, escape or rewrite any of it. + +**The runner must not wrap the body.** Wrapping in `script.on_init` would be a +convenient default and would make `probe-zoom-limits.mjs` impossible, because it +uses `on_tick` plus three `commands.add_command` registrations. + +**`literals` is the one exception**, and it exists to kill a real gotcha. Each key +becomes a Lua local declared above the consumer's `control_lua`, with its value +inside a long bracket, so base64 and quotes survive verbatim: + +```lua +local blueprint = [==[0eNq...]==] +-- the consumer's control_lua follows, unmodified +``` + +The bracket level is chosen so it cannot collide with the value's own contents. A +consumer that would rather build the string itself simply passes no `literals` and +embeds it in its own Lua, which is what every probe does today. + +**Both data-stage files are supported.** A probe mod declares no dependencies, so +its `data.lua` may run before `space-age`'s and `data.raw.resource[...]` will not +exist yet - a silent no-op. Prototype overrides belong in `data_final_fixes_lua`. + +**`map_gen_settings` is optional for `create`, and so is the seed flag.** +Measured 2026-08-16 against 2.1.14, by FactorioMapWebUI, with a probe mod that +reads back `game.surfaces[1].map_gen_settings.seed` - the seed the surface +actually got: + +| arm | file seed | flag seed | surface seed | result | +| --- | --- | --- | --- | --- | +| both | 111111 | 222222 | **222222** | flag wins | +| flag only, no settings file | - | 222222 | 222222 | works | +| neither | - | - | 3972429021 | works, random | +| settings file only | 111111 | - | 111111 | file used | + +Arms two and three generated a map, loaded the mod and produced a dump with no +settings file at all. So "always passed" was habit in the consumer repos, not a +requirement of the game. The CLI passes it when the caller supplies one and omits +it otherwise. + +**`--map-gen-seed` overrides the seed inside the settings file.** That is arm one: +the file says 111111, the flag says 222222, and the surface comes out 222222. Arm +four rules out the file simply being ignored - without the flag, the file's seed is +what you get. Precedence is flag over file. + +**So the CLI takes one `seed` field and writes both channels.** With the precedence +now known, this is not merely defensive: a tool that wrote only the file while a +caller also passed a flag would be silently overridden. Writing both from one +source makes them agree, which makes the precedence irrelevant. + +The failure this avoids is the bad kind - everything runs, nothing errors, and the +numbers come from a different map. FactorioMapWebUI has already paid for a +seed-provenance mistake once: a correct field compared against the wrong seed +convention scored 0.5% overlap where the right convention scored 99.9%, and nothing +about the failing run looked like a seed problem. + +A note for anyone reading a consumer's harness: the same measurement showed that +FactorioMapWebUI's `seed` field inside its map-gen settings JSON **has never done +anything**, because that harness always passes the flag too. No fixture there is +wrong, since the two values always agreed - but it is a dead write that looks +load-bearing, and its `mapGenOverrides` path silently discards a `seed` passed +through it. Tracked on that repo's #232. + +**The mod directory does both jobs.** For `create` and `interactive` it is an +isolated directory the runner owns *containing* the generated mod. For `dump-data` +and `preview` it is the same isolated directory, empty. The directory name must +carry the `_` suffix matching `info.json`. + +**Space Age needs three mods**, not one: `space-age` depends on `elevated-rails` +and `quality`. + +## The output contract + +**The runner returns the work directory, not "the dump".** + +This is the single most important interface decision, and it comes from +`probe-zoom-limits.mjs`. That probe runs the graphics client with a human at the +keyboard, streams newline-delimited JSON in append mode as the person scrolls, +takes input through in-game commands, writes three files, and has a three-level +success ladder: dump missing, dump present but *voided* because no character +controller ever arrived, or usable. Four of the guarantees a dump-centric interface +would offer invert for it. + +Returning the directory also covers the multi-dump and JSONL cases for free, and it +keeps freshness and voiding decisions with the consumer, who is the only party that +can make them. + +```jsonc +{ + "ok": true, + "workDir": "/tmp/factorio-oracle-abc123", + "scriptOutput": "/tmp/factorio-oracle-abc123/write/script-output", + "files": ["oracle-dump.json", "zoom-samples.jsonl"], + "exitCode": 1, + "sentinelSeen": true, + "provenance": { "factorioVersion": "2.0.77", "...": "the block described below" } +} +``` + +On failure the tool returns the same shape with `ok: false` **and the tail of +Factorio's stdout and stderr**. Every existing probe prints that tail by hand +because it is the only diagnostic there is; a JSON-out CLI that dropped it would be +a regression. The best failure message pairs the tail with the derived +`factorio_version` and the binary's own version line, because a mismatch between +them is the most common cause of an empty dump. + +**Read stdout. Factorio writes nothing to stderr.** Measured 2026-08-17 on 2.1.14 +across three cases: a `create` run whose control script called +`error("DUMPED-OK")`, a mod with an error in its `data.lua`, and an unknown +command line flag. All three printed to stdout and left stderr at zero bytes. The +stderr tail is still worth returning, in case a later version changes its mind, +but an empty one means nothing on its own. + +`sentinelSeen` reports whether `DUMPED-OK` appeared in the game's output. +Seventeen probes share that convention and nothing checks it, so it cannot +currently distinguish "the mod ran and finished" from "the mod crashed". The +runner can append the `error()` call itself and check for it. + +That check is worth more than it looks. A `create` run keys success off the dump +existing, so a probe that writes its dump and then dies on a later line still +counts as a pass. The sentinel is the only thing that catches it - which is why +reading it off the wrong stream is worse than not reporting it at all. Reported as +`false`, it looks like an answer. + +### Guards + +- **Timeout.** No consumer has one today. Default a few minutes, configurable, and + never applied in `interactive` mode. +- **Freshness.** Design it out rather than check for it. Every mode gets an + isolated `config.ini` whose `write-data` points at a scratch directory that + started empty, so a leftover dump from an older capture cannot be picked up. That + is what seventeen probes already achieve with a fresh `mkdtemp`, and it is what + FactorioTools' current script lacks - it discovers a shared user data directory + and therefore needs an mtime check to prove the dump belongs to this run. + + The mtime check stays, but as belt and braces rather than the primary defence, + and it is **optional**: `probe-zoom-limits.mjs` deliberately reuses a named + directory and appends across a session, so a mandatory check would break it. + + ~~One thing to verify during implementation rather than assume: that + `--dump-data` honours `write-data` for `script-output`.~~ **Verified 2026-08-17 + on 2.1.14: it does.** A `--dump-data` run with an isolated `config.ini` wrote + `data-raw-dump.json` and `mod-settings-dump.json` into the scratch directory's + own `script-output`, and nothing landed in the shared user data directory. So + every mode gets the isolation, and the mtime check is belt and braces + everywhere rather than load-bearing anywhere. The whole run took 2.9 seconds. +- **Contamination. On by default.** Report which mods actually loaded, and fail if + the set is not the expected one. Mods rewrite prototypes freely, so a capture + that loads them describes one person's game rather than Factorio - and it looks + completely normal, which is why this defaults on rather than being opt-in. Today + six factorio-blueprint-editor probes capture it, FactorioTools only greps stdout + for `Loading mod` (which works for `dump-data` alone), and FactorioMapWebUI + captures none. + + **The prelude registers no event at all.** Measured 2026-08-16 on 2.1.14: + `helpers.write_file` works at `control.lua` toplevel with no event, and + `script.active_mods` is populated there. So the whole prelude is one line. + + **Toplevel is for metadata, not for sampling, and the distinction matters.** + `game.surfaces[1]` does not exist at control-stage toplevel, so anything calling + `calculate_tile_properties` or `get_tile` still needs `on_init`. That is why + every sampling probe in FactorioMapWebUI registers one, and why those + registrations are load-bearing rather than habit. The two requirements compose + rather than compete: the prelude registers nothing, a sampling probe registers + exactly one `on_init`, and nothing contests the single slot. Do not read the + toplevel finding as "probes should stop using `on_init`". + + That matters because `script.on_init` takes exactly one handler, which the same + measurement proved rather than assumed: an `instrument-control.lua` registering + `on_init` had its handler silently discarded once `control.lua` registered one + too. No error - the handler simply never ran. 17 of 18 factorio-blueprint-editor + probes register an `on_init`, so any prelude using one would vanish. A toplevel + write has no collision surface and costs no ticks. + + The reported set should include the probe's own throwaway mod - it is proof the + mod loaded, which is the thing most worth knowing when a run produces no dump. + + **An empty mod directory keeps out user mods, and nothing else.** Measured + 2026-08-17 on 2.1.14, and it corrects an assumption that had gone unstated: + Factorio rewrites `mod-list.json` during startup and adds back every mod bundled + with the install that the file does not mention, with `enabled: true`. A file + naming only `base` came back naming base, elevated-rails, quality, recycler and + space-age, and all five loaded. **Omission means enabled.** + + A control arm proved both halves in one run: elevated-rails, quality and + space-age were listed with `enabled: false` and stayed out of the load order, + while recycler, left unmentioned in the same file, was added and loaded. So an + explicit `enabled: false` is honoured, and naming a mod is the only way to get a + smaller game than the install ships with. + + Loading the full bundled set is the right default, because that is what the + consumers' committed fixtures were captured against - FactorioTools' fixture + records exactly those six, counting `core`. The point is that it should be a + decision rather than a side effect, and that "the mod directory is empty" must + not be read as "only base is loaded". +- **Binary exists** before spawning. +- **Large output buffer** always, so a big dump cannot truncate the diagnostic. + +## Provenance + +Copied from FactorioMapWebUI, which has the best version of this. + +**Provenance lives beside the fixtures, not inside them.** Several fixtures are +verbatim copies of the game's own JSON and are asserted key for key, so an added +metadata key is data pollution. + +```jsonc +{ + "_comment": ["array of strings, so it stays readable in a diff"], + "fixtures": { + "": { + "factorioVersion": "2.0.77", + "factorioBuild": "build 84539, mac-arm64, full", + "branch": "stable", + "loadedMods": ["base", "core", "elevated-rails", "quality", "recycler", "space-age"], + "capturedOn": "2026-08-16", + "capturedBy": "tools/oracle/probe-rail-placement.mjs", + "targetVersionRange": "2.0.45-2.0.73", + "evidence": "stated | inferred | unknown, plus free text" + } + } +} +``` + +- `evidence` grades how the version was established. `stated` beats `inferred` + beats `unknown`. +- `branch` records stable versus experimental. FactorioTools deliberately targets + experimental, and the only planner-relevant difference between 2.0.77 stable and + 2.1.14 experimental is the pumpjack's output fluid box. A capture with no branch + marker cannot answer "which game is this". +- `targetVersionRange` is what the *consumer* targets, which can differ from the + binary captured. +- Keys are bare filenames and cover PNGs too, since nothing about a PNG hints at + which game produced it. + +**Enforcement splits in two, and the split is the point.** + +1. An always-on test that needs no Factorio: every fixture has an entry, no entry + is dangling, every entry is well formed, and a **ratchet caps the number of + `unknown` entries** so the gap can only shrink. +2. A version-comparison report that needs a binary and **always exits 0**, because + deciding whether a version gap matters needs a human. A fixture captured on + 2.1.11 is not wrong because the binary moved on. + +**A fixture's provenance is a record of the moment it was captured, not a live +claim.** Never hand-edit one to make it current, and never edit one to make a test +pass. A mismatch is a finding. + +## Reference material + +### factorio-data: read at a tag, never move HEAD + +`~/GitHub/factorio-data` is one clone with one working tree, and at least two repos +already name it as their prototype source while targeting different versions. Only +FactorioMapWebUI can currently pin it, and it does so with `git checkout`. + +The problem is live, not hypothetical. The clone is on branch `master` right now, +not detached at any tag. `refs:sync --check` reports "in sync" only because +`master` happens to equal the newest tag. That is a coincidence. MapWebUI's own +notes already record the failure inside one repo, when a second binary is used: +pointing `FACTORIO_BIN` at the 2.0.77 install de-syncs both references from the +binary the fixtures were validated against. + +So: **`refs grep` and an internal `show` read at a tag without touching `HEAD`.** +This is already where the repos are drifting - MapWebUI re-ran an entire audit at +2.1.14 "without repinning anything, because the question only needs the tags". + +For tools that need a real directory tree - ripgrep, an editor, a Lua parser - +`refs worktree ` gives each repo its own tree off one object store, with no +contention. That is better than forcing everything through `git show`. + +Two details: `git fetch --tags` is still needed before a never-seen tag can be +read, and `git grep ` prefixes every output path with `:`, so +anything parsing the output must strip it. + +The blast radius is small. No runtime code in MapWebUI reads factorio-data - every +hit under `src/`, `test/`, `crates/`, `scripts/` and `preview-service/` is a +comment - and CI never touches it. What changes is three functions in +`sync-factorio-refs.sh` plus some doc recipes. + +### API docs: cache the archive, extract on demand + +`factorioLuaAPI/` is **286 MB and 3,371 files per version**. Three repos on three +versions is about 860 MB before any history, so caching extracted trees does not +scale. Cache the published archive per version and extract when needed. + +Caching only the JSON will not do: the JSON is not a superset of the HTML. +`control:temperature:frequency` appears in `noise-expressions.html` and nowhere in +`runtime-api.json`. + +**And the JSON does not contain what people assume it does.** Verified against the +installed 2.1.14 file: 0 of 1,408 define values carry a value field, and `order` is +a dense `0..n-1` index across all 137 define tables, with values stored +alphabetically by name. So `runtime-api.json` cannot answer "what number is east", +and neither can `defines.html`. Only the running game knows. This is FactorioTools#83, +and it is the strongest argument for the `create` mode existing at all. + +## Determinism + +`--check`-style drift detection is a `diff -u` against a committed file, so any +nondeterminism turns it into a permanent false alarm. + +- Sort every map. Rust's `serde_json` does not sort by default and `HashMap` + iteration order is randomised per process, so use `BTreeMap` or sort explicitly. +- Match the existing formatting: two-space indent, trailing newline. +- **Float formatting is the trap.** `0.29`, `2.5`, `1.5` and `0.2` all appear in + FactorioTools' fixture. Any difference from Python's printer makes every future + check red. + +Acceptance test: the Rust tool must reproduce the current committed +`factorio-oracle.json` byte for byte before it replaces anything. + +### Sampled values must round-trip f32 exactly + +This is a separate requirement from the one above, on a separate path. Prototype +values are trimmed from a dump and have to match Python's printer. **Sampled +numeric values come back from the running game and have to survive as the exact +bits the game produced.** + +Requested by FactorioMapWebUI, and it is not a preference. Scoring a port by +**count of exactly matching f32 values** is a sharper instrument than any error +bound, and it only works if the capture preserves the bits. The evidence: two +candidate noise kernels had the *identical* worst absolute error, 2.682e-7, and +differed by 42 exact matches out of 512. An error bound could not tell them apart +at all. The winning variant went from 132 of 512 exact to 473 of 512. + +So: + +- Emit each value with a **shortest round-trip** representation. Rust's `{}` on + `f32` does this; `ryu` if it should be explicit. Never a fixed precision - + `{:.6}` or `%.9g` destroys the instrument. +- **Never widen f32 to f64** anywhere along the path. If the game produced an f32, + the output says so and keeps it. +- Self-test: capture, re-read, and assert `parse(serialize(v)) == v` bitwise for + every value. Cheap, and it fails loudly the day somebody tidies the formatter. + +The failure mode is what makes this worth spelling out: a capture that loses +precision still looks completely fine. Nothing errors. The consumer simply can +never again distinguish "bit-exact" from "very close". On the consumer side the +check is one line - `Math.fround(v) === v` across the fixture - which +FactorioMapWebUI now asserts before scoring anything. + +## What the tool must never do + +**Emit derived values.** Store raw prototype numbers only. Factorio's rule for +turning `supply_area_distance` into covered tiles is not one formula - poles come +out as `2*distance`, a beacon as `2*distance` plus its own footprint, and +substation fits neither reading. A guessed formula inside a fixture is confidently +wrong and drifts invisibly. Derive in the consumer, where a wrong derivation fails +loudly against a hardcoded value. + +**Hardcode an allowlist.** FactorioTools' ten wanted entities are exactly +`EntityNames.Vanilla`. A blueprint editor wants hundreds; a map tool wants none of +them. Allowlists are caller-supplied config. + +What the tool *should* keep from FactorioTools' trimmer is `find_prototype`'s type +search and its collision-box disambiguation. Most of these names exist twice, once +as the placeable entity and once as the carried item, and `data.raw["item"]["pumpjack"]` +has none of the geometry. + +## The knowledge base + +Three documents, lifted with attribution to the repo and issue each lesson came +from: + +- `docs/gotchas.md` - factorio-blueprint-editor's ~25 entries plus + FactorioMapWebUI's, each of which cost a run. +- `docs/method.md` - the epistemics. A control must be able to fail while the + hypothesis holds. Last man standing is not a measurement. Refute the rival. A + probe entity is part of the question. Ask the cheapest question that settles it. + Transcribe a proposed rule into the probe before writing code. Sweep two window + sizes and make "the wider one finds nothing new" an explicit control. +- `docs/order-of-attack.md` - factorio-data first, then the oracle, then the + binary. + +factorio-blueprint-editor's README already says its method was borrowed from +FactorioMapWebUI by hand. This gives that borrowing a home instead of a copy that +drifts. + +## Testing + +Mirror the split MapWebUI already proves works. The pure builders - `info.json`, +control-Lua assembly, `config.ini`, the argv vector, dump parsing, provenance +serialisation - are unit-tested with no Factorio present. The spawn boundary is +injectable, so a fake can assert the argv, write the dump the real game would have +written, and return a non-zero exit with `DUMPED-OK` on **stdout**, which is where +the game puts it. + +**Make the fake wrong in the same way the real game is, or the tests confirm the +bug.** The first version of the fake wrote the sentinel to stderr because the +design said stderr. Sixty unit tests passed against a `sentinelSeen` check that +was false on every real run. The fake had made the mistake unfalsifiable. + +**The integration test that runs only when an install is found is not optional.** +It caught three defects that the full unit suite passed: the sentinel read off the +wrong stream, a `mod-list.json` believed to load only `base`, and a seed hardcoded +in `main` so a spec's value went nowhere. It skips itself when no install is +present, so CI stays offline, matching all four consumers. Against 2.1.14 it runs +in under two seconds, which is cheap enough that there is no argument for +skipping it locally. + +Assert on things only a running game can answer: `defines.direction.east` is 4 and +came from the game rather than a docs index; the seed the spec asked for came back +off the surface; a consumer literal survived the trip into Lua; the probe's own mod +appears in the active-mods report, which is the proof it loaded at all. + +## Repo setup + +- Public GitHub repo at **`FactoryGameFan/factorio-oracle`**. Five of the six + Factorio repos moved to that org on 2026-08-16, so a new shared tool starting + anywhere else would be the odd one out from its first commit. +- `rust-toolchain.toml`, pinned. MapWebUI pins 1.97.1 as a correctness control + rather than a convenience, and that reasoning transfers if a crate is ever shared. +- **Renovate in the first commit.** The app runs with "Require config file" + enabled, so a new repo whose default branch has no valid config makes Renovate do + nothing at all, silently - indistinguishable from "no updates available". + `.github/renovate.json5`, JSON5 so the reasoning lives in comments beside each + rule, one weekly batch on Monday morning `America/Los_Angeles` with security + fixes outside that window, and `automerge: false` globally. Ecosystems are + `cargo` and `github-actions`. Validate with + `npx --yes --package renovate -- renovate-config-validator .github/renovate.json5`, + and keep exactly one config file in the repo. + +## Prior art, and why this is a build + +Nothing existing covers it. + +Four projects re-implement Factorio's data stage in embedded Lua: YAFC (C#, +patched Lua 5.2.1), factorio-draftsman (Python, lupa), KirkMcDonald/factorio-tools +(Go, cgo), factorio-scanner (Rust, mlua). They are fast and need no install, and +they are all approximations. YAFC ships the warning itself: "YAFC loads mods in +environment that is not completely compatible with Factorio." A tool whose premise +is that the game is the only authority should not be built on one. + +`factorio-rust-tools` is the closest match and is worth reading. Its CI downloads a +**pinned headless Factorio** from `factorio.com/get-download//headless/linux64` +and diffs against a committed 37 MB golden file. That contradicts the assumption in +several of our repos that a CI machine can never have Factorio installed, and it is +a real option for closing a gap FactorioTools already names as accepted: nothing +automatically notices a new Factorio release. + +Capturing real exported blueprint strings by running the game has **no prior art +anywhere**, confirmed on two independent search passes. + +Also confirmed: `--dump-data-raw` is not a real flag. The real set is +`--dump-data`, `--dump-prototype-locale` and `--dump-icon-sprites`. + +## Build order + +This is one coherent tool but more than one sitting. A natural spine, each step +useful on its own: + +1. **Repo skeleton.** Cargo project, pinned `rust-toolchain.toml`, Renovate config + validated, CI that builds and tests with no Factorio present. +2. **`installs list`.** Discovery, both version values, the `.app` versus directory + layouts. Smallest thing that is immediately useful, and it is the piece + duplicated most. +3. **`run` for `dump-data` and `create`.** The two modes with real consumers today. + Pure builders first, spawn boundary injectable, fake-game test. +4. **Determinism acceptance test.** Reproduce FactorioTools' committed + `factorio-oracle.json` byte for byte. This gates whether the tool can ever + replace that script. +5. **`interactive` and `preview`.** Both are one consumer each and neither blocks + the others. +6. **`provenance check`** plus the always-on completeness test and the `unknown` + ratchet. +7. **`refs`** - sync, grep at a tag, worktree, archive cache. +8. **The three knowledge documents.** Independent of all the code; can be done at + any point, including first. + +Steps 1 to 4 are the part that has to be right. Everything after is additive. + +## Out of scope for v1 + +- A shared fixture *format* beyond provenance. Each consumer needs a different + slice at a different version, so a common schema is guesswork until two of them + actually want the same field. +- A Rust library crate. Only one consumer could use it, and subprocess cost is + noise next to a 1.7 second game launch. +- Migrating any existing probe. The agreed rule is **new probes only**. +- Published binary releases. Build from source. +- Automated drift detection in CI via a pinned headless download. Worth doing, but + it is its own decision with its own cost. + +## Open questions + +1. ~~**The org move.**~~ **Done 2026-08-16/17. All six repos now live under + `FactoryGameFan`**, so a new shared tool starting anywhere else would be the odd + one out from its first commit. See "Repo setup" above. + + The blocker was specific to FactorioTools: `joelverhagen/FactorioTools#10` was + open and cross-repository with head `wormeyman:main` and 466 files, and GitHub + documents what happens to a fork on *deletion* and *detachment* but never on + *transfer*. It was closed first, deliberately, and then the repo moved. + + The move answered the question, and the answer is worth keeping because GitHub + does not document it: **a transfer rewrites a cross-repo pull request's head to + the new owner and preserves the pull request.** Verified afterwards - PR #10 + reads `head: FactoryGameFan:main`, still closed, both comments intact. Caveat: + it was closed before the transfer, so this establishes the closed case only. + + Everything else survived too: issues, fork links (FactorioTools still shows + parent `joelverhagen/FactorioTools`, factorio-blueprint-editor still shows + `teoxoy/factorio-blueprint-editor`), Actions secrets, private flags and branch + rulesets. No repo has a Cloudflare Git integration, so no deploy broke. + + Two operational notes. **The transfer API is asynchronous**: + `POST repos/{owner}/{repo}/transfer` returns the repo's pre-transfer state, so + verify with a follow-up read rather than trusting the response. And **old URLs + 301 correctly, so stale `wormeyman/` references survive silently** and still + need a sweep - the ones with teeth are each repo's `CLAUDE.md`, which is loaded + into every session and can point a future session at the old path. Do not blind + find-and-replace: the **Cloudflare account** is also called `wormeyman` and did + not move, so each hit has to be read for which one it means. +2. ~~Whether the runner injects a Lua prelude by default for + `script.active_mods`.~~ **Decided 2026-08-16: on by default**, using a + self-cancelling `on_nth_tick` rather than `on_init`. See the contamination + guard above. +3. ~~Whether `--instrument-mod` is a better launch path.~~ **Measured 2026-08-16 + on 2.1.14. Answer: no, and it makes the collision worse.** + + Instrument Mode does give earlier hooks. With `--instrument-mod`, + `instrument-data.lua` ran at 0.045s against `data.lua` at 0.129s, and + `instrument-control.lua` toplevel ran before `control.lua` toplevel. Without + the flag, neither instrument file loads at all. + + But `instrument-control.lua`'s `script.on_init` handler **never fired**, + because `control.lua` registered one too and the later registration replaced + it. So Instrument Mode gives a probe an earlier hook whose event registration + the consumer then silently destroys. + + That makes it **actively dangerous for a probe runner, not merely unhelpful.** + A tool built on Instrument Mode would appear to work on every probe that does + not register `on_init`, and fail silently on every probe that does. Keep this + entry even though the feature is not being adopted: "we watched the earlier + handler get destroyed by the later one" is a much stronger claim than "we + inferred the rule from the docs", and it is the one that survives somebody + proposing Instrument Mode again in a year. + + One quirk worth recording: `instrument-control.lua`'s toplevel logged twice in + the same run. Not investigated further, since the mode is not being adopted. + + The useful finding came out of the same run: `helpers.write_file` works at + plain `control.lua` toplevel with no event at all, and `script.active_mods` is + populated there. That is what the contamination prelude now uses. + +## First customer + +FactorioMapWebUI#234 is the first real consumer, and it is a **new** probe, so it +sits on the right side of the "new probes only" rule. + +After that repo's #214, `basisNoise` is 473 of 512 bit-exact and the remaining 39 +points come from the game's own gradient table, produced by a minimax polynomial +inside `Noise::Noise(bool)` rather than by libm. No formula recovers them. A +capture does: with `input_scale = 1`, sampling at `(I + 1/256, J)` leaves exactly +one cell corner contributing, so the value inverts directly to that slot's +gradient x component, and `(I, J + 1/256)` gives y. 256 slots, both components, +one capture. + +It needs nothing but "run this noise expression at these points and return the +numbers", which is the `create` mode shape. It is also the reason the f32 +round-trip rule above is not optional: the entire point is recovering exact f32 +constants. + +Sequencing: that repo's #220 takes priority, so there is no rush. + +## Decision log + +Settled, and not worth relitigating without new information: + +- Rust, not TypeScript. The three Node pins are mutually incompatible, and the + analysis half cannot be shared in any language. +- JSON in, JSON out. Not a probe framework. +- The runner returns a work directory, not a dump. +- The success predicate is per mode. +- Read factorio-data at a tag; never move `HEAD` in a shared clone. +- Provenance beside the fixtures, with an evidence grade and an `unknown` ratchet. +- Migration is new probes only. +- Read the game's output from stdout. Measured: stderr is always empty. +- Omission in `mod-list.json` means enabled, so a mod is only kept out by naming + it with `enabled: false`. +- One `seed` field on the spec, delivered through both channels. + +## What the first real run corrected + +Build-order steps 1 to 3 were implemented, and then run against a real 2.1.14 +install for the first time. Three things this document asserted turned out to be +wrong. All three had passed sixty unit tests. + +1. **The sentinel is on stdout, not stderr.** Factorio writes nothing to stderr at + all, in any of the three cases tried. `sentinelSeen` was therefore false on + every real run. +2. **A `mod-list.json` naming only `base` does not give a base-only game.** The + game adds back every bundled mod the file omits, enabled. +3. **The seed was hardcoded in `main`.** The design called for one `seed` field + feeding both channels, and the field did not exist, so every `create` run + generated the same map. + +Two of the three were invisible to the unit tests by construction, because the +fake game encoded the same wrong belief the code did. That is the lesson worth +carrying into the remaining plans: a fake can only be wrong in the ways its author +already considered, so any claim about what the game does has to be settled by +running it. The tool's own premise, which is that the game is the only authority, +applies to the tool. + +Three claims did survive contact and are now measured rather than assumed: +`--dump-data` honours `write-data` for `script-output`; `--create` runs with no +settings file; and a seed given once reaches the surface intact. diff --git a/tools/capture-factorio-oracle.sh b/tools/capture-factorio-oracle.sh index ee85ddf1..f422197e 100755 --- a/tools/capture-factorio-oracle.sh +++ b/tools/capture-factorio-oracle.sh @@ -19,6 +19,25 @@ # CI reads the committed fixture, which is why the fixture is committed rather than # generated on demand - CI machines have no Factorio install and never will. # +# There is a replacement, and it is proven equivalent +# --------------------------------------------------- +# FactoryGameFan/factorio-oracle is a shared Rust CLI doing this job for four repos. +# Its acceptance test reproduces this fixture BYTE FOR BYTE from a real 2.1.14 +# install, so that is checked rather than hoped for: +# +# factorio-oracle run --probe dump-data.json --work-dir /tmp/w > /tmp/run.json +# factorio-oracle trim --run /tmp/run.json --spec trim-spec.json --out +# +# The WANTED_* allowlists in trim-factorio-oracle.py move into that spec unchanged. +# This script stays anyway: the agreed migration rule across the four repos is new +# probes only. See issue #82. +# +# Two things that tool measured, which apply here too. An empty mod directory keeps +# out USER mods and nothing else - Factorio rewrites mod-list.json and re-enables +# every bundled mod the file omits, so the "base only" list written below still loads +# all six. And the fixture's `directions` come from `order` in runtime-api.json, which +# is a documentation index rather than the runtime value; see issue #83. +# # Requirements # ------------ # - A Factorio install (Steam or standalone). Only needed to re-capture, not to build. @@ -49,7 +68,10 @@ while [[ $# -gt 0 ]]; do --out) OUT="$2"; shift 2 ;; --user-data-dir) USER_DATA_DIR="$2"; shift 2 ;; --check) CHECK_ONLY=1; shift ;; - -h|--help) sed -n '2,36p' "${BASH_SOURCE[0]}"; exit 0 ;; + # Print the whole leading comment block, however long it grows. A hardcoded + # line range silently truncates the help the first time the header is edited, + # which is exactly what happened to the old '2,36p'. + -h|--help) awk 'NR>1 { if (/^#/) print; else exit }' "${BASH_SOURCE[0]}"; exit 0 ;; *) echo "unknown argument: $1" >&2; exit 2 ;; esac done