Extract a shared Factorio oracle CLI

[wormeyman]

Summary

Four repos now depend on facts pulled out of the Factorio game. Three of them run
the game headless to ask it questions, and each wrote the same plumbing from
scratch. The plan is to pull that plumbing into one shared Rust CLI,
factorio-oracle, in its own repo.

This repo is the fourth, and it is the one with the worst provenance problem.
Nothing here changes yet. The agreed migration rule is "new probes only". This
issue records what the tool would fix here so the decision is written down.

The four repos are FactorioTools, factorio-blueprint-editor, FactorioMapWebUI and
this one. Each is getting its own issue.

The problem here

data/factorioraw.json is 21.3 MB and was committed with the message "Added
Factorio raw data, very useful." Three things are wrong with it, and none of them
is visible from the filename.

It is not JSON. It is Lua table syntax, produced by a mod. The first line reads
Script @__DataRawSerpent__/data-final-fixes.lua:1: {. Anything that tries to
json.load it fails.

It is truncated. The last line is
--[[incomplete output with shared/self-references skipped]]. So it is not a
complete picture of the game's data, and nothing in the file says which parts are
missing.

It has no version stamp. Searching it for a version field returns nothing. So
there is no way to answer "which Factorio produced this", which means there is no
way to know whether it still describes the current game.

On top of that sit the hardcoded numbers in factorio_thresholds.py:24-77 - HP and
resistance tuples for every asteroid, biter, spawner, worm and pentapod - plus
factorio_productivity.py and factorio_mining.py. Those feed public wiki
pages. If Factorio changes one of them, nothing here notices. The output stays
plausible and becomes wrong, in public.

That is the exact failure the shared tool exists to catch. FactorioTools hit the
same class of bug twice: effectivity-module-N became efficiency-module-N in
2.0, and directions widened from 8-way to 16-way. Both went unnoticed for a long
time, because plans kept generating - they were just wrong.

What the shared tool would give this repo

A real capture, stamped with the version that produced it. One command that
finds the local Factorio, runs --dump-data with user mods disabled, and writes
the result with the game version, build string and the list of mods that actually
loaded. No hand-rolled serpent mod, no truncation.

Note this repo needs something the other three do not: a full, untrimmed
data.raw, consumed from Python. FactorioTools wants a small trimmed fixture of
ten entities. That difference is now in the design rather than being discovered
later.

A drift check. --check re-captures to a scratch file and diffs it against
what is committed, without dirtying the tree. Run it after a game update and it
tells you whether anything the wiki tables depend on has moved. That answers "is
the published table still true" without reading 21 MB by hand.

The migration table. Factorio ships data/*/migrations/*.json, which is a
complete list of every rename, not a guess. Checking a hardcoded name against it is
how the FactorioTools module rename was proven rather than remembered.

Provenance that is checked, not hoped for. The design copies FactorioMapWebUI's
scheme: a manifest beside the fixtures recording which game version each came from,
an evidence field grading how that was established (stated beats inferred beats
unknown), an always-on test that needs no game and fails if a fixture has no entry,
and a ratchet capping the number of unknown entries so the gap can only shrink.
Today this repo would start with everything unknown, which is at least honest.

Also worth knowing

The WikiArticles/*.json captures record no game version either. Same problem,
smaller stakes.

CLAUDE.md documents a hand-rolled probing method: /c console snippets that hook
defines.events.on_entity_damaged to count Tesla chain hits. That is exactly the
kind of question the shared tool is built to run repeatably, rather than by
retyping console commands. The other repos have 18 such probes between them and the
same technique underneath.

One caution from the FactorioTools work that applies directly here: store raw
values, never derived ones. Factorio's rule for turning a prototype number into a
usable figure is often not one formula. A guessed formula in a captured file is
confidently wrong and drifts invisibly. Derive in the code that consumes it, where
a wrong derivation fails loudly.

Prior art checked

Nothing existing covers this. Four projects re-implement Factorio's data stage in
embedded Lua (YAFC, factorio-draftsman, KirkMcDonald/factorio-tools,
factorio-scanner). They are fast and need no game install, and they are all
approximations. YAFC ships the warning itself: "YAFC loads mods in environment that
is not completely compatible with Factorio." For wiki numbers that people rely on,
an approximation is the wrong foundation.

factorio-rust-tools is the closest match and worth reading. Its CI downloads a
pinned headless Factorio from factorio.com/get-download/<version>/headless/linux64
and diffs against a committed golden file. That is a way to automate the drift
check rather than remember to run it.

Status

Design is agreed in outline: a Rust CLI, JSON in and JSON out, in a new public
repo. The write-up is not finished, and this issue will be updated with a link.

Separately, moving these repos under the FactoryGameFan org is proposed and not
decided. This repo is not a fork and has no deploy wired to GitHub, so a transfer
here is low risk when it happens. Transfers redirect clone, fetch and push,
and preserve secrets and deploy keys.

Related issues

One per affected repo, all describing the same shared tool:

• Extract a shared Factorio oracle CLI FactorioTools#82 - the trimming split, the defines bug, and the open upstream PR that gates its move
• Extract a shared Factorio oracle CLI factorio-blueprint-editor#235 - 18 probes, 1,071 lines of repeated plumbing, and the interactive probe that shaped the runner's interface
• Extract a shared Factorio oracle CLI FactorioMapWebUI#232 - the four duplicated wiring copies, the missing timeout, and the factorio-data tag contention

[wormeyman]

Status: the tool is built and the build order is done

FactoryGameFan/factorio-oracle is public, CI is green, and 167 tests pass. Nothing here changes the migration rule: it is still new probes only, and nothing in this repo needs to move.

What works today:

• factorio-oracle installs list - finds every install, reports both version forms and the three paths each one needs.
• factorio-oracle run --probe spec.json - five run modes (dump-data, create, interactive, preview, read-only), each with its own success rule. Returns the work directory, not "the dump", so a probe that writes several files or appends JSONL across a session works without special handling. Isolated config.ini per run, so a stale dump from an earlier capture cannot be picked up. There is a timeout, which none of the four repos had.
• factorio-oracle trim - cuts a data.raw dump down to a caller-supplied slice, with --check for drift.
• factorio-oracle provenance check|report - records which Factorio each fixture came from, and checks the record stays honest.

Three documents now carry the probe-writing knowledge, in docs/: order-of-attack.md, method.md, and gotchas.md with roughly 40 entries. Every borrowed lesson names the repo and issue it came from.

Not built: refs (sync, grep at a tag, worktree, archive cache). It is the last item and it is not blocking anything here.

What is settled for this repo specifically

This is the repo the provenance work was most aimed at, and it is now built.

provenance check <dir> answers, with no Factorio installed and no network:

• Does every file in a fixture directory say where it came from?
• Does every entry name a file that still exists?
• Is every entry well formed?
• Has the count of entries recording unknown grown?

That last one is a ratchet rather than a cap. It fails when the count rises above the declared maxUnknown, and it also fails when the count drops below it and the number is not lowered, naming the new value. A cap that never has to fall does not fall. That is stricter than the prior art on purpose.

A file that is deliberately not ground truth goes in a notFixtures list with a reason. There is no extension filter, because an ignore rule that costs nothing gets used without thinking - which is exactly how eight captured fixtures went unrecorded in a sibling repo (FactorioMapWebUI#240).

The second half, provenance report <dir>, compares each fixture's recorded version against an installed binary and always exits 0, because a fixture captured on an older version is not wrong, it just has not been re-validated. Deciding whether the gap matters needs a person.

For data/factorioraw.json: nothing here converts Lua table syntax to JSON, and this tool will not guess at that file's provenance. What it can do is give whatever replaces it a recorded origin from the start, so the next reader does not have to work backwards from a commit message. run with a dump-data probe produces real JSON straight from the game.

Nothing to do here yet. This is a status update.

[wormeyman]

Correction: refs shipped, so the status above is out of date

The status comment above went up about four hours before refs merged. Two
things in it are now wrong.

• It says "Not built: refs". It is built. PR feat: add Refined Flammables research tree (flamethrower vs biters/spitters) #10 merged as 6a6ddce on
  2026-08-17. It was the last item in the build order, so the tool is done.
• It says 167 tests pass. There are now 260: 247 unit and 13 integration.

I ran the full suite before writing this, and all 260 passed with none skipped.
That took setting both gate variables by hand. Left alone, two tests skip: one
wants FACTORIO_ORACLE_PROVENANCE_DIR, and one wants
FACTORIO_ORACLE_OLD_FACTORIO, meaning an install older than the crate's own
fixtures. Four more tests need a real Factorio, and four need a clone of
wube/factorio-data. Those gates are why a green run can prove less than it
looks, so it is worth saying which ones really ran.

What refs does

It reads Factorio's shipped Lua and its API docs at a version you name, and
it never moves HEAD in the shared clone. That is the whole point. One
clone of factorio-data sits on one branch, and several repos want to read
different versions of it.

Action	What it does
refs show <tag> <path>	Print one file at a tag
refs grep <term> --tag A --tag B	Search at one tag or several, and report a same-or-differs verdict
refs worktree <tag>	Make a real directory at a tag, for tools that need one. --remove cleans it up
refs docs <ver> <file>	Print a Lua API docs file. Reads an installed game before it touches the network
refs sync <ver> [--check]	Report whether a version's reference material can be read

show, grep and worktree need no Factorio installed. They need only the
clone. docs and sync do pick an install. A worktree costs 0.077 seconds and
8.6 MB, and two at the same tag can sit side by side.

What this changes here

This repo has the least Factorio of the four. The issue body says the captures
carry no version stamp, that WikiArticles/*.json records no game version
either, and that a real capture needs a pinned headless download. So the useful
fact about refs here is this one:

show, grep and worktree need no Factorio installed at all. They read
the factorio-data clone and nothing else. In the source, install::select is
reached only by the docs and sync arms. The other three never touch it. The
crate's four refs tests gate on finding the clone, which is a separate gate
from the install one, so a machine can have one and not the other.

That means this repo can read prototype definitions at a version without a
download, a Steam login, or a game.

The "identical" verdict is the half worth having

A capture with no version stamp is not wrong. It is just unchecked. What you
want to know is which values still need a second look. Both of these are real
runs from today:

$ factorio-oracle refs grep "decrease = " --tag 2.0.77 --tag 2.1.14
...
identical across 2.0.77 and 2.1.14

$ factorio-oracle refs grep damage-type --tag 2.0.77 --tag 2.1.14
...
differs between 2.0.77 and 2.1.14

The first one says: leave those captures alone. The second says: go look. That
is a triage list you can build without installing anything.

One caution, since it applies to this repo more than the others. A verdict on a
term that appears in changelog.txt will read "differs" simply because a later
tag has a longer changelog. Prototype Lua does not grow that way, so keep the
search terms to prototype fields where you can.

And the version stamp itself

provenance check <dir> is the other half, and it also runs with no Factorio
and no network. It answers whether every file in a fixture directory says where
it came from, whether every entry names a file that still exists, and whether
the count of entries recording unknown has grown. That last one is a ratchet,
not a cap: it fails when the count rises, and it also fails when the count
drops and nobody lowers the number.

For the WikiArticles/*.json captures, the realistic first step is entries
that record unknown honestly, with the ratchet holding the line so the number
can only fall.

The migration rule has not moved

Still new probes only. Nothing here has to change. This is a status update.