Read factorio-data at a tag, without ever moving its HEAD (step 7)

.gitignore

@@ -1,3 +1,15 @@
 /target
 /refs
 .DS_Store
+
+# Factorio installs kept for multi-version work. A 5.1 GB game bundle is not
+# source, and it is not ours to redistribute. The directory-level rule means
+# git never descends into it, so `git status` does not stat thousands of files.
+#
+# What lives here today: factorio-2.0.77.app, a full mac-arm64 build 84539.
+# It is deliberately OUTSIDE every candidate root in install.rs, so discovery
+# does not find it and `installs list` keeps reporting only real install
+# locations. Name it explicitly instead:
+#   cargo run -- provenance report tests/fixtures \
+#     --factorio installs/factorio-2.0.77.app
+/installs/

CLAUDE.md

@@ -43,18 +43,34 @@ cat /tmp/pj/write/script-output/oracle-dump.json
 # Reproduce FactorioTools' committed fixture
 cargo run -- run  --probe dump-data.json --work-dir /tmp/w > /tmp/run.json
 cargo run -- trim --run /tmp/run.json --spec trim-spec.json --out fixture.json [--check]
+
+# Read factorio-data at a tag. Never moves HEAD in the shared clone.
+cargo run -- refs grep support_range --tag 2.0.73 --tag 2.1.12
+cargo run -- refs worktree 2.0.77          # a real tree, under the cache
+cargo run -- refs docs 2.1.14 runtime-api.json --which
 ```
 
-Test counts to expect: **159 unit tests**, plus **8 integration tests** split
-across three files. `tests/acceptance.rs` has 3: two run offline against a
+Test counts to expect: **247 unit tests**, plus **13 integration tests** split
+across four files. `tests/acceptance.rs` has 3: two run offline against a
 committed fixture, and one (`the_real_install_reproduces_it_too`) is
-install-gated. `tests/provenance.rs` has 2: one always-on, and one gated on
-the `FACTORIO_ORACLE_PROVENANCE_DIR` environment variable naming another
-repo's fixture directory - not an install gate, so it skips even when
-Factorio is present. `tests/real_game.rs` has 3, all install-gated. That is
-**4 install-gated tests** in total. Without a real Factorio install they skip
-rather than fail, so a green run on a machine with no game proves less than it
-looks. Check which happened before trusting it.
+install-gated. `tests/provenance.rs` has 3: one always-on; one gated on the
+`FACTORIO_ORACLE_PROVENANCE_DIR` environment variable, which names another
+repo's fixture directory; and one gated on `FACTORIO_ORACLE_OLD_FACTORIO`,
+which names an install *older* than this crate's own committed fixtures.
+Neither variable is an install gate: a machine can have Factorio installed
+and still have no fixture directory to point at, and still have no *old*
+Factorio to point at either, so both tests skip even when Factorio is
+present. `tests/real_game.rs` has 3, all install-gated. That is **4
+install-gated tests** in total - `FACTORIO_ORACLE_OLD_FACTORIO`'s test is not
+one of the four, because it needs a specific old install named by hand, not
+just any install `discover` can find. `tests/refs.rs` has 4, all gated on
+finding the `~/GitHub/factorio-data` clone (or wherever `FACTORIO_DATA_DIR`
+points) - a separate gate from the install one, so a machine can have
+Factorio and no clone, or a clone and no Factorio. Without a real Factorio
+install, without an old one named by `FACTORIO_ORACLE_OLD_FACTORIO`, and
+without the clone, these tests skip rather than fail, so a green run on a
+machine with none of the three proves less than it looks. Check which
+happened before trusting it.
 
 ## Layout
 
@@ -72,6 +88,7 @@ looks. Check which happened before trusting it.
 | `numbers.rs` | Preserving the bits the game produced |
 | `trim/` | Cutting a full `data.raw` dump down to a consumer's slice |
 | `provenance/` | Which Factorio each fixture came from, and whether that record is still honest |
+| `refs/` | Reading factorio-data at a tag, and the Lua API docs, without moving HEAD |
 
 Five run modes, and **the success predicate differs per mode**: `dump-data`
 (no mod at all, the mod dir exists only to be empty), `create`, `interactive`
@@ -175,6 +192,76 @@ code did. **A fake can only be wrong in the ways its author already considered.*
   decimal, with no error. Recorded as a negative result so nobody spends an
   afternoon ruling it out. Provenance entries stay a `Value` anyway, because
   only two keys are required and the rest must round-trip untouched.
+- **`~/GitHub/factorio-data` is on `master`, and that makes one consumer's
+  drift check a coincidence.** Verified 2026-08-17: branch `master`, clean,
+  548 tags, and `base/info.json` reads 2.1.14, which is also the newest tag.
+  FactorioMapWebUI's `refs:sync --check` greps that file rather than asking
+  git anything, so it reads a coincidence as a pin. `refs` never moves `HEAD`
+  there, and `tests/refs.rs` asserts the clone is unchanged after a real run.
+- **Reading at a tag is byte-stable across platforms.** `git show
+  2.1.14:base/info.json` returned the same 193 bytes with `core.autocrlf`
+  unset, forced false and forced true, matching `git cat-file blob`. So no
+  line-ending rewriting is needed. A **worktree** checkout is a different
+  path and was not measured this way.
+- **factorio-data holds nothing binary.** At tag 2.1.14 it is 327 files: 296
+  `.lua`, 27 `.json`, 3 `.txt`, 1 `.md`, and no path contains a colon. That is
+  what makes it safe to carry `git` output through `SpawnResult`'s `String`,
+  and to split a grep line on colons.
+- **A worktree costs 0.077 seconds and 8.6 MB**, two worktrees at one tag
+  coexist, and `git worktree add` creates missing parent directories. It also
+  writes an entry into the shared clone's `.git/worktrees/`, 168 KB for three,
+  which is the one thing `refs` writes to a clone it does not own. That is why
+  `refs worktree --remove` exists.
+- **The installed game ships the API docs and the published archive too.**
+  Measured 2026-08-17 on 2.1.14: `doc-html/static/archive.zip` is byte-identical
+  to `lua-api.factorio.com/2.1.14/static/archive.zip`, both 45,547,463 bytes
+  and sha256 `87012e1c...`, and all 3,370 other files match the tree
+  FactorioMapWebUI unpacked from that archive. So for an installed version
+  there is nothing to download, and `install.rs` already resolves `doc_dir`.
+- **Single docs files are published, and the archive never wins on bytes.**
+  The server gzips HTML but not JSON. Measured **at 2.0.45**:
+  `runtime-api.json` was 1,597,033 bytes with or without
+  `Accept-Encoding: gzip`, while `defines.html` went 506,148 -> 32,038 and
+  `noise-expressions.html` 53,222 -> 11,966. That version's whole archive is
+  25,275,142 bytes, about 24 MB.
+
+  The composition below is **2.1.14**, whose archive is 45,547,463 bytes,
+  about 43 MB. Do not read the two versions as one set of numbers: a later
+  version has a bigger archive, and both figures are correct for their own
+  version. 2.1.14's archive is 96 percent HTML, 267,489,280 bytes
+  uncompressed across 1,613 pages, about 11 KB each over the wire. **Fetching
+  all 1,613 costs about 17 MB against that 43 MB**, so there is no file count
+  at which the archive is cheaper - it also ships images and a pagefind index.
+  What it would buy is one request instead of many, and search without
+  knowing the filename. That is why there is no zip cache and no zip
+  dependency. **The limit is that you cannot search a version nobody has
+  installed**, which is a real gap: the design's own example,
+  `control:temperature:frequency`, appears only in `noise-expressions.html`.
+- **`curl -f` exited 56 on a 404, not the 22 the manual suggests.** Measured
+  against an unpublished version. So `refs docs` treats any non-zero exit as
+  a failure and prints curl's own message rather than matching a number.
+- **`install::read_version` spawns a binary with no timeout.** It calls
+  `Command::new(binary).arg("--version").output()` directly. That is the one
+  subprocess in this crate not behind `Spawner`. `refs docs` and `refs sync`
+  both reach it through `install::select`, so a hung Factorio hangs either
+  command too.
+
+  Because of that, `refs sync --check` still launches the game binary on
+  every candidate root, even though it is documented as "Report only. Never
+  fetches, never writes." It only reads the version and exits, so this is
+  harmless today. But it is surprising, and worth knowing before anyone
+  treats `--check` as fully inert.
+
+  This is a known limit, not a fix. Changing `read_version`'s signature
+  would ripple through `discover`, `select`, and every caller - `installs
+  list`, `run`, and `provenance report` among them. That fix earns its own
+  branch. Found 2026-08-17.
+- **No test touches `src/main.rs`.** Every CLI guard - the tag and version
+  checks in each `refs` arm, the exit codes, the help text - runs untested.
+  The crate has no CLI test harness, and building one needs a dev-dependency
+  the five-crate limit forbids. That gap is why `worktree::ensure` and
+  `worktree::remove` stayed unguarded until a whole-branch review checked
+  every argument-vector boundary by hand, closed 2026-08-17.
 
 ### Writing a probe
 

README.md

@@ -117,6 +117,69 @@ because an ignore rule that costs nothing gets used without thinking.
 claim.** Never edit one to make it current, and never edit one to make a test
 pass. A mismatch is a finding.
 
+## Reference material
+
+Most questions about Factorio are answered by Lua that ships in the clear at
+`github.com/wube/factorio-data`, one git tag per release. The catch is that
+one clone has one working tree, and several repos want several versions of it
+at once. Checking a tag out is how one repo silently breaks another's read.
+
+So this reads at a tag and never moves `HEAD`.
+
+```bash
+# One file at a tag.
+factorio-oracle refs show 2.0.77 base/info.json
+
+# Search one tag. Output is git grep's, with the <tag>: prefix removed.
+factorio-oracle refs grep supply_area_distance --tag 2.1.14
+
+# Search several, and find out whether the answer moved.
+factorio-oracle refs grep support_range --tag 2.0.73 --tag 2.1.12 \
+  --path elevated-rails/prototypes/entity/elevated-rails.lua
+
+# A real directory, for ripgrep or an editor or a Lua parser.
+cd "$(factorio-oracle refs worktree 2.0.77)"
+factorio-oracle refs worktree 2.0.77 --remove
+
+# One Lua API docs file. An installed game answers with no network.
+factorio-oracle refs docs 2.1.14 runtime-api.json --which
+
+# Can this version be read at all?
+factorio-oracle refs sync 2.0.73
+factorio-oracle refs sync 2.0.73 --check   # exits 1 if not
+```
+
+The clone is `~/GitHub/factorio-data`, or whatever `FACTORIO_DATA_DIR` names.
+Worktrees and fetched docs go under `~/.cache/factorio-oracle`. Two overrides
+apply, in this order: `FACTORIO_ORACLE_CACHE` wins outright, and failing that
+`XDG_CACHE_HOME` is used with `factorio-oracle` appended, so a machine that
+already sets it for other tools gets the same treatment here.
+
+**The multi-tag verdict is the reason this exists.** "Is this value still the
+same two versions later" is a question every consumer repo has answered by
+hand, by checking a tag out or reading two web pages, and then written the
+answer into a fixture as prose. Two greps answer it, and the comparison
+ignores line numbers on purpose: a value that moved down the file has not
+changed.
+
+**`sync` reports availability, not state.** Since nothing here checks anything
+out, there is no pinned working tree to keep in sync and no lock file to go
+stale. `sync` answers "is the clone here, is the tag fetched, are the docs
+reachable", and may fetch tags to make the answer yes. `--check` never fetches
+and never writes.
+
+**Docs come from an installed game first.** Measured on 2.1.14: the install's
+`doc-html/` is byte-identical to the published archive's contents across all
+3,370 files, and the install even ships that archive itself, with the same
+sha256. So for a version you have there is nothing to download. For a version
+you do not have, one file is fetched and cached - `runtime-api.json` at 2.0.45
+is 1.6 MB against 24 MB for the whole archive.
+
+The limit that follows, stated rather than hidden: **you cannot search a
+version nobody has installed**, because you cannot grep files you never
+fetched. Install that version, or use `refs grep`, which searches the Lua
+rather than the docs.
+
 ## Examples
 
 - [`examples/pumpjack-terminals`](examples/pumpjack-terminals) - a `create` probe

src/lib.rs

@@ -13,6 +13,7 @@ pub mod numbers;
 pub mod outcome;
 pub mod probe;
 pub mod provenance;
+pub mod refs;
 pub mod run;
 pub mod scaffold;
 pub mod spawn;