diff --git a/.gitignore b/.gitignore index 0d99b5a..9aa8921 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,15 @@ /target /refs .DS_Store + +# Factorio installs kept for multi-version work. A 5.1 GB game bundle is not +# source, and it is not ours to redistribute. The directory-level rule means +# git never descends into it, so `git status` does not stat thousands of files. +# +# What lives here today: factorio-2.0.77.app, a full mac-arm64 build 84539. +# It is deliberately OUTSIDE every candidate root in install.rs, so discovery +# does not find it and `installs list` keeps reporting only real install +# locations. Name it explicitly instead: +# cargo run -- provenance report tests/fixtures \ +# --factorio installs/factorio-2.0.77.app +/installs/ diff --git a/CLAUDE.md b/CLAUDE.md index f58ba70..95523c7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -43,18 +43,34 @@ cat /tmp/pj/write/script-output/oracle-dump.json # Reproduce FactorioTools' committed fixture cargo run -- run --probe dump-data.json --work-dir /tmp/w > /tmp/run.json cargo run -- trim --run /tmp/run.json --spec trim-spec.json --out fixture.json [--check] + +# Read factorio-data at a tag. Never moves HEAD in the shared clone. +cargo run -- refs grep support_range --tag 2.0.73 --tag 2.1.12 +cargo run -- refs worktree 2.0.77 # a real tree, under the cache +cargo run -- refs docs 2.1.14 runtime-api.json --which ``` -Test counts to expect: **159 unit tests**, plus **8 integration tests** split -across three files. `tests/acceptance.rs` has 3: two run offline against a +Test counts to expect: **247 unit tests**, plus **13 integration tests** split +across four files. `tests/acceptance.rs` has 3: two run offline against a committed fixture, and one (`the_real_install_reproduces_it_too`) is -install-gated. `tests/provenance.rs` has 2: one always-on, and one gated on -the `FACTORIO_ORACLE_PROVENANCE_DIR` environment variable naming another -repo's fixture directory - not an install gate, so it skips even when -Factorio is present. `tests/real_game.rs` has 3, all install-gated. That is -**4 install-gated tests** in total. Without a real Factorio install they skip -rather than fail, so a green run on a machine with no game proves less than it -looks. Check which happened before trusting it. +install-gated. `tests/provenance.rs` has 3: one always-on; one gated on the +`FACTORIO_ORACLE_PROVENANCE_DIR` environment variable, which names another +repo's fixture directory; and one gated on `FACTORIO_ORACLE_OLD_FACTORIO`, +which names an install *older* than this crate's own committed fixtures. +Neither variable is an install gate: a machine can have Factorio installed +and still have no fixture directory to point at, and still have no *old* +Factorio to point at either, so both tests skip even when Factorio is +present. `tests/real_game.rs` has 3, all install-gated. That is **4 +install-gated tests** in total - `FACTORIO_ORACLE_OLD_FACTORIO`'s test is not +one of the four, because it needs a specific old install named by hand, not +just any install `discover` can find. `tests/refs.rs` has 4, all gated on +finding the `~/GitHub/factorio-data` clone (or wherever `FACTORIO_DATA_DIR` +points) - a separate gate from the install one, so a machine can have +Factorio and no clone, or a clone and no Factorio. Without a real Factorio +install, without an old one named by `FACTORIO_ORACLE_OLD_FACTORIO`, and +without the clone, these tests skip rather than fail, so a green run on a +machine with none of the three proves less than it looks. Check which +happened before trusting it. ## Layout @@ -72,6 +88,7 @@ looks. Check which happened before trusting it. | `numbers.rs` | Preserving the bits the game produced | | `trim/` | Cutting a full `data.raw` dump down to a consumer's slice | | `provenance/` | Which Factorio each fixture came from, and whether that record is still honest | +| `refs/` | Reading factorio-data at a tag, and the Lua API docs, without moving HEAD | Five run modes, and **the success predicate differs per mode**: `dump-data` (no mod at all, the mod dir exists only to be empty), `create`, `interactive` @@ -175,6 +192,76 @@ code did. **A fake can only be wrong in the ways its author already considered.* decimal, with no error. Recorded as a negative result so nobody spends an afternoon ruling it out. Provenance entries stay a `Value` anyway, because only two keys are required and the rest must round-trip untouched. +- **`~/GitHub/factorio-data` is on `master`, and that makes one consumer's + drift check a coincidence.** Verified 2026-08-17: branch `master`, clean, + 548 tags, and `base/info.json` reads 2.1.14, which is also the newest tag. + FactorioMapWebUI's `refs:sync --check` greps that file rather than asking + git anything, so it reads a coincidence as a pin. `refs` never moves `HEAD` + there, and `tests/refs.rs` asserts the clone is unchanged after a real run. +- **Reading at a tag is byte-stable across platforms.** `git show + 2.1.14:base/info.json` returned the same 193 bytes with `core.autocrlf` + unset, forced false and forced true, matching `git cat-file blob`. So no + line-ending rewriting is needed. A **worktree** checkout is a different + path and was not measured this way. +- **factorio-data holds nothing binary.** At tag 2.1.14 it is 327 files: 296 + `.lua`, 27 `.json`, 3 `.txt`, 1 `.md`, and no path contains a colon. That is + what makes it safe to carry `git` output through `SpawnResult`'s `String`, + and to split a grep line on colons. +- **A worktree costs 0.077 seconds and 8.6 MB**, two worktrees at one tag + coexist, and `git worktree add` creates missing parent directories. It also + writes an entry into the shared clone's `.git/worktrees/`, 168 KB for three, + which is the one thing `refs` writes to a clone it does not own. That is why + `refs worktree --remove` exists. +- **The installed game ships the API docs and the published archive too.** + Measured 2026-08-17 on 2.1.14: `doc-html/static/archive.zip` is byte-identical + to `lua-api.factorio.com/2.1.14/static/archive.zip`, both 45,547,463 bytes + and sha256 `87012e1c...`, and all 3,370 other files match the tree + FactorioMapWebUI unpacked from that archive. So for an installed version + there is nothing to download, and `install.rs` already resolves `doc_dir`. +- **Single docs files are published, and the archive never wins on bytes.** + The server gzips HTML but not JSON. Measured **at 2.0.45**: + `runtime-api.json` was 1,597,033 bytes with or without + `Accept-Encoding: gzip`, while `defines.html` went 506,148 -> 32,038 and + `noise-expressions.html` 53,222 -> 11,966. That version's whole archive is + 25,275,142 bytes, about 24 MB. + + The composition below is **2.1.14**, whose archive is 45,547,463 bytes, + about 43 MB. Do not read the two versions as one set of numbers: a later + version has a bigger archive, and both figures are correct for their own + version. 2.1.14's archive is 96 percent HTML, 267,489,280 bytes + uncompressed across 1,613 pages, about 11 KB each over the wire. **Fetching + all 1,613 costs about 17 MB against that 43 MB**, so there is no file count + at which the archive is cheaper - it also ships images and a pagefind index. + What it would buy is one request instead of many, and search without + knowing the filename. That is why there is no zip cache and no zip + dependency. **The limit is that you cannot search a version nobody has + installed**, which is a real gap: the design's own example, + `control:temperature:frequency`, appears only in `noise-expressions.html`. +- **`curl -f` exited 56 on a 404, not the 22 the manual suggests.** Measured + against an unpublished version. So `refs docs` treats any non-zero exit as + a failure and prints curl's own message rather than matching a number. +- **`install::read_version` spawns a binary with no timeout.** It calls + `Command::new(binary).arg("--version").output()` directly. That is the one + subprocess in this crate not behind `Spawner`. `refs docs` and `refs sync` + both reach it through `install::select`, so a hung Factorio hangs either + command too. + + Because of that, `refs sync --check` still launches the game binary on + every candidate root, even though it is documented as "Report only. Never + fetches, never writes." It only reads the version and exits, so this is + harmless today. But it is surprising, and worth knowing before anyone + treats `--check` as fully inert. + + This is a known limit, not a fix. Changing `read_version`'s signature + would ripple through `discover`, `select`, and every caller - `installs + list`, `run`, and `provenance report` among them. That fix earns its own + branch. Found 2026-08-17. +- **No test touches `src/main.rs`.** Every CLI guard - the tag and version + checks in each `refs` arm, the exit codes, the help text - runs untested. + The crate has no CLI test harness, and building one needs a dev-dependency + the five-crate limit forbids. That gap is why `worktree::ensure` and + `worktree::remove` stayed unguarded until a whole-branch review checked + every argument-vector boundary by hand, closed 2026-08-17. ### Writing a probe diff --git a/README.md b/README.md index 07c9909..ad6f744 100644 --- a/README.md +++ b/README.md @@ -117,6 +117,69 @@ because an ignore rule that costs nothing gets used without thinking. claim.** Never edit one to make it current, and never edit one to make a test pass. A mismatch is a finding. +## Reference material + +Most questions about Factorio are answered by Lua that ships in the clear at +`github.com/wube/factorio-data`, one git tag per release. The catch is that +one clone has one working tree, and several repos want several versions of it +at once. Checking a tag out is how one repo silently breaks another's read. + +So this reads at a tag and never moves `HEAD`. + +```bash +# One file at a tag. +factorio-oracle refs show 2.0.77 base/info.json + +# Search one tag. Output is git grep's, with the : prefix removed. +factorio-oracle refs grep supply_area_distance --tag 2.1.14 + +# Search several, and find out whether the answer moved. +factorio-oracle refs grep support_range --tag 2.0.73 --tag 2.1.12 \ + --path elevated-rails/prototypes/entity/elevated-rails.lua + +# A real directory, for ripgrep or an editor or a Lua parser. +cd "$(factorio-oracle refs worktree 2.0.77)" +factorio-oracle refs worktree 2.0.77 --remove + +# One Lua API docs file. An installed game answers with no network. +factorio-oracle refs docs 2.1.14 runtime-api.json --which + +# Can this version be read at all? +factorio-oracle refs sync 2.0.73 +factorio-oracle refs sync 2.0.73 --check # exits 1 if not +``` + +The clone is `~/GitHub/factorio-data`, or whatever `FACTORIO_DATA_DIR` names. +Worktrees and fetched docs go under `~/.cache/factorio-oracle`. Two overrides +apply, in this order: `FACTORIO_ORACLE_CACHE` wins outright, and failing that +`XDG_CACHE_HOME` is used with `factorio-oracle` appended, so a machine that +already sets it for other tools gets the same treatment here. + +**The multi-tag verdict is the reason this exists.** "Is this value still the +same two versions later" is a question every consumer repo has answered by +hand, by checking a tag out or reading two web pages, and then written the +answer into a fixture as prose. Two greps answer it, and the comparison +ignores line numbers on purpose: a value that moved down the file has not +changed. + +**`sync` reports availability, not state.** Since nothing here checks anything +out, there is no pinned working tree to keep in sync and no lock file to go +stale. `sync` answers "is the clone here, is the tag fetched, are the docs +reachable", and may fetch tags to make the answer yes. `--check` never fetches +and never writes. + +**Docs come from an installed game first.** Measured on 2.1.14: the install's +`doc-html/` is byte-identical to the published archive's contents across all +3,370 files, and the install even ships that archive itself, with the same +sha256. So for a version you have there is nothing to download. For a version +you do not have, one file is fetched and cached - `runtime-api.json` at 2.0.45 +is 1.6 MB against 24 MB for the whole archive. + +The limit that follows, stated rather than hidden: **you cannot search a +version nobody has installed**, because you cannot grep files you never +fetched. Install that version, or use `refs grep`, which searches the Lua +rather than the docs. + ## Examples - [`examples/pumpjack-terminals`](examples/pumpjack-terminals) - a `create` probe diff --git a/src/lib.rs b/src/lib.rs index 63f618d..0259d5c 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -13,6 +13,7 @@ pub mod numbers; pub mod outcome; pub mod probe; pub mod provenance; +pub mod refs; pub mod run; pub mod scaffold; pub mod spawn; diff --git a/src/main.rs b/src/main.rs index 3e42b14..e539389 100644 --- a/src/main.rs +++ b/src/main.rs @@ -62,6 +62,11 @@ enum Command { #[command(subcommand)] action: ProvenanceAction, }, + /// Read Factorio's shipped Lua and API docs at a version + Refs { + #[command(subcommand)] + action: RefsAction, + }, } #[derive(Subcommand)] @@ -92,6 +97,108 @@ enum ProvenanceAction { }, } +#[derive(Subcommand)] +enum RefsAction { + /// Print one file from factorio-data at a tag. Moves no HEAD. + Show { + /// The tag, for example 2.0.73 + tag: String, + /// The path inside the repo, for example base/info.json + path: String, + /// The factorio-data clone. Defaults to FACTORIO_DATA_DIR, then + /// ~/GitHub/factorio-data. + #[arg(long)] + clone: Option, + }, + /// Search factorio-data at one tag or several. Moves no HEAD. + Grep { + /// The pattern, passed to git grep + pattern: String, + /// A tag to search. Repeat it to compare versions. + #[arg(long = "tag", required = true)] + tags: Vec, + /// Limit the search to these paths + #[arg(long = "path")] + paths: Vec, + /// The factorio-data clone + #[arg(long)] + clone: Option, + /// Emit JSON instead of grep-style lines + #[arg(long)] + json: bool, + }, + /// Materialise a real tree at a tag, for tools that need a directory + Worktree { + /// The tag, for example 2.0.77 + tag: String, + /// The factorio-data clone + #[arg(long)] + clone: Option, + /// Remove the tree for this tag instead of making one. This also + /// clears the entry git wrote into the shared clone. + #[arg(long)] + remove: bool, + }, + /// Print a Lua API docs file. Uses an installed game before the network. + Docs { + /// The version, for example 2.0.45 + version: String, + /// The path inside the docs, for example runtime-api.json + path: String, + /// Select an install by path, for installs outside the usual places + #[arg(long)] + factorio: Option, + /// Print where the file is instead of printing it. Still fetches the + /// file first if it is not already present. + #[arg(long)] + which: bool, + }, + /// Report whether a version's reference material can be read + Sync { + /// The version, for example 2.0.73 + version: String, + /// The factorio-data clone + #[arg(long)] + clone: Option, + /// Select an install by path, for installs outside the usual places + #[arg(long)] + factorio: Option, + /// Report only. Never fetches, never writes, exits 1 when the version + /// cannot be read. + #[arg(long)] + check: bool, + /// Emit JSON instead of a table + #[arg(long)] + json: bool, + }, +} + +/// Resolves the factorio-data clone and checks it is one. +/// +/// `FACTORIO_DATA_DIR` is FactorioMapWebUI's own override name, reused so +/// both tools read one setting. +fn resolve_clone(explicit: Option) -> anyhow::Result { + let home = PathBuf::from(std::env::var("HOME").unwrap_or_default()); + let env_dir = std::env::var_os("FACTORIO_DATA_DIR").map(PathBuf::from); + let dir = factorio_oracle::refs::data_clone(&home, explicit.as_deref().or(env_dir.as_deref())); + if !factorio_oracle::refs::is_clone(&dir) { + anyhow::bail!( + "no factorio-data clone at {}. Clone https://github.com/wube/factorio-data there, \ + or set FACTORIO_DATA_DIR.", + dir.display() + ); + } + Ok(dir) +} + +/// Resolves this tool's cache directory. +fn resolve_cache() -> PathBuf { + let home = PathBuf::from(std::env::var("HOME").unwrap_or_default()); + let over = std::env::var_os("FACTORIO_ORACLE_CACHE").map(PathBuf::from); + let xdg = std::env::var_os("XDG_CACHE_HOME").map(PathBuf::from); + factorio_oracle::refs::cache_dir(&home, over.as_deref(), xdg.as_deref()) +} + fn main() -> anyhow::Result<()> { let cli = Cli::parse(); match cli.command { @@ -304,6 +411,230 @@ fn main() -> anyhow::Result<()> { ) ); } + Command::Refs { + action: RefsAction::Show { tag, path, clone }, + } => { + if !factorio_oracle::refs::valid_tag(&tag) { + anyhow::bail!("{tag} is not a usable tag name"); + } + let dir = resolve_clone(clone)?; + print!( + "{}", + factorio_oracle::refs::grep::show( + &factorio_oracle::spawn::RealSpawner, + &dir, + &tag, + &path + )? + ); + } + Command::Refs { + action: + RefsAction::Grep { + pattern, + tags, + paths, + clone, + json, + }, + } => { + for tag in &tags { + if !factorio_oracle::refs::valid_tag(tag) { + anyhow::bail!("{tag} is not a usable tag name"); + } + } + let dir = resolve_clone(clone)?; + let report = factorio_oracle::refs::grep::search( + &factorio_oracle::spawn::RealSpawner, + &dir, + &pattern, + &tags, + &paths, + )?; + if json { + println!( + "{}", + serde_json::to_string_pretty(&factorio_oracle::refs::grep::to_json(&report))? + ); + } else { + print!("{}", factorio_oracle::refs::grep::render(&report)); + } + // grep's convention, kept: nothing found is exit 1. A verdict of + // `differs` is NOT a failure - it is the finding, and whether it + // matters is a human's call, the same split provenance uses. + if report.empty() { + std::process::exit(1); + } + } + Command::Refs { + action: RefsAction::Worktree { tag, clone, remove }, + } => { + if !factorio_oracle::refs::valid_tag(&tag) { + anyhow::bail!("{tag} is not a usable tag name"); + } + let dir = resolve_clone(clone)?; + let cache = resolve_cache(); + if remove { + // `worktree::remove` returns early, doing nothing, when there + // is no tree for this tag - cleanup that fails on a repeat + // run cannot be run twice. So the message has to be checked + // against the same path `remove` looks at, not assumed. + let existed = factorio_oracle::refs::worktree::worktree_path(&cache, &tag).exists(); + factorio_oracle::refs::worktree::remove( + &factorio_oracle::spawn::RealSpawner, + &dir, + &cache, + &tag, + )?; + if existed { + println!("Removed the worktree for {tag}."); + } else { + println!("No worktree for {tag} to remove."); + } + } else { + let path = factorio_oracle::refs::worktree::ensure( + &factorio_oracle::spawn::RealSpawner, + &dir, + &cache, + &tag, + )?; + // The path on its own line, so a caller can capture it: + // cd "$(factorio-oracle refs worktree 2.0.77)" + println!("{}", path.display()); + } + } + Command::Refs { + action: + RefsAction::Docs { + version, + path, + factorio, + which, + }, + } => { + use factorio_oracle::refs::docs::{self, DocsSource}; + + // The version is joined onto the cache directory by `cache_path` + // and onto the URL by `url`, so it needs the same guard the tag + // arms use. The `sync` arm checks it; this one did not. + anyhow::ensure!( + factorio_oracle::refs::valid_tag(&version), + "{version} is not a usable version" + ); + anyhow::ensure!( + docs::safe_relative(&path), + "{path} is not a usable docs path" + ); + + let home = PathBuf::from(std::env::var("HOME").unwrap_or_default()); + let env_bin = std::env::var_os("FACTORIO_BIN").map(PathBuf::from); + // An installed game of that version answers with no network at + // all. Measured: its doc-html is byte-identical to the published + // archive's contents. + // + // `select` is used rather than a bare `discover`, so an install + // outside the candidate roots can be named. That is not a corner + // case: the 2.0.77 build on this machine sits in this repo at + // installs/factorio-2.0.77.app, which discovery does not search + // and should not - it is gitignored and outside every candidate + // root on purpose, so no install-gated test changes which install + // it picks. `select` also + // requires an exact triple match, so naming a 2.1.14 install + // while asking for 2.0.77 docs cannot quietly answer with the + // wrong version - it falls through to the cache instead. + let installed = install::select( + &home, + factorio.as_deref(), + env_bin.as_deref(), + Some(&version), + ) + .map(|d| d.layout.doc_dir); + let cache = resolve_cache(); + + let resolved = match docs::locate(installed.as_deref(), &cache, &version, &path) { + DocsSource::Install(p) => p, + DocsSource::Cache(p) => p, + DocsSource::Fetch { .. } => docs::fetch( + &factorio_oracle::spawn::RealSpawner, + &cache, + &version, + &path, + )?, + }; + + if which { + println!("{}", resolved.display()); + } else { + print!("{}", std::fs::read_to_string(&resolved)?); + } + } + Command::Refs { + action: + RefsAction::Sync { + version, + clone, + factorio, + check, + json, + }, + } => { + use factorio_oracle::refs::sync; + + // Matches the `docs` arm's wording: the positional is called + // `version` here, not `tag`, so the error should say so too. + anyhow::ensure!( + factorio_oracle::refs::valid_tag(&version), + "{version} is not a usable version" + ); + + let home = PathBuf::from(std::env::var("HOME").unwrap_or_default()); + let env_dir = std::env::var_os("FACTORIO_DATA_DIR").map(PathBuf::from); + let dir = + factorio_oracle::refs::data_clone(&home, clone.as_deref().or(env_dir.as_deref())); + let clone_present = factorio_oracle::refs::is_clone(&dir); + let spawner = factorio_oracle::spawn::RealSpawner; + + let mut tag_present = false; + if clone_present { + tag_present = sync::tag_present(&spawner, &dir, &version)?; + // `sync` may fetch, because that is how availability gets + // achieved. `--check` never does, because it is a report. + if !tag_present && !check { + sync::fetch_tags(&spawner, &dir)?; + tag_present = sync::tag_present(&spawner, &dir, &version)?; + } + } + + let env_bin = std::env::var_os("FACTORIO_BIN").map(PathBuf::from); + // Same reasoning as the `docs` arm: an install outside the + // candidate roots has to be nameable, and `select` will not + // return one whose version does not match. + let installed = install::select( + &home, + factorio.as_deref(), + env_bin.as_deref(), + Some(&version), + ) + .map(|d| d.layout.doc_dir); + let cache = resolve_cache(); + + let report = sync::Availability { + version: version.clone(), + clone: dir, + clone_present, + tag_present, + docs: sync::docs_standing(installed.as_deref(), &cache, &version), + }; + + if json { + println!("{}", serde_json::to_string_pretty(&sync::to_json(&report))?); + } else { + print!("{}", sync::render(&report)); + } + if check && !report.ok() { + std::process::exit(1); + } + } } Ok(()) } diff --git a/src/refs/docs.rs b/src/refs/docs.rs new file mode 100644 index 0000000..723f6d7 --- /dev/null +++ b/src/refs/docs.rs @@ -0,0 +1,466 @@ +//! The Lua API docs, without keeping a copy per version. +//! +//! Measured 2026-08-17 on 2.1.14. The installed game ships the whole docs +//! tree at `doc-html/`, 3,371 files, and one of them is +//! `doc-html/static/archive.zip` - **byte-identical to the archive published +//! at lua-api.factorio.com**, both 45,547,463 bytes and both sha256 +//! 87012e1cc45864fcda891f0e040b683ec35c9746b0e75eeab81faf5e7d6422e8. A +//! `diff -rq` of that tree against FactorioMapWebUI's `factorioLuaAPI/`, +//! which was downloaded and unpacked from the published archive, differed in +//! two files only: MapWebUI's added `VERSION`, and the install's own +//! `archive.zip`. All 3,370 others matched. +//! +//! So for a version you have installed there is nothing to fetch, and +//! `install.rs` already resolves `doc_dir` for it. +//! +//! For a version you do not have, single files are published, and the archive +//! never wins on bytes. The server gzips HTML but not JSON: at 2.0.45, +//! `runtime-api.json` came back at 1,597,033 bytes with or without +//! `Accept-Encoding: gzip`, while `defines.html` went 506,148 -> 32,038 and +//! `noise-expressions.html` 53,222 -> 11,966. The archive is 96 percent HTML, +//! 267,489,280 bytes uncompressed across 1,613 pages. Fetching **every one of +//! those pages one at a time costs about 17 MB, against 43 MB for the +//! archive**, which also carries images and a search index nobody asked for. +//! So the cache here fills one file at a time and ends up as a sparse +//! `doc-html`. +//! +//! **The limit, stated rather than hidden:** you cannot search a version +//! nobody has installed, because you cannot grep files you never fetched. +//! The design's own example is that case - `control:temperature:frequency` +//! appears in `noise-expressions.html` and nowhere in `runtime-api.json`. +//! Adding an archive cache later would fix it and nothing here blocks that. + +use crate::spawn::Spawner; +use std::path::{Path, PathBuf}; +use std::time::Duration; + +/// Where Factorio publishes its docs. +pub const DOCS_HOST: &str = "https://lua-api.factorio.com"; + +/// One file, not one archive. The biggest measured is `prototype-api.json` at +/// 1.7 MB, so this is a hang guard rather than a budget. +pub const FETCH_TIMEOUT: Duration = Duration::from_secs(120); + +/// Where a docs file will be read from. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum DocsSource { + /// That version is installed, so its own `doc-html` answers. No network. + Install(PathBuf), + /// Already fetched into this tool's cache. + Cache(PathBuf), + /// Not here yet. + Fetch { url: String, into: PathBuf }, +} + +/// The published URL for one docs file. +pub fn url(version: &str, rel: &str) -> String { + format!("{DOCS_HOST}/{version}/{rel}") +} + +/// Rejects a path that could escape the cache or be read by curl as an option. +/// +/// The path is joined onto the cache directory and also handed to curl, so it +/// has to be safe for both. No leading slash, no `.` or `..` component, no +/// empty component, and no leading dash. +pub fn safe_relative(rel: &str) -> bool { + // The backslash and colon checks are not redundant with the split below. + // Both are Windows path syntax that a `/`-split cannot see. + // + // Backslash: Windows treats it as a separator, so `..\..\etc` is one + // component to the split and a traversal to `Path::join` there. + // + // Colon: a bare drive-letter component like `C:` is a prefix without a + // root, and `PathBuf::push` documents that such a path **replaces** the + // buffer entirely rather than joining onto it. So `C:/whatever` would + // discard the cache directory and resolve against the current directory + // of that drive. Inferred from std's documented behaviour rather than + // measured, because the only Windows machine here is powered off. + // + // Both are free to reject. Measured 2026-08-17 across all 3,370 files in + // the 2.1.14 docs archive and the installed tree: not one path contains a + // backslash or a colon. + !rel.is_empty() + && !rel.starts_with('/') + && !rel.starts_with('-') + && !rel.contains('\\') + && !rel.contains(':') + && !rel + .split('/') + .any(|c| c.is_empty() || c == "." || c == "..") +} + +/// Where a fetched docs file is kept. +/// +/// The tree mirrors the docs tree exactly, so a partly filled cache is just a +/// sparse `doc-html` and existing paths resolve inside it. +pub fn cache_path(cache: &Path, version: &str, rel: &str) -> PathBuf { + let mut path = cache.join("docs").join(version); + for part in rel.split('/') { + path = path.join(part); + } + path +} + +/// Decides where a docs file comes from. An install first, then the cache, +/// then the network. +pub fn locate( + installed_doc_dir: Option<&Path>, + cache: &Path, + version: &str, + rel: &str, +) -> DocsSource { + if let Some(doc) = installed_doc_dir { + let mut path = doc.to_path_buf(); + for part in rel.split('/') { + path = path.join(part); + } + if path.is_file() { + return DocsSource::Install(path); + } + } + let cached = cache_path(cache, version, rel); + if cached.is_file() { + return DocsSource::Cache(cached); + } + DocsSource::Fetch { + url: url(version, rel), + into: cached, + } +} + +/// `curl -fsSL --max-time -o ` +/// +/// `-f` turns an HTTP error into a non-zero exit rather than a saved error +/// page. `-o` keeps the body out of stdout, which `SpawnResult` carries as a +/// `String` and would mangle for anything not UTF-8. +pub fn curl_args(url: &str, into: &Path) -> Vec { + vec![ + "-fsSL".into(), + "--max-time".into(), + FETCH_TIMEOUT.as_secs().to_string(), + "-o".into(), + into.display().to_string(), + url.to_string(), + ] +} + +/// Downloads one docs file into the cache and returns where it landed. +/// +/// It writes to a `.part` file and renames on success, so an interrupted +/// download cannot leave a truncated file that later reads as complete. That +/// is the same rule FactorioMapWebUI's sync script uses when it swaps a docs +/// tree only after a clean extract. +pub fn fetch( + spawner: &dyn Spawner, + cache: &Path, + version: &str, + rel: &str, +) -> anyhow::Result { + // Both halves are checked here and not only at the CLI, because a + // library function must not trust its caller. `cache_path` joins the + // version onto the cache directory exactly as it joins the path, so an + // unchecked version is the same traversal by another name. `valid_tag` + // is the predicate that already answers this question for tags. + anyhow::ensure!( + super::valid_tag(version), + "{version} is not a usable version" + ); + anyhow::ensure!(safe_relative(rel), "{rel} is not a usable docs path"); + let final_path = cache_path(cache, version, rel); + if let Some(parent) = final_path.parent() { + std::fs::create_dir_all(parent)?; + } + // Append rather than replace the extension, so `runtime-api.json` becomes + // `runtime-api.json.part` and not `runtime-api.part`. Two versions of the + // same file must never collide with each other's partial download. + let part = { + let mut name = final_path.clone().into_os_string(); + name.push(".part"); + PathBuf::from(name) + }; + + let args = curl_args(&url(version, rel), &part); + let out = spawner.run(Path::new("curl"), &args, Some(FETCH_TIMEOUT))?; + if out.exit_code != Some(0) { + // Measured 2026-08-17: an unpublished version gave exit 56, not the + // 22 the manual leads you to expect. So this reports whatever curl + // said rather than translating a code. + let _ = std::fs::remove_file(&part); + anyhow::bail!( + "could not fetch {}: {}", + url(version, rel), + out.stderr.trim() + ); + } + std::fs::rename(&part, &final_path)?; + Ok(final_path) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::spawn::SpawnResult; + use std::cell::RefCell; + + #[test] + fn the_url_is_the_version_then_the_path() { + assert_eq!( + url("2.0.45", "runtime-api.json"), + "https://lua-api.factorio.com/2.0.45/runtime-api.json" + ); + assert_eq!( + url("2.0.45", "auxiliary/noise-expressions.html"), + "https://lua-api.factorio.com/2.0.45/auxiliary/noise-expressions.html" + ); + } + + #[test] + fn real_doc_paths_are_accepted() { + // All four measured as HTTP 200 at 2.0.45. + for path in [ + "runtime-api.json", + "prototype-api.json", + "defines.html", + "auxiliary/noise-expressions.html", + ] { + assert!(safe_relative(path), "{path} should be accepted"); + } + } + + #[test] + fn a_path_that_could_escape_the_cache_is_rejected() { + // The path is joined onto the cache directory, so this one would + // write outside it. + assert!(!safe_relative("../../../etc/passwd")); + assert!(!safe_relative("a/../../b")); + assert!(!safe_relative("/etc/passwd")); + assert!(!safe_relative("a//b")); + assert!(!safe_relative("./a")); + assert!(!safe_relative("")); + } + + #[test] + fn a_backslash_is_rejected_because_windows_treats_it_as_a_separator() { + // The `/`-split cannot see a Windows separator, so `..\..\etc` is one + // component to it and a traversal to `Path::join` on Windows. + // + // Added after the Task 1 review found `valid_tag("..")` returning + // true: the same question was asked of every other predicate that + // guards a path join, and this was the gap. + assert!(!safe_relative("..\\..\\etc")); + assert!(!safe_relative("auxiliary\\noise-expressions.html")); + } + + #[test] + fn a_drive_letter_is_rejected_because_it_replaces_a_path_rather_than_joining() { + // `PathBuf::push` documents that a path with a prefix and no root + // replaces the buffer outright, so a `C:` component would discard the + // cache directory entirely on Windows. + // + // Found by the Task 5 review on 2026-08-17: `valid_tag` guards the + // version with a character allowlist that excludes `:` structurally, + // while this function used a blocklist and never checked for one. The + // same "guarded one argument over" gap, for the sixth time in this + // plan. + assert!(!safe_relative("C:/whatever")); + assert!(!safe_relative("C:")); + assert!(!safe_relative("classes/Lua:Entity.html")); + } + + #[test] + fn a_path_that_curl_would_read_as_an_option_is_rejected() { + assert!(!safe_relative("-o/tmp/pwned")); + } + + #[test] + fn an_installed_version_answers_from_its_own_doc_dir() { + // The whole point. No network, no cache, no copy. + let tmp = tempfile::tempdir().unwrap(); + let doc = tmp.path().join("doc-html"); + std::fs::create_dir_all(&doc).unwrap(); + std::fs::write(doc.join("runtime-api.json"), b"{}").unwrap(); + + let got = locate( + Some(&doc), + Path::new("/cache"), + "2.1.14", + "runtime-api.json", + ); + assert_eq!(got, DocsSource::Install(doc.join("runtime-api.json"))); + } + + #[test] + fn an_installed_version_missing_that_one_file_still_fetches_it() { + // A doc_dir that exists but does not hold the asked-for file is not + // proof the file does not exist, so fall through rather than fail. + let tmp = tempfile::tempdir().unwrap(); + let doc = tmp.path().join("doc-html"); + std::fs::create_dir_all(&doc).unwrap(); + + let got = locate( + Some(&doc), + Path::new("/cache"), + "2.1.14", + "runtime-api.json", + ); + assert!(matches!(got, DocsSource::Fetch { .. })); + } + + #[test] + fn an_already_cached_file_is_used_before_the_network() { + let cache = tempfile::tempdir().unwrap(); + let path = cache_path(cache.path(), "2.0.45", "runtime-api.json"); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(&path, b"{}").unwrap(); + + let got = locate(None, cache.path(), "2.0.45", "runtime-api.json"); + assert_eq!(got, DocsSource::Cache(path)); + } + + #[test] + fn an_uncached_file_names_the_url_and_where_it_goes() { + let cache = tempfile::tempdir().unwrap(); + let got = locate(None, cache.path(), "2.0.45", "runtime-api.json"); + assert_eq!( + got, + DocsSource::Fetch { + url: "https://lua-api.factorio.com/2.0.45/runtime-api.json".into(), + into: cache_path(cache.path(), "2.0.45", "runtime-api.json"), + } + ); + } + + #[test] + fn the_cache_tree_mirrors_the_docs_tree() { + // A partly filled cache is just a sparse doc-html, so a caller can + // point an existing tool at /docs/ and have the paths + // it already knows resolve. + assert_eq!( + cache_path( + Path::new("/c"), + "2.0.45", + "auxiliary/noise-expressions.html" + ), + PathBuf::from("/c/docs/2.0.45/auxiliary/noise-expressions.html") + ); + } + + #[test] + fn curl_fails_on_an_error_status_and_writes_to_a_file() { + // -f makes an HTTP error a non-zero exit instead of a saved error + // page. -o keeps the body out of stdout, which SpawnResult carries as + // a String. + let args = curl_args("https://x/y.json", Path::new("/tmp/y.json.part")); + assert!(args.contains(&"-fsSL".to_string())); + assert!(args.contains(&"-o".to_string())); + assert!(args.contains(&"/tmp/y.json.part".to_string())); + assert_eq!(args.last().unwrap(), "https://x/y.json"); + } + + struct FakeCurl { + exit: i32, + stderr: String, + body: String, + seen: RefCell>>, + } + + impl Spawner for FakeCurl { + fn run( + &self, + _binary: &Path, + args: &[String], + _timeout: Option, + ) -> anyhow::Result { + self.seen.borrow_mut().push(args.to_vec()); + if self.exit == 0 { + // Write where -o pointed, the way curl would. + let into = args[args.iter().position(|a| a == "-o").unwrap() + 1].clone(); + std::fs::write(into, self.body.as_bytes())?; + } + Ok(SpawnResult { + exit_code: Some(self.exit), + stdout: String::new(), + stderr: self.stderr.clone(), + }) + } + } + + #[test] + fn a_fetched_file_lands_at_its_cache_path() { + let cache = tempfile::tempdir().unwrap(); + let fake = FakeCurl { + exit: 0, + stderr: String::new(), + body: "{\"api_version\":6}".into(), + seen: RefCell::new(vec![]), + }; + let got = fetch(&fake, cache.path(), "2.0.45", "runtime-api.json") + .expect("the fake always answers"); + assert_eq!(got, cache_path(cache.path(), "2.0.45", "runtime-api.json")); + assert_eq!( + std::fs::read_to_string(&got).unwrap(), + "{\"api_version\":6}" + ); + } + + #[test] + fn a_fetch_writes_to_a_part_file_first() { + // Same reasoning as FactorioMapWebUI's sync script swapping only + // after a clean extract: an interrupted download must not leave a + // truncated file that reads as complete. + let cache = tempfile::tempdir().unwrap(); + let fake = FakeCurl { + exit: 0, + stderr: String::new(), + body: "{}".into(), + seen: RefCell::new(vec![]), + }; + fetch(&fake, cache.path(), "2.0.45", "runtime-api.json").unwrap(); + let seen = fake.seen.borrow(); + let into = &seen[0][seen[0].iter().position(|a| a == "-o").unwrap() + 1]; + assert!(into.ends_with(".part"), "curl should write to {into}"); + assert!(!cache_path(cache.path(), "2.0.45", "runtime-api.json.part").exists()); + } + + #[test] + fn a_version_that_could_escape_the_cache_is_rejected() { + // `cache_path` joins the version onto the cache directory, so this is + // the same hazard `valid_tag` guards for tags, one argument over. + // + // Found by auditing every path join in this plan after the Task 1 + // review caught `valid_tag("..")` returning true. The CLI checked the + // docs path and never the version. + let cache = tempfile::tempdir().unwrap(); + let fake = FakeCurl { + exit: 0, + stderr: String::new(), + body: "{}".into(), + seen: RefCell::new(vec![]), + }; + let err = fetch(&fake, cache.path(), "..", "runtime-api.json") + .expect_err("a version that is a path component must be rejected"); + assert!(err.to_string().contains("not a usable version")); + assert!( + fake.seen.borrow().is_empty(), + "it must be rejected before anything is fetched" + ); + } + + #[test] + fn a_failed_fetch_leaves_nothing_behind_and_says_what_curl_said() { + // Measured 2026-08-17 against an unpublished version: curl exited + // **56**, not the 22 the manual leads you to expect. So this keys off + // "not zero" rather than a number. + let cache = tempfile::tempdir().unwrap(); + let fake = FakeCurl { + exit: 56, + stderr: "curl: (56) The requested URL returned error: 404\n".into(), + body: String::new(), + seen: RefCell::new(vec![]), + }; + let err = fetch(&fake, cache.path(), "9.9.9", "runtime-api.json") + .expect_err("a 404 is not success"); + assert!(err.to_string().contains("404")); + assert!(!cache_path(cache.path(), "9.9.9", "runtime-api.json").exists()); + } +} diff --git a/src/refs/git.rs b/src/refs/git.rs new file mode 100644 index 0000000..59424b6 --- /dev/null +++ b/src/refs/git.rs @@ -0,0 +1,361 @@ +//! Every git command this tool runs, as an argument vector, plus the parser +//! for what comes back. +//! +//! Measured 2026-08-17 on git 2.50.1 against `~/GitHub/factorio-data`: +//! +//! - `git grep -n ` prefixes every line with `:`, so a +//! line reads `:::`. It exits 1 when nothing +//! matched, which is an answer rather than an error. +//! - `git show :` exits 128. +//! - `git show :` returned the same 193 bytes for +//! `base/info.json` with `core.autocrlf` unset, false and true, matching +//! `git cat-file blob`. So a read at a tag is byte-stable across platforms. +//! - factorio-data at 2.1.14 is 327 files: 296 `.lua`, 27 `.json`, 3 `.txt`, +//! 1 `.md`. Nothing binary, and no path holds a colon. That is what lets +//! `parse_hit` split on colons, and what makes it safe to carry the output +//! through `spawn::SpawnResult`'s `String`. +//! +//! Nothing here checks anything out. There is no `checkout`, `switch` or +//! `reset` in this file, and that is the constraint the whole command exists +//! to hold. + +use std::path::Path; +use std::time::Duration; + +/// A local git read. Measured at well under a second on a 19 MB clone, so +/// this is a hang guard rather than a budget. +pub const GIT_TIMEOUT: Duration = Duration::from_secs(30); + +/// A fetch, which is the only git call here that uses the network. +pub const FETCH_TIMEOUT: Duration = Duration::from_secs(180); + +/// One line of `git grep -n ` output, taken apart. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Hit { + pub tag: String, + pub path: String, + pub line: u32, + pub text: String, +} + +/// `git -C rev-parse -q --verify refs/tags/^{commit}` +/// +/// `^{commit}` makes this true only for a tag that resolves to a commit, and +/// `-q` keeps it silent so a missing tag is an exit code rather than noise on +/// stderr. +pub fn tag_exists_args(dir: &Path, tag: &str) -> Vec { + vec![ + "-C".into(), + dir.display().to_string(), + "rev-parse".into(), + "-q".into(), + "--verify".into(), + format!("refs/tags/{tag}^{{commit}}"), + ] +} + +/// `git -C show :` +pub fn show_args(dir: &Path, tag: &str, path: &str) -> Vec { + vec![ + "-C".into(), + dir.display().to_string(), + "show".into(), + format!("{tag}:{path}"), + ] +} + +/// `git -C grep --no-color -n -e [-- ...]` +pub fn grep_args(dir: &Path, tag: &str, pattern: &str, pathspec: &[String]) -> Vec { + let mut args = vec![ + "-C".into(), + dir.display().to_string(), + "grep".into(), + "--no-color".into(), + "-n".into(), + "-e".into(), + pattern.to_string(), + tag.to_string(), + ]; + if !pathspec.is_empty() { + args.push("--".into()); + args.extend(pathspec.iter().cloned()); + } + args +} + +/// `git -C fetch --tags --quiet origin` +/// +/// The only write this tool makes to a clone it does not own, and it writes +/// refs and objects only. No working tree changes and `HEAD` does not move. +pub fn fetch_tags_args(dir: &Path) -> Vec { + vec![ + "-C".into(), + dir.display().to_string(), + "fetch".into(), + "--tags".into(), + "--quiet".into(), + "origin".into(), + ] +} + +/// `git -C worktree add --detach ` +pub fn worktree_add_args(dir: &Path, path: &Path, tag: &str) -> Vec { + vec![ + "-C".into(), + dir.display().to_string(), + "worktree".into(), + "add".into(), + "--detach".into(), + path.display().to_string(), + tag.to_string(), + ] +} + +/// `git -C worktree remove ` +pub fn worktree_remove_args(dir: &Path, path: &Path) -> Vec { + vec![ + "-C".into(), + dir.display().to_string(), + "worktree".into(), + "remove".into(), + path.display().to_string(), + ] +} + +/// `git -C rev-parse HEAD refs/tags/^{commit}` +/// +/// Two shas on two lines: what the worktree is actually checked out at, and +/// what the tag resolves to right now. Measured 2026-08-17 on the +/// factorio-data worktree: one call, 11 ms, against 77 ms to rebuild the +/// tree with `worktree add` - cheap enough to run on every reuse rather than +/// trusting a directory's name for what it holds. +pub fn worktree_head_args(path: &Path, tag: &str) -> Vec { + vec![ + "-C".into(), + path.display().to_string(), + "rev-parse".into(), + "HEAD".into(), + format!("refs/tags/{tag}^{{commit}}"), + ] +} + +/// Parses one line of `git grep -n ` output. +/// +/// The tag is passed in rather than read off the front, because splitting on +/// the first colon would break on any tag holding one. After the tag and the +/// path, the first colon-delimited field is the line number and everything +/// left is the matched text, colons and all. +pub fn parse_hit(tag: &str, line: &str) -> Option { + let rest = line.strip_prefix(&format!("{tag}:"))?; + let (path, rest) = rest.split_once(':')?; + let (number, text) = rest.split_once(':')?; + Some(Hit { + tag: tag.to_string(), + path: path.to_string(), + line: number.parse().ok()?, + text: text.to_string(), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::path::PathBuf; + + fn dir() -> PathBuf { + PathBuf::from("/home/e/GitHub/factorio-data") + } + + #[test] + fn the_tag_check_asks_for_a_commit_and_stays_quiet() { + // Measured: this exits 0 and prints the sha for a tag that exists, + // and exits 1 printing nothing for one that does not. + assert_eq!( + tag_exists_args(&dir(), "2.0.73"), + vec![ + "-C", + "/home/e/GitHub/factorio-data", + "rev-parse", + "-q", + "--verify", + "refs/tags/2.0.73^{commit}", + ] + ); + } + + #[test] + fn show_names_the_tag_and_the_path_together() { + assert_eq!( + show_args(&dir(), "2.0.77", "base/info.json"), + vec![ + "-C", + "/home/e/GitHub/factorio-data", + "show", + "2.0.77:base/info.json", + ] + ); + } + + #[test] + fn grep_disables_colour_and_guards_the_pattern() { + // --no-color is not decoration. Someone with `color.grep = always` + // in their git config would otherwise get ANSI escapes inside every + // match, and the parser would carry them into the output. + // -e keeps a pattern beginning with `-` from being read as a flag. + assert_eq!( + grep_args(&dir(), "2.1.12", "-fluid", &[]), + vec![ + "-C", + "/home/e/GitHub/factorio-data", + "grep", + "--no-color", + "-n", + "-e", + "-fluid", + "2.1.12", + ] + ); + } + + #[test] + fn grep_puts_a_pathspec_after_a_double_dash() { + let paths = vec!["elevated-rails/prototypes/entity/elevated-rails.lua".to_string()]; + let got = grep_args(&dir(), "2.0.73", "support_range", &paths); + assert_eq!(got[got.len() - 2], "--"); + assert_eq!( + got[got.len() - 1], + "elevated-rails/prototypes/entity/elevated-rails.lua" + ); + } + + #[test] + fn fetching_tags_touches_no_working_tree() { + let got = fetch_tags_args(&dir()); + assert!(got.contains(&"fetch".to_string())); + assert!(got.contains(&"--tags".to_string())); + // The rule the whole module holds. A fetch writes refs and objects + // and nothing else, which is why it is the only write this tool makes + // to a clone it does not own. + assert!(!got + .iter() + .any(|a| a == "checkout" || a == "switch" || a == "reset")); + } + + #[test] + fn adding_a_worktree_always_detaches() { + // Measured: git detaches on its own for a tag, because a tag is not a + // branch. The flag is passed anyway so the behaviour cannot change + // the day a branch shares a name with a tag. + assert_eq!( + worktree_add_args( + &dir(), + Path::new("/home/e/.cache/factorio-oracle/worktrees/2.0.77"), + "2.0.77" + ), + vec![ + "-C", + "/home/e/GitHub/factorio-data", + "worktree", + "add", + "--detach", + "/home/e/.cache/factorio-oracle/worktrees/2.0.77", + "2.0.77", + ] + ); + } + + #[test] + fn removing_a_worktree_names_the_path_not_the_tag() { + // git tracks worktrees by path. Removing one is also what clears the + // admin entry this tool wrote into a clone it does not own. + assert_eq!( + worktree_remove_args( + &dir(), + Path::new("/home/e/.cache/factorio-oracle/worktrees/2.0.77") + ), + vec![ + "-C", + "/home/e/GitHub/factorio-data", + "worktree", + "remove", + "/home/e/.cache/factorio-oracle/worktrees/2.0.77", + ] + ); + } + + #[test] + fn checking_a_worktrees_head_reads_the_tree_not_the_shared_clone() { + // The path named is the worktree, not the clone this whole module is + // careful never to write to - one read answers both what the tree is + // at and what the tag resolves to now. + assert_eq!( + worktree_head_args( + Path::new("/home/e/.cache/factorio-oracle/worktrees/2.0.77"), + "2.0.77" + ), + vec![ + "-C", + "/home/e/.cache/factorio-oracle/worktrees/2.0.77", + "rev-parse", + "HEAD", + "refs/tags/2.0.77^{commit}", + ] + ); + } + + #[test] + fn a_grep_line_parses_into_its_four_parts() { + // Copied verbatim from a real run at the 2.0.73 tag. + let hit = parse_hit( + "2.0.73", + "2.0.73:elevated-rails/prototypes/entity/elevated-rails.lua:309: support_range = 11,", + ) + .expect("this is the exact shape git produced"); + assert_eq!(hit.tag, "2.0.73"); + assert_eq!( + hit.path, + "elevated-rails/prototypes/entity/elevated-rails.lua" + ); + assert_eq!(hit.line, 309); + assert_eq!(hit.text, " support_range = 11,"); + } + + #[test] + fn colons_inside_the_matched_text_are_kept() { + // Only the first two colons after the tag are separators. A Lua table + // key or a URL in a comment holds more, and they belong to the text. + let hit = parse_hit("2.1.14", "2.1.14:base/x.lua:7: url = \"http://a:80/b\",") + .expect("should parse"); + assert_eq!(hit.line, 7); + assert_eq!(hit.text, " url = \"http://a:80/b\","); + } + + #[test] + fn a_line_for_a_different_tag_is_not_parsed() { + // The tag is passed in rather than guessed. Guessing would mean + // splitting on the first colon, which a tag holding one would break. + assert!(parse_hit("2.1.14", "2.0.73:base/x.lua:7:text").is_none()); + } + + #[test] + fn a_line_with_no_line_number_is_not_parsed() { + assert!(parse_hit("2.1.14", "2.1.14:base/x.lua:notanumber:text").is_none()); + } + + #[test] + fn an_empty_line_is_not_parsed() { + // git's stdout ends with a newline, so splitting it yields a trailing + // empty string on every successful grep. + assert!(parse_hit("2.1.14", "").is_none()); + } + + #[test] + fn the_timeouts_are_set_and_the_fetch_gets_the_longer_one() { + // CLAUDE.md names "nothing here has a timeout by default in the + // consumer repos" as a thing this tool exists to fix, so nothing + // added here may inherit it. + assert!(GIT_TIMEOUT < FETCH_TIMEOUT); + assert_eq!(GIT_TIMEOUT, Duration::from_secs(30)); + assert_eq!(FETCH_TIMEOUT, Duration::from_secs(180)); + } +} diff --git a/src/refs/grep.rs b/src/refs/grep.rs new file mode 100644 index 0000000..70a5089 --- /dev/null +++ b/src/refs/grep.rs @@ -0,0 +1,660 @@ +//! Searching one tag, or several, and saying whether the answer moved. +//! +//! The several-tags case is what earns this command. factorio-blueprint-editor's +//! `tools/oracle/probe-elevated-rail-support.mjs:776` records by hand that +//! `support_range` is 11 on rail-support and 9 on rail-ramp "at both the +//! 2.0.73 and the 2.1.12 tags", and that sentence is copied into its fixture +//! as a version caveat. Answering it today means two checkouts of a clone +//! three repos share, or reading two web pages. Answering it here costs two +//! `git grep` calls and moves nothing. + +use super::git::{self, Hit}; +use crate::spawn::Spawner; +use std::path::Path; + +/// Every match for one tag. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct TagResult { + pub tag: String, + pub hits: Vec, +} + +/// Whether the answer moved between the tags that were asked about. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Verdict { + /// One tag was asked for, so there is nothing to compare. + Single, + /// Every tag matched the same lines in the same files. + Identical, + /// At least one tag matched something the others did not. + Differs, + /// More than one tag was asked about and none of them matched anything. + /// + /// Reported apart from `Identical` on purpose. Two absences agreeing is + /// not evidence that a value is unchanged, it is evidence the pattern is + /// absent - which is what `docs/method.md` means by refusing to treat + /// last man standing as a measurement. Without this variant a mistyped + /// pattern comes back as `verdict: "identical"`, and a consumer keying + /// on that string reads a typo as a positive finding. + NothingMatched, +} + +impl Verdict { + fn as_str(self) -> &'static str { + match self { + Verdict::Single => "single", + Verdict::Identical => "identical", + Verdict::Differs => "differs", + Verdict::NothingMatched => "nothing-matched", + } + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct GrepReport { + pub pattern: String, + pub tags: Vec, + pub verdict: Verdict, +} + +impl GrepReport { + /// True when no tag matched anything at all. + pub fn empty(&self) -> bool { + self.tags.iter().all(|t| t.hits.is_empty()) + } +} + +/// Turns one `git grep` run's stdout into hits, dropping the trailing empty +/// line every successful run ends with. +pub fn hits_from_stdout(tag: &str, stdout: &str) -> Vec { + stdout + .lines() + .filter_map(|line| git::parse_hit(tag, line)) + .collect() +} + +/// The comparison key for one tag: every match as its path and its trimmed +/// text, sorted. +/// +/// Line numbers are deliberately left out. The question a consumer asks is +/// "is this value still the same", and a value that moved down the file has +/// not changed. Including the number would report a difference every time +/// anything above the match was edited, which would make the verdict useless +/// within one release. +fn fingerprint(result: &TagResult) -> Vec<(String, String)> { + let mut keys: Vec<(String, String)> = result + .hits + .iter() + .map(|h| (h.path.clone(), h.text.trim().to_string())) + .collect(); + keys.sort(); + keys +} + +/// Compares every tag against the first one. +pub fn verdict(tags: &[TagResult]) -> Verdict { + if tags.len() < 2 { + return Verdict::Single; + } + if tags.iter().all(|t| t.hits.is_empty()) { + return Verdict::NothingMatched; + } + let first = fingerprint(&tags[0]); + if tags[1..].iter().all(|t| fingerprint(t) == first) { + Verdict::Identical + } else { + Verdict::Differs + } +} + +/// Greps every tag in turn and builds the report. +/// +/// `git grep` exits 1 when nothing matched, which is an answer rather than a +/// failure, so only an exit code above 1 is treated as an error. +pub fn search( + spawner: &dyn Spawner, + clone: &Path, + pattern: &str, + tags: &[String], + pathspec: &[String], +) -> anyhow::Result { + // Checked here and not only at the CLI, because a library function must + // not trust its caller and both of these are `pub`. `grep_args` places + // the tag as a bare positional with no `--` before it, so a tag starting + // with `-` reaches git as an option, and `git grep + // --open-files-in-pager=` runs a command. + for tag in tags { + anyhow::ensure!(super::valid_tag(tag), "{tag} is not a usable tag name"); + } + let mut results = Vec::new(); + for tag in tags { + let args = git::grep_args(clone, tag, pattern, pathspec); + let out = spawner.run(Path::new("git"), &args, Some(git::GIT_TIMEOUT))?; + match out.exit_code { + Some(0) | Some(1) => {} + other => { + anyhow::bail!( + "git grep at {tag} failed (exit {}): {}", + other + .map(|c| c.to_string()) + .unwrap_or_else(|| "killed".into()), + out.stderr.trim() + ); + } + } + results.push(TagResult { + tag: tag.clone(), + hits: hits_from_stdout(tag, &out.stdout), + }); + } + let verdict = verdict(&results); + Ok(GrepReport { + pattern: pattern.to_string(), + tags: results, + verdict, + }) +} + +/// Reads one file at one tag. `HEAD` does not move. +pub fn show(spawner: &dyn Spawner, clone: &Path, tag: &str, path: &str) -> anyhow::Result { + anyhow::ensure!(super::valid_tag(tag), "{tag} is not a usable tag name"); + let args = git::show_args(clone, tag, path); + let out = spawner.run(Path::new("git"), &args, Some(git::GIT_TIMEOUT))?; + if out.exit_code != Some(0) { + anyhow::bail!("git show {tag}:{path} failed: {}", out.stderr.trim()); + } + Ok(out.stdout) +} + +/// With one tag, this is `git grep`'s own output with the `:` prefix +/// removed, so it pipes into anything that already reads grep. With more than +/// one, a tag column is added, because without it the lines cannot be told +/// apart, and a verdict line is appended. +pub fn render(report: &GrepReport) -> String { + let mut out = String::new(); + let several = report.tags.len() > 1; + for result in &report.tags { + if result.hits.is_empty() && several { + out.push_str(&format!("{} (no match)\n", result.tag)); + continue; + } + for hit in &result.hits { + if several { + out.push_str(&format!("{} ", hit.tag)); + } + out.push_str(&format!("{}:{}:{}\n", hit.path, hit.line, hit.text)); + } + } + if several { + let names: Vec<&str> = report.tags.iter().map(|t| t.tag.as_str()).collect(); + let joined = match names.split_last() { + Some((last, rest)) if !rest.is_empty() => format!("{} and {last}", rest.join(", ")), + _ => names.join(""), + }; + out.push('\n'); + // Every variant is named rather than caught by `_`. A fourth verdict + // added later would otherwise compile silently and print "identical + // across", which is the worst wrong answer this tool can give. + match report.verdict { + Verdict::Differs => out.push_str(&format!("differs between {joined}\n")), + Verdict::NothingMatched => { + out.push_str(&format!("nothing matched at any of {joined}\n")) + } + Verdict::Identical | Verdict::Single => { + out.push_str(&format!("identical across {joined}\n")) + } + } + } + out +} + +pub fn to_json(report: &GrepReport) -> serde_json::Value { + serde_json::json!({ + "pattern": report.pattern, + "verdict": report.verdict.as_str(), + "tags": report.tags.iter().map(|t| serde_json::json!({ + "tag": t.tag, + "hits": t.hits.iter().map(|h| serde_json::json!({ + "path": h.path, + "line": h.line, + "text": h.text, + })).collect::>(), + })).collect::>(), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::spawn::SpawnResult; + use std::cell::RefCell; + use std::time::Duration; + + /// Replays canned git output, and remembers every argument vector it was + /// handed. Keyed by tag, because a multi-tag search calls git once per + /// tag and each call has to get its own answer. + struct FakeGit { + by_tag: Vec<(String, String)>, + seen: RefCell>>, + } + + impl Spawner for FakeGit { + fn run( + &self, + _binary: &Path, + args: &[String], + _timeout: Option, + ) -> anyhow::Result { + self.seen.borrow_mut().push(args.to_vec()); + let stdout = self + .by_tag + .iter() + .find(|(tag, _)| args.contains(tag)) + .map(|(_, out)| out.clone()) + .unwrap_or_default(); + Ok(SpawnResult { + exit_code: Some(if stdout.is_empty() { 1 } else { 0 }), + stdout, + stderr: String::new(), + }) + } + } + + /// The real output of `git grep -n -e support_range -- + /// elevated-rails/prototypes/entity/elevated-rails.lua`, measured + /// 2026-08-17 at both tags. Identical values, identical lines. + fn elevated_rails(tag: &str) -> String { + format!( + "{tag}:elevated-rails/prototypes/entity/elevated-rails.lua:111: support_range = 9,\n\ + {tag}:elevated-rails/prototypes/entity/elevated-rails.lua:309: support_range = 11,\n" + ) + } + + #[test] + fn a_trailing_newline_does_not_become_an_empty_hit() { + let hits = hits_from_stdout("2.0.73", &elevated_rails("2.0.73")); + assert_eq!(hits.len(), 2); + assert_eq!(hits[0].line, 111); + assert_eq!(hits[1].line, 309); + } + + #[test] + fn one_tag_is_never_a_comparison() { + let tags = vec![TagResult { + tag: "2.1.14".into(), + hits: hits_from_stdout("2.1.14", &elevated_rails("2.1.14")), + }]; + assert_eq!(verdict(&tags), Verdict::Single); + } + + #[test] + fn the_same_answer_at_two_tags_reads_as_identical() { + // This is factorio-blueprint-editor's hand-written claim, checked. + let tags = vec![ + TagResult { + tag: "2.0.73".into(), + hits: hits_from_stdout("2.0.73", &elevated_rails("2.0.73")), + }, + TagResult { + tag: "2.1.12".into(), + hits: hits_from_stdout("2.1.12", &elevated_rails("2.1.12")), + }, + ]; + assert_eq!(verdict(&tags), Verdict::Identical); + } + + #[test] + fn a_changed_value_reads_as_differs() { + let tags = vec![ + TagResult { + tag: "2.0.73".into(), + hits: hits_from_stdout("2.0.73", "2.0.73:a.lua:1: support_range = 11,\n"), + }, + TagResult { + tag: "2.1.12".into(), + hits: hits_from_stdout("2.1.12", "2.1.12:a.lua:1: support_range = 12,\n"), + }, + ]; + assert_eq!(verdict(&tags), Verdict::Differs); + } + + #[test] + fn a_line_that_only_moved_is_still_identical() { + // The question is "is this value still the same", not "is it still on + // the same line". Including the line number would report a change + // every time anything above the match was edited. + let tags = vec![ + TagResult { + tag: "2.0.73".into(), + hits: hits_from_stdout("2.0.73", "2.0.73:a.lua:309: support_range = 11,\n"), + }, + TagResult { + tag: "2.1.12".into(), + hits: hits_from_stdout("2.1.12", "2.1.12:a.lua:402: support_range = 11,\n"), + }, + ]; + assert_eq!(verdict(&tags), Verdict::Identical); + } + + #[test] + fn a_match_that_moved_to_another_file_is_a_difference() { + let tags = vec![ + TagResult { + tag: "2.0.73".into(), + hits: hits_from_stdout("2.0.73", "2.0.73:a.lua:1: support_range = 11,\n"), + }, + TagResult { + tag: "2.1.12".into(), + hits: hits_from_stdout("2.1.12", "2.1.12:b.lua:1: support_range = 11,\n"), + }, + ]; + assert_eq!(verdict(&tags), Verdict::Differs); + } + + #[test] + fn a_tag_matching_nothing_differs_from_one_that_matched() { + // A value that disappeared between versions is the most important + // difference there is, and it arrives as an empty result. + let tags = vec![ + TagResult { + tag: "2.0.73".into(), + hits: hits_from_stdout("2.0.73", "2.0.73:a.lua:1: fluidbox = {},\n"), + }, + TagResult { + tag: "2.1.12".into(), + hits: vec![], + }, + ]; + assert_eq!(verdict(&tags), Verdict::Differs); + } + + #[test] + fn three_tags_all_agreeing_read_as_identical() { + let tags = ["2.0.73", "2.1.12", "2.1.14"] + .iter() + .map(|t| TagResult { + tag: (*t).to_string(), + hits: hits_from_stdout(t, &format!("{t}:a.lua:1: support_range = 11,\n")), + }) + .collect::>(); + assert_eq!(verdict(&tags), Verdict::Identical); + } + + #[test] + fn three_tags_with_one_dissenter_read_as_differs() { + let mut tags = ["2.0.73", "2.1.12"] + .iter() + .map(|t| TagResult { + tag: (*t).to_string(), + hits: hits_from_stdout(t, &format!("{t}:a.lua:1: support_range = 11,\n")), + }) + .collect::>(); + tags.push(TagResult { + tag: "2.1.14".into(), + hits: hits_from_stdout("2.1.14", "2.1.14:a.lua:1: support_range = 12,\n"), + }); + assert_eq!(verdict(&tags), Verdict::Differs); + } + + #[test] + fn search_calls_git_once_per_tag_and_never_checks_anything_out() { + let fake = FakeGit { + by_tag: vec![ + ("2.0.73".into(), elevated_rails("2.0.73")), + ("2.1.12".into(), elevated_rails("2.1.12")), + ], + seen: RefCell::new(vec![]), + }; + let report = search( + &fake, + Path::new("/clone"), + "support_range", + &["2.0.73".to_string(), "2.1.12".to_string()], + &[], + ) + .expect("the fake always answers"); + + assert_eq!(report.tags.len(), 2); + assert_eq!(report.verdict, Verdict::Identical); + + let seen = fake.seen.borrow(); + assert_eq!(seen.len(), 2); + for args in seen.iter() { + assert!(args.contains(&"grep".to_string())); + assert!(!args + .iter() + .any(|a| a == "checkout" || a == "switch" || a == "reset")); + } + } + + #[test] + fn two_tags_that_both_matched_nothing_are_not_identical() { + // Absence at both tags is not agreement. `docs/method.md` calls this + // out directly: last man standing is not a measurement. The likeliest + // real cause is a mistyped pattern, and reporting that as "identical" + // hands back a positive finding for a question nobody asked. + let tags = vec![ + TagResult { + tag: "2.0.73".into(), + hits: vec![], + }, + TagResult { + tag: "2.1.12".into(), + hits: vec![], + }, + ]; + assert_eq!(verdict(&tags), Verdict::NothingMatched); + } + + #[test] + fn nothing_matched_renders_as_absence_rather_than_agreement() { + let report = GrepReport { + pattern: "zzz-not-a-real-token-zzz".into(), + tags: vec![ + TagResult { + tag: "2.0.73".into(), + hits: vec![], + }, + TagResult { + tag: "2.1.12".into(), + hits: vec![], + }, + ], + verdict: Verdict::NothingMatched, + }; + let text = render(&report); + assert!(text.contains("nothing matched at any of 2.0.73 and 2.1.12")); + assert!(!text.contains("identical")); + // And the machine-readable form, which is the one that can mislead + // silently: a consumer keying on the verdict string must not see + // "identical" here. + assert_eq!(to_json(&report)["verdict"], "nothing-matched"); + } + + #[test] + fn a_tag_that_git_would_read_as_an_option_never_reaches_git() { + // `grep_args` puts the tag as a bare positional with no `--` before + // it, so `git grep --open-files-in-pager=` would run a command. + // The CLI checks this, but `search` is `pub` and later tasks call + // into this module, so the guard belongs at this boundary too. + let fake = FakeGit { + by_tag: vec![], + seen: RefCell::new(vec![]), + }; + let err = search( + &fake, + Path::new("/clone"), + "support_range", + &["--open-files-in-pager=touch /tmp/pwned".to_string()], + &[], + ) + .expect_err("a tag git would read as an option must be rejected"); + assert!(err.to_string().contains("not a usable tag name")); + assert!( + fake.seen.borrow().is_empty(), + "it must be rejected before git is called at all" + ); + } + + #[test] + fn one_tag_renders_without_a_tag_column() { + // With one tag the output is git grep's own shape with the `:` + // prefix removed, which is what the design asked for. It pipes into + // anything that already reads grep. + let report = GrepReport { + pattern: "support_range".into(), + tags: vec![TagResult { + tag: "2.1.14".into(), + hits: hits_from_stdout("2.1.14", &elevated_rails("2.1.14")), + }], + verdict: Verdict::Single, + }; + // Asserted as the whole string, not with `contains`. Review on + // 2026-08-17 showed the `contains` form could not fail: adding a tag + // column would leave every substring it checked intact, and the + // `2.1.14:` check was dead too, because `parse_hit` strips that + // prefix before a `Hit` exists. So the test could not catch the one + // regression its name promises. + let text = render(&report); + assert_eq!( + text, + "elevated-rails/prototypes/entity/elevated-rails.lua:111: support_range = 9,\n\ + elevated-rails/prototypes/entity/elevated-rails.lua:309: support_range = 11,\n" + ); + } + + #[test] + fn several_tags_render_with_a_tag_column_and_a_verdict() { + let report = GrepReport { + pattern: "support_range".into(), + tags: vec![ + TagResult { + tag: "2.0.73".into(), + hits: hits_from_stdout("2.0.73", &elevated_rails("2.0.73")), + }, + TagResult { + tag: "2.1.12".into(), + hits: hits_from_stdout("2.1.12", &elevated_rails("2.1.12")), + }, + ], + verdict: Verdict::Identical, + }; + let text = render(&report); + assert!(text.contains("2.0.73 elevated-rails/")); + assert!(text.contains("2.1.12 elevated-rails/")); + assert!(text.contains("identical across 2.0.73 and 2.1.12")); + } + + #[test] + fn a_differing_result_says_so_without_deciding_what_it_means() { + // The provenance split again: a machine can say the two disagree, and + // only a human can say whether that matters. + let report = GrepReport { + pattern: "support_range".into(), + tags: vec![ + TagResult { + tag: "2.0.73".into(), + hits: hits_from_stdout("2.0.73", "2.0.73:a.lua:1: support_range = 11,\n"), + }, + TagResult { + tag: "2.1.12".into(), + hits: hits_from_stdout("2.1.12", "2.1.12:a.lua:1: support_range = 12,\n"), + }, + ], + verdict: Verdict::Differs, + }; + let text = render(&report); + assert!(text.contains("differs between 2.0.73 and 2.1.12")); + } + + #[test] + fn a_tag_with_no_matches_is_reported_rather_than_dropped() { + let report = GrepReport { + pattern: "fluidbox".into(), + tags: vec![ + TagResult { + tag: "2.0.73".into(), + hits: hits_from_stdout("2.0.73", "2.0.73:a.lua:1: fluidbox = {},\n"), + }, + TagResult { + tag: "2.1.12".into(), + hits: vec![], + }, + ], + verdict: Verdict::Differs, + }; + let text = render(&report); + assert!(text.contains("2.1.12 (no match)")); + } + + #[test] + fn the_json_carries_every_field_the_text_does() { + let report = GrepReport { + pattern: "support_range".into(), + tags: vec![TagResult { + tag: "2.1.14".into(), + hits: hits_from_stdout("2.1.14", &elevated_rails("2.1.14")), + }], + verdict: Verdict::Single, + }; + let json = to_json(&report); + assert_eq!(json["pattern"], "support_range"); + assert_eq!(json["verdict"], "single"); + assert_eq!(json["tags"][0]["tag"], "2.1.14"); + assert_eq!(json["tags"][0]["hits"][1]["line"], 309); + assert_eq!( + json["tags"][0]["hits"][1]["path"], + "elevated-rails/prototypes/entity/elevated-rails.lua" + ); + assert_eq!( + json["tags"][0]["hits"][1]["text"], + " support_range = 11," + ); + } + + #[test] + fn show_returns_the_file_at_that_tag() { + struct FakeShow; + impl Spawner for FakeShow { + fn run( + &self, + _binary: &Path, + args: &[String], + _timeout: Option, + ) -> anyhow::Result { + assert!(args.contains(&"2.0.77:base/info.json".to_string())); + Ok(SpawnResult { + exit_code: Some(0), + stdout: "{\n \"version\": \"2.0.77\"\n}\n".into(), + stderr: String::new(), + }) + } + } + let got = show(&FakeShow, Path::new("/clone"), "2.0.77", "base/info.json") + .expect("the fake always answers"); + assert!(got.contains("\"version\": \"2.0.77\"")); + } + + #[test] + fn show_reports_a_missing_path_as_an_error() { + // Measured: git exits 128 with "fatal: path ... does not exist in". + struct FakeMissing; + impl Spawner for FakeMissing { + fn run( + &self, + _binary: &Path, + _args: &[String], + _timeout: Option, + ) -> anyhow::Result { + Ok(SpawnResult { + exit_code: Some(128), + stdout: String::new(), + stderr: "fatal: path 'base/nope.lua' does not exist in '2.1.14'\n".into(), + }) + } + } + let err = show(&FakeMissing, Path::new("/clone"), "2.1.14", "base/nope.lua") + .expect_err("128 is not success"); + assert!(err.to_string().contains("does not exist")); + } +} diff --git a/src/refs/mod.rs b/src/refs/mod.rs new file mode 100644 index 0000000..8fef2fc --- /dev/null +++ b/src/refs/mod.rs @@ -0,0 +1,186 @@ +//! Reference material: the game's data Lua, and the Lua API docs. +//! +//! Both live outside this repo, and both are shared. `~/GitHub/factorio-data` +//! is one clone with one working tree that at least three repos read, so +//! nothing in this module ever moves its `HEAD`. Reads happen at a tag, with +//! `git show` and `git grep`. Anything that needs a real directory gets its +//! own worktree under this tool's cache instead. +//! +//! Measured 2026-08-17: the clone is on `master` at 2.1.14, not detached at +//! any tag. FactorioMapWebUI's `refs:sync --check` reports "in sync" only +//! because `master`'s `base/info.json` happens to equal the newest tag. That +//! is a coincidence, and it is the failure this module exists to remove. + +pub mod docs; +pub mod git; +pub mod grep; +pub mod sync; +pub mod worktree; + +use std::path::{Path, PathBuf}; + +/// The directory this tool keeps things it can fetch or build again. +/// +/// Order: an explicit override, then `XDG_CACHE_HOME`, then `~/.cache`. +/// `install.rs` reads `HOME` and nothing else, so this follows it rather than +/// inventing a second rule for where a home directory is. +pub fn cache_dir(home: &Path, override_dir: Option<&Path>, xdg_cache: Option<&Path>) -> PathBuf { + if let Some(dir) = override_dir { + return dir.to_path_buf(); + } + if let Some(dir) = xdg_cache { + return dir.join("factorio-oracle"); + } + home.join(".cache").join("factorio-oracle") +} + +/// Where the `wube/factorio-data` clone is. +/// +/// `FACTORIO_DATA_DIR` is the name FactorioMapWebUI's `sync-factorio-refs.sh` +/// already uses, so someone who has set it gets one answer out of both tools +/// instead of two. +pub fn data_clone(home: &Path, override_dir: Option<&Path>) -> PathBuf { + match override_dir { + Some(dir) => dir.to_path_buf(), + None => home.join("GitHub").join("factorio-data"), + } +} + +/// True when `dir` looks like a git checkout. +/// +/// `.git` is a directory in a clone and a file in a worktree, so this tests +/// for either rather than for a directory. +pub fn is_clone(dir: &Path) -> bool { + dir.join(".git").exists() +} + +/// Rejects a tag that could escape a directory or be read as a git option. +/// +/// Two separate problems. `refs worktree ` joins the tag onto the cache +/// path, so a tag holding a separator or `..` would write outside it. And a +/// tag starting with `-` would be read by git as a flag, which is how an +/// argument vector turns into an instruction. +pub fn valid_tag(tag: &str) -> bool { + // `.` and `..` slip past the character test below, because both are made + // only of dots and dots are allowed. They are the two names the + // filesystem treats specially, so `cache.join("..")` resolves to the + // cache's parent - the exact escape this function exists to stop. + // Measured 2026-08-17: without these two lines `valid_tag("..")` returned + // true, and the traversal test passed only because every case it tried + // also held a separator. + if tag == "." || tag == ".." { + return false; + } + !tag.is_empty() + && !tag.starts_with('-') + && tag + .chars() + .all(|c| c.is_ascii_alphanumeric() || c == '.' || c == '_' || c == '-') +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn an_explicit_cache_override_wins_over_everything() { + let got = cache_dir( + Path::new("/home/e"), + Some(Path::new("/tmp/mycache")), + Some(Path::new("/home/e/.xdg")), + ); + assert_eq!(got, PathBuf::from("/tmp/mycache")); + } + + #[test] + fn xdg_cache_home_is_used_when_there_is_no_override() { + // The tool's own directory is appended, so an XDG cache root holding + // other tools' data is not written into directly. + let got = cache_dir(Path::new("/home/e"), None, Some(Path::new("/home/e/.xdg"))); + assert_eq!(got, PathBuf::from("/home/e/.xdg/factorio-oracle")); + } + + #[test] + fn the_last_resort_cache_is_under_the_home_directory() { + let got = cache_dir(Path::new("/home/e"), None, None); + assert_eq!(got, PathBuf::from("/home/e/.cache/factorio-oracle")); + } + + #[test] + fn the_clone_defaults_to_the_path_every_repo_already_uses() { + let got = data_clone(Path::new("/home/e"), None); + assert_eq!(got, PathBuf::from("/home/e/GitHub/factorio-data")); + } + + #[test] + fn an_explicit_clone_path_wins() { + let got = data_clone(Path::new("/home/e"), Some(Path::new("/srv/factorio-data"))); + assert_eq!(got, PathBuf::from("/srv/factorio-data")); + } + + #[test] + fn a_directory_counts_as_a_clone_when_dot_git_is_a_directory() { + let tmp = tempfile::tempdir().unwrap(); + std::fs::create_dir(tmp.path().join(".git")).unwrap(); + assert!(is_clone(tmp.path())); + } + + #[test] + fn a_directory_counts_as_a_clone_when_dot_git_is_a_file() { + // Inside a worktree, `.git` is a file holding a gitdir: line. Measured + // 2026-08-17: `gitdir: /Users/ericjohnson/GitHub/factorio-data/.git/worktrees/wt-2.0.77`. + let tmp = tempfile::tempdir().unwrap(); + std::fs::write(tmp.path().join(".git"), b"gitdir: /elsewhere\n").unwrap(); + assert!(is_clone(tmp.path())); + } + + #[test] + fn a_plain_directory_is_not_a_clone() { + let tmp = tempfile::tempdir().unwrap(); + assert!(!is_clone(tmp.path())); + } + + #[test] + fn real_factorio_tags_are_valid() { + for tag in ["2.1.14", "2.0.77", "1.1.110", "0.17.79"] { + assert!(valid_tag(tag), "{tag} should be valid"); + } + } + + #[test] + fn a_tag_that_could_escape_a_directory_is_rejected() { + // `refs worktree ` joins the tag onto the cache path, so this + // one would write outside it. + assert!(!valid_tag("../../etc")); + assert!(!valid_tag("a/b")); + assert!(!valid_tag("a\\b")); + } + + #[test] + fn the_two_dot_names_the_filesystem_treats_specially_are_rejected() { + // The cases the character allowlist cannot catch on its own: both are + // made only of dots, which the allowlist permits. `cache.join("..")` + // resolves to the cache's parent. + // + // The traversal test above passes only because each of its cases + // holds a separator, so `..` alone was never exercised. Found by + // review on 2026-08-17 after `valid_tag("..")` was confirmed to + // return true. + assert!(!valid_tag("..")); + assert!(!valid_tag(".")); + // A name that is only dots but means nothing special stays legal: + // `join("...")` makes a directory literally called "...". + assert!(valid_tag("...")); + } + + #[test] + fn a_tag_that_git_would_read_as_an_option_is_rejected() { + assert!(!valid_tag("--upload-pack=touch /tmp/pwned")); + assert!(!valid_tag("-v")); + } + + #[test] + fn an_empty_tag_is_rejected() { + assert!(!valid_tag("")); + } +} diff --git a/src/refs/sync.rs b/src/refs/sync.rs new file mode 100644 index 0000000..4af75b5 --- /dev/null +++ b/src/refs/sync.rs @@ -0,0 +1,396 @@ +//! Whether a version can be read at all, and whether it can be read offline. +//! +//! This is deliberately **not** what FactorioMapWebUI's `refs:sync` does. +//! That script pins state: it checks a tag out into a shared working tree and +//! then reads `base/info.json` back to confirm. Since nothing here ever moves +//! `HEAD`, there is no working-tree state left to pin, so there is nothing to +//! keep in sync and no lock file to go stale. +//! +//! What is left is availability. Is the clone there, is the tag fetched, and +//! are that version's docs reachable without a network. `sync` may fetch tags +//! to make the answer yes. `--check` never fetches and never writes, and +//! exits 1 when the answer is no. + +use super::git; +use crate::spawn::Spawner; +use std::path::{Path, PathBuf}; + +// `docs` is imported by the test module rather than here. `docs_standing` +// builds its own `/docs/` path, so the only non-test use +// would be none at all, and CI runs clippy with `-D warnings`. + +/// Where that version's docs can be read from, if anywhere. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum DocsStanding { + /// The version is installed, so every docs file is already on disk. + Installed(PathBuf), + /// At least one file has been fetched into the cache. + Cached(PathBuf), + /// Nothing yet, which is the normal state for a version nobody asked + /// about. Files arrive one at a time, on demand. + Absent, +} + +impl DocsStanding { + fn as_str(&self) -> &'static str { + match self { + DocsStanding::Installed(_) => "installed", + DocsStanding::Cached(_) => "cached", + DocsStanding::Absent => "absent", + } + } +} + +/// What is readable for one version, right now. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Availability { + pub version: String, + pub clone: PathBuf, + pub clone_present: bool, + pub tag_present: bool, + pub docs: DocsStanding, +} + +impl Availability { + /// True when the data Lua for this version can be read. + /// + /// Docs are not part of this. They arrive one file at a time on demand, + /// so "not fetched yet" is the normal state and reporting it as a failure + /// would make `--check` red on a healthy machine. + pub fn ok(&self) -> bool { + self.clone_present && self.tag_present + } +} + +/// Whether the clone can resolve this tag. Fetches nothing. +pub fn tag_present(spawner: &dyn Spawner, clone: &Path, tag: &str) -> anyhow::Result { + let args = git::tag_exists_args(clone, tag); + let out = spawner.run(Path::new("git"), &args, Some(git::GIT_TIMEOUT))?; + Ok(out.exit_code == Some(0)) +} + +/// Fetches tags, so a tag released since the last fetch becomes readable. +/// +/// This writes refs and objects into a clone this tool does not own, and +/// nothing else. No working tree changes and `HEAD` does not move. +pub fn fetch_tags(spawner: &dyn Spawner, clone: &Path) -> anyhow::Result<()> { + let args = git::fetch_tags_args(clone); + let out = spawner.run(Path::new("git"), &args, Some(git::FETCH_TIMEOUT))?; + if out.exit_code != Some(0) { + anyhow::bail!("git fetch --tags failed: {}", out.stderr.trim()); + } + Ok(()) +} + +/// True when `dir`, or anything under it, holds at least one regular file. +/// +/// `docs::fetch` creates the *parent* directory of the file it is about to +/// write before it runs curl, and for a nested path like +/// `auxiliary/noise-expressions.html` that parent is a subdirectory. So a +/// failed fetch can leave an empty `auxiliary/` behind one level under +/// `/docs/` - a directory entry with no file in it. Measured +/// 2026-08-17: a shallow `read_dir` on the version directory alone reported +/// that case as cached, for a version with zero docs files anywhere on disk. +/// Walking into subdirectories is what tells the two apart. +fn contains_a_file(dir: &Path) -> bool { + let Ok(entries) = std::fs::read_dir(dir) else { + return false; + }; + for entry in entries.flatten() { + let path = entry.path(); + if path.is_dir() { + if contains_a_file(&path) { + return true; + } + } else if path.is_file() { + return true; + } + } + false +} + +/// Where this version's docs are, if anywhere. +pub fn docs_standing( + installed_doc_dir: Option<&Path>, + cache: &Path, + version: &str, +) -> DocsStanding { + if let Some(doc) = installed_doc_dir { + if doc.is_dir() { + return DocsStanding::Installed(doc.to_path_buf()); + } + } + let dir = cache.join("docs").join(version); + // An empty directory, or a directory holding only other empty + // directories, is what an interrupted fetch leaves, and it holds no + // answers, so it does not count. + if contains_a_file(&dir) { + return DocsStanding::Cached(dir); + } + DocsStanding::Absent +} + +pub fn render(a: &Availability) -> String { + let mut out = String::new(); + out.push_str(&format!("Factorio {} reference material:\n", a.version)); + out.push_str(&format!( + " {:<14} {}\n", + "clone", + if a.clone_present { + a.clone.display().to_string() + } else { + format!("{} (not cloned)", a.clone.display()) + } + )); + out.push_str(&format!( + " {:<14} {}\n", + "tag", + if a.tag_present { + format!("{} is readable", a.version) + } else { + format!("{} is not fetched", a.version) + } + )); + out.push_str(&format!( + " {:<14} {}\n", + "lua-api docs", + match &a.docs { + DocsStanding::Installed(p) => format!("installed at {}", p.display()), + DocsStanding::Cached(p) => format!("cached at {}", p.display()), + DocsStanding::Absent => "not fetched yet, fetched per file on demand".to_string(), + } + )); + if a.ok() { + // Say the rule out loud. Someone reading this output is the person + // most likely to reach for a checkout next. + out.push_str(" -> readable at the tag. HEAD was not moved.\n"); + } else if !a.clone_present { + out.push_str( + " -> clone https://github.com/wube/factorio-data, or set FACTORIO_DATA_DIR.\n", + ); + } else { + out.push_str(&format!( + " -> run 'factorio-oracle refs sync {}' to fetch the tag.\n", + a.version + )); + } + out +} + +pub fn to_json(a: &Availability) -> serde_json::Value { + serde_json::json!({ + "version": a.version, + "clone": a.clone, + "clonePresent": a.clone_present, + "tagPresent": a.tag_present, + "docs": a.docs.as_str(), + "ok": a.ok(), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::refs::docs; + use crate::spawn::SpawnResult; + use std::cell::RefCell; + use std::time::Duration; + + struct FakeGit { + /// Exit codes to hand back, in order. `rev-parse` exits 0 for a tag + /// that is there and 1 for one that is not. + exits: RefCell>, + seen: RefCell>>, + } + + impl FakeGit { + fn returning(exits: &[i32]) -> Self { + FakeGit { + exits: RefCell::new(exits.to_vec()), + seen: RefCell::new(vec![]), + } + } + } + + impl Spawner for FakeGit { + fn run( + &self, + _binary: &Path, + args: &[String], + _timeout: Option, + ) -> anyhow::Result { + self.seen.borrow_mut().push(args.to_vec()); + let code = if self.exits.borrow().is_empty() { + 0 + } else { + self.exits.borrow_mut().remove(0) + }; + Ok(SpawnResult { + exit_code: Some(code), + stdout: String::new(), + stderr: String::new(), + }) + } + } + + #[test] + fn a_tag_that_resolves_is_present() { + let fake = FakeGit::returning(&[0]); + assert!(tag_present(&fake, Path::new("/clone"), "2.0.73").unwrap()); + } + + #[test] + fn a_tag_that_does_not_resolve_is_absent_rather_than_an_error() { + // Measured: `rev-parse -q --verify` exits 1 and prints nothing for a + // tag that is not there. That is an answer, not a failure. + let fake = FakeGit::returning(&[1]); + assert!(!tag_present(&fake, Path::new("/clone"), "9.9.9").unwrap()); + } + + #[test] + fn checking_a_tag_never_fetches_and_never_checks_out() { + let fake = FakeGit::returning(&[0]); + tag_present(&fake, Path::new("/clone"), "2.0.73").unwrap(); + let seen = fake.seen.borrow(); + assert_eq!(seen.len(), 1); + assert!(!seen[0] + .iter() + .any(|a| a == "fetch" || a == "checkout" || a == "switch" || a == "reset")); + } + + #[test] + fn an_installed_version_reads_as_installed() { + let cache = tempfile::tempdir().unwrap(); + let doc = cache.path().join("doc-html"); + std::fs::create_dir_all(&doc).unwrap(); + let got = docs_standing(Some(&doc), cache.path(), "2.1.14"); + assert_eq!(got, DocsStanding::Installed(doc)); + } + + #[test] + fn a_version_with_files_in_the_cache_reads_as_cached() { + let cache = tempfile::tempdir().unwrap(); + let path = docs::cache_path(cache.path(), "2.0.45", "runtime-api.json"); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(&path, b"{}").unwrap(); + let got = docs_standing(None, cache.path(), "2.0.45"); + assert_eq!( + got, + DocsStanding::Cached(cache.path().join("docs").join("2.0.45")) + ); + } + + #[test] + fn an_empty_cache_directory_does_not_count_as_cached() { + // A directory made by an interrupted fetch holds no answers. + let cache = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(cache.path().join("docs").join("2.0.45")).unwrap(); + assert_eq!( + docs_standing(None, cache.path(), "2.0.45"), + DocsStanding::Absent + ); + } + + #[test] + fn a_nested_empty_directory_does_not_count_as_cached() { + // `docs::fetch` creates the *parent* of the final file before running + // curl, and for a nested path like `auxiliary/noise-expressions.html` + // that parent is a subdirectory. A failed `refs docs 2.0.45 + // auxiliary/noise-expressions.html` - the plan's own worked example, + // not a hypothetical - leaves an empty `auxiliary/` behind. + // + // Measured 2026-08-17: the old check only read the top-level + // `/docs/` directory, so it saw the `auxiliary` entry + // and reported `Cached`, for a version with zero docs files on disk. + let cache = tempfile::tempdir().unwrap(); + std::fs::create_dir_all(cache.path().join("docs").join("2.0.45").join("auxiliary")) + .unwrap(); + assert_eq!( + docs_standing(None, cache.path(), "2.0.45"), + DocsStanding::Absent + ); + } + + #[test] + fn a_version_with_neither_reads_as_absent() { + let cache = tempfile::tempdir().unwrap(); + assert_eq!( + docs_standing(None, cache.path(), "2.0.45"), + DocsStanding::Absent + ); + } + + fn available() -> Availability { + Availability { + version: "2.0.73".into(), + clone: PathBuf::from("/home/e/GitHub/factorio-data"), + clone_present: true, + tag_present: true, + docs: DocsStanding::Absent, + } + } + + #[test] + fn a_readable_version_is_ok() { + assert!(available().ok()); + } + + #[test] + fn a_missing_tag_is_not_ok() { + let mut a = available(); + a.tag_present = false; + assert!(!a.ok()); + } + + #[test] + fn a_missing_clone_is_not_ok_even_with_a_tag_somehow_present() { + // Only `clone_present` is flipped. The first version of this test + // flipped both fields, so an `ok()` that ignored `clone_present` + // entirely still passed - found by mutation testing on 2026-08-17, + // the sixth vacuous test this plan produced. + // + // The state is unreachable through the CLI, which only asks about a + // tag once it has a clone. That is exactly the point: isolating one + // field is what lets this test fail for its own reason rather than + // its neighbour's. + let mut a = available(); + a.clone_present = false; + assert!(!a.ok()); + } + + #[test] + fn absent_docs_do_not_make_it_not_ok() { + // Docs are fetched on demand, one file at a time, so "not here yet" + // is the normal state and not a finding. + assert_eq!(available().docs, DocsStanding::Absent); + assert!(available().ok()); + } + + #[test] + fn the_report_names_the_clone_and_says_head_was_not_moved() { + // The line exists so someone reading the output learns the rule, + // rather than having to find it in a design document. + let text = render(&available()); + assert!(text.contains("/home/e/GitHub/factorio-data")); + assert!(text.contains("2.0.73")); + assert!(text.contains("HEAD was not moved")); + } + + #[test] + fn the_report_says_what_to_do_when_a_tag_is_missing() { + let mut a = available(); + a.tag_present = false; + let text = render(&a); + assert!(text.contains("refs sync")); + } + + #[test] + fn the_json_carries_every_field_the_text_does() { + let json = to_json(&available()); + assert_eq!(json["version"], "2.0.73"); + assert_eq!(json["clonePresent"], true); + assert_eq!(json["tagPresent"], true); + assert_eq!(json["docs"], "absent"); + assert_eq!(json["ok"], true); + } +} diff --git a/src/refs/worktree.rs b/src/refs/worktree.rs new file mode 100644 index 0000000..ba6a32f --- /dev/null +++ b/src/refs/worktree.rs @@ -0,0 +1,352 @@ +//! A real directory tree at a tag, for the tools that need one. +//! +//! `git show` and `git grep` cover most questions, but ripgrep, an editor and +//! a Lua parser all want files on disk. A worktree gives each caller its own +//! tree off one object store, which is better than making everything go +//! through `git show` and far better than a checkout in a clone three repos +//! read. +//! +//! Measured 2026-08-17 on the 19 MB factorio-data clone: `git worktree add +//! --detach` took 0.077 seconds and produced 8.6 MB. Two worktrees at the +//! same tag coexist. A missing parent directory is created. The main tree's +//! `HEAD` stayed on `master` throughout. +//! +//! This is also the one thing in `refs` that writes to a clone this tool does +//! not own: git records an entry under that clone's `.git/worktrees/`, 168 KB +//! for three trees. `remove` exists so a caller can put that back, rather +//! than leaving stale entries behind after a cache is wiped. + +use super::git; +use crate::spawn::Spawner; +use std::path::{Path, PathBuf}; + +/// Where the tree for `tag` goes. +/// +/// One tree per tag, shared by every caller who asks for that tag. This +/// function does not validate the tag itself - `ensure` and `remove`, its +/// only callers, both check `super::valid_tag` as their first line before +/// this is reached. Closed 2026-08-17: this comment used to say the tag "is +/// validated ... before it reaches here", which was the assumption a +/// whole-branch review found unenforced at this function's own two callers. +pub fn worktree_path(cache: &Path, tag: &str) -> PathBuf { + cache.join("worktrees").join(tag) +} + +/// Returns a real tree at `tag`, making one if it is not there yet. +pub fn ensure( + spawner: &dyn Spawner, + clone: &Path, + cache: &Path, + tag: &str, +) -> anyhow::Result { + // Checked here and not only at the CLI, because a library function must + // not trust its caller - the same rule `grep::search`, `grep::show` and + // `docs::fetch` already follow. `worktree_add_args` places the tag as a + // bare positional with no `--` before it, so an unchecked tag starting + // with `-` reaches git as an option. Found 2026-08-17 by the whole-branch + // review that built the complete guard table for the first time. + anyhow::ensure!(super::valid_tag(tag), "{tag} is not a usable tag name"); + let path = worktree_path(cache, tag); + if path.exists() { + if !super::is_clone(&path) { + anyhow::bail!( + "{} exists but is not a git worktree. Remove it and try again.", + path.display() + ); + } + // Reusing a tree because its directory is named after a tag is the + // same coincidence this module exists to remove. `master`'s + // base/info.json equalling the newest tag is exactly what made + // another repo's drift check report "in sync" while pinned to + // nothing. So ask the tree what it is actually at, rather than + // trusting what it is called. + // + // A mismatch is reported, not silently rebuilt: a tag that moved is + // a finding a human should see, and the same rule the provenance + // module already follows. + let args = git::worktree_head_args(&path, tag); + let out = spawner.run(Path::new("git"), &args, Some(git::GIT_TIMEOUT))?; + if out.exit_code != Some(0) { + anyhow::bail!( + "could not check what {} is at: {}", + path.display(), + out.stderr.trim() + ); + } + let mut shas = out.stdout.split_whitespace(); + let (head, wanted) = ( + shas.next().unwrap_or_default(), + shas.next().unwrap_or_default(), + ); + anyhow::ensure!( + !head.is_empty() && !wanted.is_empty(), + "could not read two commits out of git rev-parse for {}", + path.display() + ); + anyhow::ensure!( + head == wanted, + "{} is at {head} but tag {tag} is now {wanted}. The tag moved. \ + Run `refs worktree {tag} --remove` and try again.", + path.display() + ); + return Ok(path); + } + let args = git::worktree_add_args(clone, &path, tag); + let out = spawner.run(Path::new("git"), &args, Some(git::GIT_TIMEOUT))?; + if out.exit_code != Some(0) { + anyhow::bail!("git worktree add {tag} failed: {}", out.stderr.trim()); + } + Ok(path) +} + +/// Removes the tree for `tag`, and the entry git wrote into the shared clone. +/// +/// Deleting the directory instead would leave that entry behind in a clone +/// this tool does not own. +pub fn remove(spawner: &dyn Spawner, clone: &Path, cache: &Path, tag: &str) -> anyhow::Result<()> { + // `remove` is the worse of the two unguarded entry points the review + // found, because it deletes: an unchecked tag holding `../` builds a + // path outside the cache and hands it to `git worktree remove`. Same + // rule and same date as `ensure`'s guard above. + anyhow::ensure!(super::valid_tag(tag), "{tag} is not a usable tag name"); + let path = worktree_path(cache, tag); + if !path.exists() { + return Ok(()); + } + let args = git::worktree_remove_args(clone, &path); + let out = spawner.run(Path::new("git"), &args, Some(git::GIT_TIMEOUT))?; + if out.exit_code != Some(0) { + anyhow::bail!("git worktree remove failed: {}", out.stderr.trim()); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::spawn::SpawnResult; + use std::cell::RefCell; + use std::time::Duration; + + struct FakeGit { + exit: i32, + stdout: String, + stderr: String, + seen: RefCell>>, + } + + impl FakeGit { + fn ok() -> Self { + FakeGit { + exit: 0, + stdout: String::new(), + stderr: String::new(), + seen: RefCell::new(vec![]), + } + } + + /// A fake that answers `rev-parse` with two shas, which is what the + /// reuse check reads. + fn speaking(stdout: &str) -> Self { + FakeGit { + exit: 0, + stdout: stdout.to_string(), + stderr: String::new(), + seen: RefCell::new(vec![]), + } + } + } + + impl Spawner for FakeGit { + fn run( + &self, + _binary: &Path, + args: &[String], + _timeout: Option, + ) -> anyhow::Result { + self.seen.borrow_mut().push(args.to_vec()); + Ok(SpawnResult { + exit_code: Some(self.exit), + stdout: self.stdout.clone(), + stderr: self.stderr.clone(), + }) + } + } + + /// A real commit from the 2.0.77 tag, so the fixtures below read like the + /// output git actually produces. + const AT_2_0_77: &str = "ce6741a54c3199caad4ed147e9987fd8d5653fdf"; + /// And one from 2.1.14, for the case where the tag moved. + const AT_2_1_14: &str = "a78495471dfbe7848c9c7be752e32f5065e072cb"; + + #[test] + fn a_worktree_is_named_for_its_tag_under_the_cache() { + assert_eq!( + worktree_path(Path::new("/home/e/.cache/factorio-oracle"), "2.0.77"), + PathBuf::from("/home/e/.cache/factorio-oracle/worktrees/2.0.77") + ); + } + + #[test] + fn a_tag_that_git_would_read_as_an_option_never_reaches_git() { + // Mirrors `grep::search`'s test of the same name. `worktree_add_args` + // places the tag as a bare positional with no `--` before it, so + // `git worktree add --detach --upload-pack=` would run a + // command. `ensure` is `pub`, and this module's own doc comment used + // to say the caller had already checked - closed 2026-08-17. + let cache = tempfile::tempdir().unwrap(); + let fake = FakeGit::ok(); + let err = ensure( + &fake, + Path::new("/clone"), + cache.path(), + "--upload-pack=touch /tmp/pwned", + ) + .expect_err("a tag git would read as an option must be rejected"); + assert!(err.to_string().contains("not a usable tag name")); + assert!( + fake.seen.borrow().is_empty(), + "it must be rejected before git is called at all" + ); + } + + #[test] + fn an_existing_tree_is_reused_after_one_read_confirming_its_tag() { + // Reuse costs exactly one git call, and that call is a read. It is + // not free, but 11 ms against 77 ms to rebuild, and it is what stops + // this from trusting a directory's name over its contents. + let cache = tempfile::tempdir().unwrap(); + let path = worktree_path(cache.path(), "2.0.77"); + std::fs::create_dir_all(&path).unwrap(); + std::fs::write(path.join(".git"), b"gitdir: /elsewhere\n").unwrap(); + + let fake = FakeGit::speaking(&format!("{AT_2_0_77}\n{AT_2_0_77}\n")); + let got = ensure(&fake, Path::new("/clone"), cache.path(), "2.0.77") + .expect("a tree already at that tag is reused"); + assert_eq!(got, path); + + let seen = fake.seen.borrow(); + assert_eq!(seen.len(), 1, "reuse should cost one call, not two"); + assert!(seen[0].contains(&"rev-parse".to_string())); + // A read, never a write. `add` here would mean it rebuilt the tree. + assert!(!seen[0] + .iter() + .any(|a| a == "add" || a == "checkout" || a == "switch" || a == "reset")); + } + + #[test] + fn a_tree_whose_tag_has_moved_is_reported_rather_than_reused() { + // The case the check exists for. Silently handing back the old tree + // would make every answer drawn from it wrong, with nothing on screen + // to say so - which is the failure this whole module was built to + // remove, one level down. + let cache = tempfile::tempdir().unwrap(); + let path = worktree_path(cache.path(), "2.0.77"); + std::fs::create_dir_all(&path).unwrap(); + std::fs::write(path.join(".git"), b"gitdir: /elsewhere\n").unwrap(); + + let fake = FakeGit::speaking(&format!("{AT_2_0_77}\n{AT_2_1_14}\n")); + let err = ensure(&fake, Path::new("/clone"), cache.path(), "2.0.77") + .expect_err("a moved tag must not be reused silently"); + let message = err.to_string(); + assert!(message.contains("The tag moved"), "got: {message}"); + assert!(message.contains("--remove"), "it should say how to fix it"); + } + + #[test] + fn a_missing_tree_is_added_detached() { + let cache = tempfile::tempdir().unwrap(); + let fake = FakeGit::ok(); + let got = ensure(&fake, Path::new("/clone"), cache.path(), "2.0.77") + .expect("the fake always answers"); + assert_eq!(got, worktree_path(cache.path(), "2.0.77")); + + let seen = fake.seen.borrow(); + assert_eq!(seen.len(), 1); + assert!(seen[0].contains(&"worktree".to_string())); + assert!(seen[0].contains(&"add".to_string())); + assert!(seen[0].contains(&"--detach".to_string())); + assert!(!seen[0] + .iter() + .any(|a| a == "checkout" || a == "switch" || a == "reset")); + } + + #[test] + fn a_directory_that_exists_but_is_not_a_checkout_is_an_error() { + // An interrupted add, or a caller who made the directory by hand. + // Reusing it would hand back a tree with no files in it, which reads + // as "this tag has nothing" rather than as a broken state. + let cache = tempfile::tempdir().unwrap(); + let path = worktree_path(cache.path(), "2.0.77"); + std::fs::create_dir_all(&path).unwrap(); + + let fake = FakeGit::ok(); + let err = ensure(&fake, Path::new("/clone"), cache.path(), "2.0.77") + .expect_err("a directory with no .git is not a worktree"); + assert!(err.to_string().contains("not a git worktree")); + } + + #[test] + fn a_failing_add_reports_what_git_said() { + let cache = tempfile::tempdir().unwrap(); + let fake = FakeGit { + exit: 128, + stdout: String::new(), + stderr: "fatal: invalid reference: 9.9.9\n".into(), + seen: RefCell::new(vec![]), + }; + let err = ensure(&fake, Path::new("/clone"), cache.path(), "9.9.9") + .expect_err("128 is not success"); + assert!(err.to_string().contains("invalid reference")); + } + + #[test] + fn removing_a_tree_that_is_not_there_is_not_an_error() { + // Removing is cleanup, and cleanup that fails when there is nothing + // to clean up cannot be run twice. + let cache = tempfile::tempdir().unwrap(); + let fake = FakeGit::ok(); + remove(&fake, Path::new("/clone"), cache.path(), "2.0.77").expect("nothing to do"); + assert!(fake.seen.borrow().is_empty()); + } + + #[test] + fn a_tag_that_could_escape_the_cache_is_rejected() { + // Mirrors `docs::fetch`'s test of the same shape. `worktree_path` + // joins the tag onto the cache directory, so this is the hazard + // `valid_tag` guards for `refs worktree`, one function over - and + // `remove` is the worse of the two because it deletes. + let cache = tempfile::tempdir().unwrap(); + // The intermediate component has to exist for the OS to resolve + // `..` back onto the cache directory at all, which is what makes + // this a real escape and not just a lookup failure. + std::fs::create_dir_all(cache.path().join("worktrees")).unwrap(); + + let fake = FakeGit::ok(); + let err = remove(&fake, Path::new("/clone"), cache.path(), "..") + .expect_err("a tag that is a path component must be rejected"); + assert!(err.to_string().contains("not a usable tag name")); + assert!( + fake.seen.borrow().is_empty(), + "it must be rejected before anything is removed" + ); + } + + #[test] + fn removing_an_existing_tree_calls_git_so_the_clone_stays_tidy() { + // The admin entry lives in the shared clone. Deleting the directory + // by hand would leave it behind, which is the mess this avoids. + let cache = tempfile::tempdir().unwrap(); + let path = worktree_path(cache.path(), "2.0.77"); + std::fs::create_dir_all(&path).unwrap(); + std::fs::write(path.join(".git"), b"gitdir: /elsewhere\n").unwrap(); + + let fake = FakeGit::ok(); + remove(&fake, Path::new("/clone"), cache.path(), "2.0.77").expect("the fake answers"); + + let seen = fake.seen.borrow(); + assert_eq!(seen.len(), 1); + assert!(seen[0].contains(&"worktree".to_string())); + assert!(seen[0].contains(&"remove".to_string())); + } +} diff --git a/tests/provenance.rs b/tests/provenance.rs index 66db8d7..a4399d9 100644 --- a/tests/provenance.rs +++ b/tests/provenance.rs @@ -7,6 +7,8 @@ //! different question, it needs a binary, and it needs a human, so it lives in //! `provenance report` and never fails. +use factorio_oracle::install; +use factorio_oracle::provenance::report; use factorio_oracle::provenance::{check::check, manifest, walk_fixtures}; use std::path::{Path, PathBuf}; @@ -91,3 +93,67 @@ fn a_manifest_written_by_another_repo_agrees_with_this_check() { report.malformed ); } + +/// Runs the version comparison against a binary OLDER than this crate's own +/// fixtures, which is the only way to reach `Standing::NewerThanBinary`. +/// +/// Gated on `FACTORIO_ORACLE_OLD_FACTORIO` naming that install, following +/// the `FACTORIO_ORACLE_PROVENANCE_DIR` test above. It is not an install +/// gate: a machine can have Factorio and still not have an *old* one, and on +/// this machine the old one is deliberately outside every candidate root, so +/// discovery cannot find it and must not. +/// +/// Why it is worth having at all. Every other arm of `compare` is exercised +/// by a real run somewhere, and this one never was - the fixtures are 2.1.14 +/// and the only binary anyone could reach was 2.1.14 too. A branch that has +/// only ever run against a fake is a branch whose author's beliefs are the +/// only thing holding it up. +/// +/// Run it with: +/// FACTORIO_ORACLE_OLD_FACTORIO=installs/factorio-2.0.77.app \ +/// cargo test --test provenance -- --nocapture +#[test] +fn a_binary_older_than_the_fixtures_reports_them_as_newer() { + let Some(root) = std::env::var_os("FACTORIO_ORACLE_OLD_FACTORIO").map(PathBuf::from) else { + eprintln!( + "skipping: set FACTORIO_ORACLE_OLD_FACTORIO to an install older than \ + tests/fixtures to run this." + ); + return; + }; + + let layout = install::resolve_layout(&root) + .unwrap_or_else(|| panic!("{} does not look like an install", root.display())); + let version = install::read_version(&layout.binary) + .unwrap_or_else(|| panic!("{} would not report a version", layout.binary.display())); + + let dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures"); + let manifest = manifest::load(&dir).expect("this crate's own manifest should load"); + let report = report::compare(&manifest, &version.triple()); + + eprintln!("binary {}\n{}", version.line, report::render(&report)); + + // The point of the test. Assert the arm was reached rather than asserting + // a count, because the fixture count changes whenever a fixture is added + // and the standing does not. + assert!( + report + .groups + .iter() + .any(|g| g.standing == report::Standing::NewerThanBinary), + "an install older than the fixtures should put at least one group in \ + NewerThanBinary, got: {:?}", + report + .groups + .iter() + .map(|g| (&g.version, g.standing)) + .collect::>() + ); + + // And the half that would have caught a swapped standing string: nothing + // can be stale, because every fixture is newer than this binary. + assert_eq!( + report.stale, 0, + "no fixture can predate a binary older than all of them" + ); +} diff --git a/tests/refs.rs b/tests/refs.rs new file mode 100644 index 0000000..0c64f33 --- /dev/null +++ b/tests/refs.rs @@ -0,0 +1,215 @@ +//! The tests that run against the real factorio-data clone. +//! +//! They skip themselves when there is no clone, so CI and a fresh checkout +//! stay green. Check which happened before trusting a green run: this file +//! passing on a machine with no clone proves nothing at all. +//! +//! The first test is the one that matters. Every unit test in `src/refs/` +//! asserts on an argument vector, which proves this tool never *asks* git to +//! check anything out. Only this file can prove the clone is unchanged after +//! a real run, and that is the promise three other repos are relying on. + +use factorio_oracle::refs::{self, grep}; +// `Spawner` itself is deliberately NOT imported. Coercing `&RealSpawner` to +// `&dyn Spawner` needs no trait in scope, and CI runs clippy with +// `-D warnings`, so an unused import fails the build. +use factorio_oracle::spawn::RealSpawner; +use std::path::{Path, PathBuf}; +use std::process::Command; + +/// The clone, or `None` when there is not one. +fn find_clone() -> Option { + let home = PathBuf::from(std::env::var("HOME").unwrap_or_default()); + let env_dir = std::env::var_os("FACTORIO_DATA_DIR").map(PathBuf::from); + let dir = refs::data_clone(&home, env_dir.as_deref()); + refs::is_clone(&dir).then_some(dir) +} + +/// Runs git directly, so the assertions do not depend on the code under test. +fn git(clone: &Path, args: &[&str]) -> String { + let out = Command::new("git") + .arg("-C") + .arg(clone) + .args(args) + .output() + .expect("git should run"); + String::from_utf8_lossy(&out.stdout).trim().to_string() +} + +#[test] +fn reading_at_a_tag_leaves_the_shared_clone_exactly_as_it_was() { + let Some(clone) = find_clone() else { + eprintln!("skipping: no factorio-data clone found."); + return; + }; + + // Measured 2026-08-17: master, a784954, clean. Recorded rather than + // asserted, because this is somebody's working clone and it is allowed to + // be on any branch. What is not allowed is for it to be on a different + // one afterwards. + let head_before = git(&clone, &["rev-parse", "HEAD"]); + let branch_before = git(&clone, &["rev-parse", "--abbrev-ref", "HEAD"]); + let status_before = git(&clone, &["status", "--porcelain"]); + + let spawner = RealSpawner; + let info = grep::show(&spawner, &clone, "2.0.77", "base/info.json") + .expect("2.0.77 is a real tag and base/info.json is a real path"); + assert!( + info.contains("\"version\": \"2.0.77\""), + "the file read at the tag should be the 2.0.77 one, got: {info}" + ); + + let report = grep::search( + &spawner, + &clone, + "support_range", + &["2.0.73".to_string()], + &["elevated-rails/prototypes/entity/elevated-rails.lua".to_string()], + ) + .expect("the grep should run"); + assert!(!report.tags[0].hits.is_empty()); + + assert_eq!(head_before, git(&clone, &["rev-parse", "HEAD"])); + assert_eq!( + branch_before, + git(&clone, &["rev-parse", "--abbrev-ref", "HEAD"]), + "reading at a tag must not move HEAD in a clone three repos share" + ); + assert_eq!(status_before, git(&clone, &["status", "--porcelain"])); +} + +#[test] +fn the_cross_version_verdict_reproduces_a_claim_a_consumer_wrote_by_hand() { + let Some(clone) = find_clone() else { + eprintln!("skipping: no factorio-data clone found."); + return; + }; + for tag in ["2.0.73", "2.1.12"] { + if git(&clone, &["tag", "--list", tag]).is_empty() { + eprintln!("skipping: the clone has no {tag} tag. Run 'refs sync {tag}' first."); + return; + } + } + + // factorio-blueprint-editor's tools/oracle/probe-elevated-rail-support.mjs:776 + // says, in prose it wrote by hand: "support_range is 11 on rail-support + // and 9 on rail-ramp at both the 2.0.73 and the 2.1.12 tags". That + // sentence is copied into its fixture as a versionCaveat. This is that + // sentence, checked. + let report = grep::search( + &RealSpawner, + &clone, + "support_range", + &["2.0.73".to_string(), "2.1.12".to_string()], + &["elevated-rails/prototypes/entity/elevated-rails.lua".to_string()], + ) + .expect("the grep should run"); + + assert_eq!(report.verdict, grep::Verdict::Identical); + for result in &report.tags { + let texts: Vec<&str> = result.hits.iter().map(|h| h.text.trim()).collect(); + assert!( + texts.contains(&"support_range = 11,"), + "{} should still say 11, got {texts:?}", + result.tag + ); + assert!( + texts.contains(&"support_range = 9,"), + "{} should still say 9, got {texts:?}", + result.tag + ); + } +} + +#[test] +fn a_pattern_that_matches_nothing_is_an_answer_and_not_an_error() { + let Some(clone) = find_clone() else { + eprintln!("skipping: no factorio-data clone found."); + return; + }; + // Measured: git grep exits 1 when nothing matched. Treating that as a + // failure would make every "did this get removed" question an error. + let report = grep::search( + &RealSpawner, + &clone, + "zzz-not-a-real-token-zzz", + &["2.1.14".to_string()], + &[], + ) + .expect("no match is not a failure"); + assert!(report.empty()); + assert_eq!(report.verdict, grep::Verdict::Single); +} + +/// Removes the worktree even when an assertion panics first. +/// +/// Without this, a panic between `ensure` and `remove` leaves an entry in +/// `~/GitHub/factorio-data/.git/worktrees/` with no directory behind it - +/// mess in a clone this tool does not own, left exactly in the failure case +/// where somebody is about to go looking at that clone. `TempDir` cleans up +/// the tree on disk and knows nothing about git's own bookkeeping. +/// +/// Found by the Task 7 review on 2026-08-17: the file whose whole purpose is +/// proving the shared clone is untouched could, on failure, be the thing that +/// touched it. +struct RemoveOnDrop<'a> { + clone: &'a Path, + cache: &'a Path, + tag: &'a str, +} + +impl Drop for RemoveOnDrop<'_> { + fn drop(&mut self) { + // Best effort. A failure here must not mask the assertion that is + // already unwinding. + let _ = refs::worktree::remove(&RealSpawner, self.clone, self.cache, self.tag); + } +} + +#[test] +fn a_worktree_is_a_real_tree_at_that_tag_and_removing_it_leaves_no_trace() { + let Some(clone) = find_clone() else { + eprintln!("skipping: no factorio-data clone found."); + return; + }; + + // A temporary cache, so this never touches the developer's real one. + let cache = tempfile::tempdir().expect("a temp dir"); + let worktrees_before = git(&clone, &["worktree", "list"]); + let branch_before = git(&clone, &["rev-parse", "--abbrev-ref", "HEAD"]); + + let path = refs::worktree::ensure(&RealSpawner, &clone, cache.path(), "2.0.77") + .expect("2.0.77 is a real tag"); + // Declared after `cache`, so it drops before it: the temporary directory + // is still there when the guard runs. + let _cleanup = RemoveOnDrop { + clone: &clone, + cache: cache.path(), + tag: "2.0.77", + }; + let info = std::fs::read_to_string(path.join("base/info.json")) + .expect("a worktree should hold real files"); + assert!(info.contains("\"version\": \"2.0.77\"")); + + // The main tree stayed where it was, which is the whole difference + // between adding a worktree and checking a tag out. + assert_eq!( + branch_before, + git(&clone, &["rev-parse", "--abbrev-ref", "HEAD"]), + "adding a worktree must not move the main tree's HEAD" + ); + + // The explicit remove is what the test is asserting about. The guard + // above then runs a second time at end of scope and finds nothing to do, + // because `remove` returns Ok when the path is already gone. + refs::worktree::remove(&RealSpawner, &clone, cache.path(), "2.0.77") + .expect("removing should work"); + + // The admin entry lives in a clone this tool does not own, so leaving one + // behind is leaving mess in somebody else's directory. + assert_eq!( + worktrees_before, + git(&clone, &["worktree", "list"]), + "the shared clone's worktree list should be back to what it was" + ); +}