Say what a create run's failure actually was, per the first consumer - #12
Merged
Conversation
The first probe written against this tool by a consumer wrote its result as `basis-gradient-probe.json`. The run reported "no dump was written. The most common cause is a factorio_version mismatch, which makes Factorio skip the mod in silence." That named a cause the run had already ruled out: the probe raised `DUMPED-OK`, and a mod skipped over a version mismatch never runs, so it cannot raise the sentinel. The 270 KB it had just written was listed in the report's own `files` array the whole time. So `create`'s failure message now splits on the sentinel. Without one, the silent skip really is the first thing to check and the message says so. With one, the message says the mismatch is excluded and points at the filename instead. Both arms now name `oracle-dump.json` rather than saying "no dump", because that name is this tool's contract rather than the game's and it was documented nowhere. Both tests were mutation-checked: collapsing the two arms back to the one old message fails both. The three documents get what else that probe cost, since they exist so a fifth repo does not pay it again: - gotchas: the filename contract, and that `basis_noise`'s gradient table is a constant of the engine rather than of a version - it recovers byte-identically from 2.0.77 and 2.1.14, and scores 512 of 512 against a fixture captured on 2.1.11. - method: a wrong inversion returns a full set of plausible numbers. The one-corner form the consumer's notes describe recovers 2 of 256 slots and raises nothing. Plus scoring against what the method never read, pricing what it cannot see, and a concrete case of a control that was malformed rather than failing. - README: the output filename, where a probe author will look for it. CLAUDE.md's test count was 247 and is 255. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Qg3gYjaG2uEtvv5U6AjY6
Two corrections and one addition, all from checking the claim against FactorioMapWebUI's own PROVENANCE.json rather than from memory. The fixture the recovered table scores 512/512 against was first captured on 2.1.11 and re-verified bit-for-bit on 2.1.12, so the agreement spans four versions rather than three: 2.0.77, 2.1.11, 2.1.12 and 2.1.14. And the outcome is worth stating plainly next to the tool's own measured facts. The first real use of this tool by a consumer took three of that repo's fixtures from 473/512, 36/38 and 231/266 bit-exact to 512/512, 38/38 and 266/266, all at worst error 0, with no change to its kernel. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Qg3gYjaG2uEtvv5U6AjY6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first probe a consumer wrote against this tool found a real defect in it, and
cost four lessons the three documents exist to carry. This is both.
The defect
A
createrun that wrote its dump under a different name reported:That names a cause the run had already ruled out. The probe raised
DUMPED-OK,and a mod skipped over a version mismatch never runs, so it cannot raise the
sentinel. The 270 KB it had just written was listed in the report's own
filesarray the whole time. The message sent a reader to
info.jsonfor a mod thatdemonstrably loaded.
So the message now splits on
sentinel_seen, whichRunFactsalready carriedand the
Createarm ignored:check, and the message still says so.
filename instead.
Both arms now name
oracle-dump.jsonrather than saying "no dump". That name isthis tool's contract rather than the game's, and it was documented nowhere -
not in the README, not in
docs/, only in a constant inrun.rs.Both new tests were mutation-checked: collapsing the two arms back to the one old
message fails both. One existing test in
run.rsassertedcontains("no dump")and now asserts the two properties that distinguish the arms.
The docs
docs/gotchas.md- the filename contract, and thatbasis_noise'sgradient table is a constant of the engine rather than of a version. It
recovers byte-identically from 2.0.77 and 2.1.14, and scores 512 of 512 exact
against a fixture first captured on 2.1.11 and re-verified on 2.1.12.
docs/method.md- a wrong inversion returns a full set of plausiblenumbers. The one-corner form the consumer's own notes describe recovers 2 of
256 slots, raises nothing, and returns 512 values of plausible size. Plus
scoring against what the method never read, pricing a limit the method cannot
see, and a concrete case of a control that was malformed rather than failing.
README.md- the output filename, where a probe author will look for it.CLAUDE.md- the test count was 247 and is 255, and the measured factsnow record what that probe settled.
Verification
Clean. 255 unit tests and 13 integration tests. All four gates were satisfied
on the machine that ran them, including
FACTORIO_ORACLE_PROVENANCE_DIRandFACTORIO_ORACLE_OLD_FACTORIO, so none of the 13 skipped.🤖 Generated with Claude Code
https://claude.ai/code/session_016Qg3gYjaG2uEtvv5U6AjY6