Repository navigation
Release blockers: first-run experience, and two false-decay bugs - #53
Merged
Merged
Conversation
This was referenced Sep 11, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything between v0.2 and a release I would defend.
1. The first outside run produced 22 errors from one misunderstanding (#41)
ids use hyphens, keys use underscores, and the validator printed the raw regex. Fixed with
a sentence, not a rule change:
2. Five assertions on one subject produced ten errors and an INVALID model (#40)
The check flagged every pair whose claim strings differed. Nothing reads a claim, so
"different string" was standing in for "incompatible proposition". Two findings now,
because two things were conflated:
CONTRADICTIONSUBJECT_NOT_DISCRIMINATINGMeasured on the reported case: 10 errors and INVALID → 1 warning and VALID. A test
proves the genuine contradiction stays an error, and its positive control fails if the
error is disabled.
3. #5 measured, and the answer surprised me
examples/ecommerceexamples/traceos-itselfTiers do not collapse as the graph grows — the hypothesis is refuted at scale. They
collapse on small dense models, which is what a model looks like in its first week. So
impact()reportsshare_reachedand prints DID NOT DISCRIMINATE above 50% — INV-026'sshape one level down, and a fact about the model rather than a finding about the change.
Two corrections to my own measurement before those numbers were trustworthy: a denominator
that counted only flows and nodes while the tiers also return states and externals
(producing an impossible 103%), and a summary that added medians across tiers.
4. #10 measured — the number
DIRECTION.mdsaid governs everythingUnit: a re-observation, a reference a change invalidated that someone must open and read.
A change used to force re-reading ~88% of the model's references; it now forces ~19%.
Caveat stated in the issue and the doc: n=7, one repository, one author who wrote both the
model and the code.
5. Two false-decay bugs in the extractor I shipped yesterday
git log -Lreports the range at the last commit that touched the symbol, not atHEAD. A 111-line insertion moved
validate_intentsfrom 717 to 747 and the hash wastaken over the wrong 68 lines. Now
git blame -L :<symbol> <file>, which answers at HEAD.after one read as an edit to it. Trailing blanks are dropped before hashing, the same
rule
observed_normalready applies.Both were false positives in the safe direction, and both were still wrong — a checker that
cries wolf gets discharged by rubber stamp. Sixteen references were re-observed because the
corrected extractor invalidated hashes computed by the buggy one; that is a real re-read,
not a backfill.
Checks
41/41 groups, 196/196 assertions. Every new behaviour falsified: the contradiction
error disabled, the per-pair finding restored, the discrimination flag pinned to a
constant, trailing blanks kept, and
git log -Lput back — each fails the suite.check_docs28/28,check_locators47/47, ratchet exit 0.