Skip to content

Release blockers: first-run experience, and two false-decay bugs - #53

Merged
junixlabs merged 6 commits into
mainfrom
release-blockers
Sep 11, 2026
Merged

junixlabs merged 6 commits into
mainfrom
release-blockers

Conversation

@junixlabs

Copy link
Copy Markdown
Owner

Everything between v0.2 and a release I would defend.

1. The first outside run produced 22 errors from one misunderstanding (#41)

ids use hyphens, keys use underscores, and the validator printed the raw regex. Fixed with
a sentence, not a rule change:

id: 'flow.my_flow' is not a valid id. ids use hyphens and keys use underscores:
`flow.my-flow`, not `flow.my_flow`, beside `coverage_declared` and `transitions_to`.

2. Five assertions on one subject produced ten errors and an INVALID model (#40)

The check flagged every pair whose claim strings differed. Nothing reads a claim, so
"different string" was standing in for "incompatible proposition". Two findings now,
because two things were conflated:

Finding Level Condition
CONTRADICTION error same subject, overlapping selectors, and the latest observations disagree — one supported, one refuted
SUBJECT_NOT_DISCRIMINATING warn same subject, overlapping selectors, different wording — once per subject

Measured on the reported case: 10 errors and INVALID → 1 warning and VALID. A test
proves the genuine contradiction stays an error, and its positive control fails if the
error is disabled.

3. #5 measured, and the answer surprised me

graph entities median reached by one change worst
synthetic, 10 flows 61 15% 20%
synthetic, 50 flows, fanout 4 301 9% 15%
synthetic, 200 flows, fanout 6 1,201 5% 7%
examples/ecommerce 32 61% 66%
examples/traceos-itself 45 56% 89%

Tiers do not collapse as the graph grows — the hypothesis is refuted at scale. They
collapse on small dense models, which is what a model looks like in its first week. So
impact() reports share_reached and prints DID NOT DISCRIMINATE above 50% — INV-026's
shape one level down, and a fact about the model rather than a finding about the change.

Two corrections to my own measurement before those numbers were trustworthy: a denominator
that counted only flows and nodes while the tiers also return states and externals
(producing an impossible 103%), and a summary that added medians across tiers.

4. #10 measured — the number DIRECTION.md said governs everything

Unit: a re-observation, a reference a change invalidated that someone must open and read.

n median mean max
before symbol-granular decay 20 changes 11.5 15.0 80
after symbol-granular decay 7 changes 3.0 4.6 14

A change used to force re-reading ~88% of the model's references; it now forces ~19%.
Caveat stated in the issue and the doc: n=7, one repository, one author who wrote both the
model and the code.

5. Two false-decay bugs in the extractor I shipped yesterday

  • git log -L reports the range at the last commit that touched the symbol, not at
    HEAD.
    A 111-line insertion moved validate_intents from 717 to 747 and the hash was
    taken over the wrong 68 lines. Now git blame -L :<symbol> <file>, which answers at HEAD.
  • git's funcname block includes the blank lines after a symbol, so adding a function
    after one read as an edit to it. Trailing blanks are dropped before hashing, the same
    rule observed_norm already applies.

Both were false positives in the safe direction, and both were still wrong — a checker that
cries wolf gets discharged by rubber stamp. Sixteen references were re-observed because the
corrected extractor invalidated hashes computed by the buggy one; that is a real re-read,
not a backfill.

Checks

41/41 groups, 196/196 assertions. Every new behaviour falsified: the contradiction
error disabled, the per-pair finding restored, the discrimination flag pinned to a
constant, trailing blanks kept, and git log -L put back — each fails the suite.
check_docs 28/28, check_locators 47/47, ratchet exit 0.

@junixlabs
junixlabs merged commit f95f848 into main Sep 11, 2026
9 checks passed
@junixlabs
junixlabs deleted the release-blockers branch September 11, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant