diff --git a/docs/DIRECTION.md b/docs/DIRECTION.md index 16035c1..5edc304 100644 --- a/docs/DIRECTION.md +++ b/docs/DIRECTION.md @@ -63,8 +63,9 @@ What the model declared does not quietly stop resolving. Symbol-granular decay v what it excluded (INV-025). *Claims:* the model does not rot silently. -*Kills it:* erosion fires often enough on a healthy repository to be tuned out. Unmeasured -— #10 is the number, and no horizon past this one is worth starting before it exists. +*Kills it:* erosion fires often enough on a healthy repository to be tuned out. #10 is now +measured — a median of 3 re-reads per change, down from 12 — so this horizon stands and the +ones after it are worth starting. ### H2 — Outcome carries evidence — *mechanism built, value unproven* @@ -163,9 +164,24 @@ comes: ## The number that governs all of it -**#10 — nobody has measured what it costs to keep a model current.** A peer's trial gave -a fragment: four `validate` rounds, five assertions, two files of fifty-six. Nobody has -measured the weekly cost of keeping that from rotting. +**#10 — what it costs to keep a model current. Measured, and it is not prohibitive.** + +The unit is a *re-observation*: a reference a change invalidated, which someone opens, +reads and confirms. First-time observations are excluded — that is the model growing, not +the tax of keeping it. + +| | n | median | mean | max | +|---|---|---|---|---| +| before symbol-granular decay | 20 changes | **11.5** | 15.0 | 80 | +| after symbol-granular decay | 7 changes | **3.0** | 4.6 | 14 | + +A change used to force re-reading roughly 88% of the model's references; it now forces +about 19%. The three worst figures all predate the narrowing work and were the gate +generating its own rubber stamps. + +**What this does not establish:** n=7, one repository, 16 assertions, one author who wrote +both the model and the code. The cost on a model somebody else maintains is larger and is +not knowable from here. If that number is small, every horizon above is worth building. If it is large, none of them matter, because the model will be abandoned before it is ever contradicted by an diff --git a/docs/semantic-specification.md b/docs/semantic-specification.md index e463190..ebc3704 100644 --- a/docs/semantic-specification.md +++ b/docs/semantic-specification.md @@ -515,6 +515,26 @@ behavior and is valid. ## 8. Lifecycle and Effective Reality + +#### Wording is not a contradiction + +Nothing in this system reads a claim (§6.1), so "the strings differ" cannot mean "the +propositions conflict". Treating it that way produced ten errors and an INVALID model +from five assertions on one subject on the first outside run of this tool, with nothing +actually in conflict. + +Two findings, because two different things were being conflated: + +| Finding | Level | Condition | +|---|---|---| +| `CONTRADICTION` | error | same subject, overlapping selectors, and the **latest observations disagree** — one supported, one refuted | +| `SUBJECT_NOT_DISCRIMINATING` | warn | same subject, overlapping selectors, different wording — reported **once per subject** | + +The first is decidable and is a real conflict a person must resolve, so it is reported +per pair. The second is a modelling smell: either the assertions describe different +aspects and want different subjects, or they compete and want `when` selectors. The +count of pairs was never information, only arithmetic. + ### INV-016 CURRENT-ONLY-RESOLVES ``` @@ -758,6 +778,29 @@ carry all four tiers, including an empty `unknown`. ## 13. Coverage +#### An impact result that reaches most of the model says nothing + +The tiers are useful only while they stay different sizes. Measured (#5): + +| graph | entities | median reached by one change | worst | +|---|---|---|---| +| synthetic, 10 flows | 61 | 15% | 20% | +| synthetic, 50 flows, fanout 4 | 301 | 9% | 15% | +| synthetic, 200 flows, fanout 6 | 1,201 | 5% | 7% | +| `examples/ecommerce` | 32 | **61%** | 66% | +| `examples/traceos-itself` | 45 | **56%** | 89% | + +**The tier system does not collapse with scale** — that hypothesis is refuted; as the +graph grows a change reaches proportionally *less* of it, and `certain` stays at two +entities while `inspect` absorbs the distance. Where it collapses is the opposite end: +small models in which every flow invokes another, which is what a model looks like in its +first week. + +So the result carries `share_reached` and `discriminates`, and a result above 50% is +reported as **DID NOT DISCRIMINATE** rather than as an answer — the shape of INV-026 one +level down. It is a fact about the model, not a finding about the change, and saying so +prevents the worst reading: that a change genuinely touches two thirds of a system. + ### INV-011 UNCERTAIN-NE-UNKNOWN | | Meaning | Detectable by | @@ -1072,6 +1115,19 @@ the moment it was read. The line range is derived from git's own hunk header at time and never stored, so INV-022 is untouched; what is stored is a hash of what the observer actually read, which is what an observation is for. +The range is resolved with `git blame -L : `, which answers **at HEAD**. +`git log -L` was tried first and is wrong for this: its hunk header reports the range as +it stood at the last commit that touched the symbol, so any later commit editing the file +above it shifts the answer — measured here, a 111-line insertion moved a function from +line 717 to 747 and the hash was taken over the wrong 68 lines. + +Trailing blank lines are dropped before hashing, for the same reason `observed_norm` +drops them: git's funcname block includes the blank lines that follow a symbol, so +inserting a new function *after* one otherwise reads as an edit *to* it. + +Both were false positives in the safe direction. Both were still wrong, and a checker that +cries wolf is discharged by rubber stamp (§13.0.1). + Order of answers, strongest first: | | Needs history | Answers | diff --git a/examples/traceos-itself/observations/2026-09.jsonl b/examples/traceos-itself/observations/2026-09.jsonl index d206117..60599dc 100644 --- a/examples/traceos-itself/observations/2026-09.jsonl +++ b/examples/traceos-itself/observations/2026-09.jsonl @@ -336,3 +336,19 @@ {"assertion": "assert.account.refuted-is-an-error", "reference": "tests/run_tests.py#inv_outcome_can_be_refuted", "observed_at": "2026-09-11T18:46:04.738311Z", "observed_ref": "f26f2ce06224ef1e9b123037e12cae65cad7faeb", "observed_blob": "a606ebde9c7623449f35cf98c6cfce3aa0252d17", "observed_norm": "dfce6a969090055d37d4a2d9ff0e571705741370", "supports": "supports", "observer": "agent", "kind": "test", "observed_symbol": "eb9f59f8d40526d33cfdc19a57c3aeceda4a42ef"} {"assertion": "assert.account.refuted-is-an-error", "reference": "docs/semantic-specification.md#6.4 An Outcome declares how it is checked \u2014 INV-027", "observed_at": "2026-09-11T18:46:04.889538Z", "observed_ref": "f26f2ce06224ef1e9b123037e12cae65cad7faeb", "observed_blob": "d2dbcc769ff00277b490d96ee57815ca592274ec", "observed_norm": "507d522809b19fdb877b1e1eaa57d214797f7871", "supports": "supports", "observer": "agent", "kind": "documentation", "observed_symbol": "b94b214f0662c0ac504dbd623c537b5eefd476b9"} {"assertion": "assert.account.refuted-is-an-error", "reference": "docs/semantic-specification.md#6.4 An Outcome declares how it is checked \u2014 INV-027", "observed_at": "2026-09-11T18:46:49.687941Z", "observed_ref": "9ecbb96f77e658ab42510fbc51a5ed9d6cf5c84e", "observed_blob": "e4631904a0714750114cc07fd176754330b9a058", "observed_norm": "a89789dc84043df5ee8fe5fd4a0dcce274f354fe", "supports": "supports", "observer": "agent", "kind": "documentation", "observed_symbol": "6cdaa20b4be30d3825f2cd51b6b6263948da3988"} +{"assertion": "assert.account.intent-is-provenance", "reference": "traceos/engine.py#validate_intents", "observed_at": "2026-09-11T19:33:47.753376Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "5214a57c1b1270220d65aca990a162f986b7dc45", "observed_norm": "84b72b9bbc2d810dae9855d50112212a8952b370", "supports": "supports", "observer": "agent", "kind": "implementation", "observed_symbol": "f8cef002fe697744492e6c2bc32ce6d6a9228ade"} +{"assertion": "assert.account.outcome-declares-its-check", "reference": "traceos/engine.py#validate_outcome_verification", "observed_at": "2026-09-11T19:33:47.890274Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "5214a57c1b1270220d65aca990a162f986b7dc45", "observed_norm": "84b72b9bbc2d810dae9855d50112212a8952b370", "supports": "supports", "observer": "agent", "kind": "implementation", "observed_symbol": "f3163972e97d9264f1a62296038b9471188bf28e"} +{"assertion": "assert.account.outcome-declares-its-check", "reference": "tests/run_tests.py#inv_outcome_can_be_refuted", "observed_at": "2026-09-11T19:33:48.039614Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "7f3011345598a6d0c995a9fb4953d59cb99f58e4", "observed_norm": "15c3438e2017d07c21c28ae7d9f6ad35c31652b9", "supports": "supports", "observer": "agent", "kind": "test", "observed_symbol": "7b0b6b84da830e6dc8b2fc8d9ea8f474ff987051"} +{"assertion": "assert.account.refuted-is-an-error", "reference": "traceos/engine.py#validate_outcome_verification", "observed_at": "2026-09-11T19:33:48.174059Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "5214a57c1b1270220d65aca990a162f986b7dc45", "observed_norm": "84b72b9bbc2d810dae9855d50112212a8952b370", "supports": "supports", "observer": "agent", "kind": "implementation", "observed_symbol": "f3163972e97d9264f1a62296038b9471188bf28e"} +{"assertion": "assert.account.refuted-is-an-error", "reference": "tests/run_tests.py#inv_outcome_can_be_refuted", "observed_at": "2026-09-11T19:33:48.315693Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "7f3011345598a6d0c995a9fb4953d59cb99f58e4", "observed_norm": "15c3438e2017d07c21c28ae7d9f6ad35c31652b9", "supports": "supports", "observer": "agent", "kind": "test", "observed_symbol": "7b0b6b84da830e6dc8b2fc8d9ea8f474ff987051"} +{"assertion": "assert.account.refuted-is-an-error", "reference": "docs/semantic-specification.md#6.4 An Outcome declares how it is checked \u2014 INV-027", "observed_at": "2026-09-11T19:33:48.456838Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "ebc370482855c3380efcf099ae0cd7e9a6d189fd", "observed_norm": "9b7f9882dd0d7ebaacbf3fc51955b2b18726c7dc", "supports": "supports", "observer": "agent", "kind": "documentation", "observed_symbol": "4dcf2fcd77fe80e4e09b651bfb8c4cd3609fe76e"} +{"assertion": "assert.implement.not-ours", "reference": "docs/semantic-specification.md#Non-goals", "observed_at": "2026-09-11T19:33:48.602083Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "ebc370482855c3380efcf099ae0cd7e9a6d189fd", "observed_norm": "9b7f9882dd0d7ebaacbf3fc51955b2b18726c7dc", "supports": "supports", "observer": "agent", "kind": "documentation", "observed_symbol": "e309627465118b875616e98df6463b5cd49dd2bd"} +{"assertion": "assert.reconcile.confidence-decays", "reference": "traceos/engine.py#computed_confidence", "observed_at": "2026-09-11T19:33:48.764900Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "5214a57c1b1270220d65aca990a162f986b7dc45", "observed_norm": "84b72b9bbc2d810dae9855d50112212a8952b370", "supports": "supports", "observer": "agent", "kind": "implementation", "observed_symbol": "e7d38f35bddf5544e0165922bcb37492793f120a"} +{"assertion": "assert.reconcile.unverifiable-is-not-verified", "reference": "traceos/engine.py#Git", "observed_at": "2026-09-11T19:33:48.912315Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "5214a57c1b1270220d65aca990a162f986b7dc45", "observed_norm": "84b72b9bbc2d810dae9855d50112212a8952b370", "supports": "supports", "observer": "agent", "kind": "implementation", "observed_symbol": "a60bd89e4305913604e40eff71b6418b54ec7ed5"} +{"assertion": "assert.render.no-second-implementation", "reference": "tests/run_tests.py#tool_explore", "observed_at": "2026-09-11T19:33:49.061765Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "7f3011345598a6d0c995a9fb4953d59cb99f58e4", "observed_norm": "15c3438e2017d07c21c28ae7d9f6ad35c31652b9", "supports": "supports", "observer": "agent", "kind": "test", "observed_symbol": "45025670400c01277270393f7a67c6300a7b4ab0"} +{"assertion": "assert.trace.four-tiers", "reference": "traceos/engine.py#impact", "observed_at": "2026-09-11T19:33:49.204495Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "5214a57c1b1270220d65aca990a162f986b7dc45", "observed_norm": "84b72b9bbc2d810dae9855d50112212a8952b370", "supports": "supports", "observer": "agent", "kind": "implementation", "observed_symbol": "65f836e5f398caa147047b90156eeeec8322eecc"} +{"assertion": "assert.trace.four-tiers", "reference": "tests/run_tests.py#inv_impact_ne_changed_files", "observed_at": "2026-09-11T19:33:49.354436Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "7f3011345598a6d0c995a9fb4953d59cb99f58e4", "observed_norm": "15c3438e2017d07c21c28ae7d9f6ad35c31652b9", "supports": "supports", "observer": "agent", "kind": "test", "observed_symbol": "b0dedb1f3fdb3dd7609f6a8ac2faf85c2d8e2184"} +{"assertion": "assert.trace.unknown-is-gated", "reference": "traceos/cli.py", "observed_at": "2026-09-11T19:33:49.501471Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "bb69674159ca6e6cdbccbe1013b6bfe4ffafc721", "observed_norm": "9749889510ef4ddcc4768204007cac03a02de95c", "supports": "supports", "observer": "agent", "kind": "implementation"} +{"assertion": "assert.trace.unknown-is-not-safe", "reference": "traceos/engine.py#impact", "observed_at": "2026-09-11T19:33:49.643433Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "5214a57c1b1270220d65aca990a162f986b7dc45", "observed_norm": "84b72b9bbc2d810dae9855d50112212a8952b370", "supports": "supports", "observer": "agent", "kind": "implementation", "observed_symbol": "65f836e5f398caa147047b90156eeeec8322eecc"} +{"assertion": "assert.trace.unknown-is-not-safe", "reference": "tests/run_tests.py#inv_impact_ne_changed_files", "observed_at": "2026-09-11T19:33:49.797159Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "7f3011345598a6d0c995a9fb4953d59cb99f58e4", "observed_norm": "15c3438e2017d07c21c28ae7d9f6ad35c31652b9", "supports": "supports", "observer": "agent", "kind": "test", "observed_symbol": "b0dedb1f3fdb3dd7609f6a8ac2faf85c2d8e2184"} +{"assertion": "assert.understand.observes-before-asserting", "reference": "traceos/engine.py#observe", "observed_at": "2026-09-11T19:33:49.946099Z", "observed_ref": "f77c48ef7d7e3d66a4ad6335652d4708d772cb4e", "observed_blob": "5214a57c1b1270220d65aca990a162f986b7dc45", "observed_norm": "84b72b9bbc2d810dae9855d50112212a8952b370", "supports": "supports", "observer": "agent", "kind": "implementation", "observed_symbol": "81e37c012d1ed637d3c9b13477cdd384b6da63d8"} diff --git a/tests/run_tests.py b/tests/run_tests.py index a606ebd..7f30113 100644 --- a/tests/run_tests.py +++ b/tests/run_tests.py @@ -815,7 +815,14 @@ def inv_impact_ne_changed_files(): check( "INV Impact != Files", "all four tiers are always present", - set(imp) == {"certain", "likely", "inspect", "unknown"}, + {"certain", "likely", "inspect", "unknown"} <= set(imp), + str(sorted(imp)), + ) + check( + "INV Impact != Files", + "and the result says how much of the model it reached (#5)", + {"entities", "reached", "share_reached", "discriminates"} <= set(imp), + str(sorted(imp)), ) check( "INV Impact != Files", @@ -1301,6 +1308,7 @@ def run(*a): # being rewritten, because a squash merge does exactly that. Measured on this # repository, one squash orphaned 7 of 25 observed refs. sym_before = git.symbol_blob("mod.py", "alpha") + sym_beta = git.symbol_blob("mod.py", "beta") run("checkout", "-q", "--orphan", "rewritten") run("add", "-A") run("commit", "-qm", "history rewritten, every old commit orphaned") @@ -1317,6 +1325,24 @@ def run(*a): "the old ref is now unreachable, as after a squash merge", git.reachable(head) is False, ) + # git's funcname block includes the blank lines after a symbol, so inserting a + # new function *after* this one read as an edit *to* it until trailing blanks + # were dropped. Safe-direction noise, measured on this repository's own suite. + src.write_text(src.read_text() + "\n\ndef gamma():\n return 3\n") + run("add", "mod.py") + run("commit", "-qm", "a new function after alpha and beta") + check( + "TOOL symbol decay", + "adding a function after a symbol is not an edit to it", + git.symbol_blob("mod.py", "beta") == sym_beta, + f"{git.symbol_blob('mod.py', 'beta')} vs {sym_beta}", + ) + check( + "TOOL symbol decay", + "and the range still resolves at HEAD, not at the last commit", + git.symbol_range("mod.py", "gamma")[0] is not None, + str(git.symbol_range("mod.py", "gamma")), + ) check( "TOOL symbol decay", "a content hash still narrows when the ref is gone", @@ -2446,6 +2472,138 @@ def inv_outcome_can_be_refuted(): ) +def inv_contradiction_is_evidence_not_wording(): + """INV-008. A contradiction between two sentences is not mechanically decidable. + + The original check called every pair of differently-worded assertions on one subject + a contradiction. Measured on an outsider's first run: five assertions on one subject + produced ten errors and an INVALID model, with nothing actually in conflict. Nothing + in this system reads a claim, so "different string" was standing in for "incompatible + proposition". + """ + flow = { + "id": "flow.x", + "type": "flow", + "domain": "d", + "lifecycle": "current", + "coverage_declared": "stub", + "trigger": {"kind": "user_action", "actor": "external.someone"}, + "nodes": [{"id": "node.x.s", "type": "action", "name": "S"}], + "assertions": [ + { + "id": f"assert.x.a{i}", + "claim": f"aspect {i} of the thing", + "subject": "x.thing", + "lifecycle": "current", + "about": ["node.x.s"], + "evidence": [{"kind": "implementation", "locator": f"src/a{i}.ts#f"}], + } + for i in range(5) + ], + } + externals = { + "id": "system.x.externals", + "type": "externals", + "externals": [{"id": "external.someone", "name": "Someone"}], + } + + model = scratch_model({"f": flow, "e": externals}) + findings = T.validate(model, None, {}, NOW) + smell = [f for f in findings if f.code == "SUBJECT_NOT_DISCRIMINATING"] + check( + "INV-008 contradiction", + "five assertions on one subject raise one finding, not ten", + len(smell) == 1, + str(len(smell)), + ) + check( + "INV-008 contradiction", + "wording alone is not an error, and does not make the model INVALID", + not [f for f in findings if f.level == "error"] + and T.integrity(findings, {}, None) != "INVALID", + str(sorted(codes(findings))), + ) + + # Positive control. Evidence that disagrees IS a contradiction, and must stay an + # error - otherwise this fix has traded a false alarm for a silence. + model = scratch_model({"f": flow, "e": externals}) + model.observations.append( + { + "assertion": "assert.x.a0", + "reference": "src/a0.ts#f", + "observed_at": stamp(1), + "supports": "supports", + "observer": "agent", + } + ) + model.observations.append( + { + "assertion": "assert.x.a1", + "reference": "src/a1.ts#f", + "observed_at": stamp(1), + "supports": "refutes", + "observer": "agent", + } + ) + findings = T.validate(model, None, {}, NOW) + real = [f for f in findings if f.code == "CONTRADICTION"] + check( + "INV-008 contradiction", + "evidence pointing opposite ways on one subject is still an error", + len(real) == 1 and real[0].level == "error", + str([f.message for f in real]), + ) + check( + "INV-008 contradiction", + "and it names which side is supported and which refuted", + "assert.x.a0 is supported" in real[0].message + and "assert.x.a1 is refuted" in real[0].message, + real[0].message if real else "", + ) + + +def inv_impact_says_when_it_cannot_discriminate(): + """#5. Four tiers are useful only while they stay different sizes. + + Measured across synthetic graphs of 61 to 1,201 entities: a single change reaches a + median 4-15% and the tiers stay far apart, so the tier system does not collapse with + scale - the hypothesis in #5 is refuted there. On the two reference models, 32 and 45 + entities with every flow invoking another, the median is 56-61% and the worst case + 89%. Small dense models are where it says nothing, and saying so is the answer. + """ + model = T.Model(EXAMPLE) + result = T.impact(model, ["src/payment/PaymentProcessor.ts#process"]) + check( + "INV-019 discrimination", + "impact reports how much of the model it reached", + result["entities"] > 0 and 0 <= result["share_reached"] <= 100, + str((result["reached"], result["entities"], result["share_reached"])), + ) + check( + "INV-019 discrimination", + "a change reaching most of a small dense model is flagged as not discriminating", + result["share_reached"] > 50 and result["discriminates"] is False, + str(result["share_reached"]), + ) + + # Positive control: something must still discriminate, or the flag is a constant. + itself = T.impact( + T.Model(ROOT / "examples" / "traceos-itself"), ["traceos/site.py#main"] + ) + check( + "INV-019 discrimination", + "a narrow change on a larger model does discriminate", + itself["discriminates"] is True and itself["share_reached"] <= 50, + str(itself["share_reached"]), + ) + check( + "INV-019 discrimination", + "reached never exceeds the entity count, whatever the tiers overlap", + result["reached"] <= result["entities"], + str((result["reached"], result["entities"])), + ) + + INVARIANTS = [ inv_reality_ne_code, inv_code_change_ne_behavior_change, @@ -2456,6 +2614,8 @@ def inv_outcome_can_be_refuted(): inv_evidence_ne_assertion, inv_confidence_per_reference, inv_outcome_can_be_refuted, + inv_contradiction_is_evidence_not_wording, + inv_impact_says_when_it_cannot_discriminate, ] diff --git a/traceos/cli.py b/traceos/cli.py index 7a2c456..bb69674 100644 --- a/traceos/cli.py +++ b/traceos/cli.py @@ -340,6 +340,17 @@ def main() -> int: print(f"{tier}:") for item in result[tier]: print(f" {item}") + print( + f"\nreached {result['reached']} of {result['entities']} modelled " + f"entities ({result['share_reached']}%)" + ) + if not result["discriminates"]: + print( + " DID NOT DISCRIMINATE this change reaches most of the model, so " + "the tiers\n are not telling you where to look. Usually the model " + "is small or every\n flow invokes another; it is not a finding " + "about the change." + ) return 0 if args.cmd == "ratchet": diff --git a/traceos/engine.py b/traceos/engine.py index a870a87..5214a57 100644 --- a/traceos/engine.py +++ b/traceos/engine.py @@ -389,41 +389,52 @@ def reachable(self, ref: str) -> bool: return False return self._run("merge-base", "--is-ancestor", ref, "HEAD") is not None - def symbol_blob(self, path: str, anchor: str) -> str | None: - """Content hash of the named symbol as it stands now. No history required. + def symbol_range(self, path: str, anchor: str) -> tuple[int | None, int | None]: + """The symbol's line range **at HEAD**, from git's own funcname heuristic. - `git log -L :symbol:path -1` reports the most recent commit that touched the - symbol, and its hunk header carries the symbol's line range *at HEAD*. The - range is derived here and never stored, so INV-022 is untouched - what is - stored is a hash of the text, which is what an observation is entitled to - record about an artifact it just read. + `git blame -L : ` resolves the block against the current file. + `git log -L` was tried first and was wrong: its hunk header reports the range as + it stood at the last commit that touched the symbol, so a later commit editing + the file above it shifts the answer. Measured here - a 111-line insertion moved + `validate_intents` from line 717 to 747, and the hash was taken over the wrong + 68 lines. A false decay: the safe direction, and still wrong. - This is what makes an observation survive a squash merge. Asking - `..HEAD` needs the branch's commits to still exist, and a squash - replaces them: measured here, one merge orphaned 7 of 25 observed refs, and - the next merge orphaned more. A content hash has no such dependency. + Derived at check time and never stored, so INV-022 is untouched. """ prefix = [] attributes = self._attributes_file() if attributes: prefix = ["-c", f"core.attributesfile={attributes}"] - out = self._run(*prefix, "log", "-L", f":{anchor}:{path}", "-1", "--format=%x00") + out = self._run( + *prefix, "blame", "-L", f":{re.escape(anchor)}", "--", _norm_path(path) + ) if not out: - return None - lo = hi = None + return None, None + numbers = [] for line in out.splitlines(): - if line.startswith("@@"): - try: - plus = line.split("+", 1)[1].split(" ", 1)[0] - except IndexError: - continue - start, _, count = plus.partition(",") - try: - lo = int(start) - hi = lo + (int(count) if count else 1) - 1 - except ValueError: - return None - if lo is None: + head, sep, _rest = line.partition(")") + if not sep: + continue + parts = head.rsplit(" ", 1) + if len(parts) == 2 and parts[1].isdigit(): + numbers.append(int(parts[1])) + if not numbers: + return None, None + return min(numbers), max(numbers) + + def symbol_blob(self, path: str, anchor: str) -> str | None: + """Content hash of the named symbol as it stands now. No history required. + + The range comes from git at check time; what is stored is a hash of the text, + which is what an observation is entitled to record about an artifact it read. + + This is what makes an observation survive a squash merge. Asking + `..HEAD` needs the branch's commits to still exist, and a squash + replaces them: measured here, one merge orphaned 7 of 25 observed refs, and the + next merge orphaned more. A content hash has no such dependency. + """ + lo, hi = self.symbol_range(path, anchor) + if lo is None or hi is None: return None target = (self.repo / _norm_path(path)) if self.repo else None if target is None or not target.is_file(): @@ -432,7 +443,11 @@ def symbol_blob(self, path: str, anchor: str) -> str | None: lines = target.read_text(encoding="utf-8", errors="replace").splitlines() except OSError: return None - body = "\n".join(line.rstrip() for line in lines[lo - 1 : hi]) + # Trailing blank lines are dropped for the same reason `normalised_blob` drops + # them: no language gives them meaning, and git's funcname block includes the + # blank lines after a symbol - so inserting a new function *after* this one + # otherwise reads as an edit *to* it. Safe-direction noise, and still noise. + body = "\n".join(line.rstrip() for line in lines[lo - 1 : hi]).rstrip("\n") if not body.strip(): return None return hashlib.sha1(body.encode("utf-8")).hexdigest() @@ -544,6 +559,34 @@ def selectors_overlap(a: dict | None, b: dict | None) -> bool: # ---------------------------------------------------------------- validation +ID_HINT = ( + "ids use hyphens and keys use underscores: `flow.my-flow`, not `flow.my_flow`, " + "beside `coverage_declared` and `transitions_to`. Both rules are deliberate and " + "the combination is not guessable, so it is said here rather than left to a regex." +) + + +def explain_schema_error(err) -> str: + """A regex is a specification, not an explanation. + + Measured on the first outside run of this tool: the author read the snake_case keys, + applied one substitution to make the ids match, and broke every keyword instead - + 22 errors from a message that was technically complete and practically unusable. + """ + message = err.message + if "does not match" in message and "[a-z0-9-]" in message: + return f"{message.split(' does not match')[0]} is not a valid id. {ID_HINT}" + if err.validator == "additionalProperties": + return ( + f"{message} A model document carries only the keys this entity type " + f"declares; an unexpected one is usually a typo or a field from another " + f"type." + ) + if err.validator == "enum" and "type" in str(err.path): + return f"{message} Document types are set by the `type:` key in frontmatter." + return message + + def validate_schema(model: Model) -> list[Finding]: if jsonschema is None: return [Finding("warn", "SCHEMA_SKIPPED", "jsonschema not installed")] @@ -554,7 +597,9 @@ def validate_schema(model: Model) -> list[Finding]: where = str(path.relative_to(model.root)) for err in sorted(validator.iter_errors(doc), key=lambda e: list(e.path)): loc = "/".join(str(p) for p in err.path) or "(root)" - findings.append(Finding("error", "SCHEMA", f"{loc}: {err.message}", where)) + findings.append( + Finding("error", "SCHEMA", f"{loc}: {explain_schema_error(err)}", where) + ) return findings @@ -1257,7 +1302,24 @@ def validate_confidence( def validate_contradictions(model: Model, context: dict) -> list[Finding]: - """INV-008 / §7.1: same subject, overlapping selectors, different claims.""" + """INV-008 / §7.1. Two findings, because two different things were conflated. + + The original check flagged every pair of current assertions on one subject whose + claim strings differed and whose selectors overlapped. Nothing reads a claim, so + "different string" stood in for "incompatible proposition" - and five assertions on + one subject produced ten errors and an INVALID model on an outsider's first run. + + A contradiction between two sentences is not mechanically decidable; saying it is + was the error. What *is* decidable: + + - their observations disagree - one is supported, the other refuted. That is a real + contradiction, reported per pair, because each pair is a distinct conflict a + person has to resolve. + - a subject carries several current assertions with overlapping selectors and + different wording. That is a subject not discriminating between them, which is a + modelling smell rather than an error, and it is reported **once per subject** - + the count of pairs was never information, only arithmetic. + """ findings = [] by_subject: dict[str, list[dict]] = {} for aid, assertion in model.assertions.items(): @@ -1267,21 +1329,47 @@ def validate_contradictions(model: Model, context: dict) -> list[Finding]: continue by_subject.setdefault(assertion["subject"], []).append({**assertion, "_id": aid}) - for subject, group in by_subject.items(): - for i, a in enumerate(group): - for b in group[i + 1 :]: - if a["claim"] == b["claim"]: - continue - if selectors_overlap(a.get("when"), b.get("when")): - findings.append( - Finding( - "error", - "CONTRADICTION", - f"subject '{subject}': {a['_id']} and {b['_id']} make " - f"different claims under overlapping selectors", - a["_where"], - ) + def direction(aid: str) -> str | None: + latest = latest_observation(model, aid) + return latest.get("supports") if latest else None + + for subject, group in sorted(by_subject.items()): + overlapping = [ + (a, b) + for i, a in enumerate(group) + for b in group[i + 1 :] + if a["claim"] != b["claim"] + and selectors_overlap(a.get("when"), b.get("when")) + ] + if not overlapping: + continue + for a, b in overlapping: + da, db = direction(a["_id"]), direction(b["_id"]) + if {da, db} == {"supports", "refutes"}: + supported = a if da == "supports" else b + refuted = b if da == "supports" else a + findings.append( + Finding( + "error", + "CONTRADICTION", + f"subject '{subject}': {supported['_id']} is supported and " + f"{refuted['_id']} is refuted under overlapping selectors. " + f"Evidence disagrees - this is not a wording difference", + a["_where"], ) + ) + ids = sorted({a["_id"] for pair in overlapping for a in pair}) + findings.append( + Finding( + "warn", + "SUBJECT_NOT_DISCRIMINATING", + f"subject '{subject}' carries {len(ids)} current assertions with " + f"overlapping selectors and different claims: {', '.join(ids)}. If they " + f"describe different aspects, give them different subjects; if they " + f"compete, separate them with `when` selectors (INV-008)", + group[0]["_where"], + ) + ) return findings @@ -1421,11 +1509,33 @@ def impact(model: Model, changed: list[str], depth: int = 2) -> dict: for ring in rings[1:]: inspect |= ring + entities = ( + set(model.flows) + | set(model.nodes) + | set(model.states) + | set(model.externals) + | set(model.events) + ) + reached = (certain | likely | inspect) & entities + share = round(100 * len(reached) / len(entities)) if entities else 0 + return { "certain": sorted(certain), "likely": sorted(likely), "inspect": sorted(inspect), "unknown": sorted(unknown), + # INV-019 has the same shape as INV-026 one level down: a result that reaches + # most of the model is not "everything is affected", it is "this model cannot + # discriminate". Measured (#5): on graphs of 150-1200 entities a single change + # reaches a median 4-9% and the tiers stay far apart, so the tier system does + # not collapse with scale. On the two reference models - 32 and 45 entities, + # every flow invoking another - the median is 56-61% and the worst case 89%. + # The mechanism is sound and small dense models are the place it says nothing, + # which is worth reporting rather than dressing as an answer. + "entities": len(entities), + "reached": len(reached), + "share_reached": share, + "discriminates": share <= 50, }