feat(core-engine): language-strictness table + measured fidelity coefficients replace the three-bucket tiers (#2718) - #2721
Merged
Conversation
…ficients replace the three-bucket tiers (#2718) Same equations; the constants they read come from different places. `_get_tier`'s two inline hand lists (45 of 59 languages fell through to fc 0.60 / irc 5 / ot 1.40) and the dead `FIDELITY_TIERS` are replaced by: - `analysis_lens.LANGUAGE_STRICTNESS`: one row per language, four yes/no columns (static types, enforced errors, memory safe, no implicit globals); Irc = gaps, Ot = 1 + 0.1*Irc; `None` for data/markup/config (no term); `LANGUAGE_FAMILY` resolves dialects so embedded_python reads python's row (#2653). - `gitgalaxy/standards/fidelity_table.py`, generated by tests/tools/fidelity_table.py from keyword-rosetta: fc(lang, signal) = min(1, planted / measured), read per signal (safety, test, doc, ownership) at every defence site. `--check` guards staleness; `--docs` renders both tables into docs/wiki/08-03. - `_calc_safety`'s systems_buffer_ratio retired (#2717): keyed on fc < 1.0 it scored 592 crucible files SAFER than tier 1 for identical evidence. - `_calc_documentation` scales only rule hits by fidelity; doc_loc and the umbrella are structural evidence and no longer sit inside the coefficient. Golden master re-blessed in both venvs: 8,803 risk-column diffs, 0 topological; rust/java/yaml/json untouched; ex-tier-3 languages down, ts/js/python/c/cpp slightly up. rosetta_audit 46/46, tree-sitter 30/30, tri 3/3, audit_check clear. Closes #2718 Closes #2717 Closes #2653 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
6 tasks
Contributor
…NESS annotation -- Python 3.9 evaluates module-level annotations at import Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Child of #2716 (design pass approved 2026-09-04). Same equations; the two constants they read now come from a documented table and a measured table instead of a three-bucket lookup on the language's name.
What changed
analysis_lens.LANGUAGE_STRICTNESSreplaces_get_tier+TIER_VARS(Irc / Ot). One row per language, four yes/no columns checkable against the language's documentation — static types · enforced errors · memory safe · no implicit globals.Irc= number of gaps (0–4),Ot = 1 + 0.1·Irc.Nonerows (yaml, json, css, html, …) have no runtime and carry no language-level term; an unknown language also gets none — the opposite of the old fall-through to the harshest bucket.LANGUAGE_FAMILYresolves dialects first, soembedded_pythonreads python's row (#2653). The deadFIDELITY_TIERS(one occurrence repo-wide, disagreeing with the live sets for ~20 languages) is deleted.gitgalaxy/standards/fidelity_table.pyreplaces the single per-languageFc. Generated bytests/tools/fidelity_table.pyfrom keyword-rosetta:fc(lang, signal) = min(1, planted / measured), where planted is the SPEC's plant (the corpus-wide median of the manifests) and measured is what the rule found. Asafetyrule that fires 3 times on 2 planted constructs now credits each hit at 2/3; under-firing stays at 1.0 on purpose (a rule to fix, never compensated). The engine reads it per signal —safety,test,doc,ownership— at each defence site. One deliberate consequence in_calc_documentation: only the rule hits (doc,ownership) carry the coefficient;doc_locand the GuideStar umbrella are structural evidence, not rule hits, and are no longer scaled (they used to sit inside the× Fcbracket).--checkfails when the committed module is stale against the sibling corpus (test_committed_fidelity_table_is_fresh_against_the_corpus, skipped without the corpus);--docsrenders both tables into 08-03._calc_safety'ssystems_buffer_ratiois retired (closes #2717). Keyed onfc < 1.0it made every non-tier-1 file score safer than tier 1 above 6/15 attack-weighted hits — 592 crucible files. The over-firing it compensated for (C's pointer syntax) is now scaled at the source by the fidelity coefficient.Docs. 08-03's tier paragraph is now the chain — what a regex can't see → the three inputs and where each is sourced → the invariants → the pilot that will validate the strictness weight — with the strictness and fidelity tables rendered from the data files between
<!-- generated:… -->markers, never typed. 03-02's hand-written "Tier" column is labelled as the retired 2025 buckets and points at the live table; 08-08 / 08-09 / 08-15 / 08-16 say whereIrcandFccome from.What moved, and why it's the expected shape
Golden master regenerated in both venvs and attributed with
golden_diff.deep_compare(uncapped): 8,803 substantive diffs, 0 topological — risk columns only, no structural or coordinate movement.irc 5are gone: cobol −2.9 mean over 1,700 cells, shell −2.5, lua −3.0, jcl −4.9, html −9.6, zig −7.9, embedded_python −3.3 (itsbldc_driver.pyDocumentation −45.7,vibration_driver.py−41.6 — the risk formulas: audit language-tier membership (embedded_python lands tier3) + decide how rosetta bias reporting treats documented Fc/Irc tier constants #2653 case).safetyfidelity 0.67), javascript +1.4 (Irc2→3: sloppy-mode globals), python +0.9 and c/cpp +1.0 (_calc_safety: systems_buffer_ratio keyed on fc<1.0 makes tier-2/3 files score SAFER than tier 1 above 6/15 attack hits #2717: the 0.75 attack discount is gone; python test files with heavysafety_bypassesare the +31 Error & Exception movers).constants.zigDocumentation −81.5; cobol/zig/lua Testing −77.6 (Ot1.40 → 1.20 / 1.10 / 1.30 at a sigmoid knee — pre-existing sensitivity, symmetric with the +77.5 the old constants produced).Gauntlet
audit_check.py: all clear (ruff baseline regenerated for 5 pure line-shifts; mypy 2-error baseline; dead-key; ast-accuracy).tree_sitter_accuracy_audit.py --ci --all: 30/30 OK.tri_comparison_chart.py --all --ci: 3/3 OK.rosetta_audit.py: 46/46 PASS, 0 regressions — the corpus gates §2 raw signals, which this PR does not touch, so no corpus re-bless is owed; therisk_*rows of the bias report move on the nextbias-historyregen, and keyword-rosetta#50's within-tier banding can then be retired in favour of printing the strictness offset (risk formulas: replace language tiers with a measured fidelity table + a language-strictness table (child of #2560) #2716 checklist).crucible_check.py --update --yes), scoped diff above.Not in this PR
ircis still summed into six formulas in six units — engine PR 2: per-file dynamism count replaces the flat irc term (child of #2716) #2719 swaps it for the per-file dynamism count, one formula at a time.Closes #2718
Closes #2717
Closes #2653
🤖 Generated with Claude Code