You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Design pass for #2653, approved 2026-09-04. Full write-up (measurements, retractions, the three-input model): the "Language Confidence Tiers" memo, reproduced in summary here so the issue stands alone.
Where we are
keyword-rosetta shows identical §2 raw signals producing different §3 risk scores by language. Most of that is signal_processor._get_tier: two inline hand lists pick fc / irc / ot per language, and 45 of 59 defined languages fall through to the harshest setting (fc 0.60 / irc 5 / ot 1.40).
Measured on language-crucible (2,649 scored files) by scanning once as committed and once with TIER_VARS flattened — baseline reproduces golden_master_zero_dep_audit.json with zero diffs:
78.3% of files change; 18.2% by ≥ 25 points; 11.0% by ≥ 50. Tech Debt +19.5 mean on the 31.7% it moves (max 54.9); Documentation +14.7 on 21.2% (max 81.5); Testing max +77.6.
On rosetta with content held identical: risk_documentation +87% and risk_tech_debt +56% tier 1 → tier 3.
embedded_python is tier 3 (not the literal "python"): boot.py +77.5 Testing, bldc_driver.py +55.4 Documentation. Mean doc movement +26.6 vs python's +1.0.
The heaviest-corrected languages are declarative: makefile +29.3 debt, jcl +29.1, m4 +14.9, html +9.7 doc. They have no runtime.
Why one dial can't be made right
fc / irc / ot is one three-valued lookup doing three separable jobs:
job
truth lives in
today
our rules catch less in some languages (fidelity)
rosetta measures it per language × signal
guessed by bucket; 21 of 45 tier-3 languages have full defence-rule coverage and rosetta catches their plants, yet lose 40% of credit
this file does runtime-dynamic things a regex can't follow
the file — high_risk_execution, reflection_metaprogramming, eval are already signals
flat irc by language; 80% of python files and 84% of groovy files contain zero dynamic execution and pay it anyway
this language lets you leave things unsaid (types, errors, memory, globals)
the language spec
two undocumented inline sets + a dead FIDELITY_TIERS that disagrees with them for ~20 languages
Rosetta cannot validate any §3 correction — its null hypothesis is "no correction" — so it corrects the measurement layer only. That is the reason this drifted unnoticed.
The plan
Same equations. The constants they read come from different places.
fc becomes a measured, per-signal fidelity coefficient generated from rosetta: fc(lang, signal) = min(1, planted / measured). Over-firing rules get their credit scaled to the planted truth; under-firing is a to-fix list, not a coefficient (a coefficient that compensates for a fixable gap stops the fixing). Ledgered n/a → 1.0. Retires systems_buffer_ratio, whose fc < 1.0 gate is the cause of the safety sign inversion.
Per-file dynamism replaces the flat irc in the six formulas that read it, one formula at a time, each with its own crucible before/after.
Outcome pilot: the crucible is snapshot copies (PROVENANCE.json, 218 repos pinned by SHA, no .git), so clone ~10 upstream at their pins with history, classify fix commits, and ask whether strictness / dynamism predict later fixes at equal evidence. Decides how much weight (2) may ever carry.
What "done" looks like: identical §2 gives near-identical §3 across languages; the only language-level offset left is the small documented strictness term, which rosetta prints rather than subtracts out with the F.3 within-tier band.
Design pass for #2653, approved 2026-09-04. Full write-up (measurements, retractions, the three-input model): the "Language Confidence Tiers" memo, reproduced in summary here so the issue stands alone.
Where we are
keyword-rosetta shows identical §2 raw signals producing different §3 risk scores by language. Most of that is
signal_processor._get_tier: two inline hand lists pickfc / irc / otper language, and 45 of 59 defined languages fall through to the harshest setting (fc 0.60 / irc 5 / ot 1.40).Measured on language-crucible (2,649 scored files) by scanning once as committed and once with
TIER_VARSflattened — baseline reproducesgolden_master_zero_dep_audit.jsonwith zero diffs:risk_documentation+87% andrisk_tech_debt+56% tier 1 → tier 3.embedded_pythonis tier 3 (not the literal"python"):boot.py+77.5 Testing,bldc_driver.py+55.4 Documentation. Mean doc movement +26.6 vs python's +1.0.Why one dial can't be made right
fc / irc / otis one three-valued lookup doing three separable jobs:high_risk_execution,reflection_metaprogramming, eval are already signalsircby language; 80% of python files and 84% of groovy files contain zero dynamic execution and pay it anywayFIDELITY_TIERSthat disagrees with them for ~20 languagesRosetta cannot validate any §3 correction — its null hypothesis is "no correction" — so it corrects the measurement layer only. That is the reason this drifted unnoticed.
The plan
Same equations. The constants they read come from different places.
fcbecomes a measured, per-signal fidelity coefficient generated from rosetta:fc(lang, signal) = min(1, planted / measured). Over-firing rules get their credit scaled to the planted truth; under-firing is a to-fix list, not a coefficient (a coefficient that compensates for a fixable gap stops the fixing). Ledgered n/a → 1.0. Retiressystems_buffer_ratio, whosefc < 1.0gate is the cause of the safety sign inversion.irc/otbecome a documented language-strictness table inanalysis_lens: objective yes/no columns (static types · enforced errors · memory safe · no implicit globals), one row per language, family map soembedded_python→python. No-runtime languages getNone→ no language-level term. Weight small and bounded (UEF small-file normalization isn't achieving its intent — redesign scoring so identical intent scores identically regardless of file size (design pass for Fable) #2655's invariant stays: corrects measured risk, never creates it) until the outcome pilot says otherwise. DeletesFIDELITY_TIERS.ircin the six formulas that read it, one formula at a time, each with its own crucible before/after.PROVENANCE.json, 218 repos pinned by SHA, no.git), so clone ~10 upstream at their pins with history, classify fix commits, and ask whether strictness / dynamism predict later fixes at equal evidence. Decides how much weight (2) may ever carry.What "done" looks like: identical §2 gives near-identical §3 across languages; the only language-level offset left is the small documented strictness term, which rosetta prints rather than subtracts out with the F.3 within-tier band.
Children
systems_buffer_ratiosign inversion — _calc_safety: systems_buffer_ratio keyed on fc<1.0 makes tier-2/3 files score SAFER than tier 1 above 6/15 attack hits #2717_get_tier— engine PR 1: language-strictness table + measured fidelity coefficient replace _get_tier (child of #2716) #2718irc— engine PR 2: per-file dynamism count replaces the flat irc term (child of #2716) #2719Related: #2653 (narrowed to the membership audit; closed by PR 1), #2669 (rosetta epic), #2655 (evidence-mass floor and the invariant), #2705.