length leaks: func_internal_density, avg_func_loc and control_flow_ratio read file length with content and tier held equal (rosetta F.1 design pass)

[squid-protocol]

Found by the length-leak check added to keyword-rosetta's bias_report.py in gitgalaxy#2669 F.1. Design pass requested (T1), in the #2655 shape: no engine edit until the shape is approved.

What the check does

The rosetta corpus is one 12-probe program written in 46 languages, at 46 lengths (coding_loc from 4 to 19 lines per file; every shell is under the 50-line evidence-mass floor). For each derived metric, take the languages whose measured inputs for that metric are all inside the green band (content held equal) and that share the engine's scoring tier (_get_tier; the tier-3 languages are largely the short shells, so without this a tier effect reads as a length effect), and rank-correlate the metric against coding_loc. With content and tier held, length is the only thing left to vary, so a strong correlation means the formula is reading length where it should be reading content.

Threshold: |Spearman rho| ≥ 0.6 over ≥ 8 languages. Engine a334839, corpus main, 2026-09-04.

What it found

metric	languages (tier3)	rho	inputs held in band	where length enters
func_internal_density	11	−0.96	branch, args, func_start	avg_func_complexity / avg_func_loc (record_keeper.py ~L487) — the denominator is length by construction
avg_func_loc	29	+0.92	func_start	loc / functions_found — length by definition
control_flow_ratio	21	−0.67	branch	branch / (branch + structural_boundaries) (detector.py ~L1288); structural_boundaries is a per-language token tally that grows with the file

Weak (0.4 ≤ |rho| < 0.6), reported but not asserted: risk_cognitive_load −0.58 (n=8), risk_tech_debt −0.58 (n=16), cog_raw −0.55 (n=9), risk_verification +0.48 (n=25, no LOC term located), classes_found −0.41 (n=31, nothing held). Note that cog_raw read −0.63 before tier was held equal and dropped to weak after: most of what looked like a length leak there was #2653's irc / mass_loc term. Everything under _mass_loc (cog load, safety, state flux, tech debt) is floored at 50 by #2655, so no rosetta shell should carry a length term through those densities at all — the residual weak correlations are either an unheld input (concurrency, reflection_metaprogramming are not cached) or the per-file mean over four files, and are worth a look but are not the finding.

Why it matters

These three are the per-function morphology descriptors the bias report scores as "the engine describing one program differently depending on the language it is written in". For avg_func_loc that description is tautological: a per-function line count is length, and the corpus cannot plant it. For func_internal_density the same length sits in the denominator, so a language that writes the same 13 probes in fewer lines is reported as denser logic. For control_flow_ratio the branch count is planted at 3 while the denominator is a vocabulary tally that scales with the file — #2689 bucket B already established that planting structural_boundaries cannot equalise it.

On the real-world corpus the same thing holds one level up: two files with identical branch and argument structure score differently on all three purely by line count, which is the length-dependence #2655 removed from the risk densities but not from these recorder-side descriptors.

Options (for the design pass)

1. Reclassify as length. Treat avg_func_loc as a size measure (it is), and the bias report demotes it to the context group beside coding_loc. Cheapest, but does nothing for the other two.
2. Floor the per-function denominators the way UEF small-file normalization isn't achieving its intent — redesign scoring so identical intent scores identically regardless of file size (design pass for Fable) #2655 floored the per-file ones. func_internal_density = avg_comp / max(avg_loc, F) with a per-function evidence floor; control_flow_ratio over max(branch + boundaries, F'). Same doctrine as _mass_loc, moves golden masters for every function under the floor.
3. Redefine control_flow_ratio against a planted-able denominator. Branch per function (avg_func_complexity already exists) or branch per structural boundary kind rather than per boundary token. Changes the metric's meaning; needs the wiki 08-xx pages updated.
4. Leave the engine alone, ledger the three metrics corpus-side as length-derived. Honest for the rosetta gate, but leaves the real-world length dependence in place.

Recommendation to evaluate first: (2) for func_internal_density, (1) for avg_func_loc, and a decision between (3) and (4) for control_flow_ratio, since #2689 already showed its denominator cannot be planted.

Corpus side

keyword-rosetta PR (F.1/F.2) carries the check, the tiers map in docs/bias_data.json, and the table above regenerated on every corpus push. The three metrics' rosetta cells keep whatever verdict they have today; this issue does not change the gate.

Related: #2655 (evidence-mass floor — the per-file half of this), #2689 (bucket B: structural_boundaries is unplantable), #2653 (tier constants — the confound the check now holds equal), #2669 (Batch F).

[squid-protocol]

Design pass done (2026-09-04). No engine edit yet — this is the shape for approval, in the #2655 pattern.

Recommendation: option 2 for func_internal_density, option 4 for control_flow_ratio with the cause restated, option 1 already shipped for avg_func_loc. The two are not the same defect and should not get the same fix: one is a real per-function density with no floor, the other is not a length leak at all.

Measurements below are against engine a334839 and tests/golden_master_audit.json (2817 files, 2291 with signals, 45422 functions — the real-world corpus, not rosetta).

---

1. avg_func_loc — closed, no engine change

Option 1 shipped as keyword-rosetta#49: avg_func_loc and comment_lines joined CONTEXT_METRICS, and func_internal_density now declares avg_func_loc in its DERIVED_INPUTS. avg_func_loc has dropped out of the leak table. Nothing owed here.

2. func_internal_density — floor the denominator (option 2), F = 12

The floor value follows #2655's own derivation

#2655 set EVIDENCE_MASS_FLOOR = 50 because the golden-master median coding_loc is 51 — the floor sits at the corpus median, so it neutralises the small-file regime and leaves the median-and-up regime byte-identical. The per-function analog on the same corpus:

population	median	#2655 floor	share below the floor
file coding_loc	51	50	49.5% (1299/2624)
file avg_func_loc	12.0	12 (proposed)	49.0% (1016/2074)

Same derivation, same calibration. Proposed: FUNC_EVIDENCE_MASS_FLOOR: 12 in analysis_lens.ENGINE_CONSTANTS beside EVIDENCE_MASS_FLOOR, and record_keeper.py L487 becomes

func_internal_density = avg_comp / max(avg_loc, FUNC_EVIDENCE_MASS_FLOOR)

(the avg_loc > 0 guard folds into the max).

What it does to the rosetta gate

Every rosetta shell's avg_func_loc is ≤ 5.69 (groovy, the longest), so every language is in the count regime and func_internal_density becomes avg_func_complexity / 12 — an exact rescale of a planted-signal descriptor.

	today	floored (any F ≥ 8)
leak rho vs coding_loc (n=11, tier3, branch/args/func_start held)	−0.96	+0.000
consistency (share of the 46 in the green band)	54% (21 out of band)	67% (15 out of band)

The floored column's out-of-band set is identical to avg_func_complexity's, which is the point: the metric stops reporting length and reports the branch structure the SPEC actually plants. func_internal_density is currently a derived cell, so this changes no --gate verdict — the 91 unexplained stay 91.

What it does to real code

Contained, and in the direction the doctrine predicts:

F	files moved (of 2074 with functions)	median drop	p99 drop
8	320 (15.4%)	0.042	0.875
10	456 (22.0%)	0.053	0.900
12	574 (27.7%)	0.060	0.917
15	701 (33.8%)	0.071	0.933
20	865 (41.7%)	0.082	0.950

72.3% of files are untouched at F=12 (the floor is the identity for them). The movers are exactly the small-function inflation cases: formula_installer.rb 1.50 → 0.50, and five find*.sql files at a flat 1.00 → 0.08 (functions averaging one line, one branch each). Median mover sits at 0.196 today and loses 0.060.

Blast radius: smaller than it looks, because the metric is half-dead

func_internal_density is not in either pre-trained ML vector — not the 5-dim per-function archetype vector (branch, loc, args, keyword_density, control_flow_ratio) and not the file-level raw_vector. It reaches exactly two places: the file_data DB column, and security_auditor.py L385.

And that second one never fires. signal_processor builds telemetry_payload with func_complexity_gini (L849) but never writes func_internal_density — it is computed only inside record_keeper while writing the row. So security_auditor.py:385's tel.get("func_internal_density", 0.0) has always fallen through to 0.0 for every file: a permanently-zero column in the security training frame, the same shape as the prompt_injection/agentic_rce placeholders documented at analysis_lens.py L1105. Worth filing separately — it is a pre-existing bug, not caused by anything here, and it means the floor's only live consumer is the DB column and the reports.

So the bless is 574 file rows plus topology, no archetype drift.

3. control_flow_ratio — option 3 is not safe, and length is not the cause

Option 3 has the #1020 blast radius

cfr is a pre-trained ML feature in two scoring paths:

• signal_processor L601 — the per-function archetype vector's 5th component, scaled against GENERAL_FUNCTION_INFERENCE_MODEL's SCALER_MEDIANS/SCALER_IQRS and matched to fixed centroids;
• signal_processor L665 — the file-level raw_vector, which carries both cfr and log_logic_loc (logic_loc = coding_loc × cfr), against self.SCALER_MEDIANS and GLOBAL_ARCHETYPES;
• plus control_flow_x1000 in SAT_SCHEMA.

Redefining the metric moves the feature's distribution wholesale against models trained on the old one. Measured on the golden master for the most natural redefinition (branch / max(coding_loc, 50) — the #2655-consistent branch density):

• 71.5% of files (1638/2291) change value; the distribution collapses from median 0.333 → 0.066;
• logic_loc median 15 → 4, which moves log_logic_loc too;
• so two of the file-level features and one of the five per-function features shift at once.

That is precisely the failure analysis_lens.py L1099-1112 records from #1020: "massive file_cluster_N/Repository Archetype drift, not just the intended removal". The comment there guards against reordering the schema; the same models are just as sensitive to rescaling a feature in place. Rejecting option 3 on that evidence — and note the redefinition otherwise looks great on the gate (consistency 30% → 69%, rho −0.673 → +0.000), which is exactly the trap.

The cause is vocabulary breadth, not length

structural_boundaries is not a boundary count. It is a per-language keyword tally, and the language definitions differ in breadth, not just in verbosity:

• solidity matches every type token — uint\d*, int\d*, bytes\d*, address, bool, string, mapping, plus pragma|import|contract|…
• perl matches every my; java every var, new, return; python every def|class|return|import|from|as|pass|…
• scheme matches three forms (let|let*|letrec|begin|do); yaml six literal keys; makefile assignment lines only

On the identical 12-probe program, back-calculated mean structural_boundaries runs 4.0 (makefile) to 65.2 (solidity) — 16×, while coding_loc spans 4 to 18.75 — 4.7×. Length cannot account for the spread. On real code the same thing shows as a 9× spread in the growth rate: median structural_boundaries / coding_loc is 0.047 in powershell, 0.112 in cobol, 0.264 in rust, 0.433 in lua.

So cfr = planted / (planted + vocabulary_tally) is a hybrid of a planted signal and a member of the F.1 verbosity family. It can never be cross-language consistent, and it is the weakest column in the report at 30% consistency (32/46 out of band) for that reason — not because of length.

The −0.67 length correlation is the shadow: structural_boundaries does track length within a language (rho 0.69–0.95 in every one of the 21 languages with ≥15 real files), and rosetta pins branch at 3, so the ratio has to fall as the shell gets longer.

The real-world claim in the issue body holds, with a caveat

Marginally, cfr rises with length on real code (rho +0.03 to +0.75 by language) because branch grows faster than structural_boundaries. Run rosetta's own test instead — hold language and exact branch count and correlate against coding_loc:

rho = −0.591 over 512 real files, 10 languages, 44 strata. Cobol −0.47, python −0.90, shell −0.50, assembly −0.64, groovy −0.75, lua −0.77. So the effect is real and about the size rosetta reports (−0.67); it just is not visible without holding branch.

Recommendation

Option 4, with the cause named correctly, plus nothing new in the engine. A length-and-vocabulary-invariant "how much of this program is control flow" descriptor already exists and is already gated: avg_func_complexity — 67% consistent, leak rho +0.30 over the same 21 languages, reads the planted branch count directly. The gate should read branch shape through it; cfr stays what it actually is, a pre-trained ML feature with a language-specific denominator.

Corpus side that would follow (keyword-rosetta, F.3 or its own PR):

1. A ledger entry — control-flow-ratio-vocabulary-denominator or similar — signal: branch|control_flow_ratio, covering the 32 out-of-band cfr cells, citing Design (D.1): shape-descriptor comparability — the 160 cells decompose into five mechanisms, only 15 of which are a real design question #2689 (the denominator is unplantable) and this issue for the 16× breadth measurement.
2. A small bias_report.py change so length_leaks reports this as a vocabulary leak rather than a length leak when the denominator is a registry token tally — otherwise the row reappears at every regen with the wrong cause on it.

If you would rather keep the engine honest here anyway, the only safe form is an additive one: emit branch / _mass_loc(coding_loc) as a new telemetry key and leave control_flow_ratio untouched, so no trained vector moves. That is more surface for a descriptor avg_func_complexity already covers, so I am not recommending it — but it is the option that gets the invariance without the #1020 risk.

---

Decisions requested

1. func_internal_density: floor at FUNC_EVIDENCE_MASS_FLOOR = 12 — yes/no, and is 12 the right value (the table above is the sensitivity)?
2. control_flow_ratio: ledger it as a vocabulary-denominator deviation (recommended), or add the additive branch-density key, or redefine anyway and re-train?
3. The security_auditor dead-feature find — file as its own bug, or fold into the same PR?

Related: #2655, #2653, #2689, #1020 (the ML feature-layout precedent), #2669 Batch F, keyword-rosetta#49.

[squid-protocol]

Decisions (epic owner, 2026-09-04):

1. func_internal_density: floor at FUNC_EVIDENCE_MASS_FLOOR = 12 — approved. Same corpus-median derivation as UEF small-file normalization isn't achieving its intent — redesign scoring so identical intent scores identically regardless of file size (design pass for Fable) #2655; the out-of-band set becomes identical to avg_func_complexity's and 72% of real files do not move. Keep 12 rather than a lower value: a lower floor moves fewer rows but abandons the rule that made UEF small-file normalization isn't achieving its intent — redesign scoring so identical intent scores identically regardless of file size (design pass for Fable) #2655 defensible. The wiki page must say plainly that below the floor the column is a rescale of avg_func_complexity.
2. control_flow_ratio: option 4 — ledger, cause = vocabulary. Nothing new in the engine: no additive key (surface with no consumer, avg_func_complexity already covers it), no redefinition (the 71.5% move against two pre-trained vectors is the Remove AST-dependent Security & Vulnerability metrics (Injection, RCE, Memory) #1020 failure; the good-looking gate numbers are the trap). Corpus side: the ledger entry already exists — control-flow-ratio-denominator-is-a-vocabulary-tally, validated, 21 languages; the other 11 out-of-band cells inherit from branch. The one owed action is the leak-check label: length_leaks should report a formula whose denominator is a registry token tally as a vocabulary leak, so the row stops reappearing with the wrong cause. Small keyword-rosetta tools change, rides with F.3.
3. The dead security_auditor feature: filed on its own, not fixed here. Populating the telemetry key would change an input to the security training frame for every file — a different blast-radius question from flooring a DB column, and the two must not share a bless.

Engine PR follows: one constant, one recorder line, one test in the test_uef_length_invariance.py shape, the wiki sentence; golden-master bless (~574 rows + topology) with explicit go-ahead at the time.

[squid-protocol]

Implemented as #2715 (one constant, one recorder line, tests/tools_recorders/test_func_internal_density_floor.py, wiki 08-03 paragraph). Dead-feature find filed as #2714.

One correction to the design pass: no golden-master bless is owed. crucible_check.py passes in both venvs because the audit fixtures (tests/golden_master_*.json) carry Raw Cognitive Density but not the recorder's func_internal_density column — the 574-row figure was measured on the scan file_data DB, which never reaches the fixture. So the PR is exactly as small as it looks, and crucible-audit stays green.