Skip to content

fix(core-engine): give the api rule a stated contract and audit all 46 languages against it (#2730) - #2743

Merged
squid-protocol merged 2 commits into
mainfrom
fix/2730-api-rule-contract
Sep 5, 2026
Merged

fix(core-engine): give the api rule a stated contract and audit all 46 languages against it (#2730)#2743
squid-protocol merged 2 commits into
mainfrom
fix/2730-api-rule-contract

Conversation

@squid-protocol

@squid-protocol squid-protocol commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Resolves #2730.
Part of #2669.

The problem

api had a one-line description ("Code exposed to the outside world. Captures explicit
visibility markers AND implicit architectural defaults") and no contract, so 46 languages
disagreed about what a public surface is, in both directions at once. #2729's corpus wave
measured it: seven languages cannot see the idiom their own language uses to make a function
public, and five count a bare public token anywhere in the code stream.

api feeds _calc_api_exposure and _calc_documentation, so both directions land in scored
risk. On the control corpus it reads as language bias with nothing wrong in the corpus:
identical planted intent, api 0 in one language and 3 in its neighbour, purely from what the
regex accepts.

The contract

Stated in the new docs/api_rule_contract.md, and in one line next to the rule in
how_to_add_a_language.md's output schema:

api matches a declaration that makes a named function or type visible outside the file it
is declared in.

Three corollaries, each of which the audit found a language violating:

  1. A reference is not a declaration — a call site, an import, a struct-literal field key, a
    switch case on the keyword.
  2. A modifier counts where it modifies a declaration, not wherever the word appears.
  3. Where the language is public-by-default, the declaration itself IS the marker — the rule
    python has always used, extended to the languages that needed it.

Plus a documented fallback family for languages with no per-function visibility concept at
all (dockerfile EXPOSE, yaml on:, css @property, html id=, yacc %define,
m4 AC_SUBST, cobol ENTRY, agc_assembly EQUALS, makefile, sqlite, and now
matlab), so the substitution is a recorded decision rather than an accident.

The audit — all 46 corpus languages

Full table with per-language counts is in docs/api_rule_contract.md. 23 languages were
already inside the contract and are untouched; 18 changed.

Too narrow — the language's own visibility idiom was invisible

language the idiom what was added crucible rosetta
abap FORM/FUNCTION are public by default line-anchored, so CALL FUNCTION '...' cannot match 6 0 → 13
ada a library-level subprogram is its own compilation unit procedure/function at column 0; nested (private) ones are indented 0 0 → 13
dart a top-level function with no leading _ column-0 return type + public name + ( 174 → 149 0 → 13
matlab a function file is callable by name column-0 function (the fallback family) 0 → 23 0 → 13
objective-c a method declared in an @interface a -/+ method line ending ; rather than { 5 → 13 0
ruby public :name (a top-level def is private on Object) public/public_class_method + symbol 0 12
shell export -f name the flag run between export and the name 178 12
yaml workflow_call added to the trigger set 0 4
perl a sub is package-public ^sub <name> 20 → 840 0 → 13

perl is the case the issue called out separately: every alternative it had was an export
list that also lands on a rule which already owns it (@EXPORT_OK = (...) is a
state_mutation, use Exporter|parent|base an import), so api could only move by moving
another planted count with it.

Too broad — a token counted where no declaration exists

language what it counted share of its crucible matches
c any indented declaration-shaped line, so every body-local declaration and every two-word statement 7534 of 8675 (return NULL; alone was 513)
cobol CALL/INVOKE call sites, plus END-CALL via the hyphen boundary 843 of 1396
agc_assembly the EXTEND opcode (SUBROUTINE/BEXT never fired at all) 323 of 367
go any line starting with an exported identifier — struct-literal keys, calls on exported vars 176 of 614
powershell <name>( at line start — a .NET call or a statement 61 of 199
lua function (), an anonymous function that declares no name 16 of 288
dart @pragma(...), a compiler hint 34 of 174
assembly EXTERN/IMPORT, which import a name rather than publish one 3
java csharp kotlin php groovy swift typescript a bare access modifier anywhere in the code stream see below

The bare-modifier family, with what each one was actually matching:
import okhttp3.internal.X (kotlin, 12 of 17), let package = Package(...) in a SwiftPM
manifest (swift), $public and 'path.public' (php, 7), case "public": in a compiler that
parses the keyword (typescript, csharp), a log message about a "public key ring" (groovy, 4).

java and csharp move zero matches on either corpus — real Java and C# put public in
front of a declaration nearly always, so the anchor is a precision guard rather than a recount.
That is the answer to "why did a real fix produce no diff" for those two; the strict tests hold
the guard in place.

Two findings surfaced while doing this that are not #2730's shape and are not fixed here:
powershell's #2656 keyword-exclusion set is lowercase against an re.I pattern, so If ( and
Param( matched anyway (moot now — the alternative is gone, and the test records it), and
go/java's encapsulation rules carry the same unanchored shape their api rules had.
encapsulation is a different rule with its own contract question and is left alone.

Verification

  • 7619 tests pass serially, the mode CI runs (pytest tests/ -v).
  • 91 new strict cases across 21 test_<lang>_strict.py files, in a
    test_<lang>_api_contract_2730 function each. Every case was run against the real compiled
    regex before being written down (AGENTS.md rule 3) — the harness reported 0 mismatches across
    all 91 before a line of test code was committed.
  • ReDoS: detonation on every rule that gained a new quantifier shape (c, go, dart,
    objective-c, shell, java, csharp, kotlin, groovy, php, typescript), plus a
    geometric scaling sweep at n = 2k/4k/8k/16k/32k on all of them — every one ~2.0x per doubling
    (linear), none above 5ms at n=32000. Rules that gained no new quantifier deliberately got no
    detonation: 16 extra spawn-context subprocesses measurably perturb the ratio-based
    spec_exposure timing tests elsewhere in the suite.
  • ruff_audit --ci, mypy_audit --ci, audit_check.py — all clear against their baselines.
  • tree_sitter_accuracy_audit --ci --all — 30/30 OK, no baseline regeneration needed.
  • tri_comparison_chart --all --ci — 3/3 committed baselines OK (api does not feed
    func/class precision, so no movement was expected and none occurred).

Golden masters

Both re-blessed. 3384 differences per mode, identical in structure between them, every one
attributed and zero off-target:

class count
Topological Coordinates X/Y/Z (corpus-wide mass re-solve) 2120
API Exposure 300
Documentation Exposure 257
Structural Magnitude 255
Exposed API / Public Exports 255
Directory Group Magnitude / total_mass 45 + 45
ecosystem api_exposure / documentation / composition/*/impact 45 + 41 + 16
forensic top-3 rankings, health/avg_documentation 4 + 1

The X/Y/Z bucket is why a language-scoped scope_check --expect cannot be the gate here:
api feeds Structural Magnitude, which feeds total mass, which the 3D topology solver uses to
position every file in the corpus. So the scoping claim is made on the other 1264 instead —
every per-file signal/risk diff resolves to a file the engine detected as one of the 23 touched
languages. Two looked off-target and are not: cpp/godot/object.h is detected as C
(Identity Proof: Sibling Anchor (.c)), and the extensionless perl/exiftool/exiftool is
detected as Perl by shebang. Both go through a rule this PR changed.

Direction of change matches intent everywhere — narrowed rules negative, widened rules positive:

Δ Public Exports Δ Public Exports
c −11501 perl +914
cobol −907 matlab +23
cpp/godot/object.h (C) −391 objective-c +3
agc_assembly −323
go −240
dart −31, powershell −25, lua −16, kotlin −9, php −8, assembly/groovy −3, swift/typescript −2

Cross-repo

Companion: squid-protocol/keyword-rosetta#56 ("corpus owes a re-bless against engine main",
already open before this PR).

This PR is corpus-visible per AGENTS.md rule 8 and #2669's paired-work rule, so it carries the
rosetta:rebless-owed label per keyword-rosetta/docs/GATING.md's cross-repo flow. Its
rosetta-audit is red as designed, and its classification is exactly right:

rosetta_audit: 46 language(s) checked -- 5 regression(s), 21 pre-existing, 0 broken.

Per GATING.md's flow, nothing in the corpus repo is edited beforehand:

  1. This PR merges first (labelled, audit advisory).
  2. Corpus re-bless PR against engine main: the 97 manifest cells, plus the ledger —
    api-no-plantable-idiom narrows from seven languages to objective-c + sqlite
    (objective-c stays 0 because planting an @interface would break SPEC.md rule 2's
    class_start == 0 tripwire, even though the rule can now see one), and
    api-double-count-inflates-scored-api flips still_reproduces to false — its own verdict
    says it "resolves when arch_api counts a declared-public, uncalled function twice: 3 functions record api 6 #2731 credits only the orphans the rule did not already count", which
    is what the 3 → 0 cells are. Also swaps shell's plant from export PROBE_X=1 to
    export -f probe_x, so shell's api measures an exported function rather than a
    similarly-named variable.
  3. bias-history.yml regenerates the report on that merge and closes docs: add new page to GitGalaxy documentation #56; then
    tools/issue_status.py for the languages that moved, per Rosetta to green: cause-first batch plan + multi-session coordination (child of #2560) #2669 F.6.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WfeHRWra1d6Z5uSReFajSq

…6 languages against it (#2730)

The `api` rule had no contract, so 46 languages disagreed about what
"public surface" means in both directions: seven could not see the idiom
their own language uses to make a function public, and eleven counted a
token where no declaration exists.

Contract, now stated in docs/api_rule_contract.md and next to the rule in
how_to_add_a_language.md's output schema:

    api matches a declaration that makes a named function or type visible
    outside the file it is declared in.

Too narrow (the language's own idiom was invisible): abap FORM, ada's
library-level subprogram, dart's top-level public function, matlab's
function file, objective-c's @interface method line, ruby's `public :name`,
shell's `export -f`, plus yaml's `workflow_call` and perl's `sub`.

Too broad (a reference, a statement or a bare modifier counted): c (7534 of
8675 crucible matches were body-local declarations and statements like
`return NULL;`), cobol (CALL/END-CALL), agc_assembly (the EXTEND opcode),
powershell (`<name>(` call sites), go (struct-literal field keys), lua
(anonymous `function ()`), assembly (EXTERN/IMPORT), dart (@pragma), and the
bare-`public` family: java, csharp, kotlin, php, groovy, swift, typescript.

91 new strict cases across 21 files, each verified against the real compiled
regex before being written down; ReDoS detonations on every new quantifier
shape, all measured linear (~2.0x per doubling at n=2k..32k).

Both golden masters re-blessed: 3384 differences per mode, fully attributed,
zero off-target.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WfeHRWra1d6Z5uSReFajSq
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol squid-protocol added the rosetta:rebless-owed Intentionally moves keyword-rosetta counts; audit warns, corpus re-blesses after merge label Sep 5, 2026
Conflict was confined to the two golden-master fixtures, both auto-generated:
#2742 (jcl sync_locks) re-blessed them on main while this branch was in flight.
Resolved per .agents/skills/ci-push-checklist section 6 -- never hand-merge a
generated file: took origin/main's version of both to clear the markers, then
re-ran `crucible_check.py --update --yes` so they are recalculated from the
merged code rather than stitched together.

Verified the resolution rather than trusting the regen: re-attributed the
regenerated fixtures against origin/main and got the same 3384-diff profile as
before the merge, bucket for bucket (2120 topology X/Y/Z, 300 API Exposure, 257
Documentation Exposure, 255 Public Exports, 255 Structural Magnitude, 197
roll-ups) and the same per-language counts. `jcl` appears nowhere in the
per-file diffs, which is the positive check that #2742's blessing survived
intact instead of being overwritten.

Both modes PASS, full suite 7625 passed serially, audit_check and
tree-sitter-accuracy audit clear.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WfeHRWra1d6Z5uSReFajSq
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rosetta:rebless-owed Intentionally moves keyword-rosetta counts; audit warns, corpus re-blesses after merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The api rule has no stated contract: 7 languages' own visibility idiom is invisible to it, and 5 others count any public token

1 participant