Skip to content

fix(core-engine): the api rule's own declarations are not new orphan surface (#2731) - #2734

Merged
squid-protocol merged 1 commit into
mainfrom
fix/2731-api-orphan-double-count
Sep 5, 2026
Merged

fix(core-engine): the api rule's own declarations are not new orphan surface (#2731)#2734
squid-protocol merged 1 commit into
mainfrom
fix/2731-api-orphan-double-count

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Closes #2731. Part of #2669.

The double count

galaxyscope.py's Contextual Baseline Fix converts an imported file's orphans into API
exposure. It never asked whether the language's own api rule had already counted those
declarations, so a function that is both declared public and uncalled contributed twice.

keyword-rosetta data/go/a.go — three exported functions, none called in-file, imported by
main.go:

column before after
function_count 3 3
raw_arch_api 3 3
raw_state_slop_orphans 3 3
arch_api (what scoring reads) 6 3
risk_api_exposure 5.939 3.022
risk_documentation 79.155 40.613

The fix

detector.py reports api_declared_orphans — how many of the file's orphans the api rule
matched at their own declaration — and the conversion credits only the remainder
(orphans - api_declared_orphans, clamped). The debt wipe stays unconditional: the file is
imported, so an already-public orphan is surface the rule counted already, not dead weight.

The test is by name, not by span. An orphan's name occurs exactly once in the whole file —
that is precisely what the orphan census just proved (token_counts[name] <= 1) — so a name
appearing on a line the api rule matched can only be its own declaration. No false positive is
possible. Span containment (hit_vector["api"] > 0) is both looser and tighter than that:

  • looser — C's api rule matches any non-static declaration-shaped line, including
    body-local int counter = 0;, so every long function suppresses itself. Measured: the span
    form suppressed all 73 orphans in c/cpython/ceval.c and all 27 in frameobject.c (both
    files of static helpers); the name form suppresses 6 and 0.
  • tighter — the public marker often sits outside the slicer's own span: JS/TS export
    precedes start_idx, php's span starts a line early, a java @Test line pulls start_line
    back off the public one. The span form scored javascript/typescript 0-of-3.

New api_declared_orphans is consumed by the orchestrator only — it is not a signal, does not
reach the recorder schema, and appears in no output artifact (checked against the audit JSON,
LLM digest, SBOM and the file_data schema).

Verification

  • Golden masters re-blessed, both modes. Every one of the 1254 full-precision / 1253
    zero-dependency differences is attributed, with nothing left over:

    class count
    Topological Coordinates X/Y/Z (mass re-solve, corpus-wide) 853
    API Exposure % 87
    Documentation Exposure % 71
    Exposed API / Public Exports 61
    Structural Magnitude 61
    Directory Group Magnitude / total_mass 29 + 40
    avg api_exposure / documentation roll-ups 50
    ecosystem composition/*/impact, health/avg_documentation 11
    forensic top-3 rankings (api_exposure/highest, undocumented_critical_path) 2

    61 files move on the real-world corpus, all downward, zero increases, and no structural
    signal other than api moves at all. A control A/B (same venv, origin/main vs this branch)
    measured total arch_api 35892 → 35127 (−2.13%).

  • keyword-rosetta: 21 of 46 languages move — 20 go 6 → 3 (three probe functions, three units
    of public surface), c goes 12 → 9 / 10 → 7, makefile 8 → 7. The other 25 are unchanged
    (their api rule marks export lists or non-function surfaces, which never overlapped the
    orphan set). The corpus re-bless follows as a keyword-rosetta PR against this merged engine,
    per the no-pins flow.

  • pytest tests/ — 7568 passed (3 new regression tests: the go double count, the C
    static-vs-exported discrimination, and the orchestrator's credit/clamp arithmetic).

  • ruff_audit.py --ci, mypy_audit.py --ci, dead_key_audit.py --ci — all clean against
    baseline. tree_sitter_accuracy_audit.py / tri_comparison_chart.py measure function/class/arg
    extraction and never read api (neither tool contains the string); this change cannot move them.

Follow-up for the corpus side

#2729's gate pins api_orphan_credit (arch_api - raw_arch_api) as its only assertion that the
main → a → b → c import chain resolved. That credit legitimately becomes 0 for every language
whose probes are all declared public. raw_state_slop_orphans 3 alongside state_slop_orphans 0
still proves the conversion ran, so the gate has a replacement available; it needs to be swapped
in the same PR that re-blesses the manifests.

🤖 Generated with Claude Code

…surface (#2731)

galaxyscope.py's Contextual Baseline Fix converts an imported file's orphans
into API exposure without asking whether the language's `api` rule had already
counted those same declarations, so a function that is both declared public and
uncalled contributed twice. keyword-rosetta's `data/go/a.go` -- three exported
functions, none called in-file, the file imported by main.go -- recorded
raw_arch_api 3 + orphans 3 = arch_api 6: three functions, six units of public
surface.

The overlap is the common case rather than the exception. A library's public
functions are exactly the ones with no in-repo caller, so `risk_api_exposure`
and `risk_documentation` (both read the adjusted `api`) were inflated most for
the codebases that are most library-shaped.

detector.py now reports `api_declared_orphans`: how many of the file's orphans
the api rule matched at their own declaration. The test is by NAME, not by span
-- an orphan's name occurs exactly once in the file, which is what the census
just proved, so a name appearing on a line the api rule matched can only be its
own declaration. Span containment would be both looser (C's api rule matches
every non-static local declaration, so any long body suppresses its own
function) and tighter (JS/TS `export` sits outside the slicer's start_idx, php's
span starts a line early, a java `@Test` line pulls start_line off the `public`
one). The conversion then credits only the remainder; the debt wipe stays
unconditional, since an already-public orphan is surface the rule counted
already, not dead weight.

Golden masters re-blessed (both modes). Every one of the 1254 full-precision /
1253 zero-dependency differences is attributable: 61 files lose api mass, and
with them Structural Magnitude, API Exposure %, Documentation Exposure % and
their directory/ecosystem roll-ups, plus the topological coordinates that
re-solve corpus-wide on any mass change. No api count increases anywhere, and
no other structural signal moves. On keyword-rosetta 20 languages go 6 -> 3
(three probe functions, three units of surface) and c goes 12 -> 9 / 10 -> 7;
the corpus re-bless follows in keyword-rosetta.

Fixes #2731. Part of #2669.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol

Copy link
Copy Markdown
Owner Author

CI: everything green except rosetta-audit, which is the expected corpus-side red and is the fix's own confirmation.

All 21 failing languages report only api_orphan_credit expected 3, got 0 (makefile: 1 file) —
64 mismatches, not one of them any other column, and the language list matches the local sweep
exactly. The credit is arch_api - raw_arch_api, so it going to 0 is the double count
disappearing: the api rule counted these declarations, and the orphan conversion no longer counts
them a second time.

Per the no-pins flow the corpus re-blesses against merged main, so this clears with the
keyword-rosetta PR that regenerates the 21 manifests. That PR also needs to replace what
api_orphan_credit was asserting for #2729 (that the main → a → b → c chain resolved) — the
credit is legitimately 0 now wherever every probe is declared public. raw_state_slop_orphans 3
with state_slop_orphans 0 still proves the conversion ran.

crucible-audit (both modes), tree-sitter-accuracy-audit, tri-comparison-audit,
ast-accuracy-audit, full-suite, ruff/mypy/dead-key and all 12 smoke-test matrix cells
pass.

@squid-protocol squid-protocol added the rosetta:rebless-owed Intentionally moves keyword-rosetta counts; audit warns, corpus re-blesses after merge label Sep 5, 2026
@squid-protocol
squid-protocol merged commit 860d091 into main Sep 5, 2026
43 of 44 checks passed
@squid-protocol
squid-protocol deleted the fix/2731-api-orphan-double-count branch September 5, 2026 01:10
@squid-protocol

Copy link
Copy Markdown
Owner Author

Labelled rosetta:rebless-owed — the documented mechanism for an intentional corpus-visible change. rosetta-audit now passes, reporting the same 21 languages as warnings rather than failures, so the PR is fully green. The re-bless is still owed on the corpus side exactly as described above.

squid-protocol added a commit that referenced this pull request Sep 5, 2026
main moved to 074bbef while this branch was open; #2734 (api-rule orphan
surface) rewrote both fixtures wholesale, so the merge conflicted on them.
Resolved by taking main's fixtures and regenerating from scratch on the merged
engine rather than hand-merging -- the two changes are orthogonal, and the
drift against main is again exactly 28 differences, the same jcl-only set as
before the merge (13 files' Commented-out Code counts, their derived Commented
Logic Exposure percentages, and the repo-level average).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8F1cmMsujEjWnnw4d1e4W
squid-protocol added a commit that referenced this pull request Sep 5, 2026
…eam rules (#2732) (#2736)

* feat(core-engine): jcl dead_code + jcl/yaml spec_exposure comment-stream rules (#2732)

Fills the three comment-stream rule absences #2732 found: jcl had neither
dead_code nor spec_exposure, yaml had no spec_exposure. Both jcl absences are
the #2610 leftover -- jcl's comment stream was structurally empty until
_strip_jcl_comments, and #2610 added the debt/ownership rules without revisiting
these two.

jcl/dead_code: commenting a statement out by turning `//` into `//*` is the
universal JCL practice. The operand guard (a keyword must be followed by
operand-shaped text) is load-bearing rather than decorative: cics-genapp's
CICSTS56.jcl carries `//* SET THE RETURN CODE TO CONTROL...` and `//* EXECUTE
DUMP UTILITY PROGRAM...` English banners a few lines from four genuinely
commented-out `//*        DD DSN=CSQ901...` statements. A bare
`(?:EXEC|DD|JOB|SET|INCLUDE)\b` would have counted all of them.

yaml/spec_exposure departs from the shape the issue proposed, and the departure
is the substance of this change. #2732 asked for the generic bracket-tag rule
verbatim, reasoning that "spec_exposure never sees the code stream, so YAML's
[a, b] flow-sequence syntax cannot FP against it". That premise is wrong:
coding_analysis applies every non-underscore rule to the code stream and
comment_analysis then adds a second pass over the comments -- it supplements
the code-stream pass, it does not replace it. Dropped in verbatim, the generic
rule scores spec_exposure=1 on a workflow with no comments at all, off
`needs: [audit, lint]` alone. YAML is the one language where a bracket holding
bare unquoted words is ordinary syntax rather than a tag, so the absence was
never as arbitrary as it looked. Both new spec_exposure rules are therefore
anchored to their language's comment marker (`^[ \t]*#`, `^//\*`) -- the same
anchoring yaml's own dead_code and jcl's own ownership rule already use -- which
makes them structurally comment-only, since prism strips those markers out of
the code stream.

Second measured fix carried into both: the generic rule's bare `spec` branch has
no trailing boundary, so it matches "specified" and "species". 2 of the 3
code-stream hits across the 41,815 pool .yml/.yaml files were exactly that.
Added `\b` after the alternation.

Differential Scan: 28 differences in both modes, all jcl, all attributable to
jcl/dead_code -- 13 files in cics-banking-sample-application-cbsa gaining 16
occurrences (CICSTS56 4, the twelve CREL*/CREDB2L 1 each, matching an
independent count over the 443 licensed .jcl/.prc files in the local pool),
plus the derived Commented Logic Exposure percentages and the repo average. No
coordinate churn, no other language moved, and yaml moved nothing at all --
the anchored rule finds no tag in the corpus, which is why the paired corpus
plant below is required rather than optional. The regenerated fixtures also
carry four machine-specific metadata lines per file (absolute path, remote URL,
timestamp, duration); golden_diff.load_and_sanitize pops all four.

Cross-repo: pairs with keyword-rosetta PR planting the jcl commented-out
statement and the jcl/yaml [SPEC-n] tags, and retiring ledger entry
comment-stream-rules-missing-jcl-yaml. Per docs/GATING.md's n/a semantics,
filling a None rule ends the cell's incomparability whether or not the corpus
has anything to match, so without that plant these three cells would flip from
documented-n/a to a manufactured measured-0. THIS PR MERGES FIRST (rosetta CI
checks out engine main); the corpus PR re-blesses against engine main after.

Closes #2732

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8F1cmMsujEjWnnw4d1e4W

* test: re-bless golden master after merging main (#2732)

main moved to 074bbef while this branch was open; #2734 (api-rule orphan
surface) rewrote both fixtures wholesale, so the merge conflicted on them.
Resolved by taking main's fixtures and regenerating from scratch on the merged
engine rather than hand-merging -- the two changes are orthogonal, and the
drift against main is again exactly 28 differences, the same jcl-only set as
before the merge (13 files' Commented-out Code counts, their derived Commented
Logic Exposure percentages, and the repo-level average).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8F1cmMsujEjWnnw4d1e4W

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
squid-protocol added a commit that referenced this pull request Sep 5, 2026
…2747)

#2729's fix shape asks for the split between the api rule's own count and the
Contextual Baseline Fix's converted orphans to be explicit in the engine/test
surface, because `api` in the risk equations is not the `api` rule. The gate
half landed corpus-side (keyword-rosetta reads `raw_arch_api` as `api` and
pins the conversion separately as `api_orphan_credit`), and #2731/#2734 added
the detector and galaxyscope tests for the conversion itself. This is the last
piece: the recorder test was still documenting the pre-#2731 arithmetic as an
engine invariant.

`test_raw_columns_persist_pre_adjustment_values` asserted
`arch_api == raw_arch_api + raw_state_slop_orphans` and called it "the
Contextual Baseline Fix invariant". It never proved that -- both numbers are
fed in as literals -- and since #2734 it is false in general: the conversion
credits `orphans - api_declared_orphans`, so a file whose api rule already
declared some of its orphans records an adjusted value BELOW that sum, and
total overlap (every public function uncalled -- the library case, and every
file in the keyword-rosetta corpus) makes it `raw_arch_api` exactly. A reader
would take the assertion as the engine's contract, and it is the double count
#2731 removed, written down as an expectation.

Replaced with what the raw columns are actually for: the credit is recoverable
as `adjusted - raw` without assuming what it equals -- the same subtraction
keyword-rosetta's gate reads. Added `test_raw_columns_recover_a_partial_
conversion_credit` for the shape the sum-based assertion excluded by
construction (4 declared + 3 orphans, 2 already public -> adjusted 5, credit
1), which fails against the old sum on purpose.

Header comment now says which layer owns which number, so the next reader does
not re-derive the conversion from the recorder's columns: galaxyscope.py owns
the adjusted value, detector.py owns `api_declared_orphans`, and these tests
own the round-trip only.

No production code touched; no golden-master, tri-comparison or tree-sitter
movement is possible from a test-only change. Full suite 7626 passed.


Claude-Session: https://claude.ai/code/session_01WfeHRWra1d6Z5uSReFajSq

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rosetta:rebless-owed Intentionally moves keyword-rosetta counts; audit warns, corpus re-blesses after merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

arch_api counts a declared-public, uncalled function twice: 3 functions record api 6

1 participant