Skip to content

fix(core-engine): scheme globals decided by enclosing form, not indentation (#2674) - #2701

Merged
squid-protocol merged 3 commits into
mainfrom
fix/2674-scheme-define-body-position
Sep 3, 2026
Merged

fix(core-engine): scheme globals decided by enclosing form, not indentation (#2674)#2701
squid-protocol merged 3 commits into
mainfrom
fix/2674-scheme-define-body-position

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Closes #2674.

What

Scheme's globals regex matches every (define name value), but in Scheme indentation does not track scope: (define y 5) inside (define (f x) ...) is a local binding and (define y 5) inside a file-wrapping (let () ...) is a real global, at identical indentation. On the language-crucible Chez/Racket sources the bare regex is 42.7% precise, and the #2651 column-0 anchor would have deleted 32 real globals along with 67 false positives (two of five real files have no column-0 defines at all).

This adds a small, registry-declared scope filter hook to coding_analysis and one filter, lisp_body_position:

  • gitgalaxy/standards/language_standards/languages/scheme.py declares "_scope_filters": {"globals": "lisp_body_position"} (a private key, skipped by every rules iterator the same way _args_* / _dependency_capture already are).
  • detector.py runs the filter after the regex over the same segment. It only ever removes matches, so counts, spatial_map and threat_locations stay consistent; an unknown filter name is ignored with a diagnostic, so a registry typo can never zero a metric. The structural pass is cached per segment.
  • _lisp_module_level_define_offsets is a linear tokenizer + form-head stack (strings, #\( char literals, nested #| |#, [ ] all handled). Each define climbs to its nearest non-transparent frame: a module frame (library/module/define-library, or the outermost bindings-less let-family wrapper, either only when no body frame sits above it) keeps it; a body frame (lambda / let-family with bindings / when / cond / … / any define-*) drops it; begin, let-syntax (R6RS 11.18 splicing), if, cond clauses and quoted data are transparent.
  • how_to_add_a_language.md gets engine rule 17: when the discriminator is the enclosing form, declare a scope filter rather than faking it with a column anchor.

Second defect found on the way (Prism, same PR)

#\; is a Scheme char literal ((write-char #\; p), cpnanopass.ss:7690/7805, io.ss:6393). _strip_nested_comments's ; line-comment branch claimed it and swallowed the rest of the line including its closing parens, so every paren-balanced scan downstream ran one level deep for the rest of the file — the first cut of this filter read cpnanopass.ss as 0 globals because of it. Fixed by claiming the char literal atomically before the comment branch (gated to recursive_block_lisp, same mask/unmask path as strings). Pre-existing; it also affected the Mode-B function slicer.

Measurement

Golden master (Prism code streams, i.e. what coding_analysis actually sees); full-precision and zero-dependency move identically:

file globals before after the dropped defines sit in
cpnanopass.ss 26 1 define-pass bodies (12), define-who/lambda bodies (3), meta-cond/with-output-language clauses (8), a local (module …) under a block (let () …) (2)
io.ss 10 6 (define who '…) in procedure bodies and a nested (let () …) block; the 6 kept are the wrapper-level buffer-size constants and open-files
schemify.rkt 6 0 cond/match clause bodies and procedure bodies
thread.rkt 3 1 procedure bodies; the kept one is a top-level callback

Plus one cell from the Prism fix alone: io.ss Closures and Anonymous Functions 454 → 455 (the (lambda () #\;) at io.ss:6393 the stripper used to eat). Attributed by re-scanning with each half alone: Prism-only moves exactly that one cell; the four globals cells belong to the filter.

Against the issue's raw-source paren-depth oracle (53 kept / 71 dropped) the walk deliberately differs in three shapes the oracle's "depth == 1, or any module" proxy could not express:

  • kept — defines under a (begin …) or a let-syntax body inside the wrapper: both splice into the enclosing context (R7RS 5.6.1, R6RS 11.18), so they have the same status as the wrapper's direct defines;
  • dropped — defines inside a (module …) that itself sits inside a nanopass define-pass (a procedure): pass-local state;
  • dropped — defines inside a (module …) nested under a block-scope (let () …): block-local, the same rule the oracle itself applied to syntax.ss's (define who …) runs.

(The issue's raw-source counts also include defines whose "value" was a trailing ; comment, which the code stream never shows the regex — so the golden-master figures above, not the 53/71, are the right bar.)

Verification

  • tests/extraction/languages/test_scheme_strict.py: 13 new end-to-end tests through StructuralExtractor.splice (issue example, 12 body forms, file wrapper, nested block let, begin/let-syntax splicing, library/module/define-library, local module, string/char-literal desync guard, square brackets, count/spatial/location consistency, unknown-filter fallback, linear-time check on 7 pathological payloads incl. 200k parens and an unterminated string).
  • tests/core_engine/test_prism.py: #\; / #\" / #\space survive the stripper verbatim, the real comment after them is still stripped, paren balance preserved.
  • Full suite green; ruff format; ruff / mypy / dead-key audits report nothing new.
  • scope_check.py --expect scheme: all golden-master movement confined to scheme in both modes.
  • tree_sitter_accuracy_audit.py --ci --all: 30/30 OK. tri_comparison_chart.py --ci --all: all OK.
  • keyword-rosetta verify_language.py scheme: PASS (72 assertions) before and after — the corpus plants at top level and cannot see this defect, so no re-bless is owed.
  • Golden master: not yet re-blessed — the five cells above need crucible_check.py --update --yes, which I leave to the user per repo convention.

Cross-repo

None. keyword-rosetta's scheme gate is unchanged against this build; no corpus PR and no ledger entry (the issue's fallback of ledgering scheme globals as a known over-count is superseded by the fix).

🤖 Generated with Claude Code

squid-protocol and others added 2 commits September 3, 2026 18:52
`_strip_nested_comments`'s `;` branch claimed the `#\;` char literal
(`(write-char #\; p)`, cpnanopass.ss:7690/7805, io.ss:6393) and swallowed
the rest of the line including its closing parens, so every paren-balanced
scan downstream -- the Mode-B function slicer and the new #2674 scope
filter -- ran one level deep for the rest of the file.

Claim the char literal atomically before the comment branch, gated to
`recursive_block_lisp`, through the same mask/unmask path as strings so
the code stream keeps it verbatim. `#\"` no longer opens a string either.

Golden master: exactly one cell moves from this half (io.ss closures
454 -> 455, the `(lambda () #\;)` the stripper used to eat).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tation (#2674)

Scheme's `globals` regex matches every `(define name value)`, but the same
indented line is a local binding inside a lambda/let/procedure body and a
real global inside a file-wrapping `(let () ...)` (Chez's cpnanopass.ss:
682 indented defines, none at column 0). The regex was 42.7% precise on
the crucible sources and the #2651 column-0 anchor would have traded 67
false positives for 32 real globals.

Add a registry-declared scope filter hook to coding_analysis:
`"_scope_filters": {"globals": "lisp_body_position"}` in scheme.py, and
`_lisp_module_level_define_offsets` in detector.py -- a linear tokenizer
plus form-head stack that keeps a define only when its nearest classifying
enclosing frame is module scope (top level, library/module/define-library,
or the outermost bindings-less let wrapper) and drops it under any body
form or define-* form. begin / let-syntax / cond clauses are transparent.
The filter only removes matches (counts, spatial map and threat locations
stay consistent), is cached per segment, and an unknown filter name is
ignored rather than zeroing a metric.

Golden master (scheme only, both modes): cpnanopass.ss 26 -> 1, io.ss
10 -> 6, schemify.rkt 6 -> 0, thread.rkt 3 -> 1; every dropped define is
in a procedure body, a nanopass define-pass, a cond clause, or a local
module under a block let. Rosetta scheme gate unchanged (plants are
top-level). how_to_add_a_language.md rule 17 records the pattern.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@squid-protocol squid-protocol added bug Unintended behavior or logic failure in the engine metrics Heuristics, risk exposures, and topological math updates core-engine Modifications to the central physics and parsing engine labels Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

Scope filter: cpnanopass.ss globals 26 -> 1, io.ss 10 -> 6,
schemify.rkt 6 -> 0, thread.rkt 3 -> 1 (internal defines no longer
counted as globals). Prism `#\;` fix: io.ss closures 454 -> 455 (the
`(lambda () #\;)` the comment stripper used to eat). Identical in both
modes; attribution per half in the PR description.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@squid-protocol
squid-protocol merged commit a705345 into main Sep 3, 2026
31 checks passed
@squid-protocol
squid-protocol deleted the fix/2674-scheme-define-body-position branch September 3, 2026 23:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Unintended behavior or logic failure in the engine core-engine Modifications to the central physics and parsing engine metrics Heuristics, risk exposures, and topological math updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

scheme globals: indentation doesn't track scope — the rule is 42.7% precise, and the #2651 col-0 anchor trades 67 false positives for 32 real globals

1 participant