fix(core-engine): scheme globals decided by enclosing form, not indentation (#2674) - #2701
Merged
Merged
Conversation
`_strip_nested_comments`'s `;` branch claimed the `#\;` char literal (`(write-char #\; p)`, cpnanopass.ss:7690/7805, io.ss:6393) and swallowed the rest of the line including its closing parens, so every paren-balanced scan downstream -- the Mode-B function slicer and the new #2674 scope filter -- ran one level deep for the rest of the file. Claim the char literal atomically before the comment branch, gated to `recursive_block_lisp`, through the same mask/unmask path as strings so the code stream keeps it verbatim. `#\"` no longer opens a string either. Golden master: exactly one cell moves from this half (io.ss closures 454 -> 455, the `(lambda () #\;)` the stripper used to eat). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tation (#2674) Scheme's `globals` regex matches every `(define name value)`, but the same indented line is a local binding inside a lambda/let/procedure body and a real global inside a file-wrapping `(let () ...)` (Chez's cpnanopass.ss: 682 indented defines, none at column 0). The regex was 42.7% precise on the crucible sources and the #2651 column-0 anchor would have traded 67 false positives for 32 real globals. Add a registry-declared scope filter hook to coding_analysis: `"_scope_filters": {"globals": "lisp_body_position"}` in scheme.py, and `_lisp_module_level_define_offsets` in detector.py -- a linear tokenizer plus form-head stack that keeps a define only when its nearest classifying enclosing frame is module scope (top level, library/module/define-library, or the outermost bindings-less let wrapper) and drops it under any body form or define-* form. begin / let-syntax / cond clauses are transparent. The filter only removes matches (counts, spatial map and threat locations stay consistent), is cached per segment, and an unknown filter name is ignored rather than zeroing a metric. Golden master (scheme only, both modes): cpnanopass.ss 26 -> 1, io.ss 10 -> 6, schemify.rkt 6 -> 0, thread.rkt 3 -> 1; every dropped define is in a procedure body, a nanopass define-pass, a cond clause, or a local module under a block let. Rosetta scheme gate unchanged (plants are top-level). how_to_add_a_language.md rule 17 records the pattern. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
Scope filter: cpnanopass.ss globals 26 -> 1, io.ss 10 -> 6, schemify.rkt 6 -> 0, thread.rkt 3 -> 1 (internal defines no longer counted as globals). Prism `#\;` fix: io.ss closures 454 -> 455 (the `(lambda () #\;)` the comment stripper used to eat). Identical in both modes; attribution per half in the PR description. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2674.
What
Scheme's
globalsregex matches every(define name value), but in Scheme indentation does not track scope:(define y 5)inside(define (f x) ...)is a local binding and(define y 5)inside a file-wrapping(let () ...)is a real global, at identical indentation. On the language-crucible Chez/Racket sources the bare regex is 42.7% precise, and the #2651 column-0 anchor would have deleted 32 real globals along with 67 false positives (two of five real files have no column-0 defines at all).This adds a small, registry-declared scope filter hook to
coding_analysisand one filter,lisp_body_position:gitgalaxy/standards/language_standards/languages/scheme.pydeclares"_scope_filters": {"globals": "lisp_body_position"}(a private key, skipped by every rules iterator the same way_args_*/_dependency_capturealready are).detector.pyruns the filter after the regex over the same segment. It only ever removes matches, so counts,spatial_mapandthreat_locationsstay consistent; an unknown filter name is ignored with a diagnostic, so a registry typo can never zero a metric. The structural pass is cached per segment._lisp_module_level_define_offsetsis a linear tokenizer + form-head stack (strings,#\(char literals, nested#| |#,[ ]all handled). Each define climbs to its nearest non-transparent frame: a module frame (library/module/define-library, or the outermost bindings-less let-family wrapper, either only when no body frame sits above it) keeps it; a body frame (lambda / let-family with bindings /when/cond/ … / anydefine-*) drops it;begin,let-syntax(R6RS 11.18 splicing),if, cond clauses and quoted data are transparent.how_to_add_a_language.mdgets engine rule 17: when the discriminator is the enclosing form, declare a scope filter rather than faking it with a column anchor.Second defect found on the way (Prism, same PR)
#\;is a Scheme char literal ((write-char #\; p), cpnanopass.ss:7690/7805, io.ss:6393)._strip_nested_comments's;line-comment branch claimed it and swallowed the rest of the line including its closing parens, so every paren-balanced scan downstream ran one level deep for the rest of the file — the first cut of this filter read cpnanopass.ss as 0 globals because of it. Fixed by claiming the char literal atomically before the comment branch (gated torecursive_block_lisp, same mask/unmask path as strings). Pre-existing; it also affected the Mode-B function slicer.Measurement
Golden master (Prism code streams, i.e. what
coding_analysisactually sees); full-precision and zero-dependency move identically:globalsbeforedefine-passbodies (12),define-who/lambda bodies (3),meta-cond/with-output-languageclauses (8), a local(module …)under a block(let () …)(2)(define who '…)in procedure bodies and a nested(let () …)block; the 6 kept are the wrapper-level buffer-size constants andopen-filescond/matchclause bodies and procedure bodiesPlus one cell from the Prism fix alone: io.ss
Closures and Anonymous Functions454 → 455 (the(lambda () #\;)at io.ss:6393 the stripper used to eat). Attributed by re-scanning with each half alone: Prism-only moves exactly that one cell; the fourglobalscells belong to the filter.Against the issue's raw-source paren-depth oracle (53 kept / 71 dropped) the walk deliberately differs in three shapes the oracle's "depth == 1, or any
module" proxy could not express:(begin …)or alet-syntaxbody inside the wrapper: both splice into the enclosing context (R7RS 5.6.1, R6RS 11.18), so they have the same status as the wrapper's direct defines;(module …)that itself sits inside a nanopassdefine-pass(a procedure): pass-local state;(module …)nested under a block-scope(let () …): block-local, the same rule the oracle itself applied tosyntax.ss's(define who …)runs.(The issue's raw-source counts also include defines whose "value" was a trailing
;comment, which the code stream never shows the regex — so the golden-master figures above, not the 53/71, are the right bar.)Verification
tests/extraction/languages/test_scheme_strict.py: 13 new end-to-end tests throughStructuralExtractor.splice(issue example, 12 body forms, file wrapper, nested block let, begin/let-syntax splicing, library/module/define-library, local module, string/char-literal desync guard, square brackets, count/spatial/location consistency, unknown-filter fallback, linear-time check on 7 pathological payloads incl. 200k parens and an unterminated string).tests/core_engine/test_prism.py:#\;/#\"/#\spacesurvive the stripper verbatim, the real comment after them is still stripped, paren balance preserved.ruff format; ruff / mypy / dead-key audits report nothing new.scope_check.py --expect scheme: all golden-master movement confined to scheme in both modes.tree_sitter_accuracy_audit.py --ci --all: 30/30 OK.tri_comparison_chart.py --ci --all: all OK.verify_language.py scheme: PASS (72 assertions) before and after — the corpus plants at top level and cannot see this defect, so no re-bless is owed.crucible_check.py --update --yes, which I leave to the user per repo convention.Cross-repo
None. keyword-rosetta's scheme gate is unchanged against this build; no corpus PR and no ledger entry (the issue's fallback of ledgering scheme
globalsas a known over-count is superseded by the fix).🤖 Generated with Claude Code