Skip to content

feat(core-engine): jcl api, cleanup and globals rules; high_risk_execution narrowed to the command executors (#2748–#2751) - #2755

Merged
squid-protocol merged 4 commits into
mainfrom
fix/jcl-rosetta-rules
Sep 5, 2026
Merged

feat(core-engine): jcl api, cleanup and globals rules; high_risk_execution narrowed to the command executors (#2748–#2751)#2755
squid-protocol merged 4 commits into
mainfrom
fix/jcl-rosetta-rules

Conversation

@squid-protocol

@squid-protocol squid-protocol commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Closes #2748
Closes #2749
Closes #2750
Closes #2751
Part of #2669.

Four rosetta gaps in one language, found by the 2026-09-05 pass over every non-green cell of the
keyword-rosetta bias report. Each was a "should be able to detect this, doesn't" cell whose
intended-morphology verdict was written before #2742/#2743 changed the engine's overlap posture.

The four rules

issue rule before after (crucible, 186 files)
#2748 api None //name PROC — a cataloged or in-stream procedure, the callable surface EXEC name / EXEC PROC=name invokes (api contract, fallback family). 13 declarations in 13 files; 185 of 376 EXEC steps call one
#2749 cleanup None (#2610 declined it for the io overlap) DELETE as the normal-termination disposition — DISP=(MOD,DELETE,DELETE), (OLD,DELETE), (,DELETE). 36 of 533 DISP= in 13 files. The abend-only positional of an allocation (DISP=(NEW,CATLG,DELETE), 12) is excluded: a conditional disposition on a create is not teardown
#2750 globals None //JOBLIB DD (job-wide; STEPLIB is the step-scoped twin and does not count), a // SET symbol (every later statement; a PROC parameter is the scoped twin), // EXPORT SYMLIST. 73 hits in 45 files
#2751 high_risk_execution bare PGM=<anything>: 188 hits, every step the programs that execute caller-supplied commands: IKJEFT01/1A/1B, BPXBATCH/BPXBATSL/BPXBATA2/BPXBATA8/AOPBATCH, IRXJCL, SDSF. 61 hits in 48 files, all IKJEFT01 on this corpus

Why #2610's rejection of cleanup does not hold any more. #2742 added sync_locks on the same
operand with an accepted, test-pinned io overlap, on the grounds that OLD/MOD is a narrow,
semantically distinct subset of DISP= (~9%). DELETE in the normal position is the same kind of
subset (~7%) and is the language's teardown idiom outright — IEFBR14 + DISP=(MOD,DELETE,DELETE)
is how a batch job deletes a dataset. Every hit is also an io hit (the DD's DSN=) and the
OLD/MOD forms are sync_locks hits too; test_jcl_cleanup_overlaps_io_and_sync_locks_by_design
pins all three.

Why narrowing high_risk_execution is a consistency fix, not a loss. No other language's rule
counts "runs a command" — shell/dockerfile/python count eval/exec/rm -rf /, not every command
line. JCL's counted every step: a compile-link-go job scored three high-risk executions for
compiling, and IEFBR14 (a program that does nothing) scored the same as a TSO batch step that
executes whatever SYSTSIN carries. The rosetta corpus had been avoiding the rule ("probes use
the EXEC form") rather than measuring it. Compilers, copy/catalog utilities and IEFBR14
are steps; IDCAMS/IEHPROGM/ADRDSSU are destructive-capable but execute a fixed command
language and are left out on the reasoning that keeps rm (not rm -rf /) out of shell's rule —
that boundary is the one open design question, recorded in the rule's comment.

globals' overlaps are precedented: SET is dual globals + state_mutation, dockerfile's ENV
shape exactly; JOBLIB's DD line is an io hit and a dependency edge, which is right — a JOBLIB
is a dependency of every step.

Verification

  • tests/extraction/languages/test_jcl_strict.py: 80 → 100 cases (+20, each citing its issue):
    per-rule partitions with the crucible's most frequent lookalikes as negatives (STEPLIB,
    EXEC PROC=, //SYSPROC DD, DISP=(NEW,CATLG,DELETE), IEFBR14, the compilers, a SET inside
    a SYSIN payload, a DELETE in a dataset name or a PARM), overlap pins for io/sync_locks/
    state_mutation, one end-to-end deck through prism + StructuralExtractor.splice asserting all four
    counts at once, and ReDoS detonations for every new alternation. Every case was run against the
    real compiled regex before being written down; jcl gauntlet + prism + function-extraction files
    all pass (216 passed).
  • Isolated rule probe over real Prism code streams on both corpora (raw file == code stream for
    every rule, no comment-stream leakage): the crucible counts in the table above; rosetta
    api 0 / cleanup 0 / globals 2 / high_risk_execution 0 before the paired plants (see
    Cross-repo).
  • audit_check.py — ruff (70-finding baseline, format clean), mypy, dead-key, ast-accuracy all
    clear. Files formatted with the CI-pinned ruff@0.16.0.
  • tree_sitter_accuracy_audit --ci --all — 30/30 OK. tri_comparison_chart --all --ci — 3/3 OK.
    (Neither covers jcl, but both gate this file.)
  • rosetta_audit.py46 checked, 1 regression, 0 pre-existing, 0 broken: the regression is
    jcl, this PR's own four cells, hence the rosetta:rebless-owed label.
  • Full suite: see the checks on this PR (the serial local run is recorded in the first comment).

Golden masters

Regenerated the zero-dependency audit from this branch and diffed it uncapped with
tests/golden_diff.deep_compare (the crucible_check.py printer caps at 50):

class count
Topological Coordinates X/Y/Z (corpus-wide re-solve) 428
substantive, all jcl — 418 per-file cells in .jcl/.prc files + 47 jcl directory-group / composition aggregates 465
off-target files (non-jcl) 0

Per-file field breakdown of the 418: Error & Exception Exposure 84 and Testing Exposure 79
(both read high_risk_execution, which drops from 188 to 61 — a step whose only "danger" was
PGM=IEFBR14 now reads safety 0.0% instead of 76.85%), High-Risk Execution Commands 78 and its
security-frame mirror Sec Tainted Injection 77, Global State Dependencies 45, Documentation Exposure / API Exposure 17 each and Public Exports 13 (the PROC statements), Resource Deallocation & Cleanup 13, Structural Magnitude 13. Direction matches intent everywhere:
narrowed rule negative, three new rules positive.

Blessed. Both fixtures regenerated with crucible_check.py --update --yes and committed in the second commit; the CI crucible-audit diff before the bless matched the local measurement above (jcl-only fields, zero off-target files).

Cross-repo

This PR is corpus-visible per keyword-rosetta AGENTS.md rule 8 (three None rules filled, one
narrowed), so it carries rosetta:rebless-owed and merges first per GATING.md's no-pins flow. The
paired corpus PR is prepared on keyword-rosetta branch corpus/2748-2751-jcl-rules (companion PR
opened as a draft, linked below): //ROSPROC PROC// PEND around STEP2 in main.jcl (api 1),
two teardown DDs on c.jcl's cleanup step — one of which is the DISP=OLD plant #2742 owed —
(cleanup 2, sync_locks 1, io +2 ledgered), PGM=IKJEFT01 / PGM=BPXBATCH in place of IEFBR14
(high_risk_execution stays 2), and globals 2 from b.jcl's existing SET lines with no new
plant. verify_language.py jcl passes against this branch (88 assertions); na_check.py --ci
clean; ledger: jcl-sync-locks-disp-enq-question closed, jcl-2610-rebaseline-residual-morphology
narrowed, api-no-rule-defined narrowed to markdown. Tracking: keyword-rosetta#59.

🤖 Generated with Claude Code

…ution narrowed to executors (#2748, #2749, #2750, #2751)

Four rosetta gaps in one language, found by the 2026-09-05 pass over every
non-green cell of the keyword-rosetta bias report and filed as #2748-#2751.

- api: `//name PROC` -- a cataloged or in-stream procedure is the callable
  surface `EXEC name` / `EXEC PROC=name` invokes (api contract, fallback
  family). 13 declarations in 13 crucible files; 185 of 376 EXEC steps call one.
- cleanup: DELETE as a dataset's normal-termination disposition
  (`DISP=(MOD,DELETE,DELETE)`, `(OLD,DELETE)`, `(,DELETE)`) -- JCL's teardown
  idiom. #2610 declined it for the io overlap; #2742 reversed that posture for
  sync_locks and the same narrow-subset measurement applies (36 of 533 DISP=).
  The abend-only positional of an allocation is excluded.
- globals: JOBLIB (vs STEPLIB), a job-level SET symbol (vs a PROC parameter)
  and EXPORT SYMLIST -- the scoped-vs-global distinction the ledger said JCL
  lacked. SET is dual globals+state_mutation, dockerfile's ENV shape.
- high_risk_execution: a bare `PGM=` counted every step (188 of 376 crucible
  EXECs, IEFBR14 and the compilers included). Narrowed to the programs that
  execute caller-supplied commands: IKJEFT01/1A/1B, BPXBATCH/BPXBATSL/BPXBATA2/
  BPXBATA8/AOPBATCH, IRXJCL, SDSF. 61 hits in 48 files after.

20 new strict cases: per-rule partitions, overlap pins (io/sync_locks/state),
an end-to-end deck through prism + splice, ReDoS detonations. Docs: jcl status
sections 1/3/4/6/10 and the api contract's fallback family + audit table.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@squid-protocol

Copy link
Copy Markdown
Owner Author

Local serial full suite on this branch (python -m pytest tests/, the mode CI runs): 7647 passed, 3 skipped, 9 xfailed, 3 xpassed, 1 failed in 6m27s.

The one failure is test_language_strictness.py::test_committed_fidelity_table_is_fresh_against_the_corpus, and it is pre-existing, not this PR's: gitgalaxy/standards/fidelity_table.py is untouched by this branch (git diff origin/main -- fidelity_table.py is empty), and the check compares the committed table (generated 2026-09-04 at corpus 6377eb5) against the sibling checkout's docs/bias_data.json, which bias-history.yml has since regenerated (corpus 2d18562, engine 0e521e8). The test is skipif-gated on the corpus being checked out as a sibling, so CI does not run it; the same failure reproduces on main locally. Regenerating the table is its own housekeeping item — and it will need doing again after keyword-rosetta#60 lands and the bot regenerates the report, since jcl's api/cleanup/globals coefficients are exactly the kind of cell the table carries.

@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

squid-protocol and others added 3 commits September 5, 2026 14:14
…es (#2748-#2751)

Both fixtures regenerated with crucible_check.py --update. Every substantive
difference is in jcl files or jcl directory aggregates (zero off-target files);
the rest is the corpus-wide topological re-solve. Field breakdown and direction
of change are in the PR description.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…an-census fix

Main re-blessed for #2754 after this branch's first bless, so the fixtures
conflicted. Merged main (taking its fixtures) and regenerated both from the
merged tree. Scoped against main: 893 differences, 428 topological, 465
substantive -- all jcl files / jcl aggregates plus the two corpus-wide health
averages they feed; zero off-target files. Same shape as the first bless.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@squid-protocol
squid-protocol merged commit c7d2a12 into main Sep 5, 2026
31 checks passed
@squid-protocol
squid-protocol deleted the fix/jcl-rosetta-rules branch September 5, 2026 18:26
squid-protocol added a commit that referenced this pull request Sep 5, 2026
Conflict was the two golden-master fixtures again, both auto-generated: #2755
(jcl api/cleanup/globals) and #2759 (yaml args) re-blessed them on main while
this branch was in flight. `gitgalaxy/core/detector.py` merged cleanly -- their
changes are in the rules, this one is in the Mode D slicer.

Resolved per .agents/skills/ci-push-checklist section 6: took origin/main's
version of both fixtures to clear the markers, then re-ran
`crucible_check.py --update --yes` so they are recalculated from the merged
code rather than stitched together.

Verified rather than trusted: re-attributed the regenerated fixture against
origin/main and got the same profile as before the merge, exactly -- 20
differences, 0 topology movement, `Start Line` the only field changed inside
`5. Function Analysis`, and all 17 per-file diffs on `.sh` files. #2755's and
#2759's own contributions are in the base and untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WfeHRWra1d6Z5uSReFajSq
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core-engine Modifications to the central physics and parsing engine metrics Heuristics, risk exposures, and topological math updates rosetta:rebless-owed Intentionally moves keyword-rosetta counts; audit warns, corpus re-blesses after merge

Projects

None yet

1 participant