proof(conformance): fix corpus to match grammar; wire conformance gate - #43
Merged
Merged
Conversation
Wire the parser conformance corpus into CI and fix the divergences found
when it was first run end-to-end against the real parser.
Root causes fixed:
* run_conformance.sh invoked a non-existent entry point
(`Phronesis.CLI.parse`), so the corpus had never actually executed.
Replaced with `mix run --no-start -e` calling the real
`Phronesis.parse/1` (Lexer.tokenize |> Parser.parse). `--no-start`
avoids booting the ra consensus supervision tree just to parse.
* Four "valid" samples were written against a syntax the language does
not have (per spec/grammar.ebnf v0.2.0, whose source is parser.ex):
- v02/v03/v07 used `IF` as the policy *body*; the grammar requires
`logical_expr THEN action`, with `IF` only as a conditional
*action* after THEN. They also dropped the required `:` on
PRIORITY:/EXPIRES:/CREATED_BY:, used atoms (:valid), and string
metadata.
- v05 used `x = expr` let-bindings, for which there is no production.
Rewritten to the real grammar while preserving each file's intent
(conditional action, AND/OR connectives, IMPORT + module-qualified
call, nested conditional action). Note: parenthesised *logical*
expressions are not in the grammar, so v03 uses the equal-precedence
left-associative form `a AND b OR c` = `((a AND b) OR c)`.
The parser was correct throughout; the corpus and the (also-broken)
runner were the defects. Verified locally against the actual
lib/phronesis/{lexer,parser}.ex: 7/7 valid parse, 3/3 invalid rejected.
New gate conformance.yml runs the corpus on every push/PR, reusing the
estate-vetted checkout + setup-beam pins and the project toolchain
(Elixir 1.16 / OTP 26 per .tool-versions).
https://claude.ai/code/session_01DQACj3RFmAPZaBPgR9SAaS
🔍 Hypatia Security ScanFindings: 30 issues detected
View findings[
{
"reason": "Action trufflesecurity/trufflehog@main needs attention",
"type": "unpinned_action",
"file": "secret-scanner.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "Issue in secret-scanner.yml",
"type": "missing_timeout_minutes",
"file": "secret-scanner.yml",
"action": "flag",
"rule_module": "workflow_audit",
"severity": "medium"
},
{
"reason": "Issue in scorecard-enforcer.yml",
"type": "scorecard_publish_with_run_step",
"file": "scorecard-enforcer.yml",
"action": "split_scorecard_publish_job",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "binary_to_term without :safe option -- deserialization attack (1 occurrences, CWE-502)",
"type": "elixir_send_unsanitised",
"file": "/home/runner/work/phronesis/phronesis/lib/phronesis/compiler.ex",
"action": "flag",
"rule_module": "code_safety",
"severity": "high"
},
{
"reason": "Nominal-only SAST in phronesis: codeql.yml language matrix contains no language present in the repo and lacks `actions`, so CodeQL records zero results on every commit. Remediation: set the CodeQL matrix to `language: actions`.",
"type": "StaticAnalysis",
"file": "/home/runner/work/phronesis/phronesis",
"action": "auto_fix",
"rule_module": "scorecard",
"severity": "medium",
"remediation": "Add CodeQL or equivalent SAST workflow.",
"scorecard_check": "SAST"
},
{
"reason": "Repository has 3 non-main remote branch(es). Policy: single main branch only.",
"type": "GS007",
"file": ".",
"action": "delete_remote_branches",
"rule_module": "git_state",
"severity": "medium"
},
{
"reason": "Code scanning (Hypatia): hypatia/structural_drift/SD007 -- Hypatia structural_drift: SD007 -- 2 day(s) old",
"type": "CSA001",
"file": ".claude/CLAUDE.md",
"action": "review",
"rule_module": "code_scanning_alerts",
"severity": "medium"
},
{
"reason": "Code scanning (Hypatia): hypatia/structural_drift/SD007 -- Hypatia structural_drift: SD007 -- 2 day(s) old",
"type": "CSA001",
"file": ".claude/CLAUDE.md",
"action": "review",
"rule_module": "code_scanning_alerts",
"severity": "medium"
},
{
"reason": "Code scanning (Hypatia): hypatia/structural_drift/SD007 -- Hypatia structural_drift: SD007 -- 2 day(s) old",
"type": "CSA001",
"file": ".claude/CLAUDE.md",
"action": "review",
"rule_module": "code_scanning_alerts",
"severity": "medium"
},
{
"reason": "Code scanning (Hypatia): hypatia/structural_drift/SD007 -- Hypatia structural_drift: SD007 -- 2 day(s) old",
"type": "CSA001",
"file": ".claude/CLAUDE.md",
"action": "review",
"rule_module": "code_scanning_alerts",
"severity": "medium"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes the conformance half of the "wire and conformance" task (the consensus/TLC half landed in #42). Wires the parser conformance corpus into CI and fixes the divergences uncovered when the corpus was first executed end-to-end against the real parser.
Root causes fixed
The runner never actually ran the parser.
conformance/run_conformance.shinvoked a non-existent entry point (Phronesis.CLI.parse). Replaced withmix run --no-start -ecalling the realPhronesis.parse/1(Lexer.tokenize |> Parser.parse).--no-startavoids booting theraconsensus supervision tree just to parse a file.Four "valid" samples were written against a syntax the language does not have (per
spec/grammar.ebnfv0.2.0, whose stated source isparser.ex):v02/v03/v07usedIFas the policy body, but the grammar requireslogical_expr THEN action—IFis only a conditional action afterTHEN. They also dropped the required:onPRIORITY:/EXPIRES:/CREATED_BY:, used atoms (:valid), and string metadata.v05usedx = exprlet-bindings, for which there is no production.Rewritten to the real grammar while preserving each file's intent (conditional action, AND/OR connectives, IMPORT + module-qualified call, nested conditional action).
The parser was correct throughout — the corpus and the (also-broken) runner were the defects.
Interesting / worth noting
factoronly parenthesises arithmetic expressions).v03therefore uses the equal-precedence left-associative forma AND b OR c=((a AND b) OR c), which matches the original grouping intent semantically. Flagging in case grouped boolean conditions are desired as a future language feature.Verification
Verified locally against the actual
lib/phronesis/{token,ast,lexer,parser}.ex(the parse path compiled standalone and run over the corpus): 7/7 valid parse, 3/3 invalid rejected (10/10). The remaining 64lib/modules also compile cleanly standalone, so the CImix compilestep is expected green. (Fullmix compile/mix deps.getcould not be run locally — hex is unavailable in this environment — so the end-to-endmix runpath is exercised in CI.)New gate
.github/workflows/conformance.ymlruns the corpus on every push/PR, reusing the estate-vettedactions/checkout+erlef/setup-beampins and the project toolchain (Elixir 1.16 / OTP 26 per.tool-versions).https://claude.ai/code/session_01DQACj3RFmAPZaBPgR9SAaS
Generated by Claude Code