Skip to content

[M5] Formalize root-cause label schema - #47

Merged
askmy-stack merged 1 commit into
mainfrom
m5-root-cause-label-schema
Jul 14, 2026
Merged

[M5] Formalize root-cause label schema#47
askmy-stack merged 1 commit into
mainfrom
m5-root-cause-label-schema

Conversation

@askmy-stack

Copy link
Copy Markdown
Owner

Summary

  • Add a versioned RootCauseCode / RootCauseLabel Pydantic schema in runtime/schemas/root_cause.py, replacing the ad hoc free-text root_cause strings diagnosis previously produced.
  • RootCauseLabel pairs a RootCauseCode with its FailureCategory and a schema_version, validating that the two stay consistent.
  • GroundTruthBlock.root_cause (case YAML) is now typed as RootCauseCode, so CaseRegistry.from_yaml_file / from_directory reject unknown root-cause codes at load time via BenchmarkCaseModel.model_validate.
  • BenchmarkCase.root_cause_label exposes the resolved, categorized label for a loaded case.

Scope

  • Pydantic model for root-cause labels ✅ (runtime/schemas/root_cause.py)
  • Align case YAML ground_truth.root_cause ✅ (existing tool-semantic-drift-001.yaml value already matches RootCauseCode.PLAN_IDENTIFIER_SEMANTICS_CHANGED, now enforced)

Acceptance criteria

  • Schema in runtime/schemas/
  • Validator used by CaseRegistry

Fixes #27

Test plan

  • make ci (ruff check, ruff format --check, mypy, pytest) passes locally: 18 tests pass, 0 mypy errors.
  • New tests/unit/test_root_cause.py covers: category inference from code, accepting raw strings, rejecting a mismatched category, rejecting unknown codes, CaseRegistry successfully validating the existing case file, and CaseRegistry raising ValidationError for a case file with an unrecognized root_cause.
  • Existing test suite (including tests/unit/test_detectors.py's diagnosis.root_cause == "plan_identifier_semantics_changed" assertion) still passes unchanged since RootCauseCode is a StrEnum.

Made with Cursor

Diagnosis output and case-file ground_truth.root_cause were ad hoc
free-text strings. Add a versioned RootCauseCode/RootCauseLabel
Pydantic schema under runtime/schemas/, use it to type
GroundTruthBlock.root_cause (validated by CaseRegistry on load via
BenchmarkCaseModel.model_validate), and expose the resolved label via
BenchmarkCase.root_cause_label.

Fixes #27

Co-authored-by: Cursor <cursoragent@cursor.com>
@askmy-stack
askmy-stack merged commit f327275 into main Jul 14, 2026
2 checks passed
@askmy-stack
askmy-stack deleted the m5-root-cause-label-schema branch July 14, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[M5] Formalize root-cause label schema

1 participant