[M5] Formalize root-cause label schema - #47
Merged
Conversation
Diagnosis output and case-file ground_truth.root_cause were ad hoc free-text strings. Add a versioned RootCauseCode/RootCauseLabel Pydantic schema under runtime/schemas/, use it to type GroundTruthBlock.root_cause (validated by CaseRegistry on load via BenchmarkCaseModel.model_validate), and expose the resolved label via BenchmarkCase.root_cause_label. Fixes #27 Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RootCauseCode/RootCauseLabelPydantic schema inruntime/schemas/root_cause.py, replacing the ad hoc free-textroot_causestrings diagnosis previously produced.RootCauseLabelpairs aRootCauseCodewith itsFailureCategoryand aschema_version, validating that the two stay consistent.GroundTruthBlock.root_cause(case YAML) is now typed asRootCauseCode, soCaseRegistry.from_yaml_file/from_directoryreject unknown root-cause codes at load time viaBenchmarkCaseModel.model_validate.BenchmarkCase.root_cause_labelexposes the resolved, categorized label for a loaded case.Scope
runtime/schemas/root_cause.py)ground_truth.root_cause✅ (existingtool-semantic-drift-001.yamlvalue already matchesRootCauseCode.PLAN_IDENTIFIER_SEMANTICS_CHANGED, now enforced)Acceptance criteria
runtime/schemas/Fixes #27
Test plan
make ci(ruff check, ruff format --check, mypy, pytest) passes locally: 18 tests pass, 0 mypy errors.tests/unit/test_root_cause.pycovers: category inference from code, accepting raw strings, rejecting a mismatched category, rejecting unknown codes,CaseRegistrysuccessfully validating the existing case file, andCaseRegistryraisingValidationErrorfor a case file with an unrecognizedroot_cause.tests/unit/test_detectors.py'sdiagnosis.root_cause == "plan_identifier_semantics_changed"assertion) still passes unchanged sinceRootCauseCodeis aStrEnum.Made with Cursor