Skip to content

Fail if metrics are missing for a query in yaml-tests#4194

Open
hazefully wants to merge 4 commits into
FoundationDB:mainfrom
hazefully:yamltest-fail-on-missing-metrics
Open

Fail if metrics are missing for a query in yaml-tests#4194
hazefully wants to merge 4 commits into
FoundationDB:mainfrom
hazefully:yamltest-fail-on-missing-metrics

Conversation

@hazefully
Copy link
Copy Markdown
Contributor

This PR fixes an issue in the yaml-tests planner metrics checking logic which was introduced in #4128, which makes the yaml-test pass if no planner metrics are already present in the corresponding yamsql metrics files.

In addition, e19905d fixes an issue where the tests for adding result metadata and explain expectations to YAML files was cleaning up the existing explain: and resultMetadata: lines from the input yamsql files, but this cleanup was ineffective because the files were already loaded and copied as resources to the java classpath as part of building yaml-tests, meaning that the yaml-tests framework didn't find that any explains or resultMetadatas need to be added to the file. This meant that running the test locally would fail for the first time if the yamsql files already had the explain/resultMetadata expectation in them, and would then oscillate between passing and failing in each run.

@github-actions
Copy link
Copy Markdown

📊 Metrics Diff Analysis Report

Summary

  • New queries: 9
  • Dropped queries: 0
  • Plan changed + metrics changed: 0
  • Plan unchanged + metrics changed: 0
ℹ️ About this analysis

This automated analysis compares query planner metrics between the base branch and this PR. It categorizes changes into:

  • New queries: Queries added in this PR
  • Dropped queries: Queries removed in this PR. These should be reviewed to ensure we are not losing coverage.
  • Plan changed + metrics changed: The query plan has changed along with planner metrics.
  • Metrics only changed: Same plan but different metrics

The last category in particular may indicate planner regressions that should be investigated.

New Queries

Count of new queries by file:

  • yaml-tests/src/test/resources/arrays-cardinality.metrics.yaml: 8
  • yaml-tests/src/test/resources/check-explain/shouldPass/exact.metrics.yaml: 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug fix Change that fixes a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant