Skip to content

test(integration-tests): port GTest categorization to shared YAML filters - #10243

Open
adickin-amd wants to merge 2 commits into
developfrom
users/addickin/integration_tests_yaml_filters
Open

test(integration-tests): port GTest categorization to shared YAML filters#10243
adickin-amd wants to merge 2 commits into
developfrom
users/addickin/integration_tests_yaml_filters

Conversation

@adickin-amd

Copy link
Copy Markdown
Contributor

Summary

Ports dnn-providers/integration-tests' two local GTest binaries (hipdnn_integration_tests_unit_tests, hipdnn_gpu_ref_tests) from the mixed add_unit_test_target()/add_tiered_test_target() scheme to the shared GTest-filter test_categories.yaml applied via apply_test_category_labels(), matching how miopen-provider categorizes miopen_plugin_tests/miopen_plugin_integration_tests. The hardcoded gpu-ref tiers (quick/standard/comprehensive/full) move into the YAML with identical selection/label semantics; add_tiered_test_target() is no longer used by integration-tests. The externally-driven hipdnn_integration_tests bundle binary is unaffected — it stays categorized by each provider's own test_categories_integration.yaml.

JIRA ID : ALMIOPEN-2278

Risk Assessment

Low risk. This is build/test-infrastructure plumbing confined to CTest registration and category labeling — it does not touch product source, kernel dispatch, or default runtime behavior. The shared-gate broadening in Tests.cmake was verified as a no-op for the three existing providers (they already set both YAML variables); the new categorization was verified end-to-end including a real GPU test run.

ASIC Coverage

ASIC-independent plumbing. The change only affects which CTest suites are generated and how they filter GTest cases by name pattern; it does not add, remove, or alter any op, kernel, or dispatch path. Passing standard PR CI is sufficient — no dedicated multi-arch sweep is required.

Testing Summary

  • Configured the superbuild (miopen-provider preset, includes hipDNN + miopen-provider + integration-tests) and confirmed categorization enables cleanly with no warnings.
  • Enumerated the generated CTest surface and confirmed the exact expected suite set, label cascade, and per-suite --gtest_filter/TIMEOUT/LABELS.
  • Regression-checked the shared Tests.cmake gate change against miopen-provider's existing suites (stash/diff comparison).
  • Built the real binaries and ran the new standard suite on GPU hardware to confirm the GTest filter actually selects only the intended tier.

Testing Checklist

  • Commit hooks - git commit - Status: Passed
  • Superbuild configure (miopen-provider preset) - cmake --preset miopen-provider -B build - Status: Passed (prints "YAML-based CTest categorization enabled" for both hipdnn-integration-tests and miopen-provider, zero warnings)
  • CTest suite enumeration - ctest -N - Status: Passed (9 suites: hipdnn-integration-tests_test_name_validation + 4 hipdnn_integration_tests_unit_tests_*_suite + 4 hipdnn_gpu_ref_tests_*_suite; no legacy add_tiered_test_target names, no unfiltered entries)
  • Label cascade - ctest -N -L quick|standard|full - Status: Passed (3/4/8 tests respectively, matching the YAML's cumulative labels)
  • Generated suite properties - ctest --show-only=json-v1 - Status: Passed (--gtest_filter=Standard*-*DISABLED*, TIMEOUT=1800, LABELS=comprehensive,full,slow,standard for the standard suite; all four tiers match test_categories.yaml)
  • Shared-gate regression - stashed Tests.cmake change, reconfigured, diffed miopen-provider's suite name set with vs. without the change - Status: Passed (byte-identical 28-suite set)
  • Real GPU run - hipdnn_gpu_ref_tests.exe --gtest_filter=Standard*-*DISABLED* via hipdnn-superbuild-test's cmake_run.py - ASICs: gfx1151 - Status: Passed (606/606 tests, all Standard/*, zero Smoke/Quick leakage)
  • PR CI - GitHub PR checks - Status: Pending

Technical Changes

  • dnn-providers/cmake/Tests.cmake: broaden _add_test_target_internal's raw add_test() suppression gate to fire on DNN_PROVIDER_CTEST_CATEGORIES_YAML OR DNN_PROVIDER_TEST_CATEGORY_YAMLS, since integration-tests only sets the latter (it has no external plugin ctest entry).
  • dnn-providers/integration-tests/test_categories.yaml: full rewrite from a CTest-name-regex file to a GTest-filter categories file reproducing add_tiered_test_target()'s quick/standard/comprehensive/full cascade and timeouts.
  • dnn-providers/integration-tests/tests/CMakeLists.txt: register hipdnn_gpu_ref_tests via add_integration_test_target() instead of the now-unused add_tiered_test_target().
  • dnn-providers/integration-tests/CMakeLists.txt: enable YAML categorization before add_subdirectory(gpu-ref|tests), apply apply_test_category_labels() to both binaries after finalize_test_targets() (with install-tree CTestTestfile staging mirroring miopen-provider), and create the working directory up front since it doesn't exist yet on a from-scratch configure.

Switch dnn-providers/integration-tests' two local GTest binaries
(hipdnn_integration_tests_unit_tests, hipdnn_gpu_ref_tests) from the
mixed add_unit_test_target()/add_tiered_test_target() scheme to the
shared GTest-filter test_categories.yaml applied via
apply_test_category_labels(), matching miopen-provider.

- dnn-providers/cmake/Tests.cmake: broaden the raw add_test()
  suppression gate in _add_test_target_internal to also fire on
  DNN_PROVIDER_TEST_CATEGORY_YAMLS (integration-tests has no external
  plugin ctest entry, so it never sets
  DNN_PROVIDER_CTEST_CATEGORIES_YAML). Verified no-op for existing
  providers, which set both variables.
- dnn-providers/integration-tests/test_categories.yaml: full rewrite
  reproducing add_tiered_test_target()'s quick/standard/comprehensive/
  full cascade as GTest filters instead of a CTest-name-regex file.
- dnn-providers/integration-tests/tests/CMakeLists.txt: register
  hipdnn_gpu_ref_tests via add_integration_test_target() instead of
  the now-unused add_tiered_test_target().
- dnn-providers/integration-tests/CMakeLists.txt: enable YAML
  categorization before add_subdirectory(gpu-ref|tests), then apply
  apply_test_category_labels() to both binaries after
  finalize_test_targets(), with install staging mirroring
  miopen-provider. Creates the working directory
  (CMAKE_BINARY_DIR/bin) up front since it does not exist yet on a
  from-scratch configure and apply_test_category_labels() validates
  it eagerly.

Verified via the miopen-provider superbuild preset: configure enables
categorization for both hipdnn-integration-tests and miopen-provider
with no warnings; `ctest -N` lists exactly the 9 expected suites (no
legacy add_tiered_test_target names, no unfiltered entries); -L
quick/standard/full label cascades match spec; generated suite
--gtest_filter/TIMEOUT/LABELS match the YAML. Regression-checked the
shared Tests.cmake gate: miopen-provider's 28-suite set is unchanged
with vs. without the OR. Ran hipdnn_gpu_ref_tests_standard_suite for
real on GPU (gfx1151): 606/606 tests passed, all Standard/*, zero
Smoke/Quick leakage.
Comment thread dnn-providers/integration-tests/tests/CMakeLists.txt
Comment thread dnn-providers/integration-tests/test_categories.yaml
Comment thread dnn-providers/cmake/Tests.cmake Outdated
Comment thread dnn-providers/integration-tests/CMakeLists.txt Outdated
@therock-pr-bot

therock-pr-bot Bot commented Jul 31, 2026

Copy link
Copy Markdown

✅ All Checks Passed — Ready for Review

Check Status Details
📝 PR Description ✅ Pass
Forbidden Files ✅ Pass
🧪 Unit Test ✅ Pass PR does not contain code files — Unit Test auto-passed
🔎 pre-commit ✅ Pass
🚫 Draft PR 🔜 To Be Enabled
🚩 Feature Flag 🔜 To Be Enabled
📊 Code Coverage 🔜 To Be Enabled
🤖 therock-pr-bot ✅ Pass

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🎉 All checks passed! This PR is ready for review.

… fixture

- dnn-providers/cmake/Tests.cmake: remove add_tiered_test_target() and its
  TIERED_TEST_INSTALL_STAGING install-staging path — dead code now that
  integration-tests (its only caller) uses apply_test_category_labels()
  instead. Simplify _add_test_target_internal's YAML gate from
  `DNN_PROVIDER_CTEST_CATEGORIES_YAML OR DNN_PROVIDER_TEST_CATEGORY_YAMLS`
  to just `DNN_PROVIDER_TEST_CATEGORY_YAMLS`: every existing caller that
  sets the former (miopen-provider, hipblaslt-provider, hip-kernel-provider)
  also folds that same YAML path into the latter, so checking one variable
  covers both GTest-filter-only projects and providers with an external
  CTest-name YAML.
- dnn-providers/integration-tests/CMakeLists.txt: drop the
  hipdnn_clear_miopen_test_cache FIXTURES_REQUIRED wiring — neither
  hipdnn_integration_tests_unit_tests nor hipdnn_gpu_ref_tests link
  against or call MIOpen, so the ASAN-only cache-clear fixture doesn't
  apply to them.
- dnn-providers/integration-tests/README.md: update the two stale
  add_tiered_test_target() references (timeout-override note, new-op
  registration example) left over from the port.

Reconfigured and re-verified: integration-tests still enables
categorization and produces the identical 9-suite set with matching
labels/timeouts; miopen-provider's 28-suite set is unchanged after the
gate simplification.
@adickin-amd
adickin-amd marked this pull request as ready for review July 31, 2026 21:28
@adickin-amd
adickin-amd requested a review from a team as a code owner July 31, 2026 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant