Skip to content

Unified tracing API: merge ITT/Tracy instrumentation#7250

Open
v4xsh wants to merge 10 commits intoTheHPXProject:masterfrom
v4xsh:unified-tracing-api
Open

Unified tracing API: merge ITT/Tracy instrumentation#7250
v4xsh wants to merge 10 commits intoTheHPXProject:masterfrom
v4xsh:unified-tracing-api

Conversation

@v4xsh
Copy link
Copy Markdown
Contributor

@v4xsh v4xsh commented May 5, 2026

Proposed Changes

  • Replace inline ITT instrumentation in the scheduling loop with hpx::tracing abstractions.
  • Add unified tracing API types and ITT backend implementation under libs/core/tracing.
  • Provide thread init-data helpers to drive ITT task metadata from thread descriptions.

Any background context you want to provide?

This intentionally enforces a single active backend at compile time (Tracy > ITT > none). Previously, ITT and Tracy instrumentation were separate and uncoordinated, so “both on” was incidental rather than a supported dual-profiler feature. The unified tracing layer is designed as a single entry point with compile-time dispatch, avoiding double instrumentation overhead and aligning with typical workflows where only one profiler is used at a time. Is this decision appropriate?

Checklist

Not all points below apply to all pull requests.

  • I have added a new feature and have added tests to go along with it.
  • I have fixed a bug and have added a regression test.
  • I have added a test using random numbers; I have made sure it uses a seed, and that random numbers generated are valid inputs for the tests.

v4xsh added 4 commits May 6, 2026 02:14
Signed-off-by: v4xsh <vanshdobhal11@gmail.com>
Signed-off-by: v4xsh <vanshdobhal11@gmail.com>
Signed-off-by: v4xsh <vanshdobhal11@gmail.com>
Signed-off-by: v4xsh <vanshdobhal11@gmail.com>
Copilot AI review requested due to automatic review settings May 5, 2026 21:04
@v4xsh v4xsh requested a review from hkaiser as a code owner May 5, 2026 21:04
@StellarBot
Copy link
Copy Markdown
Collaborator

Can one of the admins verify this patch?

@codacy-production
Copy link
Copy Markdown

codacy-production Bot commented May 5, 2026

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Signed-off-by: v4xsh <vanshdobhal11@gmail.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a unified hpx::tracing entry point that consolidates Tracy and ITT instrumentation, and refactors the scheduling loop to use the new abstractions instead of inline ITT calls. It also adds helpers to derive ITT task metadata from thread_data descriptions.

Changes:

  • Added ITT backend support to hpx::tracing (compile-time selected when Tracy is not enabled).
  • Replaced inline ITT instrumentation in the scheduling loop with hpx::tracing::{itt_loop_context, itt_task_region}.
  • Added threads::get_thread_region_init_data to translate thread_data descriptions into unified ITT task metadata.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
libs/core/tracing/include/hpx/tracing/tracing.hpp Adds unified tracing API types plus ITT-specific RAII types (and no-op fallbacks).
libs/core/tracing/src/tracing.cpp Implements ITT backend for unified tracing (loop context + task regions).
libs/core/tracing/CMakeLists.txt Links tracing module with hpx_itt_notify when ITTNotify is enabled.
libs/core/threading_base/include/hpx/threading_base/thread_data.hpp Declares get_thread_region_init_data (real vs. no-op depending on ITT support).
libs/core/threading_base/src/thread_data.cpp Implements get_thread_region_init_data using thread_description (string handle vs. address).
libs/core/thread_pools/include/hpx/thread_pools/scheduling_loop.hpp Replaces inline ITT code with unified tracing RAII objects in the scheduler hot path.

Comment thread libs/core/tracing/src/tracing.cpp Outdated
Comment thread libs/core/thread_pools/include/hpx/thread_pools/scheduling_loop.hpp Outdated
Copy link
Copy Markdown
Contributor

@hkaiser hkaiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can make sure (on the build system level) that there is at most one of the tracing backends enabled at any particular time. This should allow us to modify the needed data types as needed (no separate init-data type names for the different backends may be needed this way).

Comment thread libs/core/thread_pools/include/hpx/thread_pools/scheduling_loop.hpp Outdated
Comment thread libs/core/thread_pools/include/hpx/thread_pools/scheduling_loop.hpp Outdated
@v4xsh
Copy link
Copy Markdown
Contributor Author

v4xsh commented May 5, 2026

Thanks for the review @hkaiser! Both suggestions make sense. Here’s what I’m planning to do.

I’ll rename itt_loop_context to loop_context, since only one backend is active at compile time and a generic name fits better.

I’ll merge itt_task_region into region. The region constructor will take a loop_context&, and it will handle both Tracy zones and ITT task instrumentation depending on the active backend.

I’ll also merge thread_region_init_data into region_init_data, and unify get_region_init_data() and get_thread_region_init_data() into a single function.

I’ll drop the pimpl. Since backend-specific headers can be included inside the relevant #if branches in tracing.hpp, the ITT types can be stored directly as members. This avoids heap allocation and removes the need for make_unique.

With these changes, the scheduling loop becomes:

hpx::tracing::loop_context ctx;
// ...
hpx::tracing::region rctx(ctx, threads::get_region_init_data(thrdptr), num_thread);

I’ll update the PR accordingly. Let me know if this aligns with what you had in mind.

@v4xsh v4xsh force-pushed the unified-tracing-api branch from be8fe12 to 5456325 Compare May 5, 2026 21:30
Copilot AI review requested due to automatic review settings May 5, 2026 21:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comment thread libs/core/tracing/include/hpx/tracing/tracing.hpp Outdated
Comment on lines +567 to +569
return {desc.get_description(), thrdptr->get_thread_phase(),
thrdptr, thrdptr->is_stackless(), 0, false,
desc.get_description_itt().handle_};
v4xsh added 2 commits May 6, 2026 12:18
Signed-off-by: v4xsh <vanshdobhal11@gmail.com>
Signed-off-by: v4xsh <vanshdobhal11@gmail.com>
@v4xsh
Copy link
Copy Markdown
Contributor Author

v4xsh commented May 6, 2026

Kindly have a look.

Also I noticed a possible pre-existing edge case: the Tracy path calls thread_description::get_description() without checking kind(). If the description is address-type, this might assert or produce an invalid name. I’m not completely sure how often that path is hit in practice. It’s currently at thread_data.cpp. Would it be correct to adjust this (e.g., use thread_data::get_tracy_description_name(...) or fall back to threads::as_string(...))?

Copy link
Copy Markdown
Contributor

@hkaiser hkaiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be a good idea to have a dedicated CI for testing the ITTNotify support. That is more involved however, as we can't install VTune in that CI environment. Installing a 'fake' ITTNotify consumer may be viable, though (https://github.com/intel/ittapi).

Comment thread libs/core/threading_base/src/thread_data.cpp Outdated
Comment thread libs/core/threading_base/include/hpx/threading_base/thread_data.hpp Outdated
v4xsh added 2 commits May 6, 2026 22:39
Signed-off-by: v4xsh <vanshdobhal11@gmail.com>
Signed-off-by: v4xsh <vanshdobhal11@gmail.com>
Copilot AI review requested due to automatic review settings May 6, 2026 17:09
@v4xsh
Copy link
Copy Markdown
Contributor Author

v4xsh commented May 6, 2026

It might be a good idea to have a dedicated CI for testing the ITTNotify support. That is more involved however, as we can't install VTune in that CI environment. Installing a 'fake' ITTNotify consumer may be viable, though (https://github.com/intel/ittapi).

Agreed I'll add ITTNotify CI coverage using the intel/ittapi fake consumer in a follow-up PR.

Signed-off-by: v4xsh <vanshdobhal11@gmail.com>
Comment thread CMakeLists.txt
hpx_error(
"HPX_TRACY_WITH_TRACY=ON and HPX_WITH_ITTNOTIFY=ON are mutually exclusive. Only one tracing backend can be active."
)
endif()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please unify the error messages generated if more than one tracing backend is active? Also, your testing doesn't check for HPX_TRACY_WITH_TRACY and HPX_WITH_APEX being enabled at the same time (there might be other combinations we would like to avoid).

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants