Support parallel computing for treeshap function？

[HUI950319]

Feature Request

It would be beneficial to add parallel computing support for the tree function to improve performance.

Background

The tree function can be slow when processing large datasets or performing complex computations.

Proposal

• Evaluate the feasibility of implementing parallel computing for the tree function
• Consider different approaches:
  • Multi-threading
  • Multi-processing
  • GPU acceleration
• Assess potential architecture changes and compatibility issues
• Design an implementation plan if feasible

Questions

• Is parallel computing technically feasible for the tree function in the current architecture?
• What would be the recommended approach for implementation?
• Are there any breaking changes or compatibility concerns?

[mayer79]

The easiest way would be to parallelize the outer for-loop (the one that loops over observations) in the treeshap_cpp() function in https://github.com/ModelOriented/treeshap/blob/master/src/treeshap.cpp:

  for (int obs = 0; obs < x.ncol(); obs++) {
    NumericVector observation = x[obs];
    LogicalVector observation_is_na = is_na[obs];

    NumericVector shaps_row(x.nrow());

    for (int i = 0; i < roots.size(); ++i) {
      Path m;
      recurse(yes, no, missing, feature, is_leaf, value, cover, split, decision_type, observation, observation_is_na, shaps_row,
              m, roots[i], 1, 1, -1,
              0, 0, 1);
    }

    shaps(obs, _) = shaps_row;

#if !defined(WIN32) && !defined(__WIN32) && !defined(__WIN32__)
    if (verbose) {
      updateProgressBar(obs + 1, x.ncol());
    }
#endif
  }

Maybe one could do that with rcppParallel?

[mayer79]

Or we could even move the outer loop outside cpp