Skip to content

fix(trees): make DecisionTree::fit deterministic - #452

Open
mysma-9403 wants to merge 1 commit into
rust-ml:masterfrom
mysma-9403:fix/decision-tree-determinism
Open

fix(trees): make DecisionTree::fit deterministic#452
mysma-9403 wants to merge 1 commit into
rust-ml:masterfrom
mysma-9403:fix/decision-tree-determinism

Conversation

@mysma-9403

Copy link
Copy Markdown

DecisionTree::fit is not reproducible: repeated calls on identical data return differently shaped trees.

On the iris dataset, 60 fits in a single process produce 4 distinct trees, differing in leaf count (9, 10, 11, 12) and therefore in their predictions:

L[Some(0)@1,Some(1)@3,Some(2)@3,Some(2)@4,...]   (11 leaves)
L[Some(0)@1,Some(2)@2,Some(1)@3,Some(2)@4,...]   ( 9 leaves)
L[Some(0)@1,Some(2)@2,Some(2)@4,Some(1)@4,...]   (10 leaves)
L[Some(0)@1,Some(2)@3,Some(2)@4,Some(1)@4,...]   (12 leaves)

Cause

Class frequencies were carried in a HashMap<L, f32>, and HashMap iteration order is randomised per instance. Both consumers of those frequencies depend on the order:

  1. find_modal_class folded with if best_freq > freq { acc } else { Some((idx, freq)) }, so on a tie it kept the last maximum it happened to visit — the opposite of the "first class found with that frequency is returned" its own doc comment promises.

  2. gini_impurity and entropy sum the frequencies in f32, and floating-point addition is not associative. The same frequencies summed in a different order give slightly different scores.

The second one is what changes the tree shape: those small score differences flip the score < best_score comparison in TreeNode::fit at near-ties, selecting a different split and changing the whole subtree below it.

Fix

Frequencies become Vec<f32> indexed by position in a sorted class list (ClassIndex) built once per fit, so iteration order is pinned. find_modal_class now also uses >= so that ties resolve to the first class, matching its documentation.

The tree this produces is one of the trees the previous code already produced — I verified this by collecting the fingerprint set of 60 fits on master and checking that the now-deterministic output is a member of it. This changes which of several equally-valid trees you get, consistently, rather than introducing a new one.

Side effect: it is also faster

The innermost split-search loop ran, for every (sample, feature) pair:

*right_class_freq.get_mut(sample_class).unwrap() -= sample_weight;
*left_class_freq.entry(sample_class.clone()).or_insert(0.0) += sample_weight;

Two hash lookups plus a label clone — for String labels, a heap allocation. Indexing a Vec removes all three.

n d classes before after speedup
2 000 20 3 707.2 ms 271.5 ms 2.6×
8 000 20 3 2570.1 ms 860.9 ms 3.0×
8 000 20 10 2198.7 ms 861.1 ms 2.6×
16 000 50 5 14652.5 ms 4494.8 ms 3.3×

Benchmark context

  1. Plugged in (AC power, battery 100 %, charged)
  2. Power saving mode: off (lowpowermode 0)
  3. Machine otherwise idle
  4. Not thermally throttled
  5. MacBookPro15,3 — Intel Core i9-9980HK @ 2.40 GHz, 8 cores / 16 threads, 32 GB RAM, macOS 15.7.7, rustc 1.93.1
  6. Best of 5 runs of DecisionTree::params().fit(&ds), default hyperparameters, measured with a throwaway harness (not part of this PR); A/B taken on this branch with and without the change so the binary shape is identical

Notes

  • ClassIndex::frequencies replaces DatasetBase::label_frequencies_with_mask inside the tree. The only behavioural difference is that classes absent from a node are present with frequency 0.0 instead of being omitted, which is inert: a zero frequency contributes nothing to either impurity measure, and can never be the modal class of a non-empty node.
  • Added fit_is_deterministic as a regression test, plus modal_class_ties_pick_the_first_class to pin the documented tie-break.
  • The three existing unit tests for gini_impurity/entropy/find_modal_class were updated for the new slice signatures; their expected values are unchanged.

Checks

  • cargo fmt --all -- --check — clean
  • cargo clippy --workspace --all-targets -- -D warnings — clean
  • cargo test --release --workspace — 546 tests, no failures

🤖 Generated with Claude Code

https://claude.ai/code/session_01L4uUK6s1CWFwx6u7Rx28go

Repeated `fit` calls on identical data could return differently shaped
trees. On the iris dataset, 60 fits in one process produced 4 distinct
trees, differing in leaf count (9, 10, 11 and 12) and therefore in their
predictions.

Class frequencies were carried in a `HashMap<L, f32>`, and `HashMap`
iteration order is randomised per instance. Every consumer of those
frequencies depended on the order:

  * `find_modal_class` folded with `best_freq > freq`, so on a tie it kept
    the *last* maximum it happened to visit — the opposite of the "first
    class found" its doc comment promises.
  * `gini_impurity` and `entropy` sum the frequencies in `f32`, and
    floating-point addition is not associative, so the same frequencies
    summed in a different order produce slightly different scores.

Those score differences flip the `score < best_score` comparison in
`TreeNode::fit` at near-ties, which selects a different split and changes
the shape of the whole subtree.

Frequencies are now `Vec<f32>` indexed by position in a sorted class list
built once per fit, so the iteration order is pinned. The tree this
produces is one of the trees the previous code already produced, just
consistently rather than at random.

As a side effect this is also faster: the innermost split-search loop ran
`right_class_freq.get_mut(class)` and `left_class_freq.entry(class.clone())`
for every (sample, feature) pair, i.e. two hash lookups plus a label clone,
which for `String` labels meant an allocation. Indexing a `Vec` removes all
three.

  n=2000  d=20 k=3     707.2 ms ->  271.5 ms  (2.6x)
  n=8000  d=20 k=3    2570.1 ms ->  860.9 ms  (3.0x)
  n=8000  d=20 k=10   2198.7 ms ->  861.1 ms  (2.6x)
  n=16000 d=50 k=5   14652.5 ms -> 4494.8 ms  (3.3x)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4uUK6s1CWFwx6u7Rx28go
@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.04%. Comparing base (7fe5c86) to head (4a52706).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #452      +/-   ##
==========================================
+ Coverage   77.53%   78.04%   +0.51%     
==========================================
  Files         106      104       -2     
  Lines        7585     7543      -42     
==========================================
+ Hits         5881     5887       +6     
+ Misses       1704     1656      -48     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant