Repository navigation
perf(predict): complete-tree row walks; typed tree metadata for external walkers - #166
Conversation
`RegTree::depth` and `RegTree::has_categorical_splits` describe a tree's shape, and `BoostedModel::tree_outputs(t)` names the outputs tree `t` adds to (one for a scalar-leaf tree, every output for a vector-leaf tree), replacing the crate-private `tree_output`. With `tree_weights` and `shrinkage` they give an external tree walker everything it reads without a JSON round trip. The compact encoder and a SHAP test use `depth` instead of their own walks; Python's `TreeInfo` gains `depth`.
…w methods The single-row methods (`predict_row`, `predict_row_into`, their margin forms) now walk a lazily built `RowForest`: every chunk of 8 consecutive trees that are numeric, constant-leaf, at most 8 levels deep, and split on features below 2^15 is laid out as complete heap-order trees padded to the chunk's deepest tree (u32 threshold keys, u16 feature-and-mirror slots, u32 leaf slots holding the value or a leaf-vector offset), walked 8 trees in lockstep with the arena's key compare, unless padding would exceed twice the chunk's arena bytes. Other chunks walk the model's `CompactForest` by tree id with the same row keys (`walk_row_keyed`), or the `RegTree`s for linear leaves. Leaves still add in tree order onto the same intercepts with the same weights, so margins are bit-identical; a shrunk model's per-iteration shrinks now run between its trees in one walk.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
`BoostedModel::tree_outputs(t)` asserted `t < num_trees()`, a panic on a caller's argument. It is now `tree_outputs()`, one output range per tree in tree order, the shape of `tree_weights()`; internal callers keep the crate-private `tree_output(t)`.
…layout scan The row walk made both keys of every feature a model splits on before each call, whether or not a step read them: a 512-feature model walking one tree (categorical, deeper than 16, or a single-iteration range) or an empty range took up to 3.3x main's time. `RowKeys` now makes the keys on the stack at their first read, and the row walk asks for them only when it walks at least one tree per eight features; otherwise each step keys the value it reads, as wide rows already did. `walk_row` shares `RowKeys` (its policy is unchanged: lockstep groups of rows up to 128 features). `RowForest::from_trees` scans each tree once (features, numeric splits, leaves) after a bounded, allocation-free depth check, takes the features of chunks left on the arena from the arena's tree summaries instead of scanning their nodes, and trims its arrays' capacity. The first row call of a complete-layout model is now faster than main's arena build.
|
@codex review |
|
Codex Review: Didn't find any major issues. Can't wait for the next one! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
🛡️ Codex Security Review · Automatically triggeredSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
Inlining the key-making loop into every read site of `RowKeys::get` cost the arena's lockstep walk its register allocation: depth-10 arena models walked 5% slower than with keys made up front. `RowKeys::make` is now a cold, out-of-line call made once per walk; the reads stay a length check and a flag test.
|
@codex review |
🛡️ Codex Security Review · Automatically triggeredSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
|
Codex Review: Didn't find any major issues. Hooray! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Summary
An external walker that lays a model's trees out as complete trees predicted single rows 2–3× faster than hessboost's row methods. This makes the single-row methods (
predict_row,predict_row_into, their margin forms) beat it, and gives external walkers the typed metadata they had to dig out of the model JSON.perf(predict): aRowForest(tree/compact/row.rs) lays shallow numeric trees out as complete trees, 8 trees per chunk, up to depth 8 and features below 2^15: each level is one compare per lane (step_if_greater), padded paths repeat their leaf, and leaf slots areu32(thef32bits, or an offset into a shared leaf-vector array). A chunk is laid out complete only while its split and leaf-slot bytes stay within twice its trees' arena node bytes; other chunks (deep, categorical, wide-feature, chain-shaped) walk the model's ownCompactForestby tree id with the same row keys, and linear-leaf trees walk theirRegTree. Every path adds the samef32leaves in the same order aspredict, so results are bit for bitpredict's. The layout is built lazily on the first row prediction, in one scan per tree (chunks left on the arena take their features from the arena's tree summaries); after that the row methods allocate nothing and never enter rayon.Row keys (
key(v),key(-v)per feature,RowKeys) are made on the stack at their first read, and only for a walk of at least one tree per eight features (up to 512 features); shorter walks and wider rows key each value as a step reads it.CompactForest::walk_rowsharesRowKeyswith its policy unchanged.predict_rowskeeps the batch blocks.feat(model): typed tree metadata for external walkers, replacing JSON round trips:RegTree::depth,RegTree::has_categorical_splits,BoostedModel::tree_outputs()(each tree's output range, in tree order liketree_weights()), and PythonTreeInfo.depth.Review changes (since the PR opened)
BoostedModel::tree_outputs(t)assertedt < num_trees(), a panic on a caller's argument; it is nowtree_outputs(), an iterator of one range per tree liketree_weights(), and internal callers keeptree_output(t)(the backend files are unchanged from main).RowKeysnow makes keys at their first read, out of line, and only for walks of at least one tree per eight features.BoostedModel::average(feat!: model averaging, out-of-fold cross-validation with per-fold stopping; faster quantile scale pass #165) builds its model withTreeLayouts.Measurements
Two production ensembles, eight single-output models each (p50 µs per row, one pinned core; cold = caches evicted before each call)
predict_margin_row_intopredict_margin_row_intoMedians of three paired runs; the external walker is the same code in both columns.
0 bit mismatches against the external walker and against
predict_marginof a one-row matrix, on both sets. Every tree takes the complete layout.Synthetic shapes,
predict_margin_row_into(p50 µs per call; first call = the call after decoding, which builds the layouts)200 trees of 20 features unless named, trained once and read by both revisions; medians of two paired passes on one pinned core. The first call is slower than main's only where both layouts are built (the lossguide model's chunks are mixed) or the scan finds categorical splits; the production models' first calls are 25–30% faster. 0 bit mismatches against one-row
predict_margin/predicton every shape, and no allocation in a warm call.Layout bytes per tree (complete layout vs the arena): 154 B against ≈ 490 B for the production depth-4 trees (
10·2^D − 6B for depthD).Checks
cargo fmt --all --check, clippy--all-targets --all-features -D warnings(CUDA headers), x86_64 cross clippy,cargo nextest run --cargo-profile ci --features wgpu(838 passed), doctests,RUSTDOCFLAGS=-D warnings cargo doc --all-features, MSRV 1.93cargo build --all-features, Python clippy,uv run --locked pytest, pyright--verifytypes, ruff, ty.