perf(kernel): avoid per-element allocation and exploit kernel symmetry

[mysma-9403]

Dense kernel construction was doing roughly twice the necessary work, and the inner product was allocating once per matrix entry.

1. a.mul(&b).sum() → a.dot(&b)

KernelMethod::distance used a.mul(&b).sum() for the Linear and Polynomial kernels. a.mul(&b) builds a fresh Array1 for every one of the n² entries of the kernel matrix. That is why the Linear kernel — which does strictly less arithmetic than Gaussian, no exp — was measurably slower than it. a.dot(&b) computes the same value without allocating.

2. dense_from_fn only needs the upper triangle

Every KernelMethod is symmetric in its two arguments:

method	expression	symmetric?
Gaussian(eps)	exp(-Σ(xᵢ-yᵢ)² / eps)	yes
Linear	⟨a, b⟩	yes
Polynomial(c, d)	(⟨a, b⟩ + c)^d	yes

so evaluating only j >= i and mirroring halves the work. The matrix was also initialised with Array2::eye, whose values are then entirely overwritten by the loop, so that is now Array2::zeros.

Both changes are value-preserving. The new dense_from_fn_is_symmetric test guards the symmetry property that the second one relies on, for all three methods.

3. Benchmarks

linfa-kernel had no benchmarks, so this adds a criterion bench following CONTRIBUTE.md (config::set_default_benchmark_configs, pprof profiler, constant seed, data passed as an argument).

One deliberate deviation from the guidelines: sample sizes are [1_000, 2_000, 4_000] rather than [1_000, 10_000, 20_000], because a dense kernel is quadratic in memory — 20 000 samples would need ~3.2 GB for the matrix alone. This is noted in a comment in the bench.

---

Benchmark results

Context

1. Plugged in (AC power, battery 100 %, charged)
2. Power saving mode: off (lowpowermode 0)
3. Machine otherwise idle
4. Not thermally throttled
5. MacBookPro15,3 — Intel Core i9-9980HK @ 2.40 GHz, 8 cores / 16 threads, 32 GB RAM, macOS 15.7.7, rustc 1.93.1
6. Default (pure-Rust) backend, no BLAS feature

Bench command run

# before
git checkout master -- algorithms/linfa-kernel/src/lib.rs
cargo bench -p linfa-kernel --bench kernel -- --save-baseline before 2000samples
# after
cargo bench -p linfa-kernel --bench kernel -- --baseline before 2000samples

Restricted to the 2000samples subset to keep the A/B run bounded; the committed bench covers the full grid.

Kernel/Dense-Gaussian-3feats/2000samples
                        time:   [43.294 ms 44.099 ms 45.074 ms]
                        change: [-49.359% -48.003% -46.592%] (p = 0.00 < 0.05)
                        Performance has improved.
Kernel/Dense-Gaussian-8feats/2000samples
                        time:   [53.654 ms 54.183 ms 54.823 ms]
                        change: [-77.857% -76.580% -75.300%] (p = 0.00 < 0.05)
                        Performance has improved.
Kernel/Dense-Linear-3feats/2000samples
                        time:   [17.647 ms 17.925 ms 18.240 ms]
                        change: [-95.700% -95.589% -95.474%] (p = 0.00 < 0.05)
                        Performance has improved.
Kernel/Dense-Linear-8feats/2000samples
                        time:   [20.976 ms 21.452 ms 21.981 ms]
                        change: [-95.828% -95.663% -95.491%] (p = 0.00 < 0.05)
                        Performance has improved.
Kernel/Dense-Polynomial-3feats/2000samples
                        time:   [88.710 ms 91.267 ms 94.256 ms]
                        change: [-87.105% -86.643% -86.152%] (p = 0.00 < 0.05)
                        Performance has improved.
Kernel/Dense-Polynomial-8feats/2000samples
                        time:   [85.947 ms 87.571 ms 89.304 ms]
                        change: [-88.019% -87.591% -87.142%] (p = 0.00 < 0.05)
                        Performance has improved.

Summary at n = 2000:

kernel	features	before	after	speedup
Linear	8	494.6 ms	21.5 ms	23.1×
Linear	3	406.4 ms	17.9 ms	22.7×
Polynomial	8	705.7 ms	87.6 ms	8.1×
Polynomial	3	683.3 ms	91.3 ms	7.5×
Gaussian	8	231.4 ms	54.2 ms	4.3×
Gaussian	3	84.8 ms	44.1 ms	1.9×

Gaussian gains the least because it never used mul() — only the symmetry change applies to it.

End-to-end effect on Svm::fit

Since linfa-svm builds a dense kernel on every fit, measured with a separate throwaway harness (not part of this PR), accuracy identical in every case:

kernel	n / d	before	after	speedup
linear	1000 / 8	370.3 ms	25.9 ms	14.3×
linear	2000 / 50	4.4 s	394.5 ms	11.2×
polynomial	2000 / 8	2.3 s	318.2 ms	7.2×
gaussian	2000 / 50	3.4 s	1.7 s	2.0×

Numerical equivalence

a.dot(&b) and a.mul(&b).sum() both use ndarray's unrolled accumulation, and I measured a max absolute difference of exactly 0.0 between the old and new kernel matrices across n ∈ {500, 1000, 2000, 4000} and d ∈ {8, 50, 200} for all three methods. Bit-identity is not something either API contract promises, so I would not state it as a guarantee — but there is no reassociation being introduced here beyond what ndarray already does internally.

Checks

• cargo fmt --all -- --check — clean
• cargo clippy --workspace --all-targets -- -D warnings — clean
• cargo test --release --workspace — no failures

---

While looking at this I also measured a follow-up that is not in this PR: because the dense kernel is row-major and symmetric, Kernel::column(i) is a stride-n gather where the equivalent row(i) is contiguous. Switching the SMO solver to read rows is worth another ~1.7–2.1× on Svm::fit and looks relevant to #308. Happy to open that separately if it is of interest.

🤖 Generated with Claude Code

https://claude.ai/code/session_01L4uUK6s1CWFwx6u7Rx28go

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.90%. Comparing base (7fe5c86) to head (6d375b7).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #451      +/-   ##
==========================================
+ Coverage   77.53%   77.90%   +0.36%     
==========================================
  Files         106      104       -2     
  Lines        7585     7526      -59     
==========================================
- Hits         5881     5863      -18     
+ Misses       1704     1663      -41     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.