For 3.0.0 defaults, numeric return types, exact tests, and reproducible clustering, see the migration examples.
matrix-stats extends the Matrix project with statistical tools for descriptive analysis,
hypothesis testing, regression, formula/model-frame workflows, linear algebra,
interpolation, native distributions, numerical solvers, time-series diagnostics, and clustering.
Use the Matrix BOM so matrix-core, matrix-stats, and any additional Matrix modules stay aligned.
matrix-stats publishes Groovy as compileOnly, so you must add Groovy explicitly. The examples
below also use matrix-datasets for Dataset.iris().
dependencies {
implementation "org.apache.groovy:groovy:GROOVY_VERSION"
implementation platform("se.alipsa.matrix:matrix-bom:MATRIX_VERSION")
implementation "se.alipsa.matrix:matrix-core"
implementation "se.alipsa.matrix:matrix-stats"
implementation "se.alipsa.matrix:matrix-datasets"
}<dependencyManagement>
<dependencies>
<dependency>
<groupId>se.alipsa.matrix</groupId>
<artifactId>matrix-bom</artifactId>
<version>MATRIX_VERSION</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.apache.groovy</groupId>
<artifactId>groovy</artifactId>
<version>GROOVY_VERSION</version>
</dependency>
<dependency>
<groupId>se.alipsa.matrix</groupId>
<artifactId>matrix-core</artifactId>
</dependency>
<dependency>
<groupId>se.alipsa.matrix</groupId>
<artifactId>matrix-stats</artifactId>
</dependency>
<dependency>
<groupId>se.alipsa.matrix</groupId>
<artifactId>matrix-datasets</artifactId>
</dependency>
</dependencies>matrix-stats no longer carries Apache Commons Math as a transitive runtime dependency.
commons-math3 is kept as a test-only dependency to validate the native implementations.
Consumers do not need to add it unless they use Commons Math directly in their own code.
The public linear algebra facade uses EJML internally via an implementation dependency.
Consumers do not need to add EJML unless they want to use EJML APIs directly.
Use Correlation.cor() for Pearson, Spearman, and Kendall coefficients.
import static se.alipsa.matrix.stats.Correlation.*
import java.math.RoundingMode
def x = [15, 18, 21, 24, 27]
def y = [25, 25, 27, 31, 32]
assert 0.95346258924559 == cor(x, y).setScale(14, RoundingMode.HALF_EVEN)
assert 0.45643546458764 == cor([15, 18, 21, 15, 21], [25, 25, 27, 27, 27], SPEARMAN)
.setScale(14, RoundingMode.HALF_EVEN)
assert -0.47140452079103173 == cor([12, 2, 1, 12, 2], [1, 4, 7, 1, 0], KENDALL)Normalize supports logarithmic, min-max, mean, and standard-deviation scaling.
import se.alipsa.matrix.core.ListConverter
import se.alipsa.matrix.stats.Normalize
List<BigDecimal> obs = ListConverter.toBigDecimals([1200, 34567, 3456, 12, 3456, 985, 1211])
def meanNormalized = Normalize.meanNorm(obs, 7)
def minMaxScaled = Normalize.minMaxNorm(obs, 7)
def zScores = Normalize.stdScaleNorm(obs, 7)
def logScaled = Normalize.logNorm(obs, 7)
println(meanNormalized)
println(minMaxScaled)
println(zScores)
println(logScaled)matrix-stats exposes a public dense linear algebra facade in se.alipsa.matrix.stats.linalg.Linalg.
The facade accepts Matrix, Grid, and list-backed numeric vectors, computes internally in
double precision, and returns Groovy-friendly BigDecimal, List<BigDecimal>, Matrix,
Grid<BigDecimal>, and SvdResult values.
import se.alipsa.matrix.core.Grid
import se.alipsa.matrix.core.Matrix
import se.alipsa.matrix.stats.linalg.Linalg
Matrix source = Matrix.builder()
.columnNames(['x', 'y'])
.rows([
[4.0, 7.0],
[2.0, 6.0]
])
.types([Double, Double])
.build()
assert [0.6, -0.2] == Linalg.solve(source, [1.0, 0.0])
assert 10.0 == Linalg.det(source)
Matrix inverse = Linalg.inverse(source)
assert ['c0', 'c1'] == inverse.columnNames()
Grid<Number> grid = new Grid<Number>([
[4.0, 7.0],
[2.0, 6.0]
])
assert 0.6 == Linalg.inverse(grid)[0, 0]
assert Linalg.svd(grid).singularValues.size() == 2matrix-stats exposes public linear interpolation utilities in
se.alipsa.matrix.stats.interpolation.Interpolation.
The current 2.4.0 scope is intentionally narrow:
- linear interpolation only
- idiomatic Groovy numeric inputs with
BigDecimalscalar results - explicit
(x, y, targetX)interpolation as the primitive operation - convenience overloads for evenly spaced numeric series and Matrix/Grid-backed columns
- no extrapolation outside the supported domain
import se.alipsa.matrix.core.Matrix
import se.alipsa.matrix.stats.interpolation.Interpolation
assert 5.0 == Interpolation.linear(
[0.0, 2.0, 4.0],
[0.0, 10.0, 20.0],
1.0
)
assert 30.0 == Interpolation.linear([10.0, 20.0, 40.0], 1.5)
Matrix points = Matrix.builder()
.columnNames(['time', 'value'])
.rows([
[1.0, 3.0],
[2.0, 6.0],
[4.0, 12.0]
])
.types([Double, Double])
.build()
assert 9.0 == Interpolation.linear(points, 'time', 'value', 3.0)The formula pipeline supports a focused R-style subset for model-frame evaluation and fit-method
dispatch through FitRegistry.
Supported formula features include:
- additive terms and intercept control:
y ~ x + z,y ~ 0 + x - interactions and shorthand expansion:
:,*,^,/ - quoted identifiers:
`gross margin` ~ `unit price` - transformed numeric terms:
log(x),sqrt(x),exp(x),I(x + 1) - polynomial terms:
poly(x, 3) - smooth terms for GAMs:
s(x)ands(x, 6)
The Groovy-native closure DSL, Formula.build { y | x + group }, relies on dynamic
propertyMissing lookup for bare column names and is not supported inside @CompileStatic
callers; use string formulas or a @CompileDynamic helper there. In that DSL, write
intercept removal as noIntercept + x rather than 0 + x.
import se.alipsa.matrix.core.Matrix
import se.alipsa.matrix.stats.formula.ModelFrame
import se.alipsa.matrix.stats.formula.NaAction
import se.alipsa.matrix.stats.regression.FitRegistry
Matrix data = Matrix.builder()
.columnNames(['y', 'x', 'group'])
.rows([
[1.0, 1.0, 'A'],
[2.0, 2.0, 'B'],
[3.0, 3.0, 'A'],
[4.0, 4.0, 'B']
])
.types([BigDecimal, BigDecimal, String])
.build()
def frame = ModelFrame.of('y ~ x + group', data)
.naAction(NaAction.OMIT)
.evaluate()
def lm = FitRegistry.instance().get('lm').fit(frame)
assert lm.fittedValues.length == data.rowCount()Built-in fit methods through FitRegistry:
lmfor ordinary least squaresloessfor univariate local regressiongamfor additive models usings(...)smooth terms
Current limitations:
- transformed responses on the left-hand side are not supported
na.actionsupports onlyOMITandFAILloesssupports only a single predictorlm,loess, andgamreject unsupported frame metadata rather than silently ignoring it- smooth terms cannot be used inside interactions such as
s(x):zors(x) * z
The same supported formula subset is available through the Groovy operator DSL:
def stringFrame = ModelFrame.of('y ~ x + group + x:group', data).evaluate()
def dslFrame = ModelFrame.of(data) {
y | x + group + (x % group)
}.evaluate()The first Groovy DSL version uses deferred syntax:
|instead of~noInterceptinstead of0 +
The DSL is intentionally dynamic because bare column names are resolved via propertyMissing and
the formula syntax relies on Groovy operator dispatch. Use string formulas or a @CompileDynamic
helper if you are calling it from @CompileStatic code.
I { ... } is limited to the arithmetic already supported by the string parser subset: numeric
literals, unary minus, +, -, *, /, **, and supported transform helpers such as log,
sqrt, and exp.
For fitting, you can either dispatch through FitRegistry or use the convenience helpers in
FitDsl:
import static se.alipsa.matrix.stats.regression.FitDsl.lm
def fit = lm(data) {
y | x + group + interaction(x, group)
}LinearRegression fits a simple least-squares model with one predictor.
import se.alipsa.matrix.core.Matrix
import se.alipsa.matrix.stats.regression.LinearRegression
def x = [2.7, 3, 5, 7, 9, 11, 14]
def y = [4, 5, 7, 10.8, 15, 20, 40]
def model = new LinearRegression(x, y)
assert -6.23971066 == model.getIntercept(8)
assert 2.81388732 == model.getSlope(8)
assert 30.34082454 == model.predict(13, 8)
Matrix table = Matrix.builder().data(
x: x,
y: y
).build()
def matrixModel = new LinearRegression(table, 'x', 'y')
println(matrixModel.getRsquared(4))matrix-stats includes native distribution implementations used by the module's runtime and exposed
through Groovy-friendly Number/BigDecimal APIs.
import se.alipsa.matrix.stats.distribution.HypergeometricDistribution
import se.alipsa.matrix.stats.distribution.NormalDistribution
def normal = new NormalDistribution(2, 1.5)
assert 0.5 == normal.cumulativeProbability(2)
assert 2.0 == normal.inverseCumulativeProbability(0.5)
def hyper = new HypergeometricDistribution(37, 21, 17)
assert hyper.supportLowerBound == 1
assert hyper.supportUpperBound == 17
assert hyper.probability(10) > 0For low-level numerical tasks, matrix-stats includes native solver APIs.
import se.alipsa.matrix.stats.solver.BrentSolver
import se.alipsa.matrix.stats.solver.LinearProgramSolver
import se.alipsa.matrix.stats.solver.UnivariateObjective
def root = BrentSolver.solve(
{ double x -> x * x - 2.0d } as UnivariateObjective,
0.0,
2.0,
1.0e-12,
1.0e-12,
100
)
assert Math.abs((root.rootValue as double) - Math.sqrt(2.0d)) < 1e-10
def solution = LinearProgramSolver.minimize([1.0, 2.0], [[1.0, 1.0]], [1.0])
assert [1.0, 0.0] == solution.pointValues
assert solution.objectiveValue == 1.0Welch is the recommended two-sample test when equal variance is not guaranteed.
Use Student when you explicitly want the pooled-variance or paired formulation.
import se.alipsa.matrix.datasets.Dataset
import se.alipsa.matrix.stats.ttest.Welch
def iris = Dataset.iris()
def species = iris.columnIndex('Species')
def setosa = iris.subset { it[species] == 'setosa' }
def virginica = iris.subset { it[species] == 'virginica' }
def result = Welch.tTest(setosa['Petal Length'], virginica['Petal Length'])
println(result)import se.alipsa.matrix.stats.ttest.Student
def control = [14, 14, 16, 13, 12, 17, 15, 14]
def treatment = [18, 17, 19, 18, 16, 20, 18, 19]
def result = Student.tTest(control, treatment, true)
println(result)import se.alipsa.matrix.stats.ttest.Student
def before = [85, 78, 92, 91, 72, 97]
def after = [88, 80, 93, 94, 80, 100]
def paired = Student.pairedTTest(after, before)
println(paired)One-way ANOVA is available through Anova.aov().
import se.alipsa.matrix.stats.Anova
def result = Anova.aov([
control : [10, 12, 11, 13, 9],
treatmentA: [14, 15, 13, 16, 14],
treatmentB: [8, 9, 7, 10, 8]
])
println(result)
assert result.evaluate()Use UnitRoot.test(...) when you want DF, ADF, ADF-GLS, and KPSS in one report.
import se.alipsa.matrix.stats.timeseries.UnitRoot
def series = [1.2, 1.5, 1.3, 1.6, 1.4, 1.7, 1.5, 1.8, 1.6, 1.9, 1.7, 2.0]
def result = UnitRoot.test(series, 'drift')
println(result.summary())
println(result.isStationary() ? 'Stationary' : 'Needs differencing')Most matrix-stats APIs return a small result object rather than raw tuples.
Welch.tTest()returns TtestResult withtVal,pVal,df,mean1,mean2,var1,var2,sd1,sd2,n1,n2, anddescription.Student.tTest(first, second, true)returnsStudent.Result, which extendsTtestResultfor the pooled-variance case.Student.tTest(values, comparison)returnsStudent.SingleResultwithtVal,pVal,df,mean,var,sd,n, anddescription.Student.pairedTTest()returnsStudent.PairedResult, which extends the two-sample result and addssdfor the standard deviation of paired differences.Anova.aov()returnsAnova.AnovaResultwithfValueandpValue.UnitRoot.test()returnsUnitRoot.UnitRootResultwith DF, ADF, ADF-GLS, and KPSS results plus summary helpers.BrentSolver.solve()returnsBrentSolver.SolverResultwithrootValue,lowerBoundValue,upperBoundValue,evaluations, anditerations.LinearProgramSolver.minimize()returnsLinearProgramSolver.SolutionwithpointValues,objectiveValue, anditerations.FitRegistry.instance().get(...).fit(...)returnsFitResultwithcoefficients,standardErrors,fittedValues,residuals,rSquared, andpredictorNames.
All of these result types provide a readable toString() and helper getters like getT(int decimals) or getP(int decimals) where that makes sense. For the full API surface, use the GroovyDoc and the source classes above.
For time-series stationarity tests, normality tests, and clustering recipes, see the Matrix Stats cookbook.
Go to previous section | Go to next section | Back to outline