SlopeML

A simple autograd-based machine learning library for Rust.

SlopeML provides a minimal but functional toolkit for building and training neural networks, featuring automatic differentiation via a computational graph, parallel forward passes with Rayon, and an SGD optimizer with momentum.

Features

• Automatic Differentiation - Build a computational graph and compute gradients with a single backward pass
• Parallel Inference - Forward passes are parallelized across neurons and batches using Rayon
• Neural Network Primitives - Neuron, Layer, and MLP abstractions for quick model construction
• SGD with Momentum - Built-in optimizer with configurable learning rate and momentum
• Operator Overloading - Write math expressions naturally using +, -, *, /, and method calls like .relu(), .exp(), .ln()
• Zero Dependencies (beyond rand, rayon, atomic_float) - lightweight and easy to embed

Installation

Add to your Cargo.toml:

[dependencies]
slope-ml = { git = "https://github.com/VxidDev/SlopeML" }

Quick Start

use rand::Rng;
use slopeml::{Graph, MLP, SGD, Value};

fn main() {
    let mut rng = rand::rng();

    // Build a simple MLP: 2 inputs -> 16 hidden (ReLU) -> 1 output
    let mlp = MLP::new(2, &[16, 1], &mut rng);
    let mut optimizer = SGD::new(mlp.parameters(), 0.01, 0.9);

    // XOR training data
    let data = vec![
        (vec![0.0, 0.0], 0.0),
        (vec![0.0, 1.0], 1.0),
        (vec![1.0, 0.0], 1.0),
        (vec![1.0, 1.0], 0.0),
    ];

    for epoch in 0..1000 {
        let mut total_loss = Value::new(0.0);

        for (inputs, target) in &data {
            let input_vals: Vec<Value> = inputs.iter().map(|&x| Value::new(x)).collect();
            let output = &mlp.forward(&input_vals)[0];
            let target_val = Value::new(*target);
            let loss = &(output - &target_val) * &(output - &target_val);

            total_loss = &total_loss + &loss;
        }

        let avg_loss = &total_loss / &Value::new(data.len() as f64);

        let graph = Graph::build(&avg_loss);
        graph.forward();
        graph.zero_grad();
        graph.backward();
        optimizer.step();

        if epoch % 100 == 0 {
            println!("Epoch {}: loss = {:.4}", epoch, avg_loss.data());
        }
    }

    // Inference
    let test_inputs = vec![Value::new(0.0), Value::new(1.0)];
    let result = mlp.forward(&test_inputs);
    println!("0 XOR 1 = {:.4}", result[0].data());
}

Core Concepts

Value

Value is the fundamental type. It wraps an Arc<ValueData> containing the scalar data, its gradient, and the operation that produced it. Operations like +, *, -, / create new Value nodes that form a computational graph.

let a = Value::new(2.0);
let b = Value::new(3.0);
let c = &a + &b;  // creates a new Value node

assert_eq!(c.data(), 5.0);

Graph

A Graph is built from a single output Value. It topologically sorts all connected nodes, enabling:

• forward() - recompute all values from leaves to root
• zero_grad() - reset all gradients to zero
• backward() - propagate gradients from root back to leaves

let loss = /* ... some computation ... */;
let graph = Graph::build(&loss);
graph.forward();
graph.zero_grad();
graph.backward();

MLP

A multi-layer perceptron. The last layer uses no activation (linear output), while all hidden layers use ReLU.

let mlp = MLP::new(
    4,              // input dimension
    &[16, 16, 1],   // hidden layer sizes
    &mut rng,
);

let input = vec![Value::new(1.0); 4];
let output = mlp.forward(&input);  // Vec<Value> with 1 element

SGD

Stochastic gradient descent with momentum. Collects parameters from the model, then updates them in parallel.

let mut optimizer = SGD::new(mlp.parameters(), 0.01, 0.9);

// After computing gradients:
graph.backward();
optimizer.step();

Documentation

Detailed API documentation is available in the docs/ directory:

• API Reference - Full documentation for all types and methods
• Examples Guide - Walkthrough of the included examples
• Architecture - How the autograd system works

Examples

The examples/ directory contains:

• fnlm - A small language model that trains on a text dataset and generates text via an interactive REPL

Run an example with:

cargo run --example fnlm

License

GPL-3.0 - see LICENSE for details.