anofox-regression

June 26, 2026 · View on GitHub

CI Crates.io Documentation codecov MIT licensed

Need to run this on 10GB of data? Use our DuckDB extension

Need to use this in a React Dashboard? Use our npm package

A robust statistics library for regression analysis in Rust, validated against R and scikit-learn (VALIDATION).

This library provides sklearn-style regression estimators with full statistical inference support including standard errors, t-statistics, p-values, confidence intervals, and prediction intervals.

Features

  • Linear Regression

    • Ordinary Least Squares (OLS) with full inference
    • Weighted Least Squares (WLS)
    • Ridge Regression (L2 regularization)
    • Elastic Net (L1 + L2 regularization via L-BFGS)
    • Huber Regression (robust to outliers, IRLS with Huber loss)
    • Recursive Least Squares (RLS) with online learning
    • Bounded Least Squares (BLS/NNLS) with box constraints
    • Dynamic Linear Model (LmDynamic) with time-varying coefficients
    • Least Angle Regression (LARS) and LassoLars with full coefficient path
    • Streaming fits via MomentAccumulator — OLS and Ridge accept pre-accumulated sufficient statistics (n, Σx, Σy, XᵀX, Xᵀy), so panels with millions of rows can be fit without ever materialising the full design matrix (see cargo run --example streaming_ridge)

  • Robust & Bayesian Regression

    • Theil-Sen (Vardi-Zhang spatial-median, ~29.3% breakdown point)
    • RANSAC (consensus-set refit with Fischler-Bolles stop-probability)
    • Bayesian Ridge (SVD evidence maximisation)
    • ARD Regression (per-feature precisions with relevance pruning)

  • Generalized Linear Models

    • Logistic Regression (binary classification with sklearn-like API)
    • Poisson GLM (Log, Identity, Sqrt links) with offset support
    • Negative Binomial GLM (overdispersed count data with theta estimation)
    • Binomial GLM (Logistic, Probit, Complementary log-log)
    • Tweedie GLM (Gaussian, Poisson, Gamma, Inverse-Gaussian, Compound Poisson-Gamma)
    • Gamma Regression (sklearn-style wrapper for Tweedie(power=2), log link)

  • Online Learning

    • Passive-Aggressive Regressor (PA-I and PA-II) with partial_fit

  • Augmented Linear Model (ALM)

    • 24 distributions: Normal, Laplace, Student-t, Gamma, Beta, Log-Normal, and more
    • Based on the greybox R package

  • Quantile & Monotonic Regression

    • Quantile Regression (IRLS with asymmetric weights, any τ ∈ (0,1))
    • Isotonic Regression (PAVA algorithm for monotonic constraints)

  • Smoothing & Classification

    • LOWESS (Locally Weighted Scatterplot Smoothing)
    • AID (Automatic Identification of Demand) classifier with optimized closed-form fitting

  • Loss Functions

    • MAE, MSE, RMSE, MAPE, sMAPE, MASE, pinball loss

  • Model Diagnostics

    • R², Adjusted R², RMSE, F-statistic, AIC, AICc, BIC
    • Leverage, Cook's distance, VIF, studentized residuals

Installation

Add to your Cargo.toml:

[dependencies]
anofox-regression = "0.5"

Examples

The library includes runnable examples demonstrating each major feature:

# Linear
cargo run --example ols                 # Ordinary Least Squares
cargo run --example wls                 # Weighted Least Squares
cargo run --example ridge               # Ridge regression
cargo run --example elastic_net         # Elastic Net
cargo run --example rls                 # Recursive Least Squares
cargo run --example bls                 # Bounded/Non-negative LS
cargo run --example pls                 # Partial Least Squares
cargo run --example streaming_ridge     # Streaming fit via MomentAccumulator
cargo run --example lars                # LARS and LassoLars
cargo run --example lm_dynamic          # Dynamic Linear Model

# Robust & Bayesian
cargo run --example huber               # Huber regression (robust)
cargo run --example theil_sen           # Theil–Sen estimator
cargo run --example ransac              # RANSAC regression
cargo run --example bayesian            # Bayesian Ridge + ARD

# Online
cargo run --example passive_aggressive  # PA-I / PA-II + partial_fit

# GLMs
cargo run --example poisson             # Poisson GLM
cargo run --example negative_binomial   # Negative Binomial GLM
cargo run --example binomial            # Binomial GLM (logit/probit/cloglog)
cargo run --example logistic            # LogisticRegression (sklearn-style)
cargo run --example tweedie             # Tweedie GLM
cargo run --example gamma               # Gamma GLM (log link)

# Other
cargo run --example alm                 # Augmented Linear Model
cargo run --example lowess              # LOWESS smoothing
cargo run --example aid                 # Demand classification
cargo run --example quantile            # Quantile regression
cargo run --example isotonic            # Isotonic regression

Quick Start

OLS Regression

use anofox_regression::prelude::*;
use faer::{Mat, Col};

let x = Mat::from_fn(100, 2, |i, j| (i + j) as f64 * 0.1);
let y = Col::from_fn(100, |i| 1.0 + 2.0 * i as f64 * 0.1);

let fitted = OlsRegressor::builder()
    .with_intercept(true)
    .build()
    .fit(&x, &y)?;

println!("R² = {:.4}", fitted.r_squared());
println!("Coefficients: {:?}", fitted.coefficients());

Prediction Intervals

let result = fitted.predict_with_interval(
    &x_new,
    Some(IntervalType::Prediction),
    0.95,
);
println!("Fit: {:?}", result.fit);
println!("Lower: {:?}", result.lower);
println!("Upper: {:?}", result.upper);

Poisson GLM

let fitted = PoissonRegressor::log()
    .with_intercept(true)
    .build()
    .fit(&x, &y)?;

println!("Deviance: {}", fitted.deviance);
let counts = fitted.predict_count(&x_new);

Logistic Regression

let fitted = BinomialRegressor::logistic()
    .with_intercept(true)
    .build()
    .fit(&x, &y)?;

let probs = fitted.predict_probability(&x_new);

Augmented Linear Model

// Laplace regression (robust to outliers)
let fitted = AlmRegressor::builder()
    .distribution(AlmDistribution::Laplace)
    .with_intercept(true)
    .build()
    .fit(&x, &y)?;

println!("Log-likelihood: {}", fitted.log_likelihood);

Quantile Regression

// Median regression (tau = 0.5)
let fitted = QuantileRegressor::builder()
    .tau(0.5)
    .build()
    .fit(&x, &y)?;

println!("Median coefficients: {:?}", fitted.coefficients());

// 90th percentile regression
let fitted_90 = QuantileRegressor::builder()
    .tau(0.9)
    .build()
    .fit(&x, &y)?;

Isotonic Regression

// Fit monotonically increasing function
let fitted = IsotonicRegressor::builder()
    .increasing(true)
    .build()
    .fit_1d(&x, &y)?;

println!("R² = {:.4}", fitted.result().r_squared);
let predictions = fitted.predict_1d(&x_new);

Validation

This library is developed using Test-Driven Development (TDD) against established statistical oracles. Most estimators are validated against R; the sklearn-style estimators added in v0.5.5 are validated against scikit-learn 1.5.2 via a pinned reproducible Python environment under validation/python/.

R-validated estimators

RustR EquivalentPackage
OlsRegressorlm()stats
WlsRegressorlm() with weightsstats
RidgeRegressor, ElasticNetRegressorglmnet()glmnet
HuberRegressorrlm(psi=psi.huber)MASS
LogisticRegressionglm(..., family=binomial)stats
BlsRegressornnls()nnls
PoissonRegressorglm(..., family=poisson)stats
BinomialRegressorglm(..., family=binomial)stats
NegativeBinomialRegressorglm.nb()MASS
TweedieRegressortweedie()statmod
GammaRegressorglm(family=Gamma(link="log"))stats
AlmRegressoralm()greybox
QuantileRegressorrq()quantreg
IsotonicRegressorisoreg()stats
Diagnosticscooks.distance(), hatvalues(), vif()stats, car

scikit-learn-validated estimators

Rustscikit-learn equivalent
TheilSenRegressorlinear_model.TheilSenRegressor
RansacRegressorlinear_model.RANSACRegressor
PassiveAggressiveRegressorlinear_model.PassiveAggressiveRegressor
LarsRegressor, LassoLars variantlinear_model.Lars, linear_model.LassoLars
BayesianRidgelinear_model.BayesianRidge
ArdRegressionlinear_model.ARDRegression

All 499+ test cases ensure numerical agreement with the chosen oracle within appropriate tolerances.

For complete transparency on the validation process, see validation/VALIDATION.md, which documents the per-estimator tolerance rationale and reproduction instructions for both the R and Python pipelines.

Dependencies

  • faer - High-performance linear algebra
  • statrs - Statistical distributions
  • argmin - Numerical optimization (L-BFGS)

Attribution

This library includes Rust implementations of algorithms from several open-source projects. See THIRD_PARTY_NOTICES for complete attribution and license information.

Key attributions:

  • greybox - ALM distributions and AID classifier methodology (independent implementation)
  • argmin (MIT/Apache-2.0) - L-BFGS optimization
  • faer (MIT) - Linear algebra operations
  • statrs (MIT) - Statistical distributions

License

MIT License