CppNumericalSolversBenchmark

Out-of-tree benchmark comparing CppNumericalSolvers against the canonical L-BFGS reference implementations on 83 unconstrained optimization problems plus the ~290-problem CUTEst unconstrained subset, with a separate constrained track against IPOPT and ALGENCAN on 29 Hock-Schittkowski 1981 problems.

View interactive results

Solvers

Solver	History m	Wolfe c₂	Line search
cppoptlib L-BFGS	10	0.9	Moré-Thuente
cppoptlib L-BFGS (HZ)	10	0.9	Hager-Zhang
LBFGSpp	6	0.9	Moré-Thuente
LBFGS-Lite	8	0.9	Lewis-Overton
libLBFGS	6	0.9	MINPACK Moré-Thuente
Nocedal Fortran L-BFGS	5	0.9	Moré-Thuente (MCSRCH)

All drivers use each library's out-of-the-box defaults — no per-problem tuning.

Problems

Family	Count	Dim range
MGH 1–34 (Moré-Garbow-Hillstrom 1981)	34	2–20
MGH 35 Chebyquad (bounded)	1	50
Hock-Schittkowski / Toint-Petrova (NLSProblems.jl)	47	2–100
Logistic regression on Iris	1	5
CUTEst unconstrained subset (^[QSO]UR2-.+-[V0]$)	~293	2–50000

Running

Three independent tracks live in this repository:

Track	Runner	CSV output	Problems
Unconstrained	./run.sh	unconstrained.csv	83 MGH + Hock-Schittkowski + logistic regression
CUTEst	./run_cutest.sh	cutest.csv	~293 SIF problems classified as smooth-unconstrained
Constrained	./run_constrained.sh	constrained.csv	29 Hock-Schittkowski 1981 (HS001..HS030 minus HS025)

All three runners distribute problems across all available CPU cores ($(nproc)) for the build phase and serialise the timing phase for fair wall-clock numbers. Per-iteration convergence traces land in the build cache automatically.

Unconstrained track

git clone --recursive https://github.com/PatWie/CppNumericalSolversBenchmark
cd CppNumericalSolversBenchmark
./run.sh > unconstrained.csv 2> unconstrained.err
python scripts/csv_to_json.py unconstrained.csv

The run takes ~5 seconds on a recent machine with 8 cores. csv_to_json.py writes site/data.json, which drives the site/index.html page.

CUTEst track

The CUTEst runner needs a Docker image that ships ARCHDefs + SIFDecode + CUTEst + MasterSIF pre-built; build it once with:

docker/cutest/build.sh

Then run:

./run_cutest.sh > cutest.csv 2> cutest.err

run_cutest.sh automatically re-executes itself inside the image, so no manual docker run wrapping is needed. It shares the /tmp/cppopt-bench/cutest-cache directory with the host so decoded SIF files and compiled object files persist across runs; a cold run finishes in ~30 seconds, a cache-hit run in ~5 seconds plus the per-problem timing window.

CUTEst rows share the CSV schema of the unconstrained track. A combined view lives in site/data.json:

cat unconstrained.csv <(tail -n +2 cutest.csv) \
  | python scripts/csv_to_json.py -

Constrained track

The constrained runner needs a Docker image that ships IPOPT 3.11.9 and ALGENCAN 3.1.1; build it once with:

docker/build.sh

Then run:

./run_constrained.sh > constrained.csv 2> constrained.err
python scripts/build_constrained_data.py constrained.csv

run_constrained.sh automatically re-executes itself inside the image, so no manual docker run wrapping is needed. The run takes ~18 seconds on the same machine. build_constrained_data.py writes site/constrained_data.json, which drives site/constrained.html.

Serving the site

python -m http.server -d site
# open http://localhost:8000/ for the unconstrained + CUTEst track
# open http://localhost:8000/constrained.html for the constrained track

The landing page filter control lets you narrow the summary cards, performance profiles, and results table to a single problem set (MGH, HS/TP, CUTEst, or logreg) without re-running anything.

Environment overrides

CPPOPT_BENCH_CACHE            tarball cache dir (default /tmp/cppopt-bench)
CPPOPT_CONSTRAINED_IMAGE      Docker image for the constrained runner
                              (default cppoptbench-constrained:latest)
CPPOPT_CUTEST_IMAGE           Docker image for the CUTEst runner
                              (default cppoptbench-cutest:latest)
CPPOPT_CUTEST_PROBLEMS        space-separated SIF names to run
                              (default: full classification-filtered set)
CC, CXX, FC                   compilers (default gcc, g++, gfortran)
CPPOPTLIB_ROOT                override CppNumericalSolvers source path

Adding a problem

Generated (scope B):

1. Add a Spec(...) to scripts/problems_spec.py.
2. Run uv run python scripts/gen_problems.py <name>.
3. If multiple optima are valid, add to scripts/acceptance.py.

Hand-written (MGH):

1. Create problems/<name>/ with meta.h, cpp.cc, lbfgspp.cc, lbfgslite.cc, liblbfgs.c, fortran.f.

Dependencies

Git submodules under third_party/:

CppNumericalSolvers    https://github.com/PatWie/CppNumericalSolvers
LBFGSpp                https://github.com/yixuan/LBFGSpp
LBFGS-Lite             https://github.com/ZJU-FAST-Lab/LBFGS-Lite
liblbfgs               https://github.com/chokkan/liblbfgs
eigen                  https://gitlab.com/libeigen/eigen (3.4.0)

Nocedal's Fortran L-BFGS and L-BFGS-B are fetched as tarballs on first run.

License

MIT.