MATLAB Wrapper

April 10, 2026 ยท View on GitHub

This is the GTSAM MATLAB toolbox, a MATLAB wrapper around the GTSAM C++ library. To build it, enable GTSAM_INSTALL_MATLAB_TOOLBOX=ON in CMake.

The interface to the toolbox is generated automatically by the wrap tool which directly parses C++ header files. The tool generates matlab proxy objects together with all the native functions for wrapping and unwrapping scalar and non scalar types and objects. The interface generated by the wrap tool also redirects the standard output stream (cout) to matlab's console.

Building the toolbox

For non-developers, the recommended installation is outside the build tree. Enable the MATLAB toolbox, build normally, and let cmake --install . install both the shared libraries and the toolbox under the install prefix:

cd /path/to/gtsam/build
cmake .. \
  -DCMAKE_INSTALL_PREFIX=/opt/gtsam \
  -DGTSAM_INSTALL_MATLAB_TOOLBOX=ON \
  -DMatlab_ROOT_DIR=/Applications/MATLAB_R2025b.app
cmake --build . -j6
cmake --install .

With that configuration:

  • GTSAM libraries install under /opt/gtsam/lib
  • the MATLAB toolbox installs under /opt/gtsam/gtsam_toolbox
  • end users add only /opt/gtsam/gtsam_toolbox to the MATLAB path

This is the best practice for packaging, lab machines, and general non-developer use. It keeps MATLAB pointed at a stable installed toolbox instead of a changing build tree. That being said, if you need a different toolbox location, you can set GTSAM_TOOLBOX_INSTALL_PATH.

Adding the toolbox to your MATLAB path

To get started, first add the toolbox (or gtsam_toolbox) folder to your MATLAB path - in the MATLAB file browser, right-click on the folder and click 'Add to path -> This folder' (do not add the subfolders to your path).

Trying out the examples

The examples are located in the 'gtsam_examples' subfolder. You may either run them individually at the MATLAB command line, or open the GTSAM example GUI by running 'gtsamExamples'. Example:

>> cd /opt/gtsam/gtsam_toolbox % Change to wherever you installed the toolbox
>> cd gtsam_examples           % Change to the examples directory
>> gtsamExamples               % Run the GTSAM examples GUI

Running the unit tests

The GTSAM MATLAB toolbox also has a small set of unit tests located in the gtsam_tests directory. Example:

>> cd /opt/gtsam/gtsam_toolbox % Change to wherever you installed the toolbox
>> cd gtsam_tests              % Change to the examples directory
>> test_gtsam                  % Run the unit tests
Starting: testJacobianFactor
Starting: testKalmanFilter
Starting: testLocalizationExample
Starting: testOdometryExample
Starting: testPlanarSLAMExample
Starting: testPose2SLAMExample
Starting: testPose3SLAMExample
Starting: testSFMExample
Starting: testStereoVOExample
Starting: testVisualISAMExample
Tests complete!

Writing your own code

Coding for the GTSAM MATLAB toolbox is straightforward and very fast once you understand a few basic concepts! Please see the manual to get started.

Filename Case Sensitivity

Due to the file name syntax requirements of Matlab, having the files Symbol.m (for the class) and symbol.m (for the function) together on an OS with a case-insensitive filesystem is impossible.

To put it simply, for an OS like Windows or MacOS where the filesystem is case-insensitive, we cannot have two files symbol.m and Symbol.m in the same folder. When trying to write one file after another, they will end up overwriting each other. We cannot specify 2 different filenames, since Matlab's syntax prevents that.

For this reason, on Windows and MacOS, we ignore the Symbol class and request users to use the symbol function exclusively for key creation.

Developer-only build-tree install

Installing into the build tree is convenient while iterating on the MATLAB wrapper itself:

cd /path/to/gtsam/build
cmake .. \
  -DGTSAM_INSTALL_MATLAB_TOOLBOX=ON \
  -DMatlab_ROOT_DIR=/Applications/MATLAB_R2025b.app \
  -DGTSAM_TOOLBOX_INSTALL_PATH=/path/to/gtsam/build/gtsam_toolbox
cmake --build . -j6 --target gtsam_matlab_wrapper
cmake --install .

After rebuilding or reinstalling the toolbox, restart MATLAB and add the toolbox path again. MATLAB caches class and mex state aggressively, and a restart avoids stale wrapper state from an older build.

For instructions on updating the version of the wrap library included in GTSAM to the latest version, please refer to the wrap README

Known Issues

macOS stability note

On current macOS + MATLAB R2025b configurations, we have observed MATLAB exit crashes after running nonlinear optimization when GTSAM is built with TBB enabled. If you are building the MATLAB toolbox for macOS, the current recommended setting is:

-DGTSAM_WITH_TBB=OFF

This avoids a reproducible shutdown crash in the MATLAB wrapper after optimization. If you do not need TBB elsewhere in your build, it is safest to disable it for MATLAB users.

Further setup on Linux

For a normal install created with cmake --install ., the MATLAB mex wrapper is installed with an rpath back to the GTSAM library directory. In the common case where the toolbox and the GTSAM shared libraries stay under the same install prefix, no extra Linux linker setup should be necessary.

You may still need manual linker configuration if you:

  • relocate the toolbox after installation
  • relocate the GTSAM shared libraries after installation
  • use an older toolbox build that predates the current rpath handling
  • run on an older MATLAB/Linux combination with nonstandard loader behavior

For a system-wide install under a standard library location, sudo ldconfig may be sufficient.

For a custom library location, you can point the loader at the installed GTSAM libraries with LD_LIBRARY_PATH. If your install prefix is <install-prefix>, the relevant directory is usually <install-prefix>/lib:

export LD_LIBRARY_PATH=<install-prefix>/lib:$LD_LIBRARY_PATH

MATLAB CustomFactor User Guide

The MATLAB toolbox supports callback-backed CustomFactor objects with the same basic usage pattern as Python:

import gtsam.*

key = 42;
model = noiseModel.Unit.Create(1);

factor = CustomFactor(model, key, @myErrorFunction);

graph = NonlinearFactorGraph;
graph.add(factor);

The callback signature is:

function varargout = myErrorFunction(this, values)
  % Retrieve keys from the factor object ('this')
  keys = this.keys();
  residual = values.atVector(keys.at(0)) - [3];
  if nargout > 1
    varargout{1} = residual;
    varargout{2} = {1};
  else
    varargout{1} = residual;
  end
end

Rules for MATLAB CustomFactor callbacks:

  • this is the MATLAB gtsam.CustomFactor object.
  • values is a wrapped gtsam.Values object.
  • The first output must be the unwhitened residual vector.
  • If Jacobians are requested, the second output must be a cell array with one numeric matrix per factor key, in factor key order.
  • The residual dimension must match the noise model dimension.
  • Keys may be passed either as a numeric array or as a gtsam.KeyVector.

Practical notes:

  • unwhitenedError(...) exercises the residual-only path.
  • Jacobians should always be verified using numerical derivatives during development. The gtsam.numericalDerivative utility is provided for this purpose:
% Verify Jacobians using numerical derivative
expected_H = numericalDerivative.numericalDerivative11(@(v) factor.unwhitenedError(v), initial);
actual_H = factor.linearize(initial).jacobian();
EQUALITY('custom jacobian', expected_H, actual_H, 1e-5);

Wrapped overloaded methods such as factor.unwhitenedError still need an explicit lambda when passed as callbacks in MATLAB.

  • Optimization usually exercises both the residual and Jacobian paths.
  • Deleting the MATLAB factor removes its callback from the MATLAB-side registry, so repeated tests in one session do not accumulate stale entries.
  • Once a MATLAB CustomFactor has been constructed, the mex module is intentionally kept loaded for the rest of the MATLAB session to avoid unload-time crashes while callback-backed native factors are still reachable from C++.

MATLAB CustomFactor Under the Hood

The MATLAB CustomFactor support is implemented as a MATLAB-specific layer on top of the existing C++ gtsam::CustomFactor.

Files involved:

Call path:

  1. gtsam.CustomFactor(noiseModel, keys, errorFunc) registers errorFunc in gtsam.customFactorRegistry and gets back a numeric callback ID.
  2. The MATLAB facade constructs the native gtsam.MatlabCustomFactor, passing only the noise model, keys, and callback ID into C++.
  3. After construction, the MATLAB facade binds the final MATLAB gtsam.CustomFactor object into the registry under the same callback ID.
  4. When GTSAM evaluates the factor in C++, MatlabCustomFactor calls back into MATLAB with: gtsam.customFactorRegistry('invoke', callbackId, values).
  5. The registry looks up the stored MATLAB factor object and function handle, then invokes: errorFunc(this, values) or [error, H] = errorFunc(this, values).
  6. The native wrapper validates the residual size and Jacobian count, converts the outputs back into GTSAM types, and returns them to the optimizer.

Why there is a registry instead of storing the MATLAB function handle directly in C++:

  • the native factor can be copied and retained by optimizer-owned C++ objects
  • MATLAB object lifetimes do not match C++ shared pointer lifetimes
  • storing only a numeric callback ID in C++ avoids keeping raw MATLAB objects in core GTSAM state
  • the registry lets the callback receive the original MATLAB gtsam.CustomFactor object as this

Debugging checklist for CustomFactor:

  • Constructor fails: verify the toolbox contains both gtsam.CustomFactor and gtsam.MatlabCustomFactor.
  • Callback is never called: check that the factor was added to a graph and that the graph path you are exercising actually evaluates it.
  • Callback gets wrong argument types: inspect matlab/+gtsam/CustomFactor.m and matlab/+gtsam/customFactorRegistry.m first.
  • MATLAB reports wrong residual or Jacobian dimensions: the callback contract is being violated; inspect the callback outputs before suspecting the optimizer.
  • MATLAB crashes or errors while unloading the wrapper: remember that creating a MATLAB CustomFactor intentionally locks the mex module for the rest of the session.