History

1.x.x (2026-xx-xx)

• Add educational MAPIE notebooks covering regression and conformal prediction for language models. Shown in the documentation homepage and All Examples.
• Fix _compute_classification_quantile to select the intended split-conformal order statistic. For some calibration sizes and confidence levels, the previous computation selected the next higher order statistic, making prediction sets unnecessarily conservative. Classification and regression now apply the same finite-sample correction. (issue #479)

1.5.0 (2026-xx-xx)

Features

• Add ConditionalSplitConformalRegressor and ConditionalSplitConformalClassifier, implementing conformal prediction with conditional guarantees following Gibbs et al. (2023). Only in the case of finite-dimensional classes for now.
• Add CrossConformalizedQuantileRegressor for cross-conformalized quantile regression, with dedicated conformity scores and examples.
• Add PyTorch-based ConditionalExpectedRiskController for Automatically Adaptive Conformal Risk Control (AA-CRC), together with built-in recall and symmetric-interval miscoverage losses and support for custom RiskLoss instances.
• Add StdConformityScore for estimators that expose prediction standard deviations through predict(..., return_std=True), enabling methods such as J+GP.
• Add multivariate standardized residual conformity scores (in mapie.experimental for now).
• Add aucroc_score and auarc_score for uncertainty evaluation following Lin et al. (2023). (issue #551)
• Add the classwise option to expected_calibration_error for classwise ECE. (issue #277)
• Add conditional coverage diagnostics: WSC (worst_slab_coverage), ERT (excess_risk_target_coverage), and optional covmetrics integration for CovGap and WCovGap.

Improvements and refactoring

• Consolidate regression and classification quantile computation in shared utilities, replacing the former get_quantile method and _compute_quantiles wrapper. No public API changes. (issue #479)

Bug fixes

• Reduce memory usage while generating the example gallery on Read the Docs.

Documentation

• Big update to the documentation: now based on MkDocs, still hosted on Read the Docs. The legacy Sphinx tree and obsolete static-site deployment files have been removed.
• Add a dedicated conditional conformal prediction gallery, including Gibbs et al. (2023) simulations and a Communities and Crime example backed by a local copy of the dataset.
• Add scientific examples for J+GP and AA-CRC, plus an exchangeability-testing benchmark notebook.
• Update the time-series change-point notebook for the quantile refactoring.
• Generate the API reference and navigation automatically from MAPIE's public objects.
• Consolidate the conformal-prediction metrics documentation and expand the theory for conditional coverage diagnostics.
• Reorganize the documentation around high-level introductions, shared conformal-prediction foundations, refreshed v1 getting-started workflows, and one doc/content/ content folder per main section.
• Add “Other Notebooks” sections to the classification and regression galleries and make assorted wording, notation, navigation, and example refinements.

CI, release, and developer experience

• Upgrade JupyterLab, PyTorch, pytest, and other development dependencies to address security vulnerabilities.
• Add SonarQube and Bandit analysis and update GitHub Actions to supported Node runtimes.
• Simplify release automation by removing the job that rewrote CITATION.cff.
• Refresh the release checklist with GitHub release guidance.
• Add AGENTS.md as the canonical guide for coding agents.

Breaking changes

• Drop support for Python 3.9 (EOL since October 2025). Minimum supported version is now Python 3.10. This was required to upgrade pytest to 9.0.3, which fixes a security advisory (CVE on /tmp/pytest-of-{user} directory handling).

1.4.1 (2026-06-08)

Features

• Add conformity_scores attribute to all Mapie objects, exposing conformity scores through a public property. (issue #921)
• Add reset() method on CrossConformalRegressor and allow refitting via fit_conformalize (now emits a UserWarning and discards prior conformity scores instead of raising). Same pattern can be propagated to other conformal classes in follow-up PRs. (issue #710)
• Add reset() method on JackknifeAfterBootstrapRegressor and allow refitting via fit_conformalize (mirrors the pattern landed for CrossConformalRegressor in #931).
• Add reset() method on CrossConformalClassifier and allow refitting via fit_conformalize (mirrors the pattern landed in #931 and #936); completes the warn-on-refit pattern across all Cross/Jackknife conformal techniques.

Improvements and refactoring

• Relax the monotonicity check in the fixed_sequence FWER procedure: a non-monotonic risk now emits a UserWarning (suggesting split_fixed_sequence) and infers a direction, instead of raising a ValueError. (issue #942)
• Add validation that a custom BinaryRisk returns per-sample occurrence values that are binary indicators (booleans, or values equal to 0 or 1); a ValueError is now raised otherwise, as the binary Hoeffding-Bentkus guarantees require it.
• Add validation to reject Subsample as cv in CrossConformalRegressor, directing users to JackknifeAfterBootstrapRegressor instead. (issue #924)
• Add defensive validation: _MapieRegressor and _MapieClassifier now raise TypeError when sample_weight is passed as a top-level keyword argument instead of inside fit_params. Previously, top-level sample_weight was silently ignored. Also fix TimeSeriesRegressor tests that were affected by the same silent-ignore bug.
• Add notebook kernel restart warning for Kaggle/Jupyter/Colab users after installation or version changes. (issue #916)
• Simplify internal sample_weight handling in classification module: sample_weight now flows through fit_params instead of being passed as a separate argument through the call chain. No public API changes. (issue #753)
• Simplify internal sample_weight handling in quantile regression module: sample_weight now flows through fit_params instead of being passed as a separate argument through the call chain. No public API changes. (issue #753)
• Remove _prepare_fit_params_and_sample_weight utility (no longer needed after regression, classification, and quantile regression refactors). (issue #753)

Bug fixes

• Fix BinaryClassificationController so the selected best threshold is optimal: when several parameters reach the minimum secondary risk, ties are now broken in favor of the least conservative threshold. (issue #942)
• Fix optimize_beta in regression conformity scores so prediction interval width minimization actually optimizes β (was previously a no-op due to a shape-collapsing reshape); also resolves incorrect prediction interval shape when used with multiple confidence levels. (issues #588, #484)
• Propagate random_state to the internal Subsample in JackknifeAfterBootstrapRegressor so bootstrap resampling is reproducible and no longer depends on the global NumPy RNG state. (issue #940)
• Restore backward compatibility for the predict-argument renames introduced in #919: CrossConformalRegressor's aggregate_predictions and JackknifeAfterBootstrapRegressor's ensemble (in predict and predict_interval) keep working as deprecated aliases for aggregate_point_predictions, now emitting a FutureWarning instead of raising a TypeError. (issue #906)

Documentation

• Add a risk control advanced-analysis example showing how to define and control a custom risk (specificity) with BinaryRisk and the BinaryClassificationController.
• Add a repository backup of the BlogFeedback dataset (examples/data/blogData_train.csv.gz) used by the Kim et al. (2020) example, now loaded by default so the example no longer depends on the UCI download server.
• Add repository backups of the Zaffran et al. (2022) data used by the ACI comparison example — the prices dataset (examples/data/zaffran2022_prices.csv.gz) and the reference results (examples/data/zaffran2022_aci_reference.csv) — now loaded by default so the example no longer depends on external downloads (also fixes an UnboundLocalError that surfaced when the reference download failed).
• Fix the documentation site navigation dropdown and even out the vertical spacing between sidebar nav entries.

CI, release, and developer experience

• Fix the release process (release-candidate/final-release workflows, docs deployment, and release checklist).
• Make tests compatible with scikit-learn 1.9 (calibration tests).
• Silence intentional UserWarnings emitted during test runs to keep the test output clean.

1.4.0 (2026-04-30)

Features

• Add several methods for family-wise error rate (FWER) control.
• Add exchangeability testing with permutation tests and online martingale tests, plus a high-level exchangeability API.
• Add risk monitoring.
• Add experimental module.
• Expose p0_p1_output in VennAbersCalibrator.predict_proba.

Improvements and refactoring

• Change the default FWER control method for risk control from bonferroni to bonferroni_holm.
• Rename BinaryClassificationRisk to BinaryRisk and keep a deprecated alias for backward compatibility.
• Extended risks to ContinuousRisk for more general risk monitoring.
• Simplify internal sample_weight handling in regression module: sample_weight now flows through fit_params instead of being passed as a separate argument through the call chain. No public API changes. (issue #753)
• Refactor offline tests and simplify parts of regression internals (including sample_weight handling and moving _check_gamma to time_series_regression.py).

Bug fixes

• Fix sample_weight routing through scikit-learn Pipeline in _fit_estimator.
• Remove sphinx thumbnail from LAC-APS example.

Documentation

• Move documentation to GitHub Pages/MkDocs and modernize the docs stack (Material setup, content pages, API auto-generation, markdown gallery conversion, multiline API signatures, and versioning).
• Improve MkDocs navigation, layout, examples overview, and documentation page rendering.
• Add richer exchangeability documentation, including theoretical background and in-depth examples for online martingale testing.
• Improve docs site content and UX (homepage examples, contributor guide, logo/banner fixes, image path fixes, and stable-doc image rendering fixes).

CI, release, and developer experience

• Modernize CI and documentation workflows (MkDocs deployment, RTD transition support, docs build fixes, faster docs build, and workflow path/version pin updates).
• Add optional pre-commit hook (format, lint, type-check) via pre-commit; documented in CONTRIBUTING.
• Improve project tooling and contributor experience (mypy return-any warnings, refreshed CONTRIBUTING and release checklist, and new project badges).
• Simplify release checklist: migrate the release flow to setuptools-scm , smoke tests in the workflows, use release candidates for TestPypi.

1.3.0 (2026-02-02)

• Introduce the Venn-Abers calibrator for both binary and multiclass classification.
• Implement a new risk control class for multi-label classification: MultiLabelClassificationController, a new variant of PrecisionRecallController.
• Add risk control for Semantic Segmentation with a new SemanticSegmentationController inheriting from MultiLabelClassificationController.
• Add a new binary classification risk called predicted_positive_fraction and update the corresponding examples.
• Configure a self-hosted runner for minimal requirements tests.
• Add two Semantic Segmentation examples illustrating precision and recall control on satellite roof images.
• Add an example of risk control using an LLM as a judge with abstention.
• Add a comparison with naive thresholding in the risk control quick start example.
• Add risk control to the decision tree in the “choosing the right algorithm” documentation section.
• Improve documentation display: increase content width, simplify example titles, improve API tables, and refine the decision tree for algorithm selection.
• Choose better thumbnails for the example lists in the documentation.
• Fix duplicated titles in the documentation tree.
• Add previous versions to the documentation version selector.
• Fix data leakage issues in time series tutorials.
• HISTORY, CONTRIBUTING, and AUTHORS are now written in Markdown instead of reStructuredText.
• Remove the dependency of internal classes on sklearn’s check_is_fitted.
• Fix a bug in CRC and RCPS where the computed lambda was not optimal.
• Fix a bug in the update method of ACI for time series.
• Add a disclosure about LLM usage to the pull request template.
• Improve the contribution guide by simplifying the Python setup and adding instructions for new contributors.
• Created a HuggingFace repository for MAPIE: https://huggingface.co/datasets/mapie-library

1.2.0 (2025-11-17)

• Implement extension of binary risk control to multi-risk
• Implement extension of binary risk control to multi-dimensional parameters
• Reorganise code structure for risk control
• BinaryClassificationController allows BinaryClassificationRisk and string representations of risks
• Improve quick start documentation for risk control
• Add new tutorial for risk control with multiple risks
• Add new tutorial for risk control with multi-dimensional parameters
• Fix issue 614 to pass Predict Params to RAPS for conformity score calculation and EnsembleClassifier
• Fix issue 790 to make BlockBootstrap include all non-training indices in the test set
• Update Python environment: dependancies are now in pyproject.toml
• Update CI and add long_tests folder for tests requiring more time

1.1.0 (2025-09-22)

• Implement new binary risk control feature, see BinaryClassificationController and BinaryClassificationRisk
• See also the reworked risk control documentation
• Revert incorrect renaming of calibration to conformalization in PrecisionRecallController
• Fix warnings when running tests
• Add scientific references for regression conformity scores
• Fix double inference when using predict_set function in split conformal classification
• MAPIE now supports Python versions up to the latest release (currently 3.13)
• Change prefit default value to True in split methods' docstrings to remain consistent with the implementation
• Fix issue 699 to replace TimeSeriesRegressor.partial_fit with TimeSeriesRegressor.update

1.0.1 (2025-05-22)

• Patch following v1.0.0 release: removing dependence to typing_extensions, making MAPIE unusable if this package is not installed

1.0.0 (2025-05-22)

• Major update, including a complete classification and regression public API rework, and a documentation revamp
• Other parts of the public API have been improved as well
• See the v1_release_notes.rst documentation file for extensive and user-focused release notes
• This update also includes bugfixes and developer experience improvements

0.9.2 (2025-01-15)

• Fix issue 525 in contribution guidelines with syntax errors in hyperlinks and other formatting issues.
• Fix issue 495 to center correctly the prediction intervals
• Fix issue 528 to correct broken ENS image in the documentation
• Fix issue 548 to correct labels generated in tutorial
• Fix issue 547 to fix wrong warning
• Fix issue 480 (correct display of mathematical equations in generated notebooks)
• Temporary solution waiting for issue 588 to be fixed (optimize_beta not working)
• Remove several irrelevant user warnings
• Limit max sklearn version allowed at MAPIE installation
• Refactor MapieRegressor, EnsembleRegressor, and MapieQuantileRegressor, to prepare for the release of v1.0.0
• Documentation build: fix warnings, fix image generation, update sklearn version requirement
• Documentation test: add a doc testing step (in MAKEFILE and CI)
• Increase max line length from 79 to 88 characters
• Bump wheel version
• Other minor evolutions

0.9.1 (2024-09-13)

• Fix issue 511 to access non-conformity scores with previous path
• Update gitignore by including the documentation folder generated for Mondrian
• Fix (partially) the set-up with pip instead of conda for new contributors

0.9.0 (2024-09-03)

• Fix citations and license links
• Fix the CQR tutorial to have same data in both methods
• Add ** predict_params in fit and predict method for Mapie Classifier
• Add Mondrian Conformal Prediction for regression and classification
• Add ** predict_params in fit and predict method for Mapie Regression
• Update the ts-changepoint notebook with the tutorial
• Change import related to conformity scores into ts-changepoint notebook
• Replace assert np.array_equal by np.testing.assert_array_equal in Mapie unit tests
• Replace github.com/simai-ml/MAPIE by github.com/scikit-learn-contrib/MAPIEin all Mapie files
• Extend ConformityScore to support regression (with BaseRegressionScore) and to support classification (with BaseClassificationScore)
• Extend EnsembleEstimator to support regression (with EnsembleRegressor) and to support classification (with EnsembleClassifier)
• Refactor MapieClassifier by separating the handling of the MapieClassifier estimator into a new class called EnsembleClassifier
• Refactor MapieClassifier by separating the handling of the MapieClassifier conformity score into a new class called BaseClassificationScore
• Add severals non-conformity scores for classification (LAC, APS, RAPS, TopK) based on BaseClassificationScore
• Transfer the logic of classification methods into the non-conformity score classes (LAC, APS, RAPS, TopK)
• Extend the classification strategy definition by supporting method and conformity_score attributes
• Building unit tests for different Subsample and BlockBooststrap instances
• Change the sign of C_k in the Kolmogorov-Smirnov test documentation
• Building a training set with a fraction between 0 and 1 with n_samples attribute when using split method from Subsample class.

0.8.6 (2024-06-14)

• Fix the quantile formula to ensure valid coverage (deal with infinite interval production and asymmetric conformal scores).
• Fix sphinx dependencies

0.8.5 (2024-06-07)

• Issue with update from 0.8.4

0.8.4 (2024-06-07)

• Fix the quantile formula to ensure valid coverage for any number of calibration data in ConformityScore.
• Fix overloading of the value of the method attribute when using MapieRegressor and MapieTimeSeriesRegressor.
• Fix conda versionning.
• Reduce precision for test in MapieCalibrator.
• Fix invalid certificate when downloading data.
• Add citations utility to the documentation.
• Add documentation for metrics.
• Add explanation and example for symmetry argument in CQR.

0.8.3 (2024-03-01)

• Allow the use of y and groups arguments MapieRegressor and MapieClassifier.
• Add possibility of passing fit parameters used by estimators.
• Fix memory issue CQR when testing for upper and lower bounds.
• Add Winkler Interval Score.

0.8.2 (2024-01-11)

• Resolve issue still present in 0.8.1 by updating pandas.

0.8.1 (2024-01-11)

• First attemps at fixing library conda issue.

0.8.0 (2024-01-03)

• Add Adaptative Conformal Inference (ACI) method for MapieTimeSeriesRegressor.
• Add Coverage Width-based Criterion (CWC) metric.
• Allow to use more split methods for MapieRegressor (ShuffleSplit, PredefinedSplit).
• Allow infinite prediction intervals to be produced in regressor classes.
• Integrate ConformityScore into MapieTimeSeriesRegressor.
• Add (extend) the optimal estimation strategy for the bounds of the prediction intervals for regression via ConformityScore.
• Add new checks for metrics calculations.
• Fix reference for residual normalised score in documentation.

0.7.0 (2023-09-14)

• Add prediction set estimation for binary classification.
• Add Learn-Then-Test method for multilabel-classification.
• Add documentation and notebooks for LTT.
• Add a new conformity score, ResidualNormalisedScore, that takes X into account and allows to compute adaptive intervals.
• Refactor MapieRegressor and ConformityScore to add the possibility to use X in ConformityScore.
• Separate the handling of the estimator from MapieRegressor into a new class called EnsembleEstimator.
• Rename methods (score to lac and cumulated_score to aps) in MapieClassifier.
• Add more notebooks and examples.
• Fix an unfixed random state in one of the classification tests.
• Add statistical calibration tests in binary classification.
• Fix and preserve the split behavior of the check_cv method with and without random state.

0.6.5 (2023-06-06)

• Add grouped conditional coverage metrics named SSC for regression and classification
• Add HSIC metric for regression
• Migrate conformity scores classes into conformity_scores module
• Migrate regression classes into regression module
• Add split conformal option for regression and classification
• Update check method for calibration
• Fix bug in MapieClassifier with different number of labels in calibration dataset.

0.6.4 (2023-04-05)

• Fix runtime warning with RAPS method

0.6.3 (2023-03-23)

• Fix bug when labels do not start at 0

0.6.2 (2023-03-22)

• Make MapieClassifier a scikit-learn object
• Update documentation for MapieClassifier

0.6.1 (2023-01-31)

• Fix still existing bug for classification with very low scores

0.6.0 (2023-01-19)

• Add RCPS and CRC for multilabel-classification
• Add Top-Label calibration
• Fix bug for classification with very low scores

0.5.0 (2022-10-20)

• Add RAPS method for classification
• Add theoretical description for RAPS

0.4.2 (2022-09-02)

• Add tutorial for time series
• Convert existing tutorials in .py
• Add prefit method for CQR
• Add tutorial for CQR

0.4.1 (2022-06-27)

• Add packaging library in requirements
• Fix displaying problem in pypi

0.4.0 (2022-06-24)

• Relax and fix typing
• Add Split Conformal Quantile Regression
• Add EnbPI method for Time Series Regression
• Add EnbPI Documentation
• Add example with heteroscedastic data
• Add ConformityScore class that allows the user to define custom conformity scores

0.3.2 (2022-03-11)

• Refactorize unit tests
• Add "naive" and "top-k" methods in MapieClassifier
• Include J+aB method in regression tutorial
• Add MNIST example for classification
• Add cross-conformal for classification
• Add notebooks folder containing notebooks used for generating documentation tutorials
• Uniformize the use of matrix k_ and add an argument "ensemble" to method "predict" in regression.py
• Add replication of the Chen Xu's tutorial testing Jackknife+aB vs Jackknife+
• Add Jackknife+-after-Bootstrap documentation
• Improve scikit-learn pipelines compatibility

0.3.1 (2021-11-19)

• Add Jackknife+-after-Bootstrap method and add mean and median as aggregation functions
• Add "cumulative_score" method in MapieClassifier
• Allow image as input in MapieClassifier

0.3.0 (2021-09-10)

• Renaming estimators.py module to regression.py
• New classification.py module with MapieClassifier class, that estimates prediction sets from softmax score
• New set of unit tests for classification.py module
• Modification of the documentation architecture
• Split example gallery into separate regression and classification galleries
• Add first classification examples
• Add method classification_coverage_score in the module metrics.py
• Fixed code error for plotting of interval widths in tutorial of documentation
• Added missing import statements in tutorial of documentation
• Refactorize tests of n_jobs and verbose in utils.py

0.2.3 (2021-07-09)

• Inclusion in conda-forge with updated release checklist
• Add time series example
• Add epistemic uncertainty example
• Remove CicleCI redundancy with ReadTheDocs
• Remove Pep8speaks
• Include linting in CI/CD
• Use PyPa github actions for releases

0.2.2 (2021-06-10)

• Set alpha parameter as predict argument, with None as default value
• Switch to github actions for continuous integration of the code
• Add image explaining MAPIE internals on the README

0.2.1 (2021-06-04)

• Add cv="prefit" option
• Add sample_weight argument in fit method

0.2.0 (2021-05-21)

• Add n_jobs argument using joblib parallel processing
• Allow cv to take the value -1 equivalently to LeaveOneOut()
• Introduce the cv parameter to get closer to scikit-learn API
• Remove the n_splits, shuffle and random_state parameters
• Simplify the method parameter
• Fix typos in documentation and add methods descriptions in sphinx
• Accept alpha parameter as a list or np.ndarray. If alpha is an Iterable, .predict() returns a np.ndarray of shape (n_samples, 3, len(alpha)).

0.1.4 (2021-05-07)

• Move all alpha related operations to predict
• Assume default LinearRegression if estimator is None
• Improve documentation
• return_pred argument is now ensemble boolean

0.1.3 (2021-04-30)

• Update PyPi homepage
• Set up publication workflows as a github action
• Update issue and pull request templates
• Increase sklearn compatibility (coverage_score and unit tests)

0.1.2 (2021-04-27)

• First release on PyPi

0.1.1 (2021-04-27)

• First release on TestPyPi

0.1.0 (2021-04-27)

• Implement metrics.coverage
• Implement estimators.MapieRegressor