Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation

Moru Liu¹  Hao Dong²  Jessica Kelly³  Olga Fink⁴  Mario Trapp^1,3

¹Technical University of Munich, ²ETH Zurich, ³Fraunhofer IKS, ⁴EPFL

• NeurIPS 2025 •

---

Feature Mixing randomly selects a subset of N feature dimensions from each modality and swaps them to obtain new features, which are then concatenated to form the multimodal outlier features.

Code

An Example of Feature Mixing in 1D and 2D

featuremixing_example_1D_2D.py 

Multimodal OOD Detection

Follow the instructions in MultiOOD to prepare the dataset and setup the environments.

HMDB51 as ID

cd OOD_detection/HMDB-rgb-flow/

Train the Far-OOD model using A2D and Feature Mixing for HMDB:

python train_video_flow.py --dataset 'HMDB' --lr 0.0001 --seed 0 --bsz 16 --num_workers 10 --use_single_pred --use_a2d --a2d_max_hellinger --a2d_ratio 0.1 --use_featuremixing --max_ood_hellinger --a2d_ratio_ood 1.0 --ood_entropy_ratio 1.0 --nepochs 50 --appen '' --save_best --save_checkpoint --datapath '/path/to/HMDB51/' 

Save the evaluation files for HMDB:

python test_video_flow.py --bsz 16 --num_workers 2 --dataset 'HMDB' --appen 'a2d_fm_best_' --resumef '/path/to/model_best.pt'

Save the evaluation files for UCF:

python test_video_flow.py --bsz 16 --num_workers 2 --far_ood --dataset 'HMDB' --ood_dataset 'UCF' --appen 'a2d_fm_best_' --resumef '/path/to/model_best.pt'

Save the evaluation files for HAC:

python test_video_flow.py --bsz 16 --num_workers 2 --far_ood --dataset 'HMDB' --ood_dataset 'HAC' --appen 'a2d_fm_best_' --resumef '/path/to/model_best.pt'

Save the evaluation files for Kinetics:

python test_video_flow.py --bsz 16 --num_workers 2 --far_ood --dataset 'HMDB' --ood_dataset 'Kinetics' --appen 'a2d_fm_best_' --resumef '/path/to/model_best.pt'

Save the evaluation files for EPIC:

cd OOD_detection/EPIC-rgb-flow/

python test_video_flow_epic.py --bsz 16 --num_workers 2 --far_ood --dataset 'HMDB' --ood_dataset 'EPIC' --appen 'a2d_fm_best_' --resumef '/path/to/model_best.pt'

Evaluation for UCF (change --ood_dataset to UCF, EPIC, HAC, or Kinetics):

python eval_video_flow_far_ood.py --postprocessor ebo --appen 'a2d_fm_best_' --dataset 'HMDB' --ood_dataset 'UCF' --path 'HMDB-rgb-flow/'

Multimodal OOD Segmentation

The code for OOD Segmentation will be available soon.

The CARLA-OOD dataset is now available:
https://huggingface.co/datasets/Mona4399/CARLA_OOD

Citation

If you find our work useful in your research please consider citing our paper:

@article{liu2025fm,
    title={Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation},
    author={Liu, Moru and Dong, Hao and Kelly, Jessica and Fink, Olga and Trapp, Mario},
    journal={arXiv preprint arXiv:2505.16985},
    year={2025}
}

Related Projects

MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities

DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection

NNG-Mix: Improving Semi-supervised Anomaly Detection with Pseudo-anomaly Generation

Survey: Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models

Acknowledgement

Many thanks to the open-source projects SimMMDG and MultiOOD.