OpenSafeMLRM: Toolkit for Multimodal Large Reasoning Model (MLRM) Safety Evaluation 🚀

OpenSafeMLRM is the first toolkit dedicated to the safety evaluation of Multimodal Large Reasoning Models (MLRM). This toolkit provides a unified interface for mainstream models, datasets, and jailbreaking methods, enabling researchers and practitioners to assess the safety and robustness of their multimodal models efficiently.

News 📰

🚨 April 8th,2025 - Major Update Coming Soon!

Please note that the complete code is currently being organized and will be released shortly. We are working on finalizing the details and ensuring everything is thoroughly tested before the public release. Stay tuned! 🌟

Features ✨

• Unified Interface: A single API for accessing a wide range of popular multimodal models, datasets, and security testing methods.
• Safety Evaluation: Tools for testing model robustness against adversarial inputs and jailbreaking techniques.
• Multimodal Support: Works with models that handle both text and images, making it ideal for evaluating MLRMs.
• Ease of Use: Simple setup and easy-to-use interface to quickly get started with model safety evaluation.

Key Components 🧩

• Models: Supports a variety of leading multimodal models.
• Datasets: Provides access to popular datasets for safety evaluation and testing.
• Jailbreaking Methods: Implements common jailbreaking techniques to test model resilience.

Installation ⚙️

To install OpenSafeMLRM, simply clone the repository:

git clone https://github.com/yourusername/OpenSafeMLRM.git
cd OpenSafeMLRM

Install the necessary dependencies:

pip install -r requirements.txt

Usage 📚

Contributing 🤝

We welcome contributions from the community! If you would like to contribute to the project, please fork the repository, make your changes, and submit a pull request. Let's collaborate to make MLRM safety even better! 💡

---

Thank you for using OpenSafeMLRM! Stay tuned for the full release of the toolkit. 🚀