Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding (CVPR2025 Highlights)

This repository contains the official pytorch implementation of the CVPR2025 paper: "Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding".

Updates

We will update more detailed result (including dataset, training, verification) in the future

• 2025.02.27: Build project page
• 2025.03.03:Add code and .sh file
• 2025.03.04:Add Dataset
• 2025.03.04:Add detailed process description

Method: Octopus

Environment Setup

conda env create --file Octopus.yaml

or

conda create Octopus python=3.10
conda activate Octopus
git clone https://github.com/LijunZhang01/Octopus
cd Octopus
pip install -r requirements.txt

Dataset

• Please download and extract the MSCOCO 2014 dataset from this link to the ''dataset'' folder
• For AMBER evaluation, see this link,and then move to the ''data'' folder

The directory structure is as follows:

+ data
  -train.json
  -image(AMBER dataset image)
  -query
  -relation.json
  -safe_words.txt
  -annotations.json
  -metrics.txt
+ dataset
  -images(COCO dataset image)

Models

About model Pre-trained checkpoints

• LLaVA-1.5: Download LLaVA-1.5 merged 7B
• InstructBLIP: Download InstructBLIP

Train

• AMBER:

bash eval_bench/scripts/train_token_amber.sh

• Need to specify "json_path", "data_path","checkpoint_path","model_path"

• ps: Replace all "/home/zlj/Octopus" paths with your own directory paths

Evaluation

• AMBER:

bash eval_bench/scripts/eval_token_amber.sh

• Need to specify "json_path", "data_path","checkpoint_path","model_path"

• ps: Replace all "/home/zlj/Octopus" paths with your own directory paths

Results

Generative Task

Discriminative Task.

Acknowledgments

This codebase borrows from most notably VCD, AvisC, and LLaVA. Many thanks to the authors for generously sharing their codes!

Citation

If you find this repository helpful for your project, please consider citing our work :

@misc{suo2025octopusalleviatinghallucinationdynamic,
      title={Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding}, 
      author={Wei Suo and Lijun Zhang and Mengyang Sun and Lin Yuanbo Wu and Peng Wang and Yanning Zhang},
      year={2025},
      eprint={2503.00361},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2503.00361}, 
}