Aerial Mirage: Unmasking Hallucinations in Large Vision Language Models

This repository hosts two datasets:

(1) AeroCaps: the first Aerial-view Image Captioning dataset. This contains atleast four captions per image (manually annotated).

(2) LID: The Labelled Illusion Dataset, containing manually annotated fine-grained hallucinations for each description generated by LVLMs.

The datasets were introduced in WACV 2025.

The images of AeroCaps dataset can be found here! The groundtruth (reference) captions are present in AeroCaps.json file.

The LVLM-generated image captions, along with our hallucination-labelled annotations for the train and test sets, are released as the Labelled Illusion Dataset (LID). For LID, we use images from both the AeroCaps and the VisDrone datasets. The AeroCaps images are already available at the link above. The remaining images that were used from the VisDrone val set can be found in this link.

AeroCaps and LID are also released on HuggingFace!

If you use any of these datasets in your research, please cite:

@InProceedings{Debolena_WACV25,
    author    = {Basak, Debolena and Bhatt, Soham and Kanduri, Sahith and Desarkar, Maunendra Sankar},
    title     = {Aerial Mirage: Unmasking Hallucinations in Large Vision Language Models},
    booktitle = {Proceedings of the Winter Conference on Applications of Computer Vision (WACV)},
    month     = {February},
    year      = {2025},
    pages     = {5500-5508}
}