RSCC

RSCC: A Large-Scale Remote Sensing Change Caption Dataset for Disaster Events

Zhenyuan Chen, Chenxi Wang, Ningyu Zhang, Feng Zhang

Zhejiang University

Accepted by NeurIPS 2025 Datasets and Benchmarks Track

Overview

We introduce the Remote Sensing Change Caption (RSCC) dataset, a new benchmark designed to advance the development of large vision-language models for remote sensing. Existing image-text datasets typically rely on single-snapshot imagery and lack the temporal detail crucial for Earth observation tasks. By providing 62,351 pairs of pre-event and post-event images accompanied by detailed change captions, RSCC bridges this gap and enables robust disaster-awareness bi-temporal understanding. We demonstrate its utility through comprehensive experiments using interleaved multimodal large language models. Our results highlight RSCC’s ability to facilitate detailed disaster-related analysis, paving the way for more accurate, interpretable, and scalable vision-language applications in remote sensing.

📢News

[NEWS] We release RSCC subset on GoogleDrive for research purpose only.

[NEWS] 🎉 2025/09/19: Our paper "RSCC: A Large-Scale Remote Sensing Change Caption Dataset for Disaster Events" has been accepted by NeurIPS 2025 Datasets and Benchmarks Track!

[COMPLETED] Release RSCC dataset

• 2025/05/01 All pre-event & post-event images of RSCC (total: 62,351 pairs) are released.
• 2025/05/01 The change captions of RSCC-Subset (988 pairs) are released, including 10 baseline model results and QvQ-Max results (ground truth).
• 2025/05/01 The change captions based on Qwen2.5-VL-72B-Instruct of RSCC (total: 62,351 pairs) are released.
• 2025/09/09 Release RSCC change captions based on strong models (e.g., QvQ-Max, o3).

[COMPLETED] Release code for inference

• 2025/05/01 Naive inference with baseline models.
• 2025/05/15 Training-free method augmentation (e.g., VCD, DoLa, DeCo).

[COMPLETED] Release RSCCM training scripts

[COMPLETED] Release code for evaluation

• 2025/05/01 Metrics for N-Gram (e.g. BLEU, METEOR, ROUGE).
• 2025/05/01 Metrics for contextual similarity (e.g. Sentence-T5 Similarity, BERTScore).
• 2025/05/01 Auto comparison of change captions using QvQ-Max (visual reasoning VLM) as a judge.

Dataset

The dataset can be downloaded from Huggingface.

Benchmark Results

Inference

Environment Setup

cd RSCC # path of project root
conda env create -f environment.yaml # genai: env for most baseline models
conda env create -f environment_teochat.yaml # teohat: env for TEOChat
conda env create -f environment_ccexpert.yaml # CCExpert: env for CCExpert

Prepare Pre-trainined Models and Dataset

We use the same style as huggingface.co as repo_id/model_id. The model folder should be structured as below:

/path/to/model/folder/
├── moonshotai/
│   └── Kimi-VL-A3B-Instruct/
├── Qwen/
│   └── Qwen2-VL-7B-Instruct/
├── Salesforce/
│   └── xgen-mm-phi3-mini-instruct-interleave-r-v1.5/
├── microsoft/
│   └── Phi-4-multimodal-instruct/
├── OpenGVLab/
│   └── InternVL3-8B/
├── llava-hf/
│   ├── llava-interleave-qwen-7b-hf/
│   └── llava-onevision-qwen2-7b-ov-hf/
├── mistralai/
│   └── Pixtral-12B-2409/
├── Meize0729/
│   └── CCExpert_7b/
└── jirvin16/
    └── TEOChat/

{
  "mm_image_tower": "/path/to/model/folder/LanguageBind/LanguageBind_Image",
  "mm_video_tower": "/path/to/model/folder/LanguageBind/LanguageBind_Video_merge"
}

Download RSCC dataset and place them under your dataset folder:

/path/to/dataset/folder
├── EBD/
│   └── {events}/
├── xbd/
│   └── images-w512-h512/
│       └── {events}/
└── xbdsubset/
    └── {events}/

Set global variable for PATH_TO_MODEL_FOLDER and PATH_TO_DATASET_FOLDER.

# `RSCC/utils/constants.py`
PATH_TO_MODEL_FOLDER = /path/to/model/folder/ #  "/home/models"
PATH_TO_DATASET_FOLDER = /path/to/dataset/folder # "/home/datasets"

Inference

# API key for DashScope (keep this secret!)
DASHSCOPE_API_KEY="sk-xxxxxxxxxx"

# Model ID should match the official code
QVQ_MODEL_NAME="qvq-max-2025-03-25"

# API base URL
API_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1"

# Maximum concurrent workers
MAX_WORKERS=30

# Token threshold warning level
TOKEN_THRESHOLD=10000

conda activate genai
python ./inference/xbd_subset_qvq.py

# inference/xbd_subset_baseline.py
...existing codes...
INFERENCE_MODEL_LIST = [
"moonshotai/Kimi-VL-A3B-Instruct",
"Qwen/Qwen2-VL-7B-Instruct",
"Salesforce/xgen-mm-phi3-mini-instruct-interleave-r-v1.5",
"microsoft/Phi-4-multimodal-instruct",
"OpenGVLab/InternVL3-8B",
"llava-hf/llava-interleave-qwen-7b-hf",
"llava-hf/llava-onevision-qwen2-7b-ov-hf",
"mistralai/Pixtral-12B-2409",
# "Meize0729/CCExpert_7b", # omit
# "jirvin16/TEOChat", # omit
]

conda activate genai
python ./inference/xbd_subset_baseline.py
# or you can speficy the output file path, log file path and device
python ./inference/xbd_subset_baseline.py --output_file "./output/xbd_subset_baseline.jsonl" --log_file "./logs/xbd_subset_baseline.log" --device "cuda:0"

# inference/xbd_subset_baseline.py
...existing codes...
INFERENCE_MODEL_LIST = [ "jirvin16/TEOChat"]

conda activate teochat
python ./inference/xbd_subset_baseline.py
# or you can speficy the output file path, log file path and device

# inference/xbd_subset_baseline.py
...existing codes...
INFERENCE_MODEL_LIST = [ "Meize0729/CCExpert_7b"]

conda activate CCExpert
python ./inference/xbd_subset_baseline.py

Inference with Correction Decoding

python  ./inference_with_cd/inference_baseline_cd.py

Evaluation

Prepare Pre-trained Models

/path/to/model/folder
├── sentence-transformers/ # used for STS-SCS metric
│   └── sentence-t5-xxl/ # or use `sentence-t5-base` for faster evaluation
└── FacebookAI/ # used for BERTSCORE metric
    └── roberta-large/ # or use `roberta-base` for faster evaluation

Run Metrics

We calcuate BLEU, ROUGE, METEOR, BERTSCORE and Sentence-T5 Embedding Similarity for change captions between ground truth and other generated by baseline models.

conda activate genai
python ./evaluation/metrics.py \
--ground_truth_file ./output/xbd_subset_qvq.jsonl \
--predictions_file ./output/xbd_subset_baseline.jsonl > ./logs/eval.log

Fine-tuning RSCCM

cd RSCC
conda env create -f environment_qwenvl_ft.yaml
conda activate qwenvl_ft
bash train/qwen-vl-finetune/scripts/sft_for_rscc_model.sh

Auto Comparison with MLLMs (e.g. Qwen QvQ-Max)

We provide scripts that employ the latest visual reasoning proprietary model (QvQ-Max) to choose the best change caption from a series of candidates.

# API key for DashScope (keep this secret!)
DASHSCOPE_API_KEY="sk-xxxxxxxxxx"

# Model ID should match the official code
QVQ_MODEL_NAME="qvq-max-2025-03-25"

# API base URL
API_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1"

# Maximum concurrent workers
MAX_WORKERS=30

# Token threshold warning level
TOKEN_THRESHOLD=10000

conda activate genai
python ./evaluation/autoeval.py

Licensing Information

The dataset is released under the CC-BY-4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

🙏 Acknowledgement

Our RSCC dataset is built based on xBD and EBD datasets.

We are thankful to Kimi-VL, BLIP-3, Phi-4-Multimodal, Qwen2-VL, Qwen2.5-VL, LLaVA-NeXT-Interleave,LLaVA-OneVision, InternVL 3, Pixtral, TEOChat and CCExpert for releasing their models and code as open-source contributions.

The metrics implements are derived from huggingface/evaluate.

The training implements are derived from QwenLM/Qwen2.5-VL.

📜 Citation

@misc{chen2025rscclargescaleremotesensing,
      title={RSCC: A Large-Scale Remote Sensing Change Caption Dataset for Disaster Events},
      author={Zhenyuan Chen and Chenxi Wang and Ningyu Zhang and Feng Zhang},
      year={2025},
      eprint={2509.01907},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2509.01907},
}