Data preparation for Referral Video Object Segmentation

For Ref-DAVIS17 and Ref-YouTube-VOS, we follow preprocessing as in ReferFormer. Preprocessing scripts are not included in this repo.

Different from ReferFormer, download additionally captions for each dataset provided below.

Ref-DAVIS17

Downlaod the DAVIS2017 dataset from the website. Note that you only need to download the two zip files DAVIS-2017-Unsupervised-trainval-480p.zip and DAVIS-2017_semantics-480p.zip. Download the text annotations from the website. Then, put the zip files in the directory as follows.

Captions for validation videos download here.

refam
├── data
│   ├── ref-davis
│   │   ├── DAVIS-2017_semantics-480p.zip
│   │   ├── DAVIS-2017-Unsupervised-trainval-480p.zip
│   │   ├── davis_text_annotations.zip
│   │   ├── davis_gpt_captions.json

Unzip these zip files.

unzip -o davis_text_annotations.zip
unzip -o DAVIS-2017_semantics-480p.zip
unzip -o DAVIS-2017-Unsupervised-trainval-480p.zip

Preprocess the dataset to Ref-Youtube-VOS format. (Make sure you are in the main directory)

python tools/data/convert_davis_to_ytvos.py

Finally, unzip the file DAVIS-2017-Unsupervised-trainval-480p.zip again (since we use mv in preprocess for efficiency).

unzip -o DAVIS-2017-Unsupervised-trainval-480p.zip

Ref-Youtube-VOS

Download the dataset from the competition's website here. Then, extract and organize the file. We expect the directory structure to be the following:

refam
├── data
│   ├── ref-youtube-vos
│   │   ├── valid
│   │   │   ├── JPEGImages
│   │   │   ├── Annotations
│   │   │   ├── meta_expressions_challenge.json
│   │   │   ├── youtube_gpt_captions.json

Captions for validation videos download here.
In case, it is hard to find: meta_expressions_challenge.json

MeViS

Follow instructions here

Captions for validation videos download here.