VID-sentence Dataset

July 21, 2019 ยท View on GitHub

This repo contains the annotations of the VID-sentence dataset introduced in Weakly-Supervised Spatio-Temporally Grounding Natural Sentence in Video (WSSTG).

An example

Descriptions: "A large elephant runs in the water from left to right."

architecture

  1. Requirements: software
  2. Setup
  3. Tools

Requirements: software

  • python 3.6
  • cv2
  • shutil
  • commands
  • json
  • h5py
  • ffmpeg (for visualization)

Setup

  1. Download the original images Video Object Dection dataset (VID) from the official website.
  2. Create symlinks between the images of VID dataset and VID-sentence dataset.
  cd $VID-sentence_ROOT
  ln -s  $VID_ROOT/data/VID/train $VID-sentence_ROOT/data/VID/train
  ln -s  $VID_ROOT/data/VID/val $VID-sentence_ROOT/data/VID/val
  mv  $VID_ROOT/data/VID/test $VID-sentence_ROOT/data/VID/test_backup
  ln -s  $VID_ROOT/data/VID/val $VID-sentence_ROOT/data/VID/test

Note: the testing set of VID-sentence is generated by spliting the validation set of VID.

Tools

We give an example how to visualize the annotations of the dataset by running the following script.

sh vis_instance.sh

License

WSSTG is released under the CC-BY-NC 4.0 LICENSE (refer to the LICENSE file for details).

Citing WSSTG

If you find this dataset/repo useful in your research, please consider citing:

@inproceedings{chen2019weakly,
    Title={Weakly-Supervised Spatio-Temporally Grounding Natural Sentence in Video},
    Author={Chen, Zhenfang and Ma, Lin and Luo, Wenhan and Wong, Kwan-Yee K},
    Booktitle={ACL},
    year={2019}
}

Contact

You can contact Zhenfang Chen by sending email to chenzhenfang2013@gmail.com