Dataset Preparation

July 19, 2023 ยท View on GitHub

We follow VINDLU to prepare the datasets, but we DO NOT compress the videos and images. We use the original data and load the JSON files, since there are some communication problems for SQLite in our environment.

:warning: If you do not have enough resources, we suggest you follow the preprocessing of VINDLU.

:label: We use the same JSON files provided by VINDLU. However, since some vides are missing in large-scale datasets (like CC3M, CC12M and WebVid10M), we filter out those unavaliable videos.

Pretraining

Video-Text Retrieval and Video Question Answering