PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning

August 9, 2023 · View on GitHub

PUG: Photorealistic Unreal Graphics

<a href=https://pug.metademolab.com>**Website** | <a href=https://arxiv.org/abs/2308.03977>**Research Paper** | <a href=https://pug.metademolab.com/faq.html>**Datasheet**

https://github.com/facebookresearch/PUG/assets/5903040/5fd73746-a45b-4056-ae99-3726dadb51a8

This codebase contains:

download links for the PUG-datasets
dataloaders
scripts that are needed to samples images from a running interactive environment made with the Unreal Engine.
script to evaluate VLMs models with PUG: SPAR
list of the assets used to create the PUG datasets (which are listed in each PUG folders)

Downloading the PUG datasets

Here are the links to download the PUG datasets:

Dataset loaders

Please look at each PUG subfolder to get information on how to load the datasets.

How to create a PUG environment ?

The instruction are availables in the torchmultiverse folder.

LICENSE

The datasets are distributed under the CC-BY-NC, with the addenda that they should not be used to train Generative AI models, as found in the LICENSE file.

Citing PUG

If you use the PUG datasets, please cite:

@misc{bordes2023pug,
      title={PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning}, 
      author={Florian Bordes and Shashank Shekhar and Mark Ibrahim and Diane Bouchacourt and Pascal Vincent and Ari S. Morcos},
      year={2023},
      eprint={2308.03977},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}