MAC

By Runzhou Ge, Jiyang Gao, Kan Chen, Ram Nevatia.

University of Southern California (USC).

Introduction

This repository contains the code for the WACV 2019 paper, MAC: Mining Activity Concepts for Language-based Temporal Localization. arXiv

Requirements

• Python 2.7
• Tensorflow 1.0 or higher
• others

Download

The code is for Charades-STA dataset.

After cloning this repo, please donwload:

• visual features
• visual activity concepts
• ref_info

ref_info contains Charades-STA annotations, semantic activity concepts, checkpoints and others. After downloading ref_info.tar, untar it and move the folder to the root/ directory of this repo.

Please also change the visual feature and visual activity concepts directories in the main.py.

Training

For the paper results on Charades-STA dataset, run

python main.py --is_only_test True \
--checkpoint_path ./ref_info/charades_sta_wacv_2019_paper_ACL_k_results/trained_model.ckpt-10000 \
--test_name paper_results

You will get similar results listed in the row "ACL-K" of the following table.

Model	R@1,IoU=0.7	R@1,IoU=0.5	R@5,IoU=0.7	R@5,IoU=0.5
CTRL	7.15	21.42	26.91	59.11
ACL-K	12.20	30.48	35.13	64.84

To train the model from scratch, run

python main.py

The results and checkpoints will appear in root/results_history/ and root/trained_save/, respectively.

Results Visualization

Citation

If you find this work is helpful, please cite:

@InProceedings{Ge_2019_WACV,
  author = {Ge, Runzhou and Gao, Jiyang and Chen, Kan and Nevatia, Ram},
  title = {MAC: Mining Activity Concepts for Language-based Temporal Localization},
  booktitle = {The IEEE Winter Conference on Applications of Computer Vision (WACV)},
  month = {January},
  year = {2019}
}

License

MIT License

Acknowledgements

This research was supported, in part, by the Office of Naval Research under grant N00014-18-1-2050 and by an Amazon Research Award.