REMEMBERING HISTORY WITH CONVOLUTIONAL LSTM FOR ANOMALY DETECTION

By Weixin Luo, Wen Liu, Shenghua Gao

Introduction

All codes are based on Caffe. The main modifications are following

• Adding a convolutional lstm layer
• Changing the original Blob datastructure from 4-dimension tensor $(N \times C \times H \times W)$ to 5-dimension tensor $(T \times N \times C \times H \times W)$ tensor. Be careful that do not use cuDNN engien in all activation function layers in this motidified Caffe version, such as ReLU, sigmoid, tanh and so on, because cuDNN do not support 5-dimension in these activation function (but Conv layer is ok). If do that, it will be wrong.
• Adding a video data layers

Requirements for compiling

Since all the code are implemented on Caffe, so that in order to compile successfully, it must satisfy all requirements of Caffe (see the installization instruction). Whatmore, it also need to compile with Opencv 3.+, because the video data layer uses opencv libraries.

Datasets

• CUHK Avenue
• UCSD Pedestrians 1 & 2
• Subway Enter & Exit please to contact the orginal author to get the download links.

Training

As CUHK Avenue dataset for example, details are showd in the zstorm_conv_lstm_deconv/solver.prototxt and zstorm_conv_lstm_deconv/train.prototxt.

Testing or evaluation

Details are in the python shell, zstorm_conv_lstm_deconv/test.py.

License

All code are following the license of Caffe, and Caffe is released under the BSD 2-Clause license.

Citation

If you find this useful, please cite our work as follows:

@INPROCEEDINGS{luo2017remember, 
	author={W. Luo, W. Liu and S. Gao}, 
	booktitle={2017 IEEE International Conference on Multimedia and Expo (ICME)}, 
	title={Remembering history with convolutional LSTM for anomaly detection}, 
	year={2017}
}