Offcial implementation of "A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction, ICCV-2021".

HF2-VAD

Offcial implementation of "A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction, ICCV-2021".

[Paper] [Supp] [arXiv]

1. Dependencies

python==3.6
pytorch==1.5.1
mmcv-full==1.3.1
mmdet==2.11.0
scikit-learn==0.23.2
edflow==0.4.0
PyYAML==5.4.1
tensorboardX==2.4

2. Usage

2.1 Data preparation

Please follow the instructions to prepare the training and testing dataset.

2.2 Train

We train the ML-MemAE-SC at first, then train CVAE model with the reconstructed flows, and finally finetune the whole framework. All the config files are located at ./cfgs.

To train the ML-MemAE-SC, run:

$ python ml_memAE_sc_train.py

To train the CVAE model with reconstructed flows, run:

$ python trian.py

And finetune the whole HF2VAD framework together as:

$ python finetune.py

For different datasets, please modify the configuration files accordingly.

2.3 Evaluation

To evaluation the anomaly detection performance of the trained model, run:

$ python eval.py [--model_save_path] [--cfg_file] 

E.g., for the ped2 dataset:

$ python eval.py \
         --model_save_path=./pretrained_ckpts/ped2_HF2VAD_99.31.pth \
         --cfg_file=./pretrained_ckpts/ped2_HF2VAD_99.31_cfg.yaml

You can download the pretrained weights of HF2VAD for Ped2, Avenue and ShanghaiTech datasets from here.

3. Results

Acknowledgment

We thank jhaux for the PyTorch implementation of the conditional VAE.

Citation

If you find this repo useful, please consider citing:

@inproceedings{liu2021hf2vad,
title = {A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction},
author = {Liu, Zhian and Nie, Yongwei and Long, Chengjiang and Zhang, Qing and Li, Guiqing},
booktitle={Proceedings of the IEEE International Conference on Computer Vision},
year = {2021}
}