awesome-video-anomaly-detection

Papers for Video Anomaly Detection, released codes collections.

Any addition or bug please open an issue, pull requests or e-mail me by

[email protected]

Datasets

1. UMN

   Download link

2. UCSD

   Download link

3. Subway Entrance/Exit

   Download link

4. CUHK Avenue

   Download link

   • HD-Avenue Skeleton-based
5. ShanghaiTech

   Download link

   • HD-ShanghaiTech Skeleton-based
6. UCF-Crime (Weakly Supervised)
   • UCFCrime2Local (subset of UCF-Crime but with spatial annotations.)

     Download_link

     , Ano-Locality
   • Spatial Temporal Annotations

     Download_link

     Background-Bias
7. Traffic-Train
8. Belleview
9. Street Scene (WACV 2020) Street Scenes,

   Download link

10. IITB-Corridor (WACV 2020) Rodrigurs.etl
11. XD-Violence (ECCV 2020) XD-Violence

    Download link

12. ADOC (ACCV 2020) ADOC

    Download_link

The Datasets belowed are about Traffic Accidents Anticipating in Dashcam videos or Surveillance videos

1. CADP (CarCrash Accidents Detection and Prediction)
2. DAD paper,

   Download link

3. A3D paper,

   Download link

4. DADA

   Download link

5. DoTA

   Download_link

6. Iowa DOT

   Download_link

1. Driver_Anomaly Project_link

Unsupervised

2016

1. [Conv-AE] Learning Temporal Regularity in Video Sequences,

   CVPR 16

   . Code ### 2017
2. [Hinami.etl] Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge,

   ICCV 2017

   . (Explainable VAD)
3. [Stacked-RNN] A revisit of sparse coding based anomaly detection in stacked rnn framework,

   ICCV 2017

   . code
4. [ConvLSTM-AE] Remembering history with convolutional LSTM for anomaly detection,

   ICME 2017

   .Code
5. [Conv3D-AE] Spatio-Temporal AutoEncoder for Video Anomaly Detection,

   ACM MM 17

   .
6. [Unmasking] Unmasking the abnormal events in video,

   ICCV 17

   .
7. [DeepAppearance] Deep appearance features for abnormal behavior detection in video ### 2018
8. [FramePred] Future Frame Prediction for Anomaly Detection -- A New Baseline,

   CVPR 2018

   . code
9. [ALOOC] Adversarially Learned One-Class Classifier for Novelty Detection,

   CVPR 2018

   . code
10. Detecting Abnormality Without Knowing Normality: A Two-stage Approach for Unsupervised Video Abnormal Event Detection,

    ACM MM 18

    .

2019

1. [Mem-AE] Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection,

   ICCV 2019

   .code
2. [Skeleton-based] Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos,

   CVPR 2019

   .code
3. [Object-Centric] Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection,

   CVPR 2019

   .
4. [Appearance-Motion Correspondence] Anomaly Detection in Video Sequence with Appearance-Motion Correspondence,

   ICCV 2019

   .code
5. [AnoPCN]AnoPCN: Video Anomaly Detection via Deep Predictive Coding Network, ACM MM 2019. ### 2020
6. [Street-Scene] Street Scene: A new dataset and evaluation protocol for video anomaly detection,

   WACV 2020

   .
7. [Rodrigurs.etl]) Multi-timescale Trajectory Prediction for Abnormal Human Activity Detection,

   WACV 2020

   .
8. [GEPC] Graph Embedded Pose Clustering for Anomaly Detection,

   CVPR 2020

   .code
9. [Self-trained] Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection,

   CVPR 2020

   .
10. [MNAD] Learning Memory-guided Normality for Anomaly Detection,

    CVPR 2020

    . code
11. [Continual-AD]] Continual Learning for Anomaly Detection in Surveillance Videos,

    CVPR 2020 Worksop.

12. [OGNet] Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigm,

    CVPR 2020

    . code
13. [Any-Shot] Any-Shot Sequential Anomaly Detection in Surveillance Videos,

    CVPR 2020 workshop

    .
14. [Few-Shot]Few-Shot Scene-Adaptive Anomaly Detection

    ECCV 2020 Spotlight

    code
15. [CDAE]Clustering-driven Deep Autoencoder for Video Anomaly Detection

    ECCV 2020

16. [VEC]Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Events

    ACM MM 2020 Oral

    code
17. [ADOC][A Day on Campus - An Anomaly Detection Dataset for Events in a Single Camera]

    ACCV 2020

18. [CAC]Cluster Attention Contrast for Video Anomaly Detection

    ACM MM 2020

19. [STC-Graph]Scene-Aware Context Reasoning for Unsupervised Abnormal Event Detection in Videos

    ACM MM 2020

2021

1. [AMCM]Appearance-Motion Memory Consistency Network for Video Anomaly Detection

   AAAI 2021

2. [SSMT,Self-Supervised-Multi-Task]Anomaly Detection in Video via Self-Supervised and Multi-Task Learning

   CVPR 2021

3. [HF2-VAD]A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction

   ICCV 2021 Oral

4. [ROADMAP]Robust Unsupervised Video Anomaly Detection by Multipath Frame Prediction

   TNNLS 2021

5. [AEP]Abnormal Event Detection and Localization via Adversarial Event Prediction

   TNNLS 2021

Weakly-Supervised

2018

1. [Sultani.etl] Real-world Anomaly Detection in Surveillance Videos,

   CVPR 2018

   code ### 2019
2. [GCN-Anomaly] Graph Convolutional Label Noise Cleaner:Train a Plug-and-play Action Classifier for Anomaly Detection,

   CVPR 2019

   , code
3. [MLEP] Margin Learning Embedded Prediction for Video Anomaly Detection with A Few Anomalies,

   IJCAI 2019

   code.
4. [IBL] Temporal Convolutional Network with Complementary Inner Bag Loss For Weakly Supervised Anomaly Detection.

   ICIP 19

   .
5. [Motion-Aware] Motion-Aware Feature for Improved Video Anomaly Detection.

   BMVC 19

   . ### 2020
6. [Siamese] Learning a distance function with a Siamese network to localize anomalies in videos,

   WACV 2020

   .
7. [AR-Net] Weakly Supervised Video Anomaly Detection via Center-Guided Discrimative Learning,

   ICME 2020

   .code
8. ['XD-Violence'] Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision

   ECCV 2020

9. [CLAWS] CLAWS: Clustering Assisted Weakly Supervised Learning with Normalcy Suppression for Anomalous Event Detection

   ECCV 2020

   ### 2021
10. [MIST] MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection

    CVPR 2021

    Project Page
11. [RTFM] Weakly-supervised Video Anomaly Detection with Contrastive Learning of Long and Short-range Temporal Features

    ICCV 2021

    Code
12. [STAD]Weakly-Supervised Spatio-Temporal Anomaly Detection in Surveillance Video

    IJCAI 2021

13. [WSAL]Localizing Anomalies From Weakly-Labeled Videos

    TIP 2021

    Code
14. [CRFD]Learning Causal Temporal Relation and Feature Discrimination for Anomaly Detection

    TIP 2021

    ## Supervised ### 2019
15. [Background-Bias]Exploring Background-bias for Anomaly Detection in Surveillance Videos,

    ACM MM 19

    .
16. [Ano-Locality]Anomaly locality in video suveillance.

Others

2020

1. [Few-Shot]Few-Shot Scene-Adaptive Anomaly Detection

ECCV 2020

code

Reviews / Surveys

1. An Overview of Deep Learning Based Methods for Unsupervised and Semi-Supervised Anomaly Detection in Videos, J. Image, 2018.page
2. DEEP LEARNING FOR ANOMALY DETECTION: A SURVEY, paper
3. Video Anomaly Detection for Smart Surveillance paper

Books

1. Outlier Analysis. Charu C. Aggarwal ## Specific Scene

Generally, anomaly detection in recent researches are based on the datasets from pedestrian (likes UCSD, Avenue, ShanghaiTech, etc.)， or UCF-Crime (real-world anomaly). However some focus on specific scene as follows.

Traffic

CVPR workshop, AI City Challenge series.

First-Person Traffic

​ Unsupervised Traffic Accident Detection in First-Person Videos, IROS 2019.

Driving

​ When, Where, and What? A New Dataset for Anomaly Detection in Driving Videos. github

Old-man Fall Down

Fighting/Violence

1. Localization Guided Fight Action Detection in Surveillance Videos. ICME 2019.

Social/ Group Anomaly

1. Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks, Neurips 2019.

Related Topics:

1. Video Representation (Unsupervised Video Representation, reconstruction, prediction etc.)
2. Object Detection
3. Pedestrian Detection
4. Skeleton Detection
5. Graph Neural Networks
6. GAN
7. Action Recognition / Temporal Action Localization
8. Metric Learning
9. Label Noise Learning
10. Cross-Modal/ Multi-Modal
11. Dictionary Learning
12. One-Class Classification / Novelty Detection / Out-of-Disturibution Detection
13. Action Recognition.
    • Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events. ACM MM 2020 workshop.

Performance Evaluation Methods

1. AUC
2. PR-AUC
3. Score Gap
4. False Alarm Rate on Normal with 0.5 as threshold (Weakly supervised, proposed in CVPR 18)

Performance Comparison on UCF-Crime

| Model | Reported on Convference/Journal | Supervised | Feature | Encoder-based | 32 Segments | AUC (%) | [email protected] on Normal (%) | | --------------------------------------------------- | ------------------------------- | ---------- | -------- | ------- | ----------- | ------- | --------------------- | | Sultani.etl | CVPR 18 | Weakly | C3D RGB | X | √ | 75.41 | 1.9 | | IBL | ICIP 19 | Weakly | C3D RGB | X | √ | 78.66 | - | | Motion-Aware | BMVC 19 | Weakly | PWC Flow | X | √ | 79.0 | - | | GCN-Anomaly | CVPR 19 | Weakly | TSN RGB | √ | X | 82.12 | 0.1 | | ST-Graph | ACM MM 20 | Un | - | √ | X | 72.7 | | | Background-Bias | ACM MM 19 | Fully | NLN RGB | √ | X | 82.0 | - | | CLAWS | ECCV 20 | Weakly | C3D RGB | √ | X | 83.03 | - | | MIST | CVPR 21 | Weakly | I3D RGB | √ | X | 82.30 | 0.13 | | RTFM | ICCV 21 | Weakly | I3D RGB | X | √ | 84.03 | - | | WSAL | TIP 21 | Weakly | I3D RGB | X | √ | 85.38 | - | | CRFD | TIP 21 | Weakly | I3D RGB | X | √ | 84.89 | - |

Performance Comparison on ShanghaiTech

| Model | Reported on Conference/Journal | Supervision | Feature | Encoder-based | AUC(%) | [email protected] (%) | | ------------------------------------------------- | ------------------------------ | ----------------------------- | ------------------ | ------- | ------ | ----------- | | Conv-AE | CVPR 16 | Un | - | √ | 60.85 | - | | stacked-RNN | ICCV 17 | Un | - | √ | 68.0 | - | | FramePred | CVPR 18 | Un | - | √ | 72.8 | - | | FramePred* | IJCAI 19 | Un | - | √ | 73.4 | - | | Mem-AE | ICCV 19 | Un | - | √ | 71.2 | - | | MNAD | CVPR 20 | Un | - | √ | 70.5 | - | | VEC | ACM MM 20 | Un | - | √ | 74.8 | - | | ST-Graph | ACM MM 20 | Un | - | √ | 74.7 | - | | CAC | ACM MM 20 | Un | - | √ | 79.3 | | | AMMC | AAAI 21 | Un | - | √ | 73.7 | - | | SSMT | CVPR 21 | Un | - | √ | 90.2 | - | | HF2-VAD | ICCV 21 | Un | - | √ | 76.2 | - | | ROADMAP | TNNLS 21 | Un | - | √ | 76.6 | - | | MLEP | IJCAI 19 | 10% test vids with Video Anno | - | √ | 75.6 | - | | MLEP | IJCAI 19 | 10% test vids with Frame Anno | - | √ | 76.8 | - | | Sultani.etl | ICME 2020 | Weakly (Re-Organized Dataset) | C3D-RGB | X | 86.3 | 0.15 | | IBL | ICME 2020 | Weakly (Re-Organized Dataset) | I3D-RGB | X | 82.5 | 0.10 | | GCN-Anomaly | CVPR 19 | Weakly (Re-Organized Dataset) | C3D-RGB | √ | 76.44 | - | | GCN-Anomaly | CVPR 19 | Weakly (Re-Organized Dataset) | TSN-Flow | √ | 84.13 | - | | GCN-Anomaly | CVPR 19 | Weakly (Re-Organized Dataset) | TSN-RGB | √ | 84.44 | - | | AR-Net | ICME 20 | Weakly (Re-Organized Dataset) | I3D-RGB & I3D Flow | X | 91.24 | 0.10 | | CLAWS | ECCV 20 | Weakly (Re-Organized Dataset) | C3D-RGB | √ | 89.67 | | | MIST | CVPR 21 | Weakly (Re-Organized Dataset) | I3D-RGB | √ | 94.83 | 0.05 | | RTFM | ICCV 21 | Weakly (Re-Organized Dataset) | I3D-RGB | X | 97.21 | - | | CRFD | TIP 21 | Weakly (Re-Organized Dataset) | I3D-RGB | X | 97.48 | - |

Performance Comparison on Avenue

| Model | Reported on Conference/Journal | Supervision | Feature | End2End | AUC(%) | | ------------------------------------------------------------ | ------------------------------ | ----------------------------- | ---------------------- | ------- | ------ | | Conv-AE | CVPR 16 | Un | - | √ | 70.2 | | Conv-AE* | CVPR 18 | Un | - | √ | 80.0 | | ConvLSTM-AE | ICME 17 | Un | - | √ | 77.0 | | DeepAppearance | ICAIP 17 | Un | - | √ | 84.6 | | Unmasking | ICCV 17 | Un | 3D gradients+VGG conv5 | X | 80.6 | | stacked-RNN | ICCV 17 | Un | - | √ | 81.7 | | FramePred | CVPR 18 | Un | - | √ | 85.1 | | Mem-AE | ICCV 19 | Un | - | √ | 83.3 | | Appearance-Motion Correspondence | ICCV 19 | Un | - | √ | 86.9 | | FramePred* | IJCAI 19 | Un | - | √ | 89.2 | | MNAD | CVPR 20 | Un | - | √ | 88.5 | | VEC | ACM MM 20 | Un | - | √ | 90.2 | | ST-Graph | ACM MM 20 | Un | - | √ | 89.6 | | CAC | ACM MM 20 | Un | - | √ | 87.0 | | AMMC | AAAI 21 | Un | - | √ | 86.6 | | SSMT | CVPR 21 | Un | - | √ | 92.8 | | HF2-VAD | ICCV 21 | Un | - | √ | 91.1 | | ROADMAP | TNNLS 21 | Un | - | √ | 88.3 | | AEP | TNNLS 21 | Un | - | √ | 90.2 | | MLEP | IJCAI 19 | 10% test vids with Video Anno | - | √ | 91.3 | | MLEP | IJCAI 19 | 10% test vids with Frame Anno | - | √ | 92.8 |

Performance Comparison on XD-Violence

| Model | Reported on Conference/Journal | Supervision | Feature | Encoder-based | 32 Segments | AP(%) | | ----------------------------------------------------- | ------------------------------ | ------------------------ | ------------------- | ------- |-------------| ------ | | Sultani et al. | ECCV 2020 (reported by Wu) | Weakly | I3D-RGB | X | √ | 73.20 |
| Wu et al. | ECCV 2020 | Weakly | C3D-RGB | X | X | 67.19 | | Wu et al. | ECCV 2020 | Weakly | I3D-RGB+Audio | X | X | 78.64 | | RTFM | ICCV 2021 | Weakly | I3D-RGB | X | √ | 77.81 | | CRFD | TIP 2021 | Weakly | I3D-RGB | X | √ | 75.90 |