Giters

Jimmycreative / awesome-video-anomaly-detection

Papers for Video Anomaly Detection, released codes collection, Performance Comparision.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

awesome-video-anomaly-detection Awesome

Papers for Video Anomaly Detection, released codes collections.

Any addition or bug please open an issue, pull requests or e-mail me by fjchange@hotmail.com

Recent Updated

  • AAAI 2022
  • CVPR 2022

Datasets

  1. UMN Download link
  2. UCSD Download link
  3. Subway Entrance/Exit Download link
  4. CUHK Avenue Download link
  5. ShanghaiTech Download link
  6. UCF-Crime (Weakly Supervised)
  7. Traffic-Train
  8. Belleview
  9. Street Scene (WACV 2020) Street Scenes, Download link
  10. IITB-Corridor (WACV 2020) Rodrigurs.etl
  11. XD-Violence (ECCV 2020) XD-ViolenceDownload link
  12. ADOC (ACCV 2020) ADOCDownload_link
  13. UBnormal (CVPR 2022) [UBnormal] Project Link Open-Set

The Datasets belowed are about Traffic Accidents Anticipating in Dashcam videos or Surveillance videos

  1. CADP (CarCrash Accidents Detection and Prediction)

  2. DAD paper, Download link

  3. A3D paper, Download link

  4. DADA Download link

  5. DoTA Download_link

  6. Iowa DOT Download_link

  7. Driver_Anomaly Project_link

Unsupervised

2016

  1. [Conv-AE] Learning Temporal Regularity in Video Sequences, CVPR 16. Code CUHK Avenue, Subway (Enter and Exit), and UCSD Pedestrian datasets (Ped1 and Ped2)

2017

  1. [Hinami.etl] Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge, ICCV 2017. (Explainable VAD) UCSD Ped2 and Avenue datasets
  2. [Stacked-RNN] A revisit of sparse coding based anomaly detection in stacked rnn framework, ICCV 2017. code CUHK Avenue,the UCSD Pedestrian, and the Subway Entrance and Exit dataset
  3. [ConvLSTM-AE] Remembering history with convolutional LSTM for anomaly detection, ICME 2017.Code moving MNIST,CUHK Avenue,the UCSD Pedestrian dataset [20], and the Subway Entrance and Exit dataset
  4. [Conv3D-AE] Spatio-Temporal AutoEncoder for Video Anomaly Detection,ACM MM 17.
  5. [Unmasking] Unmasking the abnormal events in video, ICCV 17. CUHK Avenue, the Subway, the UCSD and the UMN
  6. [DeepAppearance] Deep appearance features for abnormal behavior detection in video CUHK Avenue and UMN

2018

  1. [FramePred] Future Frame Prediction for Anomaly Detection -- A New Baseline, CVPR 2018. code CUHK Avenue, UCSD ped1 and ped2, ShanghaiTech
  2. [ALOOC] Adversarially Learned One-Class Classifier for Novelty Detection, CVPR 2018. code UCSD Ped2
  3. Detecting Abnormality Without Knowing Normality: A Two-stage Approach for Unsupervised Video Abnormal Event Detection, ACM MM 18. Avenue, the Subway and the UMN

2019

  1. [Mem-AE] Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection, ICCV 2019.code UCSD-Ped2, CUHK Avenue and ShanghaiTech
  2. [Skeleton-based] Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos, CVPR 2019.code ShanghaiTech Campus and CUHK Avenue
  3. [Object-Centric] Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection, CVPR 2019. Avenue, ShanghaiTech, UCSD and UMN
  4. [Appearance-Motion Correspondence] Anomaly Detection in Video Sequence with Appearance-Motion Correspondence, ICCV 2019.code CUHK Avenue, UCSD Ped2, Exit Gate, and Entrance Gate
  5. [AnoPCN]AnoPCN: Video Anomaly Detection via Deep Predictive Coding Network, ACM MM 2019. ShanghaiTech Campus, CUHK Avenue, and UCSD Ped2

2020

  1. [Street-Scene] Street Scene: A new dataset and evaluation protocol for video anomaly detection, WACV 2020. Street Scene
  2. [Rodrigurs.etl]) Multi-timescale Trajectory Prediction for Abnormal Human Activity Detection, WACV 2020. ShanghaiTech, CUHK avenue.
  3. [GEPC] Graph Embedded Pose Clustering for Anomaly Detection, CVPR 2020.code ShanghaiTech
  4. [Self-trained] Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection, CVPR 2020. UCSD ped2, UMN, subway
  5. [MNAD] Learning Memory-guided Normality for Anomaly Detection, CVPR 2020. code UCSD ped2, CUHK avenue, ShanghaiTech
  6. [Continual-AD]] Continual Learning for Anomaly Detection in Surveillance Videos,CVPR 2020 Worksop. UCSD ped2, CUHK Avenue, ShanghaiTech
  7. [OGNet] Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigm, CVPR 2020. code UCSD ped 2.
  8. [Any-Shot] Any-Shot Sequential Anomaly Detection in Surveillance Videos,CVPR 2020 workshop. UCSD ped, CUHK avenue, ShanghaiTech
  9. [Few-Shot]Few-Shot Scene-Adaptive Anomaly DetectionECCV 2020 Spotlight code ShanghaiTech, UCF crime, UCSD ped 1 and 2, UR fall.
  10. [CDAE]Clustering-driven Deep Autoencoder for Video Anomaly DetectionECCV 2020 UCSD pedestrian 2, the Avenue, and the ShanghaiTech dataset
  11. [VEC]Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video EventsACM MM 2020 Oral code UCSDped2, Avenue and ShanghaiTech
  12. [ADOC][A Day on Campus - An Anomaly Detection Dataset for Events in a Single Camera] ACCV 2020 ADOC dataset
  13. [CAC]Cluster Attention Contrast for Video Anomaly Detection ACM MM 2020 CUHK Avenue, ShanghaiTech
  14. [STC-Graph]Scene-Aware Context Reasoning for Unsupervised Abnormal Event Detection in Videos ACM MM 2020 UCF-Crime, Avenue, and ShanghaiTech datasets

2021

  1. [AMCM]Appearance-Motion Memory Consistency Network for Video Anomaly Detection AAAI 2021 UCSD Pedestrian(Ped1 and Ped2), CUHK Avenue, and ShanghaiTech dataset.
  2. [SSMT,Self-Supervised-Multi-Task]Anomaly Detection in Video via Self-Supervised and Multi-Task Learning CVPR 2021 Avenue, ShanghaiTech and UCSD Ped2
  3. [HF2-VAD]A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame PredictionICCV 2021 Oral UCSD Ped2,CUHK Avenue and ShanghaiTech
  4. [ROADMAP]Robust Unsupervised Video Anomaly Detection by Multipath Frame PredictionTNNLS 2021 CUHK Avenue, ShanghaiTech Campus, and UCSD Pedestrian datasets
  5. [AEP]Abnormal Event Detection and Localization via Adversarial Event Prediction TNNLS 2021 UCSD-Ped, CUHK Avenue, Subway, and UCF-Crime data sets.

2022

  1. [Casual]A Causal Inference Look At Unsupervised Video Anomaly DetectionAAAI 2022 UCSD, Subway surveillance, UMN, and Avenue.
  2. [BDPN]Comprehensive Regularization in a Bi-directional Predictive Network for Video Anomaly DetectionAAAI 2022 UCSD Ped2, CUHK Avenue, and ShanghaiTech datasets
  3. [GCL]Generative Cooperative Learning for Unsupervised Video Anomaly DetectionCVPR 2022 UCF and ShanghaiTech datasets

Weakly-Supervised

2018

  1. [Sultani.etl] Real-world Anomaly Detection in Surveillance Videos, CVPR 2018 code UCF

2019

  1. [GCN-Anomaly] Graph Convolutional Label Noise Cleaner:Train a Plug-and-play Action Classifier for Anomaly Detection, CVPR 2019, UCF-Crime, ShanghaiTech and UCSD-Peds code
  2. [MLEP] Margin Learning Embedded Prediction for Video Anomaly Detection with A Few Anomalies, IJCAI 2019code. CUHK Avenue and ShanghaiTech
  3. [IBL] Temporal Convolutional Network with Complementary Inner Bag Loss For Weakly Supervised Anomaly Detection. ICIP 19. UCF crime
  4. [Motion-Aware] Motion-Aware Feature for Improved Video Anomaly Detection. BMVC 19. UCF crime

2020

  1. [Siamese] Learning a distance function with a Siamese network to localize anomalies in videos, WACV 2020. UCSD Ped2, ShanghaiTech and CUHK Avenue
  2. [AR-Net] Weakly Supervised Video Anomaly Detection via Center-Guided Discrimative Learning, ICME 2020.code ShanghaiTech
  3. ['XD-Violence'] Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision ECCV 2020 XD-Violence
  4. [CLAWS] CLAWS: Clustering Assisted Weakly Supervised Learning with Normalcy Suppression for Anomalous Event Detection ECCV 2020 UCF-Crime and ShanghaiTech datasets

2021

  1. [MIST] MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection CVPR 2021 Project Page UCF-Crime and ShanghaiTech
  2. [RTFM] Weakly-supervised Video Anomaly Detection with Contrastive Learning of Long and Short-range Temporal Features ICCV 2021Code ShanghaiTech, UCF-Crime, XD-Violence and UCSD-Ped
  3. [STAD]Weakly-Supervised Spatio-Temporal Anomaly Detection in Surveillance Video IJCAI 2021 UCF
  4. [WSAL]Localizing Anomalies From Weakly-Labeled VideosTIP 2021 Code e UCF-Crime and Traffic anomaly datasets.
  5. [CRFD]Learning Causal Temporal Relation and Feature Discrimination for Anomaly DetectionTIP 2021 UCF

2022

  1. [MSL]Self-Training Multi-Sequence Learning with Transformer for Weakly Supervised Video Anomaly DetectionAAAI 2022 ShanghaiTech, UCF-Crime, and XD-Violence.

Supervised

2019

  1. [Background-Bias]Exploring Background-bias for Anomaly Detection in Surveillance Videos, ACM MM 19.
  2. [Ano-Locality]Anomaly locality in video suveillance.

Others

2020

  1. [Few-Shot]Few-Shot Scene-Adaptive Anomaly Detection ECCV 2020code

Reviews / Surveys

  1. An Overview of Deep Learning Based Methods for Unsupervised and Semi-Supervised Anomaly Detection in Videos, J. Image, 2018.page
  2. DEEP LEARNING FOR ANOMALY DETECTION: A SURVEY, paper
  3. Video Anomaly Detection for Smart Surveillance paper
  4. A survey of single-scene video anomaly detection, TPAMI 2020 paper.

Books

  1. Outlier Analysis. Charu C. Aggarwal

Specific Scene

Generally, anomaly detection in recent researches are based on the datasets from pedestrian (likes UCSD, Avenue, ShanghaiTech, etc.)， or UCF-Crime (real-world anomaly). However some focus on specific scene as follows.

Traffic

CVPR workshop, AI City Challenge series.

First-Person Traffic

​ Unsupervised Traffic Accident Detection in First-Person Videos, IROS 2019.

Driving

​ When, Where, and What? A New Dataset for Anomaly Detection in Driving Videos. github

Old-man Fall Down

Fighting/Violence

  1. Localization Guided Fight Action Detection in Surveillance Videos. ICME 2019.

Social/ Group Anomaly

  1. Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks, Neurips 2019.

Related Topics:

  1. Video Representation (Unsupervised Video Representation, reconstruction, prediction etc.)
  2. Object Detection
  3. Pedestrian Detection
  4. Skeleton Detection
  5. Graph Neural Networks
  6. GAN
  7. Action Recognition / Temporal Action Localization
  8. Metric Learning
  9. Label Noise Learning
  10. Cross-Modal/ Multi-Modal
  11. Dictionary Learning
  12. One-Class Classification / Novelty Detection / Out-of-Disturibution Detection
  13. Action Recognition.
    • Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events. ACM MM 2020 workshop.

Performance Evaluation Methods

  1. AUC
  2. PR-AUC
  3. Score Gap
  4. False Alarm Rate on Normal with 0.5 as threshold (Weakly supervised, proposed in CVPR 18)

As discussed in Issue #12, the reported results below will be Micro-AUC”, if the paper provide Macro-AUC", which will be tagged with *.

Performance Comparison on UCF-Crime

Model Reported on Convference/Journal Supervised Feature Encoder-based 32 Segments AUC (%) FAR@0.5 on Normal (%)
Sultani.etl CVPR 18 Weakly C3D RGB X √ 75.41 1.9
IBL ICIP 19 Weakly C3D RGB X √ 78.66 -
Motion-Aware BMVC 19 Weakly PWC Flow X √ 79.0 -
GCN-Anomaly CVPR 19 Weakly TSN RGB √ X 82.12 0.1
ST-Graph ACM MM 20 Un - √ X 72.7
Background-Bias ACM MM 19 Fully NLN RGB √ X 82.0 -
CLAWS ECCV 20 Weakly C3D RGB √ X 83.03 -
MIST CVPR 21 Weakly I3D RGB √ X 82.30 0.13
RTFM ICCV 21 Weakly I3D RGB X √ 84.03 -
WSAL TIP 21 Weakly I3D RGB X √ 85.38 -
CRFD TIP 21 Weakly I3D RGB X √ 84.89 -
MSL AAAI 22 Weakly C3D RGB √ X 82.85 -
MSL AAAI 22 Weakly I3D RGB √ X 85.30 -
MSL AAAI 22 Weakly VideoSwin-RGB √ X 85.62 -
GCL CVPR 22 Weakly ResNext √ X 79.84 -
GCL CVPR 22 Un ResNext √ X 71.04 -

Performance Comparison on ShanghaiTech

Model Reported on Conference/Journal Supervision Feature Encoder-based AUC(%) FAR@0.5 (%)
Conv-AE CVPR 16 Un - √ 60.85 -
stacked-RNN ICCV 17 Un - √ 68.0 -
FramePred CVPR 18 Un - √ 72.8 -
FramePred* IJCAI 19 Un - √ 73.4 -
Mem-AE ICCV 19 Un - √ 71.2 -
MNAD CVPR 20 Un - √ 70.5 -
VEC ACM MM 20 Un - √ 74.8 -
ST-Graph ACM MM 20 Un - √ 74.7 -
CAC ACM MM 20 Un - √ 79.3
AMMC AAAI 21 Un - √ 73.7 -
SSMT CVPR 21 Un - √ 82.4 -
HF2-VAD ICCV 21 Un - √ 76.2 -
ROADMAP TNNLS 21 Un - √ 76.6 -
BDPN AAAI 22 Un - √ 78.1 -
MLEP IJCAI 19 10% test vids with Video Anno - √ 75.6 -
MLEP IJCAI 19 10% test vids with Frame Anno - √ 76.8 -
Sultani.etl ICME 2020 Weakly (Re-Organized Dataset) C3D-RGB X 86.3 0.15
IBL ICME 2020 Weakly (Re-Organized Dataset) I3D-RGB X 82.5 0.10
GCN-Anomaly CVPR 19 Weakly (Re-Organized Dataset) C3D-RGB √ 76.44 -
GCN-Anomaly CVPR 19 Weakly (Re-Organized Dataset) TSN-Flow √ 84.13 -
GCN-Anomaly CVPR 19 Weakly (Re-Organized Dataset) TSN-RGB √ 84.44 -
AR-Net ICME 20 Weakly (Re-Organized Dataset) I3D-RGB & I3D Flow X 91.24 0.10
CLAWS ECCV 20 Weakly (Re-Organized Dataset) C3D-RGB √ 89.67
MIST CVPR 21 Weakly (Re-Organized Dataset) I3D-RGB √ 94.83 0.05
RTFM ICCV 21 Weakly (Re-Organized Dataset) I3D-RGB X 97.21 -
CRFD TIP 21 Weakly (Re-Organized Dataset) I3D-RGB X 97.48 -
MSL AAAI 22 Weakly (Re-Organized Dataset) C3D-RGB X 94.81 -
MSL AAAI 22 Weakly (Re-Organized Dataset) I3D-RGB X 96.08 -
MSL AAAI 22 Weakly (Re-Organized Dataset) VideoSwin-RGB X 97.32 -
GCL CVPR 22 Weakly (Re-Organized Dataset) ResNext X 86.21 -
GCL CVPR 22 Un ResNext X 78.93 -

Performance Comparison on Avenue

Model Reported on Conference/Journal Supervision Feature End2End AUC(%)
Conv-AE CVPR 16 Un - √ 70.2
Conv-AE* CVPR 18 Un - √ 80.0
ConvLSTM-AE ICME 17 Un - √ 77.0
DeepAppearance ICAIP 17 Un - √ 84.6
Unmasking ICCV 17 Un 3D gradients+VGG conv5 X 80.6
stacked-RNN ICCV 17 Un - √ 81.7
FramePred CVPR 18 Un - √ 85.1
Mem-AE ICCV 19 Un - √ 83.3
Appearance-Motion Correspondence ICCV 19 Un - √ 86.9
FramePred* IJCAI 19 Un - √ 89.2
MNAD CVPR 20 Un - √ 88.5
VEC ACM MM 20 Un - √ 90.2
ST-Graph ACM MM 20 Un - √ 89.6
CAC ACM MM 20 Un - √ 87.0
AMMC AAAI 21 Un - √ 86.6
SSMT CVPR 21 Un - √ 91.5
HF2-VAD ICCV 21 Un - √ 91.1
ROADMAP TNNLS 21 Un - √ 88.3
AEP TNNLS 21 Un - √ 90.2
Causal AAAI 22 Un I3D-RGB X 90.3
BDPN AAAI 22 Un - √ 90.3
MLEP IJCAI 19 10% test vids with Video Anno - √ 91.3
MLEP IJCAI 19 10% test vids with Frame Anno - √ 92.8

Performance Comparison on XD-Violence

Model Reported on Conference/Journal Supervision Feature Encoder-based 32 Segments AP(%)
Sultani et al. ECCV 2020 (reported by Wu) Weakly I3D-RGB X √ 73.20
Wu et al. ECCV 2020 Weakly C3D-RGB X X 67.19
Wu et al. ECCV 2020 Weakly I3D-RGB+Audio X X 78.64
RTFM ICCV 2021 Weakly I3D-RGB X √ 77.81
CRFD TIP 2021 Weakly I3D-RGB X √ 75.90
MSL AAAI 2022 Weakly C3D-RGB X X 75.53
MSL AAAI 2022 Weakly I3D-RGB X X 78.28
MSL AAAI 2022 Weakly VideoSwin-RGB X X 78.59

About

Papers for Video Anomaly Detection, released codes collection, Performance Comparision.