Empirical Review of the Deep Learning Solutions for Anomaly Detection in Surveillance Videos

<span class="underline">Authors</span>

# Abstract

Deep learning has proven to be a landmark computing approach to the computer vision domain. Hence, it has been widely applied to solve complex cognitive tasks like the detection of anomalies in surveillance videos. Anomaly detection in this case is the identification of abnormal events in the surveillance videos which can be deemed as security incidents or threats.

Deep learning solutions for anomaly detection has outperformed other traditional machine learning solutions. This review attempts to provide holistic benchmarking of the published deep learning solutions for videos anomaly detection since 2016. The paper identifies, the learning technique, dataset used and the overall model accuracy.

The most common learning techniques identified herein include; autoencoders, continual learning, multiple-instance learning and transfer learning. In particular, this paper also reviews, the role of pre-trained models used in transfer and continual learning, like the C3D feature extractor, I3D and YOLOv3.

This paper identifies the most popular, underlying deep learning algorithms like CovNets, Conv2d, ConvLSTM and ConvLSTM2D. The datasets used in training these models are identified and the most popular ones like UCSD Ped1& Ped2 and UCF Crime are further explored.

Finally, the paper ranks the overall performance of the published solutions and further, it evaluates empirically a few models to determine the best deep learning solution for anomaly detection. The Receiver Operating Characteristic Curve and its underlying area under the curve have been used as the metric for evaluation of the models.

Keywords

Deep Learning, Anomaly Detection, Surveillance Automation, Surveillance Videos, Anomaly Detection Review.

# Introduction

In the recent past, the use of surveillance cameras has rapidly increased to enhance public safety. Unfortunately, the security forces ability to monitor these surveillance footages has not kept up with the surveillance data generation velocity and volume \[1\].

This scenario has resulted in a critical problem in the utilization of the surveillance footage since more human monitors are required as the surveillance cameras increase. The monitoring task requires dedicated attention since anomalous events are very rare. Hence, human monitors might miss out to signal security incidents.

Researchers have utilized supervised learning methods to develop algorithms to detect specific anomalous events. For instance; traffic accident detectors \[2\] \[3\], violence detectors, home intrusion detectors \[4\] and shoplifting detectors \[5\]. Unfortunately, these early solutions cannot be generalized to detect other abnormal events/actions since they have limited use.

To address such shortcomings, from the supervised models, other researchers proposed to use unsupervised learning algorithms. For instance, Waqas Sultani \[6\] proposes Multiple Instance Learning that can be generalized across a variety of anomalies. Chong \[7\] also proposes to use the Conv2DLSTM Autoencoder as a solution for the generalization shortcomings.

This paper examined the deep learning models published since 2016. Only the deep learning solutions in anomaly detection in surveillance videos were considered. An empirical review was conducted to cross-examine the datasets, model accuracy, learning technique and the underlying deep learning algorithms. Mostly, open-source papers were used in this study, due to their unlimited availability. A similar dataset was used to train and test the selected models where applicable.

# 2.0 Related Works

Other landmark publications in deep learning anomaly detection in surveillance videos that have received much attention include multiple instance and transfer learning publications \[8\], \[3\] \[9\] \[10\]. Other early attempts include anomaly detectors meant for specific anomalies in the insecurity problem.

The initial solutions developed towards solving the anomaly detection problem focused on specific abnormal events detection like road accidents detector \[2\], home intrusion detector \[4\] and violence detector \[11\].

Kushwaha \[12\] and others use motion sensors and a video camera to detect cases of human intrusion. This system uses frame referencing and segmentation to solve the problem of force alarms and remove the outliers.

Farooq and others used behaviour analysis to solve the anomaly detection problem. This is an example of unsupervised learning \[2\]. This system is limited to traffic streets only. The system used Gaussian Mixture Model (GMM), to identify and track objects. Kalman Filter is also used to indicate vehicle track while Euclidean distance is applied to analyse vehicles trajectory and their speed. Cases of overspeeding and sudden turns are considered anomalies. Although these assorted models are unable to address all traffic anomalies since traffic incidents are more complicated \[2\].

Other works related to this study are constrained to deep learning only. This works includes Kushwaha 2018 \[4\] which is an improvement of his earlier work on home intrusion. The model is implemented to detect intruders. Using home surveillance videos, the models utilizes Convolutional Neural Networks (CNN), to detect motion and flag intrusion. This system achieves state-of-the-art performance, although it is vulnerable to false alarms since it cannot differentiate threat and no-threat motions.

Another specialized case of anomaly detection using deep learning is the shoplifting detector. This application utilizes open pose technology from the Carnegie Mellon University and Convolutional Neural Network \[5\]. The open pose technology can identify the pose of a person and the suspicious behaviour is derived from the pose. This model suffers from false alarms since it fails to distinguish shoplifters from window shoppers and indecisive shoppers.

There are numerous attempts made by various researchers to introduce unsupervised learning solutions that can generalize on a variety of anomalies. Sabokrou and others 2017, developed a light-deep 3-D spatial autoencoder to identify anomalies in crowded scenes. Somewhat, the model can detect different anomalies occurring in crowded scenes \[13\].

Waqas Sultani \[6\] developed a Multiple Instance Learning model that utilized weakly labelled training videos, used a pre-trained C3D feature extractor and a fully connected neural network. The Sultani model is trained on both normal and abnormal videos, which are used to generate the ranking bags of normal and abnormal instances. A novel ranking loss function is used to estimate the anomaly level for every video.

Other solutions presented for anomaly detection include the use of Autoencoders. An auto-encoder can be considered as a non-linear transformation of the input, which then utilizes a differentiable equation to decode the transformed sequences to the input sequence \[14\]. Variations of auto-encoders exist, with Convolutional Auto-encoder using reconstruction obtained and mapping them to the original video frame to extrapolate the irregularity score.

Chong and Tay \[7\] Spatial-temporal auto-encoder is another variation of auto-encoders that utilizes spatial convolution, that extracts features from input images and temporal convolutional LSTM. This paradigm consists of a spatial encoder, temporal encoder, temporal decoder and spatial decoder. The basic constructs of the model are ConvNet & ConvLSTM2D.

Many deep learning models have been implemented for anomaly detection in surveillance videos. Other popular learning techniques include transfer learning, which combines pre-trained models for feature extraction like YOLOv3 \[15\], optical flow \[16\], C3D and I3D \[17\] models and other deep learning algorithms.

Rapid improvements are built continuously, to achieve state-of-the-art performance in anomaly detection in surveillance videos. Amid these refinements, the models are still challenged since false alarm rates are still present and the models still take a lot of training time. Convolutional Neural Networks and ConvLSTM algorithms are the most popular deep learning algorithms used in these models. An even bigger question arises; what is the best deep learning model so far?

# Methodology

A survey of papers published since 2016 up to date, in the field of anomaly detection in surveillance videos using deep learning was conducted in Google Scholar, the survey considered the open-source papers mostly for the ease of accessibility.

The review was constrained to deep learning proposed solutions or hybrid solutions. At least *n=*30 papers were considered. From the, *n* papers reviewed, the underlying deep learning algorithms were identified as well as their resultant ROC curve scores for their respective accuracies. From these publications, the deep learning models that had shared their complete implementation code were selected for empirical analysis. The papers were implemented to ran in Google Research Collaboratory Platform and their performance was further investigated. Frame level evaluation was conducted to frame-level performance. As a result, the best performing model was considered the best for anomaly detection.

# Experiments

## 4.1 Datasets

Publicly available, anomaly detection datasets were used for the experiments. UCF Crime dataset, UCSD Ped1 & Ped2, and Avenue Dataset. The UCF Crime dataset is 1900 hours long videos dataset that was introduced by Sultani \[6\], it is composed of real-life anomalies like Arrest, Arson, Abuse and many others. The training set has both abnormal and normal videos as well as the testing set. Although usage of both classes is dependent upon the nature of the model to be trained. For instance, in the auto-encoder model, only the normal videos are used for training while in his model was trained by both normal and abnormal videos \[6\].

University of California San Diego (UCSD) Ped1 & Ped2 datasets are used for training and testing. UCSD Ped1 is composed of 70 videos with 34 as the training set and 36 as the testing set. The videos scenery is a group of people walking in a park. Anomalies include non-pedestrian entities like bikers, skaters, carts, Wheelchairs and people walking in the grass area \[18\].

Avenue dataset \[19\] contains 16 training and 21 testing video clips. A total of 30652 frames are available in the dataset. These videos are captured on a campus street using a still camera. Strange actions like the running of persons, riding a bike in the walkway are the abnormal events presented.

## 4.2 Models

A variety of models were discovered from the empirical review. All the models selected for the empirical review had some deep learning components in them. Some models combine both deep learning and traditional machine learning algorithms like Support Vector Machine (SVM) \[6\]. The most popular underlying deep learning algorithms are 3dcnn, convlstm and conv2dlstm.

Google Colab environment was used to run the models. The models considered for empirical review were available in Github, with all the necessary implementation code and running configurations and parameters shared.

Waqas Sultani \[6\], Multiple Instance Learning model and Chong \[7\] Spatial-Temporal encoders are among the models selected for implementation and further empirical analysis from the survey conducted. Apart from these models being openly accessible and their code publicly shared in GitHub, they are very popular with the Sultani model being borrowed for improvement by many researchers like \[20\], \[9\] \[21\]. Chong \[7\] founded the idea of using unsupervised learning for anomaly detection which inspired other similar solutions like \[16\] \[22\] \[23\] \[10\]. Wide borrowing by other researchers makes these models popular, hence the motivation to further investigate.

Hybrid models required the extraction of features from videos before the videos were fed to the model. For instance, the pre-trained C3D Facebook feature extractor \[6\], Inception Version 3(I3D) \[17\], YOLO \[15\] , Optical Flow Extractor \[16\]. These features extractor models are very common video preprocessing methods.

# Results

## 5.1 Outcome of the review

The review of the Deep Learning Anomaly Detection in surveillance videos publications was the first part of this study. The study was conducted on Google Scholar, with a limitation of only papers published since 2016 to date. Papers with open access were prioritized since they were accessible and due to the budget constraint. The study reviewed *n=30* papers, which focused only on the deep learning solutions of anomaly detection in surveillance videos.

<table>
<thead>
<tr class="header">
<th><strong>Publication</strong></th>
<th><strong>Learning Technique</strong></th>
<th><strong>Deep Learning Algorithm/Models</strong></th>
<th><strong>Datasets</strong></th>
<th><strong>Overall Accuracy</strong></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>[7]</td>
<td>Auto-encoder</td>
<td>ConvLSTMAE</td>
<td><span class="underline">UCSD Ped1, UCSD Ped2</span></td>
<td><span class="underline">94%</span></td>
</tr>
<tr class="even">
<td>[23]</td>
<td>Auto-encoder</td>
<td><span class="underline">ConvAE</span></td>
<td><span class="underline">n/a</span></td>
<td><span class="underline">n/a</span></td>
</tr>
<tr class="odd">
<td>[22]</td>
<td>Fully Conv Feed Forward Auto-encoder</td>
<td>FC Convnet</td>
<td><p>UCSD Ped1</p>
<p>UCSD Ped2</p>
<p>CHUK Avenue</p></td>
<td><span class="underline">83.18%</span></td>
</tr>
<tr class="even">
<td>[24]</td>
<td>Robust PCA</td>
<td>PCA</td>
<td>Cifar10</td>
<td><span class="underline">89%</span></td>
</tr>
<tr class="odd">
<td>[6]</td>
<td>Multiple Instance Learning (MIL)</td>
<td><p>FC Convnet</p>
<p>SVM Classfier</p></td>
<td>UCF Crime Dataset</td>
<td><span class="underline">75.41%</span></td>
</tr>
<tr class="even">
<td>[3]</td>
<td>Generative Adversarial Network</td>
<td>GAN</td>
<td><span class="underline">AI City Challenge</span></td>
<td><span class="underline">94%</span></td>
</tr>
<tr class="odd">
<td>[25]</td>
<td>Variation Auto-encoder</td>
<td>2 stream VAE/ GAN</td>
<td>n/a</td>
<td><span class="underline">n/a</span></td>
</tr>
<tr class="even">
<td>[9]</td>
<td>Continual Learning</td>
<td><p>YOLOv3</p>
<p>KNN</p></td>
<td>UCSD, Avenue, Shangai Tech</td>
<td><span class="underline">85%</span></td>
</tr>
<tr class="odd">
<td>[26]</td>
<td>Feature Learning</td>
<td>CNN</td>
<td>n/a</td>
<td><span class="underline">n/a</span></td>
</tr>
<tr class="even">
<td>[27]</td>
<td>Transfer Learning</td>
<td>Binary Networks, 3DCNN</td>
<td>citySCENE</td>
<td>94.6%</td>
</tr>
<tr class="odd">
<td>[28]</td>
<td>Residual Learning</td>
<td>CNN, LSTM</td>
<td>UCF,UMN,Avenue</td>
<td><span class="underline">98.3%</span></td>
</tr>
<tr class="even">
<td>[29]</td>
<td>Conditional Generative Adversarial Network</td>
<td><p>R-CNN, SVM,</p>
<p>CGAN</p></td>
<td>Avenue,UCSD Ped1, Ped2, Shangai Tech</td>
<td><span class="underline">91.7%</span></td>
</tr>
<tr class="odd">
<td>[30]</td>
<td>Residual Network</td>
<td>ConvNet</td>
<td>CDNET2014</td>
<td><span class="underline">84.9%</span></td>
</tr>
<tr class="even">
<td>[31]</td>
<td>Auto-encoder</td>
<td>ConvLSTM</td>
<td>Avenue, Surveillance Office, Police</td>
<td><span class="underline">73.6%</span></td>
</tr>
<tr class="odd">
<td>[32]</td>
<td>Sequence Learning</td>
<td>CNN, BD-LSTM</td>
<td>UCF Crime</td>
<td><span class="underline">89.05%</span></td>
</tr>
<tr class="even">
<td>[17]</td>
<td>Ensemble Learning</td>
<td>Fully Connected Networl, Inception V3,</td>
<td>UCF Crime</td>
<td><span class="underline">n/a</span></td>
</tr>
<tr class="odd">
<td>[33]</td>
<td>Sequence Learning</td>
<td>MLP-RNN</td>
<td>n/a</td>
<td><span class="underline">n/a</span></td>
</tr>
<tr class="even">
<td>[34]</td>
<td>Feature Learning</td>
<td>Pre-trained CNNs</td>
<td>UCSD Ped2</td>
<td><span class="underline">88.93%</span></td>
</tr>
<tr class="odd">
<td>[20]</td>
<td>Reinforcement Learning</td>
<td>Deep Q Learning Network (DQN),</td>
<td>UCF Crime</td>
<td><span class="underline">n/a</span></td>
</tr>
<tr class="even">
<td>[8]</td>
<td>Transfer Learning</td>
<td>Pre-trained CNN (VGG16)</td>
<td>UCSD, UMN</td>
<td></td>
</tr>
<tr class="odd">
<td>[35]</td>
<td>Transfer Learning</td>
<td><p>Pre-trained CNN</p>
<p>ResNet</p></td>
<td>CDNET2014</td>
<td><span class="underline">85%</span></td>
</tr>
<tr class="even">
<td>[36]</td>
<td>Auto-Encoder</td>
<td>ConvAE, LSTM AE</td>
<td>n/a</td>
<td><span class="underline">n/a</span></td>
</tr>
<tr class="odd">
<td>[37]</td>
<td>Spatial Temporal Auto-encoder (STAE)</td>
<td>ConvLSTM</td>
<td>n/a</td>
<td><span class="underline">n/a</span></td>
</tr>
<tr class="even">
<td>[10]</td>
<td>Auto-Encoder</td>
<td>ConvLSTM</td>
<td>UCSD Ped1 &amp; Ped2</td>
<td><span class="underline">N/A</span></td>
</tr>
<tr class="odd">
<td>[16]</td>
<td>Auto-Encoder</td>
<td>OF-ConvAE-ConvLSTM</td>
<td>Avenue, UCSD Ped1, Ped2</td>
<td><span class="underline">91.53%</span></td>
</tr>
<tr class="even">
<td>[15]</td>
<td>Predictive Based Model</td>
<td>Pre-trained Convnet (YOLOV3) &amp; Least Square Generative Adversarial Network LS-GAN</td>
<td>CUHK, UCSD Ped2 &amp; Avenue</td>
<td><span class="underline">84.83%</span></td>
</tr>
<tr class="odd">
<td>[38]</td>
<td>Multiple Instance Learning</td>
<td>3D-CNN</td>
<td>UCF Crime</td>
<td><span class="underline">95.4%</span></td>
</tr>
<tr class="even">
<td>[21]</td>
<td>Sequential Learning</td>
<td>CNN</td>
<td>UCSD</td>
<td><span class="underline">N/a</span></td>
</tr>
<tr class="odd">
<td>[39]</td>
<td>Transfer Learning</td>
<td>Pre-trained Convnet (YOLOV3) &amp; GAN</td>
<td>UCSD PED2, CUHK, Shanghai Tech</td>
<td><span class="underline">84.87%</span></td>
</tr>
<tr class="even">
<td>[40]</td>
<td>Auto-encoder Hybrid</td>
<td>Conv-Net, GAN</td>
<td>UCSD Ped2, CHUK Avenue, Subway Entrance, Exit</td>
<td><span class="underline">91%</span></td>
</tr>
</tbody>
</table>

<span class="underline">Results Interpretation</span>

The comparisons were drawn from the publications, the overall performance was calculated from the average of the accuracies presented from different datasets. From the above table, the multiple instance learning models \[27\], ranked as the best followed by other autoencoder solutions. In general, the research trend is moving to unsupervised solutions due to the nature of the surveillance videos. Weakly supervised solutions like multiple instances, transfer and continual learning have shown good performance in very large datasets like UCF Crime.

From the above table, 3DCNN, and ConvLSTM are the most popular underlying machine learning algorithms for building anomaly detection models in surveillance videos. Therefore, 3DCNN and ConvLSTM are the basic building blocks for deep learning video analytics solutions. The most popular learning technique from the study was Autoencoders, taking 30% of the total models evaluated.

## 5.2 Results of the Experiments

Although, there are many solutions to anomaly detection in surveillance videos, only about 10% of the models in the above table have shared their whole project code. This sole reason has limited this study from further investigating all the models.

The following table shows the models that were evaluated empirically,

<table>
<thead>
<tr class="header">
<th></th>
<th>Frame Level AUC comparison</th>
<th></th>
<th></th>
<th></th>
<th></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>No.</td>
<td>Method</td>
<td>Accuracy per Dataset</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td>UCSD Ped1</td>
<td>UCSD Ped2</td>
<td>Avenue</td>
<td>UCF Crime</td>
</tr>
<tr class="odd">
<td>1.</td>
<td>Multiple Instance Learning [6]</td>
<td>n/a</td>
<td>n/a</td>
<td>n/a</td>
<td>75.41%</td>
</tr>
<tr class="even">
<td>2.</td>
<td><p>Conv2DLSTM</p>
<p>Autoencoder [7]</p></td>
<td>86.14%</td>
<td>90.23%</td>
<td>81.23%</td>
<td>78.23%</td>
</tr>
<tr class="odd">
<td>3.</td>
<td>Generative Adversarial Network Future Frame Pred [3]</td>
<td>83.15%</td>
<td>95.31%</td>
<td>84.89%</td>
<td>n/a</td>
</tr>
</tbody>
</table>

The table above shows the comparison of the ROC curve AUC of three models with distinct learning techniques. All the models have shown high accuracy in anomalies prediction. If the first model is compared with the second model, the autoencoder does better in terms of accuracy. The GAN Future Frame prediction model in some instances it outperforms the Spatial-Temporal Autoencoder. Measuring, the best model is a complex function of different factors like time, accuracy, memory resource and computational power required to run the model. Using accuracy alone as a measure is not a holistic measure but, in this case, an average accuracy can be calculated to establish some ranking.

# 6.0 Conclusion

In this study, we analysed deep learning solutions, implemented to solve the anomaly detection in videos. Deep learning models that combine various algorithms and learning techniques were analysed and underlying algorithms and pre-trained models were identified.

The study brought together different solutions and a trend was discovered that the anomaly detection solution is best served by hybrid solutions like transfer learning, continual learning and multiple instance learning especially for very large datasets. Although pure supervised solutions have shown very good results compared to others, a lot of work needs to be done to improve them and also address the online anomaly detection problem, for real-time detections.

# References

|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |

\[1\] R. Yadav and M. Rai, "Advanced Intelligent Video Surveillance System (AIVSS): A Future Aspect," *Research Gate,* 2018. \[2\] M. U. Farooq, N. A. Khan and M. S. Ali, "Unsupervised Video Surveillance for Anomaly Detection of Street Traffic," *(IJACSA) International Journal of Advanced Computer Science and Applications,,* pp. 270-275, 2017. \[3\] K. T. Nguyen, D. T. Dinh, M. N. Do and M. T. Tran, "Anomaly Detection in Traffic Surveillance Videos with GAN-based Future Frame Prediction," *Proceedings of the 2020 International Conference on Multimedia,* pp. 457-463, 2020. \[4\] A. Kushwaha, A. Mishra, K. Kamble, R. Janbhare and A. Pokhare, "Theft Detection using Machine Learning," *IOSR Journal of Engineering (IOSRJEN),* pp. 67-71, 2018. \[5\] K. Wiggers, "AI Guardsman uses computer vision to spot shoplifters," 26 June 2018. \[Online\]. Available: https://venturebeat.com/2018/06/26/ai-guardsman-uses-computer-vision-to-spot-shoplifters/.\[6\] W. Sultani, C. Chen and M. Shah, "Real-World Anomaly Detection in Surveillance Videos," *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),* pp. 6479-6488, 2018. \[7\] Y. S. Chong and Y. H. Tay, "Abnormal Event Detection in Videos Using Spatiotemporal Autoencoder," *arxiv,* vol. 1701, no. 01546v1, 2017. \[8\] S. Bansod and A. Nandedkar, "Transfer learning for video anomaly detection," *Journal of Intelligent & Fuzzy Systems,* vol. 36, no. 3, pp. 1967-1975, 2019. \[9\] K. Doshi and Y. Yilmaz, "Continual Learning for Anomaly Detection in Surveillance Videos," in *2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops*, Seattle, WA, USA, 2020. \[10\] A. Ramchandran and A. K. Sangaiah, "Unsupervised deep learning system for local anomaly event detection in crowded scenes," *Multimedia Tools and Applications,* vol. 79, no. 47/48, p. 35275–35295, 2020. \[11\] A. Sarkar, "Human Activity and Behavior Recognition in Videos. A Brief Review," 2014. \[Online\]. Available: https://www.grin.com/document/276054.\[12\] A. Kushwaha, A. Mishra, K. Kamble and R. Janbhare, "Theft-Detection using Motion Sensing Camera," *International Journal of Innovative Science and Research Technology,* pp. 90-97, 2017. \[13\] M. Sabokrou, M. Fayyaz, M. Fathy, Z. Moayed and R. Klette, "Deep-Anomaly: Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenes," *Computer Vision and Image Understanding,* pp. 1-25, 2018. \[14\] W. Badr, "Auto-Encoder: What Is It? And What Is It Used For? (Part 1)," towards data science, 22 April 2019. \[Online\]. Available: https://towardsdatascience.com/auto-encoder-what-is-it-and-what-is-it-used-for-part-1-3e5c6f017726. \[Accessed 10 May 2021\].\[15\] K. Doshi and Y. Yilmaz, "Online anomaly detection in surveillance videos with asymptotic bound on false alarm rate," *Pattern Recognition,* vol. 114, p. 107865, 2021. \[16\] E. Duman and O. A. Erdem, "Anomaly Detection in Videos Using Optical Flow and Convolutional Autoencoder," *IEEE Access,* vol. 7, pp. 183914 - 183923, 2019. \[17\] Y. Zahid, M. A. Tahir and M. N. Durrani, "Ensemble Learning Using Bagging And Inception-V3 For Anomaly Detection In Surveillance Videos," in *2020 IEEE International Conference on Image Processing (ICIP)*, Abu Dhabi, United Arab Emirates, 2020. \[18\] UCSD, "UCSD Anomaly Detection Dataset," UCSD, 2014. \[Online\]. Available: http://www.svcl.ucsd.edu/projects/anomaly/dataset.html. \[Accessed 10 May 2021\].\[19\] C. Lu, J. Shi and J. Jia, "Avenue Dataset for Abnormal Event Detection," The Chinese Univeristy of Hong Kong, 2013. \[Online\]. Available: http://www.cse.cuhk.edu.hk/leojia/projects/detectabnormal/dataset.html. \[Accessed 10 May 2021\].\[20\] S. Aberkane and M. Elarbi, "Deep Reinforcement Learning for Real-world Anomaly Detection in Surveillance Videos," in *2019 6th International Conference on Image and Signal Processing and their Applications (ISPA)*, Mostaganem, Algeria, 2019. \[21\] A. Khaleghi and M. S. Moin, "Improved anomaly detection in surveillance videos based on a deep learning method," in *2018 8th Conference of AI & Robotics and 10th RoboCup Iranopen International Symposium (IRANOPEN)*, Qazvin, Iran, 2018. \[22\] M. Hasan, J. Choi, J. Neumann, A. K. Roy-Chowdhury and L. S. Davis, "Learning Temporal Regularity in Video Sequences," *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),* pp. 733-742, 2016. \[23\] M. F. M. M. Sabokrou, "Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder," *Electronic Letters,* vol. 52, no. 13, pp. 1122-1124, 2016. \[24\] R. Chalapathy, A. K. Menon and S. Chawla, "Robust, Deep and Inductive Anomaly Detection," *Machine Learning and Knowledge Discovery in Databases,* vol. 10534, 2017. \[25\] H. Wu, J. Shao, X. Xu, F. Shen and H. Shen, "A System for Spatiotemporal Anomaly Localization in Surveillance Videos," *Proceedings of the 25th ACM international conference on Multimedia,* pp. 1225-1226, 2017. \[26\] K. Kavikuil and J. Amudha, "Leveraging Deep Learning for Anomaly Detection in Video Surveillance," *First International Conference on Artificial Intelligence and Cognitive Computing,* vol. 815, no. I, pp. 239-247, 2018. \[27\] K. Liu, M. Zhu, H. Fu, H. Ma and T.-S. Chua, "Enhancing Anomaly Detection in Surveillance Videos with Transfer Learning from Action Recognition," *Proceedings of the 28th ACM International Conference on Multimedia,* pp. 4664-4668, 2020. \[28\] W. Ullah, A. Ullah, T. Hussain, Z. A. Khan and S. W. Baik, "An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance Videos," *AI-Enabled Advanced Sensing for Human Action and Activity Recognition,* vol. 21, 2021. \[29\] T.-H. Vu, J. Boonaert, S. Ambellouis and A. Taleb-Ahmed, "Multi-Channel Generative Framework and Supervised Learning for Anomaly Detection in Surveillance Videos," *Human Activity Recognition Based on Image Sensors and Deep Learning,* vol. 21, no. 9, p. 3179, 2021. \[30\] L. P. Cinelli, L. A. Thomaz, A. F. d. Silva, E. A. B. d. Silva and S. L. Netto, "Foreground Segmentation for Anomaly Detection in Surveillance Videos Using Deep Residual Networks," *XXXV SIMPOSIO BRASILEIRO DE TELECOMUNICAC¸ ´ OES E PROCESSAMENTO DE SINAIS,* pp. 3-6, 2017. \[31\] S. Bhakat and G. Ramakrishnan, "Anomaly Detection in Surveillance Videos," *Proceedings of the ACM India Joint International Conference on Data Science and Management of Data,* p. 252–255, 2019. \[32\] W. Ullah, A. Ullah, I. U. Haq, K. Muhammad, M. Sajjad and S. W. Baik, "CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks," *Multimedia Tools and Applications ,* p. 16979–16995, 2021. \[33\] M.Murugesan and S.Thilagamani, "Efficient anomaly detection in surveillance videos based on multi layer perception recurrent neural network," in *Microprocessors and Microsystems*, 2020. \[34\] T. S. Nazare, R. F. de Mello and M. A. Ponti, "Are pre-trained CNNs good feature extractors for anomaly detection in surveillance videos?," *eprint arXiv,* no. 1811.08495v1, 2018. \[35\] L. P. Cinelli, "ANOMALY DETECTION IN SURVEILLANCE VIDEOS USING DEEP RESIDUAL NETWORKS," Universidade Federal do Rio de Janeiro, Rio de Janeiro, 2017.\[36\] V. A. Karishma Pawar, "Application of Deep Learning for Crowd Anomaly Detection from Surveillance Videos," in *2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence)*, Noida, India, 2021. \[37\] Y. Zhao, B. Deng, C. Shen, Y. Liu, H. Lu and X.-S. Hua, "Spatio-Temporal AutoEncoder for Video Anomaly Detection," *Proceedings of the 25th ACM international conference on Multimedia,* pp. 1933-1941, 2021. \[38\] N. Nasaruddin, K. Muchtar, A. Afdhal and A. P. J. Dwiyantoro, "Deep anomaly detection through visual attention in surveillance videos," *Journal of Big Data,* vol. 7, no. 87, 2020. \[39\] K. Doshi and Y. Yilmaz, "Any-Shot Sequential Anomaly Detection in Surveillance Videos," *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops,* pp. 934-935, 2020. \[40\] T.-N. Nguyen and J. Meunier, "Anomaly Detection in Video Sequence with Appearance-Motion Correspondence," in *Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)*, 2019. \[41\] H. M. Kun Liu, "Exploring Background-bias for Anomaly Detection in Surveillance Videos," *Proceedings of the 27th ACM International Conference on Multimedia,* pp. 1490-1499, 2019. \[42\] R. V. H. M. Colque, C. Caetano and M. T. L. d. Andrade, "Histograms of Optical Flow Orientation and Magnitude and Entropy to Detect Anomalous Events in Videos," *IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,* vol. 27, no. 3, pp. 673-682, 2017. \[43\] M. Sabokrou, M. Fayyaz, M. Klette and R. Fathy, "Deep-Cascade: Cascading 3D Deep Neural Networks for Fast Anomaly Detection and Localization in Crowded Scenes," *IEEE Transactions on Image Processing,* pp. 1992-2004, 2017.