Efficient and Compact Convolutional Neural Network Architectures for Non-temporal Real-time Fire Detection

Tested using Python >= 3.6.x, PyTorch >= 1.5, and OpenCV 3.x / 4.x (requires opencv extra modules - ximgproc module for superpixel segmentation).

Architectures:

Abstract:

" Automatic visual fire detection is used to complement traditional fire detection sensor systems (smoke/heat). In this work, we investigate different Convolutional Neural Network (CNN) architectures and their variants for the nontemporal real-time bounds detection of fire pixel regions in video (or still) imagery. Two reduced complexity compact CNN architectures (NasNet-A-OnFire and ShuffleNetV2-OnFire) are proposed through experimental analysis to optimise the computational efficiency for this task. The results improve upon the current state-of-the-art solution for fire detection, achieving an accuracy of 95% for full-frame binary classification and 97% for superpixel localisation. We notably achieve a classification speed up by a factor of 2.3× for binary classification and 1.3× for superpixel localisation, with runtime of 40 fps and 18 fps respectively, outperforming prior work in the field presenting an efficient, robust and real-time solution for fire region detection. Subsequent implementation on low-powered devices (Nvidia Xavier-NX, achieving 49 fps for full-frame classification via ShuffleNetV2-OnFire) demonstrates our architectures are suitable for various real-world deployment applications."

[Thomson, Bhowmik, Breckon, In Proc. International Conference on Machine Learning Applications, IEEE, 2020]

[Talk] [Example]

Our previous works on fire detection using Alexnet and InceptionVx can be found here.

Installation

The code is tested on Ubuntu 18.04, and Nvidia Jetson Xavier NX using CPU/GPU/TensorRT.

Requirements for Deskop/Laptop

Linux (Ubuntu >= 18.04 distribution)
CUDA >= 10.2, cuDNN >= 7.6.0
Python ≥ 3.6
[Optional] TensorRT

Requirements for Nvidia Jetson Xavier NX

Linux (Ubuntu 18.04 distribution for Xavier NX)
JetPack 4.4 (CUDA 10.2, cuDNN 8.0)
Python 3.6
[Optional] TensorRT - installs with JetPack

Steps

[Optional] create a new virtual environment.

sudo apt update
sudo apt install python3-dev python3-pip
sudo pip3 install -U virtualenv
virtualenv --system-site-packages -p python3 ./venv

And activate the environment.

source ./venv/bin/activate # sh, bash, ksh, or zsh

First clone the repository:

git clone https://github.com/NeelBhowmik/efficient-compact-fire-detection-cnn.git

Install pytorch >= 1.5.0 with torchvision (that matches the PyTorch installation - link). For pytorch installation on Xavier NX, please follow the steps from offical link.
Install the requirements
```
pip3 install -r requirements.txt
```
[Optional] Install torch2trt to enable TensorRT mode.

Instructions to run inference using pre-trained models:

We support inference for image/image directory, video/video directory, and webcam.

Download pre-trained models (nasnetonfire/shufflenetonfire) in ./weights directory and test video in ./demo directory as follows:

sh ./download-models.sh

This download script (download-models.sh) will create an additional weights directory containing the pre-trained models and demo directory containing a test video file.

To run {fire, no-fire} classification on full-frame:

inference_ff.py [-h]  [--image IMAGE] [--video VIDEO]
                      [--webcam] [--camera_to_use CAMERA_TO_USE]
                      [--trt] [--model MODEL]
                      [--weight WEIGHT] [--cpu] [--output OUTPUT] [-fs]

optional arguments:
  -h, --help            show this help message and exit
  --image IMAGE         Path to image file or image directory
  --video VIDEO         Path to video file or video directory
  --webcam              Take inputs from webcam
  --camera_to_use CAMERA_TO_USE
                        Specify camera to use for webcam option
  --trt                 Model run on TensorRT
  --model MODEL         Select the model {shufflenetonfire, nasnetonfire}
  --weight WEIGHT       Model weight file path
  --cpu                 If selected will run on CPU
  --output OUTPUT       A directory path to save output visualisations.If not
                        given , will show output in an OpenCV window.
  -fs, --fullscreen     run in full screen mode

e.g. as follows ....

python3 inference_ff.py \
  --video demo/test.mp4 \
  --model shufflenetonfire \
  --weight weights/shufflenet_ff.pt

To run {fire, no-fire} superpixel localisation:

python3 inference_superpixel.py [-h] [--image IMAGE] [--video VIDEO]                               
                                     [--webcam] [--camera_to_use CAMERA_TO_USE]
                                     [--trt] [--model MODEL]
                                     [--weight WEIGHT] [--cpu]
                                     [--output OUTPUT] [-fs]

optional arguments:
  -h, --help            show this help message and exit
  --image IMAGE         Path to image file or image directory
  --video VIDEO         Path to video file or video directory
  --webcam              Take inputs from webcam
  --camera_to_use CAMERA_TO_USE
                        Specify camera to use for webcam option
  --trt                 Model run on TensorRT
  --model MODEL         Select the model {shufflenetonfire, nasnetonfire}
  --weight WEIGHT       Model weight file path
  --cpu                 If selected will run on CPU
  --output OUTPUT       A directory to save output visualizations.If not given
                        , will show output in an OpenCV window.
  -fs, --fullscreen     run in full screen mode

e.g. as follows ....

python3 inference_superpixel.py \
  --video demo/test.mp4 \
  --model shufflenetonfire \
  --weight weights/shufflenet_sp.pt

Fire Detection Datasets:

The custom dataset used for training and evaluation can be found on Durham Collections - Thomson/Bhowmik/Breckon, 2020. A direct download link for the dataset is here.

DOI for datsets - http://doi.org/10.15128/r2d217qp536.

A download script download-dataset.sh is also provided which will create an additional dataset directory containing the training dataset (10.5Gb in size, works on Linux/MacOS).

References:

If you are making use of this work in any way (including our pre-trained models or datasets), you must please reference the following articles in any report, publication, presentation, software release or any other associated materials:

Efficient and Compact Convolutional Neural Network Architectures for Non-temporal Real-time Fire Detection (Thomson, Bhowmik, Breckon), In Proc. International Conference on Machine Learning Applications, IEEE, 2020.

@InProceedings{thompson20fire,
  author = {Thompson, W. and Bhowmik, N. and Breckon, T.P.},
  title = {Efficient and Compact Convolutional Neural Network Architectures for Non-temporal Real-time Fire Detection},
  booktitle = {Proc. Int. Conf. Machine Learning Applications},
  pages = {136-141},
  year = {2020},
  month = {December},
  publisher = {IEEE},
  keywords = {fire detection, CNN, deep-learning real-time, neural architecture search, nas, automl, non-temporal},
  url = {http://breckon.org/toby/publications/papers/thompson20fire.pdf},
  doi = {10.1109/ICMLA51294.2020.00030},
  arxiv = {http://arxiv.org/abs/2010.08833},
}

In addition the (very permissive) terms of the LICENSE must be adhered to.

NeelBhowmik / efficient-compact-fire-detection-cnn