owaiskaifi/Train-Your-Own-Custom-Object-Detection-Model-for-Flutter-with-Tiny-Yolov2

https://github.com/IfProgrammingIsMagicImaWizard/Train-Your-Own-Custom-Object-Detection-Model-for-Flutter-with-Tiny-Yolov2

This fork is made to train and create a TFLite of your Custom Object Detection Model compatible with Flutter and TFLite 1.1.2 library.

REQUIREMENTS

Windows 7 or 10 (64bits)
Nvidia GPU (GTX 650 or newer) for CUDA Acceleration

REMOVE FROM PATH ANY OTHER STANDALONE PYTHON YOU MAY HAVE

At "Environment Variables" > "System" > "Path" > "Edit" Remove: (e.g) H:\ProgramData\Python\Python310\Scripts
H:\ProgramData\Python\Python310\

You can add it back when finished.

INSTALLATION

Note: Everything is very version sensitive, so use the exactly version mentioned above.

0- Install Visual Studio 2017 Library

https://visualstudio.microsoft.com/vs/older-downloads/

I´m not sure if works with newer versions.

1- Install Anaconda 3

https://www.anaconda.com/products/individual

The one I´m using is "Anaconda3-2021.11-Windows-x86_64" but newer releases should be compatible as long is version 3.

2- Add to "Environment Variables" > "System" > "Path" > "New"

[YOUR DIRECTORY]\Anaconda3\Scripts\
[YOUR DIRECTORY]\Anaconda3\Library\
[YOUR DIRECTORY]\Anaconda3\Library\bin\
[YOUR DIRECTORY]\Anaconda3\Library\mingw-w64\bin\

3- Uninstall any CUDA you might have

The NVIDIA softwares you ALLOW to have:
-NVIDIA Control Panel
-NVIDIA Driver
-NVIDIA Geforce Experience
-NVIDIA RTX
-NVIDIA RTX Voice
-NVIDIA PhysX
-NVIDIA USB C
-NVIDIA FrameView SDK

Uninstall the following:
-NVIDIA CUDA
-NVIDIA Nsight
-NVIDIA Tools

Reboot the computer

4- Download CUDA CUDA 10.0

https://developer.nvidia.com/cuda-10.0-download-archive

5- Install CUDA 10.0

5.1- Choose "Custom Installation"
5.2- Create a shorter path (e.g. C:\CUDA) because the default path will be too long and we will not be able to add it to "Environment Variables Path"
5.3- It MUST be in the same directory of Windows (e.g. C:)

6- Add to "Environment Variables" > "System" > "Path" > "New"

Usually added automatically:

[YOUR DIRECTORY]\CUDA\bin (e.g. C:\CUDA\bin )
[YOUR DIRECTORY]\CUDA\libnvvp (e.g. C:\CUDA\libnvvp )

You always need to add it:

[YOUR DIRECTORY]\CUDA\extras\CUPTI\libx64 (e.g. C:\CUDA\extras\CUPTI\libx64 )

7- Download cuDNN v7.6.5 (November 5th, 2019), for CUDA 10.0

https://developer.nvidia.com/rdp/cudnn-archive

8- Install cuDNN

8.1- Under the .zip file, move "cuda > bin > *.dll" to your "CUDA > bin" folder
8.2- Create a copy and rename "cudnn64_7.dll" to "cudnn64_100.dll"

8.3- Under the .zip file, move "cuda > include > *.h" to your "CUDA > include" folder

8.4- Under the .zip file, move "cuda > lib > x64 > *.lib" to your "CUDA > lib > x64" folder

8.5- REBOOT the computer

ENVIRONMENT SETUP

1- Initiate Anaconda

1.1- Open "cmd" type: conda init
1.2- Close and reopen "cmd"

2- Setup Environment

2.1- Open "cmd" on ADMIN MODE type: conda deactivate
2.2- On "cmd" type: conda create \--name tensor_yolo_1.15 python=3.7.0
2.3- On "cmd" type: conda activate tensor_yolo_1.15
2.6- On "cmd" type: 
	 pip install tensorflow-gpu==1.15.0
	 pip install Cython==0.29.26
	 pip install opencv-python==4.5.4.60
	 pip3 install \--extra-index-url https://google-coral.github.io/py-repo/ tflite==2.4.0
	 conda install pyqt=5
	 conda install -c anaconda lxml=4.6.1
	 pip install in_place==0.5.0

3- Setup the Project

3.1- Download and extract https://github.com/IfProgrammingIsMagicImaWizard/darkflow

 With conda activate tensor_yolo_1.15:
	On "cmd" type: cd /d "PATH TO THE PROJECT FOLDER" 	    
			(e.g     cd /d "D:\Projetos\Flutter\darkflow" )
	On "cmd" type: python setup.py build_ext \--inplace

3.2- Download YOLO V2 Weights https://pjreddie.com/darknet/yolo/

3.3- Put into darkflow/bin

4- Setup the labelImg

4.1- Download labelImg and extract into darkflow/labelImg: https://github.com/tzutalin/labelImg

4.2- With conda activate tensor_yolo_1.15:
	On "cmd" type: cd /d "PATH TO labelImg" 
			(e.g     cd /d "D:\Projetos\Flutter\darkflow\labelImg" )
	On "cmd" type: pyrcc5 -o resources.py resources.qrc

4.3- Move "resources.py" and "resources.qrc" to ´libs´ folder 		(e.g darkflow\labelImg\libs )

IMAGE LABELLING

1- Data Gathering

Put all your images into darkflow\train\Images folder.

Make all images square, .jpg, up to 416x416 pixels with Photoshop and iloveimg.com.

Run labelImg.py on darkflow\labelImg folder.

2- Image labelling

With labelImg Open:

Go to view > Enable Auto Save Mode
Set ´Save Dir´ to darkflow\train\Images folder
Open Dir (Same Path)
Start Labeling
Short Cuts:
W - Draw Label
A - Next Image
D - Previous Image

3- Move .xml files

Move the .xlm files from darkflow\train\Images to darkflow\train\annotation

4- (Optional) Update Annotation´s path to the Image file

If you are using images from a database (like RoboFlow or Kaggle) you will notice that, opening the Annotation with text editor like Notepad, the Annotation´s path to the Image file is not correct.

Example:

It is this: \0_ELAADEN_CACHE_FLOW.jpg</path>
When it should be this: D:\Flutter\YOLO_V2\darkflow\train\Images\0_ELAADEN_CACHE_FLOW.jpg</path>
To update the Annotation to your liking edit the darkflow\update_annotation.py script and run on tensor_yolo_1.15 environment:

conda activate tensor_yolo_1.15 and then python PATH_TO_THE_SCRIPT

TRAIN YOUR MODEL

1- Edit `labels.txt`

Edit darkflow/labels.txt to match with your classes.

2- Edit cfg file

On darkflow\cfg create a copy (bacause we will need the original too)
of the yolov2-tiny-voc.cfg and rename it

(e.g yolov2-tiny-voc-c16.cfg because I have 16 classes )

Open with a text editor like Atom or Notepad++.
Go to the bootom and edit the line 124 and put the amount of classes you have (e.g classes = 20 )
Now we need to alter the last layer´s filter to match our amount of classes
Go to line 118 the amount will be the result of this formula:
(num classes + 5) * 5

Examples:
1 class -> (1 + 5) * 5 = 30 -> filters=30
20 classes -> (20 + 5) * 5 = 125 -> filters=125

3- (Optional) Choosing another model

In this turorial we are using Tiny YoloV2 Weights and Cfg.
If you want to use other model download the Weights and Cfg from https://pjreddie.com/darknet/yolo/
Put the Weight in the darkflow/bin folder and Cfg in the darkflow/cfg folder.

4- Train our model

With conda activate tensor_yolo_1.15:
	On "cmd" type: cd /d "PATH TO THIS PROJECT" 	     (e.g     cd /d "D:\Projetos\Flutter\darkflow\" )
	On "cmd" type:

python flow --model cfg/yolov2-tiny-voc-16.cfg --load bin/yolov2-tiny-voc.weights --train --annotation train/annotation --dataset train/Images --gpu 0.8 --epoch 1000
Explanation:

--model Path to your modifed cfg (it will look for the original too)
--load Path to the pre trained wights
-- train Command to train
--annotation Path to the annotation´s folder
--dataset Path to the images folder
--gpu Command to use the GPU, 0.8 means 80%, recommended to be safe since Windows use a bit of the GPU to function
--epoch Amount of training, default is 5000, but start with 1000 to see if everthing is working

On a Ryzen 3700x and 1660Ti training 1000 epochs took 30 min using GPU or 60 hours using CPU.

5- Test our model

With the training finished we should have darkflow\ckpt folder.

5.1 Testing CKPT on Images

Prepare the darkflow\sample_img folder with some images.

With conda activate tensor_yolo_1.15:
	On "cmd" type: cd /d "PATH TO THIS PROJECT" 	     (e.g     cd /d "D:\Projetos\Flutter\darkflow\" )
	On "cmd" type:

python flow --imgdir sample_img/ --model cfg/yolov2-tiny-voc-16.cfg --load 10750 --gpu 0.8 --threshold 0.0

Explanation:

--model Path to your modifed cfg
--load Path to your trained wights
--imgdir Path to the sample images folder
--gpu Command to use the GPU, 0.8 means 80%, recommended to be safe since Windows use a bit of the GPU to function
threshold 0.0 Set so just confidence above this number will be return, 0.0 return all, up to 1.0 (100%)

You can also get the resuts in Json format by adding --json

In the example above we load our ckpt on checkpoint 10750 that is in our ckpt folder:

If everthing works right your input image should be transform into an output like this:

5.2 Testing CKPT on Video

You can also test using a video, but in my case it didn´t work, it don´t draw the boxes But the command is the above:

python flow --model cfg\yolov2-tiny-voc-16.cfg --load 10750 --demo G:\Download\video.avi --gpu 0.8 --saveVideo --threshold 0.0

If you out of VRAM use CPU:

python flow --model cfg\yolov2-tiny-voc-16.cfg --load 10750 --demo G:\Download\video.avi --saveVideo --threshold 0.0

6- Convert to .pb

Also known as generate frozen graph or genarate saved model.

With conda activate tensor_yolo_1.15:
	On "cmd" type: cd /d "PATH TO THIS PROJECT" 	     (e.g     cd /d "D:\Projetos\Flutter\darkflow\" )
	On "cmd" type:

python flow --model cfg/yolov2-tiny-voc-16.cfg --load 10750 --savepb

Explanation:

--model Path to your modifed cfg
--load Path to your trained wights
--savepb Command to save pb file

6.1 Testing .pb on Images

Similar to testing our CKPT model we can test our .pb:

python flow --pbLoad built_graph/yolov2-tiny-voc-16.pb --metaLoad built_graph/yolov2-tiny-voc-16.meta --imgdir sample_img/

7- Convert to TFLite

With conda activate tensor_yolo_1.15:
	On "cmd" type: cd /d "PATH TO THIS PROJECT" 	     (e.g     cd /d "D:\Projetos\Flutter\darkflow\" )
	On "cmd" type:

tflite_convert --graph_def_file="built_graph\yolov2-tiny-voc-16.pb" --output_file="G:\Download\yolov2_28_12_2021_22_40.tflite" --input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE --input_shape=1,416,416,3 --input_array=input --output_array=output --inference_type=FLOAT --input_data_type=FLOAT

Change just --graph_def_file and --output_file

This will generate a model compatible with Flutter and tflite 1.1.2

Note: There is a significant loss in accuracy when converting to TFlite (In my case was about 13%~15%) but that is the price for performance.

8- Using on Flutter

Download sample project: https://github.com/IfProgrammingIsMagicImaWizard/sample_object_detection

The result is something like:

[{"label": "Cat", 
"confidence": 0.83, 
"topleft": {"x": 11, "y": 90}, 
"bottomright": {"x": 66, "y": 179}}, 

{"label": "Dog", 
"confidence": 0.93, 
"topleft": {"x": 191, "y": 274}, 
"bottomright": {"x": 246, "y": 351}}, 

{"label": "Bird", 
"confidence": 0.88, 
"topleft": {"x": 109, "y": 8}, 
"bottomright": {"x": 154, "y": 79}}]

You can use the box information do draw on a Image, Video or Live Feed.

You can know more at: https://pub.dev/packages/tflite/example

ACKNOWLEDGMENT

Thanks to Mark Jay of which most of the tutorial was based on.

owaiskaifi / Train-Your-Own-Custom-Object-Detection-Model-for-Flutter-with-Tiny-Yolov2

REQUIREMENTS

INSTALLATION

0- Install Visual Studio 2017 Library

1- Install Anaconda 3

2- Add to "Environment Variables" > "System" > "Path" > "New"

3- Uninstall any CUDA you might have

4- Download CUDA CUDA 10.0

5- Install CUDA 10.0

6- Add to "Environment Variables" > "System" > "Path" > "New"

7- Download cuDNN v7.6.5 (November 5th, 2019), for CUDA 10.0

8- Install cuDNN

ENVIRONMENT SETUP

1- Initiate Anaconda

2- Setup Environment

3- Setup the Project

4- Setup the labelImg

IMAGE LABELLING

1- Data Gathering

2- Image labelling

3- Move .xml files

4- (Optional) Update Annotation´s path to the Image file

TRAIN YOUR MODEL

1- Edit `labels.txt`

2- Edit cfg file

3- (Optional) Choosing another model

4- Train our model

5- Test our model

5.1 Testing CKPT on Images

5.2 Testing CKPT on Video

6- Convert to .pb

6.1 Testing .pb on Images

7- Convert to TFLite

8- Using on Flutter

ACKNOWLEDGMENT

About

Languages

REQUIREMENTS

INSTALLATION

0- Install Visual Studio 2017 Library

1- Install Anaconda 3

2- Add to "Environment Variables" > "System" > "Path" > "New"

3- Uninstall any CUDA you might have

4- Download CUDA CUDA 10.0

5- Install CUDA 10.0

6- Add to "Environment Variables" > "System" > "Path" > "New"

7- Download cuDNN v7.6.5 (November 5th, 2019), for CUDA 10.0

8- Install cuDNN

ENVIRONMENT SETUP

1- Initiate Anaconda

2- Setup Environment

3- Setup the Project

4- Setup the labelImg

IMAGE LABELLING

1- Data Gathering

2- Image labelling

3- Move .xml files

4- (Optional) Update Annotation´s path to the Image file

TRAIN YOUR MODEL

1- Edit labels.txt

2- Edit cfg file

3- (Optional) Choosing another model

4- Train our model

5- Test our model

5.1 Testing CKPT on Images

5.2 Testing CKPT on Video

6- Convert to .pb

6.1 Testing .pb on Images

7- Convert to TFLite

8- Using on Flutter

ACKNOWLEDGMENT

About

Languages

1- Edit `labels.txt`