On this repository, I want to share my work; detection of Chionoecetes Tanneri Crabs and estimation of their lengths for the 2021 Machine Learning MATE ROV Microsoft Azure Computer Coding Challenge Task 1.
- The Dataset is created by using Roboflow.
- YOLOv5l is used for detection on the Custom Dataset.
- Crabs are detected on the video stream.
- Lasers are masked and the distance is calculated instantaneously.
- Crab lengths are calculated with respect to boundary boxes of detected crabs and laser distances.
- About the Competition
- About the Model and the Data
- How to Detect Yourself?
- My Observations About the Task
- Discussion
Me and my friends, as DEU ROV TEAM, attend to the 2021 Machine Learning MATE ROV Microsoft Azure Computer Coding Challenge!
The Mission: Harness the power of cloud computing to identify and quantify organisms in benthic video transects. Start with pre-built Artificial Intelligence models like Microsoft Computer Vision and Video Indexer OR take it to the next level by building your own machine learning (aka coding) models.
Task 1: Create a program that annotates a video transect for one species – in this case, the grooved tanner crab, Chionoecetes tanneri.
Video transect: Computer Coding Challenge Round #1 video from the Monterey Bay National Marine Sanctuary. Video courtesy of MBARI and the ROV Ventana.
To be able to detect Crabs, the data is collected from the Video frame by frame but only 0th, 10th and 20th frames are extracted for each second to create a dataset with about 500 images.
The dataset is labeled and splitted as Train, Validate and Test sets as 80%(467 images), 13%(77 images) and 7%(38 images) respectively.
Then, another dataset version is also created and this time the rates were as 92%(924 images), 7%(66 images) and 1%(10 images) respectively which was performed much better.
Then, YOLOv5 tutorial of Ultralytics and Custom Dataset application tutorial of RoboFlow are followed to use YOLOv5l model for the detection of the crabs. Also you can check the research paper of You Only Look Once (YOLO).
Additionally, it was necessary to detect lasers on the video to calculate the length of crabs. Therefore, each frame is converted to hsv and then masked to detect laser dots and the distance is calculated for each frame as pixels. After that, the calculation is performed by using boundary boxes of the rectangles of detected crabs.
Original detect.py and utils.plots.py files are modified as detect_crab.py and utils.plots_crab.py to perform these computations.
You can simply follow my colab tutorial which was explained in detail.
- Download original YOLOv5 repository and extract the files - Install Dependencies
- Download the custom dataset
- Define Model Configuration and Architecture
Important! You need to replace and use my detect.py file and plots.py(in the utils folder) file to detect the crabs by yourself!
Besides, I have preferred to use large one instead of s, m or x. You can use any of these as you wish. TIP
- Train Custom Yolov5 Detector
- Evaluate Custom Yolov5 Detector Performance
- Run Inference With Trained Weights
- Export Trained Weights for Future Inference
- Instead of Test Set (Use Individual Images or Videos)
It was quite fun and I have learned a lot by this project. This was my first time to use YOLO in a project like this one. I was not able to extract each time and create a time table as a result and could not be able to attend the competition on time but I wanted to share this work anyway since I really enjoyed! I am still learning how to use Microsoft Azure by the way, I hope to learn to use in a short time. Additionally, i am planning to improve myself on Computer Vision and complete more projects soon.
If you have any question and suggestion, please do not hesitate to contact me.
Onurcan Köken 15th of May, 2021