Marconi-Lab / dsa_2024

Computer Vision resources for DSA 2024 in Nyeri

Intro to Computer Vision ML using YOLOv8 @DSA2024

© DSA2024. Apache License 2.0.

Contributors : Yuri Njathi, Victor Ruto, Lorna Mugambi, Dr. Andrew Katumba

Reviewers : Sam Mbatia

Introduction

As a branch of artificial intelligence, computer vision utilizes machine learning and neural networks to guide computers in extracting valuable insights from digital images, videos, and visual data. This capability allows them to offer recommendations or trigger actions when identifying flaws or issues [1].

Computer Vision is a broad concept. It covers more than 15 different applications [2]. Here we will look at 3 variations of computer vision, these are : Image Classification,Object Detection and Instance Segmentation. The variations will be pretrained and fine-tuned prediction. Zero-Shot classification is use of pretrained models to obtain predictions without training.

Here we will use DSAIL-Porini and a bone xray dataset to go through a typical end-to-end machine learning workflow for classification, detection and segementation.

Topics:

Content: Computer Vision, YOLOv8

Level: Beginner

Learning Objectives:

• Introduce you to end-to-end machine learning.

Prerequisites:

• Basic knowledge of Python Programming
• A Google Colab account
• A roboflow account
• A Hugging Face account

A. Image Classification with YOLOv8

Here we shall utilize DSAIL-Porini images to obtain classification predictions and evaluate the model's performance. Luckily, the dataset authors have provided the images and classifications. I'll take you through how to annotate using roboflow.

Here we'll use YOLOv8-cls to obtain predictions for Zebra, Impala and Other classes. We're only going to use images that have a single species of animal and the animals are detectable by YOLOv8-det.

Steps

1. We'll first practice with Image Classification prediction using pretrained yolov8-cls model

From the test image you can see that the pretrained model has a low confidence score, we'll rectify this by finetuning using annotated data.

2. Download the zipped dataset and unzip

3. Login to roboflow and create a single-label classification project

4. Upload the sample images from the zipped file above originating from DSAIL-Porini on roboflow, then save and continue.

5. Click annotate then annotate images for FineTuning using roboflow ... annotate with the classes you see fit.

6. Get raw url to annotated roboflow dataset.

7. Train YOLOv8-cls model for Image Classification and you can export model to Huggingface (optional).

8. ReCAP: above we have finetuned yolov8-cls for classifying between Zebra, Impala and Other classes.

9. Practice Work

• Try this process on your annotated dataset.
• Differentiate zero-shot, one-shot and few-shot training.

B. Image Object Detection with YOLOv8

In the object detection task, we will finetune a YOLOv8 detection model on a subset of the DSAIL-Porini camera trap dataset and use the model to obtain predictions. We will then characterise our model to see its failure modes and see how to potentially correct them.

Open and run the object detection notebook and follow the steps as outlined in the notebook.

Here is a summary of what we do in the notebook:

1. Download a pre-annotated subset of the DSAIL-Porini dataset available on Roboflow.
2. Perform an exploratory data analysis of the dataset using Fiftyone and Data-Gradients
3. Try out the YOLO-World Zero-shot open vocabulary detection model on the dataset.
4. Fine tune an Ultralytics YOLOv8 model on the dataset

After completing the steps in the walkthrough notebook, open and run the object detection exercise notebook .

Here is a summary of the exercises you will do in the notebook:

1. Try out the Grounding Dino zero-shot model
2. Annotate a few other images and add it to the already annotated dataset and retrain the YOLOv8 model with new dataset variants.

C. Image Instance Segementation with YOLOv8

For the segmentation task, we will use an open source xray dataset from roboflow.

Steps

1. Download the dataset as a zipped file in YOLOv8 format
2. Unzip the file
3. Log in to Roboflow and create a project
4. Upload your unzipped file
5. Save you dataset but apply pre-processing and augmentations of your choice from the list provided.
6. You can export your dataset again or us the raw url of the dataset.
7. Train YOLOv8-seg model for Segmentation
8. In the notebook shared above, we will have finetuned 'yolov8-seg' to carry out instance segmentation for normal vs fractured bones.
9. In the notebook, there are some exercises for you to carry out so as to familiarise yourself more with YOLOv8.

References

1. IBM Computer Vision
2. HuggingFace CV examples
3. Roboflow "What is Zero-Shot Classification"
4. How to label data for YOLOv5 Instance Segmentation training

Inspiration

1. Deep Learning Indaba intro to JAX

Name	GitHub	Affiliation
Yuri Njathi	@Yuri-Njathi	DeKUT-DSAIL
Lorna Mugambi	@lornamugambi	DeKUT-DSAIL
Victor Ruto	@vickruto	DeKUT-DSAIL
Samuel Mbatia	@mbatiasonic	DeKUT-DSAIL
Andrew Katumba	@andrewkatumba	Marconi Lab

Questions

Please ask any questions through this form