Jarvis-X / balloonDetectionAutoCalibrate

balloonDetectionAutoCalibrate

A robotic blimp project submodule that allows users to calibrate for customized color. Find # TODO flags for pending tasks.

How-To

Ras-Pi

Calibrate the color of ballons

Type in your terminal: python main.py picam 1

Calibrate the color of goals

Type in your terminal: python main.py picam 4

Run program to get info from both ballons and goals

Type in your terminal: python main.py picam 0

Laptop

Calibrate the color of ballons

Type in your terminal: python main.py cvcam 1

Calibrate the color of goals

Type in your terminal: python main.py cvcam 4

Run program to get info from both ballons and goals

Type in your terminal: python main.py cvcam 0

Components

Ballon Detection

Calibrate:

Type in your terminal: python main.py cvcam 1

Run the program while the target balloon is present in front of the cameras. A very small window will show up. Make sure the main color of the balloon is captured in that small window and wait for ~5 seconds till the window closes. A .dat file should be generated by the program in the same directory of the main.py file.

• We should experiment whether there is an advantage of using "HLS" or "HSV" color spaces. If not, we should stick to opencv's original BGR colorspace to cut down the computation overhead.

Color-based detection:

Color filter, then contour detection -> output bounding boxes of the contours.

Type in your terminal: python main.py cvcam 2 and let the camera see the object. Currently, this method can detect large color clusters based on the calibrated .dat file and bound them with rectangles. Distance or direction is not provided.

Blob detection:

Color filter, then blob detection -> output keypoints of the blobs and estimated distance.

Type in your terminal: python main.py cvcam 3 and let the camera see the object. Currently, this method can detect large color blobs based on the calibrated .dat file and mark them with keypoints. The direction is not provided. And the distance depends on the manually provided camera focal length.

Target detection

Threshold detection:

Blur, Gray, then contour detection -> output bounding boxes of the contours.

Type in your terminal: python main.py cvcam 4 and let the camera see the object.

Canny detection (*Prefered)

Blur, Gray, Threshold, Canny, then contour detection -> output bounding boxes of the contours.

Type in your terminal: python main.py cvcam 5 and let the camera see the object.

TODO

• pi camera integration
  • if possible, offload the computation to pixy: this one is hard to start with. Leave it Jiawei.
• blob detection Done
  • Profiling on a RPaspberry Pi 4B and compare the performance against the color detection method
• Distance and Direction extraction
  • Make these functionalities modular and compatible with both the color-based and blob detection
• better filter for post-detection and in-detection
  • a new class TrackingDetection is created as a placeholder for this functionality. Play with it!