OS: MacOS
python3.9+
brew install ffmpeg imagemagick libusb
pip3 install --upgrade pip
pip3 install --use-deprecated=legacy-resolver
pip3 install backgroundremover
pip3 install tflite-model-maker
pip3 install pycocotools
python3 make_image.py
Then follow the tips:
If you want to prepare the images of object bottle, take pictures of bottles with your android device (You should always take photo with landscape mode, do not use potrait mode, otherwise the extracting object work will not work as you expected). When you captured enough bottle images, connect the android device to your computer, make sure adb works.
Make a sub dir bottle under the path generated/object before you run python3 make_image.py. Then type 1 to select 'Retrieve images from android mobile phone'.
Then select the number before bottle.
When you see 'How many images to retrieve (lates)?', input the numbers of pictures you took for the bottles. Type enter then the pictures would be downloaded to the generated/object/bottle directory.
If you want to prepare the background images, you can capture some mp4 videos with your android device. Then connect the android device to your computer, make sure adb works.
Select 2 'Retrieve videos from android mobile phone' after you run python3 make_image.py. Then type the video numbers you captured. Type enter then the videos would be downloaded to the generated/background/video directory.
After you got background videos under the generated/background/video directory, you need to convert them to images.
Select the source background (video). Then input the crop size of the mp4. (e.g. The video might be 4000*3000, you many want the image with width 1000 and height 1000, from the position x = 0, y = 1000.)
Every 0.5 seconds of the video make an image, which means a 30-seconds-long mp4 would generate 60 background images. The output background image would generated under the directory generated/background/image.
If you already have the background image with a big size, you want to resize it with an appropriate ratio. You can use this command to change the resolution of background images batch.
The object images are captured with background, you should always extract object out of the background.
Input the resize ratio to fit the background size and do extracting. The output images would generated under the directory generated/extracted_object. Sub folders would generate automatically.
If some of the extract images does not fit the size you want, you can delete them and do extract again. Extract objects would only deal with the non-existed object.
First input how many training images you want to generated for each item.
Then input the maximize item count in one image.
Training image would be generated under the directory generated/training_image.
Training image info (xml) would be generated under the directory generated/training_image_info.
Training image with item border would be generated under the directory generated/training_image_with_item_border.
First input how many validation images you want to generated for each item.
Then input the maximize item count in one image.
Validation image would be generated under the directory generated/validation_image.
Validation image info (xml) would be generated under the directory generated/validation_image_info.
Validation image with item border would be generated under the directory generated/validation_image_with_item_border.
First input how many test images you want to generated for each item.
Then input the maximize item count in one image.
Test image would be generated under the directory generated/test_image.
Test image info (xml) would be generated under the directory generated/test_image_info.
Test image with item border would be generated under the directory generated/test_image_with_item_border.
Clean up training, validation and test images and related files.
python3 make_model.py
You can have many models, so input the model name.
Then the script will create model based on current training, validation and test images.
python3 check_model.py
Prepare the images for checking the model generated.
When you see 'How many images to retrieve (lates)?', input the numbers of pictures you took for checking.
First you should select model for checking.
Then the script will apply model checking with the images under folder generated/check_image, result images would under folder generated/check_result.
python3 coco_predictions.py [threshold]
Create a coco format json output which contains the labelled image boundries information generated from trained tflite model. Coco format is consistent with the evaluation tool that be used for checking AP(Average Precision) and MAP(Mean Average Precision). 'threshold' here is a numeric value between 0 and 1, usually it is ranged from 0.3 to 0.5 by default. For more information about the evaluation tool, please refer to object detection evaluation tool