Custom MaskRCNN error

Question

Custom MaskRCNN error

kavi0786 opened this issue 2 years ago · comments

Kavitha Lakshmi Ramanadham commented 2 years ago

Madam,
while i am executing the custom instance segmentation- MaskRCNN in googlecolab, i am getting the error. please see and let me know where i can modify....i am waiting for your reply..thanks in advance.

Kavitha

Mabu Manaileng · Answer 1 · Wed May 25 2022 01:21:37 GMT+0800 (China Standard Time)

Hi @kavi0786, I managed to isolate what's causing this issue and it turns out to be a series of problems.

**Issue 1: Data **
First of all, there are 2 things wrong with the dataset.

There's an incorrect image path: "imagePath": "..\butterfly (1).png".

There's a further erroneous image path in some examples:

The code below will resolve these data issues:

import json
import os
from tqdm import tqdm

DATA_DIR = "Datasets/Nature"
for _dir in os.listdir(f"{DATA_DIR}"):
    DATA_SET = f"{DATA_DIR}/{_dir}"
    for file in tqdm(os.listdir(DATA_SET)):
        if file.endswith(".json"):
            with open(os.path.join(DATA_SET, file), 'r') as f:
                data = json.load(f)
            data['imagePath'] = data['imagePath'].replace(
                '..\\', '').replace("images\\", "")
            with open(os.path.join(DATA_SET, file), 'w') as f:
                json.dump(data, f)

Issue 2: Compatibility with labelme2coco.convert()

However, solving these data issues actually raises new problems. convert actually expects an export directory as shown below, but a file name is being passed. It also handles the naming of export files differently, depending on train_split_rate:

See the below for the before and after of how to fix load_dataset. This fix is probably hacky because it still doesn't support custom train with valid set, we probably wanna fix this.

*Issue 3: labelme2coco.convert() does not respect the 0 class ID reservation
This means we'll always throw the below error:

A quick fix is a bit hacky, but it works logically!
class_id = category['id'] + 1 will ensure class IDs are > 0.

** SUCCESS! ** I got it working by following the above step. I'll create a PR and you can checkout to branch and use this fix while the PR is in review and hopefully gets some improvements.

chukwujike · Answer 2 · Wed May 25 2022 06:17:26 GMT+0800 (China Standard Time)

I am having the same issues while loading a custom dataset.
`IsADirectoryError Traceback (most recent call last)
in ()
----> 1 train_maskrcnn.load_dataset("cassava")
2 train_maskrcnn.train_model(num_epochs = 100, augmentation=True, path_trained_models="mask_rcnn_models")

1 frames
/usr/local/lib/python3.7/dist-packages/pixellib/custom_train.py in load_data(self, annotation_json, images_path)
172
173 # Load json from file
--> 174 json_file = open(annotation_json)
175 coco_json = json.load(json_file)
176 json_file.close()

IsADirectoryError: [Errno 21] Is a directory: '/content/drive/MyDrive/cassava/train.json'`

that's the error message

chukwujike · Answer 3 · Wed May 25 2022 06:18:01 GMT+0800 (China Standard Time)

I tried the solution you proposed but got a keyError: imagePath

Mabu Manaileng · Answer 4 · Wed May 25 2022 23:48:49 GMT+0800 (China Standard Time)

Looking at your error, you're missing the fix in Issue 2: Compatibility with labelme2coco.convert()
Try cloning the git repo and checkout to this branch: fix_MaskRCNN_labelme2coco, it should work.

Attila Kúr · Answer 5 · Sun Aug 14 2022 05:01:59 GMT+0800 (China Standard Time)

Hi @mabu-dev and thanks for your efforts; checking out your git repo and its branch of fix_MaskRCNN_labelme2coco really made the visualization of sample images work just OK! 👍

Any chance for also fixing the custom training?
Custom training gives this error currently:

/srv/conda/envs/saturn/lib/python3.9/site-packages/skimage/_shared/utils.py in _validate_interpolation_order(image_dtype, order)
    722 
    723     if image_dtype == bool and order != 0:
--> 724         raise ValueError(
    725             "Input image dtype is bool. Interpolation is not defined "
    726              "with bool data type. Please set order to 0 or explicitely "

ValueError: Input image dtype is bool. Interpolation is not defined with bool data type. Please set order to 0 or explicitely cast input image to another data type.

Mabu Manaileng · Answer 6 · Sun Aug 14 2022 05:25:53 GMT+0800 (China Standard Time)

Hi @kurattila, I'm glad you found the hacky fix useful. Can you describe what you're trying to do with custom training so I can reproduce the error? I'm happy to look into it.

Attila Kúr · Answer 7 · Sun Aug 14 2022 15:23:22 GMT+0800 (China Standard Time)

Sure @mabu-dev, I simply follow the steps in the tutorial for custom training, but the code seems to have an issue.

Running this training code:

###Training Code###

import pixellib
from pixellib.custom_train import instance_custom_training

train_maskrcnn = instance_custom_training()
train_maskrcnn.modelConfig(network_backbone = "resnet101", num_classes= 2, batch_size = 4)
train_maskrcnn.load_pretrained_model("mask_rcnn_coco.h5")
train_maskrcnn.load_dataset("Nature")
train_maskrcnn.train_model(num_epochs = 300, augmentation=True,  path_trained_models = "mask_rcnn_models")

Gives this output on Google Colab:

Using resnet101 as network backbone For Mask R-CNN model
There are 600 listed files in folder train.

Converting labelme annotations to COCO format: 100%|██████████| 600/600 [00:01<00:00, 393.46it/s]

There are 200 listed files in folder test.

Converting labelme annotations to COCO format: 100%|██████████| 200/200 [00:00<00:00, 331.16it/s]

Applying Default Augmentation on Dataset
Train 600 images
Validate 200 images
Checkpoint Path: /content/drive/MyDrive/Colab-Training/PixelLib/mask_rcnn_models
Selecting layers to train

/usr/local/lib/python3.7/dist-packages/skimage/transform/_warps.py:830: FutureWarning: Input image dtype is bool. Interpolation is not defined with bool data type. Please set order to 0 or explicitely cast input image to another data type. Starting from version 0.19 a ValueError will be raised instead of this warning.
  order = _validate_interpolation_order(image.dtype, order)
/usr/local/lib/python3.7/dist-packages/skimage/transform/_warps.py:830: FutureWarning: Input image dtype is bool. Interpolation is not defined with bool data type. Please set order to 0 or explicitely cast input image to another data type. Starting from version 0.19 a ValueError will be raised instead of this warning.
  order = _validate_interpolation_order(image.dtype, order)
/usr/local/lib/python3.7/dist-packages/skimage/transform/_warps.py:830: FutureWarning: Input image dtype is bool. Interpolation is not defined with bool data type. Please set order to 0 or explicitely cast input image to another data type. Starting from version 0.19 a ValueError will be raised instead of this warning.
  order = _validate_interpolation_order(image.dtype, order)

Epoch 1/300
 13/100 [==>...........................] - ETA: 1:45 - batch: 6.0000 - size: 4.0000 - loss: 7.4808 - rpn_class_loss: 0.6398 - rpn_bbox_loss: 3.0023 - mrcnn_class_loss: 0.7390 - mrcnn_bbox_loss: 2.3038 - mrcnn_mask_loss: 0.7960

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

[<ipython-input-2-55a8667fa051>](https://localhost:8080/#) in <module>()
      8 train_maskrcnn.load_pretrained_model("mask_rcnn_coco.h5")
      9 train_maskrcnn.load_dataset("Nature")
---> 10 train_maskrcnn.train_model(num_epochs = 300, augmentation=True,  path_trained_models = "mask_rcnn_models")

[...] 23 frames

[/usr/local/lib/python3.7/dist-packages/imageio/plugins/pillow.py](https://localhost:8080/#) in _palette_is_grayscale(pil_image)
    687         return False
    688     # get palette as an array with R, G, B columns
--> 689     palette = np.asarray(pil_image.getpalette()).reshape((256, 3))
    690     # Not all palette colors are used; unused colors have junk values.
    691     start, stop = pil_image.getextrema()

ValueError: cannot reshape array of size 537 into shape (256,3)

Later I plan to use my own labeled images for custom segmentation, but so far this seems to be a blocking issue even for training the sample Nature dataset from the tutorial.
Can you please look into solving this? Thanks a lot for your help! 🙂

Justin · Answer 8 · Sun Nov 20 2022 06:33:15 GMT+0800 (China Standard Time)

Hi @mabu-dev and thanks for your efforts; checking out your git repo and its branch of fix_MaskRCNN_labelme2coco really made the visualization of sample images work just OK! 👍

Any chance for also fixing the custom training? Custom training gives this error currently:
/srv/conda/envs/saturn/lib/python3.9/site-packages/skimage/_shared/utils.py in _validate_interpolation_order(image_dtype, order)
    722 
    723     if image_dtype == bool and order != 0:
--> 724         raise ValueError(
    725             "Input image dtype is bool. Interpolation is not defined "
    726              "with bool data type. Please set order to 0 or explicitely "

ValueError: Input image dtype is bool. Interpolation is not defined with bool data type. Please set order to 0 or explicitely cast input image to another data type.

I've found that this value error in scikit-image is only a warning in version 0.18.3 and is a value error in 0.19. I don't have a solution for the bool error as input but downgrading to 0.18.3 at least allows you to train a custom dataset.

skimage\transform_warps.py:830: FutureWarning: Input image dtype is bool. Interpolation is not defined with bool data type. Please set order to 0 or explicitely cast input image to another data type. Starting from version 0.19 a ValueError will be raised instead of this warning.