error in modb_evaluation.py

[Sabarinathan]

I followed the steps and placed the results in the below folder structure.

I used the below YAML file for evaluation .

PATHS:
  # Results output folder
  RESULTS: 'D:/WACV2022/mods_new_output/'
  # Path to where the dataset is stored
  DATASET: 'D:/WACV2022/mods/'
  # Path to where calibration files are stored
  DATASET_CALIB: 'D:/WACV2022/mods/calibration'
  # Path to where the segmentation predictions are stored
  SEGMENTATIONS: 'D:/WACV2022/mods_new_output/'

SEGMENTATIONS:
  # Prediciton dir structure:
  #  - False: pred_dir/method/sequence/frame.png
  #  - True:  pred_dir/sequence/method/frame.png
  SEQ_FIRST: False
  # Specify which colors are used in segmentation maps
  INPUT_COLORS: 
    - [0, 0, 0]    #[247, 195,  37] # Obstacle
    - [255, 0, 0]  #[ 41, 167, 224] # Water
    - [0, 255, 0]  #[ 90,  75, 164] # Sky

VISUALIZATION:
  # Display per-sequence evaluation progress
  SEQUENCE_PROGRESS: True

overlap_percentages, overlap_percentages_d = run_evaluation_image(self.cfg, File "D:\WACV2022\mods_evaluation-master\mods_evaluation-master\modb_evaluation.py", line 407, in run_evaluation_image raise 'Missing number of something for sequence %s frame %d' % (seq_path_split[0], frame_number) TypeError: exceptions must derive from BaseException

[Lojze Žust]

Hi! The predictions need to be placed into a subdir with the name of your method. I suggest you create a directory D:/WACV2022/predictions where you will store all your predictions. You then set this folder in the YAML file (RESULTS: 'D:/WACV2022/predictions'.

Inside the predictions folder, create a directory for your method (e.g. D:/WACV2022/predictions/method1). Then move the contents of mods_new_output to the method1 directory.

You can then call the evaluator like this:

python modb_evaluation.py --config-file /path/to/config.yaml method1

When you have new predictions just put it in a different subdir, and you can call the evaluator again.
Hope this helps!

[Sabarinathan]

@lojzezust , As per your suggestions, I have created the new folder and moved the predicted images to the method1.

But still I am getting the same error.

YAML file

PATHS:
  # Results output folder
  RESULTS: 'D:/WACV2022/predictions/'
  # Path to where the dataset is stored
  DATASET: 'D:/WACV2022/mods/'
  # Path to where calibration files are stored
  DATASET_CALIB: 'D:/WACV2022/mods/calibration'
  # Path to where the segmentation predictions are stored
  SEGMENTATIONS: 'D:/WACV2022/predictions/method1/mods_new_output/'

SEGMENTATIONS:
  # Prediciton dir structure:
  #  - False: pred_dir/method/sequence/frame.png
  #  - True:  pred_dir/sequence/method/frame.png
  SEQ_FIRST: False
  # Specify which colors are used in segmentation maps
  INPUT_COLORS: 
    - [0, 0, 0]    #[247, 195,  37] # Obstacle
    - [255, 0, 0]  #[ 41, 167, 224] # Water
    - [0, 255, 0]  #[ 90,  75, 164] # Sky

VISUALIZATION:
  # Display per-sequence evaluation progress
  SEQUENCE_PROGRESS: True

Error Message :

Traceback (most recent call last):
  File "D:\WACV2022\mods_evaluation-master\mods_evaluation-master\modb_evaluation.py", line 405, in run_evaluation_image
    seg = code_mask_to_labels(seg, cfg.SEGMENTATIONS.INPUT_COLORS)
  File "D:\WACV2022\mods_evaluation-master\mods_evaluation-master\utils\utils.py", line 21, in code_mask_to_labels
    segmentation_mask = cv2.cvtColor(segmentation_mask, cv2.COLOR_BGR2RGB)
cv2.error: OpenCV(4.6.0) D:\a\opencv-python\opencv-python\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\sabar\anaconda3\lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "D:\WACV2022\mods_evaluation-master\mods_evaluation-master\modb_evaluation.py", line 128, in process_sequence
    overlap_percentages, overlap_percentages_d = run_evaluation_image(self.cfg,
  File "D:\WACV2022\mods_evaluation-master\mods_evaluation-master\modb_evaluation.py", line 407, in run_evaluation_image
    raise 'Missing number of something for sequence %s frame %d' % (seq_path_split[0], frame_number)
TypeError: exceptions must derive from BaseException
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\WACV2022\mods_evaluation-master\mods_evaluation-master\modb_evaluation.py", line 474, in <module>
    run_evaluation()
  File "D:\WACV2022\mods_evaluation-master\mods_evaluation-master\modb_evaluation.py", line 249, in run_evaluation
    for i, result in tqdm(enumerate(eval_generator), total=len(sequences), desc='Processing sequences'):
  File "C:\Users\sabar\anaconda3\lib\site-packages\tqdm\std.py", line 1195, in __iter__
    for obj in iterable:
  File "C:\Users\sabar\anaconda3\lib\multiprocessing\pool.py", line 870, in next
    raise value
  File "C:\Users\sabar\anaconda3\lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "D:\WACV2022\mods_evaluation-master\mods_evaluation-master\modb_evaluation.py", line 128, in process_sequence
    overlap_percentages, overlap_percentages_d = run_evaluation_image(self.cfg,
  File "D:\WACV2022\mods_evaluation-master\mods_evaluation-master\modb_evaluation.py", line 407, in run_evaluation_image
    raise 'Missing number of something for sequence %s frame %d' % (seq_path_split[0], frame_number)
TypeError: exceptions must derive from BaseException

[Lojze Žust]

Then something is probably wrong with the predictions. Does this happen right at the beginning of the evaluation or is some progress already made?

It seems that the prediction segmentation mask (seg) is empty, which probably means that some of the predictions masks are missing or named incorrectly.

You can add a print statement before line 407 to print the expected path of the segmentation mask. This way you will be able to identify the source of the error.

print(os.path.join(cfg.PATHS.SEGMENTATIONS, method_name, seq_path_split[0], '{}.png'.format(img_name_split[0])))

Let me know if you encounter other problems.

[Lojze Žust]

I've also pushed a fix to the master branch, which should output a clearer exception message about the sequence and frame names where the error happens.

[Lojze Žust]

You can also compare the structure of your predictions with this example. Note that in the example we use a custom color scheme different than the default configuration, but other than that everything is the same.