ValueError: could not broadcast input array from shape (80,) into shape (160,)

[Francesco Saverio Zuppichini]

Hi there 👋

I am trying to write an object detection dataset

...
ds = YOLODataset(Path("/home/zuppif/Documents/neatly/detector/datasets/train"), padding=True)
print([el.shape for el in ds[0]])

writer = DatasetWriter("dataset.beton", {
    'image': RGBImageField(),
    'label': NDArrayField(shape=(ds.max_num_of_labels, 1), dtype=np.dtype(np.int64)),
    'bbox': NDArrayField(shape=(ds.max_num_of_labels, 4), dtype=np.dtype(np.float32)),
}, num_workers=4)


writer.from_indexed_dataset(ds)

YOLODataset returns a tuple with shapes [(1080, 1920, 3), (20, 1), (20, 4)]

I've got the following error

Traceback (most recent call last):
  File "/home/zuppif/miniconda3/envs/ffcv/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/zuppif/miniconda3/envs/ffcv/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/zuppif/miniconda3/envs/ffcv/lib/python3.9/site-packages/ffcv/writer.py", line 113, in worker_job_indexed_dataset
    handle_sample(sample, dest_ix, field_names, metadata, allocator, fields)
  File "/home/zuppif/miniconda3/envs/ffcv/lib/python3.9/site-packages/ffcv/writer.py", line 51, in handle_sample
    field.encode(destination, field_value, allocator.malloc)
  File "/home/zuppif/miniconda3/envs/ffcv/lib/python3.9/site-packages/ffcv/fields/ndarray.py", line 99, in encode
    data_region[:] = field.reshape(-1).view('<u1')
ValueError: could not broadcast input array from shape (80,) into shape (160,)

Looks like is trying to review the bboxes into 160 but not sure why.

Thanks a lot in advance

Cheers,

Fra

[Andrew Ilyas]

The most common reason for this is that your dataloader is returning something of the wrong datatype (e.g., int32 instead of int64 or float64 instead of float32). You can test this by casting the data to the declared datatype explicitly inside your dataloader.

[Francesco Saverio Zuppichini]

thanks, let me check and report back 🤗