Low Volatile GPU utility caused by Transform process which costs too much time

[yosunpeng]

I have noted that this implementation bring some box assignments in ssd.pytorch from GPU to CPU. The dataset return preprocessed bboxes (bboxes.shape=[8732, 4]) instead of ground-truth boxes. In my case, transform progress in my costumed dataset is about 10 times slower than one in original pascal voc. That cause my GPUs keep starving for low CPU data processing.
I printed the transform time by time library in python3. Here are the contradistinction:
Both transform time calculated by following codes:
t0 = time.time()
if self.transform is not None:
img, bboxes = self.transform(img, bboxes)
t1 = time.time()
delta = t1 - t0
print('sec', delta%60)
In Pascal VOC:

In my dataset:

Of cause, I have already check other part's cost time in getitem method, they are pretty much closed, details will not be presented.
The only reason I can directly recall is the different image sizes between two datasets. VOC's image shape is around (375, 500, 3), (500, 334, 3) [cv2.imread, HWC], but my dataset images' size is like (1080, 1920, 3), (1078, 1916, 3), (1500, 2000, 3), which are much bigger than VOC's.
Is there any other possible reason? Transform is kind of too complex for me. Please help me accelerate my dataloader! THINKS!