Difference inference result with and without using batch inference?

[zhihao zhao]

Hi,

First thanks for your nice library.

I try to inference a batch like #28, but the result seems not aligned with no batch inference. My code for batch is as follows:

    # images: a list of numpy array in H,W,C
    io_adapter = IOAdapter(model, images[0].shape[:2])
    inputs = io_adapter.prepare_inputs(np.array(images))
    input_images = inputs["images"][0]
    video1 = input_images[:-1]
    video2 = input_images[1:]
    input_images = torch.stack((video1, video2), dim=1)
    inputs["images"] = input_images
    out = model(inputs)
    out = io_adapter.unpad_and_unscale(out)

Best

[Henrique Morimitsu]

Hi, thanks for reporting.

Could you also show how you do the non-batch inference? I did a simple test and they give the same results, but maybe you are doing in a different way. Please try to provide a more complete example showing the difference.

Best,

[zhihao zhao]

Hi, yesterday I was testing with CPU. I'll test it with GPU and see if they are aligned.

While using GPU, I found that the algorithm takes more than 32G memory if I test a batch of images in a sequential manner, which is unreasonable. And "with torch.no_grad():" solves this problem.

Best

[zhihao zhao]

Hi,

I think the batch version should be correct. I had different input for batch and sequential before.

Best