Processing a single image with the model

[past9767]

Hello everyone,

i want to write an application that processes each frame of a video with the pretrained model of handobjectconsist
(producing the MANO-mesh and Object Pose for each frame).
I saw the code in the visualize.py file has a demonstration of the model, but it receives some sort of datastructure for the dataset which it gets by "dataset, input_res = get_dataset.get_dataset (...)", however i dont want to process a whole datastructure but just a single frame.
I would like to know, what would be the easiest way to process a single frame when each RGB-frame is given as a single opencv mat type.
I tried it like this (the getMat () funktion receives the frame from the video stream as a opencv mat):

resume = "releasemodels/fphab/hands_and_objects/checkpoint_200.pth"
opts = reloadmodel.load_opts(resume)
self.model, epoch = reloadmodel.reload_model(resume, opts)
freeze.freeze_batchnorm_stats(self.model)
self.model.cuda()
self.model.eval()

mat = getMat()
dataset = torch.utils.data.Dataset (np.array (mat))
loader = torch.utils.data.DataLoader(dataset,batch_size=1)
_, results, _ = self.model (loader)

However i get the error:
Failed to call callback: 'DataLoader' object is not subscriptable
Traceback (most recent call last):

Has anybody an idea how to fix this,
Thanks in advance,
Patrick

[Yana]

Hi @past9767 !

Note that the models are trained on small datasets which carry very specific biases (objects, points of view, ...), they will therefore not generalize to in-the-wild image !
However, if you still want to process a single image, the loader needs to be iterated upon for instance you can get the first batch data by running sample = next(iter(loader)).

Hope this helps !

Best,

Yana