About finetune.py
heguohao0728 opened this issue · comments
Hello, Great work!
I want to train my own module but I met some problems. Could you please give me some advice. Really appreciated.
In RTS, the finetune.py is same as PSM-Net, and it will encounter so problem.
Like:
if args.model == 'stackhourglass':
model = stackhourglass(args.maxdisp)
elif args.model == 'basic':
model = basic(args.maxdisp)
else:
print('no model')
if I use arg.model = RTS..or something, it will report a bug.
and in submodule.py:
class disparityregression(nn.Module):
def __init__(self, maxdisp):
super(disparityregression, self).__init__()
self.disp = torch.Tensor(np.reshape(np.array(range(maxdisp +1)), [1, maxdisp+1, 1, 1])).cuda()
this part is different from PSM-Net, and it will enconter a size bug.
How can I do with these problems?
What's more.
In submodule.py, if I change the above maxdisp + 1 to maxdisp and avoid the problem, it still have some issue:
class disparityregression(nn.Module):
def __init__(self, maxdisp):
super(disparityregression, self).__init__()
self.disp = torch.Tensor(np.reshape(np.array(range(maxdisp)), [1, maxdisp, 1, 1])).cuda()
def forward(self, x):
out = torch.sum(x * self.disp.data,2, keepdim=True)
return out
In the forward function, the "2" must be "1", or the output will have a size problem.
When I do my change work above, the finetune.py can run correctly,
but it seems the finetune.py do not use any code in the RT_Stereo.py,
and it might still be PSM-Net not RTS.
It seems that the model generation model = stackhourglass(args.maxdisp) is not correct,
I change to model = RTStereoNet(args.maxdisp), and it worked.
Is it right?
When I use the pretrained model in the test_img.py, the pretrained model have a probelm.
In the ordereddict state_dict, every key seems do not have the header module., and cannot be load with load_state_dict method.
Could you please give some advice about this? or specify how to use pretrained model? or update the finetune.py and test_img.py?
Really Appreciated!
@heguohao0728 were you able to run inference on this one with the weights provided ?
@heguohao0728 @Abhishekvats1997
The header "module" is indicated that the model is trained using nn.DataParallel().
If You do not use the nn.DataParallel(), loading the pretrained weight without this.
I will modify it as soon as possible.
@JiaRenChang
Thank you so much!
But I found that the weight and the sequence in pre-trained model is not the same as the model that is defined in code.
For example, the pre-trained model has "stage4" but the model defined in code have "attention" instead.
What's the difference?
I modify all of them.
You would load pretrained model without conflicts now.
Let me know if it has problems.
@JiaRenChang
Thank you so much! I'v tried, but my result seems not good enough:
Is it right? or I am not using the pretrained model correctly?
Besides, I notice that the attention module in RTS is deleted and replaced by stage4. Is the stage4 means attention module?
Appriciated.
@heguohao0728
Hi, I think it right.
The proposed method has around 7% 3px error on KITTI dataset, but it runs real-time on TX2.
If you want accurate disparity map instead of inference speed, you can try PSMNet.
Yep, the stage4 means that the attention module.
@JiaRenChang
That help a lot! Really appreciated for your reply!