How to implement crf-rnn in torch so it can be integrated in RIS model?
brisker opened this issue · comments
How to implement crf-rnn in torch so it can be integrated in RIS model?
https://github.com/torrvision/caffe/blob/e7f25fa2bdbb291a067930d60f337d305c128583/src/caffe/layers/multi_stage_meanfield.cpp
I am afraid that it is not straightforward. I am not aware of any implementation of crf-rnn on torch, so it would involve porting the original caffe code. It basically means to implement the bilateral filter using the permutohedral lattice to torch.
Another option could be to call that code by means of lua interfaces, but I have 0 knowledge about how to do that. Maybe other people can help here?
@bernard24
besides, why do we need hungarian algorithm on ris model? Is this necessary? Just find the max IOU and assign the ground truth to the one with maximum IOU is not a good choice?
For example,
given an IOU matrix: (3 predictions and 2 ground truth)
[ 0.1 0.2 0.5
0.2 0.4 0.3 ]
Just assign prediction 1,2 3 with the ground truth 2 (0.2>0.1) , ground truth 2(0.4>0.2) and ground truth 1 (0.5>0.3) and it's done, right?
Why bother to use hungarian algorithm?
Hi @brisker
I am sorry I missed this question.
The advantage of using the Hungarian algorithm for the loss is that if the segmentations made at first steps are mistaken (e.g. overlapping 2 or more ground truth segmentations) you still allow the model amend and make good predictions in that area in later iterations. In addition, Hungarian algorithm is not any bottleneck here, although it might be if the number of instances per image is higher than the ones in the dataset I used.
Having said that, I did try the option you mentioned, and although results are not as good as when using the Hungarian loss, they are also decent.