Issue about high feature level set.
Yangr116 opened this issue · comments
Hi, I have another question about your paper and codes.
In Eq. (8) of the paper,
img_mst_tree = self.mst(img_target)
deep_stru_feature_img = self.tree_filter(mask_pred, img_target, img_mst_tree)
lst_mst_tree = self.mst(lst_target)
deep_stru_feature_lst = self.tree_filter(deep_stru_feature_img, lst_target, lst_mst_tree, low_tree=False)
high_feature = torch.cat((deep_stru_feature_img, deep_stru_feature_lst), dim=1) * box_mask_target
loss_feat_lst = self.loss_levelset(mask_scores_phi, high_feature, pixel_num) * 5.0
, high_feature
is the refined mask score.
How can I consider it a high-level feature? Did you try to use the feature level_set_feats
in the level set loss directly?
May I see this format as the Tree Energy Loss? The difference is that the metric is changed to level set rather than L1 distance.
@Yangr116
Hello, in the early version of this work exploration, I use the feature level_set_feats
as the input data as the image for level set loss. It can also work for simple scene, like aerial scene and medical scene without the multiple objects and complex occlusion. The early exploration is here (technical report). This work can not work well for genecal coco.
For the complicate general coco, the direct level_set_feats
can also work, but without the stable and advanced performance.
To explore to obtain the better performance, I found the long-range operation can achieve the stable performance, like non-local
and 'tree filter' operation. The tree_filter here is regarded as the structural feature enhance with long-range.
I also have tried the direct l1 loss in Tree Energy Loss,which achieves the performance drop 0.9-1.2 mAP, especially with large model as Swin-L based model.
I see. Thanks for your reply!