Why don't Temporal refiner module work well in my dataset?

[Tao]

Thank you so much for your excellent work！！！
I want to make the temporal refine mechanism like your work. First, I use a trained model to extract the time series features, which fuse temporal information. Then put it into the temporal refine module, but I found that it don't work well.
I think one reason is that my dataset size is relatively small. And the feature exactor is based on CNN.
Could you give me some advice? Thank you very much!!!

[zhangtao]

Thanks for your attention. Could you please provide me with more detailed information, such as your dataset and task details, your configuration file, and training logs?

[Tao]

My task is to perform segmentation on medical videos, with only 150 videos as training samples.
In this process, I used the TemporalRefiner module for optimization. Some parameters are set as follows

   self.refine = TemporalRefiner(
            hidden_channel=512,
            feedforward_channel=512,
            num_head=8,
            decoder_layer_num=2,
            mask_dim=512,
            class_num=1,
            windows=5
        )

Sorry， I don't save the training logs, but the loss does not drop much

[Tao]

大佬可以加个好友嘛 想交流一下

[zhangtao]

好的，微信18842489947.