focal_loss

[gqy1025]

Hi,

Your project is very impressive and i am trying to re-train the model used my own dataset. Since the distribution of tags is extremely unbalanced so i chose to use focal loss for this issue.

I'm curious if you have ever tried focal-loss, and what is its final loss ? and how does the model performance compared to BCE loss?

BTW, from my perspective, the learning rate of 5 is to high for focal-loss.
Thank you again for bringing such a wonderful project!

[Kichang Kim]

I implemented and tested focal-loss, but it makes training speed be too slow. With same epoch (30), it still has too high loss and low F1 score. I think that training with focal loss need more iteration than normal loss.

You can test focal loss by changing training settings in project.json, like this: "loss": "focal_loss"

[gqy1025]

Thanks for your reply,
i am currently training resnet-v3, I would like to know what your learning rate was? still 5? I remember you mentioned earlier that the final loss was 0.006 with BCE loss.

i trained with focal_loss, and it converged really slow and the loss was around 14~15 .

[Kichang Kim]

I would like to know what your learning rate was? still 5?

Yes, it is still 5.