Calculation about focal loss

[BinsonW]

Thanks for your great works!

As far as I know, the focal loss is $ FL = -y(1-p)^\gamma \log{p} - (1-y)p^\gamma \log{1-p} $, but the code in the focal_loss.py:

        coeff = torch.abs(label - probs).pow(self.gamma).neg()
        log_probs = torch.where(logits >= 0,
                F.softplus(logits, -1, 50),
                logits - F.softplus(logits, 1, 50))
        log_1_probs = torch.where(logits >= 0,
                -logits + F.softplus(logits, -1, 50),
                -F.softplus(logits, 1, 50))
        loss = label * self.alpha * log_probs + (1. - label) * (1. - self.alpha) * log_1_probs
        loss = loss * coeff

Seems like the $ \log{p} $ in the focal loss is implemented by the function F.softplus()? But the F.softplus() comes out with $ y=\log{1+e^x} $, which is not $ \log{x} $ .

How should I understand this difference? Or did I miss something necessary about focal loss?

[CoinCheung]

That is math formula derivation in order to make expression such as log(exp + 1) more stable.

Also please do not neglect sigmoid function. It is also included in the loss function, which is same as nn.BCEWithLogitsLoss.