Focal Loss

Focal loss is designed to address extreme class imbalance by dynamically scaling the cross-entropy loss based on the prediction confidence. It down-weights the contribution of easy examples and focuses the model on hard examples.

Experiment

You can interact with the Focal Loss module directly.

1. Try changing Gamma to 0. You will see that the Focal Loss curve perfectly aligns with the Weighted Cross Entropy curve, as the modulation factor becomes 1.
2. Next, increase Gamma to 2 and change the Predicted probability to 0.99. Observe how the Focal Loss approaches zero much faster than the Weighted Cross Entropy, effectively ignoring this highly confident example.

Explore these effects here: Focal Loss Interactive Module