Sparsemax Simplex Projection

The sparsemax function maps an unnormalized vector of logits to a probability distribution, similar to softmax. However, unlike softmax, which always assigns a dense distribution with strictly positive probabilities, sparsemax computes the Euclidean projection of the logits onto the probability simplex. This Euclidean projection naturally hits the boundaries of the simplex, yielding exactly zero probability for logits that are too small relative to the leading candidates.

Experiment

You can interact with the projection operation by adjusting the logits spread and the temperature.

1. Open the Sparsemax Simplex Projection module.
2. Set the Logits control to "Separated". Observe how the smallest two logits are aggressively pushed below the threshold, resulting in exactly zero probabilities, leaving only one active coordinate.
3. Change the Logits to "Close" and lower the Temperature to 0.5 (which scales the logits to [0, 0.5, 1.0]). The increased variance pushes the lowest coordinate to zero.
4. Compare the Sparsemax probabilities to the traditional Softmax output. Softmax never assigns exactly zero probability, no matter how small the logit, whereas Sparsemax efficiently identifies a compact active set.