Jensen–Shannon Divergence
Mix two distributions to obtain a bounded divergence with explicit zero-mass conventions.
Free theory articleMix two distributions to obtain a bounded divergence with explicit zero-mass conventions.
Concept & interpretation
The Jensen–Shannon divergence is a method of measuring the similarity between two probability distributions. Unlike the Kullback-Leibler divergence, it is symmetric, always finite, and its square root is a metric.
Interactive Experiment
Explore the interactive Jensen-Shannon Divergence module here: JS Divergence Visualizer.
In this module, you can select specific distribution shapes and change the mixture weight.
- Uniform vs Point: Set P to "Point A", Q to "Uniform", and Mixture weight to "0.5". The divergence decreases to ~0.318 nats, showing that overlapping distributions have a divergence strictly less than the maximum possible entropy bound (0.693 nats).
- Asymmetric Weighting: Keep the same distributions (Point A and Point C) but change the Mixture weight to "0.25". Observe that the divergence drops to ~0.562 nats, aligning with the new lower entropy bound H(0.25). Note that at unequal weights, JS divergence is neither symmetric nor a metric.
- Zero Weight Edges: Change the Mixture weight to "0". The JS divergence becomes exactly 0, even though the unused KL term (from P to M, since M = Q) would theoretically be infinite due to the missing support in Q. The algorithm explicitly handles this by skipping zero-weight terms.