Rényi Entropy and Effective Support
Change entropy order to move from support counting to dominant outcomes.
Free theory articleChange entropy order to move from support counting to dominant outcomes.
Concept & interpretation
Rényi entropy is a family of functionals for quantifying the diversity, uncertainty, or randomness of a system. It generalizes the Shannon entropy, the Hartley entropy, the collision entropy, and the min-entropy.
Experiment
In the Rényi Entropy and Effective Support module, you can analyze different distributions and see how the entropy changes.
- Select Skew from the Distribution control to match our worked example.
- Change the Order to 2 and observe the calculated Rényi entropy and collision probability.
- Try Order Infinity to see the minimum entropy based only on the maximum probability.