Cramér–Rao Biased Variance Bound
Explain why shrinkage can fall below an unbiased variance bound without contradicting Cramér–Rao.
Free theory articleExplain why shrinkage can fall below an unbiased variance bound without contradicting Cramér–Rao.
Concept & interpretation
The Cramér-Rao lower bound establishes a fundamental limit on the variance of unbiased estimators. However, estimators that introduce a bias, such as Bayesian estimators or those using a pseudocount, can achieve a mean squared error (MSE) that is lower than this unbiased bound. This module explains why this phenomenon does not contradict the Cramér-Rao theorem, by exploring the bias-adjusted variance bound.
Experiment
You can observe the tradeoff between bias and variance interactively in the module.
- Set the True probability to 0.25, Sample size to 4, and Pseudocount to 1. Observe the MSE falls below the Unbiased CR Bound.
- Change the Pseudocount to 0. The bias becomes zero, and both the variance and the MSE exactly equal the Unbiased CR Bound.
- Change the True probability to 0.5 and the Pseudocount to 1. Notice the bias is zero at this specific point, but the variance still follows the lower bias-adjusted bound rather than the unbiased bound because the derivative of the bias is not uniformly zero.