Tweedie Gaussian Denoising

Tweedie's formula offers an elegant connection between the marginal distribution of observed data and the posterior expectation of the underlying latent variable. In Gaussian denoising, knowing the score of the marginal data density directly gives the Bayesian minimum mean squared error (MMSE) estimate of the original signal.

Experiment

You can interact with these variables directly in the module: Tweedie Gaussian Denoising. The visualization shows how the densities change and visualizes the Tweedie correction as a yellow arrow.

1. Set the Prior mean to 0, Prior variance to 1, Noise variance to 1, and Observed value to 2. Observe the Marginal score of -1.0 and Posterior mean of 1.0, matching our worked example.
2. Change the Prior variance to 0.25 and the Noise variance to 4. Notice the marginal distribution becomes much wider, and the posterior mean is strongly pulled towards the prior, yielding approximately 1.058824.
3. Set the Noise variance to 0. The marginal score becomes -2.0, but because the noise is zero, the correction is 0.0. The posterior perfectly recovers the observation y = 2, and the posterior variance is 0.
4. Set both Prior variance and Noise variance to 0. Set Observed value to 0. The marginal distribution is a consistent point mass, and the log-density score is unavailable.