Expectation-Maximization for Mixtures

The Expectation-Maximization (EM) algorithm is an iterative method to find local maximum likelihood estimates of parameters in statistical models with unobserved latent variables. In this module, we fit a two-component Gaussian mixture model to one-dimensional data.

Experiment

Follow these steps using the Expectation-Maximization for Mixtures controls to explore convergence:

1. Set Initial mean magnitude to 1, and Known variance to 1. Note the initial log-likelihood.
2. Set EM iterations to 1. Observe how the means shift towards the outer data points.
3. Set EM iterations to 2. The positive mean updates to 1.427233, improving the likelihood further.
4. Set Known variance to 4 and EM iterations to 1. The positive mean becomes 0.584576; the variance was held at 4 during this replay.
5. Set Initial mean magnitude to 0, Known variance to 1 and EM iterations to 5. Every responsibility stays 0.5, every history mean stays zero, and log-likelihood stays -8.675754.
6. Set Initial mean magnitude to 2, Known variance to 0.25 and EM iterations to 0. No update is performed: the means remain -2 and 2, and current and initial log-likelihood agree.