Probability Calibration & Brier Score
Machine Learning
About Probability Calibration & Brier Score
Compare forecast probabilities with observed frequencies and decompose binary Brier error into reliability, resolution and uncertainty.
A classifier can rank examples well while assigning misleading probabilities. Probability calibration asks whether an event occurs about 80% of the time among forecasts of 80%, and similarly at other probability levels. The Brier score evaluates the numerical forecasts themselves, using squared error against binary outcomes. This module separates that overall error into empirical reliability, resolution and uncertainty, so a low calibration penalty is not confused with an informative forecast.
The theory article is publicly readable. Interactive access follows the current free-module and PRO settings shown above.