Label Shift Prior Correction
Correct a calibrated posterior when class prevalence changes but class-conditional data do not.
Free theory articleCorrect a calibrated posterior when class prevalence changes but class-conditional data do not.
Concept & interpretation
Label shift occurs when the marginal distribution of labels P(Y) changes between the training (source) environment and the testing (target) environment, while the class-conditional distribution of features P(X|Y) remains invariant. If a model predicts a calibrated posterior q = Ptrain(Y=1|X) under the training distribution, we can correct it to reflect the target distribution by reweighting it based on the changed class priors.
Experiment
You can test the effect of changing class priors on the posterior using the Label Shift Prior Correction module.
- Set Training prior to
0.5. - Set Target prior to
0.25. - Set Training posterior to
0.75. - Observe that the calculated Target posterior is
0.5, which matches our worked example, showing that the reduced prevalence of class 1 in the target domain decreases its probability for this observation.