Noise-Contrastive Density Ratio
Recover a density ratio from binary discrimination while exposing sampling priors and support holes.
Free theory articleRecover a density ratio from binary discrimination while exposing sampling priors and support holes.
Concept & interpretation
Noise-Contrastive Estimation (NCE) reformulates the difficult problem of density estimation (finding the probability distribution of data) into a binary classification problem. We train a classifier to distinguish between samples from the true data distribution p(x) and samples from a known noise distribution q(x).
Experiment
Explore the interactive module to see how the noise distribution affects recovery:
- Open the Noise-Contrastive Density Ratio module.
- Set Data distribution to
Unevenand Noise distribution toSkewed. Set State probe to0. Observe the recovered data probability perfectly matches the true probability. - Change Noise distribution to
Support holeand State probe to2. Observe that the recovered probability is "Unavailable" because the noise distribution does not cover this state (q(x)=0).