Linear Discriminant Analysis
Find a class-aware projection by balancing mean separation against within-class covariance.
Free theory articleFind a class-aware projection by balancing mean separation against within-class covariance.
Concept & interpretation
Linear Discriminant Analysis (LDA) is a method to find a linear combination of features that characterizes or separates two or more classes of objects or events. The resulting combination may be used as a linear classifier or for dimensionality reduction.
Experiment
Try the Linear Discriminant Analysis module.
- Set Correlation to 0.5.
- Set Shrinkage to 0.
- Set Probe x to 1 and Probe y to 0.
- Observe the Fisher separation (5.333) and log likelihood ratio (2.667). The direction vector is normalized. The predicted class is 1.
- Change Probe x to 0 and Probe y to 1. The log likelihood ratio becomes -1.333, and the predicted class changes to 0.
- Change Correlation to -0.5. The direction vector's y component flips sign, and the covariance ellipses tilt in the opposite direction.
- Set Shrinkage to 1. The covariance is shrunk to the identity matrix, turning the ellipses into circles and decoupling the dimensions.