Oja Hebbian Principal Direction Flow
A local learning rule can stabilize Hebbian growth toward a principal direction, yet a finite numerical step need not preserve its unit norm.
Free theory articleA local learning rule can stabilize Hebbian growth toward a principal direction, yet a finite numerical step need not preserve its unit norm.
Concept & interpretation
The Oja learning rule modifies the classical Hebbian rule by adding a weight decay term proportional to the square of the output, preventing weights from growing indefinitely and stabilizing them near a principal direction of the input data covariance matrix.
Experiment
Explore how the single-step update direction depends on the initial state and the covariance geometry by using the Oja Hebbian Principal Direction Flow module.
- Change the Weight angle to observe how the update vector (the Oja step) pulls the weight vector towards the dominant principal axis (the major eigenvector).
- Set the Covariance rotation to 45 to see how the axes of attraction change correspondingly.
- Set the Euler step to 0.25 to see how larger learning rates increase the numerical divergence from the unit circle, even though the continuous ODE preserves it.
- Adjust the Weight radius to 0.5 or 2 to see the effect on the continuous norm derivative and how the rule either grows or shrinks the vector back toward norm 1.