Polyak Optimal-Value Step
Compare an optimal-value step with exact line search and a fixed learning rate.
Free theory articleCompare an optimal-value step with exact line search and a fixed learning rate.
Concept & interpretation
The Polyak optimal-value step determines the learning rate dynamically based on the known minimum value of the objective function. It aims to take steps that reach the optimal contour without requiring knowledge of the optimum's coordinates.
Experiment
To observe the behavior of the Polyak step compared to exact line search, you can interact with the module.
- Set the Condition ratio to
4. - Set the Start point to
Diagonal. - Select the Rule as
Exact line search. Notice how the step sizes differ significantly from the Polyak rule because exact line search minimizes along the gradient ray. - Adjust the Steps to
2and observe the trajectory on the canvas. Switch back toPolyakand compare the steps and objective values.