Gradient Descent
Finding the minimum of a cost function.
Free theory articleFinding the minimum of a cost function.
Concept & interpretation
Gradient descent is the workhorse optimization algorithm behind modern machine learning. The idea is elegantly simple: to minimize a function, take repeated steps in the direction where the function decreases most steeply. This iterative approach transforms the abstract problem of "finding the best parameters" into a concrete, mechanical procedure that scales to models with billions of parameters—the foundation upon which deep learning is built.