Mini-Batch Gradient Descent
Optimize parameters efficiently by estimating gradients on small data batches.
Free theory articleOptimize parameters efficiently by estimating gradients on small data batches.
Concept & interpretation
Mini-Batch Gradient Descent is the practical workhorse of modern deep learning. It strikes a balance between the computational efficiency of Stochastic Gradient Descent (SGD) and the stability of full Batch Gradient Descent. By computing the gradient of the cost function over a small, randomly selected subset (a "mini-batch") of the training data, it enables fast, frequent parameter updates while still providing a sufficiently accurate estimate of the true gradient to ensure steady convergence.