Explore modules

Mini-Batch Gradient Descent

Optimize parameters efficiently by estimating gradients on small data batches.

Free theory article

Optimize parameters efficiently by estimating gradients on small data batches.

Concept & interpretation

Mini-Batch Gradient Descent is the practical workhorse of modern deep learning. It strikes a balance between the computational efficiency of Stochastic Gradient Descent (SGD) and the stability of full Batch Gradient Descent. By computing the gradient of the cost function over a small, randomly selected subset (a "mini-batch") of the training data, it enables fast, frequent parameter updates while still providing a sufficiently accurate estimate of the true gradient to ensure steady convergence.

Read the full free theory article