Batch Normalization
Visualize the effects of batch normalization on neural network activations.
Free theory articleVisualize the effects of batch normalization on neural network activations.
Concept & interpretation
Batch Normalization is a technique used in deep neural networks to improve the speed, performance, and stability of the training process. During training, the distribution of inputs to internal layers changes as the parameters of preceding layers are updated—a phenomenon known as "internal covariate shift." Batch Normalization combats this by normalizing the inputs of a layer across the current mini-batch to have a mean of zero and a variance of one. It then applies learned scale and shift parameters to maintain the network's representational capacity.