Explore modules

Residual Connections

Visualize how skip connections help mitigate the vanishing gradient problem in deep networks.

Free theory article

Visualize how skip connections help mitigate the vanishing gradient problem in deep networks.

Concept & interpretation

In deep learning, residual connections (also known as skip connections) are paths that allow signals to bypass one or more layers in a neural network. Introduced in Residual Networks (ResNets), these connections solve the infamous vanishing gradient problem, which historically prevented the training of very deep networks. By adding the input of a layer block directly to its output, residual connections create a "shortcut" for both forward activations and backward gradients.

Read the full free theory article