Explore modules

Activation Functions

Visualize and compare common neural network activation functions.

Free theory article

Visualize and compare common neural network activation functions.

Concept & interpretation

Activation functions are mathematical equations attached to each neuron in a neural network, determining whether the neuron should be activated. Crucially, they introduce non-linearities into the network. Without non-linear activation functions, regardless of how many layers a neural network has, it would simply behave as a single-layer perceptron capable only of solving linearly separable problems.

Read the full free theory article