go-v.orgcodex
Menu
Page tools

codex · Psychology

Activation function

Diagram for Activation function
In artificial neural networks, the activation function of a node is a function that calculates the output of the node based on its individual inputs and their weights. Nontrivial problems can be solved using only a few nodes if the activation function is nonlinear. · image: Wikimedia Commons

In artificial neural networks, the activation function of a node is a function that calculates the output of the node based on its individual inputs and their weights. Nontrivial problems can be solved using only a few nodes if the activation function is nonlinear. Modern activation functions include the logistic (sigmoid) function used in the 2012 speech recognition model developed by Hinton et al; the ReLU used in the 2012 AlexNet computer vision model and in the 2015 ResNet model; and the smooth version of the ReLU, the GELU, which was used in the 2018 BERT model.

Wikipedia contributors, "Activation function", Wikipedia, revision 1375551500, retrieved 20 November 2026, source, CC BY-SA 4.0.

More Psychology · Search the codex