go-v.orgcodex
Menu
Page tools

codex · Philosophy

Softplus

Diagram for Softplus
In mathematics and machine learning, the softplus function is · image: Wikimedia Commons

In mathematics and machine learning, the softplus function is

f(x)=ln⁡(1+ex). It is a smooth approximation (in fact, an analytic function) to the ramp function, which is known as the rectifier or ReLU (rectified linear unit) in machine learning. For large negative x it is ln⁡(1+ex)=ln⁡(1+ϵ)⪆ln⁡1=0, so just above 0, while for large positive x it is ln⁡(1+ex)⪆ln⁡(ex)=x, so just above x. The names softplus and SmoothReLU are used in machine learning. The name "softplus" (2000), by analogy with the earlier softmax (1989) is presumably because it is a smooth (soft) approximation of the positive part of x, which is sometimes denoted with a superscript plus, x+:=max⁡(0,x).

Wikipedia contributors, "Softplus", Wikipedia, revision 1339378855, retrieved 20 November 2026, source, CC BY-SA 4.0.

More Philosophy · Search the codex