Swish function

The swish function is a family of mathematical function defined as follows:
swishβ(x)=xsigmoid(βx)=x1+e−βx. where β can be constant (usually set to 1) or trainable and "sigmoid" refers to the logistic function. The swish family was designed to smoothly interpolate between a linear function and the Rectified linear unit (ReLU) function. When considering positive values, Swish is a particular case of doubly parameterized sigmoid shrinkage function defined in . Variants of the swish function include Mish.
Wikipedia contributors, "Swish function", Wikipedia, revision 1342997104, retrieved 20 November 2026, source, CC BY-SA 4.0.