machine-learning

An activation function in a machine learning model is used to add Non-linearity to the output. This is needed, because otherwise, our model will learn to act as a linear least squares model. We usually want our models to solve complex, non-linear problems.

- ReLU - $f(x) = max(0, x)$
- Sigmoid - $f(x) = \frac{1}{1+e^{-x}}$
- Identity - $f(x) = x$