Many activation functions have been tested, but only a few have found practical applications. Four common choices – the step, sign, linear and sigmoid functions –are illustrated in Figure 6.4.
The step and sign activation functions, also called hard limit functions, areoften used in decision-making neurons for classification and pattern recognition tasks.