6.3 Learning in Artificial Neural Networks
Where alpha is a parameter that controls hoe fast the learning takes place. This is called a learning rate. The choice of the learning rate parameter can have an impact on how fast (and how correctly) a neural network learns. A high values for the learning rate can lead to too much correction in the weight values, which causes the algorithm to just go back and forth among possible weight values, never reaching the optimal values, which may lie somewhere in between the endpoints. Too low a learning rate may slow the learning process and may lead to sub-optimal weight values. In practice, a neural network analyst will try many different learning rate to achieve the optimal learning.