The Structure of a Neural Net
The structure of a neural net consists of connected units referred to as
"nodes" or "neurons". Each neuron performs a portion of the
computations inside the net: a neuron takes some numbers as inputs,
performs a relatively simple computation on these inputs, and returns
an output. The output value of a neuron is passed on as one of the
inputs for another neuron, except for neurons that generate the final
output values of the entire system.
Neurons are arranged in layers. The input layer neurons receive the
inputs for the computations, like the length, diameter, and weight of
an individual abalone. These values are passed to the neurons in the
first hidden layer, which perform computations on their inputs and
pass their outputs to the next layer. This next layer could be another
hidden layer, if there is one. The outputs from the neurons in the last
hidden layer are passed to the neuron or neurons that generate the
final outputs of the net, like the age of the abalone.
Numeric and Category Prediction
When neural nets are used to predict numeric values, they typically
have just one output. This is because single-output nets are more
reliable than multiple-output nets, and almost any prediction problem
can be addressed using single-output nets. For example, instead of
constructing a single net to predict the volume and the price for a
stock on the following day, it is better to build one net for price
predictions, and one for volume predictions. On the other hand,
neural nets have multiple outputs when used for
classification/category prediction. For example, suppose that we
want to predict if the price of a stock the following day will "rise more
that 1%", "fall more than 1%", or "not change more than 1%". Then the
net will have three numeric outputs, and the greatest output will
indicate the category selected by the net.
Training a Net
Training a net is the process of fine-tuning the parameters of the
computation, where the purpose is to make the net output
approximately correct values for given inputs. This process is guided
by training data on the one hand, and the training algorithm on the
other. The training algorithm selects various sets of computation
parameters, and evaluates each set by applying the net to each
training case to determine how good the answers given by the net are.
Each set of parameters is a "trial"; the training algorithm selects new
sets of parameters based on the results of previous trials.
Computer Processing of Neural Nets
A neural net is a model of computations that can be implemented in
various types of computer hardware. A neural net could be built
from small processing elements, with each performing the work of a
single neuron. However, neural nets are typically implemented on a
computer with a single powerful processor, like most computers
currently in use. With single-processor computers the program, like
NeuralTools, uses the same processor to perform each neuron's
computations; in this case the concept of a neuron describes part of
the computations needed to obtain a prediction, as opposed to a
physical processing element.
Types of Neural Networks
There are various types of neural networks, differing in structure,
kinds of computations performed inside neurons, and training
algorithms. One type offered in NeuralTools is the Multi-Layer
Feedforward Network. With MLF nets, a NeuralTools user can
specify if there should be one or two layers of hidden neurons, and
how many neurons the hidden layers should contain (NeuralTools
provides help with making appropriate selections, as described in the
section on MLF nets). NeuralTools also offers Generalized
Regression Neural Nets and Probabilistic Neural Nets; these are
closely related, with the former used for numeric prediction, and the
latter for category prediction/classification. With GRN/PN nets there
is no need for the user to make decisions about the structure of a net.
These nets always have two hidden layers of neurons, with one
neuron per training case in the first hidden layer, and the size of the
second layer determined by some facts about training data.
The remaining sections of this chapter discuss in more detail each
type of neural network offered in NeuralTools.