Sequentially learning the populations using
pseudorehearsal illustrates both the protective effect on A,
and also interesting consequences for the network’s ability to
generalise. We train the network to criterion A, and then
train it to criterion on B using pseudorehearsal. Specifically,
before the learning of B begins we generate a population of
256 pseudoitems. The items of the B population are then
learned not alone, but in combination with three times as
many pseudoitems randomly selected from the population of
pseudoitems for each epoch. (In other words, if the size of
the B population is 2 then every epoch 6 pseudoitems are
included in the training set). After B has been trained to
criterion we can test the network on A, B, A-gens and B-gens
in the usual way. In Fig. 4 we can see that the use of
pseudorehearsal has done an excellent job of preserving
performance on A and A-gens (cf normal learning illustrated
in Fig. 3). B is trained to criterion as expected. Of particular
interest is performance on B-gens. This has suffered
somewhat compared to the other training methods (Figures 2
and 3).