But what happens if the new set to be learned does not share the regularities of the
patterns that the network was pre-trained on? Our own neural nets, for example, have been
“pre-trained” on many instances of birds. We have, in short, learned many regularities about
birds. But without difficulty we can learn to recognize penguins, which share very few of the
previously learned “bird regularities” but are still birds, and bats, which share a great many
regularities but are not. Here are cases where “pre-training” would be of little use in
preventing interference because the regularities of the pre-training set would not be shared by
the regularities of the new information to be learned