Another potential drawback comes from the expected uncertainty being computed over the distribution p(y|x, θ u) by utilizing the currently estimated model θ u.
The currently estimated model could be far from the true model, particularly when
the number of training points is small, but the number of parameters to be estimated is large. Therefore, performing AL based only on a single estimated model
can be misleading [25]. Let us illustrate this by the following example shown in
Figure 23.5. The four existing training points are indicated by solid line contours,
test points by dashed ones. Based on these four training examples, the most likely
decision boundary is the horizontal line (dashed), even though the true decision
boundary is a vertical line (solid). If we select training input points based only on
the estimated model, subsequent training points would likely be obtained from areas along the estimated boundary, which are ineffective in adjusting the estimated
decision boundary (horizontal line) towards the correct decision boundary (vertical
line). This example illustrates that performing AL for the currently estimated model