value has a smaller average ranking than confidence. All the points in Figure 1 are above the line,
i.e., information value outperforms confidence in each experiment. Moreover, information value
has statistically significantly better performance (p = 1.70313E-60, using a pairwise t-test).
To examine what happens inside each method, we compute the average ranking of the selections
in each iterative step of the selection process. In Figures 2 and 3, the x-axis represents the i-th
step and the y-axis represents the average ranking value of choices made at that step. Under both
greedy (Figure 2) and random (Figure 3) selection, both confidence and information value are
consistently better than support. Since the performance difference between confidence and IV is
difficult to see, we calculated the difference between them in each step, as shown in Figures 4
and 5. Under greedy selection, the performance of information value is constantly better than that
of confidence, increasing through the 8th selection. After that, the improvement decreases but is
still positive. However, no such pattern is evident under random selection, and overall there is no
difference between the two values. Figures 4 and 5 support the conclusion that the performance
of information value is almost the same as that of confidence in the random selection strategy
consistently better than confidence under greedy selection.