365 (a) Agitator (b) Mockitator (c) Randoop Fig. 5. Success rate on projects when left out of the training set, relative to each tool’s overall ZeroR success rate We used two methods of cross-validation. 10-fold crossvalidation, a widely-accepted method for verifying machine learning classifiers [16], divides the data into 10 random subsets and performs cross-validation using 9 subsets for training and the remaining subset for testing. Leave-project-out (LPO) cross-validation, a method specific to our experiments, trains a decision tree on all projects except one which is left out for testing. We compared these two success rates against the success rate of the trivial ZeroR classifier that always predicts the most common classification. For example, if most of the training data got low coverage, ZeroR will predict low coverage on all inputs.