In order to build a classifier, we have to first identify potential signals (metrics), then select a classification approach, and finally implement, train and evaluate the classifier. A crucial part of training and evaluating a classifier is obtaining a standard data set which contains information about whether or not a code review comment is useful, a so called oracle. Relying on the 145 comments rated during the interviews is not sufficient as that data set is too small to build a reliable classifier. To rectify this, we manually analyzed and separately validated an additional 844 comments to enhance the oracle. In the following, we describe the manual classification, the oracle generation process, the selected signals and their calculation, the classifier, and the validation of our model. We end this section by highlighting the results of the classification approach.
In order to build a classifier, we have to first identify potential signals (metrics), then select a classification approach, and finally implement, train and evaluate the classifier. A crucial part of training and evaluating a classifier is obtaining a standard data set which contains information about whether or not a code review comment is useful, a so called oracle. Relying on the 145 comments rated during the interviews is not sufficient as that data set is too small to build a reliable classifier. To rectify this, we manually analyzed and separately validated an additional 844 comments to enhance the oracle. In the following, we describe the manual classification, the oracle generation process, the selected signals and their calculation, the classifier, and the validation of our model. We end this section by highlighting the results of the classification approach.
การแปล กรุณารอสักครู่..
