To achieve this goal, we conducted a three-stage empirical study of code review usefulness at Microsoft. In the first stage, we conducted an exploratory study in which we interviewed developers to understand their perception of what “useful” means in the context of code review feedback. In the second stage, we used our findings from the first stage to build and verify an automated classifier able to distinguish between useful and not useful review comments. Finally, we applied the classifier to ∼1.5 million code review comments from five Microsoft projects, distinguishing comments that were useful from those that were not. Using this large-scale dataset, we quantitatively investigated factors that are associated with useful review comments. The primary contributions of this study are:
• An exploratory qualitative analysis of authors’ perceptions on useful code review comments.