The SA team also provides summative information about the number of forms rated by each rater,
the number of responses rated by each rater, the number of flagged forms and flagged responses
for each rater, and the total number of flags. The AD staff keeps track of raters who are flagged more
often or on more forms, and then attempt to determine the reason for the discrepant performance
in order to provide extra training for these raters.