Reliability of the Assessment Process
Interratcr reliabilities vary across studies from a median of about .60 to over .95 (Adams & Thornton, 1989; Schmitt, 1977). Raters tend to appraise similar aspects of performance in candidates. In terms of temporal stability, an important question concerns the extent to which dimension ratings made by individual assessors change over time (i.e., in the course of a six-month assignment as an assessor). Evidence on this issue was provided by Sackett and Hakel (1979) as a result of a large-scale study of 719 individuals assessed by four assessor teams at AT&T. Mean interrater reliabilities across teams varied from .53 to .86, with an overall mean of .69. In addition to generally high stability, there was no evidence for consistent changes in assessors’ or assessor teams’ patterns of ratings over time.