Self-Other Agreement. The primary criterion of judgmental accuracy is self-other agreement: the extent to which an external judgment agrees with the target’s self-rating (17), usually operationalized as a Pearson product-moment correlation. Self-other agreement was determined by correlating participants’ scores with the judgments made by humans and computer models (Fig. 1). Since self-other agreement varies greatly with the length and context of the re- lationship (18, 19), we further compared our results with those previously published in a meta-analysis by Connely and Ones (20), including estimates for different categories of human judges: friends, spouses, family members, cohabitants, and work colleagues.
To account for the questionnaires’ measurement error, self- other agreement estimates were disattenuated using scales’ Cronbach’s α reliability coefficients. The measurement error of the computer model was assumed to be 0, resulting in the lower (conservative) estimates of self-other agreement for computer- based judgments. Also, disattenuation allowed for direct com- parisons of human self-other agreement with those reported by Connely and Ones (20), which followed the same procedure.