CHALLENGES IN CRITERION DEVELOPMENT
Competent criterion research is one of the most pressing needs of personnel psychology today as
it has been in the past. About 70 years ago, Stuit and Wilson (1946) demonstrated that
continuing attention to the development of better performance measures results in better
predictions of performance. The validity of these results has not been dulled by time
(Viswesvaran & Ones, 2000). In this section, therefore, we will consider three'types of
challenges faced in the development of criteria point out potential pitfalls in criterion research,and sketch a logical scheme for criterion development
At the outset, it is important to set certain "chronological priorities." First, criteria must be developed and analyzed, for only then can predictors be constructed or selected to predict
relevant criteria. Far too often, unfortunately, predictors are selected carefully, followed by a hasty search for "predictable criteria." To be sure, if we switch criteria, the validities of the predictors will change, but the reverse is hardly true. Pushing the argument to its logical extreme, if we use predictors with no criteria, we will never know whether or not we are selecting those individuals who are most likely to succeed. Observe the chronological priorities! At least in this process we know that the chicken comes first and then the egg follows.
Before human performance can be studied and better understood, four basic challenges
must be addressed (Ronan & Prien, 1966, l97l). These are the issues of (un)reliability of
performance, reliability of performance observation, dimensionality of performance, and
modification of performance by situational characteristics. Let us consider the first three in turn;the fourth is the focus of a later section.