The inclusion in paper and pencil tests of items that present verbal descriptions
of realistic problem situations has several attractions to test constructors. It
demonstrates that objective tests are not limited to testing for recall of isolated,
trivial factual detail s. Situation-based items cannot be answered correctly by
simple recognition of the right answer. They force the examinee to think. They
obviously require the application of knowledge to real-life problems. Realism in
the test encourages faith in the validity of the test scores. These are valuable
assets. But situation-based items also have disadvantages. They tend to be complex
and wordy. Complexi ty may obscure the crucial element in the situation ,
complicate the task of the examinee, and thus lower the discriminating power of
the items. It is true that the real problems we face in living are complex.
Unfortunately, complex, real problems seldom have single demonstrably correct
right answers. Giving a person a complex problem to solve may not be the best
way to estimate that person's capability of solving such problems.
Ordinarily a complex test question contributes only a single unit to the total
test score. It is answered correctly (l) or incorrectly (0) . But to arrive at the final
answer to a complex question , the onl y answer that counts, the examinee must
provide himself with a multitude of intermediate or contributory answers that do
not count. To reach a correct answer, each of a number of contributory steps
must be taken correctly. A single error in anyone of them may lead to a final
answer that is wrong. The value of nine correct decisions can be offset by the
penalty for one that is incorrect. Should not right and wrong decisions carry more
nearly equal weight in judging an examinee's capabilities? Would it not be more
reasonable , would it not be more informative, would it not lead to more accurate
measurement of the mental ability being tested to assess the correctness of each
step independently?