Abstract When we rely upon gains on some measure to support statements of prescription,
we have the obligation to ensure that those measures are valid. Nearly 10 years after an
influential National Research Council (2001) report on educational assessment identified an
explicit model of cognition as one of three necessary components of a valid assessment
system, we note that most measures still lack this fundamental cornerstone. In this paper,
we draw attention to the construct modeling approach to assessment that strives for
coherence and consistency with a model of cognition in which student proficiency varies
along a continuum of competence. This approach is illustrated in the context of an
assessment of conceptual understanding of certain scientific phenomena given to undergraduates
at a large public university (National Research Council 2001).
Keywords Cognition . Assessment . Validity . Construct modeling
When we rely upon gains on some measure to support statements of prescription, we have
the obligation to ensure that those measures are valid. The National Research Council
(2001) report Knowing What Students Know described three necessary components of a
valid assessment system: “a model of student cognition and learning in the domain, a set of
beliefs about the kinds of observations that will provide evidence of students’
competencies, and an interpretation process for making sense of the evidence” (p. 44).
We note that 10 years later, most measures still lack the first leg of the stool: an explicit
model of cognition.