What Are the Advantages of Using ECD?
Though ECD has many advantages in other situations, it is least
useful in the maintenance of established, ongoing testing programs
in which the primary work is writing new tasks that are similar to
the existing tasks and assembling new forms of the test that are
parallel to existing forms. Many of the most useful aspects of ECD
become irrelevant because the decisions they are designed to
facilitate have already been made for the initial forms of the test. As
long as those initial forms are simply being replicated as closely as
possible in parallel forms, the machinery of ECD will bring few
improvements.
ECD becomes more helpful for redesigning tests. The fewer
constraints there are on the changes that can be made, the more
helpful ECD becomes. ECD is even more useful for making new tests
of previously measured constructs, and is most useful for measuring
new constructs. In fact, the less experience test developers have
measuring some domain, the more useful ECD becomes because it
helps to ensure that test developers will seek the information they
require about the domain to be tested, will clearly specify the claims
to be made about test takers, will determine the evidence required
to back the claims, will develop tasks that provide the desired
evidence, and will score them appropriately.
A primary advantage of ECD is that it helps to build in validity
during the test design and development process. The Standards for
Educational and Psychological Testing (American Educational Research
Association, American Psychological Association, & National Council
on Measurement in Education, 2014) calls for a “validity argument”
supporting the appropriateness of the inferences to be made on the
basis of the assessment results. ECD provides a strong foundation for
the validity argument by requiring documented, explicit linkages
among the purpose for a test, the claims made about test takers, the
evidence supporting those claims and the test takers’ responses to
tasks that provide the evidence. ECD helps ensure that tasks are
measuring construct-relevant KSAs and makes it easier to determine
if tasks are inadvertently measuring construct-irrelevant KSAs. Note
that even if ECD is used, the gathering of validation evidence based
on such factors as expert judgments of task content, the empirical
relationships among parts of the test, and the empirical relationships
of test scores with external variables is still required.