Construct Validity
Construct validity pertains to the meaningfulness and appropriateness of the test score interpretations. Both construct definition and the characteristics of the test task should be considered when we are evaluating construct validity of a score interpretation (Bachman & Palmer, 1996). In writing section, test takers’ abilities of choosing vocabulary correctly, building grammatical structure, spelling words and using punctuation accurately, and expressing information in an organized manner are tested. Those abilities are believed to be main factors of achieving successfully writing in college and university settings. The scoring procedures of TOEFL iBT writing also reflect the construct definition. ETS (2008b) claims that test takers are not expected to produce a perfect essay. They can earn high score with a response that contains some errors.
However, the test rubrics may not match to the construct definition. Lee and Kantor (2005) point out that each of the writing tasks measures an aspect of writing, thus separate scores should be reported for each of these aspects. Furthermore, there are several possible sources of bias which lie in the task characteristics. Firstly, the writing section contains only two writing tasks. Thus, it is extremely important to generalize the writing scores across tasks and task types to ensure the test validity (Lee & Kantor, 2005). Secondly, the tasks might be appropriate for undergraduate students. But for graduate students, even though they would not have the language ability to deal with certain topics, they might have the language ability needed to succeed in their particular fields. Cumming et al. (2005) suggest that test takers should have more choices about what to write based on their individual preferences.