The linguistic analysis found in the Brown et al. (2005) study sought, among other
things, to determine whether there were differences in performance across proficiency
levels and task types. To do so, a number of linguistic analyses were conducted to explore
the production of features identified in the rater-cognition study (linguistic resources,
content, phonology, and fluency). For these analyses, the Computerized Language
Analysis (CLAN) program (MacWhinney, 2000) was used in addition to phonetic analyses
conducted by a phonetician. The results of ANOVA statistical tests indicated that
although there were a number of differences between independent and integrated tasks,
only four measures achieved an effect size that was larger than “marginal”. These effects
demonstrated that responses to independent tasks included more words from the 1K list
(the 1000 most frequent words found in a corpus of English), a higher number of modal
verbs, more word-types, and a longer mean length of run (the average number of syllables
per utterance) than responses to integrated tasks. Based on these findings, the authors
concluded that “there appears to be little evidence to support the view that integrated task
performance will be more linguistically complex or sophisticated than on independent
tasks” (pp. 105–106). One must be cautious, however, when attempting to generalize the
results of Brown et al. (2005) to operational TOEFL speaking tasks, because only one of
the prototype integrated tasks (monologic listen/speak task) is similar to the operational
tasks currently found in the TOEFL iBT