Abstract In order to study the effect of direct test preparation on TOEIC gain scores, two samples of students (i.e., English majors and non-majors) at a Japanese university were divided into three treatment groups: 1) TOEIC Preparation, 2) Business English and 3) General (four-skills) English. The results indicate that usage of TOEIC preparatory materials led to a statistically significant gain on post-test scores for the non-majors’ reading component only. The authors conclude that TOEIC preparatory materials are of little benefit to students enrolled in a comprehensive program of English language study, but might boost the score of the reading component of students enrolled in a university-level general English course in Japan.