In order to examine the students’ background information and determine a reference criterion, their TOEIC score averages are first presented here. Following that, the results of the two practice tests and the Part 5 results of the pre-test and post-test are presented.
4.1. Students’ Past TOIEC Scores
As mentioned in the methodology section, the students taking this intensive course tend to be on the lower end of the scale due to the nature of the course scheduling. The group average of the total TOEIC scores from the past three IP TOEIC tests held prior to this intensive course is 363.1, which is considerably lower than the department average score of 425. 2 The lowest score from the group is 185, and the highest 540 (STDV=102.8). The average listening score of the group is 234.1 (Lowest=107.5, Highest= 347.5, STDV=67.5; cf., Department Average=263.3). The average reading score of the group is 129.0 (Lowest=77.0, Highest=192.5, STDV=37.8; cf., Department Average=161.7). See Table 1. Graph 1and Graph 2 summarizes the distributions of the TOEIC scores of the students taking this intensive course.
Graph 1 shows the total scores and Graph 2 the reading scores.
Even though the overall level of the students taking the intensive course is low, four out of the 15 stu dents scored above the department average of 425.0 for the total score. For the reading score, three out of the 15 scored above the department average of 161.7. The top three students have both the top three total and reading scores. Individual students’ reading scores are used as a reference in the following sections, which examine the students’ performances on the timed practice tests, pre-test, and post-test.
4.2. Results of the Two Practice Tests during Review Sessions
After studying the practice test questions (40 Part 5 questions, 12 Part 6 questions, 48 Part 7 questions), the students worked on the same practice test in a timed, test-like fashion. Upon completion of the timed practice test, the students went over the answers, reviewed the questions/answers one more time, and reported their scores to the instructor. They repeated the same procedure for both Practice Test #1 and Practice Test #2. The students’ TOEIC reading scores and the Practice Tests scores are shown in Table 2.
Even though these practice tests were conducted as the fifth step described in the methodology section, the rate of correct answers was still not very high (Practice Test #1=56.9%, STDV=13.98; Practice Test #2=57.4%, STDV=15.51). Because the students only reported the total scores of the practice tests, their scores on each part were not available. However, there seem to be general tendencies that the students with higher TOEIC reading scores have higher accuracy rates when they worked on the practice test questions (Graph 3). The solid line shows the approximation of Practice Test 1 results, and the dotted line the approximation of Practice Test 2.
It is, of course, very possible that the students with higher reading scores did better even before they studied and reviewed the practice test questions. How much better they did on the timed practice tests after the review cannot be determined from these test scores alone; therefore, the following section compares the Part 5 results of the pre-test and the post-test. It investigates the students’ performance before and after the grammar review and repetition practices.
4.3. Part 5 Results of the Pre-Test and the Post-Test
The pre-test included 40 Part 5 grammar questions following the TOEIC format, and the same questions were used in the post-test. As explained earlier, out of the 40 questions, 20 questions are repeated and the other 20 questions are new Overall, the students’ scores increased by 23.5% (=Post-test Average of 57.2% ― Pre-test Average of 33.7%) after the review sessions. See Table 3 for individual student’ scores of the pre-test and the post-test. The students’ TOEIC reading scores are used again as a criterion.
The average rate of the correct answers for the pre-test is 33.7%. Because this pre-test was administered at the very beginning of the intensive course, all 40 questions were new and unfamiliar to the students at that point. The following Graph 4 shows the relation between the students’ TOEIC reading scores and the percentages of the Part 5 correct answers in the pre-test. The solid line is an approximation of the results.
The results thus show that, even though there may be a general tendency that the students with higher TOEIC reading scores did slightly better than those with lower scores, over all, students’ TOEIC scores seem to have little correlation to how well they did on the pre-test grammar questions. On the other hand, the post-test results indicate a clearer correlation with the students’ TOEIC reading scores (Graph 5). The students with higher TOEIC reading scores do much better than those with lower scores. When we compare the results of the two tests, the differences between the pre-test and the post-test as well as the correlation with the students’ TOEIC reading scores become clear. The ones with higher TOEIC reading scores clearly demonstrate higher accuracy rates in the post-test, suggesting that they perform better after working on practice tests and reviewing the practice test questions (Graph 6).
Out of the 40 Part 5 questions, the increase in the number of correct answers from the pre-test to the post-test is 9.4 on average. Graphs 7 and 8 show each student’s increase in the number of correct answers. The students with higher TOEIC reading scores had a greater increase in the number of correct answers than those with lower scores.
Because the Part 5 questions consist of both the repeated questions and the new questions, this next section looks at the results of the repeated and new questions separately. For the 20 repeated questions, the students increased the number of correct answers on the post-test by 7.3. The analysis of the repeated questions indicates that the students with higher TOEIC reading scores tend to obtain a higher number of correct answers (Graph 9). For the 20 new questions, the students’ increase in the number of correct answers is only 1.9. No differences are observable between the students with higher TOEIC reading scores and those with lower scores (Graph 10.) In fact, the two students with the highest TOEIC reading scores in the group (192.5 and 187.5, both of which are higher than the department average) demonstrated zero increase for the new questions. Two students with lower TOEIC reading scores (97.5 and 101.7) showed negative gains on the new questions. No obvious contributing factors, therefore, are identifiable for the increase/ decrease in the number of correct answers.
Graph 11 below combines Graph 9 and Graph 10, showing the students’ TOEIC reading scores and the differences of the results for the repeated questions and for the new questions.
Additionally, the increases in the number of correct answers for the repeated questions and for the new questions are compared in Graph 12. There seems no correlation between the two, and those who had a greater increase on the repeated questions did not do any better on the new questions. Therefore, for the new questions no apparent differences are identifiable between the students with lower TOEIC reading scores and those with higher scores in spite of the fact that the students with higher TOEIC reading scores clearly perform better on the repeated questions.
According to the results shown above, TOEIC grammar review using practice test questions does not look to be very effective or helpful for low-level learners even with the implementation of repetitive reviews of the questions and answers. The students with higher TOEIC reading scores may do better in understanding and retaining the information presented in the practice test questions; however, this does not necessarily mean they learn grammar better because in the post-test they were not able to apply grammar rules to the new questions. Reviewing mixed grammar points in practice tests is difficult for these learners even with repetitive review and practice. The use of practice tests should be reevaluated when it comes to teaching low-level TOEIC preparatory classes.