In addition, to help students recall what they really did for determination and
consolidation of the meaning of the new words, they were asked to bring their ESP
books in their interview sessions and to illustrate what they report by mentioning
cases from their books.
To account for interrater reliability in coding the data derived from observations and
interviews, two steps were taken. The first step dealt with the segmentation of data
and the creating of coding categories. Because segmentation of data and the creation
of the coding categories are judgmental decisions (Gass & Mackey 2000, p.102), the
researcher felt data needed to be rated by more than one person to provide for higher
levels of validity in the study. The second step involved a third rater and dealt with
using the coding scheme and verifying the coding categories.
With regard to the questionnaire, its final version which was used in the main study
was constructed by the researcher based on theoretical considerations of some
previous attempts to study VLSs (e.g., Schmitt 1997, Kudo 1999, Winke 2002, Segler
2002) and the piloted students’ responses to questions in Schmitt’s VLSs
questionnaire and their answers in interviews. The final version of the questionnaire
was used in a pilot group. As for reliability, the result turned out to be satisfying
(Cronbach alpha= 0.82 questionnaire for specialized vocabulary and alpha=0.84
questionnaire for non-specialized vocabulary). It was also read critically by some
experts in applied linguistics to clarify its possible problems. The questionnaire was
constructed for the collection of data on what the participants actually do while
comprehending and learning the vocabulary items in their ESP texts and it contained
62 items.
To check for the validity of the questionnaire, factor analysis with varimax rotation
was run. Factor analysis was attempted specifying the number of factors as two with
the hope of showing that the 62 strategies fit into the two main tentative factors as
originally hypothesized (i.e., discovery and consolidation strategies). The KMO was
0.607 which was satisfactory.