Data Coding
The coding process started with identifying interaction features
from the studies. As mentioned in the previous section, 15 studies were
utilized for this purpose. Each interaction feature that occurred in all
15 studies was carefully marked and compiled into a general list based
on this examination. Then features on this list were reviewed again
to ensure that the interactive features labeled differently in different
studies were not essentially the same by nature. Lastly, features were
classified into different categories in order to be reviewed and analyzed
in a more organized manner (see Appendix A for the list of interaction
features).
Once all interaction features were identified and placed into the
list, the nine primary studies were coded for variables including study
information, context, methods, independent variables, and study
outcomes using a coding scheme (see Appendix B). This coding scheme
was designed to capture study effects on oral interaction in English
testing and potential moderators.
The first coder is the researcher, who coded all studies. A second
coder, who is a Ph.D. student in Applied Linguistics, was trained and
coded a subset of 4 studies (i.e., 45% of the total studies) using the
coding scheme. The two coders compared and discussed their decisions
for each coded study. They were able to reach consensus on a majority
of the coding decisions. The inter-coder agreement was calculated
at 91% (i.e., the number of agreements divided by the total number
of coding decisions). Disagreement was solved through a discussion
between the two coders.