2. Coding of LAs’ impressions of the simulator
Codes emerged from students’ responses related to the
simulator’s usefulness for practicing pedagogical skills
in a college physics environment in terms of its similarities
and differences from their real-life teaching experiences
[see Fig. 2(e) for complete prompt]. Students were then
classified as finding the simulator “realistic” if they had
only advantage codes, as “nonrealistic” if they had only
disadvantage codes, and as “mixed” if they had codes of
both types. One researcher pulled common phrases from
students’ responses, which two researchers independently
applied to the response set. The initial Cohen’s kappa was
0.72. The two researchers discussed and revised the code
descriptions and reanalyzed the data, with a resulting
Cohen’s kappa of 0.85. After further discussion, all disagreements
were resolved. Finally, an external rater analyzed
the data, again resulting in Cohen’s kappa of 0.85
between the final set (presented here) and the external
rater. Importantly, the external rater only disagreed on one
LA-level classification.