In the first week of the third stage, the
academic achievement test is administered to the students
as the post-test. During the 3 weeks following the
post-test, students in both groups are asked to revise
and improve their pre-projects in accordance with the
new knowledge they have acquired throughout the
course. At the end of the 14th week, the post-projects of
the two groups are collected.projects were used to determine the students’ transfer
of learning and used only for research purposes. These
grades were not used in their actual education. Cohen’s
kappa is used as the inter-rater reliability coefficient. In
the study, the coefficients range from 0.83 to 0.93
(p < .001) for each criterion of the multimedia principles
rubric, and from 0.77 to 0.90 (p < .001) for each
criterion of the visual design principles rubric.Akappa
value above 0.60 is considered to indicate ‘substantial
agreement’ and a value above 0.80 is considered
‘almost perfect agreement’ (Landis & Koch, 1977).
Consequently, it is clear that inter-rater reliability is
high for this study.