METHODOLOGY
Product Coding: The pupils’ products were graded in four categories of success:
(1) No access, when they showed no signs of understanding the task properly or did not work on it meaningfully. (2) Basic access, when the pupils mainly understood the
problem and showed basic approach. (3) Advanced access, when they understood the
problem properly and solved it for the most part. And (4) full access, when the pupils
solved the task properly and presented appropriate reasons, if necessary.
This grading system was customized for each task with examples for each category.
Then, all the products were rated independently by the author and a research assistant.
After calculating Cohen’s kappa (κ > 0.85 for each task), we discussed and recoded the
few products with differing ratings, reaching consensus every time.