Additional Results. We did not insist that assessments be identical or formally
equivalent if studies reported only data on failure rates. To evaluate the
hypothesis that differences in failure rates recorded under traditional lecturing
and active learning were due to changes in the difficulty of examinations
and other course assessments, we evaluated 11 studies where failure
rate data were based on comparisons in which most or all examination
questions were identical. The average odds ratio for these 11 studies was 1.97 ±
0.36 (SE)—almost exactly the effect size calculated from the entire dataset.
Although we did not metaanalyze the data using “vote-counting”
approaches, it is informative to note that of the studies reporting statistical
tests of examination score data, 94 reported significant gains under active
learning whereas only 41 did not (Table S4A).
Additional results from the analyses on publication bias are reported in
Supporting Information.