The present study compared the difference between CTT and IRT approach across
samples and test forms in chemistry. The difference is demonstrated on estimates of item difficulty,
internal consistencies, and standard errors. It was found in the study that (1) IRT estimates of item
difficulty do not change across samples as compared with CTT with inconsistencies; (2) difficulty
indices were also more stable across forms of tests than the CTT approach; (3) IRT internal
consistencies are very stable across samples while CTT internal consistencies failed to be stable
across samples; (4) IRT had significantly less measurement errors than the CTT approach. These
findings further support the marked difference between the CTT and IRT approaches pertaining to
sampling and tests. Aside from demonstrating differences between IRT and CTT, the findings are
helpful for measurement experts to decide on what approach to use in analyzing test data.