Choose CTT or MFRM for open-ended test development based on practicality and reported detail, not difficulty results¶

Strategy · All strategies
Evidence · no claims cited

Description¶

Because the two theories yield similar item difficulty indices for open-ended questions, the article recommends that decisions between them should prioritize "other proper ties, such as ease of use and the comprehensiveness of the reported results". CTT is more familiar and its analyses run easily in Microsoft Excel, while MFRM synchronically computes validity, reliability, item difficulties, abilities, and rater severity, and outputs test information functions, category statistics, and unexpected responses that have no CTT counterparts.

Design Implications¶

Context¶

Requirements¶

• Researchers/practitioners must be able to judge which practical features (ease of use vs. comprehensiveness of outputs) matter for their test development context

Constraints¶

• The recommendation rests on difficulty-index agreement shown with one dataset; the authors note generalizability requires studies with different data sets

Target Learners¶

• test developers and practitioners developing achievement tests with open-ended items

Target Learning Goals¶

• item selection and difficulty estimation in open-ended achievement test development

Related Strategies¶

Examples¶

-

Key Sources¶

• Ilhan, M., & Guler, N. (2018). A Comparison of Difficulty Indices Calculated for Open-Ended Items According to Classical Test Theory and Many Facet Rasch Model. Eurasian Journal of Educational Research, 75. doi:10.14689/ejer.2018.75.6