Summed scoring and IRT pattern scoring produce very similar scale score means and failing rates within any equating procedure¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2017 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2020: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across 45 administrations, score means and percentages of failing examinees were very similar between summed and pattern scoring regardless of whether chain equating, simultaneous equating, or concurrent calibration was used. → Jungnam Kim 2017

Evidence¶

Jungnam Kim 2017¶

Jungnam Kim, Furong Gao, & Dong-In Kim. (2017). Robust IRT Scaling: Considerations in constructing item bank from tests across years. Paper presented at the annual meeting of the National Council on Measurement in Education, San Antonio, TX. https://www.nwea.org/research/publication/robust-irt-scaling-considerations-in-constructing-item-bank-from-tests-across-years/

q2 · i? · design · r2

Comparative analysis of scale score means (Figure 4) and failing percentages (Figure 5) for all 45 tests from 1992F to 2014F under six equating-by-scoring combinations; the authors report the score means were "very similar between the two scoring methods regardless of equating procedure."

"The comparison of the solid and dotted lines of a same color indicated that the score means were very similar between the two scoring methods regardless of equating procedure."

Discussion¶

Related Claims¶

• Concurrent calibration produced higher and less fluctuating failing rates than the separate-calibration equating transformations, with larger differences for older tests — related
• Concurrent calibration yielded consistently lower and less fluctuating scale score means than separate calibration with chain or simultaneous equating, especially for older tests — related
• Item parameter differences between chain and simultaneous equating were small (at most 12 items, 7%, flagged by RMSD), while comparisons involving concurrent calibration flagged up to 11% of items — related
• The number of flagged items was unrelated to equating order, and flagged items showed no meaningful content-property pattern — related