IRT item bank built from 45 licensure exam forms administered 1992-2014 under a common-item nonequivalent groups design¶

Element · All elements
Evidence · 5 claims (5 for) · 1 study (1 design), q2 · 0 of 1 report an effect size · 5 claims rest on one study

Description¶

The study's artifact is an item bank constructed from a national licensure examination administered twice yearly (spring and fall) from 1992 through 2014. As the article states, "Total of 45 tests were equated to the bank, using a common-item nonequivalent groups design." Tests were organized into two blocks split at the 2009F blueprint change, dichotomous items were calibrated with the three-parameter logistic model and polytomous items with the graded response model, and the first block was scaled into the bank before the second block was added.

Design Implications¶

Context¶

Requirements¶

• Overlapping common items between tests to support equating transformations
• Adequate common items (at least 20) for simultaneous equating grouping

Constraints¶

• A single concurrent calibration failed to run for the second block, requiring three sub-blocks
• In CH and SM, common items whose parameter estimates differed by more than 2 RMSD were excluded from the anchor set

Target Learners¶

• examinees of a national licensure examination

Target Learning Goals¶

• maintaining comparable scale scores and passing decisions across 23 years of test administrations

Claims¶

• Concurrent calibration produced higher and less fluctuating failing rates than the separate-calibration equating transformations, with larger differences for older tests [+W]
• Concurrent calibration yielded consistently lower and less fluctuating scale score means than separate calibration with chain or simultaneous equating, especially for older tests [+W]
• The number of flagged items was unrelated to equating order, and flagged items showed no meaningful content-property pattern [+W]
• Item parameter differences between chain and simultaneous equating were small (at most 12 items, 7%, flagged by RMSD), while comparisons involving concurrent calibration flagged up to 11% of items [+W]
• Summed scoring and IRT pattern scoring produce very similar scale score means and failing rates within any equating procedure [+W]

Related Elements¶

Examples¶

-

Key Sources¶

• Jungnam Kim, Furong Gao, & Dong-In Kim. (2017). Robust IRT Scaling: Considerations in constructing item bank from tests across years. Paper presented at the annual meeting of the National Council on Measurement in Education, San Antonio, TX. https://www.nwea.org/research/publication/robust-irt-scaling-considerations-in-constructing-item-bank-from-tests-across-years/