Concurrent calibration produced higher and less fluctuating failing rates than the separate-calibration equating transformations, with larger differences for older tests¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2017 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2020: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? The percentage of failing examinees under concurrent calibration was relatively less fluctuated and higher than under CH and SM, and differences between CR and the two equating transformations were larger for older tests than for middle or newer tests. → Jungnam Kim 2017
Evidence¶
Jungnam Kim 2017¶
Jungnam Kim, Furong Gao, & Dong-In Kim. (2017). Robust IRT Scaling: Considerations in constructing item bank from tests across years. Paper presented at the annual meeting of the National Council on Measurement in Education, San Antonio, TX. https://www.nwea.org/research/publication/robust-irt-scaling-considerations-in-constructing-item-bank-from-tests-across-years/
q2 · i? · design · r2
Figure 5 and Tables 2-3 failing-rate comparison across administrations. For 1992F summed scoring, %Fail was 10.54 under CH_S and SM_S but 24.44 under CR_S, consistent with the printed pattern of "higher percentage of failing examinees" under CR.
"CR produced relatively less fluctuation and higher percentage of failing examinees; larger differences were observed between CR and the two equating transformations for the older tests than the middle (i.e. tests for 2002F and 2009S) or the newer tests (i.e. 2009F and later tests)."
Discussion¶
Related Claims¶
- Concurrent calibration yielded consistently lower and less fluctuating scale score means than separate calibration with chain or simultaneous equating, especially for older tests — related
- Item parameter differences between chain and simultaneous equating were small (at most 12 items, 7%, flagged by RMSD), while comparisons involving concurrent calibration flagged up to 11% of items — related
- Summed scoring and IRT pattern scoring produce very similar scale score means and failing rates within any equating procedure — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
concurrent-calibration-higher-failing-rates |
title |
Concurrent calibration produced higher and less fluctuating failing rates than the separate-calibration equating transformations, with larger differences for older tests |
description |
Concurrent calibration produced higher and less fluctuating failing rates than the separate-calibration equating transformations, with larger differences for older tests |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2017 |
recheck_by |
2020 |
generated |
by process:wiki-ingest on 2026-10-08 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — jungnam-kim-2017 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.