The general polytomous testlet model recovers true item parameters well in simulation, with average correlations of 0.9680, 0.8608, and 0.9982 for discrimination and difficulty parameters¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2010 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2013: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? In a simulation study with 2000 examinees per data set and 20 simulated data sets, the general polytomous testlet model showed good recovery of item discrimination and difficulty parameters. → Yanmei Li 2010
Evidence¶
Yanmei Li 2010¶
Yanmei Li, Shuhong Li, and Lin Wang. (2010). Application of a General Polytomous Testlet Model to the Reading Section of a Large-Scale English Language Assessment. ETS Research Report RR-10-21. http://www.ets.org/research/contact.html doi:10.1002/j.2333-8504.2010.tb02228.x
q2 · i? · design · r2
Simulation study mimicking the real test structure (3 passages, 14 items each, last item polytomously scored 0-2), 2000 examinees per data set, ten data sets under each of two testlet-effect conditions. The report states the "average correlation across the two conditions" for discrimination and difficulty parameters and calls the RMSD values "reasonably low".
"The average correlation across the two conditions for the discrimination parameters, 1ia and 2ia , and item difficulty parameters it were 0.9680, 0.8608, and 0.9982, respectively."
Discussion¶
Related Claims¶
- In simulation, ignoring local dependence overestimates reliability, and the overestimation grows with the size of the testlet effect (17.0%-21.8% decrease under the larger testlet effect vs. 4.8%-7.4% under the small) — related
- In operational reading-test data, item parameter estimates from a standard 2PL/GPCM that ignores local dependence closely match those from the testlet model (average correlations 0.9893 and 0.9978) — related
- A multidimensional IRT model with simple structure fits testlet-based reading data better than both the general polytomous testlet model and the standard 2PL/GPCM — related
- The GPE model's parameters are recoverable via Bayesian estimation in simulation — possibly the same claim (merge candidate)
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
polytomous-testlet-model-good-parameter-recovery |
title |
The general polytomous testlet model recovers true item parameters well in simulation, with average correlations of 0.9680, 0.8608, and 0.9982 for discrimination and difficulty parameters |
description |
The general polytomous testlet model recovers true item parameters well in simulation, with average correlations of 0.9680, 0.8608, and 0.9982 for discrimination and difficulty parameters |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2010 |
recheck_by |
2013 |
generated |
by process:wiki-ingest on 2026-09-27 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — yanmei-li-2010 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.