Scorers had difficulty distinguishing rubric scoring from everyday grading and AP scoring, and training took longer than expected¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2021 · checked against its sources 2026-10-09 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Scorers had difficulty distinguishing the assignment and work scoring process from everyday grading of student work and from AP scoring, at least initially, and training and calibration took longer than expected. → Iwatani 2021
Evidence¶
Iwatani 2021¶
Iwatani, E., Means, B., Seylar, J., and Hardy, A. (2021). Rubrics on historical thinking skills for assignments and student work: Initial validity evidence. Digital Promise. https://doi.org/10.51388/20.500.12265/112
q2 · i? · design · r2
Observations from the pilot scoring session with three expert world history teachers, reported in the key findings. The article states "TUaining and calibUaWiRn WRRk lRngeU Whan Ze e[SecWed", suggesting the need for stronger, more targeted training; scorers were also reluctant to assign the low scores the rubrics required for good work or lessons.
"ScRUeUV had difficXlW\ diVWingXiVhing Whe aVVignmenW and ZRUk VcRUing SURceVV fURm eYeU\da\ gUading Rf VWXdenW ZRUk, and fURm AP VcRUing, aW leaVW iniWiall. TUaining and calibUaWiRn WRRk lRngeU Whan Ze e[SecWed, VXggeVWing Whe need fRU VWURngeU, mRUe WaUgeWed WUaining."
Discussion¶
Related Claims¶
- Trained scorers applied the historical thinking skills rubrics with good overall inter-rater consistency — related
- Scorers found all rubric dimensions important, with feedback bearing mainly on the scoring and training process rather than rubric design — related
- Scorer severity and standard difficulty were statistically adjusted using Many-Facet Rasch analysis before assignment and work measures were compared — related
- World History Project assignments scored higher on most learning opportunity rubric dimensions than other assignments — related
- Rubric scoring supports answering diagnostic questions about opportunities for and evidence of historical thinking — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
scorer-training-longer-than-expected-grading-confusion |
title |
Scorers had difficulty distinguishing rubric scoring from everyday grading and AP scoring, and training took longer than expected |
description |
Scorers had difficulty distinguishing rubric scoring from everyday grading and AP scoring, and training took longer than expected |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2021 |
source_checked |
2026-10-09 |
generated |
by process:wiki-ingest on 2026-10-09 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — iwatani-2021 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.