Agreement between K–3 screening assessments on risk classification varied from 47 percent to 85 percent depending on the assessment combination¶

Claim · All claims
Evidence · 2 studies · 2 associational r1–r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Rates of agreement between screening assessments on whether students would be considered at risk varied from 47 percent to 85 percent depending on the screening assessment combination. → Early Literacy Performance in Massachusetts: Key Findings From 2023/24 School Year Data 2025

q2 i? Rates of agreement between screening assessments on whether students would be considered at risk of reading difficulty varied from 47 percent to 85 percent depending on the screening assessment combination. → Lemke 2025

Evidence¶

Early Literacy Performance in Massachusetts: Key Findings From 2023/24 School Year Data 2025¶

Early Literacy Performance in Massachusetts: Key Findings From 2023/24 School Year Data. (2025). WestEd. https://eric.ed.gov/?id=ED677570

q2 · i? · associational · r2

Analysis of 2023/24 screening records, in which about 3 to 4 percent of students in any time period took more than one screening assessment. The brief reports agreement rates "varied from 47 percent to 85 percent depending on screening assessment combinations."

"Rates of agreement between screening assessments—that is, whether students would be considered at risk of reading difficulty on both assessments—varied from 47 percent to 85 percent depending on screening assessment combinations."

Lemke 2025¶

Lemke, M., Faulkner-Bond, M., LeBeau, B., Soo Ping Chow, A., & Quesen, S. (2025). Early literacy performance in Massachusetts: Results of ongoing analysis of literacy screening assessments (2023/24 data). WestEd. https://eric.ed.gov/?id=ED677571

q2 · i? · associational · r1

Analysis of the roughly 3 to 4 percent of students per period who took more than one screening assessment, comparing risk classifications across assessment combinations. The report prints agreement rates varying "from 47 percent to 85 percent"; no effect size is printed.

"Rates of agreement between screening assessments— that is, whether students would be considered at risk of reading difficulty— varied from 47 percent to 85 percent depending on screening assessment combinations."

Discussion¶

Related Claims¶

• Literacy screener benchmarks predict which grade 3 students will not meet MCAS ELA expectations, but not necessarily who will meet them — the general claim this finding bears on, with the studies behind it
• MAP Reading Fluency SWCPM scores classify students into ARM proficiency categories with exact match rates of 74.6–90.8%, kappa 0.60–0.85, and overall accuracy 0.81–0.96 — related
• The predictive value of grade 3 screening benchmarks for MCAS ELA proficiency varies widely: 15 to 69 percent of students classified as on track did not reach MCAS proficiency, and students flagged at risk rarely met proficiency — related
• Screening assessments vary in design, administration, and risk definitions such that there is no truly common definition of risk across assessments — related
• Screening assessment benchmarks indicating significant risk shift up and down on the MCAS scale within and across grades, so students with the same skills may be classified differently at different times — related