Screening assessments vary in design, administration, and risk definitions such that there is no truly common definition of risk across assessments¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2023 · not yet checked against its sources · not reviewed by a person · recheck by 2026: tool and model accuracy changes quickly

Subclaims¶

q2 i? Approved screening assessments differ in content, cut-score derivation, and number of risk levels, so the meaning of risk differs across assessments. → Mariann Lemke 2023

Evidence¶

Mariann Lemke 2023¶

Mariann Lemke, Dan Murphy, Aaron Soo Ping Chow, Hayley Spencer, and Angela Zhang. (2023). A First Look at Early Literacy Performance in Massachusetts: Results of Initial Analysis Based on State Grantee Literacy Screening Assessments. WestEd. https://eric.ed.gov/?id=ED638580

q2 · i? · associational · r2

Documentary review of the risk-level and cut-score definitions of the 11 assessments included in analysis (Table 2), finding benchmarks are set variously from predictive data, normative percentiles, or both; the report concludes there is "no truly common definition of risk across screening assessments."

"These differences mean there is no truly common definition of risk across screening assessments."

Discussion¶

Related Claims¶

• Literacy screener benchmarks predict which grade 3 students will not meet MCAS ELA expectations, but not necessarily who will meet them — the general claim this finding bears on, with the studies behind it
• Agreement between screening assessments on which students are at risk varies from 47 to 85 percent depending on assessment combination — related
• Early literacy screening assessments provide multiple benchmarks or performance levels defined in different ways, with no common definition of 'at risk' — possibly the same claim (merge candidate)
• Equipercentile linking shows grade 3 screening benchmark cut scores identifying at-risk students cluster in the MCAS Partially Meeting Expectations performance level, with variation among assessments — related
• Screening assessment benchmarks indicating significant risk shift up and down on the MCAS scale within and across grades, so students with the same skills may be classified differently at different times — related
• Updated 2025 MAP Growth screening thresholds are set at the 35th percentile with adjustments for the Enhanced Item-Selection Algorithm — related
• Statistical item selection that maximizes norm-referenced discrimination undermines criterion-referenced test definitions — related
• Definitions of developmental dyscalculia vary widely across studies, with diagnostic cutoffs ranging from below the 3rd to below the 25th percentile — a narrower finding that bears on this claim
• Districts defined pilot success differently, making it difficult to set a standard for evaluating product effects on student improvement — related
• Grade 3 screener benchmark scores marking any level of risk map to MCAS ELA scores from mid Partially Meeting Expectations to the beginning of Meeting Expectations — related