CSEBench items show adequate psychometric quality, with satisfactory difficulty, discrimination, and reliability for both diagnostic forms¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q3 i? The whole item bank shows mean item difficulty 0.486 and mean discrimination index 0.324, indicating appropriate difficulty and satisfactory discriminative power. → Shangchao Min 2026 q3 i? Test reliability was 0.74 for the college section and 0.82 for the junior high section, both exceeding the 0.70 threshold. → Shangchao Min 2026 (2)

Evidence¶

Shangchao Min 2026¶

Shangchao Min, Shaonan Wang, Xinyu Gao, Hui Wang, Zhiling Jin, and 2 more. (2026). Standardized Assessment of LLM English Proficiency. https://doi.org/10.21203/rs.3.rs-8820245/v1

q3 · i? · design · r2

Item quality analysis of the 624-item bank using classical item indices. The article reports "the mean item difficulty was 0.486 and the mean discrimination index was 0.324", which it interprets against Ebel and Frisbie's criteria as appropriate difficulty and satisfactory discriminative power.

"For the whole item bank (CSEBench), the mean item difficulty was 0.486 and the mean discrimination index was 0.324."

Shangchao Min 2026 (2)¶

Shangchao Min, Shaonan Wang, Xinyu Gao, Hui Wang, Zhiling Jin, and 2 more. (2026). Standardized Assessment of LLM English Proficiency. https://doi.org/10.21203/rs.3.rs-8820245/v1

q3 · i? · design · r2

Reliability analysis of the two large-scale diagnostic forms. The article reports Cronbach's α of "0.74 for the college section and 0.82 for the junior high section", both exceeding the commonly accepted 0.70 threshold for educational assessments.

"Test reliability, as measured by Cronbach's α, was 0.74 for the college section and 0.82 for the junior high section."

Discussion¶

Related Claims¶

• Benchmark items showed high mean difficulty and low item discrimination, with many trivial items and only one broken item — related
• Spanish Reading items show gradually increasing mean p values and stable discrimination with no negatively discriminating items — related
• The ten-construct instrument showed strong psychometric quality, with the four ARCS dimensions forming a psychometrically sound higher-order motivation construct — related
• Negative affective experience items were dropped due to floor effects and poor discrimination — related
• CTAT shows robust psychometric properties for higher education CS/AI students, with reliable internal consistency and adequate IRT item parameters — related