HDR scores correlate only weakly with reading comprehension (α = 0.36 and 0.41), indicating the task measures practical application ability rather than reading skill¶

Claim · All claims
Evidence · 1 study · design r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q3 i? Cronbach's α between HDR problems and a reading-comprehension control problem was low (0.36 for the original, 0.41 for the ChatGPT-generated problem), indicating HDR measures an ability unrelated to simple reading comprehension. → Satoshi Takahashi 2026

Evidence¶

Satoshi Takahashi 2026¶

Satoshi Takahashi, Atsushi Yoshikawa, Megumi Kose, Kenichi Suzuki, Chieko Inoue, Yumi Watanabe, and Mari Sawada. (2026). Automating Constructive Assessment with Large Language Models: Toward Scalable and Repeated Evaluation of Practical Competence. arXiv. https://arxiv.org/abs/2609.25790 doi:10.48550/arxiv.2609.25790

q3 · i? · design · r2

Experiment 1 internal-consistency analysis including a reading-comprehension control problem (a Tokyo metropolitan high-school entrance exam task); the discussion reports "the low correlation with the reading-comprehension problem (α=0.36,0.41)".

"the low correlation with the reading-comprehension problem (α=0.36,0.41) indicates that HDR measures practical application ability rather than superficial reading comprehension."

Discussion¶

Learner Variables¶

• Reading and Language — predictor: learners who differ on it differ in outcomes

Related Claims¶

• ChatGPT-generated HDR problems measure the same ability as human-created problems, with high internal consistency (Cronbach's α = 0.78) and no significant difficulty difference — related
• Automated HDR problem generation makes repeated and longitudinal assessment feasible by producing structurally equivalent cases with different contexts — related