Shallow complexity measures for word problems correlate only weakly with difficulty¶

Claim · All claims
Evidence · 1 study · associational r2 · q2
Currency · newest study 2022 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? For a diverse set of 1000 word problems, shallow, easily computable complexity measures correlate only very weakly with difficulty; the only larger correlation is between the length of the text and the response time. → Pelánek 2022

Evidence¶

Pelánek 2022¶

Pelánek, R., Effenberger, T., & Čechák, J. (2022). Complexity and Difficulty of Items in Learning Systems. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-021-00252-4

q2 · i? · associational · r2

The article reports an analysis of "a diverse set of 1000 word problems" showing that "the shallow, easily computable comp lexity measures correlate only very weak- ly with difficulty".

"The analysis showed that the shallow, easily computable comp lexity measures correlate only very weak- ly with difficulty. The only larger correla tion is between the length of the text and the response time"

Discussion¶

Related Claims¶

• Complexity measures for logic puzzles vary in correlation with difficulty across puzzle types — a broader claim this one bears on
• In the random forest, word count was the most important readability feature while traditional readability formulas ranked near the bottom — related
• Revising texts to improve readability-formula scores often produces nonexistent or insignificant comprehension gains — related