Perceived task difficulty ratings show a strong relation to Rasch-estimated relative task difficulty, supporting construct validity of the single-item scale¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r1 · q2 · i3 large
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i3 Perceived task difficulty and relative task difficulty are strongly correlated (r = -0.91), with relative task difficulty explaining 83.2% of variance in perceived task difficulty. → Schuessler 2025 q2 i? In a simple linear regression at task level, relative task difficulty accounted for most variance in perceived task difficulty, leaving 16.8% unexplained. → Schuessler 2025 (2)

Evidence¶

Schuessler 2025¶

Schuessler, K., Fischer, V., & Walpuski, M. (2025). Investigating construct validity of cognitive load measurement using single-item subjective rating scales. Instructional Science. https://doi.org/10.1007/s11251-024-09692-6

q2 · i3 · associational · r1

Task-level reanalysis of the Fischer (2019) chemistry competence data: 32 tasks rated on perceived task difficulty and analyzed with a 1-pl Rasch model. The printed effect size r = -0.91 is negative only because of Rasch scaling, meaning relatively difficult tasks were rated as more difficult.

"the Rasch value for perceived task difficulty and relative task difficulty estimated using the Rasch model are significantly correlated at a high level, r = −0.91, 95% BCa CI [−0.87, −0.95], p ≤ 0.001, Ntasks = 32."

Schuessler 2025 (2)¶

Schuessler, K., Fischer, V., & Walpuski, M. (2025). Investigating construct validity of cognitive load measurement using single-item subjective rating scales. Instructional Science. https://doi.org/10.1007/s11251-024-09692-6

q2 · i? · associational · r1

Regression analysis on the same 32 tasks; the article reports that "only 16.8% of perceived task difficulty remain unexplained". No standardized effect size is printed for this regression beyond R², which is not treated as an effect size.

"Using simple linear regression reveals that relative task difficulty can account for 83.2% of variance in perceived task difficulty, R 2 = 0.83, F(1, 30) = 148.89, p ≤ 0.001, Ntasks = 32 (see Table 1). Thus, only 16.8% of perceived task difficulty remain unexplained."

Discussion¶

Related Claims¶

• Items requiring integration have higher relative item difficulty than items requiring selecting or organizing, and delayed perceived task difficulty ratings appear closer to integrating-item difficulty, though that correlation comparison is not significant
• Both the competence test and the cognitive load ratings fit two-dimensional Rasch models significantly better than unidimensional models, supporting separate modeling of the two load dimensions
• Physiological measures have limited validity for effort, especially for distinguishing data-driven from goal-driven effort — related
• Multiple-choice single-select items are less difficult than open-ended response questions, and delayed perceived task difficulty ratings relate more strongly to relative item difficulty for open-ended items — a narrower finding that bears on this claim
• Delayed perceived task difficulty ratings relate more strongly to the relative difficulty of the more demanding items within a task set, consistent with delayed ratings being affected by the most demanding item — related
• Easier items were perceived as better aligned with the curriculum (strong difficulty–alignment association) — related
• Separate validation experiments confirmed that masked speech evokes a subjective experience of relative disfluency compared with pure speech — related
• Invested mental effort and perceived task difficulty are strongly correlated yet relate differently to relative task difficulty, indicating the two ratings measure different aspects of overall cognitive load — related
• Direct zero-shot GPT-4o difficulty ratings beat a grade-level dummy regressor and correlate strongly with true Rasch difficulties overall, for K-5 math and reading items — related
• The scale showed convergent and incremental validity through medium-size correlations with established cognitive load measures — related
• Re-analyses support PSVT:R structural validity: degree of rotation explains 66.9% of item-difficulty variance, though modified problematic items show no rotation-difficulty relationship — related
• Most elisions of the test were judged of average difficulty (83.2%), with 9.3% more-than-average and 7.5% very difficult, yielding frequency-based weights of .048, .427, and .533. — related