Single-item effort ratings show high internal consistency but their reliability is difficult to establish¶
Claim · All claims
Evidence · 1 study · reviewr3·q2
Currency · newest study 2020 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Internal consistency of multiple effort ratings was high (Cronbach's alpha .90 and .84), but determining the reliability of a single-item effort appraisal is problematic. → Scheiter 2020
Evidence¶
Scheiter 2020¶
Scheiter, K., Ackerman, R., & Hoogerheide, V. (2020). Looking at Mental Effort Appraisals through a Metacognitive Lens: Are they Biased? Educational Psychology Review. https://doi.org/10.1007/s10648-020-09555-9
q2 · i? · review · r3
A reanalysis of data from Paas (1992) and Paas and van Merriënboer (1994b) determined internal consistency by administering multiple test problems with effort ratings after each. The review reports "Cronbach 's alpha across these multiple effort ratings was .90" and .84, but notes that "determining the reliability of the single-item effort appraisal is problematic."
"Cronbach 's alpha across these multiple effort ratings was .90 (Paas 1992) and .84 (Paas and van Merriënboer 1994b), respectively."
Discussion¶
Related Claims¶
- The post-study survey instrument showed high internal consistency (alpha = 0.90) — related
- A high Cronbach's alpha across items measuring distinct knowledge aspects indicates the test is not working as intended — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
single-item-effort-rating-reliability-problematic |
title |
Single-item effort ratings show high internal consistency but their reliability is difficult to establish |
description |
Single-item effort ratings show high internal consistency but their reliability is difficult to establish |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2020 |
generated |
by process:wiki-ingest on 2026-10-21 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — scheiter-2020 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.