Schellings' questionnaire-thinking-aloud convergent correlation of .51 rests on a small sample and is questionable against other studies¶

Claim · All claims
Evidence · 1 study · theoretical r3 · q2
Currency · newest study 2011 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person

Subclaims¶

q2 i? Schellings (this issue) found a non-significant correlation of .51 between questionnaire and thinking-aloud scores, based on N=16, and its convergence with other studies' correlations of −.07 to .42 is doubtful. → Marcel V. J. Veenman 2011

Evidence¶

Marcel V. J. Veenman 2011¶

Marcel V. J. Veenman. (2011). Alternative assessment of strategy use with self-report instruments: a discussion. Metacognition and Learning. https://doi.org/10.1007/s11409-011-9080-x

q2 · i? · theoretical · r3

Discussion of a convergent-validity validation study comparing a task-specific questionnaire with thinking-aloud protocols of text studying. The article also notes reliabilities for Schellings' subscales were remarkably low and comprehension was not assessed, leaving external validity indeterminate.

"Schellings (this issue) found a non-significant correlation of .51 between questionnaire and thinking-aloud scores. Her conclusions, however, should be formulated with modesty for two reasons. First, results were based on N=16"

Discussion¶

Related Claims¶

• External validity of the special-issue strategy measures was mixed: low-to-moderate comprehension correlations for Bräten and Strømsø and Magliano et al., indeterminate for Schellings and Winne and Muis — related
• Questionnaire measures of curiosity are difficult to use validly with young children — related
• Low questionnaire–think-aloud correlations may have two alternative explanations besides poor self-report ability: the methods may measure different strategies or be aimed at different tasks — related
• At scale level, the questionnaire's Elaboration & evaluation scale correlated significantly with the same think-aloud scale (r=.60) while the Orientation & planning scale did not (r=.24) — reports the opposite
• Most subscale reliabilities of the task-specific questionnaire were moderate to low (three of four subscales with 16 students, all four with 190), despite acceptable full-instrument reliability — related
• A task-specific questionnaire built item-by-item from a think-aloud coding taxonomy correlated r=.51 with think-aloud protocols, higher than the -.07 to .42 regularly reported — possibly the same claim (merge candidate)
• Self-report factors correlate weakly with metacognitive behavior while within-factor subcomponent correlations are larger, per studies reviewed for the meta-analysis — related
• Self-report subscales do not reliably assess distinct metacognitive subcomponents, with structures of two to nine components found across studies — related
• In the case study, students answering the questionnaire while thinking aloud showed misunderstanding of items, general rather than task-specific reference perspectives, middle-category 'neutral' answering, and probable social desirability — related
• Task-based and questionnaire-based EF measures show consistently low correlations and may assess partially distinct phenomena — related
• Cromley and Azevedo's prompted strategy-use measure correlated .23–.74 with other measures and accounted for about 30% of comprehension variance on average — related