In the case study, students answering the questionnaire while thinking aloud showed misunderstanding of items, general rather than task-specific reference perspectives, middle-category 'neutral' answering, and probable social desirability¶

Claim · All claims
Evidence · 1 study · qualitative r2 · q1
Currency · newest study 2011 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2014: tool and model accuracy changes quickly

Subclaims¶

q1 i? Think-aloud questioning revealed that some items were misunderstood (e.g., the term 'concepts', abstract phrases like 'drawing conclusions'), some answers referred to general learning rather than the task, uncertain students chose the middle scale point as a 'neutral' answer, and social desirability may have affected ratings. → Gonny Schellings 2011

Evidence¶

Gonny Schellings 2011¶

Gonny Schellings. (2011). Applying learning strategy questionnaires: problems and possibilities. Metacognition and Learning, 6, 91–109. https://doi.org/10.1007/s11409-011-9069-5

q1 · i? · qualitative · r2

Case study in which four ninth-graders thought aloud while answering the questionnaire; the author summarizes the answering problems observed. Many questions were answered in a clear, task-specific way, and the author notes "we have presented almost all notable verbalizations."

"sometimes students were not understanding the question, sometimes the students were 'rating' their general learning, and sometimes social desirability or uncertainty were probably playing a role."

Discussion¶

Related Claims¶

• A task-specific questionnaire built item-by-item from a think-aloud coding taxonomy correlated r=.51 with think-aloud protocols, higher than the -.07 to .42 regularly reported — related
• Low questionnaire–think-aloud correlations may have two alternative explanations besides poor self-report ability: the methods may measure different strategies or be aimed at different tasks — a broader claim this one bears on
• Self-report factors correlate weakly with metacognitive behavior while within-factor subcomponent correlations are larger, per studies reviewed for the meta-analysis — related
• At scale level, the questionnaire's Elaboration & evaluation scale correlated significantly with the same think-aloud scale (r=.60) while the Orientation & planning scale did not (r=.24) — related
• Schellings' questionnaire-thinking-aloud convergent correlation of .51 rests on a small sample and is questionable against other studies — related
• Most subscale reliabilities of the task-specific questionnaire were moderate to low (three of four subscales with 16 students, all four with 190), despite acceptable full-instrument reliability — related
• Think-aloud reports are incomplete and sometimes nonveridical, particularly for metacognitive processes — related