Off-line self-report instruments show low concurrent validity with on-line measures and do not predict learning outcomes very well¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2011 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2014: tool and model accuracy changes quickly

Subclaims¶

q2 i? Off-line methods such as self-report instruments show low concurrent validity with on-line measures and do not predict learning outcomes very well. → Gonny Schellings & Bernadette Van Hout-Wolters 2011

Evidence¶

Gonny Schellings & Bernadette Van Hout-Wolters 2011¶

Gonny Schellings & Bernadette Van Hout-Wolters. (2011). Measuring strategy use with self-report instruments: theoretical and empirical considerations. Metacognition and Learning. https://doi.org/10.1007/s11409-011-9081-9

q2 · i? · review · r2

Narrative review passage in which the editorial attributes to Veenman the position that off-line self-report utility should be reconsidered and "on-line methods should be preferred". The article reports this as Veenman's argument, not as a new empirical test.

"V eenman (2005, 2011) argues that the utility of off-line methods such as self-report instruments should be reconsidered and, for the time being, on-line methods should be preferred."

Discussion¶

Related Claims¶

• GLAT scores predicted performance on GenAI-supported tasks better than perceived ChatGPT proficiency — a narrower finding that bears on this claim
• Strategic users score lowest on standard AI-outcome measures, reflecting instrument validity limits that reward AI-driven outcomes over writing quality — related
• Mathematics achievement correlates more strongly with learners' knowledge, skills and cognitive abilities than with their personality traits, goals or emotions — related
• Veenman's 2007 discussion addresses assessment and instruction of self-regulation in computer-based environments with attention to metacognitive skill measurement and intelligent tutoring systems — related
• Metacognition assessment is shifting from self-report questionnaires toward process-oriented multimodal methods, which face validity and data-alignment challenges — possibly the same claim (merge candidate)
• External validity of the special-issue strategy measures was mixed: low-to-moderate comprehension correlations for Bräten and Strømsø and Magliano et al., indeterminate for Schellings and Winne and Muis — related