Validate new strategy-use assessment methods against three validity criteria: internal consistency, construct validity, and external validity¶

Strategy · All strategies
Evidence · 3 claims (1 for, 2 mixed) · 1 study (1 theoretical), q2 · 0 of 1 report an effect size · 3 claims rest on one study

Description¶

The article prescribes that introduction of any new assessment method for strategy use be accompanied by validation on three criteria: "internal consistency or reliability of a measure, construct validity, and external or predictive validity." Convergent validity should be established in a multi-method design against other assessments of the same construct, and a strategy-use instrument should adequately predict reading comprehension, as most theories claim better strategy use leads to better comprehension.

Design Implications¶

Context¶

Requirements¶

• New methods should be validated at introduction (Veenman 2007); convergent validity only makes sense if both measurements represent the same construct of strategy use

Constraints¶

• Within-method comparisons only establish stability of measures, not construct validity; external validity cannot be determined when the performance measure is incorporated in the calculated score

Target Learners¶

• researchers and assessment designers studying learner strategy use

Target Learning Goals¶

• valid measurement of strategy use in reading

Claims¶

• Cromley and Azevedo's prompted strategy-use measure correlated .23–.74 with other measures and accounted for about 30% of comprehension variance on average [+W]
• External validity of the special-issue strategy measures was mixed: low-to-moderate comprehension correlations for Bräten and Strømsø and Magliano et al., indeterminate for Schellings and Winne and Muis [~W]
• Schellings' questionnaire-thinking-aloud convergent correlation of .51 rests on a small sample and is questionable against other studies [~W]

Related Strategies¶

Examples¶

-

Key Sources¶

• Marcel V. J. Veenman. (2011). Alternative assessment of strategy use with self-report instruments: a discussion. Metacognition and Learning. https://doi.org/10.1007/s11409-011-9080-x