Have respondents think aloud while completing a questionnaire, and interview them afterwards, to diagnose item interpretation problems¶

Strategy · All strategies
Evidence · 1 claim (1 for) · 1 study (1 qualitative), q1 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

The article proposes examining questionnaire validity by having a few respondents answer the questionnaire while thinking aloud, so researchers can see how items are interpreted and whether items affect recollection. The author reports this "implies new possibilities in examining validity and reliability issues of questionnaires" and, since many verbalizations were short, advises "extending the think-aloud session by interviewing the respondents afterwards", citing Karabenick et al.'s cognitive interviewing.

Design Implications¶

Context¶

Requirements¶

• Access to a small volunteer sample willing to think aloud; audio-recording and transcription of verbalizations

Constraints¶

• The case study involved only four ninth-graders and one questionnaire, so the diagnostic value for other instruments is not established in this article

Target Learners¶

• secondary school students

Target Learning Goals¶

• improving the validity of self-report questionnaires about learning strategies

Claims¶

• In the case study, students answering the questionnaire while thinking aloud showed misunderstanding of items, general rather than task-specific reference perspectives, middle-category 'neutral' answering, and probable social desirability [+W]

Related Strategies¶

• Construct task-specific strategy questionnaires directly from a think-aloud coding taxonomy, item by item

Examples¶

-

Key Sources¶

• Gonny Schellings. (2011). Applying learning strategy questionnaires: problems and possibilities. Metacognition and Learning, 6, 91–109. https://doi.org/10.1007/s11409-011-9069-5