Design think-aloud tasks with training (instruction, demonstration, practice) and marker-based prompting to enhance methodological rigour¶

Strategy · All strategies
Evidence · 3 claims (2 for, 1 mixed) · 2 studies (1 review, 1 design), q2–q3 · 0 of 2 report an effect size · 3 claims rest on one study

Description¶

The review recommends two highlighted strategies for improving think-aloud validity. Training includes clear instruction covering why and how to think aloud, demonstration sessions (via videotapes, acting out, or illustrative comments), and practice opportunities before the actual task. Prompting involves reminding participants to verbalize, ideally "by placing markers" in texts, which Crain-Thoreson et al. (1997) found elicited more veridical protocols. The review also recommends discouraging justification or explanation during think-aloud and asking for it only in follow-up interviews.

Design Implications¶

Context¶

Requirements¶

• Practice tasks should differ from tasks under investigation; all participants should receive the same examples
• Instruction should cover reasons for thinking aloud and ways to think aloud

Constraints¶

• Demonstration and practice risk participants verbalizing what is not in the mind; over-practice can automatize processes making them nonverbalizable
• Prompting during reading can interfere with the process being observed; how much practice to give is unclear

Target Learners¶

• Participants in self-regulated reading studies across grade levels and universities

Target Learning Goals¶

• Complete and accurate verbal reports of reading processes

Affordances¶

• Think Aloud Reactivity Veridicality Framework

Claims¶

• Think-aloud reports are incomplete and sometimes nonveridical, particularly for metacognitive processes [+W]
• Think-aloud reactivity effects on reading outcomes are inconclusive, moderated by report type, task language, and participant age [~W]
• The combined theory-driven and data-driven validation approach improved the match rate between trace-based SRL processes and think aloud data from 38.97% to 54.24% (training) and 34.54% to 55.09% (testing) [+W]

Related Strategies¶

• Think-Aloud Modeling
• Use a multi-study mixed method design combining process data (eye tracking), concurrent think-aloud, and retrospective interviews to study SRL adaptation

Examples¶

-

Key Sources¶

• Hu, Jingjing; Gao, Xuesong (Andy). (2017). Using think-aloud protocol in self-regulated reading research. Educational Research Review, v.22, pp. 181-193. https://doi.org/10.1016/j.edurev.2017.09.004