Critical constructive feedback is neglected at multiple, independently measurable processing stages¶

Claim · All claims
Evidence · 1 study · causal r2 · q3 · n=46 Swedish fifth-graders (36 with complete eye-tracking data), 424-451 feedback instances
Currency · newest study 2020 · not yet checked against its sources · not reviewed by a person

Using eye-tracking and interaction logs to separate noticing, reading, acting-upon, and progress, only 4.5% of feedback instances in a real classroom digital game survived all four measured stages of the authors' five-stage model (the third, making sense, was not measured directly); a pointing, gazing pedagogical agent significantly increased noticing and reading (but not acting-upon or progress), while an equivalent arrow cue did not.

Subclaims¶

q3 i? Of 424 tracked instances of critical constructive feedback, only 19 (4.5%) proceeded from noticing through to making measurable progress without dropping out at some stage; the largest drop-out (77%) occurred between reading and acting upon the feedback. → Tärning et al. 2020

q3 i? A pointing, gazing pedagogical agent significantly increased both noticing (66%→75% of instances) and reading (40%→52%) of feedback relative to a no-signaling control (Z=3.38, p<.001 for noticing; Z=3.69, p<.001 for reading). → Tärning et al. 2020

q3 i? An animated arrow pointing at the same feedback text produced no significant improvement in noticing or reading over the no-signaling control (Z=-0.53, p=.60 for noticing; Z=0.49, p=.63 for reading). → Tärning et al. 2020

q3 i? Whether feedback was signaled by agent, arrow, or not at all had no significant effect on whether students subsequently acted on it or made progress on the related task, though sample sizes at these later stages were small after upstream drop-out. → Tärning et al. 2020

q1 i? In a post-hoc self-report questionnaire, students were evenly split on whether they found the feedback helpful (33% affirmative, 33% "somewhat," 33% negative), indicating no dominant perceived-helpfulness explanation for neglect. → Tärning et al. 2020

Evidence¶

Tärning et al. 2020¶

Tärning, B., Lee, Y. J., Andersson, R., Månsson, K., Gulz, A., & Haake, M. (2020). Assessing the black box of feedback neglect in a digital educational game for elementary school. Journal of the Learning Sciences, 29(4-5), 511-549. https://doi.org/10.1080/10508406.2020.1770092

q3 · peer-reviewed classroom experiment with eye-tracking validation, within-subjects randomized signaling conditions; not pre-registered · i? · no d-family effect size printed; a quantified baseline (4.5% full-pipeline completion) plus statistically significant improvements at the first two of the model's five stages (noticing and decoding/reading), with null results at the later measured stages · n=46 Swedish fifth-graders (36 with complete eye-tracking data), 424-451 feedback instances · causal · r2

Forty-six fifth-graders played a digital history game requiring three teaching activities; a teachable agent ("Timy") produced critical constructive feedback whenever the student answered incorrectly, under three randomized signaling conditions (embodied agent gaze/point, animated arrow, no signaling). Eye-tracking (120 Hz, SVM-classified fixation-vs-reading) measured noticing and reading; game interaction logs measured whether students acted on the feedback and whether they subsequently improved. The pedagogical agent signaling design significantly increased noticing and reading but had no measurable effect further downstream, and a non-social arrow cue had no effect at any stage — indicating the benefit depends on social cueing specifically, not mere visual pointing.

Discussion¶

The study's central contribution is methodological: by instrumenting each processing stage separately, it shows that "feedback neglect" is not one phenomenon but several, with the largest bottleneck (77% drop-out) occurring after students have already read the feedback — a stage the tested interventions did not touch. This means the reported positive effect of agent signaling should not be read as evidence that agent signaling improves learning outcomes; it is evidence only that it improves the first two stages of a five-stage pipeline, and the paper's own null results at the later stages make that boundary explicit. The self-report finding is q1/i0 and included mainly to show that perceived helpfulness does not obviously explain who reads feedback.

Related Claims¶

• (none yet linked)
• Outcome feedback (mere knowledge of results) impedes learning of complex tasks in the multiple-cue probability learning paradigm — related
• Using generative AI as a dialogic partner for brainstorming, feedback, and co-construction can support constructive and interactive learning — related