The ChatGPT group achieves significantly greater essay score improvement than the other three groups¶

Claim · All claims
Evidence · 1 study · causal r2 · q3 · i1 small
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i1 Essay score improvement after revision differed significantly across groups, with the AI group improving significantly more than CN, HE and CL groups. → Fan 2025

Evidence¶

Fan 2025¶

Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gasevic, D. (2025). Beware of Metacognitive Laziness: Effects of Generative Artificial Intelligence on Learning Motivation, Processes, and Performance. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13544

q3 · i1 · causal · r2

ANOVA on researcher-scored essays (inter-rater ICCs all above 0.85) in the randomised four-group experiment. Pairwise Tukey comparisons showed the AI group's improvement exceeded CN (mean difference = 1.970, p-adjusted = 0.037), HE (2.120, 0.025) and CL (2.200, 0.012).

"The ANOVA results indicated no significant differences in essay scores (F=1.275, p=0.286, η2=0.033) before revision, but significant differences in score improvements (F=4.549, p=0.005, η2=0.108) after revision."

Discussion¶

Related Claims¶

• Process mining shows the ChatGPT group's revision loop centred on consulting ChatGPT, while the human-expert group strengthened connections between revising and reading, orientation and evaluation — related
• During revision, supported groups (AI, human expert, checklist) engage more in elaboration, organisation and orientation, while AI and human-expert groups read less than the control group — related
• Checklist tool use significantly increases Evaluation processes, an effect not observed in the ChatGPT and human-expert groups — related
• Learners supported by ChatGPT, a human expert, checklist tools, or no support show no significant difference in post-task intrinsic motivation — related
• Chain-of-thought prompting yields higher-quality GenAI feedback on argumentative essays than Zero-shot prompting and teacher feedback — related
• Essay quality was highest for AI-generated drafts and lowest with no AI, negatively correlated with ownership, suggesting an ownership-quality tradeoff. — related
• Students receiving instructor feedback showed significantly greater improvements in lab assignment scores than students receiving AI-generated feedback, after controlling for initial knowledge — related