Recent AI-in-education studies often rely on subjective or performance-during-support measures and weak comparison designs, limiting claims about learning effects¶

Claim · All claims
Evidence · 1 study (2 entries) · theoretical r3 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Studies often measure satisfaction or self-assessed learning rather than actual learning processes and outcomes, and treat AI-supported performance as evidence of learning. → Bauer 2025 q2 i? Pre-post designs without comparison conditions and no-instruction controls cannot clarify how AI-enhanced instruction compares to other forms of instruction. → Bauer 2025 (2)

Evidence¶

Bauer 2025¶

Bauer, E., Greiff, S., Graesser, A. C., Scheiter, K., & Sailer, M. (2025). Looking Beyond the Hype: Understanding the Effects of AI on Learning. Educational Psychology Review. https://doi.org/10.1007/s10648-025-10020-8

q2 · i? · theoretical · r3

This is the review's own critical analysis of publication trends in AI-enhanced learning research. It argues that demonstrating actual learning effects "would require demonstrating subsequent performance or knowledge improvements that persist without AI support".

"studies often focus on subjective variables like satisfaction or self-assessments of learning to make claims about the usefulness of the new AI-approaches, neglecting actual learning processes and outcomes."

Bauer 2025 (2)¶

Bauer, E., Greiff, S., Graesser, A. C., Scheiter, K., & Sailer, M. (2025). Looking Beyond the Hype: Understanding the Effects of AI on Learning. Educational Psychology Review. https://doi.org/10.1007/s10648-025-10020-8

q2 · i? · theoretical · r3

The review's methodological analysis of study designs in AI-enhanced learning research. It also notes that no-instruction controls reveal only that an AI intervention beats doing nothing, offering limited insight into its instructional quality.

"pre-post designs without a comparison condition can indicate whether there has been a positive or negative change during the AI-enhanced intervention period. However, such designs do not clarify how the AI-enhanced instruction compares to other forms of instruction (with or without AI or other technologies), nor do they confirm whether changes are due to the investigated intervention or other factors"

Discussion¶

Related Claims¶

• AI assistance reduces subjective mental effort across all tasks even when it does not reduce completion time, dissociating time and effort — related
• Subjective effort ratings are affected by invalid cues and only weakly valid as cues to judged learning outcomes — a broader claim this one bears on
• Teachers' self-reported AI literacy correlates only weakly with their objectively measured performance (r = 0.07–0.24) — a broader claim this one bears on
• Fluent Illusions Mislead Self Assessment — a broader claim this one bears on