Metacognition and critical-thinking tutors show mixed results: Operation ARA gained 1.4 σ, SEEK showed no significant improvement, and MetaTutor's learning-efficiency advantage disappeared when agent time was counted¶

Claim · All claims
Evidence · 1 study (2 entries) · review r2 · q2 · i2–i3
Currency · newest study 2014 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i3 Operation ARA evaluations reported learning gains of 1.4 σ on a test of scientific research skills versus no-instruction controls, with students using ARA between 8 and 12 h. → Nye 2014 q2 i2 MetaTutor reported higher learning efficiency (0.84 σ) when tutors assisted students with prompts and feedback, but learning efficiency was approximately equal after considering the entire time spent using the system. → Nye 2014 (2)

Evidence¶

Nye 2014¶

Nye, B. D., Graesser, A. C., & Hu, X. (2014). AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-014-0029-5

q2 · i3 · review · r2

Review reports Operation ARA/ARIES serious-game evaluations for scientific inquiry skills; students used ARA between 8 and 12 h with game-narrative motivational features. Printed effect size: 1.4 σ versus no-instruction controls.

"evaluations of Operation ARA reported learning gains of 1.4 σ on a test of scientific research skills when comp ared with controls who received no instruction (Halpern et al. 2012)"

Nye 2014 (2)¶

Nye, B. D., Graesser, A. C., & Hu, X. (2014). AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-014-0029-5

q2 · i2 · review · r2

Review reports the MetaTutor hypermedia biology evaluation with agent prompts and feedback. Printed effect size: 0.84 σ learning efficiency counting only material-interaction time; approximately equal when total system time was counted.

"Azevedo et al. (2012)r e p o r t e dh i g h e rl e a r n i n ge f f i c i e n c yw h e nt u t o r sa s s i s t e ds t u d e n t s with prompts and feedback as they explored the hypermedia pages (0.84 σ)."

Discussion¶

Learner Variables¶

• Critical Thinking — outcome: instruction changes it

Related Claims¶

• WISE Agent feedback significantly increased Truth-seeking and CT Self-confidence disposition scores — related
• Metacognitive prompts improve learning — related
• No significant differences in proportions of fixations on single areas of interest by prior knowledge group during learning with MetaTutor — related
• Evidence suggests over-reliance on generative AI may negatively affect students' self-regulated learning behaviors, potentially harming retention and transfer to novel contexts — related
• Students with low prior knowledge show significantly higher proportional learning gains than students with high prior knowledge when learning with MetaTutor — related
• A three-month WISE Agent intervention did not significantly increase sixth-graders' aggregate critical thinking disposition scores — related