LLM-based simulated students surpass rule-based systems in natural language generation and situational flexibility, but concerns about bias and evaluation reliability persist¶

Claim · All claims
Evidence · 1 study · review r2 · q1
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q1 i? The review asserts that LLMs surpass rule-based systems in natural language generation and situational flexibility for student simulation. → Luis Marquez-Carpintero 2025

Evidence¶

Luis Marquez-Carpintero 2025¶

Luis Marquez-Carpintero, Alberto Lopez-Sellers, Miguel Cazorla. (2025). Simulating Students with Large Language Models: A Review of Architecture, Mechanisms, and Role Modelling in Education with Generative AI. https://arxiv.org/abs/2511.06078 doi:10.48550/arxiv.2511.06078

q1 · i? · review · r2

The review's own thematic synthesis comparing LLM-based simulated students with earlier rule-based and cognitive-model approaches. It reports the LLM advantage in "situational flexibility" while flagging persistent concerns; no effect sizes are printed.

"While LLMs surpass rule-based systems in nat- ural language generation and situational flexibility, ongoing concerns persist regarding algorithmic bias, evaluation reliability, and align- ment with educational objectives."

Discussion¶

Related Claims¶

• BEAGLE achieves higher behavioral fidelity to real student cognitive-behavior distributions than LLM prompting and agent baselines on a Particle Simulator task (DKL = 0.31 vs. baselines ≥ 0.53) — reports the opposite
• The most reported risks of AI tools in science and chemistry education are ethical issues, including gender and racial bias, hallucinations, copyright infringement, and plagiarism — related
• BEAGLE's error recurrence rate (86.2%) falls within the realistic novice envelope, far above vanilla LLMs (7.8%) and below rule-based SimStudent's mechanical saturation (92.0%) — related
• Teachers judged LLM-simulated student ideas realistic and most useful when the ideas sparked sensemaking about student thinking and suggested scaffolds — related
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — related
• Phase VI AI-enabled systems face six documented challenges: explainability, hallucination risk, prompt sensitivity, computational cost, validation complexity, and limited large-scale evidence — related
• Open issues for dynamic assessment include cost-effectiveness, training, reliability and validity, and generalization — related
• LLMs help bridge the gap between theoretical curricula and real-world skills by creating simulated scenarios, cases and problem contexts that are hard to reproduce in traditional classrooms — related