Predicted post-test distributions differed significantly from real distributions in four dimensions, with no significant difference only for Technology Acceptance¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Paired t-tests comparing predicted and true post-test score distributions found no significant difference only in the Technology Acceptance dimension. → Jianxiao Jiang 2025

Evidence¶

Jianxiao Jiang 2025¶

Jianxiao Jiang, Yu Zhang. (2025). Student Development Agent: Risk-free Simulation for Evaluating AIED Innovations. Preprint. https://arxiv.org/abs/2510.09183 doi:10.48550/arxiv.2510.09183

q2 · i? · design · r2

Distribution and paired t-test comparisons across the five non-cognitive dimensions (Figure 5) show significant differences for the agent's predictions in motivation, self-efficacy, grit, and self-regulated learning, but not technology acceptance; no effect sizes are printed.

"Among all dimensions, results of our student development agents showed no significance with true distribution only in Technology Acceptance dimension, which aligned with the results from RMSE and MAE estimations."

Discussion¶

Related Claims¶

• In the MAIC case study, student development agent predictions (Concept and Scales methods) outperformed the mean baseline across all five non-cognitive dimensions on RMSE and MAE — related
• The agent's predictions show a performance gap versus a regression reference that uses real students' post-test data — related
• Students' technology acceptance increased significantly after taking a MAIC course