The omnibus ANOVA on total QM score did not reach conventional significance, though a Friedman test suggested an overall difference with no significant pairwise Wilcoxon comparisons¶

Claim · All claims
Evidence · 1 study · causal r1 · q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? The repeated-measures ANOVA on total score was non-significant (F(2,38)=2.66, p=.083), while the Friedman test suggested an overall difference (χ²(2)=6.24, p=.044) with no significant pairwise Wilcoxon comparisons after Holm correction. → Jiayi Wang 2025

Evidence¶

Jiayi Wang 2025¶

Jiayi Wang, Ruiwei Xiao, Xinying Hou, John Stamper. (2025). Enabling Multi-Agent Systems as Learning Designers: Applying Learning Sciences to AI Instructional Design. https://arxiv.org/abs/2508.16659 doi:10.48550/arxiv.2508.16659

q3 · i? · causal · r1

Human expert evaluation with 20 teachers rating all three systems; the article reports the omnibus ANOVA "did not reach con- ventional significance" while the Friedman test suggested an overall difference, and post-hoc Wilcoxon tests with Holm correction revealed no significant pairwise comparisons.

"For the total score, the omnibus ANOV A did not reach con- ventional significance,F (2, 38) = 2.66,p =.083, although the non-parametric Friedman test suggested an overall dif- ference across systems,χ2(2) = 6.24,p =.044."

Discussion¶

Related Claims¶

• MAS-CMD was rated significantly higher than MAS-Roles on QM criterion 5.2C active learning by 20 expert teachers, with a large effect size — related
• GenAI models' grades differ significantly across models, with GPTo1 grading more leniently than Llama2, Mistral7B and Cohere — related
• Comprehension-test expectancy tended to be more effective than delayed-keywords listing and rereading, though interventions did not substantially differ overall — related
• Instructional approaches do not differ significantly in monitoring or reflecting, and omnibus differences in orienting and planning show no significant pairwise differences — related
• In Experiment 2, the heuristic worked examples condition significantly affected socio-discursive MAS, with maximal worked examples descriptively best and no significant post-hoc differences — related
• The revised Round II grading workflow reduced AI over-awarding and total-score MAD, most clearly for OE1 (7.1% to 4.8%) and QM (9.4% to 3.8%) — related
• Teachers qualitatively preferred MAS-CMD activities for creativity, real-world context, and classroom-ready completeness — related