The individualized generative AI condition showed no significant effect on relevance as identification with math compared with the control group or the collective condition¶

Claim · All claims
Evidence · 1 study · causal r2 · q3 · i1 small
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i1 Planned comparisons found no significant differences between E1 and C (p = .261, partial η² = .009), or between E1 and E2 (p = .175, η² = .013), on post relevance as identification with math. → Guo 2025

Evidence¶

Guo 2025¶

Guo, Z., Fryer, L. K., & Shum, A. (2025). Utilizing generative AI to promote high school students' personal relevance to math and interest in the math class: An intervention. doi:10.31234/osf.io/srxga_v1

q3 · i1 · small effect at most: partial η² = .013 (E1 vs E2, about d 0.23); .009 (E1 vs C) is negligible · causal · r2

Planned comparisons following the MANCOVA in the quasi-experiment of 218 students; the article reports that "The differences between E1 and C, E1 and E2 were not significant" for post relevance as identification with math. Table 3 prints the ANCOVA results: E1 vs C, F = 1.28, p = .261, η² = .009; E1 vs E2, F = 1.86, p = .175, η² = .013, so E1 vs C falls below and E1 vs E2 just above the authors' 0.01 benchmark for a small effect. Equivalence was not tested, so these are nonsignificant differences rather than a demonstration of no effect.

"The differences between E1 and C, E1 and E2 were not significant."

Discussion¶

Learner Variables¶

• Domain Context — outcome: instruction changes it

Related Claims¶

• An IES practice-guide presentation recommends personalized problem contexts, asserting without cited studies that students engage more actively and perform better in them — the general claim this finding bears on, with the studies behind it
• The generative AI relevance intervention did not significantly increase students' interest in the math class — related
• Interviews indicate individualized AI-generated identity-based relevance was perceived as overly abstract, while collective AI dialogue connected math to current concerns and cross-domain applications — related
• A collective generative AI relevance intervention significantly improved high school students' relevance as identification with math immediately after the intervention compared with a control group — related