No significant differences emerged between GPT-3.5 Turbo and Claude 3 Opus on affective, collaborative, or performance measures¶

Claim · All claims
Evidence · 1 study · causal r2 · q3 · i0 negligible
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q3 i0 All GPT-3.5 Turbo versus Claude 3 Opus comparisons showed Cohen's d < 0.2 and were not statistically significant. → Fan 2025

Evidence¶

Fan 2025¶

Fan, G., Liu, D., Zhang, R., & Pan, L. (2025). The impact of AI-assisted pair programming on student motivation, programming anxiety, collaborative learning, and programming performance: a comparative study with traditional pair programming and individual approaches. International Journal of STEM Education. https://doi.org/10.1186/s40594-025-00537-3

q3 · i0 · causal · r2

Direct comparison of the two AI-assisted year cohorts (n = 38 GPT-3.5, n = 39 Claude 3) on IMI, PAS, CLS, SPQ, and performance; the largest difference was in anxiety (d = 0.166, p = 0.149), and "Cohen's d < 0.2 for all comparisons" indicated similar impacts.

"The small effect sizes (Cohen's d < 0.2 for all comparisons) suggest that the two AI models had similar impacts on students' affective states, collaborative perceptions, and programming performance."

Discussion¶

Related Claims¶

• On benchmarks, GPT-3.5-turbo outperformed the fine-tuned Phi model in knowledge and story completion, while PhinetunEd scored best in story generation — related
• FSM-based evaluation discriminates interactive quality across models, with GPT-5 Mini scoring highest and most tiers significantly separated — related
• In blind verification, model selection matters more than source type: Bradley-Terry ranking interleaves human coders and LLMs, with per-model win rates spanning significant human advantage to numerical LLM advantage — related