Across three physics assessments, students who declared GenAI use scored slightly higher than non-users, but none of the differences were statistically significant¶

Claim · All claims
Evidence · 1 study (3 entries) · associational r1 · q2 · i0–i2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i2 In the scaffolded presentation (A1), GenAI users scored 15.76 ± 1.06 versus 14.80 ± 1.08 for non-users, a moderate effect size (Cohen's d = 0.63) that was not statistically significant. → Zachery Quince and Emily Faulconer 2026 q2 i0 In the lab workbook (A2), users achieved a mean of 41.18 ± 3.64 versus 40.60 ± 4.95 for non-users, a negligible effect (d = 0.12), not significant. → Zachery Quince and Emily Faulconer 2026 (2) q2 i0 In the final take-home exam (A3), users performed slightly better (23.44 ± 2.23 vs. 22.77 ± 3.73; d = 0.19), not significant. → Zachery Quince and Emily Faulconer 2026 (3)

Evidence¶

Zachery Quince and Emily Faulconer 2026¶

Zachery Quince and Emily Faulconer. (2026). Student GenAI use under changing institutional policies: a mixed-methods case study in undergraduate physics. International Journal of STEM Education. https://doi.org/10.1186/s40594-026-00651-w

q2 · i2 · associational · r1

Observational comparison of assessment grades in a single-case study of an undergraduate physics unit; Table 6 reports the contrast as "Not significant (P > 0.05)" with "Cohen's d = 0.63" for the A1 presentation (marked /20).

"The scaffolded presentation (A1), GenAI users scored 15.76 ± 1.06 compared with 14.80 ± 1.08 for non-users, yielding a moderate effect size (Cohen's d = 0.63)"

Zachery Quince and Emily Faulconer 2026 (2)¶

Zachery Quince and Emily Faulconer. (2026). Student GenAI use under changing institutional policies: a mixed-methods case study in undergraduate physics. International Journal of STEM Education. https://doi.org/10.1186/s40594-026-00651-w

q2 · i0 · associational · r1

Assessment-level comparison of percentage-converted marks for the A2 laboratory workbook (/50) in the same case study; the article reports the difference as "indicating a negligible effect" and not significant (P > 0.05).

"In the lab workbook (A2), users achieved a mean of 41.18 ± 3.64, while non-users averaged 40.60 ± 4.95 (d = 0.12), indicating a negligible effect."

Zachery Quince and Emily Faulconer 2026 (3)¶

Zachery Quince and Emily Faulconer. (2026). Student GenAI use under changing institutional policies: a mixed-methods case study in undergraduate physics. International Journal of STEM Education. https://doi.org/10.1186/s40594-026-00651-w

q2 · i0 · associational · r1

Assessment-level comparison for the A3 take-home quiz (/30) in the same case study; Table 6 reports "Not significant (P > 0.05)" with Cohen's d = 0.19.

"In the final take-home exam (A3), users again performed slightly better (23.44 ± 2.23 vs. 22.77 ± 3.73; d = 0.19)."

Discussion¶

Related Claims¶

• ChatGPT was the most used GenAI tool across all three physics assessments, with Copilot a distant second — related
• Exam environment critically moderates GenAI learning effects: significant large gains when GenAI is permitted during assessment (g = 0.76) versus non-significant slightly negative effects when it is not (g = -0.06). — related
• Student GenAI use patterns shifted across assessments: Learning and Formatting dominated under purpose-specific use, while Feedback became the dominant use (66.7%) under open use — related
• Descriptive user-study patterns that the authors read as GPT output possibly scaffolding IVDA students and reducing CV students' performance, an interpretation by prior knowledge that the study did not measure — related
• Within-user exercise accuracy does not differ significantly between high- and low-score sessions — related
• Approximately two-thirds of teacher education students chose not to adopt GenAI in assessments even when its use was allowed — related
• CLASP students' MCAT Physical Science scores were slightly higher than Physics 5 students', but the difference was not statistically significant; Biological Science scores were identical on average — related
• Students who used LLMs on assignments scored significantly lower than non-users on the first midterm (large effect size), but this gap narrowed and was non-significant on the later midterms — reports the opposite
• Students received significantly higher homework scores during GenAI-evaluation periods, but this advantage did not transfer to downstream exams — related