Students' recognition did not differ between AI-generated and NLE MCQs¶

Claim · All claims
Evidence · 1 study · associational r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Correct and incorrect recognition showed no significant difference between generative AI and NLE MCQs. → Vogt P 2026

Evidence¶

Vogt P 2026¶

Vogt P, Wolf N, Jordan S, Volz-Willems S, Jäger J, Dupont F. (2026). Can students identify AI? – A cross-sectional quantitative study about AI recognition in tablet-based MCQ assessment among fifth-year undergraduate medical students at Saarland University, Germany. BMC Medical Education. https://doi.org/10.1186/s12909-026-10410-8

q3 · i? · associational · r2

Welch's t-tests comparing recognition of AI (n = 21 items) versus NLE (n = 18 items) items in the exam of 119 students. The article reports "no significant difference" for correct and incorrect recognition; a complementary mixed-effects model gave OR = 1.43, 95% CI 0.88–2.34, p = 0.147. No effect size printed.

"Differences were not significant for correct recognition (t(29.6) = −1.24, p = 0.225) nor for incorrect recognition (t(29.6) = 1.18, p = 0.246)."

Discussion¶

Related Claims¶

• AI-generated assessment items frequently need faculty correction, supporting AI as a draft generator within quality assurance — the general claim this finding bears on, with the studies behind it
• Recognition was not associated with item difficulty — related
• Exploratory analysis found item difficulty differed across sources, driven by the Google Gemini versus NLE contrast — related
• Students identified MCQ item source at a rate significantly above the 50% chance level in a real exam setting — related