Students identified MCQ item source at a rate significantly above the 50% chance level in a real exam setting¶

Claim · All claims
Evidence · 1 study · associational r1 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Mean correct source recognition across 39 items was 63.9%, significantly above a 50% chance level, though recognition remained imperfect. → Vogt P 2026

Evidence¶

Vogt P 2026¶

Vogt P, Wolf N, Jordan S, Volz-Willems S, Jäger J, Dupont F. (2026). Can students identify AI? – A cross-sectional quantitative study about AI recognition in tablet-based MCQ assessment among fifth-year undergraduate medical students at Saarland University, Germany. BMC Medical Education. https://doi.org/10.1186/s12909-026-10410-8

q3 · i? · associational · r1

One-sample t-test across the 39 items of the tablet-based FM exam (60 MCQs) taken by 119 fifth-year students. Mean correct source recognition was "63.9% (95% CI 58.6–69.2%)" and "significantly above a 50% chance level (t(38) = 5.30, p < 0.001)". No effect size is printed.

"Across the 39 items included in the recognition analyses, mean correct source recognition was 63.9% (95% CI 58.6–69.2%) and was significantly above a 50% chance level (t(38) = 5.30, p < 0.001)."

Discussion¶

Related Claims¶

• AI-generated assessment items frequently need faculty correction, supporting AI as a draft generator within quality assurance — the general claim this finding bears on, with the studies behind it
• Students' recognition did not differ between AI-generated and NLE MCQs — related
• Recognition was not associated with item difficulty — related
• Student-perceived curricular alignment did not differ between AI and NLE MCQs or among ChatGPT, Gemini and NLE — related