ChatGPT-generated SBMCQs contained a higher rate of nonfunctional distractors than faculty-generated SBMCQs (33.75% vs. 13.75%)¶
Claim · All claims
Evidence · 1 study · causalr2·q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? Of distractors in ChatGPT-generated items, 27 (33.75%) were nonfunctional (chosen by <5% of students), versus 11 (13.75%) in faculty-generated items. → Chauhan 2025
Evidence¶
Chauhan 2025¶
Chauhan, A., Khaliq, F., & Nayak, K. R. (2025). Assessing Quality of Scenario-Based Multiple-Choice Questions in Physiology: Faculty-Generated vs. ChatGPT-Generated Questions among Phase I Medical Students. International Journal of Artificial Intelligence in Education, 35, 2315–2344. https://doi.org/10.1007/s40593-025-00471-z
q3 · i? · causal · r2
Distractor-effectiveness analysis across the four physiology test sessions in the same crossover item-analysis study. The article reports "27 (33.75%) nonfunctional distractors in questions generated by ChatGPT" versus 11 (13.75%) for faculty; no effect size is printed.
"The study revealed that there were 27 (33.75%) nonfunctional distractors in questions generated by ChatGPT and 11 (13.75%) nonfunctional distractors for faculty generated questions."
Discussion¶
Related Claims¶
- Faculty-generated SBMCQs showed superior discrimination indices and distractor effectiveness compared with ChatGPT-generated SBMCQs — related
- Faculty-generated SBMCQs met all three ideal item-analysis criteria more often than ChatGPT-generated SBMCQs (19 vs. 9 of 40) — related
- Faculty focus groups raised concerns about authenticity, validity, and reliability of ChatGPT-generated SBMCQs, including cultural and linguistic bias — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
chatgpt-sbmcqs-more-nonfunctional-distractors |
title |
ChatGPT-generated SBMCQs contained a higher rate of nonfunctional distractors than faculty-generated SBMCQs (33.75% vs. 13.75%) |
description |
ChatGPT-generated SBMCQs contained a higher rate of nonfunctional distractors than faculty-generated SBMCQs (33.75% vs. |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2025 |
generated |
by process:wiki-ingest on 2026-10-27 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — chauhan-2025 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.