ChatGPT answered unsupervised web-based MCQ assessments with a mean accuracy of 70% (range 35–100%)¶

Claim · All claims
Evidence · 1 study (2 entries) · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Across individual web-based assessments, ChatGPT's mean MCQ accuracy was 70%, with module-level averages of 74%. → Ivory 2026 q2 i? On scenario-based 'higher MCQ' class-test questions, ChatGPT scored as highly as 87% by applying declarative knowledge to novel scenarios. → Ivory 2026 (2)

Evidence¶

Ivory 2026¶

Ivory, M., Finnerty, S., Dunn, K., Philpot, R., & Warmelink, L. (2026). The Integrity of Psychology Assessments in the AI Age: A Critical Examination. Journal of Academic Ethics. https://doi.org/10.1007/s10805-026-09729-0

q2 · i? · design · r2

Automated scoring of ChatGPT answers on unsupervised weekly web-based assessments; individual WBA mean was 70% (SD = 17.30), and when grouped by module average accuracy increased to 74% (min = 58, max = 93, SD = 14.90).

"Across all individual WBAs, the mean average score was 70% (min = 35, max = 100, SD = 17.30), highlighting their susceptibility to being answered by Gen-AI."

Ivory 2026 (2)¶

Ivory, M., Finnerty, S., Dunn, K., Philpot, R., & Warmelink, L. (2026). The Integrity of Psychology Assessments in the AI Age: A Critical Examination. Journal of Academic Ethics. https://doi.org/10.1007/s10805-026-09729-0

q2 · i? · design · r2

In class tests using a 'higher MCQ' format with described scenarios (e.g. patients presenting symptoms in PSYC317), ChatGPT scored as highly as 87% when given the same information as students.

"The Gen-AI was adept at breaking down the scenarios and typically choosing the correct answers, scoring as highly as 87%, highlighting that the application of declarative knowledge to novel scenarios is a task very easily managed by Gen-AI when given the same information as is given to students."

Discussion¶

Related Claims¶

• ChatGPT could not execute R code for statistics MCQs, achieving 60% accuracy when its code was run with corrected dataset naming — related
• Phi-2's answers to its own generated MCQs align with human experts at least as strongly as the experts agree with each other — related
• Hint-based prompts raise ChatGPT scoring agreement with human experts to 98–100%, while no-hint prompts are unstable (Q2 precision 0.42) — related
• A physics-education review reports one study (Ding et al. 2023) in which high-trust students agreed with ChatGPT's answers 100% of the time although those answers were 82% accurate — related
• Judging learning-objective alignment of generated MCQs is more subjective than answering them, with only moderate human agreement — related
• Module 3 code execution success dropped to approximately 60% before a missing-dependency infrastructure fix, recovering to match the 86–87% rates of Modules 1 and 2 — related
• LLM-generated retrieval practice was practical to deploy and well received, but high correctness rates (about 70% of items at p=1.0) restricted score variance — related
• GPT-4o and GPT-5.2 achieve higher accuracy and more fully correct answers on university IVDA multiple-choice questions than the smaller Mistral-7B and DeepSeek-R1 models — related