The authors argue that LLMs such as GPT-3 threaten OBE validity by enabling answers without deep understanding, particularly in unproctored settings¶

Claim · All claims
Evidence · 1 study (2 entries) · quant-synthesis r1 · q1
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q1 i? LLMs have the potential to inflate scores by enabling students to answer questions without deep understanding, especially in unproctored environments where AI use goes unchecked. → Mizori R 2025 q1 i? Test design moderates LLM impact: LLMs perform poorly on certain tasks, so questions may need to evolve toward complex, application-based scenarios. → Mizori R 2025 (2)

Evidence¶

Mizori R 2025¶

Mizori R, Sadiq M, Ahmad MT, Siu A, Ahmad RR, Yang Z, et al. (2025). STEM exam performance: Open‐ versus closed‐book methods in the large language model era. The Clinical Teacher. https://doi.org/10.1111/tct.13839

q1 · i? · quant-synthesis · r1

Theoretical argument in the Discussion, not a tested result of this review; all included studies predate major LLMs. The authors argue AI can readily answer short-answer recall questions and that unproctored use raises fairness and validity concerns.

"LLMs have the potential to inflate scores by enabling students to answer questions without a deep understanding of the subject."

Mizori R 2025 (2)¶

Mizori R, Sadiq M, Ahmad MT, Siu A, Ahmad RR, Yang Z, et al. (2025). STEM exam performance: Open‐ versus closed‐book methods in the large language model era. The Clinical Teacher. https://doi.org/10.1111/tct.13839

q1 · i? · quant-synthesis · r1

Authors' interpretation citing external evidence (e.g. poor LLM performance on biomedical NLP tasks) that test design influences LLM impact. No data in this review tests this; the authors note LLM assistance will become more sophisticated over time.

"Test questions may need to evolve from traditional formats to more complex, application‐based scenarios that LLMs struggle to navigate effectively."

Discussion¶

Related Claims¶

• Bootstrapped self-consistency at temperature 1.0 flags potentially incorrect GPT-4 answers: about half of inconsistent responses were incorrect versus about 5% of consistent ones — related
• Narrative synthesis: three studies found no significant exam-score difference between OBEs and CBEs, with differences emerging only under specific conditions — related