Ontology-based generation produced exam-appropriate case-based MCQs, with 30% of a stratified sample rated appropriate by both expert raters and 50% by at least one¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2019 · not yet checked against its sources · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? In a user study with 15 medical experts rating a stratified random sample of 435 generated questions, 129 (30%) were considered appropriate for exams by both experts and a further 216 (50%) by at least one expert. → J. Leo 2019
Evidence¶
J. Leo 2019¶
J. Leo, G. Kurdi, N. Matentzoglu, B. Parsia, U. Sattler, S. Forge, G. Donato, W. Dowling. (2019). Ontology-Based Generation of Medical, Multi-term MCQs. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-018-00172-w
q2 · i? · design · r2
Evaluation of EMCQG's generated questions: a stratified random sample of 435 questions, of which "316 were rated by two experts", was judged by 15 medical experts for exam appropriateness. The article reports the counts and percentages above; no effect size or test statistic is printed.
"We find that using a stratified random sample of 435 questions out of which 316 were rated by two experts, 129 (30%) are considered appropriate to be used in exams by both experts and a further 216 (50%) by at least one expert."
Discussion¶
Related Claims¶
- EMCQG generated more than 3 million distinct case-based questions for four physician specialities from the EMMeT ontology — related
- Prior automatic MCQ generation approaches produce structurally simple questions (at most two stem entities) testing mainly recall — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
emcqg-expert-rated-exam-appropriate-questions |
title |
Ontology-based generation produced exam-appropriate case-based MCQs, with 30% of a stratified sample rated appropriate by both expert raters and 50% by at least one |
description |
Ontology-based generation produced exam-appropriate case-based MCQs, with 30% of a stratified sample rated appropriate by both expert raters and 50% by at least one |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2019 |
generated |
by process:wiki-ingest on 2026-10-27 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — j-leo-2019 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.