GPT-4 significantly outperforms the average Dutch student on VWO English reading comprehension, scoring 8.3 (March) and 8.1 (June) without re-prompting¶
Claim · All claims
Evidence · 1 study · designr2·q3
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q3 i? GPT-4-0314 and GPT-4-0613 achieved overall mean grades of 8.3 and 8.1 on the three exams via fully automated API prompting, above the student mean grade of 6.99. → Joost C. F. de Winter 2024
Evidence¶
Joost C. F. de Winter 2024¶
Joost C. F. de Winter. (2024). Can ChatGPT Pass High School Exams on English Language Comprehension? International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-023-00372-z
q3 · i? · design · r2
Comparison of GPT-4 scores against the distribution of 35,698 students on Exam 1 (Fig. 3). Students' "mean of their grades was 6.99"; GPT-4 grades (8.3/8.1 across exams, Table 1) exceeded this mean, with GPT-4 at the 97th and 91st percentiles on Exam 1.
"The students’ mean number of points was 35.88 ( SD = 6.48) out of a maximum of 49, and the mean of their grades was 6.99."
Discussion¶
Related Claims¶
- GPT-3.5 passes Dutch VWO English reading comprehension exams with a mean grade of 7.3 with re-prompting, and 6.5 when only its first responses are scored — related
- GPT-4's March and June 2023 versions showed no notable overall difference on the English exams, but differed on individual questions — related
- Bootstrapped self-consistency at temperature 1.0 flags potentially incorrect GPT-4 answers: about half of inconsistent responses were incorrect versus about 5% of consistent ones — related
- Exam questions that GPT-4 answered inconsistently or incorrectly were harder for human examinees, who scored about 63–68% on them versus about 75% on the rest — related
- GPT-3.5 performed worst on open one-point exam questions, while GPT-4 improved mainly on multiple-choice items — related
Page metadata
| Field | Value |
|---|---|
status |
review |
generated |
by process:wiki-ingest on 2026-10-28 |
trust |
unverified — no one has confirmed this page's content |
Frontmatter is the page's machine-readable record — see the Schema & Guide.