GPT-3.5 passes Dutch VWO English reading comprehension exams with a mean grade of 7.3 with re-prompting, and 6.5 when only its first responses are scored¶

Claim · All claims
Evidence · 1 study · design r2 · q3
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly

Subclaims¶

q3 i? GPT-3.5 would pass each of the English exams, with an overall mean grade across the exams of 7.3; limited to first (non-)responses the mean score was 6.5. → Joost C. F. de Winter 2024

Evidence¶

Joost C. F. de Winter 2024¶

Joost C. F. de Winter. (2024). Can ChatGPT Pass High School Exams on English Language Comprehension? International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-023-00372-z

q3 · i? · design · r2

Benchmark evaluation of three 2022 VWO English exams (124 questions total) submitted via the ChatGPT web interface; grades converted using official CITO tables. The results table reports GPT-3.5 with "an overall mean grade across the exams of 7.3"; without re-prompting the mean was 6.5 (Table 1).

"It can be seen that GPT-3.5 would pass each of the English exams, with an overall mean grade across the exams of 7.3, whereas GPT-4-0314 and GPT-4-0613 had overall mean grades of 8.3 and 8.1, respectively."

Discussion¶

Related Claims¶

• GPT-4 significantly outperforms the average Dutch student on VWO English reading comprehension, scoring 8.3 (March) and 8.1 (June) without re-prompting — related
• GPT-4's March and June 2023 versions showed no notable overall difference on the English exams, but differed on individual questions — related
• Bootstrapped self-consistency at temperature 1.0 flags potentially incorrect GPT-4 answers: about half of inconsistent responses were incorrect versus about 5% of consistent ones — related