ChatGPT shows high success on open-ended general chemistry exam questions but lower accuracy at the university level, and unverified copying of its answers is a reported risk¶

Claim · All claims
Evidence · 1 study (2 entries) · review r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

Reading the evidence codes

q is the design tier, i the impact, n the sample, and the kind with its r the rigour judged within that kind: what each code means.

This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.

q2 i? ChatGPT showed high success in open-ended and skill-free general chemistry exam questions and could correctly define concepts in closed-format questions despite chemical symbols. → Ali Kürşat Erümit 2025 q2 i? ChatGPT provides understandable answers for middle and high school level questions but its accuracy rate is low at the university level, creating a risk of copying unverified answers. → Ali Kürşat Erümit 2025 (2)

Evidence¶

Ali Kürşat Erümit 2025¶

Ali Kürşat Erümit, Rabia Özdemir Sarıalioğlu. (2025). Artificial intelligence in science and chemistry education: a systematic review. Discover Education. https://doi.org/10.1007/s44217-025-00622-3

q2 · i? · review · r2

The review reports, citing Clark, an examination of ChatGPT's answers to general chemistry exam questions, in which the tool "showed high success in open-ended and skill-free questions". Second-hand report of a single study; no effect size.

"the artificial intelligence tool showed high success in open-ended and skill-free questions and could correctly define concepts in closed-format questions despite the chemical symbols in the question content"

Ali Kürşat Erümit 2025 (2)¶

q2 · i? · review · r2

The review reports, citing Tassoti (study A1 with 23 chemistry teacher candidates), that "ChatGPT provides understandable answers for middle and high school level questions, but its accuracy rate is low at the university level". Second-hand report; no effect size.

"ChatGPT provides understandable answers for middle and high school level questions, but its accuracy rate is low at the university level"

Discussion¶

Teachers find ChatGPT a creative, fast, iterative brainstorming partner that adds specific numbers and details when posing math questions from photos — related
ChatGPT zero-shot relevance extraction for instruction-quality assessment retrieves mostly irrelevant utterances and overestimates instruction quality — related
Teachers report ChatGPT-generated math questions misalign with grade level and topic, are unreliable or unanswerable, and cost time — related
Mathematics teachers critique ChatGPT-generated generative learning lessons for poor fit to students, boredom, preparation time, missing student-facing materials, mathematical flaws, and required materials — related
ChatGPT and conversational robots are the most used AI tools in science and chemistry education research, with ChatGPT used in 50% of reviewed studies — related
GenAI chemistry outputs can be fluent and locally persuasive while chemically invalid across representations — related
LLMs perform worse on complex, open-ended, or subjective assignments than on short, clearly formulated answers — related
General-purpose LLMs show accuracy and reliability limits, including hallucinations and task-specific accuracy below perfect, making them risky as expert advice sources — related

Page metadata

Field	Value
`type`	`claim`
`id`	`chatgpt-accuracy-varies-by-question-level`
`title`	`ChatGPT shows high success on open-ended general chemistry exam questions but lower accuracy at the university level, and unverified copying of its answers is a reported risk`
`description`	`ChatGPT shows high success on open-ended general chemistry exam questions but lower accuracy at the university level, and unverified copying of its answers is a reported risk`
`status`	`review`
`evidence_strength`	`weak`
`newest_evidence`	`2025`
`recheck_by`	`2028`
`generated`	by `process:wiki-ingest` on `2026-10-10`
`trust`	unverified — no one has confirmed this page's content
`sources`	2 — `ali-kürşat-erümit-2025`, `ali-kürşat-erümit-2025-2`

Frontmatter is the page's machine-readable record — see the Schema & Guide.