ChatGPT zero-shot relevance extraction for instruction-quality assessment retrieves mostly irrelevant utterances and overestimates instruction quality¶

Claim · All claims
Evidence · 1 study · design r2 · q1
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly

Subclaims¶

q1 i? Preliminary analysis shows the relevant utterances retrieved by ChatGPT as a zero-shot summarization model are not helpful; it falsely identified leading questions as relevant to REMED regardless of whether they addressed student errors at a conceptual level. → Paiheng Xu 2024

Evidence¶

Paiheng Xu 2024¶

Paiheng Xu, Jing Liu, Nathan Jones, Julie Cohen, and Wei Ai. (2024). The Promises and Pitfalls of Using Language Models to Measure Instruction Quality in Education. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). https://aclanthology.org/volumes/2024.naacl-long/

q1 · i? · design · r2

Preliminary zero-shot analysis using ChatGPT as the first-stage summarization/relevance model. The authors report ChatGPT overestimated the mathematical instruction quality of utterances and failed to make inferences based on the level of pedagogical expertise required, consistent with Wang and Demszky (2023); poor results and high cost warranted no further large-scale experiments.

"our preliminary analysis shows that the relevant utterances retrieved by ChatGPT are not helpful."

Discussion¶

Related Claims¶

• Zero-shot LLM errors follow four recurring patterns: over-interpretation, failure to detect relevant information, hallucination, and failure to generate a response — a broader claim this one bears on
• Curriculum document type and competency framework significantly predict LLM prediction accuracy, and zero-shot LLMs systematically overestimate competency coverage — related
• Curricular chain-of-thought prompting improves competency-classification accuracy over zero-shot, with gains concentrated in larger models, while definition-based prompting does not improve performance — related
• Zero-shot LLMs perform only marginally above random in five-class competency classification but exceed 70% accuracy on binary classification — related
• A two-stage relevance-then-classification strategy improves PLM scoring of metacognitive modeling components whose assessment relies on multiple relevant sentences — related
• ChatGPT's exam scores were within 10% of human grades 70% of the time in a study of AI-based grading — related
• ChatGPT shows high success on open-ended general chemistry exam questions but lower accuracy at the university level, and unverified copying of its answers is a reported risk — related
• ML-based scoring approaches more often overestimated expert-assigned scores, whereas LLM-based approaches more often underestimated them — related
• Hint-based prompts raise ChatGPT scoring agreement with human experts to 98–100%, while no-hint prompts are unstable (Q2 precision 0.42) — related
• Llama3 quote extraction failed and sentiment outputs required substantial post-processing, with 20% of Reflection 1 labels unmatched by the human coder — related
• Current LLMs misalign instructional-strategy sensitivity with pedagogical relevance, including hallucinated relevance of irrelevant traits — related
• Retrieval context has a sweet spot at Top-K = 10: raising it to 15 did not improve recall but significantly worsened context precision via the distraction issue — related
• Switching from reference-based to contextual evaluation prompts improved GenAI grading accuracy — related
• Hints that instruction is suboptimal trigger compensatory self-regulation, but self-management carries cognitive and motivational costs that deplete over time — related