LLM-based assessment can replace similarity-based methods because it needs no ground-truth response and no correctness threshold¶

Claim · All claims
Evidence · 1 study · design r2 · q1
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q1 i? The authors argue LLMs could effectively replace similarity-based methods because they operate without a ground-truth response and without a threshold for determining correctness. → Arun-Balajiee Lekshmi-Narayanan 2026

Evidence¶

Arun-Balajiee Lekshmi-Narayanan 2026¶

Arun-Balajiee Lekshmi-Narayanan, Mohammad Hassany, Peter Brusilovsky. (2026). Exploring the Effectiveness of Using LLMs for Automated Assessment of Student Self–Explanations in Programming Education. https://arxiv.org/abs/2605.21614 doi:10.48550/arxiv.2605.21614

q1 · i? · design · r2

Authors' interpretation in the Discussion section, grounded in their comparison results. They add that nonlinear LLM approaches capture latent-space information about meaning or intent beyond lexical or semantic features, and that non-state-of-the-art GPT-3.5 still beat contemporary similarity methods.

"Our results suggest that LLMs could be an effective replacement for similarity-based methods for 2 reasons: they can operate without a ground-truth response and do not require a threshold to determine the correctness of student explanations."

Discussion¶

Related Claims¶

• LLM-based automated assessment of student code self-explanations outperforms semantic similarity methods on a balanced classification task — related
• Despite generally accurate feedback, the LLM's scoring explanations could be unpredictable and prone to logical inconsistency, failing to award points even when citing the correct rubric directive — related
• Teacher and student feedback indicates LLM-based formative assessment can reduce grading burden, surface misconceptions, and build student trust without replacing teacher judgment — related
• LLM-as-judge automated evaluation can achieve human-level agreement when carefully validated — related