LLMs show discoverability bias: they accurately identify only high-quality open-ended biology responses and miss nuanced misconceptions¶

Claim · All claims
Evidence · 1 study · review r1 · q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? LLMs used for formative assessment of open-ended biology questions accurately identify high-quality responses but fail to distinguish nuanced misconceptions in less accurate answers, because low-quality answers cluster poorly in embedding space. → Filippo Sciarrone 2025

Evidence¶

Filippo Sciarrone 2025¶

Filippo Sciarrone, Pierpaolo Vittorini, Christothea Herodotou, Elvira Popescu, Nuno Palmero Otero, Chee-Kit Looi, Marcelo Milrad. (2025). Guest Editorial: Special Issue on “Bridging Futures: The Convergence of Technology Enhanced Learning and Generative AI”. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-025-00516-3

q3 · i? · review · r1

The editorial reports, from a special-issue study of LLM-based formative assessment, that models identify "only high-quality responses in open-ended biology questions" while missing misconceptions; it attributes this to poor embedding-space clustering of low-quality answers. No statistics are printed.

"LLMs have demonstrated a bias toward accurately identifying only high-quality responses in open-ended biology questions, failing to distinguish nuanced misconceptions in less accurate answers."

Discussion¶

Related Claims¶

• GPT-4 scoring errors clustered on multiple possible answers, unrecognized vocabulary, and missed incorrect or missing answers, while faculty were overly generous graders — related
• LLM grading performance relative to human graders is mixed: comparable in 12 reviewed cases, worse in 11, with the strongest alignment on short, straightforward tasks — a broader claim this one bears on
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — a broader claim this one bears on
• Accurate LLM solution diagnosis does not reliably produce pedagogically actionable feedback, with lowest rubric scores on error identification and actionability — related
• LLMs predict common distractors above chance but fall 15 percentage points below chance at predicting which errors students rarely select — related
• A discoverability bias favors correct responses: the highest-quality KP is retrieved far better than all other KPs across embedding models and clustering algorithms — possibly the same claim (merge candidate)
• Clusters computed from pre-trained LLM embeddings show low overall alignment with theory-driven Knowledge Profiles of student biology responses — related
• The Anna Karenina effect: the lower the quality of a Knowledge Profile, the less similar its responses are to each other in LLM embedding space — related