LLMs underperformed on image-based dental examination questions compared with text-based questions¶

Claim · All claims
Evidence · 1 study · review r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person · recheck by 2028: tool and model accuracy changes quickly

Subclaims¶

q2 i? Even multimodal LLMs that can process images performed worse on image-based tasks than on text-based questions, a limitation consequential for dental education where radiographs and clinical visual imagery are essential. → Kong Mengjie 2025

Evidence¶

Kong Mengjie 2025¶

Kong Mengjie, Fok Elise Hoi Wan, Yiu Cynthia Kar Yung. (2025). A Scoping Review of Large Language Models in Dental Education: Applications, Challenges, and Prospects. International Dental Journal. https://doi.org/10.1016/j.identj.2025.103854

q2 · i? · review · r2

In the review's discussion of technical challenges, it states that "their performance on image-based tasks remains inferior to that on text-based questions", citing multiple included studies on image-based examination performance.

"Although some multimodal models have recently emerged, empirical studies indicate that their performance on image-based tasks remains inferior to that on text-based questions."

Discussion¶

Related Claims¶

• LLMs have been applied across seven distinct domains in dental education, with dental examinations the most frequent (n = 24) — related
• LLM accuracy drops from 96% on text-only to 74% on multimodal physics problems, with significant differences emerging among models — related
• Studies using both text- and image-based questions show higher pooled LLM accuracy (78%) than text-only studies (71%) — reports the opposite
• Multimodal input consistently improves accuracy but not F1, model size correlates moderately with performance, and reasoning models show mixed results — reports the opposite
• LLM errors on multimodal physics problems fall into four categories, with visual processing errors most prevalent, and error type distribution does not differ significantly across models — related
• The Multimodal Interference Effect: mapping visual representations to physical concepts degrades LLM performance even on non-visual problem aspects — a broader claim this one bears on