Before instruction, physics students show skepticism about LLM learning impacts, agreeing that over-reliance risks dependence, false confidence, and reduced conceptual understanding¶

Claim · All claims
Evidence · 1 study · causal r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2031: baselines and conditions change

Subclaims¶

q2 i? Pre-lesson, majorities agreed that LLM use can cause dependence (81%), false confidence (58%), and reduced conceptual understanding (63%), while 77% agreed LLMs help learning by checking work. → Jake O'Brien 2026

Evidence¶

Jake O'Brien 2026¶

Jake O'Brien, Margaret Ellis, Alma Robinson, John Simonetti, and Naren Ramakrishnan. (2026). Skepticism vs. Convenience: Physics Students' Perceptions and Use of Large Language Models Before and After Instruction. https://arxiv.org/abs/2609.21037 doi:10.48550/arxiv.2609.21037

q2 · i? · causal · r1

Pre-lesson Likert survey results (agreement %, disagreement %) from the same FYE physics cohort. The article reports majority agreement on "usage-false-confidence (58%, 23%), usage-dependence (81%, 10%)" and reduced conceptual understanding from solution generation; percentages are descriptive with no effect size printed.

"The statements related to learning impacts with majority agreement were usage-false-confidence (58%, 23%), usage-dependence (81%, 10%), helpful-checking-work (77%, 12%), and usage-reduced-conceptual (63%, 27%)."

Discussion¶

Related Claims¶

• Despite skepticism about learning benefits, physics students report using LLMs regularly, motivated primarily by convenience and deadline pressure — related
• First-year physics students overwhelmingly believe LLMs can solve introductory physics problems with high accuracy before instruction — related
• A reflective lesson on LLM functioning and educational impacts shifted physics students toward greater skepticism about LLM learning impacts — possibly the same claim (merge candidate)
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — a broader claim this one bears on
• Students report negative experiences with ChatGPT including inaccurate outputs, lack of emotional connection, and risk of overreliance, which constrain competence — related
• The convenience and low opportunity cost of LLMs poses a risk of over-reliance, deskilling, and reduced judgment competence when learners outsource tasks that are essential learning opportunities. — related
• Among 245 Indonesian undergraduates, majorities reported concern about reduced critical thinking when depending on AI (75%), excessive reliance (73%), misinformation (70%) and unintentional plagiarism (69%), and 40% about algorithmic bias — related
• Most students report using LLMs to get solutions more often than they believe is ideal for learning, and this gap widened after the lesson — related
• A physics-education review reports one study (Ding et al. 2023) in which high-trust students agreed with ChatGPT's answers 100% of the time although those answers were 82% accurate — related