Three-part reflective lesson on LLM use: hands-on problem solving with and without LLMs, instruction on LLM internals, and discussion of educational-impacts research¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The article describes a two-session lesson combining a problem-solving activity in which students solve a physics problem with an LLM then attempt a transfer problem without one, instruction on LLM internal functioning (probabilistic token prediction, transformers, chain-of-thought and self-consistency techniques), and discussion of research on LLMs' educational impacts. The authors report that "three components of the lesson (problem-solving activity, discussion of LLM internals, and effects of LLMs on learning) were likely beneficial toward the goal of promoting reflective usage of LLMs." Group discussion and reflection are embedded in each component.

Design Implications¶

Context¶

Requirements¶

• Students must have LLM access for the problem-solving activity and be encouraged to prompt for conceptual understanding, not just answers

Constraints¶

• The article states shifts may reflect differing pre/post populations due to attrition and unmatched anonymous surveys; the study is exploratory and hypothesis-gathering

Target Learners¶

• first-year undergraduate physics majors

Target Learning Goals¶

• critical evaluation of LLM outputs
• self-regulated learning and avoidance of over-reliance on LLMs

Related Strategies¶

• Use staged problem-solving reflection assignments to build metacognition around LLM use and surface student help-seeking perspectives

Examples¶

-

Key Sources¶

• Jake O'Brien, Margaret Ellis, Alma Robinson, John Simonetti, and Naren Ramakrishnan. (2026). Skepticism vs. Convenience: Physics Students' Perceptions and Use of Large Language Models Before and After Instruction. https://arxiv.org/abs/2609.21037