A general-purpose LLM used as a tutor without pedagogical prompting overwhelmingly provides high assistance, with more than 95% of responses classified as Explaining or Solving¶
Claim · All claims
Evidence · 1 study · associationalr2·q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? In an introductory AI course, more than 95% of the LLM's 14,637 responses were classified as Explaining (40.3%) or Solving (55.2%), with Prompting, Hinting and Minimal together under 5%. → Suhyeon Lee 2026
Evidence¶
Suhyeon Lee 2026¶
Suhyeon Lee, Juneha Baek, Jaehyeong Park, Donghyuk Shin. (2026). LLM Pedagogical Behavior in AI Tutoring Interactions. https://arxiv.org/abs/2608.22993 doi:10.48550/arxiv.2608.22993
q2 · i? · associational · r2
Observational analysis of 14,637 paired turns from 203 students using a GPT-4o-mini assistant with the system prompt "You are a helpful assistant." Responses were LLM-classified on the five-level scaffolding scale; the mean level was 3.43 (SD = 0.84).
"More than half of responses provide a complete, task-specific solution, and another 40.3% provide detailed explanations. Prompting, Hinting, and Minimal responses together account for fewer than 5% of responses."
Discussion¶
Related Claims¶
- LLM scaffolding level is associated with the student's next dialogue act, and the association persists within broad request categories — related
- The relationship between scaffolding level and students' subsequent provide-context turns is not monotonic — related
- Scaffolding level provides no consistent additional predictive information about subsequent exam performance beyond prior achievement and observed dialogue behavior — related
- Random assignment to AI access produces a strong first stage: about 70 percent of treated students use AI, most often to explain concepts — related
- First-year physics students overwhelmingly believe LLMs can solve introductory physics problems with high accuracy before instruction — related
- Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — related
- Reviews citing one randomised trial (Bastani et al. 2025) report that unrestricted GPT-4 access in high-school mathematics raised assisted practice performance but left students worse off after withdrawal — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
default-llm-tutoring-high-assistance-concentration |
title |
A general-purpose LLM used as a tutor without pedagogical prompting overwhelmingly provides high assistance, with more than 95% of responses classified as Explaining or Solving |
description |
A general-purpose LLM used as a tutor without pedagogical prompting overwhelmingly provides high assistance, with more than 95% of responses classified as Explaining or Solving |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
recheck_by |
2029 |
generated |
by process:wiki-ingest on 2026-10-11 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — suhyeon-lee-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.