Prompt-literacy guidance was associated with higher transcript-coded rule-following compliance than unrestricted LLM access, mainly in process-over-answer and stepwise-hint behaviors¶

Claim · All claims
Evidence · 1 study · causal r2 · q2 · i3 large
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person
Evidence quality · No study behind this claim is a peer-reviewed article with a DOI: it rests on a source with no registered publication record (a web page, book or unpublished report). It is kept because 2 design pages cite it; treat it as a lead to check, not an established finding.

Subclaims¶

q2 i3 Guided-LLM students showed higher overall rule-family compliance than Unrestricted-LLM students (d = 0.81, p = .025), in a non-randomized design the authors treat as associational. → Mohammad Amanlou 2026 q2 i? The contrast was driven mainly by process-over-answer (100% vs 22%) and stepwise-hint seeking (41% vs 7%), while concept tutoring was common in both groups. → Mohammad Amanlou 2026

Evidence¶

Mohammad Amanlou 2026¶

Mohammad Amanlou, Yasaman Amou-Jafari, Fereshte Bagheri, Fatemeh Boloukazari, Mehrad Liviyan, Elahe Khodaverdi Nadrabadi, Shahab Sherafat, Behnam Bahrak. (2026). Beyond Access: Guided LLM Scaffolding for Independent Learning in Undergraduate Statistics. Proceedings of the 34th International Conference on Computers in Education. https://github.com/MohammadAmanlou/Guided-LLM-Scaffolding

q2 · i3 · causal · r2

Manipulation check comparing the two LLM-access groups across Practices 2–4, with three raters coding 67 student-practice transcripts against six rule families (Fleiss' κ = .896). Table 2 reports "Overall compliance, assigned" at "2.43 / 6" vs "3.21 / 6", "d = 0.81; p = .025"; rule-level R1 was 22% vs 100% and R3 7% vs 41%. Assignment to the four-week quasi-experimental conditions was not randomized, and the authors state that their findings are associational rather than causal.

"Overall compliance, assigned 2.43 / 6 3.21 / 6 +0.78 / 6; d = 0.81; p = .025"

Discussion¶

Related Claims¶

• A scaffolded AI Study Coach produced no assignment-performance or concept-inventory advantage over unrestricted AI use in a master's programming course pilot — related
• Unguarded answer-giving AI harmed unaided exam performance while a guarded version of the same model erased the harm (Bastani et al., 2025, as reported) — related
• The Coach's hint budget was managed poorly: students split into early-spenders and savers, and within the Coach condition only those with a deliberate hint strategy scored higher — related
• Learning-by-teaching an LLM improved assignment and project performance versus historical cohorts, but not exam performance, in an undergraduate CS course — related
• A 4B per-student simulator solves a held-out Socratic-guidance chess case where prompted GPT-5.4 fails, indicating multi-turn training adds guidance-following capability beyond hint copying — related
• In an immediate post-test design, participants with an unrestricted chatbot achieved significantly higher learning deltas than participants with a Socratic chatbot (d = 0.80) or an EEG-adaptive exercise system (d = 0.85) — related
• Falsifiable hypothesis: unrestricted AI use on deep-processing tasks will produce a product–process dissociation—higher-rated assignments but lower unaided delayed transfer — related