GPT responses reliably incorporated broad ideas from instructions but resisted precise control¶

Claim · All claims
Evidence · 1 study · qualitative r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? Students observed that the only consistent relationship between cGPT materials and instructions and GPT responses was that broad ideas included in instructions would likely appear in responses, while identical instructions could still yield divergent response structures. → Lo S 2026

Evidence¶

Lo S 2026¶

Lo S, Lea T, Mavromoustakos A, Joseph B, Yii M, Exintaris B, Karunaratne N, Pearson DL, Ritchie T, Yuriev E and Pyun J (2026) Co-creating custom GPTs: an autoethnographic study of undergraduate students as partners in generative AI innovation. Front. Educ. 11:1818781. https://doi.org/10.3389/feduc.2026.1818781

q2 · i? · qualitative · r2

Inductive thematic analysis of summer-scholarship SPAR logs, particularly the consolidation phase where GPTs with identical instructions diverged in response structure. Students interpreted this as generative AI internalising high-level patterns while remaining resistant to precise control.

"The only consistent relationship between GPT materials and instructions, and GPT responses, is if a broad idea is included, then it will likely appear in a response"

Discussion¶

Related Claims¶

• Students reported that continued interaction with a cGPT improved its adherence to instructions — related
• Small edits to cGPT instructions and knowledge bases had unpredictable effects on GPT responses during iterative co-design — related
• Students' evaluative judgment of cGPT responses shifted from strict principle compliance toward contextual appropriateness — related
• LLM-generated ideas diverge from lesson ideas in grounding, misconception substance, vocabulary, uncertainty markers, and analogy use — related