GenAI's default output on a take-home exam question remained at the quantitative, multistructural SOLO phase, producing polished but pedagogically thin responses¶

Claim · All claims
Evidence · 1 study · theoretical r2 · q1
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q1 i? In the authors' demonstration, the chatbot's initial essay-style answer met assignment criteria but organised relevant ideas without supporting relational understanding, remaining largely at the quantitative phase. → Linus Brunnström & Lisa Palmqvist 2026

Evidence¶

Linus Brunnström & Lisa Palmqvist 2026¶

Linus Brunnström & Lisa Palmqvist. (2026). AI-interaction literacy: reflections on how generative AI might be used to support self-regulated learning in higher education. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2026.2731261

q1 · i? · theoretical · r2

Reflective demonstration: the authors, role-playing a naive student, prompted Claude Opus 4.6 with the exam question 'Do machines think?' with course-literature and word-limit restrictions. The polished essay-style answer "would most likely receive a pass" yet listed replies to Searle's Chinese Room argument without explaining them.

"In SOLO terms, the response remained largely at the quantitative phase: it organised relevant ideas but did not support the learner in building their own qualitative, relational understanding."

Discussion¶

Related Claims¶

• Reaching the qualitative SOLO phase in GenAI interaction required repeated meta-level learner interventions, demanding the very self-regulatory skills the tool was expected to support — related
• Existing upskilling frameworks accelerate single stages and leave four gaps: fragmentation, missing verification, shallow default LLM pedagogy, and lack of external outcome measurement — related
• Shallow, copy-pasting engagement occurred but was not the dominant mode of GenAI Tutor use, and was lowest during exam preparation — related
• GenAI chemistry outputs can be fluent and locally persuasive while chemically invalid across representations — related
• Automated verification of AI-drafted content produced mixed results: strong accuracy and integrity scores but only 5 of 10 chapters meeting the pedagogical-progression bar — related
• In a narrative educator assessment, coding assistants (Cursor and Claude Code) produced the most accurate, complete, and pedagogically sound slides from textbook chapters — related
• As cited second-hand in an AI-literacy framework paper: ChatGPT use improved short-term essay scores but triggered metacognitive laziness with no significant knowledge-transfer gains — related
• Reviewed literature reports risks of overreliance, AI errors in complex tasks, and students' uncritical acceptance of AI-generated outputs — related
• Most student essays remained at a descriptive reflection level despite repeated AI feedback — related