Use MAPS-based reflection questions to prepare students to critique AI-generated physics solutions¶

Strategy · All strategies
Evidence · 1 claim (1 for) · 1 study (1 causal), q2 · 0 of 1 report an effect size · 1 claim rests on one study

Description¶

Before evaluating an AI solution, students critique an AI output to a related preparation problem using structured reflection questions organized around the MAPS rubric categories. In this study, Critique-condition groups used "15 reflection questions organized around five MAPS categories, three per category, rather than the full MAPS rubric or scoring-level descriptors", followed by a common task evaluating a more complex AI solution with MAPS-aligned ratings and written justifications.

Design Implications¶

Context¶

Requirements¶

• A curated AI-generated solution to a related preparation problem, MAPS-aligned reflection questions, and a common evaluation task with scoring questions and written justifications

Constraints¶

• In this study the critiqued AI solution was fixed rather than student-prompted; the authors call for future work on more authentic AI use including students' own prompts and follow-up questions

Target Learners¶

• introductory undergraduate physics lab groups

Target Learning Goals¶

• expert-aligned critique of AI-generated reasoning
• calibrated trust in AI outputs

Affordances¶

• Jonassen Prompt Variation Maps Framework For Lrm Reasoning

Claims¶

• MAPS-guided critique preparation led students to identify more expert-aligned weaknesses in an AI solution than independent problem-solving preparation [+W]

Related Strategies¶

• Design PjBL projects so students must evaluate, critique, and improve AI-generated outputs rather than prohibiting AI use

Examples¶

-

Key Sources¶

• Nikhil Sanjay Borse, Amir Bralin, Sean Savage, N. Sanjay Rebello. (2026). Probing AI-generated physics solutions and preparing students to critique them. https://arxiv.org/abs/2608.12533