Using AI to evaluate AI outputs was often challenging because AI act as yes people and humans struggled to agree on evaluation rubrics¶

Claim · All claims
Evidence · 1 study · qualitative r? · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person · recheck by 2027: tool and model accuracy changes quickly

Subclaims¶

q2 i? Cohort teams found it difficult to use AI to evaluate AI outputs, both because models reinforced what had been stated and because humans could not agree on rubrics to automate evaluation. → ai-eval

Evidence¶

ai-eval¶

Noakes, S., Shell, A., Murillo, A., Van Nostrand, P., Ruiz, P., Cornick, S., and Karim, S. (2024, October). An Ethical and Equitable Vision of AI in Education: Learning Across 28 Exploratory Projects. Digital Promise. https://doi.org/10.51388/20.500.12265/232

q2 · i? · qualitative · r?

Technical findings across the cohort on using AI to evaluate AI outputs, reported in Table 5. The report attributes the difficulty to models reinforcing stated content and to humans struggling to agree on rubrics for automated evaluation.

"This was often challenging, particularly because AI are "yes people" and reinforce what has been stated. Further, cohort members found it to be difficult for humans to agree on rubrics to automate evaluation."

Discussion¶

Related Claims¶

• A hybrid human-AI workflow using GPT-4o with retrieval-augmented generation supported efficient inductive thematic analysis while preserving researcher judgment — related
• Human guidance remains essential for ethical grounding when integrating AI with situated learning, including resisting complacency toward generative AI outputs — related
• AI was found lacking in diagnosing student misconceptions because models do not understand lack of logic or sense making — related
• A student-facing chatbot was pulled after it reinforced a student's self-deprecating views about math, illustrating substantial risk of direct AI engagement with students — related
• Fairness of AI outputs remains under-evaluated in edtech research — related
• Underlying all AI risks in education is that generative AI behavior is difficult to explain, inspect, or predict due to its statistical mechanisms — related
• LLM-as-judge automated evaluation can achieve human-level agreement when carefully validated — reports the opposite
• The VCP rubric was difficult for teams to use because it lacked standardized calibration and its process and content categories were unclear — related
• Trustworthiness could not be reduced to a fully automated objective; human judgment remains necessary for resolving complex or context-dependent cases — related
• Despite generally accurate feedback, the LLM's scoring explanations could be unpredictable and prone to logical inconsistency, failing to award points even when citing the correct rubric directive — related
• Case-based learning benefits were reported particularly when GenAI was embedded in faculty-guided teaching, while complex cases and team workflows showed null or unfavorable effects — related
• The primary learning challenge shifts from navigating networks to evaluating and integrating AI-generated output whose sources are not visible — related
• Humans are more certain when judging that facilitation is not needed than when judging it is — related
• Global-judgment criteria of counselor effectiveness lack clarity, making replication fruitless and the research unproductive — related