Artifact-grounded agent responses are rated significantly higher than transcript-only responses on groundedness, analytical depth, helpfulness, relevance, and overall quality¶

Claim · All claims
Evidence · 1 study (2 entries) · causal r1–r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Across 21 matched query pairs rated by four blind expert raters, the full agent outperformed the transcript-only baseline overall (r=.792, p=.001; means 4.24 vs 3.67 on a 5-point scale), with the largest effects on helpfulness (r=.737), analytical depth (r=.723), and groundedness (r=.693). → Dawei Xie 2026 q2 i? Accuracy ratings did not differ significantly between the full and baseline agents (r=.476, p=.054). → Dawei Xie 2026 (2)

Evidence¶

Dawei Xie 2026¶

Dawei Xie, Khalil Anderson, Tochukwu Eze, Chenghong Lin, Bookyung Shin, and Marcelo Worsley. (2026). CLARA: An AI-Augmented Analytics Dashboard for Collaboration Literacy. https://arxiv.org/abs/2605.17259 doi:10.48550/arxiv.2605.17259

q2 · i? · causal · r1

Blind rating experiment in which four external education raters evaluated 46 agent responses (21 matched full-vs-baseline pairs plus 4 artifact-specific queries) on five 5-point Likert dimensions. Wilcoxon signed-rank tests showed the full agent significantly higher on four of five dimensions and overall; the article reports "r=.792,p=.001".

"The overall comparison was significant (r=.792,p=.001), with the full agent averaging 4.24 and the baseline 3.67 on the 5-point scale."

Dawei Xie 2026 (2)¶

Dawei Xie, Khalil Anderson, Tochukwu Eze, Chenghong Lin, Bookyung Shin, and Marcelo Worsley. (2026). CLARA: An AI-Augmented Analytics Dashboard for Collaboration Literacy. https://arxiv.org/abs/2605.17259

q2 · i? · causal · r2

In the same paired rating study, accuracy showed no statistically significant difference between conditions; the article attributes the artifact advantage to depth and grounding of analysis rather than factual correctness. Equivalence was not tested.

"Accuracy did not differ significantly between conditions (r=.476,p=.054)."

Discussion¶

Related Claims¶

• Indexing LLM-generated artifacts nearly doubles retrieval recall on analytical queries compared with transcript-only retrieval, while direct queries perform comparably across configurations — related
• Disciplinary AI interaction competencies are observable in student response artifacts and scorable with rubrics grounded in student data and expert knowledge — related
• LLM-generated misconceptions grounded in quiz content, response patterns, and lecture transcripts are rated excellent by subject matter experts — related
• MAIC's script generation function outscores baseline and human script generation on five Likert-rated dimensions — related
• Research-question responses score higher Answer Relevancy (0.87) than follow-ups (0.72), while Faithfulness, Context Relevance and Response Groundedness are lower — related