Claims

An LLM-based educational agent-enriched learning context improved task performance in most experimental classes but not all, with one experimental class underperforming the control on the complex task

Claim · All claims
Evidence · 1 study (2 entries) · causal r2 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims

Reading the evidence codes

q is the design tier, i the impact, n the sample, and the kind with its r the rigour judged within that kind: what each code means.

This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.

q3 i? Experimental classes Y02 and Y03 scored significantly higher than the control class on Task 2, while Y03 also significantly outperformed the control on Task 1. → Li 2026 q3 i? Experimental class Y01 significantly underperformed the control class on Task 2 (Z = -2.97, p < .001, r = .38), so agent support did not always improve outcomes. → Li 2026 (2)

Evidence

Li 2026

Li, X., Liu, Z., Jiang, S., Chen, J., & Chen, W. (2026). The impact of an LLM-based educational agent on learning achievement, cognitive dynamics, and student perceptions in computer science education. International Journal of STEM Education. https://doi.org/10.1186/s40594-026-00641-y

q3 · i? · causal · r2

Mann-Whitney U comparison of practical-task scores between the control class J01 (n = 67) and experimental classes using DBagent. Y02 (Mean = 87.21) and Y03 (Mean = 89.63) "both scored higher than J01 (Mean = 85.97)" with p < .01; Y03 also significantly outperformed control on Task 1 (Z = 3.49, Z = 5.15, p < .001).

"Y02 (Mean = 87.21) and Y03 (Mean = 89.63) both scored higher than J01 (Mean = 85.97), with significant differences ( p < .01)."

Li 2026 (2)

Li, X., Liu, Z., Jiang, S., Chen, J., & Chen, W. (2026). The impact of an LLM-based educational agent on learning achievement, cognitive dynamics, and student perceptions in computer science education. International Journal of STEM Education. https://doi.org/10.1186/s40594-026-00641-y

q3 · i? · causal · r2

Mann-Whitney U test on Task 2 scores shows experimental class Y01 (Mean = 84.42), comprising primarily Geoscience majors, "underperformed J01" (Z = -2.97, p < .001, r = .38). The article attributes this to moderated efficacy tied to learners' domain-specific digital readiness.

"The negative Z-score and significant metric (Z = -2.97, p < .001, r = .38) indicate that the Y01 (Mean = 84.42) underperformed J01."

Discussion

Page metadata
Field Value
type claim
id dbagent-improves-achievement-mostly
title An LLM-based educational agent-enriched learning context improved task performance in most experimental classes but not all, with one experimental class underperforming the control on the complex task
description An LLM-based educational agent-enriched learning context improved task performance in most experimental classes but not all, with one experimental class underperforming the control on the complex task
status review
evidence_strength mixed
newest_evidence 2026
generated by process:wiki-ingest on 2026-10-11
trust unverified — no one has confirmed this page's content
sources 2 — li-2026, li-2026-2

Frontmatter is the page's machine-readable record — see the Schema & Guide.