Lack of standardised AI reporting undermines reproducibility, peer review, meta-analysis, and impact¶

Claim · All claims
Evidence · 1 study · theoretical r2 · q1
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q1 i? Without standardised reporting of AI tools, configurations, and data, published AI-in-education research faces problems for reproducibility, peer review, meta-analysis, and impact. → Allison 2026

Evidence¶

Allison 2026¶

Allison, J. (2026). RAISE the Standard: A Framework for Transparent Reporting of Artificial Intelligence Studies in Education. Journal of Educational Computing Research, 64(1), 3-15. https://doi.org/10.1177/07356331251377430

q1 · i? · theoretical · r2

Editorial argument by the author, grounded in observed manuscript deficiencies: submissions often omit which model was used, how it was configured, the AI's role, human oversight, and how limitations or ethical risks were addressed. This is a reasoned editorial claim, not an empirical test; no effect size is reported.

"A lack of standardised reporting creates problems for reproducibility, peer review, meta-analysis, and ultimately, impact. In this context, the RAISE framework offers a shared vocabulary and minimal reporting standard to elevate the quality and consistency of published research."

Discussion¶

Related Claims¶

• A single configurable analytics codebase replicates prior research findings across three educational AI assistant domains — related
• Research questions are rarely conceived in partnership with practitioners, which undermines research impact on educational decision making — related
• The pipeline's contingency table option applied identically across all three domains reproduced previous descriptive findings — a narrower finding that bears on this claim
• A systematic search found almost no rigorous studies contrasting online and face-to-face instruction for K–12 students: none with computable effect sizes through 2006, and only five meeting meta-analysis criteria through July 2008. — related