The pipeline's contingency table option applied identically across all three domains reproduced previous descriptive findings¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? The contingency table statistic, run on domain-specific variable pairs (JW usage with age group; VERA usage with gender; SAMI usage with age and gender), produced descriptive results matching earlier findings. → Y. Bai 2025

Evidence¶

Y. Bai 2025¶

Y. Bai, P. Thajchayapong, A. Goel. (2025). Generalizing a Highly Configurable Analytics Pipeline to Replicate and Support Educational Research Across Multiple Domains. https://arxiv.org/abs/2605.30303 doi:10.48550/arxiv.2605.30303

q2 · i? · design · r2

Descriptive replication in Results §5.3: the same contingency table option was configured per domain — "the contingency table used JW usage and student age group" for JW, JW usage in VERA and gender for VERA, and SAMI usage with age and with gender for SAMI.

"For example, the Analytics Pipeline's contingency table option was applied to all three domains and produced descriptive statistic results that matched previous findings."

Discussion¶

Related Claims¶

• A single configurable analytics codebase replicates prior research findings across three educational AI assistant domains — a broader claim this one bears on
• Statistical power analysis for Welch's t-test originally developed for the VERA domain was extended to the SAMI domain through configuration alone — related
• Lack of standardised AI reporting undermines reproducibility, peer review, meta-analysis, and impact — a broader claim this one bears on