Guided ASR practice improves overall pronunciation accuracy of Korean EFL learners more than ordinary classroom pronunciation practice alone¶

Claim · All claims
Evidence · 2 studies (3 entries) · 1 causal r2, 1 quant-synthesis r2 · q2–q3 · i2 medium · n=15 studies (38 effect sizes)
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i2 The treatment group showed a significant reduction in segmental error rate with a moderate effect size (d = 0.52), while the control group showed a slight non-significant increase. → Dillon 2023 q2 i? The between-group difference in SER error rate was statistically significant, with the treatment group outperforming the control group. → Dillon 2023 (2) q3 i2 Across 15 studies, ESL/EFL learners practising with ASR outperformed a non-ASR condition on pronunciation (g = 0.69), with a larger effect for learners 18 and over (g = 1.20) and a negligible one for treatments of one to four weeks (g = 0.07). → Ngo et al. 2024

Evidence¶

Dillon 2023¶

Dillon, Thomas, & Wells, Donald. (2023). Effects of pronunciation training using automatic speech recognition on pronunciation accuracy of Korean English language learners. English Teaching, 78(1), 3-23. https://doi.org/10.15858/engtea.78.1.202303.3

q2 · i2 · causal · r2

Paired samples t-test on pre/post segmental error rate in a four-week quasi-experiment; treatment group (n=34) mean error rate fell from 26 (SD = 11.12) to 23.03 (SD = 9.41), a "significant reduction in mean error rate with a moderate effect size" (d = 0.52).

"The findings of the paired samples t-tests show that there was a significant reduction in mean error rate with a moderate effect size for the Treatment group (t = 3.01, p = 0.005, d = 0.52)."

Dillon 2023 (2)¶

Dillon, Thomas, & Wells, Donald. (2023). Effects of pronunciation training using automatic speech recognition on pronunciation accuracy of Korean English language learners. English Teaching, 78(1), 3-23. https://doi.org/10.15858/engtea.78.1.202303.3

q2 · i? · causal · r2

One-way ANOVA comparing SER error rate between treatment (n=34) and control (n=25) groups in the quasi-experiment; the article reports a "statistically significant difference in SER error rate between the treatment and control group" (F(1,57) = 5.176, p = 0.027, ηp2 = 0.83 as printed). Coded i? (2026-10-15): the printed ηp2 = 0.83 cannot follow from F(1,57) = 5.176, which gives about 0.08, and the authors themselves call the effect moderate, so the printed value does not establish a magnitude.

"As shown in Table 4 above there was a statistically significant difference in SER error rate between the treatment and control group with a moderate effect size. (F(1,57) = 5.176, p = 0.027, ηp 2 = 0.83)."

Ngo et al. 2024¶

Ngo, T. T.-N., Chen, H. H.-J., & Lai, K. K.-W. (2024). The effectiveness of automatic speech recognition in ESL/EFL pronunciation: A meta-analysis. ReCALL, 36(1), 4–21. doi:10.1017/S0958344023000113

q3 · meta-analysis (three-level, random effects) · i2 · medium effect, g=0.69 · n=15 studies (38 effect sizes) · quant-synthesis · r2

A meta-analysis of 15 studies (2008–2021, 38 effect sizes) comparing ESL/EFL learners who practised pronunciation with automatic speech recognition (ASR) against a non-ASR condition. The pooled effect was g = 0.69 (95% CI 0.31 to 1.08), with substantial heterogeneity (Q = 227.70; I² 56.56% between studies). The authors did not test for publication bias, and name the small number of primary studies as the main limitation. Read from the publisher's open-access full text; the article was published online in 2023. For learners aged 18 and above the effect was g = 1.20 (11 studies, 95% CI 0.41 to 1.98) and for learners under 18 g = 0.67 (4 studies, 95% CI −0.11 to 1.46). Treatments of one to four weeks gave g = 0.07 (3 studies, 95% CI −1.02 to 1.15), against g = 1.01 for five to eight weeks and g = 0.72 for nine weeks or more.

"The pooled effect size was medium (g = 0.69), and the confidence interval was not across zero (95% CI = [0.31, 1.08])."

Discussion¶

Ngo et al. (2024) support the general claim that ASR practice improves pronunciation more than a non-ASR condition. Their duration moderator qualifies it: the three studies lasting one to four weeks pooled to a negligible effect, and the Dillon & Wells (2023) programme reported here lasted four weeks, so its gain is larger than the synthesis would predict for a programme that short.

Related Claims¶

• CAPT software shows promising results for segmental pronunciation, while prosodic features and fluency still require further research and development — the general claim this finding bears on, with the studies behind it
• Guided ASR practice did not produce statistically significant improvement in any specific pronunciation error type, with some errors unimproved or regressed — related
• ASR transcription serves as a diagnostic tool identifying individual learners' pronunciation errors, with function words most commonly mispronounced — related
• Amount of ASR practice (days per week, session length) showed no observable difference in pronunciation improvement — related
• Students responded positively to ASR pronunciation training and the Rainbow passage, but were mixed on technical aspects of recording and voice typing — related
• An online word error rate checker correlates strongly with the study's segmental error rate measure, offering a quick pronunciation accuracy score — related
• The Fluspeak ASR-based pronunciation software gave good results with beginners focusing on phoneme production but poor results overall for advanced learners seeking fluency — related
• Whether individual or peer practice works better with ASR-based pronunciation training is unsettled: one study found individual work best, a meta-analysis found peer practice gave larger effects — related
• An AI-supported reading progress tool produced large effect sizes for speed and accuracy but not significant prosody gains for third-grade EFL learners — related
• Profy practice shows larger observed pronunciation-intelligibility improvement than elicited imitation, with non-overlapping pre/post confidence intervals — related
• In the 12-week quasi-experiment the experimental group's learning trajectory was significantly steeper than the control group's (group × time β = 0.52, 95% CI 0.31–0.73), with comparable baseline D-scores — related
• Against no-treatment controls, ITS produce a large statistically significant effect (g = 1.23) under a random-effects model — related
• Students reported significantly less difficulty with comprehensibility and accuracy of pronunciation between the first and final audio discussion activity — related
• Teachers attribute students' lack of pronunciation progress mainly to instruction and practice factors rather than learner factors — related