Quantile regression estimates show little evidence of heterogeneous Mathematics Mastery treatment effects across the post-test score distribution¶

Claim · All claims
Evidence · 1 study · causal r2 · q2
Currency · newest study 2015 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Exploratory quantile regression at every 5th percentile between p15 and p85 found estimates close to the OLS estimate, with coefficients below p40 or above p65 not statistically different from zero at the 10% level. → Vignoles et al. 2015

Evidence¶

Vignoles et al. 2015¶

Anna Vignoles, John Jerrim, Richard Cowan. (2015). Mathematics Mastery: Primary Evaluation Report. Education Endowment Foundation. http://educationendowmentfoundation.org.uk/library/maths-mastery-primary

q2 · i? · causal · r2

Exploratory analysis reported in Figure 4: a series of quantile regression models estimated at every 5th percentile between p15 and p85, with standard errors bootstrapped by cluster using 100 replications. The report states there is little evidence of heterogeneous treatment effects across the post-test distribution.

"There is some, but relatively little, evidence that the intervention was less effective for the bottom half of the test distribution. The quantile regression estimates all sit relatively closely to the dashed horizontal line (OLS estimate), with all those below p40 or above p65 not statistically different from zero at even the 10% level."

Discussion¶

Related Claims¶

• The effect of Mathematics Mastery did not differ significantly between children with different levels of prior achievement — related
• Quantile regression revealed larger intervention effects on GMRT reading comprehension at the .25–.30 quantiles (g = .35, ~9 points) and the .50 quantile (g = 0.22, ~5 points), but not at other quantiles or on WJ-III Passage Comprehension — related
• The contextual-mean effect remains negative across every achievement decile and across MSC quantiles, with no systematic attenuation at the top — related
• Chatbot treatment effects on final grades were statistically significant between the 35th and 75th quantiles of the course grade distribution and at the 95th — related
• Gains from verified AI materials are concentrated in the bottom quintile, which accounts for roughly three-quarters of the average effect — related
• The positive association between sleep and performance is strongest among lower-achieving students — related
• Quantile regression showed significant Passport to Literacy effects only between the .40 and .70 quantiles of GMRT posttest comprehension — related