The WWC Adolescent Literacy review rates PALS as having potentially positive effects on reading comprehension from one trial of students averaging 9.8 years old, with an effect size of 0.48 that is not statistically significant when adjusted for clustering¶

Claim · All claims
Evidence · 1 study (3 entries) · causal r2 · q3 · i2 medium
Currency · newest study 2012 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i2 In the Fuchs et al. (1997) RCT, the PALS group outscored the regular-instruction comparison group on the CRAB Questions Correct measure (mean difference 0.84, effect size 0.48, improvement index +19), but the effect was not statistically significant when adjusted for clustering (p = 0.12). → Peer-Assisted Learning Strategies 2012 q3 i? The WWC rates PALS as having potentially positive effects on comprehension based on one study showing substantively important positive effects and no contrary evidence. → Peer-Assisted Learning Strategies 2012 (3)

Evidence¶

Peer-Assisted Learning Strategies 2012¶

Peer-Assisted Learning Strategies. (2012). WWC Intervention Report: Adolescent Literacy — Peer-Assisted Learning Strategies. U.S. Department of Education, What Works Clearinghouse. https://kc.vanderbilt.edu/pals/

q3 · i2 · causal · r2

Findings table in Appendix C for the Fuchs et al. (1997) cluster-randomized study of 12 schools and 120 students. The intervention group mean was 6.02 (1.79) versus 5.18 (1.66) for comparison, a mean difference of 0.84, effect size 0.48, improvement index +19, with the original study reporting p < 0.05.

"CRAB: Questions Correct Average age: 9.8 years 12 schools/ 120 students 6.02 (1.79) 5.18 (1.66) 0.84 0.48 +19 < 0.05"

Peer-Assisted Learning Strategies 2012 (2)¶

Peer-Assisted Learning Strategies. (2012). WWC Intervention Report: Adolescent Literacy — Peer-Assisted Learning Strategies. U.S. Department of Education, What Works Clearinghouse. https://kc.vanderbilt.edu/pals/

q3 · i? · causal · r2

WWC re-analysis of the same study, correcting for clustering of students within schools. The note states the adjusted effect "was not statistically significant (p = 0.12)", so significance differs from the original study's report; equivalence was not tested.

"When adjusted for clustering, the WWC-calculated effect on the Questions Correct score of the CRAB was not statistically significant (p = 0.12)."

Peer-Assisted Learning Strategies 2012 (3)¶

Peer-Assisted Learning Strategies. (2012). WWC Intervention Report: Adolescent Literacy — Peer-Assisted Learning Strategies. U.S. Department of Education, What Works Clearinghouse. https://kc.vanderbilt.edu/pals/

q3 · i? · review · r2

The Effectiveness Summary section states the WWC rating rationale: one study showed substantively important positive effects and no studies showed negative effects, yielding a rating of "potentially positive effects, with a small extent of evidence."

"Thus, for the comprehension domain, one study showed substantively important positive effects. This results in a rating of potentially positive effects, with a small extent of evidence."

Discussion¶

Related Claims¶

• Peer-assisted instruction and small-group instruction by adults have large, robust short-term effect sizes for K-6 struggling students — the general claim this finding bears on, with the studies behind it
• The evidence base for PALS in the Adolescent Literacy review rests on a single randomized trial of 120 students across 12 schools, rated as meeting standards with reservations — related
• PALS effects on the learning-disability subsets analyzed separately were not statistically significant, while the full-sample effect was significant in one study (Fuchs et al., 1997) and not in the other (Saenz et al., 2005) — related
• PALS shows no discernible effects on mathematics for students with learning disabilities — related
• PALS shows potentially positive effects on reading comprehension for students with learning disabilities (average effect size 0.75, not statistically significant) — related
• PALS shows potentially positive effects on reading fluency for students with learning disabilities, with a substantively important but not statistically significant WWC-calculated effect size of 0.41 in Saenz et al. (2005) — related
• Clustering effects in education random-assignment trials vary by design but are typically large, requiring large school samples — related
• Disciplinary Literacy Instruction Improves Comprehension — related
• The WWC rates The Expert Mathematician as having potentially positive effects on middle school mathematics achievement — a narrower finding that bears on this claim
• Earlier meta-analyses report small positive effects for computer-assisted literacy instruction (0.19; 0.16) — related