Sub-group analysis: observational studies show a larger OBE advantage (MD 6.89) than quasi-experimental studies (MD 3.79), with no statistically significant difference between designs¶

Claim · All claims
Evidence · 1 study (2 entries) · quant-synthesis r1 · q3
Currency · newest study 2025 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Observational studies showed a mean difference of 6.89 (95% CI 2.45 to 11.32; I² = 98%) favouring OBEs, while quasi-experimental studies showed 3.79 (95% CI 1.96 to 5.62; I² = 0%). → Mizori R 2025 q3 i? A Chi-square test showed no statistically significant variance in effect sizes between observational and quasi-experimental sub-groups (Chi2 p = 0.21). → Mizori R 2025 (2)

Evidence¶

Mizori R 2025¶

Mizori R, Sadiq M, Ahmad MT, Siu A, Ahmad RR, Yang Z, et al. (2025). STEM exam performance: Open‐ versus closed‐book methods in the large language model era. The Clinical Teacher. https://doi.org/10.1111/tct.13839

q3 · i? · quant-synthesis · r1

Sub-group meta-analysis by study design (Figure 2b) across the eight included studies. Observational designs showed "a strong effect favouring OBEs" (MD 6.89, I² = 98%); quasi-experimental designs a more modest but significant benefit (MD 3.79, I² = 0%).

"the mean difference for observational studies was 6.89 (95% CI: 2.45 to 11.32; I 2 = 98%), indicating a strong effect favouring OBEs. Quasi‐experimental studies presented a mean difference of 3.79 (95% CI: 1.96 to 5.62; I 2 = 0%)"

Mizori R 2025 (2)¶

Mizori R, Sadiq M, Ahmad MT, Siu A, Ahmad RR, Yang Z, et al. (2025). STEM exam performance: Open‐ versus closed‐book methods in the large language model era. The Clinical Teacher. https://doi.org/10.1111/tct.13839

q3 · i? · quant-synthesis · r1

Test of the sub-group interaction in the same meta-analysis. The review reports no statistically significant difference between the two design sub-groups (Chi2 p = 0.21); equivalence was not tested.

"a Chi‐square test for sub‐group differences showed no statistically significant variance in effect sizes between observational and quasi‐experimental studies (Chi2 p = 0.21)."

Discussion¶

Related Claims¶

• Narrative synthesis: three studies found no significant exam-score difference between OBEs and CBEs, with differences emerging only under specific conditions — related
• Meta-analysis shows open-book exams yield higher STEM exam scores than closed-book exams, with an overall mean difference of 5.91 despite very high heterogeneity — related
• Contiguity effects are larger under one-on-one testing (d = .89) than under group testing (d = .65), but remain educationally significant in both — related