Model specificity correlates positively with database length and negatively with the proportion of relevant studies¶

Claim · All claims
Evidence · 1 study (2 entries) · associational r2 · q2 · i2–i3
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i2 Database length correlated positively with model specificity (ρ = .35). → Campos 2024 q2 i3 The rate of relevant studies in an abstract collection correlated negatively with algorithm specificity (ρ = −0.65). → Campos 2024 (2)

Evidence¶

Campos 2024¶

Campos, D. G., Fütterer, T., Gfrörer, T., Lavelle-Hill, R., Murayama, K., König, L., Hecht, M., Zitzmann, S., & Scherer, R. (2024). Screening Smarter, Not Harder: A Comparative Analysis of Machine Learning Screening Algorithms and Heuristic Stopping Criteria for Systematic Reviews in Educational Research. Educational Psychology Review. https://doi.org/10.1007/s10648-024-09862-5

q2 · i2 · associational · r2

Exploratory Spearman correlation across the 27 abstract collections. The article reports "a positive correlation between database length and model specificity (ρ = .35, 95% CI [0.22, 0.48])".

"Exploratory analyses suggested a positive correlation between database length and model specificity (ρ = .35, 95% CI [0.22, 0.48])"

Campos 2024 (2)¶

Campos, D. G., Fütterer, T., Gfrörer, T., Lavelle-Hill, R., Murayama, K., König, L., Hecht, M., Zitzmann, S., & Scherer, R. (2024). Screening Smarter, Not Harder: A Comparative Analysis of Machine Learning Screening Algorithms and Heuristic Stopping Criteria for Systematic Reviews in Educational Research. Educational Psychology Review. https://doi.org/10.1007/s10648-024-09862-5

q2 · i3 · associational · r2

Same exploratory correlational analysis across the 27 collections. The article reports "a negative correlation between the rate of relevant studies in an abstract collection and the specificity of the algorithms (ρ = − 0.65, 95% CI [− 0.72, − 0.58])".

"a negative correlation between the rate of relevant studies in an abstract collection and the specificity of the algorithms (ρ = − 0.65, 95% CI [− 0.72, − 0.58])."

Discussion¶

Related Claims¶

• Response length is only weakly correlated with originality for most LLMs, suggesting CPIG's originality gains are not purely elaboration — related
• Sensitivity under mixed stopping criteria correlated positively with collection length in some strategy groups but not others — related
• Active learning algorithms reduce abstract-screening workload in education systematic reviews, saving on average 58% of irrelevant-record screening and 1.66 days at 95% sensitivity — related