LLM-based extraction of three variables from 156 meta-analyses cost about $50 in computing fees, far less than estimated human-assistant labor costs¶

Claim · All claims
Evidence · 1 study · design r3 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Total computing fees were $50 for the three LLMs, versus an estimated $26,832 for human research assistants at roughly 3 hours per study. → Jansen 2026

Evidence¶

Jansen 2026¶

Jansen, T., Liebenow, L. W., Schaller, N.-J., Hattie, J., & Möller, J. (2026). Automated Data Extraction by Large Language Models: Assessing Accuracy in Comparison to Human Experts Using the Example of Visible Learning. Educational Psychology Review. https://doi.org/10.1007/s10648-026-10136-5

q3 · i? · design · r3

Cost accounting of the LLM extraction phase in the evaluation study. The article reports "total computing fees were $50" across the three models, and separately estimates human research assistants would have cost about $26,832 assuming roughly 3 hours per study at $20 per hour; automated extraction took less than 6 minutes per LLM.

"The total computing fees were $50 to extract the three variables from 156 PDFs using three LLMs (GPT 4.1 = $ 8, Gemini 2.5 Pro 2 = $2, o3 = $40)."

Discussion¶

Related Claims¶

• State-of-the-art LLMs extract effect sizes, study counts, and participant counts from educational meta-analyses with excellent agreement with adjudicated gold and silver standards — related
• A page-by-page, rubric-guided multimodal LLM workflow graded an authentic handwritten exam in about three hours at roughly $100 in token costs, versus about $3,500 in TA time — related
• Neither LLM nor human coders showed systematic over- or under-estimation in extracting data from educational meta-analyses — related
• Human scoring and feedback for HDR takes about 11 minutes 30 seconds per learner, while GPT-based scoring and feedback is nearly instant — related