OSF repository

Product or Programme · All products and programmes
Evidence · 6 claims (6 for) · 3 studies (2 causal, 1 design), q1–q2 · 0 of 3 report an effect size · 6 claims rest on one study

Description

An OSF-hosted replication package from Zambrano et al. containing the prompts, GPT-generated codebooks, and code used to develop and compare LLM-assisted qualitative codebooks.

Components

• OSF repository of prompts, codebooks, and code for LLM-assisted codebook development: A released replication package hosting the code for the article's codebook development process together with all employed prompts and all codebooks produced by GPT across both studies. The article states: "The code for our codebook development process and all the employed prompts and codebooks produced by GPT are available for replication purposes at: https://osf.io/g3z4x". It supports researchers reproducing or extending the three-step human-AI workflow and the four prompting-strategy comparisons in new theoretical domains or datasets. (Zambrano et al. (2026))
• Open-source OSF repository of emotional micro-intervention resources under Creative Commons licence: The author provides a set of implementation resources "under Creative Commons licence in a dedicated OSF storage" at https://osf.io/tuw2a/overview, alongside the article's appendix of extensive emotional micro-interventions. The appendix includes catalogs of teacher-expression reformulations, emotion tables (discovery and power, challenge and frustration, learning and collaboration, AI and paradigm shifts), and situation-specific feedback phrases. (François PARLANT (2025))
• Open Science Framework supplementary materials with R code and interactive design-parameter tables: The article is accompanied by extensive online supplementary materials hosted on the Open Science Framework (https://osf.io/qz7fy). These include "details on the applied systematic search for IPD, measures, and methods (OSM A), the R code (R Core Team, 2024) for reproducibility and replicability, and detailed interactive tables of the design parameters that are required for sample size planning (OSM B)". The materials support researchers applying the parameters in a priori power analyses. (Martin Brunner et al. (2025))
• Open-access OSF repository with scripts to replicate results and implement the comparative modeling framework: The article releases open-access computational materials in an OSF repository (https://osf.io/84bzp/). The authors state: "We provide open-access scripts to replicate results and implement the proposed comparative modeling framework in various research settings." The scripts serve two functions: replicating the reanalyses reported in the article and applying the comparative framework to other research settings. (Fernando Núñez-Regueiro et al. (2025))
• Open Science Framework repository with data, R code, and forest plots for the label-effect meta-analysis: The article releases its research materials openly: "Data and R-Code underlying all results and comprehensive forest plots are available at the repository of the Open Science Framework". The repository contains the extracted effect-size dataset (284 effects nested in 60 experiments), the R analysis code, and comprehensive forest plots, enabling reuse and verification of the meta-analytic results. It supports the multilevel meta-analytic approach implemented with the metafor package in R. (Franz et al. (2023))
• Open Science Framework and research data repositories as infrastructure for sharing: The article describes digital infrastructure that makes open practices possible and easy. End-of-life repositories archive materials "not commonly preserved or rarely published, such as protocols, stimuli, questionnaires, raw data, analytical code". The Open Science Framework (OSF) is an open-source workspace connecting project pages to registries, repositories, and preprint servers, providing persistent, citable, timestamped links so authors need not reorganize materials at project's end. The Re3data registry catalogues repositories, including many supporting restricted access for sensitive data. (Mellor (2021))
• OSF repository with individual-study data, analysis scripts, and an integrated cross-study dataset: The article's open-data disclosure points to an Open Science Framework repository containing the study data and code. It states that "The data and analysis scripts from all individual studies, along with the integrated dataset and analytical codes for cross-study comparisons, are available via the Open Science Framework (OSF)" at the printed URL. This supports independent re-analysis of the individual studies and cross-study comparison of the ability-attribution and cheating findings. (Li Zhao et al. (2024))
• Preregistered analysis plan for the Social Perspective Taking professional development study hosted on the Open Science Framework: The article's Open Practices Disclosure identifies a time-stamped, immutable registration of the study's analysis plan in a public, open-access repository, at https://osf.io/suhvj/. The form states the registration was made "prior to examination of the data or observing the outcomes" and that no additional registrations exist and no changes were made to the preregistered primary confirmatory analysis. This artifact allows independent researchers to compare the reported analyses against the registered plan. (Hunter Gehlbach et al. (2019))

Claims

• GPT-4o prompted only with SRL theory references generates a codebook accurately representing most key elements of both foundational SRL theories [+W]
• Human review remains essential after GPT-based codebook generation, producing a final refined SRL codebook of eleven constructs [+W]
• Fully inductive LLM codebook development risks importing unexamined sensitizing concepts, such as folk theories and scientific misconceptions, from the model's training data [+W]
• Adding think-aloud data to theory prompts improves GPT-4o codebook completeness and alignment with the data [+W]
• Adult ability attributions of young children's success increase test cheating, per the article's title [+W]
• The Social Perspective Taking professional development study was preregistered with its confirmatory analysis plan registered before data examination and unchanged thereafter [+W]

Related Products and Programmes

• Reference implementation of the classroom feedback mediation layer
• Open online dataset and statistical analyses of the mutual-modelling emotions study
• Annotated R syntax and OSF repository for replicating MAIHDA analyses
• Open-source database of SRL constructs and multimodal indicators

Key Sources

• Zambrano, A.F., Wei, Z., Zhang, J., Baker, R.S., Ocumpaugh, J., Barany, A., Liu, X., Ginger, J., Paquette, L., Zhou, Y., & Borchers, C. (2026). Data Plus Theory Equals Codebook: Leveraging LLMs for Human-AI Codebook Development. Journal of Educational Data Mining, Volume 18, No 1. https://osf.io/g3z4x. doi:10.35542/osf.io/4wyqc_v1

Page metadata

Field	Value
type	product
id	osf-repository
title	OSF repository
description	An OSF-hosted replication package from Zambrano et al.
status	review
moved_from	element/osf-gpt-codebook-replication-artifact
product_kind	dataset
generated	by process:sweep-kinds on 2026-10-10
trust	unverified — no one has confirmed this page's content
sources	1 — zambrano-2026

Frontmatter is the page's machine-readable record — see the Schema & Guide.