Preliminary Data Collection¶

Strategy · All strategies
Evidence · 1 claim (1 for) · 2 studies (1 causal, 1 review), q2–q3 · 0 of 2 report an effect size

Description¶

Preliminary data collection involves gathering contextual data to gain a deeper understanding of the local settings where a problem exists. It combines quantitative sources (achievement records, attendance, discipline referrals disaggregated by subgroup) with qualitative sources (interviews, observations, walkthroughs, student voice) to build a comprehensive picture before designing interventions. The strategy treats data as evidence for problem definition rather than as a record-keeping exercise.

Design Implications¶

Data-informed problem framing improves the fit between interventions and local conditions; decisions grounded in multiple data types are more likely to target root causes than surface symptoms Coburn & Turner's review finds that combining data types and building capacity to interpret them is central to productive data use. [+M]. Because raw data underdetermines interpretation, collection should be paired with structured sense-making — protocols that force examination of disaggregated patterns and disconfirming evidence Data use improves decisions when paired with routines that surface assumptions rather than confirm intuitions. [~M].

Context¶

Requirements¶

• Access to data sources: records systems, observation protocols, survey instruments, and interviewees
• Analytic skills for both quantitative (disaggregation, trend analysis) and qualitative (coding, triangulation) data
• Structured protocols for interpretation, such as Data-Driven Dialogue or the Data Wise cycle
• Time and facilitation for collaborative sense-making, not just collection

Constraints¶

• Data collection is time-consuming and often requires navigating gatekeepers, consent processes, and privacy constraints (e.g., FERPA)
• Quantitative indicators alone can mislead: discipline and achievement data reflect policy and practice, not just learner behavior, and decontextualized numbers invite deficit interpretations [-M]
• Confirmation bias operates strongly: teams tend to select data that supports pre-existing theories of the problem Data use routines that do not surface assumptions tend to reinforce existing beliefs. [-M]
• Over-collection without a clear problem focus produces paralysis rather than insight

Implementation Variability¶

• Rapid-cycle versions collect a small, targeted dataset in days (e.g., five student interviews plus one week of referral data) within improvement-science approaches
• Participatory versions involve students and community members as co-collectors, e.g., youth participatory action research (YPAR)
• Audit-style versions focus on disaggregated system data to surface inequity patterns before any human data gathering

Target Learners¶

• Educator teams, school leaders, and improvement communities diagnosing a local problem of practice
• Students conducting inquiry projects, where collection builds authentic research skills Case-based and inquiry-oriented learning improves engagement and exam performance. [+M]
• Community researchers and equity audits examining disproportionality

Target Learning Goals¶

• Problem definition: moving from vague concerns to specific, evidence-grounded problem statements
• Data literacy: reading, disaggregating, and triangulating quantitative and qualitative sources
• Contextual understanding: how local culture, policies, and practices produce observed patterns
• Question formulation: deriving researchable questions from evidence rather than assumption

Instructions¶

1. Define a preliminary problem of practice and identify what evidence would distinguish competing explanations (Challenge Identification)
2. Gather quantitative indicators, disaggregated by relevant subgroups, and note anomalies or gaps
3. Collect qualitative data — interviews, Check-In conversations, classroom observations — to contextualize the numbers and surface lived experience
4. Triangulate sources and code qualitative data for patterns (Data Analysis)
5. Facilitate collaborative sense-making with a structured protocol that requires teams to generate multiple interpretations before acting (Peer Collaboration)
6. Summarize findings into a problem statement and revise research questions (Summarization and Synthesis)

Related Strategies¶

• Action Research — preliminary data collection is the diagnosing phase of an action-research cycle
• Data-Driven Dialogue — the structured sense-making protocol that converts collected data into shared interpretations
• Equity Audits — a systematic application focused on disaggregated inequity data
• Conduct a needs assessment combining baseline district data and projective opinion data before designing a middle school
• Use a Data-Informed Decision Matrix linking data types to strategic questions
• Differentiate root-cause facilitation, protocols and tools according to the school's organizational culture and past performance results
• Engineer purpose-built solutions using fit-to-context analysis and a Heilmeier Catechism-style question set
• Surface best practices for crisis-era learning support from district, school, and teacher leaders' perspectives
• Use data collaboratively through a structured data analysis process among educators
• Use a family engagement survey within a data inquiry cycle to gather family feedback and drive changes to policies and practices

Examples¶

• A school team reviewing achievement and behavioral data disaggregated by race, then conducting walkthroughs and student focus groups to understand school culture before redesigning discipline practices — surfacing how specific policies reinforced disproportionality.
• Data Wise (Harvard Graduate School of Education) — an improvement process whose "prepare and organize" and "inspect data" phases operationalize preliminary data collection for school teams.
• YPAR projects in which students survey peers and analyze school data to document issues such as restroom access or curriculum representation, then present findings to decision-makers.

Key Sources¶

• Coburn, C. E., & Turner, E. O. (2011). Research on data use: A framework and analysis. Measurement: Interdisciplinary Research and Perspectives, 9(4), 173–206. doi:10.1080/15366367.2011.626729
• Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America's schools can get better at getting better. Harvard Education Press.
• Mandinach, E. B. (2012). A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist, 47(2), 71–85. doi:10.1080/00461520.2012.667064
• Earl, L., & Katz, S. (2006). Leading schools in a data-rich world: Harnessing data for school improvement. Corwin Press.
• Ikemoto, G. S., & Marsh, J. A. (2007). Cutting through the "data-driven" mantra: Different conceptions of data-driven decision making. In P. A. Moss (Ed.), Evidence and decision making (NSSE Yearbook, pp. 105–131). Blackwell.