Skip to content

Digital learning platform process-data analysis

Research Method · All research methods
Evidence · 10 claims (8 for, 2 mixed) · 6 studies (3 associational, 1 review, 1 design, 1 theoretical), q2 · 1 of 6 report an effect size · 10 claims rest on one study

Description

The report urges researchers to use the robust log data DLPs generate, such as which tasks students begin, time spent, and detailed progress. Such data can reveal attrition or compliance indirectly and "may serve as a measure of important intermediate outcomes, or mediators, that are often very difficult to collect in conventional research settings." Researchers unfamiliar with log data should work with platform teams and review codebooks and data dictionaries.

Accounts

  • Leverage DLP process data on user behavior to address analytic questions and measure intermediate outcomes: The report urges researchers to use the robust log data DLPs generate, such as which tasks students begin, time spent, and detailed progress. Such data can reveal attrition or compliance indirectly and "may serve as a measure of important intermediate outcomes, or mediators, that are often very difficult to collect in conventional research settings." Researchers unfamiliar with log data should work with platform teams and review codebooks and data dictionaries. (Considerations for Conducting Research in Digital Learning Platforms)
  • Session-level behavioral measures as content-independent, practitioner-aligned detectors of self-regulated learning: The article proposes that session-level measures like delayed start provide "a new opportunity for content-independent detectors, adding a behavioral component to the traditional usage and progress based on student engagement with content." Because the daily class period in K-12 is a bounded work space, its start and end times serve as behavioral anchors for defining self-regulation measures. The framework holds that detectors built from DLP log data already in use require no new diagnostics and frame results in minutes of delay, matching what educators observe. It bridges education research and classroom practice through interpretable measures. (Jordan Gutterman et al. (2026))
  • Use complete AI-tutoring dialogue transcripts as fine-grained analytics linking help-seeking behavior to performance: The article argues that Socratic AI systems can serve instruction and research simultaneously by logging full interaction transcripts. The authors state that AI-driven Socratic dialogue "generates fine-grained analytics for physics education research, establishing a scalable dual-purpose tool for instruction and learning analytics." Transcript-based coding of question specificity linked help-seeking behavior to expected grades, a methodological gap the authors say existing studies leave open because they focus on aggregate performance metrics. (Syed Furqan Abbas Hashmi (2025))
  • Two-stage learning analytics framework for GenAI Tutor engagement: session-level engagement types plus student-level transition patterns: The article proposes a learning analytics framework that first identifies engagement types at the conversation session level through clustering on behavioral, cognitive, and temporal features, then aggregates them into student-level engagement patterns via process mining. The authors state they "propose a learning analytics framework that identifies conversation session-level engagement types and integrates with student-level engagement patterns, which can be applied to other human-AI interaction data." It builds on prior process-mining work that segments logs into sessions and aggregates tactics into strategies. (Youjie Chen et al. (2026))
  • Explore existing platform data before fully specifying a research plan to inform questions, design, and sample selection: The report advises that DLPs offer access to data on large numbers of users or full user populations, usable for secondary analysis or to refine hypotheses and run power calculations. The stated consideration is "taking time to explore this available data before fully specifying a research plan," because existing data reveal available data structures and user behavior and help relate a planned experimental contrast to current platform activity. (Considerations for Conducting Research in Digital Learning Platforms)
  • Researchers should take time to understand a digital learning platform's learning environment and capabilities before planning research within it: The report's first consideration holds that researchers must conduct due diligence on a DLP before planning studies, because research must fit within learning environments that already exist. Researchers should inquire about the learning setting, theory of action, typical implementation, and available content, since "researchers will need to be prepared to explore, modify, and study their ideas within learning environments that already exist." This saves the expense of building a platform while requiring creativity in fitting research interests to it. (Considerations for Conducting Research in Digital Learning Platforms)
  • Record game telemetry at the finest usable grain size with sufficient context to reconstruct events: The chapter defines "finest usable grain size" as "a data element that has a clear definition associated with it"; a bare click is unusable, whereas a qualified datum such as clicking the reset button is usable. It recommends that "game telemetry should contain sufficient information to describe the context in which the event occurred" in enough detail to link data to a specific school, teacher, period, player, game level, and game state, illustrated by an index-card reconstruction test. (Gregory K.W.K. Chung (2015))
  • Measure utilization of instructional features via process data rather than only comparing learning outcomes: The article argues that research focusing only on the direct path—the effect of digital tools on outcomes versus another scenario—allows only limited interpretation. Instead, students' interaction with implemented instructional features (the external behavioral side of learning activities) should be treated as a theoretical and empirical mediator of achievement. The framework treats "process data" as an umbrella term encompassing tool-logged interactions and other process indicators obtained from accompanying assessments during learning. (Frank Reinhold et al. (2024))

Claims

Key Sources

  • Schellinger, A., Zacamy, J., Roschelle, J., Closser, A., & Zepeda, C. D. (2024, April). Considerations for Conducting Research in Digital Learning Platforms. Digital Promise. https://doi.org/10.51388/20.500.12265/210
  • Jordan Gutterman, Ashish Gurung, Lee Branstetter, Kenneth Koedinger, and Vincent Aleven. (2026). Cross-Subject Predictive Validity for Learning Outcomes of Delayed Start Behavior. https://arxiv.org/abs/2606.25308
  • Syed Furqan Abbas Hashmi, N. Sanjay Rebello. (2025). Analyzing Undergraduate Problem-Solving in Physics Through Interaction With an AI Chatbot. https://arxiv.org/abs/2508.14778
  • Youjie Chen, Xixi Shi, Xinyu Liu, Shuaiguo Wang, Tracy Xiao Liu, Dragan Gašević. (2026). Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use. arXiv preprint. https://arxiv.org/abs/2602.00447
Page metadata
Field Value
type research-method
id digital-learning-platform-process-data-analysis
aliases exploratory-digital-learning-platform-data-analysis-for-research-planning
title Digital learning platform process-data analysis
description The report urges researchers to use the robust log data DLPs generate, such as which tasks students begin, time spent, and detailed progress.
status review
generated by process:settle-candidates on 2026-10-09
trust unverified — no one has confirmed this page's content

Frontmatter is the page's machine-readable record — see the Schema & Guide.