Leveraging complexity frameworks to refine theories of engagement: Advancing self-regulated learning in the age of artificial intelligence

Capturing evidence for dynamic changes in self-regulated learning (SRL) behaviours resulting from interventions is challenging for researchers. In the current study, we identified students who were likely to do poorly in a biology course and those who were likely to do well. Then, we randomly assigned a portion of the students predicted to perform poorly to a science of learning to learn intervention where they were taught SRL study strategies. Learning outcome and log data (257 K events) were collected from n = 226 students. We used a complex systems framework to model the differences in SRL including the amount, interrelatedness, density and regularity of engagement captured in digital trace data (ie, logs). Differences were compared between students who were predicted to (1) perform poorly (control, n = 48), (2) perform poorly and received intervention (treatment, n = 95) and (3) perform well (not flagged, n = 83). Results indicated that the regularity of students' engagement was predictive of course grade, and that the intervention group exhibited increased regularity in engagement over the control group immediately after the intervention and maintained that increase over the course of the semester. We discuss the implications of these findings in relation to the future of artificial intelligence and potential uses for monitoring student learning in online environments.

Practitioner notes

What is already known about this topic

• Self-regulated learning (SRL) knowledge and skills are strong predictors of postsecondary STEM student success.
• SRL is a dynamic, temporal process that leads to purposeful student engagement.
• Methods and metrics for measuring dynamic SRL behaviours in learning contexts are needed.

What this paper adds

• A Markov process for measuring dynamic SRL processes using log data.
• Evidence that dynamic, interaction-dominant aspects of SRL predict student achievement.
• Evidence that SRL processes can be meaningfully impacted through educational intervention.

Implications for theory and practice

• Complexity approaches inform theory and measurement of dynamic SRL processes.
• Static representations of dynamic SRL processes are promising learning analytics metrics.
• Engineered features of LMS usage are valuable contributions to AI models.