Learning from Learning Kits: gStudy Traces of Students’ Self-Regulated Engagements with Computerized Content

Researching self-regulated learning (SRL) as a process that evolves across multiple episodes of studying poses large methodological challenges. While self-report data provide useful information about learners’ perceptions of learning, these data are not reliable indicators of studying tactics learners actually use while studying, especially when learners are young children. We argue that self-reports about SRL need to be augmented by fine-grained traces that are records of learners’ actual activities as they study. We describe how gStudy software unobtrusively collects detailed trace data about learners’ use of study tactics as they engage with content presented in learning kits—collections of documents (e.g., texts, graphics, video clips) and tasks (e.g., notes, concept maps) on which learners operate to study. We suggest that trace data can advance research about how learners select, monitor, assemble, rehearse, and translate information to learn it, and provide raw materials for mapping SRL and its effects. Examples from the Life Cycles Learning Kit that supports grade 1 students learning about the life cycles of humans and frogs are given.

List of Guest Reviewers—Volume 15

This study compared the affordances of 4 multimedia learning environments for specific learning processes. The environments covered the same domain but used different instructional approaches: (a) hypermedia learning, (b) observational learning, (c) self-explanation-based learning, and (d) inquiry learning. Although they all promote an active attitude, they differ in the specific learning processes they intend to foster. In earlier research (Eysink et al., 2009 Eysink, T. H. S., de Jong, T., Berthold, K., Kolloffel, B., Opfermann, M. and Wouters, P. 2009. Learner performance in multimedia learning arrangements: An analysis across instructional approaches. American Educational Research Journal, 46: 1107–1149. [Crossref], [Web of Science ®] , [Google Scholar]), we found that learners involved in self-explanation-based or inquiry learning had the highest learning outcomes. In these approaches learners were required to generate (parts of) the subject matter, from which we concluded that they presumably stimulated learners to elaborate. Therefore, in the present study we expected that learners involved in self-explanation-based or inquiry learning would engage in more learning processes connected to elaboration than would learners involved in hypermedia or observational learning. Forty participants worked through the learning environments while thinking aloud; their protocols were coded using a generic learning processes scheme. Results showed that self-explanation-based learning and inquiry learning led to greater engagement in learning processes in general and more elaborative processes in particular. The results suggest that elaboration is indeed the key process explaining differences in learning across different instructional approaches within multimedia learning environments.     

Analysis of self-regulated learning processing using statistical models for count data

Researchers often use measures of the frequency of self-regulated learning (SRL; Zimmerman, American Educational Research Journal, 45(1), 166–183, 2000) processing as a predictor of learning gains. These frequency data, which are really counts of SRL processing events, are often non-normally distributed, and the accurate analysis of these data requires the use of specialized statistical models. In this study, we demonstrate how to implement and interpret count statistical models in path and latent profile analyses to investigate the role of SRL as a mediator of the relation between pretest and posttest conceptual understanding. Our findings from a sample of 170 college students using a computer to learn about the circulatory system show that SRL does mediate the aforementioned relation, and that count models are a more accurate representation of SRL processing data than standard analysis models based on ordinary least squares estimation. The results of our path analyses revealed which specific SRL processes were related to learning, above and beyond the effect of prior knowledge. Our latent profile analysis revealed two groups of participants, indicative of Brophy’s (2004) model of schematic and aschematic learners. We conclude with implications and future directions for basic and applied SRL research.     

Differences in students' metacognitive strategy knowledge,motivation, and strategy use: A typology of self-regulated learners

Metacognitive strategy knowledge, motivation, and learning strategies play an important role in self-regulated learning (SRL). However, little is known about different profiles of self-regulated learners in schools that prepare students for the university entrance certificate. The aim of this study was to examine intraindividual differences in the patterns of students' SRL. In this 2-wave longitudinal study, 897 students were involved. Latent class analyses revealed four-cluster solutions at the beginning as well as at the end of the school year. Maximal self-regulated learners with the highest levels on all cognitive, metacognitive, and motivational components of SRL reported the highest grades in the academic subject of German (first language) at both measurement points, followed by motivated and strategic learners. Students with a low level on several SRL components reported the lowest grades. Further, the results indicated changes in profiles of SRL over time.     

Why is externally-facilitated regulated learning more effective than self-regulated learning with hypermedia?

We examined how self-regulated learning (SRL) and externally-facilitated self-regulated learning (ERL) differentially affected adolescents’ learning about the circulatory system while using hypermedia. A total of 128 middle-school and high school students with little prior knowledge of the topic were randomly assigned to either the SRL or ERL condition. Learners in the SRL condition regulated their own learning, while learners in the ERL condition had access to a human tutor who facilitated their self-regulated learning. We converged product (pretest-posttest shifts in students’ mental models and declarative knowledge measures) with process (think-aloud protocols) data to examine the effectiveness of self- versus externally-facilitated regulated learning. Findings revealed that learners in the ERL condition gained statistically significantly more declarative knowledge and that a greater number of participants in this condition displayed a more advanced mental model on the posttest. Verbal protocol data indicated that learners in the ERL condition regulated their learning by activating prior knowledge, engaging in several monitoring activities, deploying several effective strategies, and engaging in adaptive help-seeking. By contrast, learners in the SRL condition used ineffective strategies and engaged in fewer monitoring activities. Based on these findings, we present design principles for adaptive hypermedia learning environments, engineered to foster students’ self-regulated learning about complex and challenging science topics.

In Defense of Clarity in the Study of Personal Epistemology

Andrew Elby (this issue) argues that researchers in the field of personal epistemology should beware insistence on a narrow definition of epistemology to guide this work. His argument is a response to suggestions (Hofer & Pintrich, 1997 Hofer, B. K. and Pintrich, P. R. 1997. The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning.. Review of Educational Research, 67(1): 88–140. [Crossref], [Web of Science ®] , [Google Scholar]; Sandoval, 2005 Sandoval, W. A. 2005. Understanding students' practical epistemologies and their influence on learning through inquiry.. Science Education, 89: 634–656. [Crossref], [Web of Science ®] , [Google Scholar]) that the study of personal epistemology should focus on people's views about knowledge and knowing and not conflate those with views about learning. His main concern is that learners' views about knowledge and their views about learning may, in fact, be conflated and that an insistence on definitional clarity could lead to a mischaracterization of cognitive structures. In this response I argue that clarity in the definition of theoretical constructs does not imply exclusion of views about learning from the study of personal epistemology. Furthermore, given the history of this area of research, failing to more clearly define our constructs makes real theoretical progress difficult.     

Assertion‐reason multiple‐choice testing as a tool for deep learning: a qualitative analysis

This paper reflects on the ongoing debate surrounding the usefulness (or otherwise) of multiple‐choice questions (MCQ) as an assessment instrument. The context is a graduate school of business in Australia where an experiment was conducted to investigate the use of assertion‐reason questions (ARQ), a sophisticated form of MCQ that aims to encourage higher‐order thinking on the part of the student. It builds on the work of Connelly (2004 Connelly, L. B. 2004. “Assertion‐reason assessment in formative and summative tests: results from two graduate case studies”. In Educational innovation in economics and business VIII: pedagogy, technology and innovation, Edited by: Ottewill, R., Borredon, E., Falque, L., Macfarlane, B. and Wall, A. 359–378. Dordrecht: Kluwer Academic Publishers.  [Google Scholar]) which produced a quantitative analysis of the use of ARQ testing in two economics course units in a flexibly‐delivered Master of Business Administration (MBA) program. Connelly's main findings were that ARQ tests were good substitutes for the more conventional type of multiple‐choice/short‐answer type questions and, perhaps more significantly, ARQ test performance was a good predictor of student performance in essays—the assessment instrument most widely favoured as an indicator of deeper learning. The main focus of this paper is the validity of the second of these findings, analysis of questionnaire data casting some doubt over whether student performance in ARQ tests can, indeed, be looked upon as a sound indicator of deeper learning—student reactions and opinions suggesting instead that performance might have more to do with one's proficiency in the English language.     

Note-taking within MetaTutor: interactions between an intelligent tutoring system and prior knowledge on note-taking and learning

Hypermedia learning environments (HLE) unevenly present new challenges and opportunities to learning processes and outcomes depending on learner characteristics and instructional supports. In this experimental study, we examined how one such HLE—MetaTutor, an intelligent, multi-agent tutoring system designed to scaffold cognitive and metacognitive self-regulated learning (SRL) processes—interacts with learner’s prior domain knowledge to affect their note-taking activities and subsequent learning outcomes. Sixty (N = 60) college students studied with MetaTutor for 120 min and took notes on hypermedia content of the human circulatory system. Log-files and screen recordings of learner-system interactions were used to analyze notes for several quantitative and qualitative variables. Results show that most note-taking was a verbatim copy of instructional content, which negatively related to the post-test measure of learning. There was an interaction between prior knowledge and pedagogical agent scaffolding, such that low prior knowledge students took a greater quantity of notes compared to their high prior knowledge counterparts, but only in the absence of MetaTutor SRL scaffolding; when agent SRL scaffolding was present, the note-taking activities of low prior knowledge students were statistically equivalent to the number of notes taken by their high prior knowledge counterparts. Theoretical and instructional design implications are discussed.     

The Why of Learning

Alexander, Schallert, and Reynolds's (2009 Alexander, P. A., Schallert, D. L. and Reynolds, R. E. 2009. What is learning anyway? A topological perspective considered. Educational Psychologist, 44: 176–192. this issue[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]/this issue) what, where, who, and when framework situates different perspectives on learning in different places in this multidimensional space and by doing so helps us to better understand seemingly disparate approaches to learning. The framework is in need of a fifth, why dimension. The why dimension helps to place learning within an evolutionary and cultural perspective and to better understand students' motivation to learn and their preferences for what, where, and how to learn.     

Defining Personal Epistemology: A Response to Hofer & Pintrich (1997) and Sandoval (2005)

Some researchers, including B. K. Hofer and P. R. Pintrich (1997) Hofer, B. K. and Pintrich, P. R. 1997. The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning.. Review of Educational Research, 67(1): 88–140. [Crossref], [Web of Science ®] , [Google Scholar] and W. A. Sandoval (2005) Sandoval, W. A. 2005. Understanding students' practical epistemologies and their influence on learning through inquiry.. Science Education, 89: 634–656. [Crossref], [Web of Science ®] , [Google Scholar], argue for defining personal epistemology as views about the nature of knowledge and knowing but not views about the nature of learning. Others continue using a more expansive definition of personal epistemology that includes views about learning. I argue that the scope of personal epistemology should not be decided entirely a priori. If people's views about the nature of knowing and knowledge turn out to be separable from (despite being intertwined with) their views about the nature of learning, then it makes sense to define 2 separate areas of study corresponding to those 2 separable sets of psychological constructs. From some theoretical perspectives, however, empirical results may support the interpretation that views about knowledge are inseparably entangled with views about learning. In that case, excluding views about learning from personal epistemology obscures rather than elucidates the content and cognitive structure of students' views. To be clear, I do not think the community should decide, now, to etch “views about the nature of learning” into the definition of personal epistemology. I argue instead that it is more productive not to converge on a definition until further empirical and theoretical progress points us toward the best way to “cut up [nature] … along its natural joints” (Plato, 1995 Plato. 1995. Phaedrus, Edited by: Nehamas, A. and Woodruff, P. Indianapolis, IN: Hackett.  [Google Scholar], p. 64).     

Analysing novices analysing Prolog: what stories do novices tell themselves about Prolog?

Informal observations of Prolog learners showed that, despite being presented with correct information and models, students still tended to construct their own idiosyncratic explanations of events, and, characteristically, they defended these ‘stories’ fiercely when tutorial intervention was attempted. Although the stories were often so flawed that the student's future progress was potentially hampered, it was nevertheless true that learning could not have proceeded at all without them. It seems that if we are to understand the novice Prolog programmer, we need to know about these stories, their source, and what, if anything, they have in common from one learner to another. Pain and Bundy (1987) posed the question “What stories should we tell novice Prolog programmers?” in order to teach them Prolog. In our research, we ask: “What stories do novices Prolog programmers tell themselves?” in order to learn Prolog. Observational studies undertaken showed that students used tacit knowledge of human discourse processes both to interpret the language used to communicate with the computer and to interpret the behaviour of the machine. Students did not appreciate the fundamental differences between natural discourse (as takes place amongst humans) and formal discourse (as takes place between humans and machines), and confused elements of the discourse levels. This can be an effective initial learning strategy, but unless its limitations are recognised, programs are inevitably incomplete at some level. Examples from these studies are reported here with illustrative protocol fragments.     

Design and use of hypermedia systems

Hypermedia makes it possible for learners to explore interactive multimedia-based instructional systems. To realize the potential of hypermedia to increase student involvement and learning, important design and use issues will have to be addressed. For example, how can we ensure that learners will be able to effectively regulate their own learning as they exercise the control inherent in hypermedia-based systems? How can motivation be stimulated and maintained, so that individuals will go beyond superficial browsing to really explore and learn? A theoretical and research-based foundation is summarized here which suggests answers to these questions, answers that revolve around learner control, self-regulated learning, and continuing motivation. Recommendations are offered for the effective design of hypermedia systems and the appropriate preparation of learners who will use them.     

Exploring School Improvement in Hong Kong Secondary Schools

Scholars have increasingly sought to understand how the process of school improvement differs among schools operating in different school levels, conditions, and contexts. Using Rosenholtz's (1985 Rosenholtz, S. 1985. Effective schools: Interpreting the evidence. American Journal of Education, 93: 352–388. [Crossref], [Web of Science ®] , [Google Scholar]) conception of “moving” and “stuck” schools as a framework for thinking about school improvement, this study examines the learning outcomes of 39 Hong Kong secondary schools over a 3-year period. We examine whether features of leadership and school capacity differed with respect to these learning outcomes within the sample of moving and stuck schools. This research in Hong Kong has identified several factors that appear to synergistically contribute to differences in patterns of improvement in learning across different subjects in both moving and stuck schools. These factors include resource management of principals and school capacity in terms of professional learning community; workload of teachers; alignment, coherence, and structure; and resource capacity. This study extends the research on leadership and capacity building as a means of school improvement, in the process elaborating on their impact within a non-Western society.     

Preschool Student Learning in Literacy and Mathematics: Impact of Teacher Experience,Qualifications, and Beliefs on an At-Risk Sample

Few studies investigating the impacts of teacher characteristics and beliefs about the importance of early skill learning have included measures of children's learning outcomes. This study investigated how teachers' educational attainment, experience, and beliefs impact the development of letter identification and number concepts (enumeration, cardinality, and numeral identification). One hundred thirty-eight 4-year-old children from low-income homes attending public preschool programs were the focus of a study based on findings that early learning is impacted by family characteristics and teachers' perceptions of children's eagerness to learn (West, Denton, & Germino Hausken, 2000 West, J., Denton, K. and Hausken, Germino E. 2000. America's Kindergartners (NCES 2000–070), Washington, DC: National Center for Education Statistics..  [Google Scholar]). Children's skills were assessed fall and spring, with more change found in spring measures of letter identification than in measures of number concept skills. Teachers' educational attainment was found to strongly influence development of letter identification, with teacher experience a weaker influence. For number concepts, teacher education and experience were equivalent influences. Teachers' beliefs about literacy and mathematics were weakly related to children's learning outcomes, but added to the variance accounted for beyond the influence of teacher education and experience in the development of numeral recognition. More information is needed from studies focusing on children learning across the school year on how structural and process features influence young children's learning.     

Creativity in the Early Childhood Classroom: Perspectives of Preservice Teachers

This study examined how preservice teachers view the nature and role of creativity in light of the complexities of contemporary early childhood classrooms. A multiple methods approach was utilized and data were collected with the Questionnaire Examining Student Teachers' Beliefs about Creativity (Diakidoy & Kanari, 1999 Diakidoy, I. and Kanari, E. 1999. Student teacher beliefs about creativity. British Educational Research Journal, 25: 225–243. [Taylor & Francis Online] , [Google Scholar]) survey instrument and through a qualitative examination of a learning community whose members were preservice teachers engaged in their student-teaching semester. Findings from both the survey and qualitative measures indicate that the vast majority of preservice teachers valued creative thinking; however, they voiced concerns over their own unclear understandings of how to support children's creative endeavors in complex classroom environments. Additionally, this research strongly suggests that early childhood teacher educators take a supportive role in the development of preservice teachers' understandings of the interrelationships between content, pedagogy, and creativity in early childhood classrooms.     

Leveraging complexity frameworks to refine theories of engagement: Advancing self-regulated learning in the age of artificial intelligence

Capturing evidence for dynamic changes in self-regulated learning (SRL) behaviours resulting from interventions is challenging for researchers. In the current study, we identified students who were likely to do poorly in a biology course and those who were likely to do well. Then, we randomly assigned a portion of the students predicted to perform poorly to a science of learning to learn intervention where they were taught SRL study strategies. Learning outcome and log data (257 K events) were collected from n = 226 students. We used a complex systems framework to model the differences in SRL including the amount, interrelatedness, density and regularity of engagement captured in digital trace data (ie, logs). Differences were compared between students who were predicted to (1) perform poorly (control, n = 48), (2) perform poorly and received intervention (treatment, n = 95) and (3) perform well (not flagged, n = 83). Results indicated that the regularity of students' engagement was predictive of course grade, and that the intervention group exhibited increased regularity in engagement over the control group immediately after the intervention and maintained that increase over the course of the semester. We discuss the implications of these findings in relation to the future of artificial intelligence and potential uses for monitoring student learning in online environments.

Practitioner notes

What is already known about this topic

• Self-regulated learning (SRL) knowledge and skills are strong predictors of postsecondary STEM student success.
• SRL is a dynamic, temporal process that leads to purposeful student engagement.
• Methods and metrics for measuring dynamic SRL behaviours in learning contexts are needed.

What this paper adds

• A Markov process for measuring dynamic SRL processes using log data.
• Evidence that dynamic, interaction-dominant aspects of SRL predict student achievement.
• Evidence that SRL processes can be meaningfully impacted through educational intervention.

Implications for theory and practice

• Complexity approaches inform theory and measurement of dynamic SRL processes.
• Static representations of dynamic SRL processes are promising learning analytics metrics.
• Engineered features of LMS usage are valuable contributions to AI models.

     

The mediating effects of needs satisfaction on the relationships between prior knowledge and self-regulated learning through artificial intelligence chatbot

The anthropomorphic characteristics of artificial intelligence (AI) can provide a positive environment for self-regulated learning (SRL). The factors affecting adolescents' SRL through AI technologies remain unclear. Limited AI and disciplinary knowledge may affect the students' motivations, as explained by self-determination theory (SDT). In this study, we examine the mediating effects of needs satisfaction in SDT on the relationship between students' previous technical (AI) and disciplinary (English) knowledge and SRL, using an AI conversational chatbot. Data were collected from 323 9th Grade students through a questionnaire and a test. The students completed an AI basic unit and then learned English with a conversational chatbot for 5 days. Confidence intervals were calculated to investigate the mediating effects. We found that students' previous knowledge of English but not their AI knowledge directly affected their SRL with the chatbot, and that satisfying the need for autonomy and competence mediated the relationships between both knowledge (AI and English) and SRL, but relatedness did not. The self-directed nature of SRL requires heavy cognitive learning and satisfying the need for autonomy and competence may more effectively engage young children in this type of learning. The findings also revealed that current chatbot technologies may not benefit students with relatively lower levels of English proficiency. We suggest that teachers can use conversational chatbots for knowledge consolidation purposes, but not in SRL explorations.

Practitioner notes

What is already known about this topic

• Artificial intelligence (AI) technologies can potentially support students' self-regulated learning (SRL) of disciplinary knowledge through chatbots.
• Needs satisfaction in Self-determination theory (SDT) can explain the directive process required for SRL.
• Technical and disciplinary knowledge would affect SRL with technologies.

What this paper adds

• This study examines the mediating effects of needs satisfaction in SDT on the relationship between students' previous AI (technical) and English (disciplinary) knowledge and SRL, using an AI conversational chatbot.
• Students' previous knowledge of English but not their AI knowledge directly affected their SRL with the chatbot.
• Autonomy and competence were mediators, but relatedness was not.

Implications for practice and/or policy

• Teachers should use chatbots for knowledge consolidation rather than exploration.
• Teachers should support students' competence and autonomy, as these were found to be the factors that directly predicted SRL.
• School leaders and teacher educators should include the mediating effects of needs satisfaction in professional development programmes for digital education.