Improving learning achievements,motivations and problem-solving skills through a peer assessment-based game development approach

In this study, a peer assessment-based game development approach is proposed for improving students’ learning achievements, motivations and problem-solving skills. An experiment has been conducted to evaluate the effectiveness of the proposed approach in a science course at an elementary school. A total of 167 sixth graders participated in the experiment, 82 of whom were assigned to the experimental group and learned with the peer assessment-based game development approach, while 85 students were in the control group and learned with the conventional game development approach. From the empirical results, it was found that the proposed approach could effectively promote students’ learning achievement, learning motivation, problem-solving skills, as well as their perceptions of the use of educational computer games. Moreover, it was found from the open-ended questions that most of the students perceived peer assessment-based game development as an effective learning strategy that helped them improve their deep learning status in terms of “in-depth thinking,” “creativity,” and “motivation.”     

The role of learning theory in multimodal learning analytics

This study presents the outcomes of a semi-systematic literature review on the role of learning theory in multimodal learning analytics (MMLA) research. Based on previous systematic literature reviews in MMLA and an additional new search, 35 MMLA works were identified that use theory. The results show that MMLA studies do not always discuss their findings within an established theoretical framework. Most of the theory-driven MMLA studies are positioned in the cognitive and affective domains, and the three most frequently used theories are embodied cognition, cognitive load theory and control–value theory of achievement emotions. Often, the theories are only used to inform the study design, but there is a relationship between the most frequently used theories and the data modalities used to operationalize those theories. Although studies such as these are rare, the findings indicate that MMLA affordances can, indeed, lead to theoretical contributions to learning sciences. In this work, we discuss methods of accelerating theory-driven MMLA research and how this acceleration can extend or even create new theoretical knowledge.

Practitioner notes

What is already known about this topic

• Multimodal learning analytics (MMLA) is an emerging field of research with inherent connections to advanced computational analyses of social phenomena.
• MMLA can help us monitor learning activity at the micro-level and model cognitive, affective and social factors associated with learning using data from both physical and digital spaces.
• MMLA provide new opportunities to support students' learning.

What this paper adds

• Some MMLA works use theory, but, overall, the role of theory is currently limited.
• The three theories dominating MMLA research are embodied cognition, control–value theory of achievement emotions and cognitive load theory.
• Most of the theory-driven MMLA papers use theory ‘as is’ and do not consider the analytical and synthetic role of theory or aim to contribute to it.

Implications for practice and/or policy

• If the ultimate goal of MMLA, and AI in Education in general, research is to understand and support human learning, these studies should be expected to align their findings (or not) with established relevant theories.
• MMLA research is mature enough to contribute to learning theory, and more research should aim to do so.
• MMLA researchers and practitioners, including technology designers, developers, educators and policy-makers, can use this review as an overview of the current state of theory-driven MMLA.

     

Learning problem-solving through making games at the game design and learning summer program

Today’s complex and fast-evolving world necessitates young students to possess design and problem-solving skills more than ever. One alternative method of teaching children problem-solving or thinking skills has been using computer programming, and more recently, game-design tasks. In this pre-experimental study, a group of middle school students (n = 18) with an age average of 12.6 attended a game-design summer program for 10 days. Students were assessed in their problem-solving skills, specifically in system analysis and design, decision-making, and troubleshooting domains, at the beginning and end of the program. The results indicated that there were significant improvements in students’ problem-solving skills after attending the summer program, Wilks’ Λ = .258, F (3, 15) = 14.397, p < .001, η ² = .742. For system analysis and design, and decision-making follow-up t-tests pointed to large and medium effect sizes, while for troubleshooting the gains were not significant. This study is a contributes to the growing body of literature investigating the benefits of designing games for young children by adding that game-design activities can be suitable venues for young children to learn and practice problem-solving skills.     

Exploring autonomous learning capacity from a self-regulated learning perspective using learning analytics

Practising self-regulated learning (SRL) has been proposed to develop learning autonomy. However, there is lack of empirical evidence on how SRL strategies affect autonomous learning capacity. This study attempts to bridge that gap by utilizing the learners’ trace data for measuring the learners’ autonomous interactions, and investigates the effects of four SRL strategies on learners’ autonomous choices. The goal is to explain how the employed SRL strategies impact autonomous control (in terms of frequencies of self-enforced decisions, as well as time-spent on decision making). The results from an exploratory study with undergraduate learners (N = 113) shown that goal-setting and time-management have strong positive effects on autonomous control, effort-regulation moderately positively affects learners’ autonomy, while help-seeking has a strong negative effect. These findings provide empirical evidence and contribute to clarifying the role of each one of the SRL strategies in the development of autonomous learning capacity, from a learning analytics perspective. Limitations and potential implications for research and practice are also discussed.     

An overview of learning analytics

Learning analytics, the analysis and representation of data about learners in order to improve learning, is a new lens through which teachers can understand education. It is rooted in the dramatic increase in the quantity of data about learners and linked to management approaches that focus on quantitative metrics, which are sometimes antithetical to an educational sense of teaching. However, learning analytics offers new routes for teachers to understand their students and, hence, to make effective use of their limited resources. This paper explores these issues and describes a series of examples of learning analytics to illustrate the potential. It argues that teachers can and should engage with learning analytics as a way of influencing the metrics agenda towards richer conceptions of learning and to improve their teaching.     

Ethical issues in learning analytics: a review of the field

Learning analytics (LA) collects, analyses, and reports big data about learners to optimise learning. LA ethics is an interdisciplinary field of study that addresses moral, legal, and social issues; therefore, institutions are responsible for implementing frameworks that integrate these topics. Many of the ethical issues raised apply equally to educational data sets of any size. However, in this study, we focus on big data that increases the scale and granularity of data gathered. The purpose of this study is twofold: (a) to critically review the published (2011–2018) scientific literature on LA ethics issues and (b) to identify current trends and answer research questions in the field. This study’s research questions are as follows: what is essential in LA ethics for key educational stakeholders, and what should a proposed checklist for LA ethics include for specific educational stakeholders? After systematically searching online bibliographic databases, journals, and conferences, a literature review identified 53 articles from a sample of 562. The selected articles, based on critical and qualitative content analysis, were exhaustively analysed. The findings demonstrate the shortage of empirical evidence-based guidelines on LA ethics and highlight the need to establish codes of practices to monitor and evaluate LA ethics policies. Finally, this work proposes a useful checklist as an instructional design model for scholars, policymakers, and instructional designers, so that trusted partners may use LA responsibly to improve teaching and learning.

     

Collaborative learning and teaching: Reflections on a facilitated journey

Abstract

This report summarises the first year of an innovative and multilayered teaching and learning collaborative experience within the context of an undergraduate unit in early childhood. The facilitative influence is two‐fold: (a) generic information skills training (GIST) project which is a joint initiative between the Institute of Early Childhood (IEC), the Library and the Centre for Flexible Learning to teach information literacy skills which are contextualised within the unit presentation mode, its requirements and assessment tasks, and (b) redesign of the unit from a former third‐year level to a first‐year level to incorporate the notion of problem‐based, collaborative teaching and learning. Many challenges were faced by those involved in the unit in its first year, mainly class size and heterogeneity of student levels, incorporation of technology into learning and teaching, and flow of information for smooth operations for students, the teaching team and Library staff. Multilevel collaborations were central to this experience, and examples of student‐student, student‐teacher, and teacher‐staff collaborations are provided. The first of the 2‐year cycle of the GIST project has generated substantive data, as has the implementation of the unit in an innovative format. The complexity and challenges presented in the unit seemed to account for student withdrawal of about 25% in the first month; final evaluations are obtained from 292 students. Survey results presented here are both from the withdrawn students, as well as those who completed the unit. Modifications to the delivery of the unit in its second year and ways of strengthening the communities of learners and teachers towards a common goal are briefly reviewed.     

Towards mapping competencies through learning analytics: real-time competency assessment for career direction through interactive simulation

Abstract

The selection of career paths and making of academic choices is a difficult and often confusing task for young people. The impact on their lives, however, is enormous as it can determine entire future career possibilities. In India, a general remedy to this stress is that instead of choosing a field of study tailored to individual preferences and strengths, topics are chosen that align with the choices of the students’ families or their friends. This can have the effect of entrenching patterns of intergenerational inequity. The aim of this research is to give students greater access to the knowledge capital which will help them make better choices. This is achieved by engaging students in the career planning process, in order to convey information in a likeable and credible way. The COMPCAT (Competency and Career Assessment Tool) game engine combines the use of learning analytics and real time, interactive computer simulations designed to gain insights into the students’ engagement in the making of these complex decisions. This paper presents the conceptual architecture of the game and demonstrates its role in enhancing the learning effectiveness of the students.     

Formative assessment strategies for students' conceptions—The potential of learning analytics

Formative assessment is considered to be helpful in students' learning support and teaching design. Following Aufschnaiter's and Alonzo's framework, formative assessment practices of teachers can be subdivided into three practices: eliciting evidence, interpreting evidence and responding. Since students' conceptions are judged to be important for meaningful learning across disciplines, teachers are required to assess their students' conceptions. The focus of this article lies on the discussion of learning analytics for supporting the assessment of students' conceptions in class. The existing and potential contributions of learning analytics are discussed related to the named formative assessment framework in order to enhance the teachers' options to consider individual students' conceptions. We refer to findings from biology and computer science education on existing assessment tools and identify limitations and potentials with respect to the assessment of students' conceptions.

Practitioner notes

What is already known about this topic

• Students' conceptions are considered to be important for learning processes, but interpreting evidence for learning with respect to students' conceptions is challenging for teachers.
• Assessment tools have been developed in different educational domains for teaching practice.
• Techniques from artificial intelligence and machine learning have been applied for automated assessment of specific aspects of learning.

What does the paper add

• Findings on existing assessment tools from two educational domains are summarised and limitations with respect to assessment of students' conceptions are identified.
• Relevent data that needs to be analysed for insights into students' conceptions is identified from an educational perspective.
• Potential contributions of learning analytics to support the challenging task to elicit students' conceptions are discussed.

Implications for practice and/or policy

• Learning analytics can enhance the eliciting of students' conceptions.
• Based on the analysis of existing works, further exploration and developments of analysis techniques for unstructured text and multimodal data are desirable to support the eliciting of students' conceptions.

     

The effectiveness of learning analytics for identifying at-risk students in higher education

Abstract

In this article we investigate the effectiveness of learning analytics for identifying at-risk students in higher education institutions using data output from an in-situ learning analytics platform. Amongst other things, the platform generates ‘no-engagement’ alerts if students have not engaged with any of the data sources measured for 14 consecutive days. We tested the relationship between these alerts and student outcomes for two cohorts of first-year undergraduate students. We also compared the efficiency of using these alerts to identify students at risk of poorer outcomes with the efficiency of using demographic data, using widening participation status as a case study example. The no-engagement alerts were found to be more efficient at spotting students not progressing and not attaining than demographic data. In order to investigate the efficacy of learning analytics for addressing differential student outcomes for disadvantaged groups, the team also analysed the likelihood of students with widening participation status generating alerts compared with their non-widening participation counterparts. The odds of students with widening participation status generating an alert were on average 43% higher, demonstrating the potential of such a system to preferentially target support at disadvantaged groups without needing to target directly based on immutable factors such as their socio-economic background.     

Improving workplace-based assessment and feedback by an E-portfolio enhanced with learning analytics

Electronic portfolios (E-portfolios) are crucial means for workplace-based assessment and feedback. Although E-portfolios provide a useful approach to view each learner’s progress, so far options for personalized feedback and potential data about a learner’s performances at the workplace often remain unexploited. This paper advocates that E-portfolios enhanced with learning analytics, might increase the quality and efficiency of workplace-based feedback and assessment in professional education. Based on a 5-phased iterative design approach, an existing E-portfolio environment was enhanced with learning analytics in professional education. First, information about crucial professional activities for professional domains and suited assessment instruments were collected (phase 1). Thereafter probabilistic student models were defined (phase 2). Next, personalized feedback and visualization of the personal development over time were developed (phase 3). Then the prototype of the E-portfolio—including the student models and feedback and visualization modules—were implemented in professional training-programs (phase 4). Last, evaluation cycles took place and 121 students and 30 supervisors from five institutes for professional education evaluated the perceived usefulness of the design (phase 5). It was concluded that E-portfolios with learning analytics were perceived to assist the development of students’ professional competencies and that the design is only successful when developed and implemented through the eyes of the users. Feedback and assessment methods based upon learning analytics can stimulate learning at the workplace in the long run. Practical, technological and ethical challenges are discussed.     

The compatibility of action learning with inner game coaching

Using ‘inner game’ coaching techniques in the remediation of a challenged programme at a Global Investment Bank the environment was transformed into a delivery focused culture. The techniques included group sessions that would be familiar to anyone aware of action learning and were an integral part of the strategy to ensure sustainable change was achieved, with the programme delivered to an agreed plan and under budget.     

Using a computer simulation to compare expert/novice problem-solving subroutines

Hierarchical problem-solving strategies employed in solving exercise science problems were examined in this study, which also tested the validity of an educational computer simulation. Hypothesis testing was used as the theoretical base for the study of differences in problem-solving within the computer simulation. In a previous study two groups of undergraduate (novices) and graduate students were compared in their ability to solve exercise science problems. The present study added a group of faculty (experts) who were presented with the same simulation protocol as the other subjects. Protocol analysis and the Pitt coding system were used to analyse verbal data. Group differences were examined statistically. The faculty were superior in interpreting data and used the Basic Heuristic and Pattern Extraction strategies for the generation and use of algorithms. The problem-solving strategies varied for each group based on the perceived difficulty of the problem, the knowledge base available, and the similarity of the given problem to previous problems.