Learning analytics initiatives in Latin America: Implications for educational researchers,practitioners and decision makers

This study presents several Latin American research initiatives in the field of learning analytics (LA). The study’s purpose is to enhance awareness and understanding of LA among researchers, practitioners and decision makers, and to highlight the importance of supporting research on LA. We analyzed case studies of LA research conducted at four levels of the educational system (the national, institutional, classroom and student levels), which were implemented in four countries (Brazil, Ecuador, Mexico and Uruguay). Diversified cases were selected to demonstrate the use of LA in primary, secondary and higher education, and to allow the inclusion of different types of datasets. These cases also showed the development of legal frameworks for handling ethical issues, and they met the requirements for data privacy protection in Latin America. The study concludes with a discussion of the findings and their implications for further research and practice in the field of LA for teaching and learning.     

Adaptation and evaluation of a learning analytics dashboard to improve academic support at three Latin American universities

Despite the success of academic advising dashboards in several higher educational institutions (HEI), these dashboards are still under-explored in Latin American HEI's. To close this gap, three different Latin American universities adapted an existing advising dashboard, originally deployed at the KU Leuven to their own context. In all three cases, the context was the main ruling factor to these adaptations. In this paper, we describe these adaptions using a framework that focuses on four different elements of the context: Objectives, Stakeholders, Key moment and Interactions. Evaluation of the adapted dashboards in the three different Latin American universities is conducted through pilots. This evaluation shows the value of the dashboard approach in different contexts in terms of satisfaction, usefulness and impact in academic decision-making and advising tasks. The main contribution of this paper is the systematic reporting of the adaptations to an academic advising dashboard and showing the value of an academic advising dashboard on academic decision-making and advising tasks.     

Learning design and learning analytics in mobile and ubiquitous learning: A systematic review

Mobile and Ubiquitous Learning (m/u-learning) are finding an increasing adoption in education. They are often distinguished by hybrid learning environments that encompass elements of formal and informal learning, in activities that happen in distributed settings (indoors and outdoors), across physical and virtual spaces. Despite their purported benefits, these environments imply additional complexity in the design, monitoring and evaluation of learning activities. The research literature on learning design (LD) and learning analytics (LA) has started to deal with these issues. This paper presents a systematic literature review of LD and LA, in m/u-learning. Apart from providing an overview of the current research in the field, this review elicits elements of common ground between both communities, as shown by the similar learning contexts and complementary research contributions, and based on the research gaps, proposes to: address m/u-learning beyond higher education settings, reinforce the connection between physical and virtual learning spaces, and more systematically align LD and LA processes.     

A systematic review of the role of learning analytics in enhancing feedback practices in higher education

Learning analytics (LA) offers new opportunities to enrich feedback practices in higher education, but little is understood about the ways different LA can enhance feedback practices for educators and students. This systematic literature review maps the current state of implementation of LA to improve feedback practices in technology-mediated learning environments in higher education. We used strict inclusion criteria to select relevant studies that have investigated the role of LA on feedback practices. To identify common features of LA for feedback studies, we coded relevant publications using an analytical framework that identifies four key dimensions of LA systems: what (types of data), how (analytic methods), why (objectives), and how educators and students are served by LA (stakeholders). Based on findings, we propose a conceptual framework that can guide the implementation of LA for feedback systems and also suggest future empirical research in this area.     

Environmental education in Latin American and the Caribbean: the challenges and limitations of conducting a systematic review of evaluation and research

Abstract

This systematic review extends the boundary of prior reviews in the environmental education (EE) field by analyzing publications focused on Latin America and the Caribbean (LATAM). We examined peer-reviewed literature and dissertations/theses in four languages (English, French, Spanish and Portuguese) published between January 2000 and February 2018 to examine outcomes of EE interventions occurring in both formal and nonformal settings among LATAM youth up to age 18 years. Our findings reflect recurring critiques of EE, such as the field’s limited empirical research and a bias toward measuring knowledge gain. Our findings also highlight themes specific to LATAM, related to issue-based EE programs, EE versus Education for Sustainable Development, barriers to EE implementation and potential challenges posed by bias, privilege and power in LATAM-based EE research. Access to potentially appropriate publications was limited by weaknesses in search databases. Only a small selection of LATAM countries is represented in publications that met the search criteria, limiting our capacity to draw stronger conclusions about the region as a whole.     

Motivation beliefs of secondary school teachers in Canada and Singapore: A mixed methods study

A mixed methods approach was used to explore secondary teachers’ motivation beliefs in Canada and Singapore. Results from Study 1 revealed that socio-economic status (SES) was the strongest predictor of school climate in Canada, and that collective efficacy mediated the effect of SES on school climate in Singapore, but not in Canada. In Study 2, interviews were conducted with 10 teachers in Canada and 14 teachers in Singapore. Teachers in both settings discussed students’ social and behavior problems, but the range of the social problems was greater in Canada than in Singapore, and had a stronger impact on teachers’ motivation beliefs.     

Sources of math and science self-efficacy in rural Appalachia: A convergent mixed methods study

Rural living, particularly in economically distressed areas, may reduce students’ educational opportunities and alter their self-beliefs. According to social cognitive theory, the contexts in which people live influence how they feel about their capabilities. The purpose of this study was to examine the experiences that raise and lower the math and science self-efficacy of students living in a rural, high-poverty area in Central Appalachia. A convergent mixed methods design was used to examine quantitative and qualitative survey data from 673 students in Grades 6–12 who took part in a multi-year study on academic motivation (Year 1 = 511; Year 2 = 391; Year 3 = 418). In the quantitative phase, structural equation models showed that Year 1 mastery experience raised and physiological states lowered students’ math and science self-efficacy in Year 2. Deductive coding of students’ responses to 4 open-ended questions in Year 3 indicated that other sources were also salient and differed by domain, their effect on self-efficacy, and student gender. Integrative analyses showed that students consider information from multiple sources when judging their capabilities. This research extends findings related to the sources of self-efficacy to the understudied population of rural learners, the less-studied context of science, and to the factors that not only increase but decrease perceived efficacy.     

A meta-framework for conducting mixed methods impact evaluations: Implications for altering practice and the teaching of evaluation

Educational interventions are becoming increasingly more complex, far-reaching, and high-stakes. Thus, there is a need for an evaluation meta-framework that is comprehensive, flexible, and meets enhanced complexity. Therefore, we provide a new and comprehensive definition of impact evaluations—what we call a comprehensive impact evaluation—that draws out the importance of collecting and analyzing both quantitative and qualitative data, thereby resulting in a rigorous approach that can allow for strong inferences. Further, we provide an overview of impact evaluation designs that can be used in comprehensive impact evaluations along with a rationale for using both quantitative and qualitative research techniques. This leads to the central purpose of the article: to provide a meta-framework for conducting what we call a Mixed Methods Theory-Based Impact Evaluations, wherein mixed methods techniques are used at every phase of the process. Building on White’s (2009) work, we outline an 8-phase Mixed Methods Theory-Based Impact Evaluation.     

A comparative study of student attitude,learning and teaching practices in Pakistan and Britain

Good teaching is open to change and it involves constantly trying to find out what the effects of instruction are on learning, and modifying that instruction in the light of the evidence. Similarly, the ultimate outcome of instructional practice is effective student learning. Therefore, in this perspective, adequately meeting the varying needs of students is a major challenge for the modern education system in every country. As the title indicates, this paper attempts to explore students’ attitudes to studying and learning in order to help encourage greater engagement with quality learning. The study is based on primary data with regard to postgraduate education and learning in respective British and Pakistani education institutions—i.e. Moray House School of Education (Edinburgh University) and the Institute of Education and Research (University of the Punjab, Lahore). In the context of the collaboration of these two institutions, the paper focuses on the ways and means to enhance the system‐wide capacity for research‐based practice.     

Beginning teachers' experience of the workplace learning environment in alternative teacher certification programs: A mixed methods approach

Workplace learning in early entry (EE) teacher education programs has been proposed as an alternative to traditional programs and as bridging the theory-practice gap in teacher education. However, there is little empirical evidence for the underlying assumption that one can become a highly qualified teacher by merely being a teacher. This article discusses to what extent students of teaching in EE programs experience their work environment as a stimulating learning environment. The results of semi-structured interviews and an online survey suggest that schools tend to hinder rather than serve the purpose of workplace learning. Obstructions and supporting conditions are discussed.     

Exploring the effect of task instructions on topic beliefs and topic belief justifications: A mixed methods study

The purpose of this mixed methods study was to investigate whether task instructions that asked adolescents to evaluate the merit of both sides of a controversial issue would affect their topic beliefs and topic belief justifications after they read belief-consistent and belief-inconsistent information. In the quantitative phase, we conducted an experiment in which high school students (n = 45) were randomly assigned to one of four conditions and received their respective pre-reading task instructions. Quantitative analyses showed that task instructions affected topic beliefs and belief justifications. However, inspection of topic belief scores within each condition indicated that some individuals’ beliefs became weaker, whereas others’ became stronger. In the qualitative phase, we conducted interviews to explain why this occurred. The interview data revealed two distinct reader profiles: belief-reflection and belief-protection. The data sets were complementary: the quantitative data indicated group differences in topic beliefs and belief justifications, and the qualitative data allowed us to explain differences within and across groups.     

Effectiveness and relevance of feedback in Higher Education: A study of undergraduate students

This paper draws upon a wider study on assessment in higher education. It focuses on students’ perceptions of the effectiveness and relevance of feedback in regard to assessment methods and self-regulation of learning. In total, 605 undergraduates participated in the study from five Portuguese public universities. Data were collected through questionnaires with open and closed-ended questions. Results revealed that feedback is perceived as more relevant, effective and in a more positive way by students who are assessed by learner-centred methods than by those assessed by traditional methods. Also, participants who are assessed by learner-centred methods or mixed methods perceived feedback as more effective in all phases of self-regulation learning than students who are assessed by traditional methods. Implications of the findings for feedback and assessment in Higher Education are discussed.     

Validity evidence for the Experiences of Teaching and Learning Questionnaire (ETLQ) in evaluations of quality learning: A systematic critical literature review

The use of the Experiences of Teaching & Learning Questionnaire (ETLQ) for the evaluation of learning quality in higher education has been expanding during the last decade, thus a review of the instrument’s validity evidence is warranted. The design of the study was a systematic critical literature review. We evaluated the strength of the validity evidence of 17 included studies with a quality appraisal framework reflecting current standards for educational testing. The evidence supporting the central validity assumptions of the ETLQ scales is currently weak to moderate and incomplete. Thus, caution against the uncritical use of ETQL scores for high-stakes educational decisions is warranted. The appraisal framework used was useful for creating an overview of the evidence. However, attention to more general aspects of study quality, and consensus deliberations with three to four raters was also important for sufficiently reliable appraisal of the evidence.     

Explaining discrepancies between teacher beliefs and teacher interventions in a problem-based learning environment: A mixed methods study

The purpose of this study was to explore the discrepancy between teacher beliefs and behavior in a Problem-Based Learning (PBL) environment. Using a survey and observations, this study demonstrated that tutors prefer learner-oriented beliefs, but in their teacher behavior they showed a more traditional approach to teaching. Analysis of semi-structured interviews indicated that this inconsistency could be attributed to the way in which problem-based learning is embedded in the curriculum, the confidence teachers have in the self-directed capabilities of students, and the self-confidence of teachers regarding their own facilitation skills.     

The evaluation of school-university partnerships that improve teaching and learning practices: A systematic review

This paper presents a systematic literature review of School-University Partnerships (SUPs) that aim to promote changes in teaching and learning practices. Our analysis focuses on the evaluation of such SUPs, throughout their development process (from Exploration to Implementation and Sustainability). Our results highlight the nonlinearity of such development processes. The evaluation of the SUPs mainly relied on qualitative approaches and occurred primarily during the Implementation phase. Furthermore, the evaluation instruments used have seldom been validated or reused within the research community. Future research can focus on the development of reliable evaluation instruments, useable throughout the lifecycle of SUPs, to guide their decision-making.     

School context and instructional capacity: A comparative study of professional learning communities in rural and urban schools in China

This study examines the teaching gap between rural and urban schools in China from the perspective of teacher professional learning communities (PLCs). Drawing on in-depth interviews with 36 primary school teachers, the study finds striking disparities between rural and urban schools in the working of Teaching and Research Groups (TRGs). These disparities in TRGs result in divergent patterns of instructional capacity building in rural and urban schools. The evidence shows that teaching and teachers are strongly shaped by the school organizational context. It suggests that strengthening school-wide PLCs is an important way of narrowing the rural-urban teaching and learning gaps.