Towards learning analytics adoption: A mixed methods study of data-related practices and policies in Latin American universities

In Latin American universities, Learning Analytics (LA) has been perceived as a promising opportunity to leverage data to meet the needs of a diverse student cohort. Although universities have been collecting educational data for years, the adoption of LA in this region is still limited due to the lack of expertise and policies for processing and using educational data. In order to get a better picture of how existing data-related practices and policies might affect the incorporation of LA in Latin American institutions, we conducted a mixed methods study in four Latin American universities (two Chilean and two Ecuadorian). In this paper, the qualitative data were based on 37 interviews with managers and 16 focus groups with 51 teaching staff and 45 students; the quantitative data were collected through two surveys answered by 1884 students and 368 teachers, respectively. The findings reveal opportunities to incorporate LA services into existing data practices in the four case studies. However, the lack of reliable information systems and policies to regulate the use of data imposes challenges that need to be overcome for future LA adoption.     

Evaluation of a learning analytics tool for supporting teachers in the creation and evaluation of accessible and quality open educational resources

The learning analytics (LA) field seeks to analyze data about students’ interactions, and it has been applied in the development of tools for supporting both learning and teaching processes. Recent research has paid attention on how LA may benefit teachers in the creation of educational resources. However, most of the research on LA solutions is carried out to support teachers in analyzing students’ behavior data collected as they interact with virtual learning environments. While solutions to support teachers in different virtual learning and teaching contexts have become important, to date little research has been done on how LA solutions can help teachers to create and evaluate Open Educational Resources (OERs). This study aims at presenting the evaluation of a LA tool for supporting teachers in the creation and evaluation of accessible and quality OERs considering that both processes fall within the competences that teachers can acquire and strengthen by participating as authors (creation) and evaluators (evaluation) of OERs. The study was conducted with Colombian teachers and the results obtained highlight the positive effect the tool had on the teachers’ acquisition of the competences and the positive attitude they had toward using the tool.     

Ethical issues in learning analytics: a review of the field

Learning analytics (LA) collects, analyses, and reports big data about learners to optimise learning. LA ethics is an interdisciplinary field of study that addresses moral, legal, and social issues; therefore, institutions are responsible for implementing frameworks that integrate these topics. Many of the ethical issues raised apply equally to educational data sets of any size. However, in this study, we focus on big data that increases the scale and granularity of data gathered. The purpose of this study is twofold: (a) to critically review the published (2011–2018) scientific literature on LA ethics issues and (b) to identify current trends and answer research questions in the field. This study’s research questions are as follows: what is essential in LA ethics for key educational stakeholders, and what should a proposed checklist for LA ethics include for specific educational stakeholders? After systematically searching online bibliographic databases, journals, and conferences, a literature review identified 53 articles from a sample of 562. The selected articles, based on critical and qualitative content analysis, were exhaustively analysed. The findings demonstrate the shortage of empirical evidence-based guidelines on LA ethics and highlight the need to establish codes of practices to monitor and evaluate LA ethics policies. Finally, this work proposes a useful checklist as an instructional design model for scholars, policymakers, and instructional designers, so that trusted partners may use LA responsibly to improve teaching and learning.

     

Coordinating learning analytics policymaking and implementation at scale

Many Latin-American institutions recognise the potential of learning analytics (LA). However, the number of actual LA implementations at scale remains limited, notwithstanding considerable effort made to formulate guidelines and frameworks to support the LA policy development. Guidance on how to coordinate the interaction between the LA policymaking and implementation is mostly missing, leaving a difficult challenge up to practitioners. In this study we propose a coordination model to support future LA initiatives at scale. We explore the problem by comparing two cases in Belgium and Ecuador. Following up we use the LA implementation timeline as a driver for planning the interaction between the policymaking and implementation. We continue by testing an application of the model with LA experts predominantly from Latin-American institutions, asking them to map low-level items of the SHEILA policy framework to four implementation phases. The results of this mapping support that LA policy building can be spread over time, that it can coincide with LA implementation at scale, and that both efforts can be coordinated. It is hoped that this study will provide additional guidance for future Latin-American and other LA initiatives.     

A systematic review of the role of learning analytics in enhancing feedback practices in higher education

Learning analytics (LA) offers new opportunities to enrich feedback practices in higher education, but little is understood about the ways different LA can enhance feedback practices for educators and students. This systematic literature review maps the current state of implementation of LA to improve feedback practices in technology-mediated learning environments in higher education. We used strict inclusion criteria to select relevant studies that have investigated the role of LA on feedback practices. To identify common features of LA for feedback studies, we coded relevant publications using an analytical framework that identifies four key dimensions of LA systems: what (types of data), how (analytic methods), why (objectives), and how educators and students are served by LA (stakeholders). Based on findings, we propose a conceptual framework that can guide the implementation of LA for feedback systems and also suggest future empirical research in this area.     

Learning design and learning analytics in mobile and ubiquitous learning: A systematic review

Mobile and Ubiquitous Learning (m/u-learning) are finding an increasing adoption in education. They are often distinguished by hybrid learning environments that encompass elements of formal and informal learning, in activities that happen in distributed settings (indoors and outdoors), across physical and virtual spaces. Despite their purported benefits, these environments imply additional complexity in the design, monitoring and evaluation of learning activities. The research literature on learning design (LD) and learning analytics (LA) has started to deal with these issues. This paper presents a systematic literature review of LD and LA, in m/u-learning. Apart from providing an overview of the current research in the field, this review elicits elements of common ground between both communities, as shown by the similar learning contexts and complementary research contributions, and based on the research gaps, proposes to: address m/u-learning beyond higher education settings, reinforce the connection between physical and virtual learning spaces, and more systematically align LD and LA processes.     

Action research approach on mobile learning design for the underserved

This paper discusses an action research study focused on developing a mobile learning model of literacy development for underserved migrant indigenous children in Latin America. The research study incorporated a cyclical action model with four distinctive stages (Strategize, Apply, Evaluate, and Reflect) designed to guide constituencies involved in the study to design, test, and enhance a mobile learning model. The findings, to date, reveal some of the contextual phenomena that create both challenges and opportunities for a mobile learning model. From this, design strategies are evolving focused on sustained literacy exposure for extremely marginalized (economically, educationally, geographically, and technologically) migrant indigenous children who have no consistent access to a formal education system.     

The effect of curricular activities on learner autonomy: the perspective of undergraduate mechanical engineering students

This study researches how first-year engineering students perceived the influence of curricular activities on their own learning autonomy, measured with an adaptation of the Personal Responsibility Orientation to Self-direction in Learning Scale (PRO-SDLS). Participants were questioned to assess the influence of the teacher's role. The results indicate that learners’ characteristics (motivation and self-efficacy) contribute more to learner autonomy (LA) than the teaching–learning transaction (control and initiative), as in the original PRO-SDLS validation. The most autonomous learners presented higher values in all LA components and dimensions, but the differences were greater in motivation and initiative. The participants with higher LA were not as dependent on the teacher, regarding assessment, the completion of classroom tasks and deadlines. Regardless of the degree of autonomy in learning, all participants viewed teachers as the main source of information. Therefore, LA plays an important role in teaching activities planning. Suggestions for adjustments and more flexible learning scenarios are formulated.     

Foreign language learning difficulties in Italian children: are they associated with other learning difficulties?

A group of seventh- and eighth-grade Italian students with low achievement (LA) in learning English as a foreign language (FL) was selected and compared to a group with high achievement (HA) in FL learning. The two groups were matched for age and nonverbal intelligence. Two experiments were conducted to examine the participants' verbal and nonverbal learning skills, such as native language reading accuracy, speed and comprehension, calculation, and attention and self-regulation. Both experiments showed that the LA group seemed at risk for reading comprehension difficulties, but its reading speed and accuracy were within the average range according to Italian norms. The results also excluded the possibility that FL learning difficulties of LA participants could be associated with a deficit in calculation. Furthermore, according to teachers' ratings, children with LA appeared at risk for attention-deficit disorder (ADD). The pattern of learning difficulties of seventh- and eighth-grade participants with LA appeared to be not completely comparable with that of high school students at risk of FL learning difficulties as described in the literature.     

院校研究与学习分析

院校研究与学习分析在学习研究方面同异并存,皆利用大数据探寻学生学习的优劣.院校研究主要依赖大学的管理和调查数据,学习分析主要使用学习管理系统收集的数据.院校研究聚焦于帮助大学“知己知彼”,学习分析旨在助力“知学生”“知学习”“知教育”.院校研究为学校层面决策提供实证研究证据,学习分析识别学生个体学习行为从而支持个性化学...     

Learning achieved in structured online debates: levels of learning and types of postings

The purpose of this study was to examine the learning process exhibited in restrained online debates in terms of to what extent each of Bloom’s six levels of cognitive learning were exhibited among four types of message (argument, critique, evidence, and explanation). Thirty-three graduate students enrolled in an online entry-level course in distance learning participated in this study. The threaded discussion postings from four structured online debates were analyzed. The results indicated that five of the six levels of cognitive learning (except for knowledge learning) were achieved to a certain extent among the four types of postings. Chi square tests indicated that higher levels of learning were most likely to be exhibited in critique and argument postings. Message–response exchanges ending with critiques or starting with argument messages were most likely to elicit higher levels responses. The results were discussed in light of better understanding of student learning and implications for instructional designs of discussion topics and restraints.     

Early Cognitive Precursors of Children's Mathematics Learning Disability and Persistent Low Achievement: A 5-Year Longitudinal Study

Mathematical difficulties have been distinguished as mathematics learning disability (MLD) and persistent low achievement (LA). Based on 1,880 Finnish children who were followed from kindergarten (age 6) to fourth grade, this study examined the early risk factors for MLD and LA. Distinct groups of MLD (6.0% of the sample) and LA (25.7%) children were identified on the basis of their mathematics performance between first and fourth grades with latent class growth modeling. Impairment in the same set of cognitive skills, including language, spatial, and counting skills, was found to underlie MLD and LA. The finding highlights the importance of monitoring mathematical development across the early grades and identifying early cognitive precursors of MLD and LA for screening and intervention efforts.     

Science Education Research Trends in Latin America

The purpose of this study was to survey and report on the empirical literature at the intersection of science education research in Latin American and previous studies addressing international research trends in this field. Reports on international trends in science education research indicate that authors from English-speaking countries are major contributors of publications. Despite the internationalization of the science education community, as represented by the publication of the work generated in non-English-speaking countries, research trends in science education in Latin America are uncommon in the literature. Therefore, it was deemed important to explore the scholarly productivity of science education researchers from Latin America to learn about the research orientations pursued by scholars from this region. Collective review findings are presented with respect to author’s nationality, publication volume generated in each country, research type and topic, collaborative research, and areas for future research. Of the ten countries represented in this study, Brazilian authors were the most research active scholars followed by their colleagues from Venezuela, Mexico, and Argentina. The History, Philosophy and the Nature of Science (HPNOS) was the topic that most attracted the interest of Latin American science education researchers, and the Empirical Qualitative studies was the most frequent research type combination in the analyzed publications. Findings in this study suggest a relationship between investment power in Research and Development (R&D) and the scholarly productivity not only in science education but also in the scientific field in the countries of the region.     

LD status and achievement: confounding variables in the study of children's social status, self-esteem, and behavioral functioning

The purpose of the present study was to examine the role of achievement in explaining the poor social and behavioral functioning associated with LD status, and to evaluate potential gender differences in patterns of interpersonal functioning among youth with learning disabilities (LD) and nondisabled (NLD) youth. Thirty-two students with learning disabilities (21 boys, 11 girls) were matched with same-sex, same-race classmates whose reading achievement was low (LA) or average (AA), and these groups were compared on peer ratings of liking and disliking, perceptions of self-worth and social acceptance, and teacher ratings of conduct problems, anxiety-withdrawal, and attention problems. Students with learning disabilities were less accepted and less well-liked than children in the LA or AA groups and also perceived their self-worth and social acceptance to be lower than LA or AA students. Group by Sex interactions were apparent for several of the peer rating and behavioral variables, indicating that different patterns of social and behavioral functioning distinguished LD boys and LD girls from their NLD peers. The findings highlight the potential role of low achievement in peers' dislike of LD girls and suggest the importance of investigating well-defined subgroups of youth with LD in future research.     

More than science: family learning in a Mexican science museum

Latin American audiences living in their countries of origin are poorly understood as museum learners due to the scarcity of research in this field. Through a case study approach, I investigate and report on the ways of learning of 20 Mexican family groups. In particular, I examine the influence of the Mexican sociocultural context on the participant family members’ learning outcomes from a Mexican science museum. Conducted in Universum Museo de las Ciencias, a science museum located in Mexico City, this research study is based on the premise that understanding the role of the sociocultural elements of learning is essential to understanding the nature of learning in museums. The cognitive and social outcomes of the participants are discussed in the light of the sociocultural elements that define Mexicans as museum learners.     

The effects of peer assessment on learner autonomy: An empirical study in a Chinese college English writing class

The study reports on a one-semester-long intervention study of peer assessment in a college English writing class. The purpose of the study was to investigate the effects of peer assessment on learner autonomy (LA). Seventy English major sophomores from an independent college in China participated in the study, who were randomly divided into two groups of 35 students each. Traditional teacher feedback was used in the control group while peer assessment was adopted for the experimental group. A questionnaire on LA was administered to both groups as the pre-test and the post-test. A number of ANCOVA analyses were run to measure the effects of peer assessment on students’ LA. The results indicate that peer assessment enhanced the students’ learner autonomy. Peer assessment significantly reduced learners’ dependence on the teacher and boosted the students’ confidence in learning ability, while failing to make noticeable improvement in the other aspects of LA.     

The validity and reliability of the cross-national comparison of degree programme levels in European countries. What have students learnt?

A cross-national comparison of degree programme levels became relevant when the borders of European countries opened for students and graduates, and higher education institutions were restructured into bachelor’s and master’s programmes. This new situation foregrounded the questions of what students are learning in the degree programmes of European countries and how to compare their achievements. Therefore, we conceptualised a valid and reliable ‘level’ construct that included a cognitive (‘disciplinary thinking’) and an affective aspect (‘professional attitude’). The main research question for our exploratory study was: ‘What procedure can lead to a valid and reliable cross-national comparison of degree programme levels?’ To achieve this comparison, we designed a Three-Step Procedure, in which level was operationalised (step 1), measured and analysed (step 2), and compared cross-nationally (step 3). The study was conducted in collaboration with four bachelor programmes in Hotel Management from four European countries; a total of 783 participants were involved. Four themes were generated to operationalise the concept of level: professional management, hospitality business research, leading management, and strategic management; their respective learning outcomes were measured with a questionnaire. Principal component analysis identified the conceptualised themes and measured their components with eigenvalues ≥1, which explained 66 % of the variance. The reliability of the components exceeded a Cronbach’s alpha coefficient of 0.70. Analysis of the components and of the single samples showed strong validity and reliability for the learning outcomes. Thus, we believe this study has produced a rigorous means to compare degree programme levels across countries.

     

Application of learning analytics using clustering data Mining for Students’ disposition analysis

Learning Analytics (LA) is an emerging field in which sophisticated analytic tools are used to improve learning and education. It draws from, and is closely tied to, a series of other fields of study like business intelligence, web analytics, academic analytics, educational data mining, and action analytics. The main objective of this research work is to find meaningful indicators or metrics in a learning context and to study the inter-relationships between these metrics using the concepts of Learning Analytics and Educational Data Mining, thereby, analyzing the effects of different features on student’s performance using Disposition analysis. In this project, K-means clustering data mining technique is used to obtain clusters which are further mapped to find the important features of a learning context. Relationships between these features are identified to assess the student’s performance.     

Factors that influence learning during a scientific field trip in a natural environment

This study deals with the educational effectiveness of field trips. The main purpose was to obtain insight concerning factors that might influence the ability of students to learn during a scientific field trip in a natural environment. The research was conducted in the context of a 1-day geologic field trip by 296 students in Grades 9 through 11 in high schools in Israel. The study combined qualitative and quantitative research methods. Data were collected from three different sources (student, teacher, and outside observer) in three stages (before, after, and during the field trip). Using observations and questionnaires we investigated: a) the nature of student learning during the field trip, b) student attitudes toward the field trip, and c) changes in student knowledge and attitudes after the field trip. Our findings suggest that the educational effectiveness of a field trip is controlled by two major factors: the field trip quality and the “Novelty space” (or Familiarity Index). The educational quality of a field trip is determined by its structure, learning materials, and teaching method, and the ability to direct learning to a concrete interaction with the environment. The novelty space consists of three prefield variables: cognitive, psychological, and geographic. The learning performance of students whose “Novelty Space” was reduced before the field trip was significantly higher than that of students whose “Novelty Space” had not been so reduced. Thus, the former group gained significantly higher achievement and attitude levels. It is suggested that a field trip should occur early in the concrete part of the curriculum, and should be preceded by a relatively short preparatory unit that focuses on increasing familiarity with the learning setting of the field trip, thereby limiting the “Novelty Space” factors.