Computer-aided authoring of assessment instruments: An activity-theoretical approach

Abstract

Various researches have been conducted on the role and importance of assessment in education as well as its impact on the learner and the overall learning process. In fact, the way assessment is formulated in a particular subject shapes the way students learn. They focus their learning to comply with assessment requirements that they anticipate. In this article, the study is focused on the written examination papers (teacher-made tests) that are normally prepared at the end of a semester or an academic year to assess students of secondary and tertiary levels. The study also investigates how well papers are set and balanced according to the cognitive levels defined by Bloom (1956) and the learning outcomes/objectives as defined for the subjects. A collaborative process model as a framework for the design of such tests that can enhance the evaluation process is proposed. A brief argument is made for a case for a computer-supported collaborative environment to implement such a framework and which is based on activity theory. Such a framework is implemented in the form of MYSTIC; a collaborative authoring software for assessment instruments. The software allows stand-alone as well as collaborative authoring of examination papers and also helps academics' decision-making concerning the examination paper balancing and moderating process by graphically displaying and comparing marks allocated per question paper against the learning objectives     

Investigating agentive urban learning: an assembly of situated experiences for sustainable futures

ABSTRACT

In this article we explore the dynamic between the pedagogical and the urban, attending to ‘agentive urban learning’. By this we mean processes by which young people build agency in the urban context, in using the resources of the city to develop their own agency, and of developing agency to act within the city. By agency, we refer to the capacity to imagine and act to create individual and collective futures. Our interest is how young people develop such agentive urban learning themselves and how it might be enhanced pedagogically at school and university. Three case studies explore different facets—the first how young people themselves develop this agency in situated settings and the tools that they use to reflect upon the future; the second how digital tools might be used to enhance students’ understanding of the city as a site of change, in this instance, climate change; and the third how such agency might be developed collectively in partnership with other city dwellers. We conclude that a diversity of students’ engagement in urban contexts of learning offers ways from which to further investigate how identity, setting, and stakeholder relationships matter as part of potentially sustainable agentive learning futures.     

Teacher facilitating of group learning in science with digital technology and insights into students’ agency in learning to learn

ABSTRACT

The study provides an insight into how teachers may facilitate students’ group learning in science with digital technology, which was examined when Norwegian lower secondary school students engaged in learning concepts of mitosis and meiosis. Quantitative and qualitative analyses of the teacher’s assistance draw on Galperin’s conceptualisation of learning.

Findings reveal patterns in the teacher’s guidance: the teacher fulfilled the orienting, executive and controlling functions while assisting students in identifying the key features of mitosis and meiosis and solving the compare and contrast task. The teacher relied on and interplayed with the available mediational resources: compare and contrast task, digital animations, and collaborating peers. However, it was the compare and contrast task that demonstrated an approach to study scientific concepts which may have contributed to the development of learners’ understanding about to engage in learning in science. By adopting such an approach, learning activity has the potential to not only help students to achieve learning outcomes but it acquires a functional significance, becoming a tool in the learning process aimed at the development of students’ as learners. The digital animations, in turn, demonstrated scientific processes that were otherwise invisible for students and triggered group discussions. The study, therefore, raises questions about the need for practitioners’ awareness of the type of support the technology and other resources provide to assist both conceptual learning and enhancing students’ agency in learning to learn.     

Qualitative research and making meaning from adventure: A case study of boys' experiences of outdoor education at school

Abstract

This article details the process and analysis of a case study, conducted over a six week period, involving an outdoor education class in an all-boys Catholic, New Zealand secondary school. The questions explored by the case study were the subjective meanings of adventure experiences in outdoor education and the benefits of qualitative research for assessing the value of outdoor education. The methodological techniques used were observation, involving some researcher participation, and in-depth interviewing. Subsequent data analysis was based upon theories of experiential education and adventure education, and concepts of leisure and human agency. The results of the study suggest that the meanings participants make of their experiences, and the value they derive from them, exceed those that may conventionally be sought and measured as an improvement in self-concept. These findings suggest that learning through adventure is potentially valuable as a holistic and life-long form of activity that enhances the capacity to enjoy and engage in living. This is an important extension beyond its often limited and compartmentalised applications, which are rationalised by specific outcome based objectives. A qualitative methodology was indispensable to an inquiry of this kind and warrants further attention in the process of understanding the meanings of adventure and learning.     

Havruta Goes to University: Havruta-style Text Study in a College Education Class

ABSTRACT

This article presents original qualitative research applying paired text study, havruta learning, to the secular college classroom. I adapted this method to a first-year seminar in a public university and found that students perceived that havruta improved their abilities to verbalize their understandings through reading text aloud and debating one another, opened their eyes to new perspectives, engaged them in argumentation, and empowered them to take charge of learning. Despite students’ overall resoundingly positive reflections upon their havruta experience, I also inquired as to perceived disadvantages, which included unbalanced participation, discomfort with disagreeing, and agreement leading to stagnation. Strategies I posit to address these challenges include planning for and evaluating future havruta learning and evaluating the balance and quality of work of partners.     

Beyond the tools: analysing personal and group learning environments in a university course / Más allá de la tecnología: análisis de los entornos de aprendizaje personales y grupales de estudiantes en una asignatura universitaria

Abstract

In this paper we present the results of a naturalistic study carried out with higher education students which describes their learning environments when working on academic tasks in groups and individually. In this study we want to analyse how they organize their learning activities and knowledge processes (reading, reflecting and sharing) and how these learning processes are integrated into their Personal Learning Environments (PLE). In order to achieve that, students had to reflect about the basic ‘components’ that make up their learning process: reading (using not only by text but also multimedia), doing/reflecting (creating cognitive artefacts) and sharing (discussing, showing, and offering and receiving feedback from and to a community of reference). Students formed relationships between those components and any technological tools they used and created mind maps to represent their PLEs. The main objective is to try to understand how PLEs are organized and how they are perceived by learners, not from a technological perspective but from a learning perspective. The main results confirm the strong nature of PLEs as a pedagogical approach with a strong technological base. In addition, those results suggest a close relationship between students’ beliefs and expectations regarding academic tasks as well as the importance they give to each component in their learning process.     

‘I’ll take care of the flowers!’ Researching agency through initiatives across different learning environments

ABSTRACT

This case study examined how the agency of a fifth-grade pupil appeared across different learning environments in the primary school context. In this study, agency is defined as the initiatives taken by an individual in interactive situations. The research question is: how does a pupil’s agency manifest and vary through taking initiatives across different learning environments within curriculum-based education? The empirical data consist of 50 video episodes including learning activities in a classroom, in a vegetable garden and on a nearby farm. The findings show that the pupil’s agency varied across learning environments, manifesting itself through complementing, supportive, constructive and challenging initiatives. When developing pedagogy that supports learning across formal and informal learning environments, it is essential to enable pupils to take different kinds of initiatives and to exercise their agency in versatile contexts.     

Exceeding expectations: scaffolding agentic engagement through assessment as learning

Background: The active involvement of learners as critical, reflective and capable agents in the learning process is a core aim in contemporary education policy in Australia, and is regarded as a significant factor for academic success. However, within the relevant literature, the issue of positioning students as agents in the learning process has not been fully examined and needs further exploration.

Purpose: This study aims to explore ways in which aspects of self-regulated learning theory may be integrated with the concept of agentic engagement into classroom practice. Specifically, the study seeks to scaffold students’ self-assessment capabilities and self-efficacy by using a formative assessment-as-learning process. The research examines how scaffolded planning, as part of the forethought phase in the Assessment as Learning (AaL) process, influences self-regulation and student agency in the learning process.

Sample: 126 students from school years two, four and six (student age groups 7, 9 and 11 years), and 7 teachers at an independent (co-educational, non-religious) primary school in the Northern Territory, Australia, participated in the study.

Design and methods: Conducted as a one-setting, cross-sectional practitioner research study, the data sources included students’ planning templates, writing samples, interviews with students and teachers and email correspondence with teachers. The data were analysed for emerging themes and interpreted from a framework of social cognitive theory.

Findings: In this study, students were given the opportunity and support to exercise agentic engagement. Findings suggested that, in particular, students who were identified by their teachers as low-achieving and/or with poor motivation, were perceived by the teachers as exceededing expectations by demonstrating relatively greater motivation, persistence, effort and pride in their work than would be the case usually.

Conclusions: The findings from this formative AaL study suggest that AaL has the potential to help scaffold primary students’ development of assessment capabilities.     

Escaping the subject-based class: A Finnish case study of developing transversal competencies in a transdisciplinary course

Abstract

In 2014, the Finnish National Board of Education launched a new core curriculum with the aim of meeting the skills and competence requirements of the 21st century. The purpose of this case study was to find out what transversal competencies Finnish eighth graders developed and how they experienced studying in a problem-based transdisciplinary course, which was arranged for the transitional stage between the former and the new curriculum. The qualitative data consisted of questionnaires and interviews. The analysis followed the methods of qualitative content analysis. When asked about their learning, the students commented on not having learnt much, referring to the discipline-based knowledge. Instead, they had learnt skills through teamwork, problem solving, and expression of their views and opinions, an aspect that they did not clearly connect with the things to be learnt at school. Many students felt that they acquired the competencies that they would need later in their lives. The students’ teamwork had a crucial impact on their learning experience. The timeframe for the integrative approach needs careful consideration; the learning process in this course was perceived as being successful because it was long enough. Our study highlights that focusing on students’ views is critical when reforming curriculum.     

Learning in the Finnish outdoor classroom: Pupils’ views

ABSTRACT

Finnish nature schools are environmental education organisations offering outdoor programmes for classes in natural environments. They are intended to contribute to the aims of the national curriculum in various subjects. This qualitative study examines how students describe their learning in a nature school context and how they describe differences between learning in nature and learning in the classroom. After a nature school day, 30 10–11-year-old pupils participated in focus groups interviews. The data were analysed through inductive content analysis. Nature school learning was described as cognitive, affective, social and as providing practical skills learning, although some pupils claimed that they had not learned anything. The differences between the learning environments were described as differences in learning activities, concreteness, external conditions, learning topics, perceived ability to concentrate and differences in motivation. We stress the importance of reflection on learning in order to raise pupils’ awareness of their outdoor learning.     

Relation of General Intelligence to the Persistence of Educational and Vocational Plans of High-School Pupils

ABSTRACT

The author aimed to explore primary pupils’ perceptions of learning and to what extent they perceive learning is important to them. Fifty-five primary school pupils in Istanbul and Canakkale (Turkey) were interviewed. To analyze the data, A. Strauss and J. Corbin's (1998) grounded theory methodology was followed. NVivo 7 (QSR, Australia) a software program for qualitative analysis, was also used during the analysis process. Research findings revealed that pupils who have a low-socioeconomic background have stated that learning is for a reason of employability. The interviews also revealed that many pupils associated learning with memorization and listening. This showed that they perceive learning as a passive transmission.     

Assessing learning in a sociology department: what do students say that they learn?

Assessment plays a central role in evaluating and strengthening student learning in higher education, and sociology departments, in particular, have increasingly become interested in engaging in assessment activities to better understand students’ learning. This qualitative study builds on previous research on assessment by asking what students in one American university department see themselves learning in the sociology major. Rather than asking students to reflect on what we think they are learning, we asked open-ended questions about skills, topics and modes of education they considered most significant to their learning. The 25 sociology majors in our study included second-year students, graduating fourth-year students and alumni who had graduated five years prior, enabling us to compare what students have learned or are learning across cohorts. Our findings demonstrate that students emphasise a common collection of skills, topics and – especially – modes of learning in the major, despite their various course selections and interests within the discipline, and also that majors’ orientations to sociology vary as they move through, and beyond, the undergraduate curriculum.     

Pedagogy of Learning and Teaching: A Phenomenographic Perspective from Selected Classrooms in Rural South Africa

Abstract

Classroom practitioners are expected to facilitate effective learning under the Curriculum and Assessment Policy Statement (CAPS) in South Africa. During this facilitation process, educators are required to become more resourceful in terms of their learning and teaching strategies. This article is based on a case study – within the phenomenographic paradigm – of Grade 9 educators and learners in four rural schools in North West, South Africa. The study focussed on a specific economics and management sciences (EMS) topic relating to the understanding of the value or price determination of the South Africa rand on the foreign exchange market. The article is divided into three parts: firstly, the concepts of phenomenography and the variation theory of learning are explored and constituted as the conceptual framework; secondly, educators’ experiences with the variation theory as a classroom resource for learning; and thirdly, the impact of educators’ classroom experiences with the variation theory on learners’ quantitative and qualitative learning outcomes. The results showed an improvement in the learners’ understanding of the EMS topic studied.     

Teacher students’ meaningful learning in widening learning worlds

ABSTRACT

This qualitative metasynthesis investigated students’ meaningful learning experiences amid andragogical teacher education programmes. The programmes catered to the specific learning needs of adult students in the context of university-based teacher education. This study aimed to provide a wider picture of the frames of students’ meaningful learning in andragogical teacher education programmes. The findings revealed three major learning worlds of adult students’ meaningful learning, which formed a common system widening from the professional awakening to the transformative community and agency in society. Based on the findings, this study argues that in andragogical teacher education, which emphasises collaboration and networking in accordance with the current trend in higher education, teacher students may become empowered participants and active agents in society. The findings can be used in planning curricula, and developing programmes of higher education in general and teacher education in particular.     

Photovoice as an evaluation tool for student learning on a field trip

Background: Photovoice is one method that enables an educator to view an experience from a student’s perspective. This study examined how teachers might use photovoice during an informal learning experience to understand the students’ experiences and experiential gain.

Design and methods: Participants in this study consisted of six students, three male and three female, ranging from ninth through twelfth grade at a rural Ohio high school, who attended a field trip to a biological field station for a four-day immersive science experience. Students were provided cameras to photograph what they believed was important, interesting, or significant during an immersive four-day science trip to a biological field station, individualizing their observations in ways meaningful to them, and enabling them to assimilate or accommodate the experiences to their schema.

Results: Analysis identified five positive benefits to use photovoice as an evaluation tool: teachers were provided qualitative evidence to evaluate student interaction on the field trip; teachers could evaluate the students’ photographs and captions to determine if the field trip met the learning objectives; students were empowered to approach the goals and objectives of the field trip by making the field trip personally relevant; students assimilated and accommodated the new observations and experiences to their own schema; students automatically reflected upon the learning experience as they captioned the photos.

Conclusions: Through photovoice, the teachers were enabled to qualitatively assess each student’s experience and learning from the field trip by illustrating what the students experienced and thought was significant; providing the teachers a method to evaluate all participating students, including those who are secretive or do not normally contribute to class discussions.     

How elementary school students' motivation is connected to self-regulation

Empirical research reveals that students face difficulties engaging in learning and achieving their goals in a variety of learning contexts. To study effectively, students need to regulate their learning process. In spite of increased understanding of cognitive aspects of self-regulation, motivational aspects of regulation have not yet been thoroughly probed. This study investigates how motivation is connected to self-regulated learning when elementary school students (N = 32) study science in real classroom contexts using gStudy software. This was done by: (1) identifying students' situated motivation during the learning process, (2) analyzing how the students with different motivational approaches activated cognitive self-regulation in authentic learning situations across multiple learning episodes, and (3) complementing the understanding with the students' individual accounts of their motivation regulation during the learning process. The results show that there are qualitative differences in the self-regulation tactics used by the high- and low-motivation students as they study. Motivation is linked closely to active self-regulation.     

Questioning to learn and learning to question: Structure and function of problem-based learning scenarios in environmental science education

In problem-based learning, scenarios relating to real life areused as a point of departure for the learning process. Eventhough the importance of suitable cases or scenarios in bringingabout a fruitful learning process is emphasised in theliterature, few studies focus on how they actually function inthe learning process. This study focuses on how the scenariosused in a ten-week introductory course of a new four-yearundergraduate programme in environmental science functioned interms of the structure and content of the questions they evoked.Data were gathered through diary notes from nine groups ofstudents, comprising 5–8 students per group. The data weresubjected to a qualitative analysis aimed at describing thestructure and content of the questions generated by the groups.Five different kinds of questions were identified and labelled;encyclopaedic, meaning-oriented, relational,value-oriented and solution-oriented. All scenarios generated questions pertaining to all five categories in all groups, butthe emphasis varied. The results are discussed in relation to thedesign of scenarios, and in relation to students' approaches tolearning.     

A jigsaw cooperative learning application in elementary science and technology lessons: physical and chemical changes

Cooperative learning is an active learning approach in which students work together in small groups to complete an assigned task. Students commonly find the subject of ‘physical and chemical changes’ difficult and abstract, and thus they generally have many misconceptions about it.

Purpose

This study aimed to investigate the effects of jigsaw cooperative learning activities developed by the researchers on sixth grade students’ understanding of physical and chemical changes.

Sample

Participants in the study were 61 sixth grade students in a public elementary school in Izmir, Turkey.

Design and methods

A pre-test and post-test experimental design with a control group was used, and students were randomly assigned to the experimental and control groups. Instruction of the subject was conducted via jigsaw cooperative learning in the experimental group and via teacher-centered instruction in the control group. During the jigsaw process, experimental group students studied the subjects of changes of state, changes in shape and molecular solubility from physical changes, and acid–base reactions, combustion reactions and changes depending on heating from chemical changes in their jigsaw groups.

Results

The concept test results showed that jigsaw cooperative learning instruction yielded significantly better acquisition of scientific concepts related to physical and chemical changes, compared to traditional learning. Students in the experimental group had a lower proportion of misconceptions than those in the control group, and some misconceptions in the control group were identified for the first time in this study.

Conclusions

Jigsaw cooperative learning is an effective teaching technique for challenging sixth grade students’ misconceptions in the context of physical and chemical changes, and enhancing their motivation, learning achievements, self-confidence and willingness in the science and technology lesson. This technique could be applied to other chemistry subjects and other grade levels.