Collaborative research for assessment for learning

This paper reports on a collaborative research study between three secondary teachers and two university researchers, initiated by the teachers, into the practice and impacts of assessment for learning in science, geography and history classes. The research provides insights into how teachers and researchers can collaborate to develop a research and practice agenda. The study illustrates the need for a dynamic interconnection between the personal, micro or school-related level and the macro or policy level factors. The findings highlight the importance of external support, shared teacher knowledge and beliefs, professional experimentation, and shared reflection on student responses to classroom innovations.     

Analytics-supported reflective assessment for 6th graders' knowledge building and data science practices: An exploratory study

Preparing data-literate citizens and supporting future generations to effectively work with data is challenging. Engaging students in Knowledge Building (KB) may be a promising way to respond to this challenge because it requires students to reflect on and direct their inquiry with the support of data. Informed by previous studies, this research explored how an analytics-supported reflective assessment (AsRA)-enhanced KB design influenced 6th graders' KB and data science practices in a science education setting. One intact class with 56 students participated in this study. The analysis of students' Knowledge Forum discourse showed the positive influences of the AsRA-enhanced KB design on students' development of KB and data science practices. Further analysis of different-performing groups revealed that the AsRA-enhanced KB design was accessible to all performing groups. These findings have important implications for teachers and researchers who aim to develop students' KB and data science practices, and general high-level collaborative inquiry skills.

Practitioner notes

What is already known about this topic

• Data use becomes increasingly important in the K-12 educational context.
• Little is known about how to scaffold students to develop data science practices.
• Knowledge Building (KB) and learning analytics-supported reflective assessment (AsRA) show premises in developing these practices.

What this paper adds

• AsRA-enhanced KB can help students improve KB and data science practices over time.
• AsRA-enhanced KB design benefits students of different-performing groups.
• AsRA-enhanced KB is accessible to elementary school students in science education.

Implications for practice and/or policy

• Developing a collaborative and reflective culture helps students engage in collaborative inquiry.
• Pedagogical approaches and analytic tools can be developed to support students' data-driven decision-making in inquiry learning.

     

Capturing temporal and sequential patterns of self-, co-, and socially shared regulation in the context of collaborative learning

This study examined how temporal sequences of regulated learning events, such as types and processes of regulated learning, emerge during different stages of collaborative learning. Earlier research has focused on individual learning and not on the captured temporal sequences of regulation in collaborative learning. The data were collected during a two-month math didactics course taken by teacher education students who collaborated in three member groups. Twenty-two hours of video data were collected to follow how sequences of regulated learning events, along with task execution, emerged within the six groups as their collaboration advanced. The data were analyzed using qualitative content analysis and lag sequential analysis. The results showed that the groups engaged mostly in co-regulated planning and monitoring. Temporal analysis showed that collaborative interactions focusing on task execution promoted socially shared planning, indicating that task execution provided grounding for socially shared planning and regulation to occur. The sequential analysis illustrated that metacognitive monitoring played a facilitative role in the progress of task execution.     

Fostering highly engaged knowledge building communities in socioemotional and sociocognitive hybrid learning spaces

With the aim of understanding sociocognitive and socioemotional hybridity in learning spaces, we examined a semester-long learning community where students were given the freedom to advance their epistemological and social agendas across face-to-face and online settings. We collected and analyzed 1,780 online notes written by students throughout the semester and coded them based on their sociocognitive or socioemotional values. We then examined the conversation chains that students engaged in vis-a-vis their developments as a group. In addition to showing how the group developmental stages served as a macro-level context for the socioemotional and sociocognitive spaces, the analysis highlighted how deep, rapid, community knowledge building conversations spontaneously emerged in relation to the timing of socioemotional developments. This study elucidates an important dimension of hybrid spaces, emphasizing the need to design activities to support both sociocognitive and socioemotional spaces in technology-enhanced learning communities.     

Fruitful student interactions and perceived learning improvement in DSLab: A dynamic assessment tool for distributed programming

This work presents an automated assessment tool for online distributed programming, called DSLab. It is a web-based environment that provides a transparent deployment and execution of assignments in remote computers, a transparent initialization and the possibility to add new logs by the students. DSLab has been evaluated in a real distributed learning environment by analysing students' own perceptions on their learning improvement and exploring whether students' interactions with the tool yielded a fruitful learning experience. Current research provides no evidence that the effects of such a tool have been investigated in the field of online distributed programming. Two types of analysis were performed: a quantitative analysis of data coming from students answering a questionnaire and an analysis of the log files of students' interactions with the tool. Our results showed that students perceived substantial learning improvement from the use of the automated assessment tool. Moreover, students produced fruitful interaction with the tool as soon as they achieved high familiarization and constant activity with it, which ultimately helped them improve their academic performance. Finally, the limitations of the current study and directions for future research are presented.     

Embedding librarians in online tertiary classrooms: A new model for learner support

Embedded librarians working in online classrooms present a new avenue of learner support direct to students in the space where they learn. The development of embedded librarian practice has evolved since 2004 and is becoming commonplace in online tertiary classrooms, but what is not yet so common is theory underlying this new practice. A multiple case study of embedded librarians in online tertiary classrooms in New Zealand was conducted to identify what makes this practice effective in providing support. Six factors were identified as contributing to effectiveness: relationality, immediacy, relevancy, interactivity, parity and faculty–librarian collaboration. The Embedded Librarian Support model was derived from, and is presented as the culmination of, the findings of this research.     

Student group presentations: a learning instrument in undergraduate mathematics for engineering students

In this study we created an environment for peer learning, where students teach students by making oral presentations in groups about solving mathematical problems and explaining the theoretical background in mathematics, during the first year of an undergraduate engineering programme at the Norrköping campus of the Linköping University. In order to strengthen the students’ understanding and perception of central mathematical concepts, the study was designed to take the students through five different learning experiences, preparing the presentation, presenting the mathematics, listening to others presenting, discussion by all students after the presentation and feedback by the teacher to the small group of students separate from the other students. We study how oral presentations work as a learning and assessment method. The study consisted of three stages. After a first run of the presentations as a learning instrument, three guidebooks with recommendations for students and teachers were developed in order to assist students as well as teachers about their role in this learning environment. Students’ and teachers’ views on the student presentations as learning instrument were surveyed before and after the intervention. In stage three, students were interviewed individually to ascertain the relevant success of the different learning experiences.     

Assessing assessment quality: Criteria for quality assurance in design of (peer) assessment for learning - A review of research studies

The interest in assessment for learning (AfL) has resulted in a search for new modes of assessment that are better aligned to students’ learning how to learn. However, with the introduction of new assessment tools, also questions arose with respect to the quality of its measurement. On the one hand, the appropriateness of traditional, psychometric criteria is being questioned in the context of assessment for learning. On the other hand, it has been argued that new criteria need to be formulated to acknowledge the unique features of assessment for learning. The issue dealt with in this review is what quality criteria are specifically relevant to AfL. Studies using (peer) assessment for learning were evaluated with regard to use of quality criteria under two perspectives: their recognition of educational measurement criteria, as well as their consideration of student involvement in the assessment of learning. The selected studies were analyzed to determine what specific criteria were utilized in (successive parts of) the assessment cycle (i.e., the process of construction, administration and follow up of an assessment). Our results indicate that quality criteria are dissimilarly connected to the distinct steps of an assessment cycle, which holds as well for the attention given to student involvement in assessment for learning.     

Assessment for learning in the Hong Kong assessment reform: A case of policy borrowing

This paper describes Hong Kong’s borrowing, primarily from the UK, of the assessment for learning policy, in the context of prolonged use of formal summative public examinations. The narrative review and analysis are guided by a social positivist critique of the assessment for learning policy in the Hong Kong context. This paper concludes that Hong Kong’s attempts to implement assessment for learning are ambitious but somewhat futile because of persistent use of public examinations for important decision-making. Change in Hong Kong may only happen through a gradual implementation of assessment for learning practices that takes into account cultural, societal, and historic norms. Critical issues are identified and recommendations are proposed for further implementation of assessment for learning in Hong Kong.     

Recognizing socially shared regulation by using the temporal sequences of online chat and logs in CSCL

The increasing amount of empirical research shows that the role of regulatory processes is critical in CSCL and collaborative learning settings. However, the current conceptual definitions and specificity of the findings vary. This is most probably because of limitations in the methods investigating regulated learning in a collaborative learning context. This study aimed to provide empirical evidence for how self- and shared regulation activities are used and whether they are useful for collaborative learning outcomes. Eighteen graduate students worked in collaborative groups for seven weeks in a CSCL course and the data of this study focuses on three one week online collaborative learning phases in the course. Temporal and sequential analysis of chat discussions and log file traces were matched to find evidence about whether the students' collaboratively planned regulatory activities became shared in practice. The results show evidence that collaborative planned regulatory activities become shared in practice. The groups that achieved good learning results used multiple regulatory processes to support their learning and also reached shared regulation. The four microlevel examples demonstrate simplified patterns of the activation of self-regulation and shared regulation. In conclusion, individual socially shared regulation plays a critical role in successful collaborative learning.     

Accounting for structure and agency in ‘close-up’ research on teaching,learning and assessment in higher education

This paper examines research into teaching, learning and assessment (TLA) in higher education in terms of structure and agency. It argues that although issues of structure and agency are seen as crucial in social theory, they are very little discussed in research into TLA in higher education and that a consideration of structure and agency raises some important questions about this research and the quality of the explanations that it generates. It is therefore time to reconsider this research from the standpoint of structure and agency so that more sophisticated approaches to researching, and generating explanations of, teaching, learning and assessment in higher education can be developed.     

A systematic review of prerequisites for implementing assessment for learning in classroom practice

Although many researchers acknowledge that Assessment for Learning can significantly enhance student learning, the factors facilitating or hindering its implementation in daily classroom practice are unclear. A systematic literature review was conducted to reveal prerequisites needed for Assessment for Learning implementation. Results identified prerequisites regarding the teacher, student, assessment and context. For example, teachers must be able to interpret assessment information on the spot, student engagement in the assessment process is vital, assessment should include substantial, constructive and focussed feedback, and the school should have a school-wide culture that facilitates collaboration and encourages teacher autonomy. The results of this review contribute to a better understanding of the multiple facets that need to be considered when implementing Assessment for Learning, from both a theoretical and a practical standpoint.     

Formative and shared assessment in higher education. Lessons learned and challenges for the future

Formative assessment is the process by which teachers provide information to students during the learning process to modify their understanding and self-regulation. An important process within this is shared assessment, which refers to student involvement in the assessment and learning practice, a process of dialogue and collaboration between teacher and students aimed at improving the learning process, both individually and collectively. The purpose of this paper is to review the current state of affairs in depth. This paper therefore highlights, on the one hand, the lessons learned through research and development in higher education (i.e. providing clear learning goals and feedback, guiding learning, involving students in learning and assessment, promoting feedback as a process of dialogue, and making processes viable). On the other hand, these lessons also suggest some challenges and difficulties that must be addressed in the future in order to further improve formative and shared assessment in higher education. These include the need for more research on its effects, further conceptual clarification, the intersubjectivity of the process, recognition of the divergent processes and ethical principles, students’ involvement not only in assessment but also in determining academic grades, and broading learning goals and objetives in FA & SA.     

Using technology to facilitate effective assessment for learning and feedback in higher education

The aims of this paper are to examine and critically evaluate a selection of different technological methods that were specifically chosen for their alignment with, and potential to enhance, extant assessment for learning practice. The underpinning perspectives are that: (a) both formative and summative assessment are valuable opportunities for learning, and (b) using technology may enhance learning in assessment and feedback processes. Drawing on the literature and empirical evidence from a research study in a Scottish university, the advantages and drawbacks of using technology are examined. It is asserted that, by adopting a flexible approach and taking small incremental steps, the use of different types of technology can be beneficial in facilitating effective assessment for learning and feedback in higher education.     

Integrating assessment for learning in the teacher education programme at the University of Oslo

Abstract

This article provides an analysis of the integration of assessment for learning principles in the newly revised five-year Master of Education programme at the University of Oslo, Norway, across didactic subjects, pedagogy and school practice. The analysis draws on lecture notes, student videos and student exam papers among 143 student teachers, aiming to identify (a) the operationalisation of the assessment curriculum at the university campus, and in school practice, (b) how the student teachers use assessment principles as tools in their instructional designs and (c) how they self-assess their teaching practice. Our main finding is that student teachers seem to be more concerned with assessing their students than using self-assessment to improve their instruction. Based on the findings, we argue the importance of relating the teaching and learning activities with the assessment situations used in the teacher education programme.     

Supporting teacher responsiveness in assessment for learning through disciplined noticing

Abstract

Assessment for learning (AfL) practices are commonly recommended as effective classroom strategies for providing teachers with information about student understanding. For teachers, the substantive potential of these AfL practices to inform student learning actions depends on what teachers notice and select as a focus and how they interpret and act on the information they have. This paper suggests research on teacher professional noticing has something to offer in understanding how teacher AfL attention and actions are framed in the moment. It explains noticing as a responsive act that invites action that is an inclusive, dynamic and purposeful response to evidence of student ideas. Noticing is what enables a teacher to act on the fly because it informs and underpins possible actions. Three frames of noticing are introduced to guide noticing and action – curriculum connoisseurship, cultural and community connectedness, and collaborative ways of working. Through consideration of classroom events using these frames, teachers become better prepared to choose to respond in the moment creatively. The three frames provide teachers and researchers with a conceptual language to articulate and sharpen their AfL practices as part of a flexible and rigorous responsiveness to their students’ learning.     

Integrating Skills and Wills Instruction in Self-Regulated Science Text Reading for Secondary Students

This study investigated the effectiveness of cognitive-metacognitive versus motivational components of the IMPROVE self-regulatory model, used while reading scientific texts, for 10th graders’ scientific literacy and self-regulated learning (SRL). Three treatment groups (N?=?198) received one type of self-addressable questions while reading scientific texts: cognitive-metacognitive (CogMet), motivational (Mot), or combined (CogMetMot). Control group received no self-addressed questions (noSRL). One measure assessed scientific literacy, and two measures assessed SRL: (a) as an aptitude—pre/post questionnaires assessing self-perceived SRL, and (b) as an event—audiotaping participants’ thinking-aloud SRL behaviors in real-time learning experiences and data coding illustrating SRL changes. Findings indicated that treatment groups significantly outperformed the non-treatment group. No differences emerged between CogMet and Mot, whereas fully combined SRL support (CogMetMot) was most effective. Theoretical and practical implications of this preliminary study are discussed.     

Is assessment for learning feasible in large classes? Challenges and coping strategies from three case studies

ABSTRACT

While assessment for learning (AfL) has been widely used in college English classrooms in China where teachers usually teach large classes, how AfL is implemented in the large-class higher education (HE) context is relatively underexplored. To address this gap, this study explores some of the challenges presented by large classes and the teachers’ coping strategies through case studies of three university English teachers’ AfL practices. Challenges are identified surrounding how the teachers dealt with (a) inadequate attention to individual students, (b) reduced opportunities for individualised feedback, (c) overwhelming marking responsibilities, and (d) involving students in assessment, with strategies of putting students in groups, matching names with faces, conducting teacher-student conferences, utilising technology to facilitate feedback processes, enabling peer feedback, and transforming students from assessees to assessors. Findings are framed around useful forms of AfL in large classes and the feasibility of AfL in terms of compromises between teacher agency and their temporal and contextual conditions. The paper concludes with implications for teacher assessment training, teacher education and policy-making.     

Feedback in Hong Kong secondary writing classrooms: Assessment for learning or assessment of learning?

While much of L2 teacher feedback research has focused on the effectiveness of feedback and its impact on student revision and writing, little has been done to examine teachers’ feedback in the larger classroom context of teaching and learning to ascertain the functions teacher feedback serves from an assessment-for-learning perspective. Using multiple sources of data from 26 secondary teachers’ written feedback to 174 student texts, interviews with six of the teachers and 18 students, the present study investigates the nature of teacher feedback and the functions it serves in the teaching-learning-assessment process in the writing classroom. The findings show that teacher feedback focuses largely on assessing writing summatively, primarily serving the purpose of assessment of learning, rather than assessment for learning – i.e., using feedback as a pedagogical tool for improving the teaching and learning of writing. The study calls for greater attention to the implementation of assessment for learning in the writing classroom, and specifically the use of feedback for formative purposes.