Improving assessment of learning and for learning

The focus of this article is recent work by the Assessment Reform Group (ARG) on the role of teachers' judgements in the summative use of assessment. A brief overview of the early work of the ARG is followed by discussion of the desirable properties of assessment for summative uses. The work of the ARG's Assessment Systems for the Future project provided evidence and arguments concerning the validity, reliability, impact and cost of tests and of summative assessment by teachers. Whilst there is ample evidence that the teachers' judgements are more valid than, and equally reliable as, tests, there is a danger of unwanted impact on teaching as long as results are used for ‘high stakes’ evaluation of teachers and schools. Implications for policy include an end to the practice of using the results of pupils' summative assessment, however they are derived, as the sole basis for target setting and school accountability.     

Peer assessment as a learning tool for enhancing student teachers' preparation

This study investigates how peer assessment contributes to enhancing student teachers' preparation during field experience. A semi-structured interview was conducted with 72 student teachers. The findings showed that the student teachers have positive beliefs about peer assessment. They think that it can be beneficial if some changes are made in the way it is employed in teacher education programs. In light of the study findings, the researchers put forth a number of suggestions and recommendations for employing peer assessment in teacher education programs in Jordan.     

Effective formative e-assessment of student learning: a study on a statistics course

The process of formative assessment in universities has the potential to engage students in reflection and to take greater ownership of their learning. We report on a study involving 104 students taking a statistics course in a degree programme in rehabilitation science in Hong Kong. The assessment strategy was redesigned to include a weekly online formative assessment activity for 10 weeks. In this case, it was observed that there was a high participation rate across weeks. An end-of-course online survey was conducted to explore the learning experiences during formative assessment of the cohort, and students reported high satisfaction with the process, particularly how this contributed to a change in their learning approach. Students not only showed regular commitment of time on tasks but also took active steps to find answers for themselves and with their peers.     

Academic support model for post-secondary school students with learning disabilities: student and instructor perceptions

ABSTRACT

While post-secondary school students with learning difficulties confront many challenges, there is little research on methods of support. In the current study, the ‘adapted course’ model was qualitatively examined from the perspective of both students and instructors. The main goal was to explore how students and instructors perceived the instruction in adapted courses. Five pre-service students with learning disabilities (LD) and/or attention-deficit hyperactivity disorder and three instructors were interviewed. Responses were analysed using subjective analysis, with central content categories identified based on student and instructor statements. Six main themes emerged: (1) small class size; (2) emotional support; (3) adjusted teaching methods; (4) independent practice exercises; (5) instruction by modelling; and (6) instructor characteristics. The current study can inform pedagogical principles on which a support model for post-secondary students with LD can be based. It provides analyses of the effectiveness of one support model and presents the perceptions of both faculty and students. Moreover, the study provides an opportunity to learn about potential components of effective teaching-learning, not only in the specific context of adapted courses, but in any supportive learning situation (e.g. tutoring) or course.     

Teacher and student perceptions of intermediate assessment in higher education

Universities introduce intermediate assessment because it is understood to have positive effects on student behaviour and achievement. Yet, how intermediate assessment is perceived might be conditional for its success. The current study investigates both teachers’ and students’ perceptions of intermediate assessment. Teachers and students were interviewed and Student Evaluations of Teaching were examined. Results indicate that both teachers and students had generally positive perceptions of intermediate assessment. However, the two groups provided different reasons for their positive perceptions. Teachers and students agreed that intermediate assessment helps students to keep up with their study work. Moreover, teachers mentioned that they could assess various knowledge and skills with intermediate assessment, whereas students preferred intermediate assessments to test the same knowledge and skills as the final exam. This finding suggests that teachers and students in our study had different goals for intermediate assessment.     

Triangulation of language assessment modes: learning benefits and socio-cognitive prospects

An experimental design was adopted in the present study to cross-compare the effect of two formative assessments, namely peer assessment (PA) and teacher assessment (TA), with summative assessment (SA) on the improvement of language learners’ writing skill and self-efficacy. Writing excellence was operationalized by structural (grammar), lexical (word choice), and discoursal (cohesion and coherence) well-formedness. Furthermore, focused group interview was run to elicit the learners' attitudes and preferences toward type of assessment. Three cohorts of proficiency-matched language learners sat for a semester-long experimentation (20 sessions × 90 min = 30 h) and three testing sessions. The results revealed that PA group significantly improved in the writing skill, but TA and SA groups did not. Self-efficacy was not enhanced in any of the assessment groups under study. The interviews and follow-up member checkings indicated that the pupils did not have positive attitudes toward PA and preferred TA. Among the sources of this disfavor, they referred to misunderstanding, lack of knowledge or attention, tiredness, poor handwriting of the assesse, shyness, friendship bias, and even feeling not qualified enough to judge their peers’ tasks. Overall, then, the study reported a mismatch between the students’ actual performance and attitudinal beliefs under different assessment conditions, implying that (a) Iranian students are more inclined toward teacher-dominated pedagogies and (b) triangulation of assessment modes provides more reliable optimal results.     

Governing student assessment: administrative rules,silenced academics and performing students

In this paper, the author reports on an analysis of an Australian university’s assessment procedure. The procedure – a major governance document of the university – is deconstructed by way of a Foucauldian discourse analysis in order to consider how students and academics are governed. There were three major findings. The dominant discourse operating in relation to student assessment was that of administration; a finding consistent with previous work in this area and at other universities. The main subject position in this discourse of administration is one of silence for the academic and of a performer for the student, one who is rewarded with the award of a degree if the performance is satisfactory. The passion to learn and to teach is nowhere to be found in this administrative discourse, one not unique to the said university, indeed one that flourishes in Western universities today.     

Formative assessment and teaching practice: the point of view of Italian teachers

Over the years the debate on the aims, approaches, and impacts of formative assessment has never stopped to grow. Parallel to this growth is the awareness that it is crucial to understand what conceptions teachers have of assessment in order to guarantee more effective teacher education. This paper tries to connect these two fields of research: formative assessment and teacher education through an analysis of teachers’ conceptions: What do teachers think about assessment? What aims do they pursue through it? Do teachers really distinguish between formative and summative assessment? On the backdrop of the formative assessment literature, the article reports on an exploratory research study. Limitations and issues are analysed in order to shed light on teacher education implication and on future educational research paths in the educational assessment field.     

Students' choice of college major and their perceived fairness of the procedure: evidence from Turkey

This study investigated the relative influence of factors and persons in college major choice and procedural justice perceptions of 449 undergraduate students enrolled in 1 private and 2 public universities in Turkey. Results showed that students found themselves to be the most influential person in their choice of college major, followed by parents/other family members. With regard to the factors, interest in the subject, guaranteed employment, expected earnings in the field, the university entrance exam score, and a prestigious career were ranked, respectively. Striking differences were found among majors. Findings also suggested that, whereas male and female students' choices were influenced by very similar factors and persons, female students declared more procedural justice.     

Feedback as dialogue: exploring the links between formative assessment and social software in distance learning

The paper explores the relationship between formative assessment and social software. Formative assessment practices though beneficial for student learning become marginalised and constrained in open and distance learning environments in higher education. Feedback is a key factor in formative assessment and learners can benefit from the deployment of emerging technologies and the opportunities for participation and dialogue afforded by social software. This paper explores and proposes a conceptual framework for this relationship. The claim is that the social dimensions of emerging technologies – specifically, blogs and wikis – allow for formative assessment practices to be re‐invented or at the very least facilitated by essentially participative and student‐focussed interventions. A comparison of these technologies against formative assessment mechanisms identifies the types of processes that these new tools might best support to encourage effective feedback approaches that both empower the learner and enhance their learning experience.     

福建省高职高专人才培养工作水平评估之探微

近年来，福建省高等职业教育在跨越式发展过程中存在着许多不容忽视的问题和矛盾，以期通过对高职高专人才培养工作水平评估加以控制与改善．就福建省高职高专评估工作中存在的若干问题，提出了行之有效的解决思路与建议．     

Key Issues in Sex Education: reflecting on teaching,learning and assessment

Drawing upon critical reflections of staff and student experiences of teaching, learning and assessment on an undergraduate module entitled Key Issues in Sex Education, we discuss the strategies used to engage students in debates around sex and relationships education (SRE). To date, there is little research which evaluates how formal assessments can be made relevant to students' exploration of this subject. In the case of this module, assessment requires students to first deliver a seminar presentation that focuses upon a self-selected issue within the field of sexual politics, policy and practice. Students then submit a manifesto, in the form of posters, leaflets or flyers pitched at relevant stakeholders, and which adheres more to the conventions for political campaigning, rather than the standard essays or examinations commonly used to assess undergraduates. In the light of this, we critically reflect upon student evaluations in respect of this module, address various problems and paradoxes, and offer recommendations for encouraging students to find their own voices when debating complex issues of not only SRE, but also sex, gender and sexualities more widely.     

An assessment of the effectiveness of a classroom activity designed to teach the value of lifelong learning

Many agree that educational systems should instill in students the value of lifelong learning (LLL), but few have suggested how to accomplish that or how to measure the effectiveness of those curricular initiatives. We developed a technique intended to strengthen students' beliefs about the value of LLL and piloted use of a recently developed scale to assess changes in commitment to LLL. Using a quasi-experimental procedure, we presented students with faculty members’ career biographies describing their personal career paths, including how they learned from their experiences and how they had to adapt and adjust to changing circumstances. To assess beliefs about LLL, students completed Bath and Smith's Propensity for Lifelong Learning (PLLL) scale. We found that students who listened to career biographies had significantly stronger beliefs regarding the value of LLL than the students who were not exposed to the career biographies. We concluded that listening to career biographies strengthened commitment to LLL and that the PLLL scale is a useful outcome measure of this commitment. We discussed limitations and future directions of our method and measurement instrument.     

Quality of assessment of prior learning (APL) in university programmes: perceptions of candidates,tutors and assessors

Formal diplomas and certificates have been accepted as proof that students may receive exemption for parts of their educational programme. Nowadays, though, it is socially desirable that informal and non-formal learning experiences are also recognised. Assessment of prior learning (APL) addresses this issue. In APL, the candidate's knowledge, skills or competences required in informal and non-formal learning are measured against a standard to determine whether they match the learning objectives. Although APL is frequently used in workplaces and vocational education, it is practised less in universities, and research is lacking in this context. This study aims to evaluate the first APL procedure in an academic computer science programme, and an adjusted APL procedure in an educational science masters programme. This is done from the perspective of the APL candidates, tutors and assessors, using the theoretical framework by Baartman et al. (2006). The computer science participants comprised 23 candidates from a police software company, four tutors and four assessors. From educational science, nine candidates, two tutors and two assessors participated. The results show that the APL procedure in educational science is viewed significantly more positively than that in computer science; further, the computer science assessors differ considerably from the other participants in their perceptions relating to the quality criterion ‘cognitive complexity’. Explanations for the difference between the two programmes are discussed in this article and assessor and tutor training highly recommended.     

Disaggregating assessment to close the loop and improve student learning

This study examined student learning outcomes for accelerated degree students as compared to conventional undergraduate students, disaggregated by class levels, to develop strategies for then closing the loop with assessment. Using the National Survey of Student Engagement, critical thinking and oral and written communication outcomes were examined. Within- and between-group learning outcome differences with freshmen and seniors in both instructional formats were tested to determine if instructional goals were met and student learning transpired. A quantitative, single case analysis method was used to determine significance with the items relating to these outcomes. Results indicated that seniors in both instructional formats typically reported greater engagement scores than freshmen, and that accelerated students tended to report more engagement than their conventional counterparts. These findings were most robust for the critical thinking outcome. Ramifications for student learning and relevant instructional formats are then explored, so as to close the loop with assessment.     

Vignette question design for the assessment of graduate sustainability learning outcomes

The development of university wide graduate attributes has become an important mechanism for setting generic learning outcomes across higher education institutions. Many universities around the world have developed sets of graduate attributes that they aim to develop through degree programs. Sustainability related attributes have been included in numerous university attribute statements, however, little research exists on methods that can be used to capture data on the application and use of these attributes by graduates. This paper evaluates an assessment tool, based on a scenario/vignette question design, which can be used to capture data on graduate attributes in context and has the potential to be used across a large graduate population which would enable comparative research of learning outcomes to be undertaken. An evaluation of the usefulness of the scenario/vignette design will be presented based on a pilot of the question design with a sample of RMIT University graduates. The results of the pilot show that scenario/vignettes address some of the limitations of existing self-assessment methods and represent a novel way to collect data on graduate sustainability attributes post-degree completion.     

Formative assessment strategies for students' conceptions—The potential of learning analytics

Formative assessment is considered to be helpful in students' learning support and teaching design. Following Aufschnaiter's and Alonzo's framework, formative assessment practices of teachers can be subdivided into three practices: eliciting evidence, interpreting evidence and responding. Since students' conceptions are judged to be important for meaningful learning across disciplines, teachers are required to assess their students' conceptions. The focus of this article lies on the discussion of learning analytics for supporting the assessment of students' conceptions in class. The existing and potential contributions of learning analytics are discussed related to the named formative assessment framework in order to enhance the teachers' options to consider individual students' conceptions. We refer to findings from biology and computer science education on existing assessment tools and identify limitations and potentials with respect to the assessment of students' conceptions.

Practitioner notes

What is already known about this topic

• Students' conceptions are considered to be important for learning processes, but interpreting evidence for learning with respect to students' conceptions is challenging for teachers.
• Assessment tools have been developed in different educational domains for teaching practice.
• Techniques from artificial intelligence and machine learning have been applied for automated assessment of specific aspects of learning.

What does the paper add

• Findings on existing assessment tools from two educational domains are summarised and limitations with respect to assessment of students' conceptions are identified.
• Relevent data that needs to be analysed for insights into students' conceptions is identified from an educational perspective.
• Potential contributions of learning analytics to support the challenging task to elicit students' conceptions are discussed.

Implications for practice and/or policy

• Learning analytics can enhance the eliciting of students' conceptions.
• Based on the analysis of existing works, further exploration and developments of analysis techniques for unstructured text and multimodal data are desirable to support the eliciting of students' conceptions.