Student evaluation surveys: anonymous comments that offend or are unprofessional

Student comments are routinely collected in university evaluation surveys for the purpose of improving teaching and learning. Whilst student comments provide valuable insights into their experiences, there is limited published research reporting the extent to which student comments are offensive and professional. The aim of this study was to investigate the number of student comments that were identified as being offensive or unprofessional in an online unit evaluation survey collected in a semester in 2010 from an Australian university. One person read 30,684 comments taken from 17,855 surveys and identified comments considered to be abusive or unprofessional. Comments were categorised as either abusive or unprofessional and by the intended target (that is, teacher, unit, resource). Thirteen abusive comments (0.04 % of the sample) were identified. Five abusive comments were directed at the teacher and eight were targeted at teaching and learning experiences. Forty-six comments (0.15 % of the sample) were identified as unprofessional. Of these, seven comments were directed at the teacher and 34 were about units. This suggests that the vast majority of students do not abuse the privilege of giving anonymous feedback. Strategies identified in this paper to educate students and give appropriate feedback can be adopted by universities to minimise offensive comments. Universities can educate students and teachers in appropriate and professional ways of working together, in providing professional feedback to improve the student experience in teaching and learning and to support and mentor teachers in their academic careers.     

Monitoring Trends in Student Satisfaction

Over the last decade, the assessment of student experience has gained significant prominence in Australian higher education. Universities conduct internal surveys in order to identify which of their services students rate higher or lower on importance and performance. Thus, institutions can promote highly performing areas and work on those needing improvement, while students are given an opportunity to influence decision-making in both academic and non-academic spheres of campus life. National student surveys conducted by the government aim to maintain quality assurance in the sector, benchmark outcomes, and, in some cases, reward better performing institutions. This paper outlines the findings of a study which monitored the entire student experience in a large metropolitan multi-campus university during 2005–2011. A specially designed biennial survey was repeated across the years, and completed over the period by 10,562 students from all key cohorts. This paper discusses the trends in perceived importance and performance of various university services and key issues the university has been addressing in order to enhance student experience. It concludes that: (a) the time series data provide a powerful lens into the university’s strategies, initiatives and actions which worked well and those needing further effort or adjustment; (b) it is the total experience of the university that shapes students’ judgements, not just what happens in the classroom; and (c) taking a systematic approach to student feedback, and acting on the results in a timely manner, can significantly improve student engagement and satisfaction.     

Development,Evaluation and Use of a Student Experience Survey in Undergraduate Science Laboratories: The Advancing Science by Enhancing Learning in the Laboratory Student Laboratory Learning Experience Survey

Student experience surveys have become increasingly popular to probe various aspects of processes and outcomes in higher education, such as measuring student perceptions of the learning environment and identifying aspects that could be improved. This paper reports on a particular survey for evaluating individual experiments that has been developed over some 15 years as part of a large national Australian study pertaining to the area of undergraduate laboratories—Advancing Science by Enhancing Learning in the Laboratory. This paper reports on the development of the survey instrument and the evaluation of the survey using student responses to experiments from different institutions in Australia, New Zealand and the USA. A total of 3153 student responses have been analysed using factor analysis. Three factors, motivation, assessment and resources, have been identified as contributing to improved student attitudes to laboratory activities. A central focus of the survey is to provide feedback to practitioners to iteratively improve experiments. Implications for practitioners and researchers are also discussed.     

Feeling feedback: students’ emotional responses to educator feedback

Assessment feedback allows students to obtain valuable information about how they can improve their future performance and learning strategies. However, research indicates that students are more likely to reject or ignore comments if they evoke negative emotional responses. Despite the importance of this issue, there is a lack of research exploring if certain types of students are more likely to experience negative emotional responses than others. This study builds on extant qualitative studies through a quantitative examination of two previously identified student variables: different citizenship backgrounds (domestic and international) and different grade expectations (higher or lower than expected). The participants were 4514 students from two Australian universities. Analysis of survey data revealed that, regardless of language experience, international students were more likely than domestic students to find feedback comments to be discouraging, upsetting and too critical. Students who received grades lower than they expected on a particular assessment task were more likely than students who received grades higher than they expected to feel sad, shameful and angry as a result of the feedback comments. This paper concludes with several recommendations, including the need to modify assessment feedback practices in order to be sensitive to different student cohorts.     

Local university students and intercultural interactions: conceptualising culture,seeing diversity and experiencing interactions

This paper examines the intercultural interaction experiences of local, first-year students (n?=?25) in their first few weeks at university. The focus on local students complements existing intercultural interaction literature, which has tended to concentrate on the experience of the ‘cultural other’ student. Employing qualitative analysis, the study revealed relationships between how students conceptualise culture, see diversity and experience their initial intercultural interactions on campus. A link between students' cultural backgrounds and the depth of their intercultural interaction experiences emerged.     

Show and tell: Student and instructor perceptions of screencast assessment

This study addresses students’ and instructors’ perceptions of screencast assessment and compares this feedback method to traditional paper or digital text-based comments. Screencast assessment allows for asynchronous audio and visual commenting on student papers using screencast software. A pilot study using surveys of 39 students enrolled in lower-level composition classes and surveys of five composition instructors have indicated that screencast assessment promotes detailed and effective feedback on student writing. This feedback method reveals instructor's thought process while grading, and this transparency fosters student–teacher communication about writing. Screencast assessment changes the detail and types of comments, fostering an increased use of macro-level comments over micro-level concerns. The method does not necessarily save time grading but allows for feedback in the same amount of time. Because of the suggestion that students and instructors value screencast assessment, this study warrants a larger scale project in order to investigate the student preference for this feedback and whether the method leads to improvement in student performance.     

Is students’ qualitative feedback changing,now it is online?

Student feedback can be used to enable institutions to achieve the best possible outcomes for students through informing changes which enhance the quality of teaching and learning. Following the introduction of an online survey platform to gather student feedback at a top-performing UK university, anecdotal concerns raised by academics suggested students were being more critical. This exploratory study investigated the effects of switching from a paper to an online survey by comparing the open-ended responses between the two methods of collection. The main contribution of this study is the development of an analytical framework which can be used by other institutions to evaluate student comments, in order to understand and improve the student experience. Three key findings in this study were uncovered. Firstly, the quality of student feedback is not undermined by a switch to a more efficient online collection. Secondly, student comments via both methods of collection predominantly continue to focus on fulfilling basic needs, such as study resources. Finally, a small number of comments online revealed a lack of behavioural constraint and were considered to be inappropriate. These findings have important policy implications for the global higher education sector, highlighting the need for students to be given guidance on providing constructive feedback.     

Evaluating student satisfaction - restricting lecturer professionalism: outcomes of using the UK national student survey questionnaire for internal student evaluation of teaching

Abstract

In the neo-liberal context of a UK university, responding to student feedback in order to raise student satisfaction levels is important in improving National Student Survey (NSS) scores. This article focuses on the impact of a UK university’s new student feedback questionnaire - for individual modules - which used the NSS questions. The research draws on survey data (N?=?101) to identify lecturers’ views and three student focus groups. The outcomes raised issues relating to performativity, professionalism and ‘provision’, the latter defined as the university’s contract with each student, including the aspects that affect the student learning experience but are beyond the lecturers’ control, for example, class sizes and timetables. The results indicate that by recognising the impact of provision university managers may be better able to develop systemic improvements to student experience and (in the UK) a corresponding uplift in NSS and Teaching Excellence Framework (TEF) results. The article puts forward a model linking performativity, professionalism and provision to the relationships between university managers, academics and students. This model could enrich understandings of professionalism and performativity, extend the range of issues affecting student experience in SETs and support data analysis in future research studies.     

Enhancing student engagement in student experience surveys: a mixed methods study

Background

Measuring the student experience is becoming increasingly important in higher education in the UK. Student experience surveys are used as indicators of quality and form the basis of rankings of higher education institutions. They are also used by them as tools to assist their quality enhancement initiatives. However, these surveys frequently suffer from low response rates, which can reduce the reliability and usefulness of their data. The UK Postgraduate Taught Experience Survey (PTES) is a relatively new survey and suffers from a low response rate. As this survey is new, little is known about why students do not respond to it.

Purpose

This study aimed to explore the reasons why postgraduate students do not respond to the PTES.

Sample

Three hundred and fifty-five postgraduate taught students from four health faculties in one UK higher education institution completed an online survey. Of these, seven participated in one of two focus groups.

Design and methods

The online survey was completed both by students who completed the PTES in 2011 and those who did not. This provided us with cross-sectional data to compare both groups’ knowledge of PTES and their reasons for completing or not completing it. We used multivariate regression analysis to explore which variables were associated with response to PTES. We led two focus groups to explore the themes that emerged from the survey in more depth. This data was analysed by two researchers using thematic analysis.

Results

The cross-sectional data found that students who were not clear about the purpose of PTES were less likely to respond, independent of other potential predictor variables. Focus group data indicated that if postgraduate students felt a stronger connection to the university community they may be more likely to respond to PTES.

Conclusions

This study suggests that higher education institutions may wish to review their strategies for advertising student experience surveys to focus more on their purpose rather than their impact.     

Cultural dimensions of feedback at an Australian university: a study of international students with English as an additional language

International students with English as an additional language face transitional challenges when entering a new academic culture. One such challenge involves optimising feedback to help foster their academic development, bearing in mind that feedback is not a culturally neutral entity (Nazif, A., Biswas, D., &; Hilbig, R. (2004–2005). Towards an understanding of student perceptions of feedback. Carleton Papers in Applied Language Studies, 21/22, 166–192). The current study of 134 postgraduate international students examines pre-course and in-course perceptions, experiences and expectations of feedback at university in Australia. Responses to questionnaires and in focus groups revealed that students’ previous feedback experiences had largely been summative, with an emphasis on error correction, but they expected to receive more feedback, particularly formative, throughout their courses. They also had concerns about understanding feedback, and about potentially negative remarks. These concerns were partially borne out by a follow-up survey (n?=?43) in which only 29% said they understood everything their lecturers said. Teachers giving feedback to international students may, therefore, need to consider the nature of their comments, their method of delivery and how their feedback affects student development.     

Academic feedback in veterinary medicine: a comparison of school leaver and graduate entry cohorts

This study analysed the expectations and experiences of students on a five-year undergraduate (n?=?91) and four-year graduate entry (n?=?47) veterinary medicine degree programme relating to academic feedback. Qualitative and quantitative methodologies were used to explore new students’ expectations and prior experiences of feedback and capture experiences across one academic year. The majority of the students expect no less feedback at university than at school. Students’ experiences of the course highlighted themes of not knowing what was expected of them, a perceived need for more guidance and the importance of timely feedback. The impact of the staff–student relationship on how students obtain and perceive feedback and the emotional impact of positive and negative feedback were also highlighted. In addition, a recurring theme was the social context of veterinary medicine with issues relating to high academic achievers, competition between students and the need to gain professional and clinical skills. This study confirms a mismatch in student expectations versus experience. The paper draws on a rich data-set based on both quantitative and qualitative methods and is the first study of this type to be carried out in the context of students of veterinary medicine.     

Closing the loop on student feedback: the case of Australian and Scottish universities

Universities have a long history of collecting student feedback using surveys and other mechanisms. The last decade has witnessed a significant shift in how student feedback is systematically collected, analysed, reported, and used by governments and institutions. This shift is due to a number of factors, including changes in government policy related to quality assurance, and the increased use of the results by various stakeholders such as governments, institutions, and potential students and employers. The collection, analysis, and reporting of results are systematically carried out in many institutions worldwide. However, how to use student feedback to effectively improve student learning experience remains an issue to be addressed. This paper will contribute to this debate by comparing how Australian and Scottish universities use student feedback results to inform improvements. Based on thematic analysis of external quality audit reports of all Australian and Scottish universities, this paper suggests that universities have systematic processes to collect student feedback using a range of mechanisms, but limited work is done to use the data to inform improvements. This paper argues the need for universities to genuinely listen to student voice by facilitating partnership between students and institutions to act on their feedback as part of quality assurance.     

Do students value feedback? Student perceptions of tutors’ written responses

The topic of feedback to students is an under‐researched area, and there has been little empirical research published which focuses on student perceptions. This study explores student perceptions of written feedback and examines whether feedback received demonstrated a student‐centred approach to learning. A multi‐method approach of qualitative and quantitative data collection and analysis was used to survey 44 students in the faculties of Business and Art & Design. Student responses show feedback is valued, but believed tutor comments could be more helpful. Survey results indicate that students may need advice on understanding and using feedback before they can engage with it. Content analysis of feedback samples and student responses uncovered four main themes of feedback considered unhelpful to improve learning: comments which were too general or vague, lacked guidance, focused on the negative, or were unrelated to assessment criteria. It is suggested that by focusing on messages conveyed by their writing, providing feedback set in the context of assessment criteria and learning outcomes, and by ensuring that it is timely, tutors could greatly improve the value of feedback.     

A written,reflective and dialogic strategy for assessment feedback that can enhance student/teacher relationships

In response to the shortcomings of current assessment feedback practice, this paper presents the results of a study designed to examine students’ and teachers’ experience of engaging in a written, reflective and dialogic feedback (WRDF) strategy. The strategy was designed to enhance the learning experience of students undertaking a large first-year core course at a regional Australian university in semester 2, 2012. The evaluation consisted of three components: student surveys pre- and post-WRDF; a student focus group post-WRDF; and a teacher survey post-WRDF. Participating students’ and teachers’ perceptions of the WRDF assessment feedback suggested that students value feedback highly, and show a preference for feedback combining written, reflective and dialogic processes. The research findings suggest that the WRDF framework can be utilised to address the immediate, practical problem of students’ and teachers’ dissatisfaction with the practice of assessment feedback. Thus, WRDF may be used to nurture teacher/student relationships and enhance the learning process. Although a relatively intensive process, the WRDF strategy can serve an integral role in enhancing feedback practices and supporting students.     

Facilitating student learning in distance education: a case study on the development and implementation of a multifaceted feedback system

This paper reports on a case study conducted in an American university investigating the role of feedback within a distance education environment. Based on data gathered from online and hybrid undergraduate students in a teacher education program and supported by existing research, we describe how we support online learners by implementing a feedback system that incorporates multi-draft formative assessments and various feedback strategies. We describe these strategies (i.e. explicit feedback directions and feedback surveys) as well as data from formative assessments to demonstrate the impact of clear expectations and feedback on student performance in distance education. We present recommendations for online feedback and propose a feedback model for further study.     

Exploring cultures of feedback practice: the adoption of learning-focused feedback practices in the UK and Australia

ABSTRACT

In recent years, there have been calls in the literature for the dominant model of feedback to shift away from the transmission of comments from marker to student, towards a more dialogic focus on student engagement and the impact of feedback on student learning. In the present study, we sought to gain insight into the extent to which such a shift is evident in practice, and how practice is shaped by national and disciplinary cultures. A total of 688 higher education staff from the UK and Australia completed a survey, in which we collected data pertaining to key influences on the design of feedback, and the extent to which emphasis is placed on student action following feedback. Our respondents reported that formal learning and development opportunities have less influence on feedback practice than informal learning and development, and prior experience. Australian respondents placed greater emphasis on student action following feedback than their counterparts in the UK, and were also more likely than UK respondents to judge the effectiveness of feedback by seeking evidence of its impact on student learning. We contextualise these findings within the context of disciplinary and career stage differences in our data. By demonstrating international differences in the adoption of learning-focused feedback practices, the findings indicate directions for the advancement of feedback research and practice in contemporary higher education.     

Closing the feedback loop? Iterative feedback between tutor and student in coursework assessments

We evaluate the case for using feedback iteratively, to improve student engagement and learning. In this model, students were invited to respond to tutor feedback with students’ own responses. Among the three courses/modules (three tutors) studied, differences in feedback styles were evident from: (a) thematic analysis of tutor comments and, for one course, and (b) comments in a student focus group. Students were inconsistent in their evaluation of tutor feedback but, in a more detailed study of one of the courses, there was clear evidence for the benefits of iterative feedback. Lessons from the main study were then applied to a course that had not previously incorporated iterative feedback. Using this experience, we provide suggestions for applying iterative feedback in assessments.     

Group Projects in Social Work Education: The Influence of Group Characteristics and Moderators on Undergraduate Student Outcomes

This study examined the impact of group size, group formation, group conflict, and division of labor on student outcomes in a group project for a sample of 112 BSW research seminar students at a large university in the Midwest. Students completed surveys on their experiences with the group project at the end of the semester. Multiple regression was used along with analysis of student comments and answers to open-ended questions. Group size was not associated with outcomes. Instructor input into group formation was related to higher self-assessment of student performance. Greater division of labor predicted greater learning, whereas more group conflict predicted less learning; less conflict occurred when students divided tasks in the most equitable way possible. Strengths and limitations of the study are discussed, along with suggestions for improving the group project experience, including preventative measures to reduce conflict.     

Improving the student learning experience through dialogic feed-forward assessment

Abstract

Assessment feedback from teachers gains consistently low satisfaction scores in national surveys of student satisfaction, with concern surrounding its timeliness, quality and effectiveness. Equally, there has been heightened interest in the responsibility of learners in engaging with feedback and how student assessment literacy might be increased. We present results from a five-year longitudinal mixed methods enquiry, thematically analysing semi-structured interviews and focus groups with undergraduate students who have experienced dialogic feed-forward on a course in a British university. We use inferential statistics to compare performance pre and post-assessment intervention. The assessment consisted of submitting a draft coursework essay, which was discussed and evaluated face-to-face with the course teacher before a self-reflective piece was written about the assessment process and a final essay was submitted for summative grading. We evidence that this process asserted a positive influence on the student learning experience in a number of inter-related cognitive and affective ways, impacting positively upon learning behaviour, supporting student achievement and raising student satisfaction with feedback. We advocate a cyclic and iterative approach to dialogic feed-forward, which facilitates learners’ longitudinal development. Programme teams should offer systematic opportunities across curricula for students to understand the rationale for and develop feedback literacy.     

Developing evidence for action on the postgraduate experience: an effective local instrument to move beyond benchmarking

Summative and benchmarking surveys to measure the postgraduate student research experience are well reported in the literature. While useful, we argue that local instruments that provide formative resources with an academic development focus are also required. If higher education institutions are to move beyond the identification of issues and benchmarking practices, the scope of survey results and their reporting need to enable and foster appropriate changes in disciplinary practices. Robust, locally developed instruments can provide detailed, programme-specific information and foster timely changes in practice with direct benefits for postgraduate respondents. Unlike benchmarked surveys, local tools can adapt to explore and examine specific concerns of students, supervisors and academic developers. Coupling high-response rates and follow-on engagement with participant feedback, well-designed local instruments provide clear and irrefutable indicators to programme and university administrators of specific disciplinary strengths and weaknesses in postgraduate pathways. In this paper, we discuss the development of a research student survey specifically designed to support academic development purposes in strengthening and enhancing the postgraduate experience.