Interpreting context to the UK’s National Student (Satisfaction) Survey data for science subjects

Universities capture and use student feedback to improve the student experience, but how should information from national scale surveys be used at local and institutional levels? The authors explored the UK’s National Student (Satisfaction) Survey (NSS) data relevant to science and engineering programmes using percentages of students who were satisfied or very satisfied. For brevity, one NSS national dataset was explored, but the patterns found were consistent for the following year. Simple exploratory data analysis techniques underlined the care that is needed when interpreting NSS outputs, corroborating previous research into its international precursors. Factor analysis supported claims of a high internal consistency of the survey. Subject groupings showed consistent differences in responses, with some subjects consistently recording higher satisfaction. This reduces the usefulness of the NSS for comparing different subject groupings within a university. Universities provide different subject compositions, so direct comparisons between institutions are not straightforward. Subject groupings should be compared only against similar subjects, and then with due care to understand the complexity of satisfaction. Further analysis of national questionnaires like this is required to contextualise its outputs. For example, there is a national trend of low satisfaction with assessment feedback in all subjects, but the relationship between feedback satisfaction and overall satisfaction is complex. There are marked differences between subjects which may, in science subjects, be associated with mathematical content. There is scope for future elucidations of the ‘overall satisfaction’ value and for use of the measures of dissatisfaction.     

The UK National Student Survey: An amalgam of discipline and neo-liberal governmentality

The UK National Student Survey (NSS) has high status on the agenda of UK universities. Its rise in status is linked to its influence on national rankings and associated funding streams referenced to the Teaching Excellence Framework (TEF). Consequently, many universities have implemented further assessments of student satisfaction, thereby putting additional internal performative pressures on courses and individual lecturers. The research contribution of this article comprises an analysis of the NSS through Foucault's notion of ‘governmentality’, with a particular focus on his work on ‘discipline’ and ‘neo-liberal governmentality’. More specifically, by utilising qualitative data from interviews, research diaries and observations, it will be demonstrated how the NSS functions as a ‘disciplinary’ technology of government which subjects lecturers, departments and universities to intersecting panoptic gazes and perpetual ratings. In addition, the NSS can also be considered ‘neo-liberal’ in that it governs the academic population through narrow conceptions of ‘freedom’ and omnipresent competition. The article proposes that it is through the amalgamated forces of intersecting panoptic gazes, on the one hand, and neo-liberal free-market principles, on the other, that student feedback develops its power to govern.     

Coherence between text comments and the quantitative ratings in the UK’s National Student Survey

Institutions are understandably interested in the profile of their own reputations based upon publicly available data about student experiences. The UK’s National Student Survey (NSS) metrics are integrated into several ‘Good University’ calculations, whereas teaching teams most often use the survey’s text comments to change practices, rather than the metrics directly. There is little information about how messages from the national survey’s text comments relate to the accompanying numerical ratings, partly because text comments are confidential to the institution and unavailable for wide-scale research. We categorised institutional NSS text comments into themes that mirrored those of the original questionnaire. Comparisons were made between frequencies of thematic comments and the national ratings of satisfaction for several subject areas. For the first time, we demonstrate broad agreement between comments about measures of teaching staff and course organisation with the performance of the subject areas (compared to metrics of their national counterparts). These findings are consistent with previous quantitative models predicting the most important factors that most influence overall satisfaction ratings. We intend this study to be a catalyst for other institutions to explore their non-publicly available, textual returns in a similar way. The outcomes of this type of work are pertinent to all countries that use large-scale surveys. However, institutions will need to release findings to a public audience if we are to gain a national/international perspective on this key linkage between publicly available metrics and the associated text comments.     

A conceptual framework for analysing the impact of influences on student engagement and learning

The notion that ‘customer satisfaction’ should be the ultimate measure of quality provision of any service organization is often accepted in the higher education context. However, measuring the quality of an educational institution based on students’ satisfaction is insufficient as it diverts the focus from student development, advancement and growth to an affective evaluation of the service. More appropriate measures of quality of higher education institutions are student engagement and learning. This study describes the likely impact of perceived overall quality of higher education institutions on student engagement. The contribution of this study is twofold. First, it draws attention to the importance of higher education institution quality as the main institutional-level determinant of student engagement and, second, it suggests a comprehensive multilevel conceptual framework for its empirical testing.     

Evaluating student satisfaction - restricting lecturer professionalism: outcomes of using the UK national student survey questionnaire for internal student evaluation of teaching

Abstract

In the neo-liberal context of a UK university, responding to student feedback in order to raise student satisfaction levels is important in improving National Student Survey (NSS) scores. This article focuses on the impact of a UK university’s new student feedback questionnaire - for individual modules - which used the NSS questions. The research draws on survey data (N?=?101) to identify lecturers’ views and three student focus groups. The outcomes raised issues relating to performativity, professionalism and ‘provision’, the latter defined as the university’s contract with each student, including the aspects that affect the student learning experience but are beyond the lecturers’ control, for example, class sizes and timetables. The results indicate that by recognising the impact of provision university managers may be better able to develop systemic improvements to student experience and (in the UK) a corresponding uplift in NSS and Teaching Excellence Framework (TEF) results. The article puts forward a model linking performativity, professionalism and provision to the relationships between university managers, academics and students. This model could enrich understandings of professionalism and performativity, extend the range of issues affecting student experience in SETs and support data analysis in future research studies.     

Student satisfaction,league tables and university applications: Evidence from Britain

We investigate the impact of information about student satisfaction on university choice, using data from the UK's National Student Survey (NSS) and on applications to undergraduate degrees. We show that NSS scores have a small statistically significant effect on applications at the university-subject level. This effect operates via the influence of the NSS on a university's position in separately published, subject-specific league tables, suggesting that information contained in the league table rankings is more salient. The impact of rankings is greater for more able students, for universities with entry standards in the upper-middle tier, and for subject-departments facing more competition.     

Student satisfaction and perceptions of quality: testing the linkages for PhD students

Student satisfaction and the quality of education are of compelling interest to students, academic staff, policy-makers and higher education researchers internationally. There is a widespread belief in their ‘cause and effect’ relationship. This paper tests these beliefs and explores how the level of student satisfaction is linked with the perceived quality of PhD education. Using expectancy value theory as a framework and interview data from PhD students and their supervisors, this paper suggests that satisfaction is not necessarily perceived as an indicator of quality education. Levels of student satisfaction can be influenced by students’ expectations prior to their study and their preconceived beliefs regarding the value of a PhD education. Concern is raised that an overemphasis on student satisfaction may pose a threat to the quality of PhD programmes, making it increasingly difficult for universities to retain their integrity and reducing the intellectual challenges that PhD students need to experience.     

National Student Survey: are differences between universities and courses reliable and meaningful?

The National Student Survey (NSS) of course experience satisfaction is sent to final year students of all higher education institutions in England, Northern Ireland, Scotland and Wales (N = 171,290 and 157,342 students in 2005 and 2006) and results are made publicly available. The present investigation assesses the reliability and appropriate use of the NSS from a multilevel perspective. Although NSS responses provide a limited basis for discriminating amongst universities and courses within universities, the ratings of universities are highly reliable and stable over time due to the large number of students (2005 and 2006 rankings correlated r = .86). The unresolved question is whether very small (only 2–3% of the variance explained) but reliable and stable differences between universities provide useful information for benchmarking universities, self‐improvement, and informing student choice.     

Satisfied teachers are good teachers: The association between teacher job satisfaction and instructional quality

Instructional quality is associated with better academic outcomes for students. This study aimed to investigate how teachers' job satisfaction was associated with clarity of instruction and cognitive activation as measures of instructional quality. In addition, we investigated whether this association between teachers' job satisfaction and instructional quality was mediated by teacher–student relationships. Drawing on the 2018 Teaching and Learning International Survey (TALIS), we compared participants from both Eastern (N = 27,106; Japan, Taipei, Korea, Shanghai) and Western sociocultural contexts (N = 20,209; Canada, Australia, New Zealand, United States, United Kingdom). Multilevel structural equation modelling results indicated that teachers' job satisfaction was positively associated with instructional quality across Eastern and Western settings. The relationship between teachers' job satisfaction and instructional quality was partially mediated by better student–teacher relationships. There were some differences between the cultural settings in how job satisfaction correlated with clarity of instruction and cognitive activation. We suggest that these differences may be accounted for by cultural characteristics leading to different approaches to teaching. Our results suggest that teachers' job satisfaction and the quality of classroom-level relationships may be important indicators of positive instructional outcomes. While schools focus on student outcomes, they should also address teachers' job satisfaction and prioritise the importance of relationships between teachers and students in classrooms.     

Assessing antecedents and consequences of student satisfaction in higher education: evidence from Malaysia

This paper explores the interrelationships of the key influences on student satisfaction via multivariate analysis from three groups of university students in two popular private universities in Malaysia. The correlation coefficient and structural model indicated that student satisfaction is influenced not only by academic quality, but also by the university core services, information technology services, and skill building. These dimensions are linked to one another. The strong interrelationships between these dimensions are indicative that holistic approach needs to be adopted rather than dealing with each dimension in isolation in creating a valued education. Significant performance gaps of key influences indicated that the satisfaction levels are below students’ expectation despite the satisfaction scores being above average. Academic assessment, teaching quality, and IT are areas where a university management needs to prioritize resources in order to increase student satisfaction. Students who perceived higher value are more likely to have higher repurchase intention.     

Postgraduate student satisfaction: A multilevel analysis of PTES data

Student satisfaction has received growing attention in Higher Education systems in recent years, and are increasingly used for internal and external accountability in the sector. This does leave us with questions on the extent to which variance in student satisfaction can be explained by Higher Education Institution (HEI) and course attended rather than by individual student characteristics; and on what factors may predict student satisfaction. In this study we used the Postgraduate Taught Experience Survey (PTES) to look at these questions in a sample of approximate 70000 UK postgraduate students from 100 HEI's. Firstly, confirmatory factor analyses were used to test the validity of the proposed multidimensional structure of student satisfaction in PTES. Then multilevel models were used to look at variance at three levels (HEI, course and student), and the relationships between institutional and student factors collected in PTES and student satisfaction. Two years of data (2014 and 2015) were used. Findings suggest that over 90% of variance is explained at the student level for all dimensions of student satisfaction in both years, with variance at the HEI level being particularly low. Individual student characteristics explain more variance than institutional characteristics, but only some (such as BME status) are significant, and explained variance is low. These findings suggest using student satisfaction as an accountability measure in HE may be highly problematic.     

Student evaluation of courses: what predicts satisfaction?

The main goals of course evaluations are to obtain student feedback regarding courses and teaching for improvement purposes and to provide a defined and practical process to ensure that actions are taken to improve courses and teaching. Of the items on course evaluation forms, the one that receives the most attention and consequently the most weight is the question, ‘Overall, I was satisfied with the quality of this course.’ However, no attention has been placed on examining the predictors of students being ‘satisfied with the quality of this course’ overall. This study attempts to address this gap. The findings show that while student characteristics and reasons for enrolling in a course are predictors of overall satisfaction, it is the evaluation questions that predict the majority of the variation in course satisfaction. The findings also reveal that faculty‐selected optional questions are stronger predictors of overall satisfaction than compulsory questions.     

Student perception of Moodle learning management system: a satisfaction and significance analysis

This research study addresses differences in student perception of the significance of Moodle learning management system (LMS) quality characteristics and differences in student satisfaction in regard to such characteristics. Therein, it has been proven in this study that male and female students are equally satisfied with Moodle LMS quality characteristics and that there is a difference in the significance that students give to these characteristics. When students were observed in regard to their age and year of study, it was found that these groups assigned different significance levels to quality characteristics and were not equally satisfied with them. It was also found that there is a substantial statistical difference in the significance students gave to quality characteristics and in student satisfaction itself, according to how much time they spent using the Moodle application, which is also noted as one of the most important aspects of the research conducted. A further analysis of the variables demonstrated that the following components of quality characteristics were more important to female students: average waiting time for a response, feedback quality, material thoroughness, material clarity, website user-friendliness, cooperation diversity, and material quantity.     

对教育服务质量和学生满意度的实证分析

在对天津市中学生教育的实证调研基础上,建立了教育服务质量感知和重要程度的四分表,并发现学校服务质量和学生满意度正相关,其中有形性、保证性、实践性和可靠性与满意度高度相关。     

Influence of improved teaching practices on student satisfaction ratings for two undergraduate units at an Australian university

To track the quality of instruction delivered at the University of Western Australia, the university surveys all units using its Students’ Unit Reflective Feedback (SURF) metrics, and faculties use these metrics to benchmark student satisfaction. Consequently, teaching staff are actively encouraged to adopt teaching practices that will increase the average levels of these metrics. Using a ‘before-after’ approach, we compared these metrics before and after the implementation of improved teaching practices that addressed specific weaknesses identified through student responses for two undergraduate units. Despite the implementation of improved teaching practices, SURF scores did not increase significantly for the two units assessed, due possibly to the components of the academic programme covered by the SURF questions, the field-based nature of the units, and the difficulty in measuring an increase in the SURF scores of units which already score highly. This poor sensitivity of the SURF metrics to the implementation of improved teaching practices could have implications for their use for performance assessment by the university.     

Determinants of PhD student satisfaction: the roles of supervisor,department, and peer qualities

Understanding the determinants of PhD student satisfaction is likely to become increasingly vital for universities as student satisfaction rankings already ubiquitous at undergraduate and master degree levels extend more broadly to the PhD level. Moreover, as PhD student populations and university competition become increasingly transnational, there is a growing need to understand cross-nationally common determinants of satisfaction. Building on prior research into PhD student satisfaction, and drawing upon relevant conceptual and metrical refinements in the measurement of satisfaction from cognate domains of psychology, we use cross-sectional data (N?=?409) from PhD candidates across the sciences, social sciences and humanities in 63 universities from 20 countries to examine how overall PhD student satisfaction is determined by, respectively and in combination, supervisor, department and peer-group, in terms of both their academic qualities and supportiveness. Taken together, we find that supervisor supportiveness is the greatest predictor of PhD student satisfaction, but that supervisor academic qualities have no significant effect. However, both the academic qualities and supportiveness of departments significantly predict PhD student satisfaction, suggesting university departments and PhD supervisors would ideally work jointly, and perhaps more closely than many currently do, to achieve competitive levels of PhD student satisfaction.     

Multilevel analysis of the effects of student and course characteristics on satisfaction in undergraduate liberal arts courses

The purpose of this study is to analyze the effects of student and course characteristics on student satisfaction in courses by using a two-level hierarchical linear model (HLM). Based on literature reviews, 13 research variables (1 student gender, 2 student academic year, 3 student major, 4 student reason for taking the course, 5 student level of course participation, 6 student expected grade, 7 student achieved grade, 8 faculty gender, 9 faculty age, 10 faculty status, 11 academic field of the course, 12 class size, and 13 course type) were selected and specified as fixed effects in the analysis model. Data were 57,216 ratings of 1,481 undergraduate liberal arts courses at Seoul National University in 2006. The result of unconditional model analysis revealed that student characteristics’ effects on student satisfaction in courses (within-course effects, 88.1%) were much larger than course characteristics’ effects (between-courses effects, 11.9%). The result of conditional model analysis specifying student and course level predictors revealed that those 12 research variables, with the exception of student gender, had statistically significant effects on student satisfaction in courses. The explained variance was 22.0% in student level, 65.8% in course level, and 27.2% of the total variance. An earlier version of this paper was presented at the 8^th International Conference on Education Research, Seoul National University, Seoul, Korea, October 23–25, 2007.     

Preparing for success: one institution's aspirational and student focused response to the National Student Survey

This paper describes quality enhancement (QE) focused response to the 2006 National Student Survey (NSS) by a post-1992 Higher Education Institution. Recognising the increasing importance of the NSS to a wide range of stakeholders, the University established a task team to explore, from a QE perspective, why the institution received particular scores (both high and low), and to engage in dialogue with students about our response to the results. This article describes just one aspect of this approach: an innovative, student-focused institutional level event, which was forward looking and resulted in tangible actions and outputs for colleagues and students.     

基于毕业生满意度的高校服务型管理模式建构

调查了应届毕业大学生对四年大学生活的满意度,通过分析他们对大学生活的满意度、教师上课效果的认可度,对就读大学的情感归属度,对大学未来发展的期望度,以及对职业素质认知度等五个角度,有效解读了高校服务型管理模式所面临的挑战。围绕"学生的满意度和高等教育质量有机结合,大学教育的目标和学生的需求有机结合,美好的办学理想同有效的办学载体有机结合"三个结合,提出了服务型高校管理模式的建构路径。