The value of institutional research in the assessment of institutional effectiveness

A survey of 167 institutions engaged in the regional reaccreditation process between 1987 and 1992 revealed that the presence of institutional research activities was positively related to the institution carrying out a systematic assessment of institutional effectiveness. Institutional research activity was more predictive of assessment efforts than was the type of governance (public or private), the level of degree offered, or enrollment size. An alarmingly low proportion of institutions had begun to define outcomes at the time of the survey—a first step in the development of assessment procedures. Even fewer had identified ways of measuring outcomes. Budgets and staff expertise were felt to be adequate for current responsibilities but insufficient to assume responsibility for assessing institutional effectiveness. Institutional research activity is urgently needed in order to support the data collection efforts required by institutional assessment mandates. Whether these activities are conducted by institutional researchers or are dispersed across the campus, they provide the underpinnings for assessment.     

The importance of institutional image to student satisfaction and loyalty within higher education

This paper outlines the findings of a study employing a partial least squares (PLS) structural equation methodology to test a customer satisfaction model of the drivers of student satisfaction and loyalty in higher education settings. Drawing upon a moderately large sample of students enrolled in four ‘types’ of Australian universities, the findings suggest that student loyalty is predicted by student satisfaction, which is in turn predicted by the perceived image of the host university. While the perceived quality of “humanware” (e.g., people and process) and “hardware” (e.g., infrastructure and tangible service elements) has an impact on perceived value, this was found to be weak and indeterminate. Of most importance was the impact of the institution’s institutional image, which strongly predicted perceived value, and to a lesser extent student satisfaction. The findings have implications for newer, less prestigious universities seeking to compete in a more deregulated, market driven environment.     

The commuter student and student satisfaction

This study examined the variables associated with student satisfaction on a commuter campus. A sample of 38 classes was used and resulted in 782 undergraduates responding to questionnaires. The variables included were: age, student level, student's grade, perception of educational benefits, and satisfaction with college experience. A causal model was built and tested in order to determine the direct and indirect effects of these variables upon student satisfaction. While the student level had a negative influence upon satisfaction, both age and grade had a moderate effect upon student satisfaction. The perception of educational benefits seemed to be the most critical variable in predicting student satisfaction.     

The effect of working condition on math teacher effectiveness: value-added scores and student satisfaction in teaching

The purpose of this study is to better understand how math teachers’ effectiveness as measured by value-added scores and student satisfaction with teaching is influenced by school’s working conditions. The data for the study were derived from 2009 to 2010 Teacher Working Condition Survey and Student Perception Survey in Measures of Effective Teaching Project. Using the structural equation modeling and other related methods, several models of teacher effectiveness were estimated. The findings indicate that among the examined working condition factors, support for instruction and for student conduct management have significant effects on teachers’ value-added scores in mathematics. Moreover, the student satisfaction with teaching seems to have a mediating effect on value-added scores. The findings of the study significantly contribute to a better understanding of the effects of working environment on math teachers’ effectiveness and how improvement in working conditions can enhance math teachers’ performance.     

Understanding student/alumni satisfaction from a consumer's perspective: The effects of institutional performance and program outcomes

The benefits of understanding and promoting student/alumni satisfaction are numerous. While this article draws on previous studies of the determinants of satisfaction in higher education, it has a different research focus. The study investigates the student/alumni satisfaction phenomenon in a consumer satisfaction framework. Results indicate that the formation of satisfaction judgments is a multidimensional process, and that the process is dependent on the degree of goal development that a student has for a particular aspect of his or her educational experience. If students have poorly formed goals for an aspect of education, they are likely to base their satisfaction judgments on institutional performance. If their goals are well formed, they are likely to base satisfaction judgments on the outcomes of the institutional performance. In general, however, student/alumni assessments of satisfaction with higher education are influenced by both the perceived quality of the service provider's performance and the perceived outcomes of that performance.The authors' names are ordered alphabetically to reflect equal contributions.     

The role of relationships and emotions in student perceptions of learning and assessment

The paper addresses the way in which participants in a qualitative study drew upon accounts of relationships and emotions in sharing their perceptions of assessment. By first exploring ideas about emotions and relationships in learning and assessment through the literature and subsequently discussing an interpretation of participant narratives, the author suggests that emotions and relationships surrounding past learning and assessment contexts can influence current perceptions of assessment and learning in powerful ways. The study also reveals how students value opportunities to express their beliefs, feelings and emotions during the assessment process. They also expect teachers to balance objectivity in assessment with empathy for those parts of themselves shared in the process. The conclusion is drawn that there are important implications for learning in the emotional response of students to assessment and in the nature of teaching and learning relationships that are worthy of further investigation.     

The influence of technology-mediated and in-person communication on student satisfaction: The moderating role of national culture

This article analyses the effects of technology-mediated (i.e., social media) and in-person communication (i.e., non-verbal cues) on student satisfaction in a higher education context. Data were collected among 221 college students from the University of Valencia (UVEG) in Spain and analysed from the perspective of the respondents' national culture. Contrary to expectations, the results show that neither one of the social media aspects drives student satisfaction, thus providing support for the technology paradox literature. Instead, three non-verbal communication cues are found to influence student satisfaction, i.e., paralanguage, kinesics, and chronemics. The moderating role of national culture on the examined relationships is also considered. The results reveal that national culture seems to only affect the relationship between chronemics and satisfaction.     

A Theory of Motivation and Ontological Enhancement: The role of disability policy in student empowerment and institutional change

As debate continues around the nature and values of education, it is important to ask the question of what factors motivate a student to engage with the ends of an educational institution. In this paper, a broad, holistic view of learner motivation, derived from Aristotelian ethics, is used to provide a model to drive institutional change.
Focussing on the approach of one Higher Education institution to the particular accommodations required for students with disabilities, the paper identifies three factors which motivate students, a failure to engage with the aims and ends of the educational project, a failure to see that a particular learning aim is worth attaining, and a simple lack of will-power to attain it. To each of these failures a social cause is identified, and a change in both the institutional culture and the individual learner's approach to their education is suggested.     

A Theory of Motivation and Ontological Enhancement: The role of disability policy in student empowerment and institutional change

As debate continues around the nature and values of education, it is important to ask the question of what factors motivate a student to engage with the ends of an educational institution. In this paper, a broad, holistic view of learner motivation, derived from Aristotelian ethics, is used to provide a model to drive institutional change.

Focussing on the approach of one Higher Education institution to the particular accommodations required for students with disabilities, the paper identifies three factors which motivate students, a failure to engage with the aims and ends of the educational project, a failure to see that a particular learning aim is worth attaining, and a simple lack of will‐power to attain it. To each of these failures a social cause is identified, and a change in both the institutional culture and the individual learner's approach to their education is suggested.     

Predicting student progression: The influence of race and other student and institutional characteristics on college student performance

In this study of college student retention and progression, significant differences were found between black and white student cohorts in terms of their attrition rates, overall progression rates (defined as length of time to graduate), and tendency to follow the prescribed progression pattern (sophomore in the second year, junior in the third year, senior in the fourth year, and graduate after the fourth year). However, multiple regression analyses show that racial differences disappear when the effects of other student and institutional characteristics are statistically controlled. Therefore, colleges and universities would do well to rethink special retention and counseling programs designed especially to serve minority group students.     

The influence of student gender on the assessment of undergraduate student work

The aim of this study was to investigate the influence of perceived student gender on the feedback given to undergraduate student work. Participants (n = 12) were lecturers in higher education and were required to mark two undergraduate student essays. The first student essay that all participants marked was the control essay. Participants were informed that the control essay was written by Samuel Jones (a male student). Participants then marked the target essay. Although participants marked the same essay, half of the participants (n = 6) were informed that the student essay was written by Natasha Brown (a female student), while the remaining participants were informed that it was written by James Smith (a male student). In-text and end-of-text feedback were qualitatively analysed on six dimensions: academic style of writing; criticality; structure, fluency and cohesion; sources used; understanding/knowledge of the subject; and other. Analysis of feedback for both the control and target essay revealed no discernible differences in the number of comments (strengths of the essay, areas for improvement) made and the content and presentation of these comments between the two groups. Pedagogical implications pertaining to the potential impact of anonymous marking on feedback processes are discussed.     

The role of institutional research in college management

Abstract

Institutional research is well established in postsecondary institutions in the USA as an effective tool for provision of accurate, reliable data to support management decision making. Institutional research offices are much less prevalent in such institutions in the UK however, particularly in colleges. The purpose of this paper is to outline the principal roles and functions of an institutional research office, briefly discuss relevant staffing issues (including important qualifications), and suggest some practical considerations regarding the role of institutional research in college management.     

Personal and institutional characteristics of student failure

There is a limited literature on academic failure in higher education despite the fact that failure is a common experience among undergraduates. Student and course characteristics were examined in relation to proportions of fail grades assigned in a Faculty of Arts. Concerns had been expressed that combinations of two grades, F (failure to reach a satisfactory standard) and FNS (failure to submit assessment work), resulted in high fail rates. Regression analyses indicated that students assigned higher proportions of fail grades tended to be male, Indigenous, younger, had lower entry scores, had deferred fees, were not full‐time day students, lived at home, and had histories of incomplete studies. However, university entrance score was the strongest predictor of F, while being male and deferring payment of fees were the strongest predictors of FNS. Variations in awarding types and levels of fail grades were evident between schools in the faculty, indicating different grading practices. The findings suggest that personal and institutional factors interacted to increase the risk of academic failure for these students.     

The role of statements of institutional purpose

This paper assesses the utility and role of institutional purpose statements. If they are as effective as one is led to believe, what intrinsic facts or elements make them so? Is there value in having a purpose statement, or is the value attributed to the exercise of creating and discussing it? If there is such a value, what forms and circumstances create the value? What forms do mission statements usually take? Do different forms have different attributes? In addressing these questions, two methods are employed. One is conventional in that literature documenting the theory and research of others has been carefully reviewed, but from the particular perspectives of form and effectiveness. The other is to examine a series of actual statements of institutional purpose, with particular regard to form, content, and context of planning. To elucidate the context of planning, a series of master plans and mission statements for systems of higher education are also examined. In total, 32 institutional statements and 12 system plans or statements are examined. (The plans that are examined are listed in Appendixes A and B.) The paper observes that mission statements are effective in some situations, but not in all. In some situations they may be disadvantageous. Although planning theorists suggest that mission statements follow an approximately common form, the study of actual statements indicates several different types.     

The role of scales in student ratings

Questions used in course evaluations should mainly measure the quality of teaching, and students' answers to those questions should not be influenced by other factors. This paper investigates how seemingly neutral rating scales and multiple-choice questions might have an impact on the results of such evaluations. In several studies, it has been shown that the way in which a scale is constructed may strongly influence the answers it elicits in surveys and tests. Whether and to what extent common course-related ratings of students are also affected by the kinds of scales used is the main topic of this paper. Four studies examined the influence of scale polarity (unipolar vs. bipolar scales), the role of different ranges in time scales, and the impact of ordering choices in a certain way. Participants' ratings and answers were strongly influenced by all these manipulations. It is recommended to pay special attention to the role of scales when constructing questionnaires for course evaluations and when interpreting course evaluation reports.