A conducive environment? The role of need support in the higher education workplace and its effect on academics' experiences of research assessment in the UK

Little is known about how researchers in higher education institutions (HEIs) experience and respond to support received from their departments. The present study investigated how support for researchers' autonomy (choice and self‐expression), relatedness (through connections with colleagues) and competence (feeling effective in one's work) influenced their attitudes towards an external assessment of research. To do so, we surveyed 598 academics from four HEIs in the UK about their attitudes towards one such external assessment: the Research Excellence Framework (REF), a nationwide assessment of research quality and the subject of debate about research evaluation. Our findings, drawing on self‐determination theory, show that departments can shape responses to the REF: individuals whose psychological needs were supported by their academic departments held more positive, and less negative, attitudes towards the REF. This occurred both directly and indirectly through researchers' recognition that the REF had a more positive influence on their research activities and outputs.     

Student responses to criteria‐referenced self‐assessment

This paper reports on a study of undergraduate students’ experiences with criteria‐referenced self‐assessment. Fourteen students who had taken a course involving self‐assessment were interviewed in focus groups segregated by gender. The findings suggest that students had positive attitudes toward self‐assessment after extended practice; felt they can effectively self‐assess when they know their teacher’s expectations; claimed to use self‐assessment to check their work and guide revision; and believed the benefits of self‐assessment include improvements in grades, quality of work, motivation and learning. There were indications that some students sensed a tension between their own standards for good work and some of their teachers’ standards. There was no evidence of differences in the responses of male and female students. The paper concludes with the suggestion that self‐assessment involves a complex process of internalization and self‐regulation, and with implications for research and practice.     

Looking for a balance between internal and external evaluation of school quality: evaluation of the SVI model

This article describes the results of a study into the utility of the SVI model, a model in which internal and external evaluation are balanced. The model consists of three phases: school self‐evaluation, visitation and inspection. Under the guidance of school consultants, 27 Dutch primary schools have built up two years of experience with the SVI model. The results show that the school leaders developed a positive attitude towards school self‐evaluation and visitation. They found that both self‐evaluation and visitation have improved their insight into the quality of the school. However, a content analysis of the school self‐evaluation reports shows that the school self‐evaluations are often of low quality. For example, it appeared that most of the self‐evaluation reports do not provide answers to questions the schools have formulated at the beginning of the self‐evaluation. Moreover, the teams of critical friends and the inspectors concluded that the school self‐evaluations do have many shortcomings. Based on these results, we conclude that school self‐evaluation is a very difficult task for most schools. It is therefore crucial that schools receive external support and that they build up experience with school self‐evaluations over a period of years.     

Standardization techniques for grade‐inflation problems at higher educational institutions of Ethiopia: the case of Addis Ababa

Cumulative Grade Point Average (CGPA) is a measure that is used to display the achievement of college students in Ethiopia. It also serves as a key yardstick in career and scholarship assessment. In recent time there has been a rapid massification of higher educational institutions (HEIs). Many academics believe that the expansion has increased a tendency for grade inflation in Ethiopia. As an objective comparison of the quality of graduates from different HEIs is not easy, this study has attempted to discern the CGPA differentials across sample HEIs in Ethiopia and to derive a new standardization technique for their comparison. The study identifies different grading patterns, not only across college categories but also along various batches of HEIs. To generate the standardization technique, both CGPA and the ESLCE (Ethiopian School Leaving Certificate Examination) are employed to derive the Normalized Index. The index is used to adjust not only the inflated but also the deflated CGPAs. The quality of the index could be further improved by incorporating additional variables and also by validating the method over a large set of HEIs.     

Qualitatively different ways of experiencing student self‐assessment

In the literature, student self‐assessment as a practice and as a goal in higher education is generally emphasized for academics. This paper reports a study that investigates what academics emphasize in their experiences of student self‐assessment. The investigation focused on the different ways academics described their understanding and practise of self‐assessment. A phenomenographic approach was used to research and identify a set of progressive variations of academics’ ways of experiencing student self‐assessment. Altogether, 16 academics from a variety of disciplines and programs of study in three Australian universities participated in the investigation. The consequent research findings describe five qualitatively different conceptions that depict how academics understand and use student self‐assessment. These findings are subsequently discussed in terms of the potential for academics to understand and use student self‐assessment to enhance students’ self‐assessment ability, to further students’ lifelong learning and to empower, rather than discipline, students.     

Introducing Students to Self‐assessment

This study describes the introduction of self‐assessment to third‐year education students as a strategy for improving learning. Students were required to self‐assess a literature review using the marking guide developed by the course lecturer. The results showed a good level of agreement between students and the teacher but with a tendency for more students to overestimate than underestimate their grades. Interviews with students indicated that those who over estimated their grades were less clear about the standard by which to judge their work and looked for more specific guidance on what the lecturer ‘wanted’ than students whose estimates were closer to the lecturer's. The former group also tended to include ‘effort’ as a factor in their judgement. Students who responded more positively to the self‐assessment exercise were also more positive about the value of having a marking guide given out with the assignment. The paper concludes with suggestions for improving the study and some advice to other teachers who are contemplating the introduction of self‐assessment.     

Accuracy of students' self‐assessment and their beliefs about its utility

The purpose of the two studies presented here was to evaluate the accuracy of students' self‐assessment ability, to examine whether this ability improves over time and to investigate whether self‐assessment is more accurate if students believe that it contributes to improving learning. To that end, the accuracy of the self‐assessments of 3588 first‐year students enrolled in a post‐secondary institution was studied throughout a semester during which each student made approximately 80 self‐assessments about his or her own learning process. These self‐assessments were then compared with multiple judgements by peers and tutors. The overall correlations between the scores of self‐, peer and tutor assessments suggest weak to moderate accuracy of student self‐assessment ability. The findings also reveal an ability effect; students judged as more academically competent were able to self‐assess with higher accuracy than their less competent peers. Comparing the accuracy of student self‐assessment averaged over four consecutive periods indicates that the accuracy does not improve over time. In a second study, a questionnaire aimed at eliciting student's beliefs about the effects of self‐assessment on their learning was administered to 936 first‐year students. Based on their responses, sub‐groups of students were identified: those who either believed in the usefulness of self‐assessment or did not. Results suggest that there is no significant association between student beliefs about the utility of self‐assessment and the accuracy of their self‐assessments.     

Students’ conceptions of assessment: Links to outcomes

Students conceive of assessment in at least four major ways (i.e., assessment makes students accountable; assessment is irrelevant because it is bad or unfair; assessment improves the quality of learning; and assessment is enjoyable). A study in New Zealand of 3469 secondary school students’ conceptions of assessment used a self‐report inventory and scores from a standardised curriculum‐based assessment of reading comprehension. Four inter‐correlated conceptions based on 11 items were found with good psychometric properties. A path‐model linking the four correlated conceptions with student achievement in reading, while taking into account student ethnicity, student sex, and student year, had good psychometric properties. The conception that assessment makes students accountable loaded positively on achievement while the three other conceptions (i.e., assessment makes schools accountable, assessment is enjoyable, and assessment is ignored) had negative loadings on achievement. These findings are consistent with self‐regulation and formative assessment theories, such that students who conceive of assessment as a means of taking responsibility for their learning (i.e., assessment makes me accountable) will demonstrate increased educational outcomes.     

Relationships between student characteristics and self‐, peer and tutor evaluations of oral presentations

There are many influences on how assessors grade themselves and others. Oral presentations are useful for exploring such factors in peer, self‐ and tutor marked assessments, being rapidly completed and assessed, commonly used in HE and very difficult to anonymize. This opportunistic study examined the effects of gender and level of attainment on the triangulation of marks awarded to student presenters. Grades generated by peer assessment were associated more strongly with tutor‐awarded marks than those from self‐assessment. For self‐assessment there was a strong effect of gender (female students undervalued their performance compared with tutor grades). Peer assessment produced higher marks than from tutors, perhaps because of the close‐knit community developed during residential courses. For tutor marks, the greatest variability was at the lower end of the scale, whereas peer assessors were most variable when marking students who self‐evaluated or peer assessed highly. Students awarded a narrower range of marks to peers compared with tutors, but when self‐assessing used a larger range. Presentations by students who admitted to little sleep the night before received lower grades from both peers and tutors, but this was not reflected by self‐assessments, suggesting they were unaware of their poorer performances. Sessions with fewer talks (four rather than seven) reduced the ‘dip’ in marks previously observed in the middle of sessions. Findings are discussed in the context of bias in this mode of assessment.     

AN INSTRUMENT TO MEASURE SELF‐RESPONSIBILITY FOR WELLNESS IN OLDER ADULTS

Literature relating to the well‐being of older adults was reviewed to identify indicators relevant to the construct of self‐responsibility for wellness. The wellness model proposed by Travis (1981) has produced a variety of concepts which can be useful in improving the quality of life for older adults. The purpose of this study was to develop an instrument which would assess an individual's self‐responsibility for wellness. A 47‐item instrument developed for this purpose was evaluated by experts in gerontology and psychology. After revision and reevaluation it was field‐tested on a sample of 180 older adults (60 years of age and over). In order to take preliminary steps in establishing the validity and reliability of this instrument, the data were evaluated and an item analysis conducted to identify poor items. Cronbach's coefficient alpha was also computed (α = .90). A test‐retest correlation coefficient was computed, and an analysis of variance was performed to test for the relationship between self‐responsibility for wellness and demographic variables obtained during the field test.

The field testing of the instrument served as an educational needs assessment study. Evidence has been provided that there is a significant need for education programs which can provide training in the wellness skills as assessed by the instrument.     

A Study in Self‐assessment: tutor and students’ perceptions of performance criteria

This paper reports a study which implemented and evaluated a method of student self‐assessment. The theme of the importance of marking criteria (Orsmond et al., 1996) is developed. Pairs of first‐year undergraduate biology students were asked to complete a poster assignment on a specific aspect of nerve physiology. The study was designed to allow the evaluation of (1) student self and tutor marking for individual marking criteria; and (2) student vs student marking of their poster work for individual marking criteria. A comparison between the tutor and the student self‐assessed mark reveals how important it is to consider the individual marking criteria rather than the overall mark. These results support our findings on peer assessment reported earlier (Orsmond et al., 1996). The results also illustrate the potential pitfalls that exist when making assumptions about the degree of student/student interaction and students’ marking ability. This study supports previous work which showed that self‐assessment is extremely useful in helping students reach their learning goals, i.e. it is a strong formative educational tool and can be used in order to bring about behavioural changes in students with regard to their own learning processes.     

Assessing ‘inexperienced’ students' ability to self‐assess: exploring links with learning style and academic personal control

The study sought to establish the level of students' self‐assessment skill—particularly inexperienced students—and to examine the relationship between self‐assessment skill and learning style, student perceptions of academic locus of control and academic self‐efficacy. Students were asked to evaluate and provide estimated marks for their own work, were which compared with tutors' actual marks. Students also completed measures of learning style, academic locus control and academic self‐efficacy. Comparisons of student estimated and tutor marks indicated a good level of self‐assessment skill in the majority of students. A significant minority of students did however fail to exhibit such skills. There was also some evidence of a tendency for students to underestimate their performance. While both strategic and deep approaches to learning were shown to be positively correlated with tutor mark, only surface approach was negatively correlated with students' estimated mark, suggesting that surface learners are inclined to provide lower evaluations of their own performance. Deep approach was also correlated with accuracy of student self‐assessment skill, suggesting that deep learning is associated with self‐assessment competency. No clear or convincing associations between self‐assessment skill and perceptions of academic locus of control or academic self‐efficacy were identified. Findings suggest that while self‐assessment skill undoubtedly develops, becoming more effective during students' academic career, inexperienced students do have the capacity for self‐evaluation and should therefore be included in self‐assessment activities. In the light of findings related to learning style and the heterogeneous nature of student groups, student monitoring and skill development are proposed in order to allow the integration of self‐assessment into the learning and assessment process.     

Self‐assessment in university assessment of prior learning procedures

Competency‐based university education, in which lifelong learning and flexible learning are key elements, demands a renewed vision on assessment. Within this vision, Assessment of Prior Learning (APL), in which learners have to show their prior learning in order for their goals to be recognised, becomes an important element. This article focuses on a first step in APL, namely students’ self‐assessment of their prior learning before entering university education. The main aim of the presented study is to examine the suitability of the use of self‐assessment in APL. First, in an explorative study, the main sources for self‐assessment are derived and the relation between sources and domain of study is investigated. Second, in a pre‐test post‐test research design, the hypothesis that students’ self‐assessment of prior learning related to a course changes after studying a domain‐specific course is tested. Pre‐test results reveal that students indicate that they have prior knowledge related to the chosen university programme. In general, this prior learning is obtained from study experience, work experience, books, newspapers, magazines, the Internet, TV, radio, film or video. A relation is found between the type of source and the university programme. The hypothesis that students change their self‐assessment after a study period could not be confirmed. Based on these results, it is concluded that self‐assessment in APL might be a suitable tool. Implications for further research are discussed.     

Self‐regulation empowerment program: A school‐based program to enhance self‐regulated and self‐motivated cycles of student learning

This article describes a training program, Self‐Regulation Empowerment Program (SREP), that school professionals can use to empower adolescent students to engage in more positive, self‐motivating cycles of learning. It is a two‐part approach whereby self‐regulated learning coaches (SRC) (a) use microanalytic assessment procedures to assess students' self‐regulation beliefs and study strategies and (b) train students to use these strategies in a cyclical, self‐regulation feedback loop. Ultimately, students learn how to set goals, select and monitor strategy effectiveness, make strategic attributions, and adjust their goals and strategies. The program was developed from social‐cognitive theory and research and integrates many of the essential features of the problem‐solving model. Interventions used in the SREP include graphing, cognitive modeling, cognitive coaching, and structured practice sessions. A case study is presented to illustrate procedures for implementing the program. Implications for school psychologists and teachers also are presented and discussed. © 2004 Wiley Periodicals, Inc. Psychol Schs 41: 537–550, 2004.     

Peer and self‐assessment in the first year of university

This paper reviews the literature about peer and self‐assessment in university courses from the point of view of their use, and the suitability of their use, in the first year of university study. The paper is divided into three parts. The first part argues that although first‐year students are involved in many of the studies that report on the use of peer and self‐assessment in higher education, the proportion of these studies that do so is somewhat less than in other year levels. In addition, relatively little of this work directly and explicitly discusses the suitability of peer and self‐assessment for students and courses at this year level. The second part of the paper provides an introductory exploration of the relationship between peer and self‐assessment, and specific features of first‐year assessment, learning and teaching. Three issues relating directly to the suitability of peer and self‐assessment in the first year are explored. In the third part, the paper briefly discusses the desirability of implementing peer and self‐assessment, in general, before seeking to extend this specifically to the first year. The paper concludes by recommending that greater use can and should be made of peer and self‐assessment in the first year of university study.     

Discrete Choice Modeling in Education: An Innovative Method to Assess Teaching Practices

Self‐reported measures are an easy, time‐efficient, and low‐cost way to collect research data compared to other assessment methods. They are, however, characterized by several limitations regarding the quality and the clarity of the data they produce, especially when they are implemented in education. The main purpose of the current study was to use a method called discrete choice modeling (DCM) in education, in order to develop a self‐reported instrument that will reduce the bias for socially desirable responses and to assess teachers’ practices in physical activity. DCM method has the ability to overcome the respondents’ tendency to answer in a socially desirable way. A nationally representative sample of 531 Greek early educators participated in this study and were administered a self‐assessment questionnaire based on DCM. Results showed that the DCM based newly developed instrument manages to assess effectively educators’ practices and provided evidence of the applicability of the method in education. Further implications and future recommendations regarding the effective application of DCM in education are discussed.     

Trainees' perspectives on the assessment and self‐assessment of surgical skills

Self‐assessment is argued to be valuable educationally and is being advocated through policies for wider adoption within medical practice. However, students are often reluctant to self‐assess and little research has examined the reasons for this. This study explored the perceptions of trainees and postgraduates on assessment and self‐assessment of surgical skills. Data were collected through semi‐structured interviews conducted with six trainees and postgraduate students in Oral Surgery. They had previously been assessed, and self‐assessed themselves for the first time, while surgically removing a mandibular third molar tooth. Assessments were judged against both an objective checklist scale and a global rating scale. Eight themes were identified and trainees’ comments on these themes analysed. The themes identified were assessment and stress, pressure to over‐ or under‐score, the impact of self‐assessment on learning, reflection and performance, confidence and the necessity of preparation, acceptability of the scales and criteria used, feedback, and the acceptability of self‐assessment. In conclusion we need to instil in our trainees a culture whereby they feel capable of and comfortable making judgements about their own performance and giving and receiving constructive assessment from their peers.     

Quality,pedagogy and governance in private Higher Education Institutions in Egypt

Building on a mixed method research approach, this article reports on an analysis of the difference between public and private higher education institutions (HEIs) in Egypt in terms of teaching methods, quality assessment approaches and alumni engagement. An analysis of the survey data compared the experiences of 1 713 graduates of both private and public HEIs. Qualitative case study data seeks to explain the noted differences and similarities. The analysis showed that both types of institutions fall short of providing student-activating teaching methods; engaging students in quality assessment; or supporting graduates through job placement services. The analysis related the similarity between the two types of institutions to the state governance approach and its focus on education inputs, recommending a shift to an accountability regime that is focused on education outcomes and performance.     

Promoting Professional Development in Higher Education through Portfolio Assessment

The evaluation of the capacity for reflective and critical thinking at the transition stage from pre‐service education to working life is a challenging task for the development of quality assessment in higher education. Beginning teachers can organise their plans, reflections, observations and work samples in what is called a portfolio, reflecting on their experiences. The purpose of the present study is to evaluate professional development in higher education based on personal portfolios during a year‐long induction programme. The trainees were new teachers about to enter working life. The development essays and interviews with the participants based on their personal portfolios were analysed through qualitative methods. The results indicate that self‐assessment is an important tool for professional development.     

QUALITY ASSURANCE AND INSTITUTIONAL ACCREDITATION IN EASTERN AND CENTRAL EUROPEAN HIGHER EDUCATION SYSTEMS: PROCEDURES AND OPERATIONAL ASPECTS: WORKING DOCUMENT

This article forms the Working Document for the Conference bearing the same title. It reflects the fact that this Conference is the second in what is planned to be a series on quality assessment and accreditation in higher education. Thus it calls for an appraisal of the transition from policy statements to operational developments. The participants in the Conference are invited to describe developments in their countries in terms of the adoption of legal provisions; the establishment of bodies in charge of quality assessment and institutional accreditation; the identification of criteria, performance indicators, and assessment standards; the development of procedures for self‐study and peer‐review; and the identification of the policy and administrative consequences for higher education of the processes of quality assessment and accreditation.