Three views of fairness in a school-based assessment scheme of practical work in biology

This interpretive study of the implementation of a school-based assessment scheme of biology practical work in Hong Kong examines three teachers' struggles with the issue of fairness in relation to their classroom actions. Though the teachers' discourses were dominated by, and their classroom actions were pre-eminently influenced by, the notion of fairness, they did so in three qualitatively different ways: (1) fair in the sense of assessing students on a fair basis; (2) fair in the sense of not jeopardizing students' chances to learn the subject matter while they are being assessed; and (3) fair in the sense of not depriving students' of opportunities of receiving all-round education. The implication is that assessment innovation is a necessary, but not a sufficient, mechanism for changes within our educational system. The role of the teacher is challenged by the new assessment scheme?the co-existence of assessment and learning requires a significant change in the teacher's pedagogy. For teachers to implement the new programme their existing understanding and beliefs concerning assessment must be challenged and opportunities provided for them to come to terms with the philosophy of the new assessment scheme. Most importantly, the teachers themselves must undertake such a learning process.     

Raising the stakes: the challenges for teacher assessment

These are changing times in Australia for teachers and their students, with the introduction of a national curriculum and standards driven reform. While countries in Europe such as England, and in Asia such as Singapore, are changing policy to use assessment in the support of and improvement of learning it appears that we in Australia are moving towards creating policy that will raise the assessment stakes for the putative purposes of transparency, accountability and fairness. What can be learnt from countries that have had years of high stakes testing? How can Australia avoid the mistakes of past curriculum and assessment reform efforts? And how can Australian teachers build their capacity to maximise their use of the learning power of assessment? These are the questions that are addressed in this article, with reference to innovative research from global networks that have maintained the assessment focus on learning rather than accountability practices.     

Assessment drives learning: An unavoidable truth?

The debate around which factors drive medical students' learning is ongoing and controversial. What is the influence of an assessment's weighting on the motivation of students to study the particular subject? One medical school in London is in a unique position to investigate this question. At our institution, the weighting of Anatomy within the overall scheme of assessment has changed twice in recent years, a trend of increased weighting. This enabled a comparative investigation into the effect these changes have had on the students' motivation to learn Anatomy. A five‐point Likert‐scale questionnaire survey was used to evaluate students. A section within a broad survey of Anatomy teaching and learning at our institution was dedicated to the evaluation of the amount of weighting Anatomy received within the assessment structure and the effect this had on students' motivation toward learning the subject. Increasing Anatomy's weighting within the scheme of assessment produced a dramatic increasing trend toward students' motivation to learn Anatomy. The weighting of Anatomy has a profound effect on students' motivation to learn it. Although multifactorial and complex in nature, medical students' self‐reported drive to study a subject is directly influenced by the weighting of the subject in the overall scheme of assessment. Anat Sci Educ 2:199–204, 2009. © 2009 American Association of Anatomists.     

Assessing students’ performance by measured patterns of perceived strengths: does preference make a difference?

This paper is an empirical field study of whether college students’ preferences for assessment type correspond to their performance in assessment that tests that particular strength. For example, if students say they prefer assessment that tests their creativity, do they actually perform better on assessment tasks requiring the use of creativity? Seventy-eight students in three different courses were surveyed to determine their preferences in four types of assessment: memorisation, analysis, creativity and practical application. These preferences were then compared to student grades on corresponding forms of assessment to see if the preferences corresponded to actual performance. The study found that, while students had a clear preference for memorisation, they were not likely to deliver their best performances on memorisation tasks. There was no relationship at all between student preferences in assessment type and their performance in the respective assessments. These results indicate that, while in theory assessing students based on their preferences is reasonable for improved learning, we were not able to find evidence that it actually leads to higher performance.     

关于高校德育评估信度问题的探讨

德育评估信度是指德育评估结果的可靠或稳定程度。可分为重测信度、复分信度、对半信度和主评信度。评估主体因素、方案因素、客体因素和过程因素都会影响评估信度。选拔优秀评估人员、制订合适方案、有效控制评估过程、取得客体良好配合是提高评估信度的可行办法。     

Working toward integrated practice: Understanding the interaction among formative assessment strategies

The purpose of this study was to directly observe and investigate the factors that support or hinder the implementation of an integrated approach to formative assessment and to understand how the implementation of formative assessment strategies interact. The authors present findings from a collective case study that included the observation of six teachers (three mathematics and three English language arts) who were purposefully sampled from a large, public, suburban high school in the northeast United States. The analysis of the case studies resulted in the identification of different patterns in teachers’ formative assessment practice. These patterns illustrate how the complex interaction between formative assessment practices can either support or hinder the implementation of an integrated approach to formative assessment and suggests classroom observations as a potential mechanism for identifying challenges and informing just-in-time professional development.     

Active citizenship: a preliminary study into student teacher understandings

Background:?Assessment grades are ‘estimates’ of ability or performance and there are many reasons why an awarded grade might not meet a candidate's expectations, being either better or poorer than anticipated. Although there may be some obvious reasons for grade discrepancies, such as a lack of preparation or under-performance, there are a number of technical issues to consider, such as the potential effects of random measurement error, human error and grade misclassification. However, traditionally, there has been limited information available to the public about such issues.

Purpose:?This study formed part of a two-year investigation into the reliability of public examination outcomes in England and the current paper explores participants’ narratives relating to one of the themes that emerged from the study of public perceptions of assessment reliability. It examines how individuals interpreted and rationalised their examination results, particularly those that failed to meet expectations, and discusses the impact that such results may have on individuals’ academic self-concept.

Sample and method:?Ten focus groups were conducted across five qualification user groups:?two each with employees, employers, teachers, trainee teachers, and job-seekers (74 participants in total). A flexible discussion schedule was employed to explore participants’ experiences and perceptions of assessment reliability.

Main findings:?Participants tended to internalise ‘blame’ for results that were poorer than expected by constructing explanations that focused on a perceived lack of preparation, ability or knowledge. These experiences appeared to have a negative impact on individuals' academic self-concept. Secondary school teacher participants shared experiences of marking, technical and standard setting errors, and were more aware than other qualification user groups of the external factors that can impact on assessment outcomes.

Conclusion:?Examination results that are poorer than expected can threaten individuals’ academic self-concept, confidence in their ability, and influence their study and career intentions and opportunities. A better understanding of educational measurement issues may offer individuals a more informed framework for understanding their examination results, especially where results do not meet expectations.     

Student Loans: The Failure to Consolidate an Emerging Political Consensus

This article contends that over a long period of time there was growing support for a student loan scheme. In the event, the Thatcher Government adopted proposals which failed to consolidate that emerging consensus. What was it about the Government's legislation that achieved this political own goal? How can such politically maladroit behaviour be explained? What elements should the legislation have contained if the consensus were to be firmed up and enhanced? These are the key questions which we attempt to answer in this article.     

Interdisciplinary project working in engineering education

The purpose of this paper is to give an overview of the interdisciplinary project programme implemented by the Faculty of Technology at Plymouth University teamed with three Royal Academy of Engineering Visiting Professors in Principles of Engineering Design. We examine the challenges encountered in developing such a scheme and identify the processes that were put in place to ensure the quality of the course and to support and maximize student performance. Prior to commencement of the programme the Dean and Vice Chancellor had developed clear objectives and expectations for building upon the significant achievements of the six schools comprising the Faculty of Technology. These objectives were: (a) For the Faculty: ? Provide a focus and impetus for the development of multidisciplinary design in a new portfolio of Masters-level (4 year) engineering courses. ? Enhance the teaching of design in all engineering courses. ? Strengthen a design-oriented approach to all teaching in engineering courses. ? Enhance students' enthusiasm and awareness through greater contact with senior practitioners. ? Improve the understanding of both staff and students concerning the principles of modern engineering design and manufacturing/construction in contemporary professional practice. ? Provide experience and an understanding of the value of interdisciplinary working. (b) For the students: ? Give students the opportunity to tackle problems outside their disciplines. ? Help students understand the relationships between different disciplines. ? Develop innovative and creative solutions to open-ended problems. ? Develop management skills. ? Develop personal and interpersonal skills. ? Focus on developing team-working skills. ? Develop communications and presentation skills. In addition to providing students with a significant intellectual challenge the interdisciplinary project is designed to help them gain a broader insight into problem-solving and in dealing with uncertainty--the essences of engineering. Our experience leads us to believe that both undergraduate and postgraduate engineering students would benefit from working in interdisciplinary teams whilst at university and that having done so they would leave university better fitted for careers in engineering.     

基于层次分析法的国外典型巨灾风险保险模式评价

通过分析美国国家洪水保险计划、英国洪水保险、日本地震保险三种典型巨灾风险保险模式的主要因素,提出一个由保险险种、保险方式、保险金额、保险费率、经营方式、经营风险六大要素构成的巨灾风险保险模式评价指标体系和评价标准,建立多层次的巨灾风险保险模式评价模型,并利用专家判断法和层次分析法确定各种方案的权重值,得出最优的巨灾风险保险模式,从而为我国巨灾风险保险模式的建立提供决策依据。     

Does calibration reduce variability in the assessment of accounting learning outcomes?

Reliable, consistent assessment process that produces comparable assessment grades between assessors and institutions is a core activity and an ongoing challenge with which universities have failed to come to terms. In this paper, we report results from an experiment that tests the impact of an intervention designed to reduce grader variability and develop a shared understanding of national threshold learning standards by a cohort of reviewers. The intervention involved consensus moderation of samples of accounting students’ work, with a focus on three research questions. First, what is the quantifiable difference in grader variability on the assessment of learning outcomes in ‘application skills’ and ‘judgement’? Second, does participation in the workshops lead to reduced disparity in the assessment of the students’ learning outcomes in ‘application skills’ and ‘judgement’? Third, does participation in the workshops lead to greater confidence by reviewers in their ability to assess students’ skills in application skills and judgement? Our findings suggest consensus moderation does reduce variability across graders and also builds grader confidence.     

The internal consistency of a concept mapping scoring scheme and its effect on prediction validity

This study examines the internal consistency of Novak and Gowin's scoring scheme and its effect on the prediction validity of concept mapping as an alternative science classroom achievement assessment. Data were collected in three typical situations: very limited concept mapping experience with free‐style concept mapping; some concept mapping experience with questions provided; extensive concept mapping experience with a list of concepts provided for. It was found that Novak's scoring scheme was not internally consistent, and therefore there was generally no significant correlation between students’ scores on concept mapping and students’ scores on conventional classroom achievement assessments. The need for a new scoring scheme when concept mapping is used as an alternative science assessment is discussed.     

Quality assessment of undergraduate education in China: impact on different universities

This research analyzes the scheme proposed to assess the quality of higher education institutions in China, namely, the Quality Assessment of Undergraduate Education (QAUE) scheme. This article aims to determine the impact of the QAUE on universities and explore the reasons that intended effects have or have not been generated in the evaluated universities by conducting case studies of three Chinese universities with different statuses. The empirical studies show the effects on the various dimensions of quality provisions at different universities were not the same. It was found that the impact of the QAUE was not a linear consequence of policy implementation, but the result of an interaction between the external quality assessment scheme and the evaluated universities. Quality assessment is regarded to be an external force to cause universities to change. This empirical study of the QAUE shows that changes will only take place when the external force is integrated with the evaluated universities’ internal motivation and capacity.     

Applying computerized-scoring models of written biological explanations across courses and colleges: prospects and limitations

Our study explored the prospects and limitations of using machine-learning software to score introductory biology students’ written explanations of evolutionary change. We investigated three research questions: 1) Do scoring models built using student responses at one university function effectively at another university? 2) How many human-scored student responses are needed to build scoring models suitable for cross-institutional application? 3) What factors limit computer-scoring efficacy, and how can these factors be mitigated? To answer these questions, two biology experts scored a corpus of 2556 short-answer explanations (from biology majors and nonmajors) at two universities for the presence or absence of five key concepts of evolution. Human- and computer-generated scores were compared using kappa agreement statistics. We found that machine-learning software was capable in most cases of accurately evaluating the degree of scientific sophistication in undergraduate majors’ and nonmajors’ written explanations of evolutionary change. In cases in which the software did not perform at the benchmark of “near-perfect” agreement (kappa > 0.80), we located the causes of poor performance and identified a series of strategies for their mitigation. Machine-learning software holds promise as an assessment tool for use in undergraduate biology education, but like most assessment tools, it is also characterized by limitations.     

Students’ reception of peer assessment of group-work contributions: problematics in terms of race and gender emerging from a South African case study

Participatory assessment is increasingly employed in higher education worldwide as a formative mechanism to support students’ active learning. But do students in an increasingly relationally diverse environment perceive that peer assessment of individuals’ contributions to group-work tasks enhances their learning? Recognising the impact of students’ conceptions on the quality of their learning, this study considers students’ perspectives of peer assessment of group-work contributions at a South African university. Questionnaires elicited students’ perspectives of and general attitudes towards assessment of and by their peers. A growing measure of discontent with the process of assessing peer contributions to group tasks emerged, including actual and perceived racial and gender stereotyping, and related rejection-sensitivity. These initial findings were checked against the students’ experiences in a report-and-respond process that enabled probing discussions of the interpretations. This paper examines and explores the implications of such identifications and receptions for learning engagement and group-work curriculum development in the context of a rapidly transforming higher education sector.     

Education students and their teachers: comparing views on participative assessment practices

Beliefs about the validity and merit of self‐, peer‐ and group‐assessment practices are presented from 213 pre‐service primary teachers and 30 staff who teach them. Both groups were surveyed using comparable items. A subset of seven staff participated in semi‐structured interviews. Staff were far more supportive of peer‐ and self‐assessment practices than their students with both groups indicating modest support for group assignments. While pedagogical factors best represent the staff support for engaging students in assessment of their own and their peers’ work, several staff in interviews revealed that their high level of support for peer‐assessment was closely linked to a time‐saving factor. Peer‐assessment was more often used than self‐assessment while both practices were reported to have increased over the past two to three years. These findings are consistent with the reported increase in participative assessment practices in higher education generally. An important implication of this research is that in order to optimise the use of participative assessment, staff need to better prepare their students by modelling and communicating their reasons for adopting such practices.     

把高校招生考试制度改革的目标愿景锁定好

高校招生考试制度改革不仅是理论上“为什么要改、改什么”和思路上“怎么改”的问题,更重要的是实践的问题。要进行改革,就需要做一个基础工作,即把长远改革目标愿景锁定好。一旦锁定,鉴于全国各地不同现状的不同起点,我们可从中设计出若干条不同起点而归于目标愿景的光滑曲线,即改革方案。本文从多元模式、科学考试、综合评价、自主招生、和谐管理等五个方面,分析了高校招生考试制度的现状,并提出了改革的目标建议。     

Les Langues Étrangères: apprenant,enseignant, situation

What determines the success of individuals when learning foreign languages in school? This article describes briefly the most important factors. These are grouped in three categories according to their relevance for the learner, the teacher or the learning situation. Factors such as intelligence, attitude, personality, age and sex are presented in a model which attempts to determine their comparative importance and their interrelations.     

Affording the Unaffordable: Cost Sharing in Higher Education in Sub-Saharan Africa

The purpose of this article is to discuss some financial aspects of the future development of higher education in Africa entering the 21st century. Two main aspects can be distinguished: first, problems that concern the economy as a whole, the macroeconomic problems; second, questions that apply to the individual institutions of higher education. The former group includes questions such as: How much will the future expansions of higher education in Africa cost? How large is the capacity of African economies to finance this expansion and to what extent can tuition be applied? The article makes three recommendations to Africa's higher education: flexible tuition fees; a well-designed loan scheme; and a proactive measure to promote access and quality.     

Changing the marks-based culture of learning through peer-assisted tutorials

We describe and evaluate an approach to student learning that aims to instil a culture of formative assessment based on peer-assisted learning. The idea is for suitably qualified undergraduates to assist in the running of weekly first-year tutorials. They mark submitted work, provide written and verbal feedback and lead problem-solving discussions during tutorials. However, contrary to normal practice, the marks they award do not contribute to the students’ end-of-year total; all tutorial work becomes essentially voluntary. We report results from a pilot implementation of the scheme over a 12 month period in an engineering department at a leading academic institution. The scheme was such that a comparative and triangulated assessment was possible among the students and tutor team. Results show no discernible degradation in student attendance, submission rates and performance in either the weekly exercises or end-of-year examinations. Important benefits to the peer tutors are also found.