Formative computer‐based assessment in higher education: the effectiveness of feedback in supporting student learning

A formative computer‐based assessment (CBA) was one of three instruments used for assessment in a Bachelor of Education course at Queen’s University (Ontario, Canada) with an enrolment of approximately 700 students. The formative framework fostered a self‐regulated learning environment whereby feedback on the CBA was used to support rather than measure student learning. The four types of feedback embedded in the CBA included: (a) directing students to a resource, (b) rephrasing a question, (c) providing additional information and (d) providing the correct answer. Although students originally reported positive experiences with the formative CBA, two follow‐up surveys revealed that they found the four types of feedback to be moderately effective in supporting their learning.     

Mixing distance and face‐to‐face higher education

There is an unreal demarcation in higher education between distance and face‐to‐face providers, argues Malcolm Tight, Director of the Centre for Research and Development in Part‐time Higher Education at Birkbeck College, University of London. Given the undoubted success of distance education, established by the Open University and others, there is now a strong case for diversifying methods of provision in higher education, in the interests of expanding opportunities for students and increasing the effectiveness of teachers.     

Variation in the development of teachers’ understandings of assessment and their assessment practices in higher education

This paper reports a study into the development of staff understanding of assessment and assessment practice. Eight teachers from two universities constructed an initial concept map about assessment that was discussed in a one-to-one semi-structured interview. A year later, a new map was created and the interview focused on change in thinking and practice. Multiple models of assessment were evident in the participants’ understandings at the same time and change was characterised by subtle evolution in thinking. Development in practice was more significant and often associated with the foregrounding of assessment for learning. Vignettes are used to illustrate the variation in nature and scale of development. Interplay between this development of practice and understanding was multidirectional and external context played an important role. The approach offers detailed insight into the relationship between assessment thinking and practice and demonstrates that both research and academic development need to go beyond conventional approaches to conceptualising the development of academics and take account of the finer grained complexities of assessment thinking and practices.     

Curriculum‐embedded performance assessment in higher education: maximum efficiency and minimum disruption

Increasingly, institutions of higher education are required to evaluate student progress and programme effectiveness through implementation of performance assessment practices. Faculty members frequently resist performance assessment because of concerns that assessment activities will increase workloads, reduce time for scholarly activities, eliminate professional autonomy, and reduce faculty work into component parts or discrete technical competences. This paper describes how curriculum‐embedded performance assessment can be used to evaluate student and programme effectiveness without placing an undue burden on faculty. Examples of the use of curriculum‐embedded performance assessment strategies in a graduate‐level educational psychology programme are provided.     

The myth of objective assessment — An analysis in relation to primary education

This article focuses on the recent growth and development of the field of Early Childhood Education and Care (ECEC) in England and discusses the issue of ‘quality’ in relation to the diversity of provision on which these developments are based. In particular the article discusses the ways in which cultural constructions of childhood and the theories of child development and learning espoused by stakeholders impact on assessments of ‘quality’.     

The value of non‐participation in higher education

The paper seeks to link the structural and the institutional to learning outcomes in order to articulate a research agenda capable of evaluating the impact of consumerism on learning and teaching in higher education. Consumerist mechanisms are situated in the context of quasi‐market and new managerial regulatory frameworks and concepts developed by Pierre Bourdieu are drawn on to establish a theoretical model of the uneven impact of consumerism across different types of universities. Empirical studies, conducted in a variety of national settings, are outlined to confirm the plausibility of the model. The possible interactions between first, changes in academic identity, teaching and the curriculum; and secondly, on student identities and their impact on teaching and assessment and their consequent learning outcomes, are outlined. Some important questions about the consequences for the labour market are also raised. The paper hypothesizes that attempts to restructure pedagogical cultures and identities to comply with consumerist frameworks may unintentionally deter innovation, promote passive and instrumental attitudes to learning, threaten academic standards and further entrench academic privilege. The paper concludes by outlining key areas that require investigation in order to address some of the problems posed by consumerism in a mass higher education system.     

E‐learning and the transformation of social interaction in higher education

This article examines the way in which e‐learning is transforming the nature of social interaction in higher education. In this new educational environment, radical societal transitions and the opportunities afforded by modern communication technologies together produce formidable challenges. Significant as these challenges may be, concentration upon problems of a practical kind draws attention away from the more theoretical concerns in understanding e‐learning. By drawing together developments in social, educational and communicational theory and Gilly Salmon’s hands‐on approach to teaching and learning online, this article reveals some unintended consequences: e‐moderation and the use of e‐tivities may perpetuate the very conditions that limit our chances of dealing successfully with the challenges posed by e‐learning. While theory may muddle what might otherwise be communicated meaningfully to those in search of practical answers, theoretical developments provide concepts and frameworks that can be placed in the service of a critical understanding of e‐learning and the transformation of social interaction in higher education.     

Academics’ perceptions of the benefits and challenges of self and peer assessment in higher education

Despite compelling evidence of its potential effectiveness, uptake of self and peer assessment in higher education has been slower than expected. As with other assessment practices, self and peer assessment is ultimately enabled, or inhibited, by the actions of individual academics. This paper explores what academics see as the benefits and challenges of implementing self and peer assessment, through the analysis of interviews with 13 Australian academics. Thematic analysis of our qualitative data identified seven themes of benefits and five challenges. Our academics showed strong belief in the power of self and peer assessment as formative assessment, contrary to past literature which has focussed on the accuracy of students’ marking. This paper therefore brings insights as to not only what academics value about self and peer assessment but also identifies potential inhibitors in practice. Recommendations are made about improving the design and implementation of self and peer assessment in higher education.     

Validity and reliability of portfolio assessment in pre‐service teacher education

This article concentrates on the validity and reliability of portfolio assessment as used in pre‐service teacher education. It is not possible to make general pronouncements about the validity of portfolio assessment in pre‐service teacher education as there are multiple portfolio applications. The validity depends on the purpose, namely the divers competencies which the course organisers wish to assess with it. Therefore, three categories of competencies and consequently three types of portfolios were distinguished in order to determine the validity of portfolio assessment. For the assessment of teaching and partnership competencies, it is argued that the validity is low due to the roundabout nature of the assessment. On the contrary, the validity of portfolio assessment for learning competencies can be high. The execution of a self‐regulated learning process can be accurately assessed using portfolios. The reliability of portfolio assessment is problematic, since it is incapable of fulfilling the classic psychometric requirement of reliability. Nevertheless, provided that the necessary measures are taken, the reliability of portfolio assessment can still be brought to an acceptable level. Five measures are proposed.     

Who aspires to higher education? Axes of inequality,values of education and higher education aspirations in secondary schools in Luxembourg and the Swiss Canton of Bern

This article reports a study that investigated secondary school students’ higher education aspirations (towards university studies, ISCED 6 and above) and how these differ between student groups as well as how these are impacted by values of education. Panel data of more than 300 secondary school students in two countries, Luxembourg and Switzerland (the Swiss Canton of Bern) was analysed. Schools are structured differently in the education systems of Luxembourg and the Swiss Canton of Bern. The results of our analysis show that students in the Luxembourgish sample more often aspire to higher education than in the Swiss sample. Disparities in higher education aspirations were also more pronounced in the Luxembourgish sample, boys and students from families of low socio‐economic status (SES) were less likely to aspire to higher education. While the effects of values of education are generally scarce, stimulation in terms of anticipated enjoyment and interest derived from participation in higher education seems to have a positive effect on higher education aspirations.     

Quality assurance of higher education in transition countries: Accreditation ‐ accountability and assessment

Abstract

One issue facing countries in transition concerns how to ensure quality for a broader and more diverse set of institutions. Quality assurance processes (i.e. accreditation, accountability and assessment) have particular relevance today as higher education undergoes dramatic changes in countries in transition, as the world becomes more internationalised, and as private higher education institutions increase in number and enrol increasing numbers of students. Three questions emerge. Do differences among transition countries relative to central control prior to independence explain the current set of quality assurance mechanisms? Has the growth of private higher education in transition countries affected how quality assurance is conceptualised? Have quality assurance mechanisms been affected by the opening of branch campuses in transition countries?     

Benefits of computer‐assisted instruction for struggling readers in middle school

This study investigated the benefits of computer‐assisted instruction (CAI) for middle‐school students attending remedial reading classes. As a supplement to a strong language‐based reading curriculum, a CAI programme that works on strengthening phonics word identification skills was provided to students in the treatment group. These students were compared with control students taught by the same teacher but without CAI support. Students in the treatment group showed greater reading gains than control students. Group differences in gain scores were significant in the area of decoding skills. These results show that a skills‐based CAI programme can boost decoding for struggling readers in middle school.     

Governance and decision‐making in higher education: Comparative aspects

In this paper, I shall put forward a model which highlights key issues to be taken into account in an attempt to improve academic excellence by the means of human resources management and leadership. Practices at the University of Eastern Finland and elsewhere are used as examples. It is maintained that logical and coherent human resources policies and capable leadership can support and contribute to academic performance and excellence. A key issue is not to interfere too much with active academic work, but still give sufficient support and prerequisites for academic basic tasks.     

E‐learning in higher education: some key aspects and their relationship to approaches to study

While there has been systematic and on‐going research into e‐learning in universities for over two decades, there has been comparatively less evidence‐based research into how key aspects of e‐learning are internally constituted from a student perspective and how these aspects might be related to university students’ learning experiences. The purpose of this paper is to explore key aspects of e‐learning that might be related to university student approaches to study, so that a better understanding of the internal structure of these aspects is achieved. Student responses to surveys are analysed at the level of each item to identify which items made the most sense to over 200 third‐year economics students. Data was also analysed both at the variable level, to identify which items coalesce to determine the structure of e‐learning variables, and at the student level, to see if there were groups of students in the sample that shared similar experiences of e‐learning when it was used to support a predominately campus‐based learning experience. The results suggest several implications for improving particular aspects of the student experience of e‐learning when it is used to support a campus‐based experience.     

Part‐time higher education in Northern Ireland

Previous studies have shown that tutors usually enjoy teaching students and working for the University. During 1985 Reg Melton and colleagues in the Student Research Centre in the Institute of Educational Technology interviewed a number of tutors in depth to ascertain their reactions to the experience of reductions in their teaching. Job satisfaction and goodwill appeared to be under considerable strain.     

Learning to write in higher education: students’ perceptions of an intervention in developing understanding of assessment criteria

This paper reports on the second phase of a project designed to improve students’ understanding of assessment demands. In Stage 1, Level 1 students were involved in a range of activities culminating in peer marking. This peer assessment was, itself, marked by the tutors to encourage students to engage positively with the process. Stage 2 of the project investigated whether these various intervention activities had any long-term impact on sports studies students’ approach to writing assignments. Interviews were conducted with six students who participated in Stage 1. For comparison purposes, a matched group of students from another vocationally-related course were also interviewed. The findings suggest that the peer assessment did encourage students to pay attention to assessment information. However, the students placed greater stress on the role of informal support, particularly verbal clarification of written guidance and feedback. The article concludes with a discussion of the implications for practice.     

Evaluating computer‐based assessment in a risk‐based model

There are three purposes for evaluation: evaluation for action to aid the decision making process, evaluation for understanding to further enhance enlightenment and evaluation for control to ensure compliance to standards. This article argues that the primary function of evaluation in the ‘Catherine Wheel’ computer‐based assessment (CBA) cyclic model is evaluation for action and secondary functions are evaluation for understanding and control. By studying segment and cyclic dependencies, conducting a risk analysis and identifying key stakeholders it is possible to identify where action evaluation has the most effect, what is required to be evaluated, who will perform the action and how it will be controlled. Each segment and cycle in the CBA model must therefore be monitored and the observations made must have meaning for the elimination or reduction of pedagogic, operational, technical non web‐based, web‐based and financial risks. It is important to consider stakeholders because they have differing views on what is most important in the evaluation process and inevitably will be involved in the evaluation process itself. CBA system stakeholders include management, academic, support and administrative staff together with external verifiers and students. The information gained from these stakeholders provides quality enhancement and assurance and forms the basis for change.