How to evaluate rankings of academic entities using test data

In the field of scientometrics, impact indicators and ranking algorithms are frequently evaluated using unlabelled test data comprising relevant entities (e.g., papers, authors, or institutions) that are considered important. The rationale is that the higher some algorithm ranks these entities, the better its performance. To compute a performance score for an algorithm, an evaluation measure is required to translate the rank distribution of the relevant entities into a single-value performance score. Until recently, it was simply assumed that taking the average rank (of the relevant entities) is an appropriate evaluation measure when comparing ranking algorithms or fine-tuning algorithm parameters.With this paper we propose a framework for evaluating the evaluation measures themselves. Using this framework the following questions can now be answered: (1) which evaluation measure should be chosen for an experiment, and (2) given an evaluation measure and corresponding performance scores for the algorithms under investigation, how significant are the observed performance differences?Using two publication databases and four test data sets we demonstrate the functionality of the framework and analyse the stability and discriminative power of the most common information retrieval evaluation measures. We find that there is no clear winner and that the performance of the evaluation measures is highly dependent on the underlying data. Our results show that the average rank is indeed an adequate and stable measure. However, we also show that relatively large performance differences are required to confidently determine if one ranking algorithm is significantly superior to another. Lastly, we list alternative measures that also yield stable results and highlight measures that should not be used in this context.     

Teaching Assistants managing behaviour – who knows how they do it? A review of literature

This article reflects on the specific challenges facing teaching assistants (TAs) when managing behaviour. It considers the variety of existing research into this area, and why the paucity of specific research is problematic. It discusses how difficulties in access to training, its availability, the levels of preparation for lessons, the lack of guidance on teacher and TA working relationships, all impact on the way TAs manage behaviour. The article also highlights some of the conflicts inherent within broader issues which impact, either positively or negatively, on TAs ability to manage behaviour, including TA role definition, deployment and the wider pedagogical aspects of their evolving role.     

Primary exclusions: are they rising?

Reducing the number of pupils excluded from school has formed part of the Government's strategy for school improvement in recent years. The study reported here was conducted within an inner-city LEA. In their article, Sue Pavey, research in special education, and John Visser, senior fellow lecturer in special education, both of whom work in the School of Education at the University of Birmingham, question the veracity of collected data on exclusions relating to the primary sector. Furthermore, they suggest that findings from their study indicate that exclusions are considered ineffective for the pupils concerned and are seen as a last resort by primary headteachers. More effective strategies for meeting the pupils' needs, argue Sue Pavey and John Visser, lie within enhanced, in-school support services and multi-agency approaches. The authors conclude that statistics on exclusion do not give a representative picture of real levels of exclusion in the LEA. This situation adversely affects strategic planning and the distribution of resources. In concluding their article, Sue Pavey and John Visser propose that resources should be spent on developing alternative strategies to exclusion and methods of identifying children at risk of disaffection.     

Assessment revisited: a review of research in Assessment and Evaluation in Higher Education

A review of articles published in Assessment and Evaluation in Higher Education, over the last eight years (2006–2013) on assessment in higher education, since the introduction of the Bologna process, is the subject of the paper. The first part discusses the key issue of assessment in higher education and the method used for selecting articles. The second part presents results according to the main emerging themes arising from data analysis: assessment methods, modes of assessment and assessment related to a given teaching and learning method. The paper concludes that the foci of the studies are aligned with assessment practices other than the written test, in accordance with a learner-oriented perspective. Although the implementation of the Bologna process has had different kinds of impact in different European countries, the review shows that the use and effects of a diversity of assessment methods in higher education have been investigated, particularly those pointing to the so-called alternative methods. Implications of the findings are discussed.     

Protection styles of prostitutes' clients: intentions,behavior, and considerations in relation to AIDS

The perceived costs and benefits of condom use within the context of commercial sex were examined in interviews with 87 clients of female Dutch prostitutes. 8 types of condom users were identified: convinced (46% of the study group), guilty conscience (11%), anti-ridden (13%), defaulting (6%), maximum selective (8%), minimum selective (3%), indifferent (8%), and recalcitrant (5%). Clients who consistently used condoms take responsibility for both the pleasure and safety of their sexual encounter; their attitude toward prostitution is positive and straightforward. Guilty conscience users are motivated to use condoms by feelings of guilt toward their wife and are ambivalent about their participation in commercial sex. Angst-ridden users are motivated to seek out prostitutes more by loneliness than the pursuit of sexual pleasure; they view their personal risk of acquired immunodeficiency syndrome (AIDS) as higher than that reported by consistent users and thus are motivated by fear to use condoms. Defaulting users perceive themselves as the victims of temptation and express a lack of control over the sexual encounter that places them at risk of unprotected sex. Maximum selective users are consistent in using condoms in sex with prostitutes, except when it involves a prostitute whom they see regularly and frequently. Minimum selective users seek frequent contact with prostitutes and base condom use on a subjective assessment of whether the woman is healthy or not. Indifferent users, often disabled and lonely men with no other source of sexual contact, base their condom use or nonuse solely on the wishes of the prostitute. Finally, recalcitrant users are characterized by a desire for power and protest condom use. In general, consistent condom use was related to a positive evaluation of prostitutes, positive attitudes toward the advantages of condom use, an internal locus of control with regard to health, and a higher level of education.     

‘Luring the academic soul’: promoting academic engagement in South African universities

There is widespread pressure that universities should become more responsive and accountable to multiple demands in their local, national and global contexts. Academics grapple to identify appropriate organisational responses to the pressures of state steering and incentive programmes. The empirical focus of the paper is a survey of academics' community engagement activities, mapping their interaction with external partners at five South African universities. The paper addresses a basic question: Who are the academics that engage, and what are their main academic influences? We argue that engagement is more likely to ‘lure the academic soul’ when it is driven by substantive growth in a disciplinary field and enhances reputations. The analysis seeks to identify how disciplinary, institutional and positional attributes are correlated with engaged activities. ‘Luring the academic soul’ through an understanding of the influences on academics' propensity to engage can facilitate more effectively targeted incentives and policies.     

‘Selective cosmopolitans’: tutors’ and students’ experience of offshore higher education in Dubai

As the offshore mobility of higher education has increased in recent times, the question of how it interacts with the recipient cultures has become ever more significant. Using ethnographic methods, this empirical study examined the adaptation of the UK teacher education model – the Postgraduate Certificate in Education – to the context of Dubai. The study asks ‘how do students and tutors experience the adaptation of British education in the context of Dubai?’ This paper will argue that tutors and students in offshore Dubai teacher education are ‘selective cosmopolitans’ who negotiate cross-cultural influences pragmatically and ambivalently. The study addresses a significant gap in the literature, as there is little written on the internationalisation of higher education in the context of Gulf Cooperation Council countries. There is also an inadequate appreciation of the role of local culture and religion in offshore education and tutors and students’ role as active agents in negotiating cross-cultural dynamics in the offshore educational setting.     

TORAH LITERACY IN THE ELEMENTARY SCHOOL CLASSROOM

As a lover of language and literature, as a serious yet secular Jew, and as a long-time educator in Jewish schools who strives to implement the best practices possible, the author found herself in an educational trap. Typically, even in liberal Jewish practice, Written Law, i.e. the Torah, and Oral Law, i.e. the rabbinic commentaries, are tightly bound together. In educational practice, this results in introducing commentaries on a primary text far earlier than done in Language Arts studies. According to the author, students are bound by a tradition and methodology that do not serve them well, as it recognizes the text but not the reader or the context. This article grows out of a desire to break out of traditional practices and teach Torah with the contemporary awareness of reading theory, without actually breaking with tradition.