Measuring Search Engine Quality

The effectiveness of twenty public search engines is evaluated using TREC-inspired methods and a set of 54 queries taken from real Web search logs. The World Wide Web is taken as the test collection and a combination of crawler and text retrieval system is evaluated. The engines are compared on a range of measures derivable from binary relevance judgments of the first seven live results returned. Statistical testing reveals a significant difference between engines and high intercorrelations between measures. Surprisingly, given the dynamic nature of the Web and the time elapsed, there is also a high correlation between results of this study and a previous study by Gordon and Pathak. For nearly all engines, there is a gradual decline in precision at increasing cutoff after some initial fluctuation. Performance of the engines as a group is found to be inferior to the group of participants in the TREC-8 Large Web task, although the best engines approach the median of those systems. Shortcomings of current Web search evaluation methodology are identified and recommendations are made for future improvements. In particular, the present study and its predecessors deal with queries which are assumed to derive from a need to find a selection of documents relevant to a topic. By contrast, real Web search reflects a range of other information need types which require different judging and different measures.     

Medical history for tomorrow—preserving the record of today

This paper addresses the need to recognize the importance, and the challenges, of preserving the hugely expanding record of medicine today for historians of the future. We are faced not only with an increase in traditional publishing formats, but also with the rapid growth in electronic communications. Two celebrated medical cases, one from the late seventeenth century and one from the late twentieth, are used to illustrate the differences and the difficulties. The need for active and effective professional collaboration is stressed; libraries should work together to address these issues.     

Accurate assessment of the brake torque on a rope-braked cycle ergometer

Sports Engineering - A Monark cycle ergometer is a device globally used in physiological studies to measure the work and energy levels of exercising humans. In this paper a rope-braked cycle...     

Developing the evidence base for UK film strategy: the research process at the UK Film Council

The UK Film Council established a Research and Statistics Unit in order to gather data relating to film to inform the development of UK Film Council strategy and to provide an information service to the industry, government, the arts and cultural sector and the wider research community. The Research and Statistics Unit draws data from both official and unofficial sources and commissions its own special-purpose studies to gather information relevant to the strategic objectives of the Council. Key tasks are the measurement of the size of the market for film and the various elements of the film value chain, the performance of films supported by the UK Film Council and the performance of UK films in general. Special-purpose research projects currently include a detailed survey of the film production workforce, a study of the economic impact of the UK screen industries and studies of the social impact of local cinemas and the experience of Black- and Minority-led film production companies. A range of industry and official partners are collaborating in these studies. The Research and Statistics Unit also provides statistical and policy analysis relating to the wider policy environment of UK film, including issues such as the future of film tax incentives. This analysis has been developed within the HM Treasury ‘Green Book’ framework with particular reference to understanding market failure in relation to film. Central to the market failure argument is the cultural value of film in both its qualitative and its quantitative aspects. UK Film Council research is placed in the context of the literature on hedonic pricing and contingent valuation. The industrial challenges of increasing cultural value are discussed. Finally, consideration is given to the potential of film to contribute directly to ‘public value’.