Measuring Search Engine Quality

The effectiveness of twenty public search engines is evaluated using TREC-inspired methods and a set of 54 queries taken from real Web search logs. The World Wide Web is taken as the test collection and a combination of crawler and text retrieval system is evaluated. The engines are compared on a range of measures derivable from binary relevance judgments of the first seven live results returned. Statistical testing reveals a significant difference between engines and high intercorrelations between measures. Surprisingly, given the dynamic nature of the Web and the time elapsed, there is also a high correlation between results of this study and a previous study by Gordon and Pathak. For nearly all engines, there is a gradual decline in precision at increasing cutoff after some initial fluctuation. Performance of the engines as a group is found to be inferior to the group of participants in the TREC-8 Large Web task, although the best engines approach the median of those systems. Shortcomings of current Web search evaluation methodology are identified and recommendations are made for future improvements. In particular, the present study and its predecessors deal with queries which are assumed to derive from a need to find a selection of documents relevant to a topic. By contrast, real Web search reflects a range of other information need types which require different judging and different measures.     

Interlibrary loan and commercial document supply: Finding the right fit

This study examined the cost and performance of six commercial document suppliers accessible via OCLC, and comapared their service to that of traditional interlibrary loan (ILL). Results indicated that, while commercial firms cannot be recommended as a wholesale replacement for ILL, they can add much-needed flexibility to a library's overall plan for document delivery.     

Developing a general practice library: a collaborative project between a GP and librarian

The authors report on a self‐completed questionnaire study from a North Yorkshire based general practice regarding the information needs of its clinicians. The work was carried out with a particular focus on the practice library, and the findings identified that a new approach to maintaining and developing the library was needed. The literature regarding the information needs of primary care clinicians and the role of practice libraries is considered, and compared to those of the clinicians at the practice. Discussion follows on how a collaborative project was set up between the practice and a librarian based at the local NHS Trust library in order to improve the existing practice library. Difficulties encountered and issues unique to the project are explored, including training implications presented by the implementation of electronic resources. Marketing activities implemented are discussed, how the library will operate in its new capacity, and how ongoing support and maintenance of the library will be carried out. It is concluded that although scepticism still exists regarding librarian involvement in practice libraries, collaboration between clinicians and librarians is an effective approach to the successful development and maintenance of a practice library, and recommendations are therefore made for similar collaborative work.     

Collaborative Research: Policy and the Management of Knowledge Creation in UK Universities

Collaboration in research activity is now the rule not the exception. It is encouraged by government, funding bodies and research councils. However, the concept of collaboration is difficult to define. It occurs at many different levels, driven by a complex research system-policy dynamic. Three different models of collaboration – inter-personal, team and corporate – are identified, each with their own rationale, structure, benefits and costs. The paper examines the institutional implications of these models. It argues that institutions and individual researchers conceptualise and operationalise research collaboration in different ways. Although vital to institutional mission, collaborative research is rarely mapped by senior managers with any precision. In general, institutional approaches to the management of collaborative research lag behind the policy rhetoric. The paper concludes with an overview of the key dilemmas for institutional strategists and policy makers posed by the shift towards more collaborative approaches to research.     

International co-production of distance-teaching courses

Co-production occurs when two or more institutions contribute funds and labour to the processes of making course components for distance teaching. The author summarises reasons for international co-production of multimedia distance-teaching courses and provides a checklist of questions that should be considered before entering upon such co-production.
The second half of the article consists of a composite case study based on the UK's Open University experience of international co-production. The paper ends with three questions: Is international co-production of this kind worth trying? Can the chances of success be made high enough by careful planning to justify risking capital in such ventures? Are other forms of collaboration more likely to be cost-effective?