A probabilistic model of information retrieval: development and comparative experiments: Part 2

The paper combines a comprehensive account of the probabilistic model of retrieval with new systematic experiments on TREC Programme material. It presents the model from its foundations through its logical development to cover more aspects of retrieval data and a wider range of system functions. Each step in the argument is matched by comparative retrieval tests, to provide a single coherent account of a major line of research. The experiments demonstrate, for a large test collection, that the probabilistic model is effective and robust, and that it responds appropriately, with major improvements in performance, to key features of retrieval situations.Part 1 covers the foundations and the model development for document collection and relevance data, along with the test apparatus. Part 2 covers the further development and elaboration of the model, with extensive testing, and briefly considers other environment conditions and tasks, model training, concluding with comparisons with other approaches and an overall assessment.Data and results tables for both parts are given in Part 1. Key results are summarised in Part 2.     

KDD, data mining, and the challenge for normative privacy

The present study examines certain challenges that KDD (Knowledge Discovery in Databases) in general and data mining in particular pose for normative privacy and public policy. In an earlier work (see Tavani, 1999), I argued that certain applications of data-mining technology involving the manipulation of personal data raise special privacy concerns. Whereas the main purpose of the earlier essay was to show what those specific privacy concerns are and to describe how exactly those concerns have been introduced by the use of certain KDD and data-mining techniques, the present study questions whether the use of those techniques necessarily violates the privacy of individuals. This question is considered vis-à-vis a recent theory of privacy advanced by James Moor (1997). The implications of that privacy theory for a data-mining policy are also considered.