A pipelined architecture for distributed text query evaluation

Two principal query-evaluation methodologies have been described for cluster-based implementation of distributed information retrieval systems: document partitioning and term partitioning. In a document-partitioned system, each of the processors hosts a subset of the documents in the collection, and executes every query against its local sub-collection. In a term-partitioned system, each of the processors hosts a subset of the inverted lists that make up the index of the collection, and serves them to a central machine as they are required for query evaluation. In this paper we introduce a pipelined query-evaluation methodology, based on a term-partitioned index, in which partially evaluated queries are passed amongst the set of processors that host the query terms. This arrangement retains the disk read benefits of term partitioning, but more effectively shares the computational load. We compare the three methodologies experimentally, and show that term distribution is inefficient and scales poorly. The new pipelined approach offers efficient memory utilization and efficient use of disk accesses, but suffers from problems with load balancing between nodes. Until these problems are resolved, document partitioning remains the preferred method. Alistair Moffat was supported by the Australian Research Council, the ARC Special Research Center for Perceptive and Intelligent Machines in Complex Environments, and the NICTA Victoria Laboratory. William Webber and Justin Zobel were supported by the Australian Research Council. Ricardo Baeza-Yates was supported by Grant P01-029-F from Millennium Initiative of Mideplan, Chile; and by the University of Melbourne as a visiting scholar at the time this project was undertaken.     

India: Demographics for Publishers

The essential demographic data and related information for India is provided specifically for the publishing industry. The focus has been placed on the economics, education and computer industry in relation to the complex cultural factors of different languages and religions. The role of India in relation to the other three BRIC countries or Brazil, China and Russia is explored.     

Investigation of the flow of scientific-technical information on the basis of the Russian technical literature on nanostructures and nanotechnologies

The objective of this investigation of the Russian technical literature on nanostructures and nanotechnologies is a study of the dynamics and the cumulative number of publications (CPN) concerning nanostructures and nanotechnologies, the dynamics an2 CPN for different types of publications, the publication productivity of Russian scientists and Russian cities concerning nanostructures and nanotechnologies, the scientific activity of Russian scientific-research and educational institutions concerning nanostructures and nanotechnologies, the investigation of Russian periodicals and continuing publications concerning nanostructures and nanotechnologies, and the simulation of the CPN and prediction of the CPN for nanostructures and nanotechnologies.     

Searching strategies for the Bulgarian language

This paper reports on the underlying IR problems encountered when indexing and searching with the Bulgarian language. For this language we propose a general light stemmer and demonstrate that it can be quite effective, producing significantly better MAP (around + 34%) than an approach not applying stemming. We implement the GL2 model derived from the Divergence from Randomness paradigm and find its retrieval effectiveness better than other probabilistic, vector-space and language models. The resulting MAP is found to be about 50% better than the classical tf idf approach. Moreover, increasing the query size enhances the MAP by around 10% (from T to TD). In order to compare the retrieval effectiveness of our suggested stopword list and the light stemmer developed for the Bulgarian language, we conduct a set of experiments on another stopword list and also a more complex and aggressive stemmer. Results tend to indicate that there is no statistically significant difference between these variants and our suggested approach. This paper evaluates other indexing strategies such as 4-gram indexing and indexing based on the automatic decompounding of compound words. Finally, we analyze certain queries to discover why we obtained poor results, when indexing Bulgarian documents using the suggested word-based approach.