Application of the Levenshtein Distance Metric for the Construction of Longitudinal Data Files

The analysis of longitudinal data in education is becoming more prevalent given the nature of testing systems constructed for No Child Left Behind Act (NCLB). However, constructing the longitudinal data files remains a significant challenge. Students move into new schools, but in many cases the unique identifiers (ID) that should remain constant for each student change. As a result, different students frequently share the same ID, and merging records for an ID that is erroneously assigned to different students clearly becomes problematic. In small data sets, quality assurance of the merge can proceed through human reviews of the data to ensure all merged records are properly joined. However, in data sets with hundreds of thousands of cases, quality assurance via human review is impossible. While the record linkage literature has many applications in other disciplines, the educational measurement literature lacks details of formal protocols that can be used for quality assurance procedures for longitudinal data files. This article presents an empirical quality assurance procedure that may be used to verify the integrity of the merges performed for longitudinal analysis. We also discuss possible extensions that would permit merges to occur even when unique identifiers are not available.     

The impact of technology and a no remediation policy on non-traditional students

A Hunter College librarian, with public library experience, offered a series of drop-in workshops Saturday afternoons open to all students and questions regarding Web or Windows use in the academic year, 2000-2001. The intention of the workshops was to address the continuous needs of students lacking technological and informational literacy. The success of these workshops led to this investigation of the implications of 21st century technology on non-traditional students from the perspective of bibliographic instruction as well as reference desk service. The perspective is that of the urban educational environment of City University of New York and Hunter College. The histories of open admissions at CUNY and the current efforts to abolish remediation are examined.     

A pipelined architecture for distributed text query evaluation

Two principal query-evaluation methodologies have been described for cluster-based implementation of distributed information retrieval systems: document partitioning and term partitioning. In a document-partitioned system, each of the processors hosts a subset of the documents in the collection, and executes every query against its local sub-collection. In a term-partitioned system, each of the processors hosts a subset of the inverted lists that make up the index of the collection, and serves them to a central machine as they are required for query evaluation. In this paper we introduce a pipelined query-evaluation methodology, based on a term-partitioned index, in which partially evaluated queries are passed amongst the set of processors that host the query terms. This arrangement retains the disk read benefits of term partitioning, but more effectively shares the computational load. We compare the three methodologies experimentally, and show that term distribution is inefficient and scales poorly. The new pipelined approach offers efficient memory utilization and efficient use of disk accesses, but suffers from problems with load balancing between nodes. Until these problems are resolved, document partitioning remains the preferred method. Alistair Moffat was supported by the Australian Research Council, the ARC Special Research Center for Perceptive and Intelligent Machines in Complex Environments, and the NICTA Victoria Laboratory. William Webber and Justin Zobel were supported by the Australian Research Council. Ricardo Baeza-Yates was supported by Grant P01-029-F from Millennium Initiative of Mideplan, Chile; and by the University of Melbourne as a visiting scholar at the time this project was undertaken.     

India: Demographics for Publishers

The essential demographic data and related information for India is provided specifically for the publishing industry. The focus has been placed on the economics, education and computer industry in relation to the complex cultural factors of different languages and religions. The role of India in relation to the other three BRIC countries or Brazil, China and Russia is explored.     

Searching strategies for the Bulgarian language

This paper reports on the underlying IR problems encountered when indexing and searching with the Bulgarian language. For this language we propose a general light stemmer and demonstrate that it can be quite effective, producing significantly better MAP (around + 34%) than an approach not applying stemming. We implement the GL2 model derived from the Divergence from Randomness paradigm and find its retrieval effectiveness better than other probabilistic, vector-space and language models. The resulting MAP is found to be about 50% better than the classical tf idf approach. Moreover, increasing the query size enhances the MAP by around 10% (from T to TD). In order to compare the retrieval effectiveness of our suggested stopword list and the light stemmer developed for the Bulgarian language, we conduct a set of experiments on another stopword list and also a more complex and aggressive stemmer. Results tend to indicate that there is no statistically significant difference between these variants and our suggested approach. This paper evaluates other indexing strategies such as 4-gram indexing and indexing based on the automatic decompounding of compound words. Finally, we analyze certain queries to discover why we obtained poor results, when indexing Bulgarian documents using the suggested word-based approach.