An intelligent spelling error corrector

This paper describes an intelligent spelling error correction system for use in a word processing environment. The system employs a dictionary of 93,769 words and provided the intended word is in the dictionary it identifies 80 to 90% of spelling and typing errors.     

The generation and use of text fragments for data compression

An analysis of 31,369 bibliographic titles was carried out to obtain statistics on frequently occurring character groups to increase the effective character set with the aim of estimating possible compression factors for text. It was found that a common set could be used to obtain compression ranging between 30 and 53% over a wide variety of original text.     

The rules of spelling errors

This paper demonstrates that the vast majority of spelling errors follow specific rules which are based on phonological and sequential considerations. It introduces and describes three categories of spelling errors (consonantal, vowel and sequential) and presents the results of the analysis of 1377 spelling error forms.     

Developing an automated writing placement system for ESL learners

ABSTRACT

There are quite a few challenges in the development of an automated writing placement model for non-native English learners, among them the fact that exams that encompass the full range of language proficiency exhibited at different stages of learning are hard to design. However, acquisition of appropriate training data that are relevant to the task at hand is essential in the development of the model. Using the Cambridge Learner Corpus writing scores, which have been subsequently benchmarked to Common European Framework of Reference for Languages (CEFR) levels, we conceptualize the task as a supervised machine learning problem, and primarily focus on developing a generic writing model. Such an approach facilitates the modeling of truly consistent, internal marking criteria regardless of the prompt delivered, which has the additional advantage of requiring smaller dataset sizes and not necessarily requiring re-training or tuning for new tasks. The system is developed to predict someone’s proficiency level on the CEFR scale, which allows learners to point to a specific standard of achievement. We furthermore integrate our model into Cambridge English Write & Improve^TM—a freely available, cloud-based tool that automatically provides diagnostic feedback to non-native English language learners at different levels of granularity—and examine its use.     

An efficient file structure for specialized dictionaries and other “lumpy” data

There are many cases where it is necessary to store sets of data that are variable in length, and to search these in order to satisfy requests for subsets with a common characteristic. This article presents a file structure that holds an integrated English dictionary used to locate clusters of words for presentation to an intelligent spelling error correction system. Although the emphasis has been on misspelling, the structure presented is capable of handling any other types of lumpy data provided the characteristics used in search requests can be translated into a set of integer numbers.