A probabilistic model for stemmer generation |
| |
Institution: | 1. Institute of Informatics, Hacettepe University, Turkey;2. Department of Computer Engineering, Hacettepe UniversityBeytepe Campus P.O. 06800, Ankara, Turkey;1. Department of Biochemistry and Molecular Biology, Capital Medical University, Beijing 100069, PR China;2. Biomedical Engineering Institute, Capital Medical University, Beijing 100069, PR China |
| |
Abstract: | In this paper we will present a language-independent probabilistic model which can automatically generate stemmers. Stemmers can improve the retrieval effectiveness of information retrieval systems, however the designing and the implementation of stemmers requires a laborious amount of effort due to the fact that documents and queries are often written or spoken in several different languages. The probabilistic model proposed in this paper aims at the development of stemmers used for several languages. The proposed model describes the mutual reinforcement relationship between stems and derivations and then provides a probabilistic interpretation. A series of experiments shows that the stemmers generated by the probabilistic model are as effective as the ones based on linguistic knowledge. |
| |
Keywords: | |
本文献已被 ScienceDirect 等数据库收录! |
|