We present GliLem -- a novel hybrid lemmatization system for Estonian that enhances the highly accurate rule-based morphological analyzer Vabamorf with an external disambiguation module based on GliNER -- an open vocabulary NER model that is able to match text spans with text labels in natural language. We leverage the flexibility of a pre-trained GliNER model to improve the lemmatization accuracy of Vabamorf by 10\% compared to its original disambiguation module and achieve an improvement over the token classification-based baseline. To measure the impact of improvements in lemmatization accuracy on the information retrieval downstream task, we first created an information retrieval dataset for Estonian by automatically translating the DBpedia-Entity dataset from English. We benchmark several token normalization approaches, including lemmatization, on the created dataset using the BM25 algorithm. We observe a substantial improvement in IR metrics when using lemmatization over simplistic stemming. The benefits of improving lemma disambiguation accuracy manifest in small but consistent improvement in the IR recall measure, especially in the setting of high k.

本研究提出了GliLem，一种用于爱沙尼亚语的新型混合词形还原系统，通过增强高准确性的基于规则的形态分析器Vabamorf，利用基于GliNER的外部分歧模块，解决了词形还原准确性不足的问题。实验结果表明，与原始分歧模块相比，GliLem在词形还原准确性上提高了10%，并在信息检索任务中展现出显著的改进。

GliLem：基于GliNER的爱沙尼亚语上下文化词形还原