We present mahaNLP, an open-source natural language processing (NLP) library
specifically built for the Marathi language. It aims to enhance the support for
the low-resource Indian language Marathi in the field of NLP. It is an
easy-to-use, extensible, and modular toolkit for Marathi text analysis built on
state-of-the-art MahaBERT-based transformer models. Our work holds significant
importance as other existing Indic NLP libraries provide basic Marathi
processing support and rely on older models with restricted performance. Our
toolkit stands out by offering a comprehensive array of NLP tasks, encompassing
both fundamental preprocessing tasks and advanced NLP tasks like sentiment
analysis, NER, hate speech detection, and sentence completion. This paper
focuses on an overview of the mahaNLP framework, its features, and its usage.
This work is a part of the L3Cube MahaNLP initiative, more information about it
can be found at this https URL .

mahaNLP 是一个针对马拉地语开发的开源自然语言处理（NLP）库，旨在增强 NLP 领域中对资源匮乏的印度语言马拉地的支持，它是一个易于使用、可扩展和模块化的工具包，基于最先进的 MahaBERT-based transformer 模型进行马拉地文本分析。

mahaNLP：一个马拉地文自然语言处理库

mahaNLP: A Marathi Natural Language Processing Library

Indian language machine translation performance is hampered due to the lack
of large scale multi-lingual sentence aligned corpora and robust benchmarks.
Through this paper, we provide and analyse an automated framework to obtain
such a corpus for Indian language neural machine translation (NMT) systems. Our
pipeline consists of a baseline NMT system, a retrieval module, and an
alignment module that is used to work with publicly available websites such as
press releases by the government. The main contribution towards this effort is
to obtain an incremental method that uses the above pipeline to iteratively
improve the size of the corpus as well as improve each of the components of our
system. Through our work, we also evaluate the design choices such as the
choice of pivoting language and the effect of iterative incremental increase in
corpus size. Our work in addition to providing an automated framework also
results in generating a relatively larger corpus as compared to existing
corpora that are available for Indian languages. This corpus helps us obtain
substantially improved results on the publicly available WAT evaluation
benchmark and other standard evaluation benchmarks.

本文提供并分析了一个用于印度语神经机器翻译系统的大规模多语言句子对齐语料库和强大基准的自动化框架，其中包括基线 NMT 系统，检索模块和用于公共网站的对齐模块，通过迭代增加语料库来改进系统。我们的工作还评估了设计选择，例如枢轴语言的选择和迭代式增量语料库增加的影响。与现有的印度语语料库相比，本工作不仅提供了自动化框架，还产生了一个相对较大的语料库。这个语料库使我们能够在公开的 WAT 评估基准和其他标准评估基准上获得明显改善的结果。