Recent language models use subwording mechanisms to handle
Out-of-Vocabulary(OOV) words seen during test time and, their generation
capacity is generally measured using perplexity, an intrinsic metric. It is
known that increasing the subword granularity results in a decrease of
perplexity value. However, the study of how subwording affects the
understanding capacity of language models has been very few and only limited to
a handful of languages. To reduce this gap we used 6 different tokenization
schemes to pretrain relatively small language models in Nepali and used the
representations learned to finetune on several downstream tasks. Although
byte-level BPE algorithm has been used in recent models like GPT, RoBERTa we
show that on average they are sub-optimal in comparison to algorithms such as
SentencePiece in finetuning performances for Nepali. Additionally, similar
recent studies have focused on the Bert-based language model. We, however,
pretrain and finetune sequential transformer-based language models.

使用 6 种不同的分词方案预先训练了相对较小的尼泊尔语言模型，并发现相比于字节级 BPE 算法，SentencePiece 等算法在尼泊尔的微调性能上表现更好。此外，我们还预先训练和微调了基于顺序转换器的语言模型。

用于尼泊尔语序列语言模型的标记化效果对微调性能的预测能力调查

Can Perplexity Predict Fine-Tuning Performance? An Investigation of  Tokenization Effects on Sequential Language Models for Nepali

Efforts on the research and development of OCR systems for Low-Resource
Languages are relatively new. Low-resource languages have little training data
available for training Machine Translation systems or other systems. Even
though a vast amount of text has been digitized and made available on the
internet the text is still in PDF and Image format, which are not instantly
accessible. This paper discusses text recognition for two scripts: Bengali and
Nepali; there are about 300 and 40 million Bengali and Nepali speakers
respectively. In this study, using encoder-decoder transformers, a model was
developed, and its efficacy was assessed using a collection of optical text
images, both handwritten and printed. The results signify that the suggested
technique corresponds with current approaches and achieves high precision in
recognizing text in Bengali and Nepali. This study can pave the way for the
advanced and accessible study of linguistics in South East Asia.

该研究使用编码器 - 解码器转换器针对 OCR 系统的研究和发展，以低资源语言（如孟加拉语和尼泊尔语）为例，通过对手写和打印的光学文本图像的集合进行评估，结果表明该技术与当前方法相符，并在孟加拉语和尼泊尔语的文本识别中取得了高精度，从而为东南亚地区的语言学研究打开了先机。

尼泊尔语和孟加拉语的光学文本识别：基于 Transformer 的方法

Optical Text Recognition in Nepali and Bengali: A Transformer-based  Approach

Video captioning in Nepali, a language written in the Devanagari script,
presents a unique challenge due to the lack of existing academic work in this
domain. This work develops a novel encoder-decoder paradigm for Nepali video
captioning to tackle this difficulty. LSTM and GRU sequence-to-sequence models
are used in the model to produce related textual descriptions based on features
retrieved from video frames using CNNs. Using Google Translate and manual
post-editing, a Nepali video captioning dataset is generated from the Microsoft
Research Video Description Corpus (MSVD) dataset created using Google
Translate, and manual post-editing work. The efficacy of the model for
Devanagari-scripted video captioning is demonstrated by BLEU, METOR, and ROUGE
measures, which are used to assess its performance.

本研究论文利用 LSTM 和 GRU 模型开发了一种新颖的编码器 - 解码器范例，用于尼泊尔视频字幕生成，利用 Devanagari 文字进行相关描述，并通过 BLEU、METOR 和 ROUGE 等指标评估其性能。