Informal language is a style of spoken or written language frequently used in
casual conversations, social media, weblogs, emails and text messages. In
informal writing, the language faces some lexical and/or syntactic changes
varying among different languages. Persian is one of the languages with many
differences between its formal and informal styles of writing, thus developing
informal language processing tools for this language seems necessary. Such a
converter needs a large aligned parallel corpus of colloquial-formal sentences
which can be useful for linguists to extract a regulated grammar and
orthography for colloquial Persian as is done for the formal language. In this
paper we explain our methodology in building a parallel corpus of 50,000
sentence pairs with alignments in the word/phrase level. The sentences were
attempted to cover almost all kinds of lexical and syntactic changes between
informal and formal Persian, therefore both methods of exploring and collecting
from the different resources of informal scripts and following the phonological
and morphological patterns of changes were applied to find as much instances as
possible. The resulting corpus has about 530,000 alignments and a dictionary
containing 49,397 word and phrase pairs.

构建一个包含 50,000 个句对的并行语料库，用来提取波斯语非正式语言的规范文法和字词拼写变化。

制定一部波斯语非正式 - 正式语料库

Developing an Informal-Formal Persian Corpus

Morphological and syntactic changes in word usage (as captured, e.g., by
grammatical profiles) have been shown to be good predictors of a word's meaning
change. In this work, we explore whether large pre-trained contextualised
language models, a common tool for lexical semantic change detection, are
sensitive to such morphosyntactic changes. To this end, we first compare the
performance of grammatical profiles against that of a multilingual neural
language model (XLM-R) on 10 datasets, covering 7 languages, and then combine
the two approaches in ensembles to assess their complementarity. Our results
show that ensembling grammatical profiles with XLM-R improves semantic change
detection performance for most datasets and languages. This indicates that
language models do not fully cover the fine-grained morphological and syntactic
signals that are explicitly represented in grammatical profiles.
An interesting exception are the test sets where the time spans under
analysis are much longer than the time gap between them (for example,
century-long spans with a one-year gap between them). Morphosyntactic change is
slow so grammatical profiles do not detect in such cases. In contrast, language
models, thanks to their access to lexical information, are able to detect fast
topical changes.

在这项研究中，我们探究了大型预先训练的上下文感知语言模型和语法特征分析在语义变化检测方面的异同。结果表明，将语法特征分析和语言模型相结合能够提高语义变化检测的性能，尤其是对于大部分数据集和语言而言。但对于一些跨度时间相差较长的测试集，语法特征分析检测效果较差，而语言模型则表现良好。