We investigate how pretrained language models (PLM) encode the grammatical
category of verbal aspect in Russian. Encoding of aspect in transformer LMs has
not been studied previously in any language. A particular challenge is posed by
"alternative contexts": where either the perfective or the imperfective aspect
is suitable grammatically and semantically. We perform probing using BERT and
RoBERTa on alternative and non-alternative contexts. First, we assess the
models' performance on aspect prediction, via behavioral probing. Next, we
examine the models' performance when their contextual representations are
substituted with counterfactual representations, via causal probing. These
counterfactuals alter the value of the "boundedness" feature--a semantic
feature, which characterizes the action in the context. Experiments show that
BERT and RoBERTa do encode aspect--mostly in their final layers. The
counterfactual interventions affect perfective and imperfective in opposite
ways, which is consistent with grammar: perfective is positively affected by
adding the meaning of boundedness, and vice versa. The practical implications
of our probing results are that fine-tuning only the last layers of BERT on
predicting aspect is faster and more effective than fine-tuning the whole
model. The model has high predictive uncertainty about aspect in alternative
contexts, which tend to lack explicit hints about the boundedness of the
described action.

我们调查了预训练语言模型（PLM）如何编码俄语动词相貌的语法类别。实验表明 BERT 和 RoBERTa 确实在其最后几层对相貌进行编码，并且对添加界定性含义对完美相貌和不完美相貌的影响相反，这与语法一致。我们的研究结果的实际应用是，仅对 BERT 的最后几层进行微调以预测相貌比对整个模型进行微调更快更有效。该模型对于具有界定性提示不明确的替代环境中的相貌预测存在较高的预测不确定性。

探究变形金刚式语言模型中的动词体类别

Probing the Category of Verbal Aspect in Transformer Language Models

The interpretation of the lexical aspect of verbs in English plays a crucial
role for recognizing textual entailment and learning discourse-level
inferences. We show that two elementary dimensions of aspectual class, states
vs. events, and telic vs. atelic events, can be modelled effectively with
distributional semantics. We find that a verb's local context is most
indicative of its aspectual class, and demonstrate that closed class words tend
to be stronger discriminating contexts than content words. Our approach
outperforms previous work on three datasets. Lastly, we contribute a dataset of
human--human conversations annotated with lexical aspect and present
experiments that show the correlation of telicity with genre and discourse
goals.

研究动词词汇体现出的语态对于文本蕴涵和学习语篇层推断至关重要。我们使用分布语义学有效地模拟了动态类别的两个基本维度，状态与事件以及目的性与非目的性事件。我们发现动词的本地上下文最能表明其语态，并证明封闭类词往往比内容词更具区分能力。我们的方法优于先前的三个数据集。最后，我们提供了一个由人 - 人对话注释的词汇视角数据集，并呈现实验结果，表明终止性与文本体裁和话语目标的相关性。