The outstanding performance of transformer-based language models on a great
variety of NLP and NLU tasks has stimulated interest in exploring their inner
workings. Recent research has focused primarily on higher-level and complex
linguistic phenomena such as syntax, semantics, world knowledge, and common
sense. The majority of the studies are anglocentric, and little remains known
regarding other languages, precisely their morphosyntactic properties. To this
end, our work presents Morph Call, a suite of 46 probing tasks for four
Indo-European languages of different morphology: English, French, German and
Russian. We propose a new type of probing task based on the detection of guided
sentence perturbations. We use a combination of neuron-, layer- and
representation-level introspection techniques to analyze the morphosyntactic
content of four multilingual transformers, including their less explored
distilled versions. Besides, we examine how fine-tuning for POS-tagging affects
the model knowledge. The results show that fine-tuning can improve and decrease
the probing performance and change how morphosyntactic knowledge is distributed
across the model. The code and data are publicly available, and we hope to fill
the gaps in the less studied aspect of transformers.

本研究提出了 Morph Call，它是针对四种不同形态的印欧语言（英语，法语，德语和俄语）的 46 个探测任务套件，使用诱导句子扰动的检测方法，在神经元，层和表示水平上分析了四个多语言 transformers 的形态句法内容，其中包括其不太被探究的蒸馏版本，并研究了精调 POS 标记对模型知识的影响。研究结果表明，精调可以改善和降低探测性能，并改变形态句法知识在模型中的分布。代码和数据公开，希望填补 transformers 的研究空白。