This paper explores the ability of Transformer models to capture subject-verb
and noun-adjective agreement dependencies in Galician. We conduct a series of
word prediction experiments in which we manipulate dependency length together
with the presence of an attractor noun that acts as a lure. First, we evaluate
the overall performance of the existing monolingual and multilingual models for
Galician. Secondly, to observe the effects of the training process, we compare
the different degrees of achievement of two monolingual BERT models at
different training points. We also release their checkpoints and propose an
alternative evaluation metric. Our results confirm previous findings by similar
works that use the agreement prediction task and provide interesting insights
into the number of training steps required by a Transformer model to solve
long-distance dependencies.

本文探讨 Transformer 模型在捕捉加利西亚语中的主谓和名词形容词一致性依赖方面的能力，并通过一系列词语预测实验来评估加利西亚语的现有单语和多语模型的性能，并对不同训练点上两个单语 BERT 模型的不同成就程度进行比较，得出了所需的训练步骤的有趣见解。

Galician BERT 模型在句法能力与训练时间交互界面的计算心理语言学评估

A computational psycholinguistic evaluation of the syntactic abilities of Galician BERT models at the interface of dependency resolution and training time