Observing that for certain NLP tasks, such as semantic role prediction or
thematic fit estimation, random embeddings perform as well as pretrained
embeddings, we explore what settings allow for this and examine where most of
the learning is encoded: the word embeddings, the semantic role embeddings, or
``the network''. We find nuanced answers, depending on the task and its
relation to the training objective. We examine these representation learning
aspects in multi-task learning, where role prediction and role-filling are
supervised tasks, while several thematic fit tasks are outside the models'
direct supervision. We observe a non-monotonous relation between some tasks'
quality score and the training data size. In order to better understand this
observation, we analyze these results using easier, per-verb versions of these
tasks.

本文研究使用随机嵌入在某些自然语言处理任务中（如语义角色预测或主题合适度估计）可以取得与预训练嵌入一样好的表现，探讨了哪些情况下可以实现这一点，并考察了学习主要编码的部分：单词嵌入、语义角色嵌入或网络结构，研究了多任务学习中的表征学习方面，观察到某些任务的质量得分与训练数据大小之间的非单调关系，并使用更简化的动词版本的任务分析这些结果，结论有助于更好地了解 NLP 中的随机嵌入、多任务学习和主题合适度等。

表示学习在组合语义中的学习和主题契合案例中的学习在哪里

Where's the Learning in Representation Learning for Compositional Semantics and the Case of Thematic Fit

Prior research has explored the ability of computational models to predict a
word semantic fit with a given predicate. While much work has been devoted to
modeling the typicality relation between verbs and arguments in isolation, in
this paper we take a broader perspective by assessing whether and to what
extent computational approaches have access to the information about the
typicality of entire events and situations described in language (Generalized
Event Knowledge). Given the recent success of Transformers Language Models
(TLMs), we decided to test them on a benchmark for the \textit{dynamic
estimation of thematic fit}. The evaluation of these models was performed in
comparison with SDM, a framework specifically designed to integrate events in
sentence meaning representations, and we conducted a detailed error analysis to
investigate which factors affect their behavior. Our results show that TLMs can
reach performances that are comparable to those achieved by SDM. However,
additional analysis consistently suggests that TLMs do not capture important
aspects of event knowledge, and their predictions often depend on surface
linguistic features, such as frequent words, collocations and syntactic
patterns, thereby showing sub-optimal generalization abilities.

本文研究了计算模型预测给定谓词的语义匹配能力。作者考虑了问提中的更广泛视角，评估了计算方法在评估语言描述的事件和情况（广义事件知识）的典型性信息方面的应用。结果表明 TLM 与 SDM 相当，但 TLM 的预测通常取决于表面语言特征，因此显示出次优的推广能力。