The ability to understand and generate languages sets human cognition apart
from other known life forms'. We study a way of combing two of the most
successful routes to meaning of language--statistical language models and
symbolic semantics formalisms--in the task of semantic parsing. Building on a
transition-based, Abstract Meaning Representation (AMR) parser, AmrEager, we
explore the utility of incorporating pretrained context-aware word
embeddings--such as BERT and RoBERTa--in the problem of AMR parsing,
contributing a new parser we dub as AmrBerger. Experiments find these rich
lexical features alone are not particularly helpful in improving the parser's
overall performance as measured by the SMATCH score when compared to the
non-contextual counterpart, while additional concept information empowers the
system to outperform the baselines. Through lesion study, we found the use of
contextual embeddings helps to make the system more robust against the removal
of explicit syntactical features. These findings expose the strength and
weakness of the contextual embeddings and the language models in the current
form, and motivate deeper understanding thereof.

该论文提出了一种名为 AmrBerger 的新的语义分析器，它探索了预训练上下文感知的单词嵌入在 AMR 分析中的可用性，并发现这些丰富的词汇特征与额外的概念信息的结合可以提高系统的性能。同时，通过分析，也展现了上下文嵌入的优点和局限，这为进一步深入理解提供了启示。

利用上下文嵌入的基于转移的抽象含义表示解析

Transition-based Abstract Meaning Representation Parsing with Contextual Embeddings

Homographs, words with different meanings but the same surface form, have
long caused difficulty for machine translation systems, as it is difficult to
select the correct translation based on the context. However, with the advent
of neural machine translation (NMT) systems, which can theoretically take into
account global sentential context, one may hypothesize that this problem has
been alleviated. In this paper, we first provide empirical evidence that
existing NMT systems in fact still have significant problems in properly
translating ambiguous words. We then proceed to describe methods, inspired by
the word sense disambiguation literature, that model the context of the input
word with context-aware word embeddings that help to differentiate the word
sense be- fore feeding it into the encoder. Experiments on three language pairs
demonstrate that such models improve the performance of NMT systems both in
terms of BLEU score and in the accuracy of translating homographs.

该研究利用词义消歧方法，在词输入编码器之前使用上下文感知的词嵌入来区分单词的多义性，解决神经机器翻译（NMT）系统在翻译歧义单词时存在的问题，实验表明该方法提高了 NMT 系统的翻译准确率。