Context-aware translation can be achieved by processing a concatenation of
consecutive sentences with the standard translation approach. This paper
investigates the intuitive idea of adopting segment embeddings for this task to
help the Transformer discern the position of each sentence in the concatenation
sequence. We compare various segment embeddings and propose novel methods to
encode sentence position into token representations, showing that they do not
benefit the vanilla concatenation approach except in a specific setting.

本研究采用 segement embeddings 帮助 Transformer 识别翻译串联中每个句子的位置，并通过比较不同的 segment embeddings 和提出的方法，发现除了在特定情况下，它们并不会对原始串联方法产生益处。

利用链接方式增强上下文感知的神经机器翻译中的编码句子位置

Encoding Sentence Position in Context-Aware Neural Machine Translation with Concatenation

Contextualized word embeddings, i.e. vector representations for words in
context, are naturally seen as an extension of previous noncontextual
distributional semantic models. In this work, we focus on BERT, a deep neural
network that produces contextualized embeddings and has set the
state-of-the-art in several semantic tasks, and study the semantic coherence of
its embedding space. While showing a tendency towards coherence, BERT does not
fully live up to the natural expectations for a semantic vector space. In
particular, we find that the position of the sentence in which a word occurs,
while having no meaning correlates, leaves a noticeable trace on the word
embeddings and disturbs similarity relationships.

本文研究了语境化单词嵌入，并集中讨论了 BERT 这一深度神经网络，该网络产生了语境化的单词嵌入并在多个语义任务中创下了最好的记录，并研究了其嵌入空间的语义一致性。该文章表明，BERT 具有一定的语义一致性，但并未完全符合语义向量空间的自然期望，尤其是发现单词出现在的句子位置虽然不具含义，但在单词嵌入中留下了明显的痕迹，破坏了相似性关系。