Even though Transformers are extensively used for Natural Language Processing
tasks, especially for machine translation, they lack an explicit memory to
store key concepts of processed texts. This paper explores the properties of
the content of symbolic working memory added to the Transformer model decoder.
Such working memory enhances the quality of model predictions in machine
translation task and works as a neural-symbolic representation of information
that is important for the model to make correct translations. The study of
memory content revealed that translated text keywords are stored in the working
memory, pointing to the relevance of memory content to the processed text.
Also, the diversity of tokens and parts of speech stored in memory correlates
with the complexity of the corpora for machine translation task.

这篇论文研究了在 Transformer 模型解码器中添加的符号工作内存的属性，该工作内存增强了机器翻译任务中模型预测的质量，并作为信息的神经符号表示，对于模型进行正确翻译至关重要。通过研究记忆内容发现，翻译文本的关键词存储在工作内存中，指向记忆内容与处理文本的相关性。此外，存储在内存中的标记和词性的多样性与机器翻译任务的复杂性相关。