In the evolving landscape of Neural Machine Translation (NMT), the
pretrain-then-finetune paradigm has yielded impressive results. However, the
persistent challenge of Catastrophic Forgetting (CF) remains a hurdle. While
previous work has introduced Continual Learning (CL) methods to address CF,
these approaches grapple with the delicate balance between avoiding forgetting
and maintaining system extensibility. To address this, we propose a CL method,
named $\textbf{F-MALLOC}$ ($\textbf{F}$eed-forward $\textbf{M}$emory
$\textbf{ALLOC}ation)$. F-MALLOC is inspired by recent insights highlighting
that feed-forward layers emulate neural memories and encapsulate crucial
translation knowledge. It decomposes feed-forward layers into discrete memory
cells and allocates these memories to different tasks. By learning to allocate
and safeguard these memories, our method effectively alleviates CF while
ensuring robust extendability. Besides, we propose a comprehensive assessment
protocol for multi-stage CL of NMT systems. Experiments conducted following
this new protocol showcase the superior performance of F-MALLOC, evidenced by
higher BLEU scores and almost zero forgetting.

在神经机器翻译中，前预训练再微调模式取得了令人印象深刻的结果，但灾难性遗忘仍然是一个挑战。为了解决这个问题，我们提出了一种连续学习方法，名为 F-MALLOC (前馈记忆分配)，通过将前馈层分解为离散的记忆单元并将这些记忆分配给不同的任务，从而有效缓解了灾难性遗忘并确保了系统的可扩展性。此外，我们还提出了一个多阶段连续学习评估协议。根据这个新协议进行的实验显示了 F-MALLOC 的出色性能，体现在较高的 BLEU 得分和几乎没有遗忘。