We present AutoNMT, a framework to streamline the research of seq-to-seq
models by automating the data pipeline (i.e., file management, data
preprocessing, and exploratory analysis), automating experimentation in a
toolkit-agnostic manner, which allows users to use either their own models or
existing seq-to-seq toolkits such as Fairseq or OpenNMT, and finally,
automating the report generation (plots and summaries). Furthermore, this
library comes with its own seq-to-seq toolkit so that users can easily
customize it for non-standard tasks.

AutoNMT 是一个自动化的框架，可以对 seq-to-seq 模型的研究进行流程化，包括数据管道、实验工具箱和报告生成等，并配有自定义 seq-to-seq 工具箱以方便用户进行非标准任务。

AutoNMT: 简化 Seq2Seq 模型研究的框架

AutoNMT: A Framework to Streamline the Research of Seq2Seq Models

Compositional generalization is a fundamental trait in humans, allowing us to
effortlessly combine known phrases to form novel sentences. Recent works have
claimed that standard seq-to-seq models severely lack the ability to
compositionally generalize. In this paper, we focus on one-shot primitive
generalization as introduced by the popular SCAN benchmark. We demonstrate that
modifying the training distribution in simple and intuitive ways enables
standard seq-to-seq models to achieve near-perfect generalization performance,
thereby showing that their compositional generalization abilities were
previously underestimated. We perform detailed empirical analysis of this
phenomenon. Our results indicate that the generalization performance of models
is highly sensitive to the characteristics of the training data which should be
carefully considered while designing such benchmarks in future.

本文针对基础的 seq-to-seq 模型缺乏组合概括能力的问题，重点讨论了使用一次性原语概括来增强这种能力的方法，发现通过修改训练数据的方式，能够使标准的 seq-to-seq 模型实现接近完美的概括能力表现，并且对该现象进行了详细的实证分析，指出了模型的概括性能对于训练数据的特征非常敏感，因而应该在设计这样的基准测试时仔细考虑训练数据的选择。

神经序列模型的组合泛化能力再探讨

Revisiting the Compositional Generalization Abilities of Neural Sequence Models

Popular solutions to Named Entity Recognition (NER) include conditional
random fields, sequence-to-sequence models, or utilizing the question-answering
framework. However, they are not suitable for nested and overlapping spans with
large ontologies and for predicting the position of the entities. To fill this
gap, we introduce a new model for NER task -- an RNN transducer (RNN-T). These
models are trained using paired input and output sequences without explicitly
specifying the alignment between them, similar to other seq-to-seq models.
RNN-T models learn the alignment using a loss function that sums over all
alignments. In NER tasks, however, the alignment between words and target
labels are available from the human annotations. We propose a fixed alignment
RNN-T model that utilizes the given alignment, while preserving the benefits of
RNN-Ts such as modeling output dependencies. As a more general case, we also
propose a constrained alignment model where users can specify a relaxation of
the given input alignment and the model will learn an alignment within the
given constraints. In other words, we propose a family of seq-to-seq models
which can leverage alignments between input and target sequences when
available. Through empirical experiments on a challenging real-world medical
NER task with multiple nested ontologies, we demonstrate that our fixed
alignment model outperforms the standard RNN-T model, improving F1-score from
0.70 to 0.74.

提出一种新的序列到序列模型 -- 一个 RNN transducer (RNN-T) 来解决名词实体识别 (NER) 的任务，利用对齐的人类标注信息作为特征来训练模型，并通过实验表明该方法可以提高 F1-score 的表现。