Learning to construct text representations in end-to-end systems can be
difficult, as natural languages are highly compositional and task-specific
annotated datasets are often limited in size. Methods for directly supervising
language composition can allow us to guide the models based on existing
knowledge, regularizing them towards more robust and interpretable
representations. In this paper, we investigate how objectives at different
granularities can be used to learn better language representations and we
propose an architecture for jointly learning to label sentences and tokens. The
predictions at each level are combined together using an attention mechanism,
with token-level labels also acting as explicit supervision for composing
sentence-level representations. Our experiments show that by learning to
perform these tasks jointly on multiple levels, the model achieves substantial
improvements for both sentence classification and sequence labeling.

该研究探讨了不同颗粒度的目标如何用于学习更好的语言表示，并提出了一种架构来联合学习标记句子和标记令牌，最终通过注意力机制将每个级别的预测组合，其中令牌级别的标签也作为显式监督来合成句子级别的表示，实验结果表明，通过在多个级别上联合执行这些任务，模型在句子分类和序列标记方面都取得了实质性的改进。

联合学习标记句子和单词

Jointly Learning to Label Sentences and Tokens

Current dialogue systems are not very engaging for users, especially when
trained end-to-end without relying on proactive reengaging scripted strategies.
Zhang et al. (2018) showed that the engagement level of end-to-end dialogue
models increases when conditioning them on text personas providing some
personalized back-story to the model. However, the dataset used in Zhang et al.
(2018) is synthetic and of limited size as it contains around 1k different
personas. In this paper we introduce a new dataset providing 5 million personas
and 700 million persona-based dialogues. Our experiments show that, at this
scale, training using personas still improves the performance of end-to-end
systems. In addition, we show that other tasks benefit from the wide coverage
of our dataset by fine-tuning our model on the data from Zhang et al. (2018)
and achieving state-of-the-art results.

本文介绍了一个包含 500 万个人物角色和 7 亿个依据人物角色的对话的新数据集，研究表明，使用此数据集训练对话系统，尤其是当基于文本人物角色时，可以提高其表现，并且在 Zhang 等人（2018）的数据上进行微调可以实现最新的成果。

训练数百万个个性化对话代理

Training Millions of Personalized Dialogue Agents

We highlight a practical yet rarely discussed problem in dialogue state
tracking (DST), namely handling unknown slot values. Previous approaches
generally assume predefined candidate lists and thus are not designed to output
unknown values, especially when the spoken language understanding (SLU) module
is absent as in many end-to-end (E2E) systems. We describe in this paper an E2E
architecture based on the pointer network (PtrNet) that can effectively extract
unknown slot values while still obtains state-of-the-art accuracy on the
standard DSTC2 benchmark. We also provide extensive empirical evidence to show
that tracking unknown values can be challenging and our approach can bring
significant improvement with the help of an effective feature dropout
technique.

本文介绍了一种基于指针网络 (PtrNet) 的 E2E 框架，可以有效地提取未知槽值，同时仍然能够在标准 DSTC2 基准测试中获得具有代表性的精度，我们还提供了广泛的实证证据表明，跟踪未知值可能具有挑战性，在有效的特征 dropout 技术的帮助下，我们的方法可以带来显著的改进。