Learning to construct text representations in end-to-end systems can be
difficult, as natural languages are highly compositional and task-specific
annotated datasets are often limited in size. Methods for directly supervising
language composition can allow us to guide the models based on existing
knowledge, regularizing them towards more robust and interpretable
representations. In this paper, we investigate how objectives at different
granularities can be used to learn better language representations and we
propose an architecture for jointly learning to label sentences and tokens. The
predictions at each level are combined together using an attention mechanism,
with token-level labels also acting as explicit supervision for composing
sentence-level representations. Our experiments show that by learning to
perform these tasks jointly on multiple levels, the model achieves substantial
improvements for both sentence classification and sequence labeling.

该研究探讨了不同颗粒度的目标如何用于学习更好的语言表示，并提出了一种架构来联合学习标记句子和标记令牌，最终通过注意力机制将每个级别的预测组合，其中令牌级别的标签也作为显式监督来合成句子级别的表示，实验结果表明，通过在多个级别上联合执行这些任务，模型在句子分类和序列标记方面都取得了实质性的改进。