Transfer and multi-task learning have traditionally focused on either a
single source-target pair or very few, similar tasks. Ideally, the linguistic
levels of morphology, syntax and semantics would benefit each other by being
trained in a single model. We introduce a joint many-task model together with a
strategy for successively growing its depth to solve increasingly complex
tasks. Higher layers include shortcut connections to lower-level task
predictions to reflect linguistic hierarchies. We use a simple regularization
term to allow for optimizing all model weights to improve one task's loss
without exhibiting catastrophic interference of the other tasks. Our single
end-to-end model obtains state-of-the-art or competitive results on five
different tasks from tagging, parsing, relatedness, and entailment tasks.

本研究提出一种多任务学习模型，可以同时训练多种语言学层次的任务，并通过一种逐步增加深度的策略来解决越来越复杂的任务。该模型使用一个简单的正则化项来实现在提高某一任务的损失函数时，不影响其他任务的学习效果。实验结果表明，这个端到端的模型在标签、句法分析、相关性与蕴含等五种任务上取得了最先进或有竞争力的结果。