We propose an end-to-end Multitask Learning Transformer framework, named
MulT, to simultaneously learn multiple high-level vision tasks, including depth
estimation, semantic segmentation, reshading, surface normal estimation, 2D
keypoint detection, and edge detection. Based on the Swin transformer model,
our framework encodes the input image into a shared representation and makes
predictions for each vision task using task-specific transformer-based decoder
heads. At the heart of our approach is a shared attention mechanism modeling
the dependencies across the tasks. We evaluate our model on several multitask
benchmarks, showing that our MulT framework outperforms both the state-of-the
art multitask convolutional neural network models and all the respective single
task transformer models. Our experiments further highlight the benefits of
sharing attention across all the tasks, and demonstrate that our MulT model is
robust and generalizes well to new domains. Our project website is at
this https URL

我们开发了一种称为 MulT 的端到端多任务学习 Transformer 框架，用于同时学习多个高级视觉任务，包括深度估计、语义分割、重着色、表面法线估计、2D 关键点检测和边缘检测。我们的框架基于 Swin transformer 模型，利用任务特定的基于 Transformer 的解码器头为每个视觉任务进行预测，并采用共享的注意机制模型来建模跨任务的依赖关系。我们在多个多任务基准测试中评估了我们的模型，显示出我们的 MulT 框架优于最先进的多任务卷积神经网络模型和所有各自的单任务 Transformer 模型。我们的实验进一步突出了共享所有任务的注意力的优点，并证明我们的 MulT 模型具有鲁棒性和很好的泛化性能。

MulT: 一种端到端的多任务学习变压器

MulT: An End-to-End Multitask Learning Transformer

Automatic post-editing (APE) systems aim to correct the systematic errors
made by machine translators. In this paper, we propose a neural APE system that
encodes the source (src) and machine translated (mt) sentences with two
separate encoders, but leverages a shared attention mechanism to better
understand how the two inputs contribute to the generation of the post-edited
(pe) sentences. Our empirical observations have showed that when the mt is
incorrect, the attention shifts weight toward tokens in the src sentence to
properly edit the incorrect translation. The model has been trained and
evaluated on the official data from the WMT16 and WMT17 APE IT domain
English-German shared tasks. Additionally, we have used the extra 500K
artificial data provided by the shared task. Our system has been able to
reproduce the accuracies of systems trained with the same data, while at the
same time providing better interpretability.

本文提出了一种神经自动后编辑系统，该系统使用两个独立编码器对源句子和机器翻译句子进行编码，并利用共享注意力机制更好地理解两个输入对于后编辑句子生成的贡献，其已在 WMT16 和 WMT17 APE IT domain 英德共享任务的官方数据以及附加的 500K 人工数据上进行了训练和评估，同时提供了更好的可解释性。