Machine translation (MT) of user-generated content (UGC) poses unique challenges, including handling slang, emotion, and literary devices like irony and sarcasm. Evaluating the quality of these translations is challenging as current metrics do not focus on these ubiquitous features of UGC. To address this issue, we utilize an existing emotion-related dataset that includes emotion labels and human-annotated translation errors based on Multi-dimensional Quality Metrics. We extend it with sentence-level evaluation scores and word-level labels, leading to a dataset suitable for sentence- and word-level translation evaluation and emotion classification, in a multi-task setting. We propose a new architecture to perform these tasks concurrently, with a novel combined loss function, which integrates different loss heuristics, like the Nash and Aligned losses. Our evaluation compares existing fine-tuning and multi-task learning approaches, assessing generalization with ablative experiments over multiple datasets. Our approach achieves state-of-the-art performance and we present a comprehensive analysis for MT evaluation of UGC.

本研究解决了机器翻译用户生成内容时情感表达和语言特性的评估问题。我们提出了一种新的多任务学习框架，结合情感分类和翻译质量评估，采用创新的损失函数以实现任务的并行处理。研究发现，该方法在多个数据集上达到了最先进的性能，为UGC的机器翻译评估提供了新的视角。

情感载体用户生成内容的机器翻译评估多任务学习框架