Data augmentations (DA) are the cores to achieving robust
sequence-to-sequence learning on various natural language processing (NLP)
tasks. However, most of the DA approaches force the decoder to make predictions
conditioned on the perturbed input representation, underutilizing supervised
information provided by perturbed input. In this work, we propose a
framework-level robust sequence-to-sequence learning approach, named BLISS, via
self-supervised input representation, which has the great potential to
complement the data-level augmentation approaches. The key idea is to supervise
the sequence-to-sequence framework with both the \textit{supervised}
("input$\rightarrow$output") and \textit{self-supervised} ("perturbed
input$\rightarrow$input") information. We conduct comprehensive experiments to
validate the effectiveness of BLISS on various tasks, including machine
translation, grammatical error correction, and text summarization. The results
show that BLISS outperforms significantly the vanilla Transformer and
consistently works well across tasks than the other five contrastive baselines.
Extensive analyses reveal that BLISS learns robust representations and rich
linguistic knowledge, confirming our claim. Source code will be released upon
publication.

本文提出了一种名为 BLISS 的架构级强韧序列到序列学习方法，通过自监督输入表示具有潜在的补充数据级扩充方法的潜力，在各种任务中得到验证，包括机器翻译，语法纠错和文本摘要。