Despite the remarkable success deep models have achieved in Textual Matching
(TM) tasks, it still remains unclear whether they truly understand language or
measure the semantic similarity of texts by exploiting statistical bias in
datasets. In this work, we provide a new perspective to study this issue -- via
the length divergence bias. We find the length divergence heuristic widely
exists in prevalent TM datasets, providing direct cues for prediction. To
determine whether TM models have adopted such heuristic, we introduce an
adversarial evaluation scheme which invalidates the heuristic. In this
adversarial setting, all TM models perform worse, indicating they have indeed
adopted this heuristic. Through a well-designed probing experiment, we
empirically validate that the bias of TM models can be attributed in part to
extracting the text length information during training. To alleviate the length
divergence bias, we propose an adversarial training method. The results
demonstrate we successfully improve the robustness and generalization ability
of models at the same time.

本文研究文本匹配任务中深度模型是否真正理解语言或仅仅是利用数据集的统计偏差来衡量文本的语义相似性，发现普遍存在长度偏差启发式规则，并提出对抗训练方法以减缓长度偏差的影响，得到改进后的模型具有更强鲁棒性和泛化能力。