Experimental evidence indicates that simple models outperform complex deep networks on many unsupervised similarity tasks. We provide a simple yet rigorous explanation for this behaviour by introducing the concept of an optimal representation space, in which semantically close symbols are mapped to representations that are close under a similarity measure induced by the model's objective function. In addition, we present a straightforward procedure that, without any retraining or architectural modifications, allows deep recurrent models to perform equally well (and sometimes better) when compared to shallow models. To validate our analysis, we conduct a set of consistent empirical evaluations and introduce several new sentence embedding models in the process. Even though this work is presented within the context of natural language processing, the insights are readily applicable to other domains that rely on distributed representations for transfer tasks.

简单模型在无监督相似性任务中优于复杂深度网络的实验证据，本文提供了一种基于最佳表示空间的简单而严格的解释，介绍了一种直接的方法，使深度复发模型（DRM）在不重新训练或结构修改的情况下，达到与浅层模型同等或更优的表现。

解码解码器：为无监督相似性任务找到最佳表示空间