An important component of achieving language understanding is mastering the
composition of sentence meaning, but an immediate challenge to solving this
problem is the opacity of sentence vector representations produced by current
neural sentence composition models. We present a method to address this
challenge, developing tasks that directly target compositional meaning
information in sentence vector representations with a high degree of precision
and control. To enable the creation of these controlled tasks, we introduce a
specialized sentence generation system that produces large, annotated sentence
sets meeting specified syntactic, semantic and lexical constraints. We describe
the details of the method and generation system, and then present results of
experiments applying our method to probe for compositional information in
embeddings from a number of existing sentence composition models. We find that
the method is able to extract useful information about the differing capacities
of these models, and we discuss the implications of our results with respect to
these systems' capturing of sentence information. We make available for public
use the datasets used for these experiments, as well as the generation system.

文章介绍了一种可以精确和控制地针对句向量中的组合含义信息的方法，利用一个专门的句子生成系统，创建遵循特定句法、语义和词汇约束条件的带注释的大型句子集。实验表明，这种方法能够从多种现有的句子表示模型中提取有用的信息，有助于理解这些系统对句子信息的捕获能力。