Most work on natural language question answering today focuses on answer selection: given a candidate list of sentences, determine which contains the answer. Although important, answer selection is only one stage in a standard end-to-end question answering pipeline. This paper explores the effectiveness of convolutional neural networks (CNNs) for answer selection in an end-to-end context using the standard TrecQA dataset. We observe that a simple idf-weighted word overlap algorithm forms a very strong baseline, and that despite substantial efforts by the community in applying deep learning to tackle answer selection, the gains are modest at best on this dataset. Furthermore, it is unclear if a CNN is more effective than the baseline in an end-to-end context based on standard retrieval metrics. To further explore this finding, we conducted a manual user evaluation, which confirms that answers from the CNN are detectably better than those from idf-weighted word overlap. This result suggests that users are sensitive to relatively small differences in answer selection quality.

本研究使用标准的TrecQA数据集检验了卷积神经网络（CNNs）在端到端环境中用于答案选择的有效性，发现优化后的idf加权单词重叠算法是一个很强的基线，深度学习在这个数据集上的表现有限，而并不清楚在基于标准检索指标的端到端环境中CNN是否比基线更有效。而进行的人工用户评估证实了CNN产生的答案比idf加权单词重叠更好，这表明用户对答案选择质量的差异非常敏感。

探究卷积神经网络在端到端问答中的答案选择效果