Code search is an important task that has seen many developments in recent
years. However, previous attempts have mostly considered the problem of
searching for code by a text query. We argue that using a code snippet (and
possibly an associated traceback) as a query and looking for answers with
bugfixing instructions and code samples is a natural use case that is not
covered by existing approaches. Moreover, existing datasets use comments
extracted from code rather than full-text descriptions as text, making them
unsuitable for this use case. We present a new SearchBySnippet dataset
implementing the search-by-code use case based on StackOverflow data; it turns
out that in this setting, existing architectures fall short of the simplest
BM25 baseline even after fine-tuning. We present a new single encoder model
SnippeR that outperforms several strong baselines on the SearchBySnippet
dataset with a result of 0.451 Recall@10; we propose the SearchBySnippet
dataset and SnippeR as a new important benchmark for code search evaluation.

提出了一个新的 SearchBySnippet 数据集并建立了一个单一编码器模型 SnippeR，用于针对代码搜索的基准测试。在这一设置中，现有的架构即使是经过精细调整后也远远不及最简单的 BM25 基准。