Retrieval-based code question answering seeks to match user queries in
natural language to relevant code snippets. Previous approaches typically rely
on pretraining models using crafted bi-modal and uni-modal datasets to align
text and code representations. In this paper, we introduce ProCQA, a
large-scale programming question answering dataset extracted from the
StackOverflow community, offering naturally structured mixed-modal QA pairs. To
validate its effectiveness, we propose a modality-agnostic contrastive
pre-training approach to improve the alignment of text and code representations
of current code language models. Compared to previous models that primarily
employ bimodal and unimodal pairs extracted from CodeSearchNet for
pre-training, our model exhibits significant performance improvements across a
wide range of code retrieval benchmarks.

基于检索的代码问答模型通过预训练模型使用构建的双模和单模数据集来匹配自然语言查询与相关代码片段。本文介绍了 ProCQA，一个从 StackOverflow 社区提取的大规模编程问答数据集，提供自然结构的混合模态问答对。为了验证其有效性，我们提出了一种模态不可知的对比预训练方法，以改善当前代码语言模型的文本和代码表示之间的对齐。与先前主要使用从 CodeSearchNet 中提取的双模和单模对进行预训练的模型相比，我们的模型在广泛的代码检索基准上表现出显著的性能改进。