Neural information retrieval requires costly annotated data for each target
domain to be competitive. Synthetic annotation by query generation using Large
Language Models or rule-based string manipulation has been proposed as an
alternative, but their relative merits have not been analysed. In this paper,
we compare both methods head-to-head using the same neural IR architecture. We
focus on the BEIR benchmark, which includes test datasets from several domains
with no training data, and explore two scenarios: zero-shot, where the
supervised system is trained in a large out-of-domain dataset (MS-MARCO); and
unsupervised domain adaptation, where, in addition to MS-MARCO, the system is
fine-tuned in synthetic data from the target domain. Our results indicate that
Large Language Models outperform rule-based methods in all scenarios by a large
margin, and, more importantly, that unsupervised domain adaptation is effective
compared to applying a supervised IR system in a zero-shot fashion. In addition
we explore several sizes of open Large Language Models to generate synthetic
data and find that a medium-sized model suffices. Code and models are publicly
available for reproducibility.

比较了使用大型语言模型生成查询和基于规则的方法进行合成注释来进行神经信息检索，结果表明大型语言模型在所有情景中的性能优于基于规则的方法，而无监督领域适应相对于零样本方式应用监督信息检索系统是有效的。