Few-shot dense retrieval (DR) aims to effectively generalize to novel search
scenarios by learning a few samples. Despite its importance, there is little
study on specialized datasets and standardized evaluation protocols. As a
result, current methods often resort to random sampling from supervised
datasets to create "few-data" setups and employ inconsistent training
strategies during evaluations, which poses a challenge in accurately comparing
recent progress. In this paper, we propose a customized FewDR dataset and a
unified evaluation benchmark. Specifically, FewDR employs class-wise sampling
to establish a standardized "few-shot" setting with finely-defined classes,
reducing variability in multiple sampling rounds. Moreover, the dataset is
disjointed into base and novel classes, allowing DR models to be continuously
trained on ample data from base classes and a few samples in novel classes.
This benchmark eliminates the risk of novel class leakage, providing a reliable
estimation of the DR model's few-shot ability. Our extensive empirical results
reveal that current state-of-the-art DR models still face challenges in the
standard few-shot scene. Our code and data will be open-sourced at
this https URL

本文提出 FewDR 数据集和统一评估基准，并消除了现有方法中出现的一些问题。经实证结果表明，当前最先进的 DR 模型仍然在标准 few-shot 场景中面临挑战。

重新思考稠密检索的少样本能力

Rethinking Dense Retrieval's Few-Shot Ability

We introduce ASPiRe (Adaptive Skill Prior for RL), a new approach that
leverages prior experience to accelerate reinforcement learning. Unlike
existing methods that learn a single skill prior from a large and diverse
dataset, our framework learns a library of different distinction skill priors
(i.e., behavior priors) from a collection of specialized datasets, and learns
how to combine them to solve a new task. This formulation allows the algorithm
to acquire a set of specialized skill priors that are more reusable for
downstream tasks; however, it also brings up additional challenges of how to
effectively combine these unstructured sets of skill priors to form a new prior
for new tasks. Specifically, it requires the agent not only to identify which
skill prior(s) to use but also how to combine them (either sequentially or
concurrently) to form a new prior. To achieve this goal, ASPiRe includes
Adaptive Weight Module (AWM) that learns to infer an adaptive weight assignment
between different skill priors and uses them to guide policy learning for
downstream tasks via weighted Kullback-Leibler divergences. Our experiments
demonstrate that ASPiRe can significantly accelerate the learning of new
downstream tasks in the presence of multiple priors and show improvement on
competitive baselines.

ASPiRe 通过特定数据集来学习不同的技能先验（即行为先验），并学习如何将它们相互组合来解决新任务，期望通过自适应的权重分配来指导政策学习以加速学习的下游任务。