Image retrieval is a fundamental task in computer vision. Despite recent
advances in this field, many techniques have been evaluated on a limited number
of domains, with a small number of instance categories. Notably, most existing
works only consider domains like 3D landmarks, making it difficult to
generalize the conclusions made by these works to other domains, e.g., logo and
other 2D flat objects. To bridge this gap, we introduce a new dataset for
benchmarking visual search methods on flat images with diverse patterns. Our
flat object retrieval benchmark (FORB) supplements the commonly adopted 3D
object domain, and more importantly, it serves as a testbed for assessing the
image embedding quality on out-of-distribution domains. In this benchmark we
investigate the retrieval accuracy of representative methods in terms of
candidate ranks, as well as matching score margin, a viewpoint which is largely
ignored by many works. Our experiments not only highlight the challenges and
rich heterogeneity of FORB, but also reveal the hidden properties of different
retrieval strategies. The proposed benchmark is a growing project and we expect
to expand in both quantity and variety of objects. The dataset and supporting
codes are available at this https URL

图像检索是计算机视觉中的一个基本任务。尽管该领域取得了一些进展，但许多技术仅在有限数量的领域和少数实例类别上进行了评估。为了填补这一空白，我们提出了一个新的数据集，用于在具有多样的平面图案的平面图像上评估视觉搜索方法，以及出于领域外分布评估图像嵌入质量。