The systematic evaluation and understanding of computer vision models under
varying conditions require large amounts of data with comprehensive and
customized labels, which real-world vision datasets rarely satisfy. While
current synthetic data generators offer a promising alternative, particularly
for embodied AI tasks, they often fall short for computer vision tasks due to
low asset and rendering quality, limited diversity, and unrealistic physical
properties. We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and
assets to generate fully customized synthetic data for systematic evaluation of
computer vision models, based on the newly developed embodied AI benchmark,
BEHAVIOR-1K. BVS supports a large number of adjustable parameters at the scene
level (e.g., lighting, object placement), the object level (e.g., joint
configuration, attributes such as "filled" and "folded"), and the camera level
(e.g., field of view, focal length). Researchers can arbitrarily vary these
parameters during data generation to perform controlled experiments. We
showcase three example application scenarios: systematically evaluating the
robustness of models across different continuous axes of domain shift,
evaluating scene understanding models on the same set of images, and training
and evaluating simulation-to-real transfer for a novel vision task: unary and
binary state prediction. Project website:
this https URL

介绍了 BEHAVIOR Vision Suite（BVS），它是一套用于生成全面定制的合成数据以系统评估计算机视觉模型的工具和资源，并且支持在场景、对象和摄像机级别进行可调参数的任意变化，以执行可控实验。

行为视觉套件：通过模拟生成可定制的数据集

BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation

The information retrieval community has recently witnessed a revolution due
to large pretrained transformer models. Another key ingredient for this
revolution was the MS MARCO dataset, whose scale and diversity has enabled
zero-shot transfer learning to various tasks. However, not all IR tasks and
domains can benefit from one single dataset equally. Extensive research in
various NLP tasks has shown that using domain-specific training data, as
opposed to a general-purpose one, improves the performance of neural models. In
this work, we harness the few-shot capabilities of large pretrained language
models as synthetic data generators for IR tasks. We show that models finetuned
solely on our unsupervised dataset outperform strong baselines such as BM25 as
well as recently proposed self-supervised dense retrieval methods. Furthermore,
retrievers finetuned on both supervised and our synthetic data achieve better
zero-shot transfer than models finetuned only on supervised data. Code, models,
and data are available at this https URL .

利用大型预训练语言模型作为综合数据生成器，对于信息检索任务的未监督数据的微调，可以使检索器在零样本学习和有监督学习的任务中获得更好的表现