Humans rely on strong inductive biases to learn from few examples and abstract useful information from sensory data. Instilling such biases in machine learning models has been shown to improve their performance on various benchmarks including few-shot learning, robustness, and alignment. However, finding effective training procedures to achieve that goal can be challenging as psychologically-rich training data such as human similarity judgments are expensive to scale, and Bayesian models of human inductive biases are often intractable for complex, realistic domains. Here, we address this challenge by introducing a Bayesian notion of generative similarity whereby two datapoints are considered similar if they are likely to have been sampled from the same distribution. This measure can be applied to complex generative processes, including probabilistic programs. We show that generative similarity can be used to define a contrastive learning objective even when its exact form is intractable, enabling learning of spatial embeddings that express specific inductive biases. We demonstrate the utility of our approach by showing how it can be used to capture human inductive biases for geometric shapes, and to better distinguish different abstract drawing styles that are parameterized by probabilistic programs.

我们引入了一种贝叶斯的生成相似性概念，通过该概念我们可以定义一个对比学习目标，即使其精确形式难以计算，从而使得能够学习到表达特定归纳偏差的空间嵌入。我们通过展示如何捕捉几何形状的人类归纳偏差以及更好地区分由概率程序参数化的不同抽象绘画风格，证明了我们方法的实用性。

利用对比学习与生成相似性来学习能捕捉人类归纳偏好的空间