Vision transformers (ViTs) are top performing models on many computer vision
benchmarks and can accurately predict human behavior on object recognition
tasks. However, researchers question the value of using ViTs as models of
biological learning because ViTs are thought to be more data hungry than
brains, with ViTs requiring more training data to reach similar levels of
performance. To test this assumption, we directly compared the learning
abilities of ViTs and animals, by performing parallel controlled rearing
experiments on ViTs and newborn chicks. We first raised chicks in impoverished
visual environments containing a single object, then simulated the training
data available in those environments by building virtual animal chambers in a
video game engine. We recorded the first-person images acquired by agents
moving through the virtual chambers and used those images to train self
supervised ViTs that leverage time as a teaching signal, akin to biological
visual systems. When ViTs were trained through the eyes of newborn chicks, the
ViTs solved the same view invariant object recognition tasks as the chicks.
Thus, ViTs were not more data hungry than newborn visual systems: both learned
view invariant object representations in impoverished visual environments. The
flexible and generic attention based learning mechanism in ViTs combined with
the embodied data streams available to newborn animals appears sufficient to
drive the development of animal-like object recognition.

视觉变换器 (ViTs) 是许多计算机视觉基准测试的最佳模型，可以准确预测对象识别任务上的人类行为。然而，研究人员对使用 ViTs 作为生物学习模型的价值产生了质疑，因为人们认为 ViTs 需要比大脑更多的训练数据才能达到类似的性能水平。为了测试这个假设，我们通过对 ViTs 和新生小鸡进行平行控制饲养实验，直接比较了它们的学习能力。当 ViTs 在新生小鸡的视角下进行训练时，ViTs 能够解决与小鸡相同的视角不变的对象识别任务。因此，ViTs 并不比新生视觉系统更加贪婪于数据：在贫乏的视觉环境中，两者都学习到了视角不变的对象表示。ViTs 的灵活和通用的基于注意力的学习机制与新生动物可用的具体数据流似乎足以推动动物样的对象识别的发展。