To make sense of their surroundings, intelligent systems must transform
complex sensory inputs to structured codes that are reduced to task-relevant
information such as object category. Biological agents achieve this in a
largely autonomous manner, presumably via self-\allowbreak super-\allowbreak
vised learning. Whereas previous attempts to model the underlying mechanisms
were largely discriminative in nature, there is ample evidence that the brain
employs a generative model of the world. Here, we propose that eye movements,
in combination with the focused nature of primate vision, constitute a
generative, self-supervised task of predicting and revealing visual
information. We construct a proof-of-principle model starting from the
framework of masked image modeling (MIM), a common approach in deep
representation learning. To do so, we analyze how core components of MIM such
as masking technique and data augmentation influence the formation of
category-specific representations. This allows us not only to better understand
the principles behind MIM, but to then reassemble a MIM more in line with the
focused nature of biological perception. From a theoretical angle, we find that
MIM disentangles neurons in latent space, a property that has been suggested to
structure visual representations in primates, without explicit regulation.
Together with previous findings of invariance learning, this highlights an
interesting connection of MIM to latent regularization approaches for
self-supervised learning. The source code is available under
this https URL

智能系统通过自我监督学习中的生成模型将复杂的感知输入转化为结构化编码，以获取与任务相关的信息，如物体类别和视觉表征。

掩码图像建模作为一种跨眼动自监督学习的框架

Masked Image Modeling as a Framework for Self-Supervised Learning across  Eye Movements

Humans have a remarkable ability to disentangle complex sensory inputs (e.g.,
image, text) into simple factors of variation (e.g., shape, color) without much
supervision. This ability has inspired many works that attempt to solve the
following question: how do we invert the data generation process to extract
those factors with minimal or no supervision? Several works in the literature
on non-linear independent component analysis have established this negative
result; without some knowledge of the data generation process or appropriate
inductive biases, it is impossible to perform this inversion. In recent years,
a lot of progress has been made on disentanglement under structural
assumptions, e.g., when we have access to auxiliary information that makes the
factors of variation conditionally independent. However, existing work requires
a lot of auxiliary information, e.g., in supervised classification, it
prescribes that the number of label classes should be at least equal to the
total dimension of all factors of variation. In this work, we depart from these
assumptions and ask: a) How can we get disentanglement when the auxiliary
information does not provide conditional independence over the factors of
variation? b) Can we reduce the amount of auxiliary information required for
disentanglement? For a class of models where auxiliary information does not
ensure conditional independence, we show theoretically and experimentally that
disentanglement (to a large extent) is possible even when the auxiliary
information dimension is much less than the dimension of the true latent
representation.

本文针对非线性独立成分分析的负面结果，试图研究在没有条件独立性的辅助信息下如何实现解缠以及如何减少需要的辅助信息量。在一类模型中，我们证明理论上和实验上都可以实现解缠，即使辅助信息的维度远小于真实潜在表示的维度。