Multi-view representation learning aims to derive robust representations that
are both view-consistent and view-specific from diverse data sources. This
paper presents an in-depth analysis of existing approaches in this domain,
highlighting a commonly overlooked aspect: the redundancy between
view-consistent and view-specific representations. To this end, we propose an
innovative framework for multi-view representation learning, which incorporates
a technique we term 'distilled disentangling'. Our method introduces the
concept of masked cross-view prediction, enabling the extraction of compact,
high-quality view-consistent representations from various sources without
incurring extra computational overhead. Additionally, we develop a distilled
disentangling module that efficiently filters out consistency-related
information from multi-view representations, resulting in purer view-specific
representations. This approach significantly reduces redundancy between
view-consistent and view-specific representations, enhancing the overall
efficiency of the learning process. Our empirical evaluations reveal that
higher mask ratios substantially improve the quality of view-consistent
representations. Moreover, we find that reducing the dimensionality of
view-consistent representations relative to that of view-specific
representations further refines the quality of the combined representations.
Our code is accessible at: this https URL

多视角表示学习中的重叠与纯化问题得到了深入分析，提出了一种创新的多视角表示学习框架，结合了一种名为 “distilled disentangling” 的技术，通过引入遮蔽交叉视角预测的概念，实现了从多种数据源中提取紧凑、高质量的视角一致的表示，进一步减少了与视角特定表示的冗余，提升了学习过程的整体效率。