Multi-view clustering (MVC) can explore common semantics from unsupervised
views generated by different sources, and thus has been extensively used in
applications of practical computer vision. Due to the spatio-temporal
asynchronism, multi-view data often suffer from view missing and are unaligned
in real-world applications, which makes it difficult to learn consistent
representations. To address the above issues, this work proposes a deep MVC
framework where data recovery and alignment are fused in a hierarchically
consistent way to maximize the mutual information among different views and
ensure the consistency of their latent spaces. More specifically, we first
leverage dual prediction to fill in missing views while achieving the
instance-level alignment, and then take the contrastive reconstruction to
achieve the class-level alignment. To the best of our knowledge, this could be
the first successful attempt to handle the missing and unaligned data problem
separately with different learning paradigms. Extensive experiments on public
datasets demonstrate that our method significantly outperforms state-of-the-art
methods on multi-view clustering even in the cases of view missing and
unalignment.

本文提出一种深度多视角聚类 (Multi-view Clustering) 框架，将数据恢复和对齐融合在一个层次一致的方式中，通过最大化不同视角之间的互信息，并确保它们的潜在空间的一致性来解决多视角数据在现实世界应用中的视角缺失和不对齐问题。实验结果表明，我们的方法在多视角聚类中显著优于现有方法，即使在视角缺失和不对齐的情况下也是如此。