As we all know, multi-view data is more expressive than single-view data and
multi-label annotation enjoys richer supervision information than single-label,
which makes multi-view multi-label learning widely applicable for various
pattern recognition tasks. In this complex representation learning problem,
three main challenges can be characterized as follows: i) How to learn
consistent representations of samples across all views? ii) How to exploit and
utilize category correlations of multi-label to guide inference? iii) How to
avoid the negative impact resulting from the incompleteness of views or labels?
To cope with these problems, we propose a general multi-view multi-label
learning framework named label-guided masked view- and category-aware
transformers in this paper. First, we design two transformer-style based
modules for cross-view features aggregation and multi-label classification,
respectively. The former aggregates information from different views in the
process of extracting view-specific features, and the latter learns subcategory
embedding to improve classification performance. Second, considering the
imbalance of expressive power among views, an adaptively weighted view fusion
module is proposed to obtain view-consistent embedding features. Third, we
impose a label manifold constraint in sample-level representation learning to
maximize the utilization of supervised information. Last but not least, all the
modules are designed under the premise of incomplete views and labels, which
makes our method adaptable to arbitrary multi-view and multi-label data.
Extensive experiments on five datasets confirm that our method has clear
advantages over other state-of-the-art methods.

本文提出了一种基于 transformer 的多视图多标签学习框架，以解决多视图数据和多标签注释于模式识别中的应用。该框架通过设计两个基于 transformer 的模块对特征进行聚合和分类，并提出一种自适应视图融合模块解决视图表达的不平衡性，同时将标签约束到样本级表示学习中。五个数据集上的实验证实了该方法的有效性。