In existing visual representation learning tasks, deep convolutional neural
networks (CNNs) are often trained on images annotated with single tags, such as
ImageNet. However, a single tag cannot describe all important contents of one
image, and some useful visual information may be wasted during training. In
this work, we propose to train CNNs from images annotated with multiple tags,
to enhance the quality of visual representation of the trained CNN model. To
this end, we build a large-scale multi-label image database with 18M images and
11K categories, dubbed Tencent ML-Images. We efficiently train the ResNet-101
model with multi-label outputs on Tencent ML-Images, taking 90 hours for 60
epochs, based on a large-scale distributed deep learning framework,i.e.,TFplus.
The good quality of the visual representation of the Tencent ML-Images
checkpoint is verified through three transfer learning tasks, including
single-label image classification on ImageNet and Caltech-256, object detection
on PASCAL VOC 2007, and semantic segmentation on PASCAL VOC 2012. The Tencent
ML-Images database, the checkpoints of ResNet-101, and all the training
codehave been released at this https URL It is
expected to promote other vision tasks in the research and industry community.

提出使用多标记图像训练深度学习 ResNet-101 模型以提高图像的可视化质量，通过在大型分布式深度学习框架下进行多标签输出的高效训练，建立了一个大规模的多标签图像数据库，并通过传输学习任务验证了其可视化表示的良好质量。

Tencent ML-Images：大规模多标签图像数据库用于视觉表征学习

Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual  Representation Learning

Convolutional neural networks (CNNs) have shown great performance as general
feature representations for object recognition applications. However, for
multi-label images that contain multiple objects from different categories,
scales and locations, global CNN features are not optimal. In this paper, we
incorporate local information to enhance the feature discriminative power. In
particular, we first extract object proposals from each image. With each image
treated as a bag and object proposals extracted from it treated as instances,
we transform the multi-label recognition problem into a multi-class
multi-instance learning problem. Then, in addition to extracting the typical
CNN feature representation from each proposal, we propose to make use of
ground-truth bounding box annotations (strong labels) to add another level of
local information by using nearest-neighbor relationships of local regions to
form a multi-view pipeline. The proposed multi-view multi-instance framework
utilizes both weak and strong labels effectively, and more importantly it has
the generalization ability to even boost the performance of unseen categories
by partial strong labels from other categories. Our framework is extensively
compared with state-of-the-art hand-crafted feature based methods and CNN based
methods on two multi-label benchmark datasets. The experimental results
validate the discriminative power and the generalization ability of the
proposed framework. With strong labels, our framework is able to achieve
state-of-the-art results in both datasets.

本文提出了一种基于卷积神经网络的多视图多实例框架，用于提高物体检测中的特征表达和泛化能力，通过利用局部信息、弱标签和强标签有效地识别包含多个物体的多标签图像，实验结果表明该框架具有优异的鉴别力和泛化能力。

利用边界框注释进行多标签物体识别

Exploit Bounding Box Annotations for Multi-label Object Recognition

With the rapid growth of web images, hashing has received increasing
interests in large scale image retrieval. Research efforts have been devoted to
learning compact binary codes that preserve semantic similarity based on
labels. However, most of these hashing methods are designed to handle simple
binary similarity. The complex multilevel semantic structure of images
associated with multiple labels have not yet been well explored. Here we
propose a deep semantic ranking based method for learning hash functions that
preserve multilevel semantic similarity between multi-label images. In our
approach, deep convolutional neural network is incorporated into hash functions
to jointly learn feature representations and mappings from them to hash codes,
which avoids the limitation of semantic representation power of hand-crafted
features. Meanwhile, a ranking list that encodes the multilevel similarity
information is employed to guide the learning of such deep hash functions. An
effective scheme based on surrogate loss is used to solve the intractable
optimization problem of nonsmooth and multivariate ranking measures involved in
the learning procedure. Experimental results show the superiority of our
proposed approach over several state-of-the-art hashing methods in term of
ranking evaluation metrics when tested on multi-label image datasets.

使用神经网络结合深度学习的方式，针对多标签图像，提出了一种能够保持多层语义相似度的哈希函数学习方法，并在多标签图像数据集上的排名评估中证明了该方法的优越性。

基于深度语义排名的多标签图像检索哈希

Deep Semantic Ranking Based Hashing for Multi-Label Image Retrieval

Convolutional Neural Network (CNN) has demonstrated promising performance in
single-label image classification tasks. However, how CNN best copes with
multi-label images still remains an open problem, mainly due to the complex
underlying object layouts and insufficient multi-label training images. In this
work, we propose a flexible deep CNN infrastructure, called
Hypotheses-CNN-Pooling (HCP), where an arbitrary number of object segment
hypotheses are taken as the inputs, then a shared CNN is connected with each
hypothesis, and finally the CNN output results from different hypotheses are
aggregated with max pooling to produce the ultimate multi-label predictions.
Some unique characteristics of this flexible deep CNN infrastructure include:
1) no ground truth bounding box information is required for training; 2) the
whole HCP infrastructure is robust to possibly noisy and/or redundant
hypotheses; 3) no explicit hypothesis label is required; 4) the shared CNN may
be well pre-trained with a large-scale single-label image dataset, e.g.
ImageNet; and 5) it may naturally output multi-label prediction results.
Experimental results on Pascal VOC2007 and VOC2012 multi-label image datasets
well demonstrate the superiority of the proposed HCP infrastructure over other
state-of-the-arts. In particular, the mAP reaches 84.2% by HCP only and 90.3%
after the fusion with our complementary result in [47] based on hand-crafted
features on the VOC2012 dataset, which significantly outperforms the
state-of-the-arts with a large margin of more than 7%.

本研究提出了一个深度卷积神经网络结构 (HCP)，可灵活应对多标签图像任务，并成功应用于 Pascal VOC2007 和 VOC2012 数据集中，表现优于其他同类算法，最大平均精度达到 84.2%。