How do the neural networks distinguish two images? It is of critical
importance to understand the matching mechanism of deep models for developing
reliable intelligent systems for many risky visual applications such as
surveillance and access control. However, most existing deep metric learning
methods match the images by comparing feature vectors, which ignores the
spatial structure of images and thus lacks interpretability. In this paper, we
present a deep interpretable metric learning (DIML) method for more transparent
embedding learning. Unlike conventional metric learning methods based on
feature vector comparison, we propose a structural matching strategy that
explicitly aligns the spatial embeddings by computing an optimal matching flow
between feature maps of the two images. Our method enables deep models to learn
metrics in a more human-friendly way, where the similarity of two images can be
decomposed to several part-wise similarities and their contributions to the
overall similarity. Our method is model-agnostic, which can be applied to
off-the-shelf backbone networks and metric learning methods. We evaluate our
method on three major benchmarks of deep metric learning including CUB200-2011,
Cars196, and Stanford Online Products, and achieve substantial improvements
over popular metric learning methods with better interpretability. Code is
available at this https URL

本文提出了一种新的深度可解释度度量学习方法 (DIML)，基于结构匹配策略和最优匹配流来弥补现有方法 (基于特征向量比较) 的缺点，实现了深度模型以更人性化的方式学习度量，将图像相似性分解为几个子相似性及其对整体相似性的影响，该方法适用于各种骨干网络和度量学习方法，并在三个深度度量学习基准测试中表现出了显著的改进，但不影响模型性能。

结构匹配下可解释的深度度量学习

Towards Interpretable Deep Metric Learning with Structural Matching

We develop a robust multi-scale structure-aware neural network for human pose
estimation. This method improves the recent deep conv-deconv hourglass models
with four key improvements: (1) multi-scale supervision to strengthen
contextual feature learning in matching body keypoints by combining feature
heatmaps across scales, (2) multi-scale regression network at the end to
globally optimize the structural matching of the multi-scale features, (3)
structure-aware loss used in the intermediate supervision and at the regression
to improve the matching of keypoints and respective neighbors to infer a
higher-order matching configurations, and (4) a keypoint masking training
scheme that can effectively fine-tune our network to robustly localize occluded
keypoints via adjacent matches. Our method can effectively improve
state-of-the-art pose estimation methods that suffer from difficulties in scale
varieties, occlusions, and complex multi-person scenarios. This multi-scale
supervision tightly integrates with the regression network to effectively (i)
localize keypoints using the ensemble of multi-scale features, and (ii) infer
global pose configuration by maximizing structural consistencies across
multiple keypoints and scales. The keypoint masking training enhances these
advantages to focus learning on hard occlusion samples. Our method achieves the
leading position in the MPII challenge leaderboard among the state-of-the-art
methods.

本文提出了一种多尺度结构感知神经网络，通过多尺度监督、多尺度回归网络、中间监督和结构感知损失以及关键点掩蔽训练方案等四个方面对深度卷积 - 反卷积沙漏模型进行改进，以有效地提高人体姿势估计的性能。该网络不仅可以解决尺度差异、遮挡和复杂多人场景等问题，而且可以全局优化多尺度特征的结构匹配，并在 MPII 挑战排行榜上取得了领先地位。