Existing state-of-the-art human pose estimation methods require heavy
computational resources for accurate predictions. One promising technique to
obtain an accurate yet lightweight pose estimator is knowledge distillation,
which distills the pose knowledge from a powerful teacher model to a
less-parameterized student model. However, existing pose distillation works
rely on a heavy pre-trained estimator to perform knowledge transfer and require
a complex two-stage learning procedure. In this work, we investigate a novel
Online Knowledge Distillation framework by distilling Human Pose structure
knowledge in a one-stage manner to guarantee the distillation efficiency,
termed OKDHP. Specifically, OKDHP trains a single multi-branch network and
acquires the predicted heatmaps from each, which are then assembled by a
Feature Aggregation Unit (FAU) as the target heatmaps to teach each branch in
reverse. Instead of simply averaging the heatmaps, FAU which consists of
multiple parallel transformations with different receptive fields, leverages
the multi-scale information, thus obtains target heatmaps with higher-quality.
Specifically, the pixel-wise Kullback-Leibler (KL) divergence is utilized to
minimize the discrepancy between the target heatmaps and the predicted ones,
which enables the student network to learn the implicit keypoint relationship.
Besides, an unbalanced OKDHP scheme is introduced to customize the student
networks with different compression rates. The effectiveness of our approach is
demonstrated by extensive experiments on two common benchmark datasets, MPII
and COCO.

本文提出了一种新型的在线知识蒸馏方法 OKDHP，通过多分支网络和特征聚合单元以及像素级 Kullback-Leibler 散度来保证人体姿态估计的效率和准确性。实验结果表明，该方法在 MPII 和 COCO 数据集上表现良好。

在线知识蒸馏用于高效姿态估计

Online Knowledge Distillation for Efficient Pose Estimation

Most of unsupervised person Re-Identification (Re-ID) works produce
pseudo-labels by measuring the feature similarity without considering the
distribution discrepancy among cameras, leading to degraded accuracy in label
computation across cameras. This paper targets to address this challenge by
studying a novel intra-inter camera similarity for pseudo-label generation. We
decompose the sample similarity computation into two stage, i.e., the
intra-camera and inter-camera computations, respectively. The intra-camera
computation directly leverages the CNN features for similarity computation
within each camera. Pseudo-labels generated on different cameras train the
re-id model in a multi-branch network. The second stage considers the
classification scores of each sample on different cameras as a new feature
vector. This new feature effectively alleviates the distribution discrepancy
among cameras and generates more reliable pseudo-labels. We hence train our
re-id model in two stages with intra-camera and inter-camera pseudo-labels,
respectively. This simple intra-inter camera similarity produces surprisingly
good performance on multiple datasets, e.g., achieves rank-1 accuracy of 89.5%
on the Market1501 dataset, outperforming the recent unsupervised works by 9+%,
and is comparable with the latest transfer learning works that leverage extra
annotations.

本文提出一种基于 intra-inter camera 相似度的方法，生成更可靠的 pseudo-labels 用于训练 Re-ID 模型在多分支网络中，并在多个数据集上取得了 89.5% 的 rank-1 准确率，优于最新的无监督方法 9+%，同时与使用额外标注的最新迁移学习方法相当。

无监督个体再识别的摄像头内部 - 外部相似性

Intra-Inter Camera Similarity for Unsupervised Person Re-Identification

The method introduced in this paper aims at helping deep learning
practitioners faced with an overfit problem. The idea is to replace, in a
multi-branch network, the standard summation of parallel branches with a
stochastic affine combination. Applied to 3-branch residual networks,
shake-shake regularization improves on the best single shot published results
on CIFAR-10 and CIFAR-100 by reaching test errors of 2.86% and 15.85%.
Experiments on architectures without skip connections or Batch Normalization
show encouraging results and open the door to a large set of applications. Code
is available at this https URL

本文介绍了一种替换标准分支求和的随机仿射组合方法，用于改善深度学习中过拟合问题，结合掷骰子正则化可提高 CIFAR-10 和 CIFAR-100 测试准确度，对于跳跃连接或批量归一化的架构也有鼓舞人心的结果，具有广泛的应用前景。