One of the mainstream schemes for 2D human pose estimation (HPE) is learning
keypoints heatmaps by a neural network. Existing methods typically improve the
quality of heatmaps by customized architectures, such as high-resolution
representation and vision Transformers. In this paper, we propose
\textbf{DiffusionPose}, a new scheme that formulates 2D HPE as a keypoints
heatmaps generation problem from noised heatmaps. During training, the
keypoints are diffused to random distribution by adding noises and the
diffusion model learns to recover ground-truth heatmaps from noised heatmaps
with respect to conditions constructed by image feature. During inference, the
diffusion model generates heatmaps from initialized heatmaps in a progressive
denoising way. Moreover, we further explore improving the performance of
DiffusionPose with conditions from human structural information. Extensive
experiments show the prowess of our DiffusionPose, with improvements of 1.6,
1.2, and 1.2 mAP on widely-used COCO, CrowdPose, and AI Challenge datasets,
respectively.

本文提出了 DiffusionPose，将 2D 人体姿态估计问题定义为噪声热图的关键点热图生成问题，并通过添加噪声，将关键点扩散到随机分布中，学习扩散模型从噪声热图中恢复与图像特征相关的地面真值热图，从而实现从初始化热图的漸進式降噪方式生成热图。此外，本文还进一步探究了从人体结构信息中提取条件以提高 DiffusionPose 的性能。经过广泛的实验，证明了 DiffusionPose 的优越性，在广泛使用的 COCO、CrowdPose 和 AI Challenge 数据集上分别提高了 1.6、1.2 和 1.2 mAP。

学习结构引导的扩散模型用于二维人体姿态估计

Learning Structure-Guided Diffusion Model for 2D Human Pose Estimation

In previous deep-learning-based methods, semantic segmentation has been
regarded as a static or dynamic per-pixel classification task, \textit{i.e.,}
classify each pixel representation to a specific category. However, these
methods only focus on learning better pixel representations or classification
kernels while ignoring the structural information of objects, which is critical
to human decision-making mechanism. In this paper, we present a new paradigm
for semantic segmentation, named structure-aware extraction. Specifically, it
generates the segmentation results via the interactions between a set of
learned structure tokens and the image feature, which aims to progressively
extract the structural information of each category from the feature. Extensive
experiments show that our StructToken outperforms the state-of-the-art on three
widely-used benchmarks, including ADE20K, Cityscapes, and COCO-Stuff-10K.

本文提出了一种新的语义分割范式，名为结构感知抽取，并通过一组学习到的结构单元与图像特征之间的交互逐步提取每个类别的结构信息来生成分割结果，实验证明其优于目前广泛使用的三个基准测试。

StructToken：基于结构先验的语义分割方法再思考

StructToken : Rethinking Semantic Segmentation with Structural Prior

The impressive performance of Convolutional Neural Networks (CNNs) when
solving different vision problems is shadowed by their black-box nature and our
consequent lack of understanding of the representations they build and how
these representations are organized. To help understanding these issues, we
propose to describe the activity of individual neurons by their Neuron Feature
visualization and quantify their inherent selectivity with two specific
properties. We explore selectivity indexes for: an image feature (color); and
an image label (class membership). Our contribution is a framework to seek or
classify neurons by indexing on these selectivity properties. It helps to find
color selective neurons, such as a red-mushroom neuron in layer Conv4 or class
selective neurons such as dog-face neurons in layer Conv5 in VGG-M, and
establishes a methodology to derive other selectivity properties. Indexing on
neuron selectivity can statistically draw how features and classes are
represented through layers in a moment when the size of trained nets is growing
and automatic tools to index neurons can be helpful.

本文为了帮助理解 CNNs 的黑盒特性以及它们构建的表示和这些表示如何组织，提出了通过神经元功能可视化描述神经元的活动，并用两个特定的属性量化它们固有的选择性，探索特定选择性指标，并建立了一个通过选择性属性对神经元进行索引的框架，可帮助找到颜色选择性神经元和类选择性神经元，从而在神经元索引方面提供了一种方法。