Millimetre wave (mmWave) radar is a non-intrusive privacy and relatively
convenient and inexpensive device, which has been demonstrated to be applicable
in place of RGB cameras in human indoor pose estimation tasks. However, mmWave
radar relies on the collection of reflected signals from the target, and the
radar signals containing information is difficult to be fully applied. This has
been a long-standing hindrance to the improvement of pose estimation accuracy.
To address this major challenge, this paper introduces a probability map guided
multi-format feature fusion model, ProbRadarM3F. This is a novel radar feature
extraction framework using a traditional FFT method in parallel with a
probability map based positional encoding method. ProbRadarM3F fuses the
traditional heatmap features and the positional features, then effectively
achieves the estimation of 14 keypoints of the human body. Experimental
evaluation on the HuPR dataset proves the effectiveness of the model proposed
in this paper, outperforming other methods experimented on this dataset with an
AP of 69.9 %. The emphasis of our study is focusing on the position information
that is not exploited before in radar singal. This provides direction to
investigate other potential non-redundant information from mmWave rader.

通过概率地图引导的多格式特征融合模型 ProbRadarM3F，本文提出了一种用于人体姿态估计的新型毫米波雷达特征提取框架，有效地实现了人体的 14 个关键点的估计。实验证明了该模型的有效性，其 AP 达到 69.9%，突出了研究中未被利用的位置信息，为探索毫米波雷达中的其他潜在非冗余信息提供了方向。

ProbRadarM3F: 基于 mmWave 雷达的人体骨骼姿态估计与概率图引导的多格式特征融合

ProbRadarM3F: mmWave Radar based Human Skeletal Pose Estimation with  Probability Map Guided Multi-Format Feature Fusion

Multi-Object Tracking (MOT) aims to maintain stable and uninterrupted
trajectories for each target. Most state-of-the-art approaches first detect
objects in each frame and then implement data association between new
detections and existing tracks using motion models and appearance similarities.
Despite achieving satisfactory results, occlusion and crowds can easily lead to
missing and distorted detections, followed by missing and false associations.
In this paper, we first revisit the classic tracker DeepSORT, enhancing its
robustness over crowds and occlusion significantly by placing greater trust in
predictions when detections are unavailable or of low quality in crowded and
occluded scenes. Specifically, we propose a new framework comprising of three
lightweight and plug-and-play algorithms: the probability map, the prediction
map, and the covariance adaptive Kalman filter. The probability map identifies
whether undetected objects have genuinely disappeared from view (e.g., out of
the image or entered a building) or are only temporarily undetected due to
occlusion or other reasons. Trajectories of undetected targets that are still
within the probability map are extended by state estimations directly. The
prediction map determines whether an object is in a crowd, and we prioritize
state estimations over observations when severe deformation of observations
occurs, accomplished through the covariance adaptive Kalman filter. The
proposed method, named MapTrack, achieves state-of-the-art results on popular
multi-object tracking benchmarks such as MOT17 and MOT20. Despite its superior
performance, our method remains simple, online, and real-time. The code will be
open-sourced later.

在本文中，我们提出了一种名为 MapTrack 的方法，利用概率图、预测图和协方差自适应卡尔曼滤波器的三个轻量级插件算法，通过增强经典跟踪器 DeepSORT 在拥挤和遮挡场景中的鲁棒性，实现在常用的多目标跟踪基准集（如 MOTS17 和 MOT20）上的最先进效果。

MapTrack：在地图中的跟踪

MapTrack: Tracking in the Map

Recently, talking face generation has drawn ever-increasing attention from
the research community in computer vision due to its arduous challenges and
widespread application scenarios, e.g. movie animation and virtual anchor.
Although persevering efforts have been undertaken to enhance the fidelity and
lip-sync quality of generated talking face videos, there is still large room
for further improvements of synthesis quality and efficiency. Actually, these
attempts somewhat ignore the explorations of fine-granularity feature
extraction/integration and the consistency between probability distributions of
landmarks, thereby recurring the issues of local details blurring and degraded
fidelity. To mitigate these dilemmas, in this paper, a novel CLIP-based
Attention and Probability Map Guided Network (CPNet) is delicately designed for
inferring high-fidelity talking face videos. Specifically, considering the
demands of fine-grained feature recalibration, a clip-based attention condenser
is exploited to transfer knowledge with rich semantic priors from the
prevailing CLIP model. Moreover, to guarantee the consistency in probability
space and suppress the landmark ambiguity, we creatively propose the density
map of facial landmark as auxiliary supervisory signal to guide the landmark
distribution learning of generated frame. Extensive experiments on the
widely-used benchmark dataset demonstrate the superiority of our CPNet against
state of the arts in terms of image and lip-sync quality. In addition, a cohort
of studies are also conducted to ablate the impacts of the individual pivotal
components.

本文提出了一个基于 CLIP 的关注力和概率图引导网络 (CPNet)，用于推断高保真的人脸合成视频。通过 fine-grained 特征整合和概率分布的一致性，该方法克服了局部细节模糊和降级保真度问题，取得了比现有技术更好的成果。