Remote photoplethysmography (rPPG) is a non-contact method for detecting
physiological signals based on facial videos, holding high potential in various
applications such as healthcare, affective computing, anti-spoofing, etc. Due
to the periodicity nature of rPPG, the long-range dependency capturing capacity
of the Transformer was assumed to be advantageous for such signals. However,
existing approaches have not conclusively demonstrated the superior performance
of Transformer over traditional convolutional neural network methods, this gap
may stem from a lack of thorough exploration of rPPG periodicity. In this
paper, we propose RhythmFormer, a fully end-to-end transformer-based method for
extracting rPPG signals by explicitly leveraging the quasi-periodic nature of
rPPG. The core module, Hierarchical Temporal Periodic Transformer,
hierarchically extracts periodic features from multiple temporal scales. It
utilizes dynamic sparse attention based on periodicity in the temporal domain,
allowing for fine-grained modeling of rPPG features. Furthermore, a fusion stem
is proposed to guide self-attention to rPPG features effectively, and it can be
easily transferred to existing methods to enhance their performance
significantly. RhythmFormer achieves state-of-the-art performance with fewer
parameters and reduced computational complexity in comprehensive experiments
compared to previous approaches. The codes are available at
this https URL

RhythmFormer 是一种基于 Transformer 的全端对端方法，通过明确利用 rPPG 的准周期性特性来提取 rPPG 信号。在多个时间尺度上，层次化时间周期性 Transformer 核心模块从中提取周期性特征，利用时间域的动态稀疏注意力来对 rPPG 特征进行精细建模。此外，还提出了融合模块以有效地指导自注意力到 rPPG 特征，能够轻松地转移到现有方法以显著提升它们的性能。与之前的方法相比，在综合实验中，RhythmFormer 表现出更少的参数和降低的计算复杂度，达到了最先进的性能。