Remote photoplethysmography (rPPG) based physiological measurement is an
emerging yet crucial vision task, whose challenge lies in exploring accurate
rPPG prediction from facial videos accompanied by noises of illumination
variations, facial occlusions, head movements, \etc, in a non-contact manner.
Existing mainstream CNN-based models make efforts to detect physiological
signals by capturing subtle color changes in facial regions of interest (ROI)
caused by heartbeats. However, such models are constrained by the limited local
spatial or temporal receptive fields in the neural units. Unlike them, a native
Transformer-based framework called Dual-path TokenLearner (Dual-TL) is proposed
in this paper, which utilizes the concept of learnable tokens to integrate both
spatial and temporal informative contexts from the global perspective of the
video. Specifically, the proposed Dual-TL uses a Spatial TokenLearner (S-TL) to
explore associations in different facial ROIs, which promises the rPPG
prediction far away from noisy ROI disturbances. Complementarily, a Temporal
TokenLearner (T-TL) is designed to infer the quasi-periodic pattern of
heartbeats, which eliminates temporal disturbances such as head movements. The
two TokenLearners, S-TL and T-TL, are executed in a dual-path mode. This
enables the model to reduce noise disturbances for final rPPG signal
prediction. Extensive experiments on four physiological measurement benchmark
datasets are conducted. The Dual-TL achieves state-of-the-art performances in
both intra- and cross-dataset testings, demonstrating its immense potential as
a basic backbone for rPPG measurement. The source code is available at
\href{https://github.com/VUT-HFUT/Dual-TL}{this https URL}

使用基于学习 Token 的双路径 TokenLearner（Dual-TL）的原生 Transformer 框架，综合利用面部视频的空间和时间信息，实现高准确度的远程光电体积脉搏测量。

基于面部视频的远程光电容数据测量的双路径 TokenLearner

Dual-path TokenLearner for Remote Photoplethysmography-based  Physiological Measurement with Facial Videos

This work introduces a novel DeepFake detection framework based on
physiological measurement. In particular, we consider information related to
the heart rate using remote photoplethysmography (rPPG). rPPG methods analyze
video sequences looking for subtle color changes in the human skin, revealing
the presence of human blood under the tissues. In this work we investigate to
what extent rPPG is useful for the detection of DeepFake videos.
The proposed fake detector named DeepFakesON-Phys uses a Convolutional
Attention Network (CAN), which extracts spatial and temporal information from
video frames, analyzing and combining both sources to better detect fake
videos. This detection approach has been experimentally evaluated using the
latest public databases in the field: Celeb-DF and DFDC. The results achieved,
above 98% AUC (Area Under the Curve) on both databases, outperform the state of
the art and prove the success of fake detectors based on physiological
measurement to detect the latest DeepFake videos.

本文介绍了一种基于生理测量的新型 DeepFake 检测框架，利用远程光电测量心率（rPPG）方法来分析视频序列中与人类皮肤微小颜色变化相关的信息，以便更好地检测制造的 DeepFake 视频。我们提出的 DeepFakesON-Phys 假检测器使用卷积注意网络（CAN）从视频帧中提取空间和时间信息，分析并结合两个来源以更好地检测假视频。探究结果表明，该检测方法在 Celeb-DF 和 DFDC 数据库上达到了 98% 以上的 AUC（曲线下的面积），超过了现有技术的水平，并证明了基于生理测量的假检测器成功检测了最新的 DeepFake 视频。