Advances in Deep Learning have made possible reliable landmark tracking of
human bodies and faces that can be used for a variety of tasks. We test a
recent Computer Vision solution, MediaPipe Holistic (MPH), to find out if its
tracking of the facial features is reliable enough for a linguistic analysis of
data from sign languages, and compare it to an older solution (OpenFace, OF).
We use an existing data set of sentences in Kazakh-Russian Sign Language and a
newly created small data set of videos with head tilts and eyebrow movements.
We find that MPH does not perform well enough for linguistic analysis of
eyebrow movement -- but in a different way from OF, which is also performing
poorly without correction. We reiterate a previous proposal to train additional
correction models to overcome these limitations.

深度学习、关键点追踪、面部特征、语言分析和手语是该篇研究的主要关键词，通过测试不同解决方案的性能，提出了训练额外校正模型来克服局限性的建议。

测试 MediaPipe Holistic 对手语中非手标记的语言分析

Testing MediaPipe Holistic for Linguistic Analysis of Nonmanual Markers  in Sign Languages

Facial expressions are widely used in the behavioral interpretation of
emotions, cognitive science, and social interactions. In this paper, we present
a novel method for fully automatic facial expression recognition in facial
image sequences. As the facial expression evolves over time facial landmarks
are automatically tracked in consecutive video frames, using displacements
based on elastic bunch graph matching displacement estimation. Feature vectors
from individual landmarks, as well as pairs of landmarks tracking results are
extracted, and normalized, with respect to the first frame in the sequence. The
prototypical expression sequence for each class of facial expression is formed,
by taking the median of the landmark tracking results from the training facial
expression sequences. Multi-class AdaBoost with dynamic time warping similarity
distance between the feature vector of input facial expression and prototypical
facial expression, is used as a weak classifier to select the subset of
discriminative feature vectors. Finally, two methods for facial expression
recognition are presented, either by using multi-class AdaBoost with dynamic
time warping, or by using support vector machine on the boosted feature
vectors. The results on the Cohn-Kanade (CK+) facial expression database show a
recognition accuracy of 95.17% and 97.35% using multi-class AdaBoost and
support vector machines, respectively.

本研究提出了一种全自动面部表情识别的新方法，该方法利用弹性束图匹配位移估计跟踪连续视频帧中的面部特征点，提取各个特征点的特征向量并进行归一化，通过多类 AdaBoost 和支持向量机方法识别面部表情，并在 Cohn-Kanade（CK +）面部表情数据库上实现了 95.17％和 97.35％的识别准确率。