This report presents our team's 'PCIE_LAM' solution for the Ego4D Looking At
Me Challenge at CVPR2024. The main goal of the challenge is to accurately
determine if a person in the scene is looking at the camera wearer, based on a
video where the faces of social partners have been localized. Our proposed
solution, InternLSTM, consists of an InternVL image encoder and a Bi-LSTM
network. The InternVL extracts spatial features, while the Bi-LSTM extracts
temporal features. However, this task is highly challenging due to the distance
between the person in the scene and the camera movement, which results in
significant blurring in the face image. To address the complexity of the task,
we implemented a Gaze Smoothing filter to eliminate noise or spikes from the
output. Our approach achieved the 1st position in the looking at me challenge
with 0.81 mAP and 0.93 accuracy rate. Code is available at
this https URL

该报告介绍了我们团队在 CVPR2024 的 Ego4D Looking At Me 挑战中的 'PCIE_LAM' 解决方案。我们的方案 InternLSTM 由 InternVL 图像编码器和 Bi-LSTM 网络组成，通过提取空间和时间特征来准确确定场景中的人是否面向佩戴摄像头的摄像机。鉴于此任务中存在的面部图像模糊问题，我们采用了 Gaze Smoothing 滤波器来消除噪音或波动，并在挑战中获得了第一名，mAP 为 0.81，准确率为 0.93。代码可在链接中访问。