Depth perception is crucial for a wide range of robotic applications.
Multi-frame self-supervised depth estimation methods have gained research
interest due to their ability to leverage large-scale, unlabeled real-world
data. However, the self-supervised methods often rely on the assumption of a
static scene and their performance tends to degrade in dynamic environments. To
address this issue, we present Motion-Aware Loss, which leverages the temporal
relation among consecutive input frames and a novel distillation scheme between
the teacher and student networks in the multi-frame self-supervised depth
estimation methods. Specifically, we associate the spatial locations of moving
objects with the temporal order of input frames to eliminate errors induced by
object motion. Meanwhile, we enhance the original distillation scheme in
multi-frame methods to better exploit the knowledge from a teacher network. MAL
is a novel, plug-and-play module designed for seamless integration into
multi-frame self-supervised monocular depth estimation methods. Adding MAL into
previous state-of-the-art methods leads to a reduction in depth estimation
errors by up to 4.2% and 10.8% on KITTI and CityScapes benchmarks,
respectively.

多帧自监督单目深度估计方法中，我们通过使用运动感知损失以及在教师网络和学生网络间增强的蒸馏方案来解决动态环境下性能下降的问题。插入 Motion-Aware Loss 到之前最先进的方法中可使深度估计错误率在 KITTI 和 CityScapes 基准测试上分别减少 4.2% 和 10.8%。

MAL: 具有时空和蒸馏提示的动态感知损失以用于自监督深度估计

MAL: Motion-Aware Loss with Temporal and Distillation Hints for  Self-Supervised Depth Estimation

3D human body shape and pose estimation from RGB images is a challenging
problem with potential applications in augmented/virtual reality, healthcare
and fitness technology and virtual retail. Recent solutions have focused on
three types of inputs: i) single images, ii) multi-view images and iii) videos.
In this study, we surveyed and compared 3D body shape and pose estimation
methods for contemporary dance and performing arts, with a special focus on
human body pose and dressing, camera viewpoint, illumination conditions and
background conditions. We demonstrated that multi-frame methods, such as PHALP,
provide better results than single-frame method for pose estimation when
dancers are performing contemporary dances.

使用 RGB 图像进行三维人体形状和姿势估计是一个具有挑战性的问题，具有增强 / 虚拟现实、医疗保健和健身技术以及虚拟零售等潜在应用。本研究调查了并比较了当代舞蹈和表演艺术领域中的三维人体形状和姿势估计方法，重点关注人体姿势和着装、摄像机视角、光照条件和背景条件。我们证明了当舞者进行当代舞蹈表演时，多帧方法（如 PHALP）比单帧方法更能提供更好的姿势估计结果。

现代舞应用中 3D 人体姿势和形状估计方法综述

Survey of 3D Human Body Pose and Shape Estimation Methods for  Contemporary Dance Applications

Occlusions between consecutive frames have long posed a significant challenge
in optical flow estimation. The inherent ambiguity introduced by occlusions
directly violates the brightness constancy constraint and considerably hinders
pixel-to-pixel matching. To address this issue, multi-frame optical flow
methods leverage adjacent frames to mitigate the local ambiguity. Nevertheless,
prior multi-frame methods predominantly adopt recursive flow estimation,
resulting in a considerable computational overlap. In contrast, we propose a
streamlined in-batch framework that eliminates the need for extensive redundant
recursive computations while concurrently developing effective spatio-temporal
modeling approaches under in-batch estimation constraints. Specifically, we
present a Streamlined In-batch Multi-frame (SIM) pipeline tailored to video
input, attaining a similar level of time efficiency to two-frame networks.
Furthermore, we introduce an efficient Integrative Spatio-temporal Coherence
(ISC) modeling method for effective spatio-temporal modeling during the
encoding phase, which introduces no additional parameter overhead.
Additionally, we devise a Global Temporal Regressor (GTR) that effectively
explores temporal relations during decoding. Benefiting from the efficient SIM
pipeline and effective modules, StreamFlow not only excels in terms of
performance on the challenging KITTI and Sintel datasets, with particular
improvement in occluded areas but also attains a remarkable $63.82\%$
enhancement in speed compared with previous multi-frame methods. The code will
be available soon at this https URL

针对光流估计中的连续帧遮挡问题，我们提出了一个简化的批处理多帧（SIM）流程，包括高效的时空建模方法和全局时间回归器（GTR），可在解码过程中有效地探索时间关系。相对于以前的多帧方法，该方法不仅在 KITTI 和 Sintel 数据集上的性能表现出色，尤其是在遮挡区域方面有显著改进，而且在速度方面取得了显著的 63.82% 提升。