Considering the complementarity of scene flow estimation in the spatial
domain's focusing capability and 3D object tracking in the temporal domain's
coherence, this study aims to address a comprehensive new task that can
simultaneously capture fine-grained and long-term 3D motion in an online
manner: long-term scene flow estimation (LSFE). We introduce SceneTracker, a
novel learning-based LSFE network that adopts an iterative approach to
approximate the optimal trajectory. Besides, it dynamically indexes and
constructs appearance and depth correlation features simultaneously and employs
the Transformer to explore and utilize long-range connections within and
between trajectories. With detailed experiments, SceneTracker shows superior
capabilities in handling 3D spatial occlusion and depth noise interference,
highly tailored to the LSFE task's needs. The code for SceneTracker is
available at this https URL

本研究介绍了一种名为 SceneTracker 的基于学习的长期场景流估计（LSFE）网络，通过迭代方法来近似最优轨迹，并同时动态索引和构建外观和深度相关特征，并采用 Transformer 探索和利用轨迹之间的长程连接。通过详细实验，SceneTracker 展示了处理三维空间遮挡和深度噪声干扰的卓越能力，非常适合 LSFE 任务的需求。SceneTracker 的代码可在此 URL 获取：https:// 该网址。

SceneTracker：长期场景流估计网络

SceneTracker: Long-term Scene Flow Estimation Network

Text-to-motion generation has gained increasing attention, but most existing
methods are limited to generating short-term motions that correspond to a
single sentence describing a single action. However, when a text stream
describes a sequence of continuous motions, the generated motions corresponding
to each sentence may not be coherently linked. Existing long-term motion
generation methods face two main issues. Firstly, they cannot directly generate
coherent motions and require additional operations such as interpolation to
process the generated actions. Secondly, they generate subsequent actions in an
autoregressive manner without considering the influence of future actions on
previous ones. To address these issues, we propose a novel approach that
utilizes a past-conditioned diffusion model with two optional coherent sampling
methods: Past Inpainting Sampling and Compositional Transition Sampling. Past
Inpainting Sampling completes subsequent motions by treating previous motions
as conditions, while Compositional Transition Sampling models the distribution
of the transition as the composition of two adjacent motions guided by
different text prompts. Our experimental results demonstrate that our proposed
method is capable of generating compositional and coherent long-term 3D human
motions controlled by a user-instructed long text stream. The code is available
at
\href{https://github.com/yangzhao1230/PCMDM}{this https URL}.

我们提出了一种新方法，利用过去条件扩散模型和两种可选的相干采样方法：过去修复采样和组合过渡采样，实现由用户指导的长文本流控制的组合和连贯的长期三维人体动作生成。

通过一致的采样使用扩散模型合成长期人类运动

Synthesizing Long-Term Human Motions with Diffusion Models via Coherent  Sampling

Learning the spatial-temporal representation of motion information is crucial
to human action recognition. Nevertheless, most of the existing features or
descriptors cannot capture motion information effectively, especially for
long-term motion. To address this problem, this paper proposes a long-term
motion descriptor called sequential Deep Trajectory Descriptor (sDTD).
Specifically, we project dense trajectories into two-dimensional planes, and
subsequently a CNN-RNN network is employed to learn an effective representation
for long-term motion. Unlike the popular two-stream ConvNets, the sDTD stream
is introduced into a three-stream framework so as to identify actions from a
video sequence. Consequently, this three-stream framework can simultaneously
capture static spatial features, short-term motion and long-term motion in the
video. Extensive experiments were conducted on three challenging datasets: KTH,
HMDB51 and UCF101. Experimental results show that our method achieves
state-of-the-art performance on the KTH and UCF101 datasets, and is comparable
to the state-of-the-art methods on the HMDB51 dataset.

本文提出了一种名为连续深层轨迹描述符（sDTD）的长期运动描述符，它通过将稠密轨迹投影到二维平面上，再使用 CNN-RNN 网络来学习长期运动的有效表示，进而将 sDTD 流引入三流框架以从视频序列中识别动作。实验证明该方法在 KTH 和 UCF101 数据集上取得了最先进的性能，且在 HMDB51 数据集上与最先进的方法相媲美。