Given a video of a person in action, we can easily guess the 3D future motion
of the person. In this work, we present perhaps the first approach for
predicting a future 3D mesh model sequence of a person from past video input.
We do this for periodic motions such as walking and also actions like bowling
and squatting seen in sports or workout videos. While there has been a surge of
future prediction problems in computer vision, most approaches predict 3D
future from 3D past or 2D future from 2D past inputs. In this work, we focus on
the problem of predicting 3D future motion from past image sequences, which has
a plethora of practical applications in autonomous systems that must operate
safely around people from visual inputs. Inspired by the success of
autoregressive models in language modeling tasks, we learn an intermediate
latent space on which we predict the future. This effectively facilitates
autoregressive predictions when the input differs from the output domain. Our
approach can be trained on video sequences obtained in-the-wild without 3D
ground truth labels. The project website with videos can be found at
this https URL

本篇论文提出了一种可以根据过去的视频输入，预测人体未来三维网格模型序列的方法，通过引入自回归模型在中间潜在空间上进行预测，有效地促进了自回归预测。

从视频预测三维人体动态

Predicting 3D Human Dynamics from Video

We consider the problem of next frame prediction from video input. A
recurrent convolutional neural network is trained to predict depth from
monocular video input, which, along with the current video image and the camera
trajectory, can then be used to compute the next frame. Unlike prior next-frame
prediction approaches, we take advantage of the scene geometry and use the
predicted depth for generating the next frame prediction. Our approach can
produce rich next frame predictions which include depth information attached to
each pixel. Another novel aspect of our approach is that it predicts depth from
a sequence of images (e.g. in a video), rather than from a single still image.
We evaluate the proposed approach on the KITTI dataset, a standard dataset for
benchmarking tasks relevant to autonomous driving. The proposed method produces
results which are visually and numerically superior to existing methods that
directly predict the next frame. We show that the accuracy of depth prediction
improves as more prior frames are considered.

通过训练递归卷积神经网络，从视频输入中预测深度来产生下一个帧，此方法可以较其他直接预测下一个帧的方法产生更精确的深度预测，并且通过考虑更多的先前帧可以提高预测准确率。