In the evolving landscape of digital media and video production, the precise
manipulation and reproduction of visual elements like camera movements and
character actions are highly desired. Existing SLAM methods face limitations in
dynamic scenes and human pose estimation often focuses on 2D projections,
neglecting 3D statuses. To address these issues, we first introduce a reverse
filming behavior estimation technique. It optimizes camera trajectories by
leveraging NeRF as a differentiable renderer and refining SMPL tracks. We then
introduce a cinematic transfer pipeline that is able to transfer various shot
types to a new 2D video or a 3D virtual environment. The incorporation of 3D
engine workflow enables superior rendering and control abilities, which also
achieves a higher rating in the user study.

通过反向拍摄行为估计技术，优化相机轨迹，并借助可微分渲染器 NeRF 和细化的 SMPL 轨迹来解决现有 SLAM 方法在动态场景和人物姿态估计方面的局限性。结合 3D 引擎工作流的电影化转换管线，在用户研究中取得了更高的评级。

基于 NeRF 的可微电影化的电影行为转移

Cinematic Behavior Transfer via NeRF-based Differentiable Filming

We present a dataset for evaluating the tracking accuracy of monocular visual
odometry and SLAM methods. It contains 50 real-world sequences comprising more
than 100 minutes of video, recorded across dozens of different environments --
ranging from narrow indoor corridors to wide outdoor scenes. All sequences
contain mostly exploring camera motion, starting and ending at the same
position. This allows to evaluate tracking accuracy via the accumulated drift
from start to end, without requiring ground truth for the full sequence. In
contrast to existing datasets, all sequences are photometrically calibrated. We
provide exposure times for each frame as reported by the sensor, the camera
response function, and dense lens attenuation factors. We also propose a novel,
simple approach to non-parametric vignette calibration, which requires minimal
set-up and is easy to reproduce. Finally, we thoroughly evaluate two existing
methods (ORB-SLAM and DSO) on the dataset, including an analysis of the effect
of image resolution, camera field of view, and the camera motion direction.

提出了一个包含 50 个具有独特多样性的现实中场景的数据集，用于评估单目视觉里程计和 SLAM 方法的跟踪精度，并在此基础上提出了一个相机光度校准的简单方法以及一种非参数化暗角校准方法，该研究详细评估了两种现有方法（ORB-SLAM 和 DSO）在该数据集上的效果。