In this paper, we propose MoDGS, a new pipeline to render novel-view images
in dynamic scenes using only casually captured monocular videos. Previous
monocular dynamic NeRF or Gaussian Splatting methods strongly rely on the rapid
movement of input cameras to construct multiview consistency but fail to
reconstruct dynamic scenes on casually captured input videos whose cameras are
static or move slowly. To address this challenging task, MoDGS adopts recent
single-view depth estimation methods to guide the learning of the dynamic
scene. Then, a novel 3D-aware initialization method is proposed to learn a
reasonable deformation field and a new robust depth loss is proposed to guide
the learning of dynamic scene geometry. Comprehensive experiments demonstrate
that MoDGS is able to render high-quality novel view images of dynamic scenes
from just a casually captured monocular video, which outperforms baseline
methods by a significant margin.

在本文中，我们提出了 MoDGS，这是一个新的管道，可以利用仅有的随意捕捉的单目视频生成动态场景的新视角图像。MoDGS 采用了最近的单视图深度估计方法来指导动态场景的学习，并提出了一种新的三维感知初始化方法来学习合理的变形场和新的鲁棒深度损失来指导动态场景几何结构的学习。全面的实验证明，MoDGS 能够从仅有的随意捕捉的单目视频中生成高质量的动态场景新视角图像，优于基线方法的效果显著。

MoDGS: 动态高斯分布从偶然捕获的单目视频

MoDGS: Dynamic Gaussian Splatting from Causually-captured Monocular  Videos

Synthesizing multi-view 3D from one single image is a significant and
challenging task. For this goal, Zero-1-to-3 methods aim to extend a 2D latent
diffusion model to the 3D scope. These approaches generate the target-view
image with a single-view source image and the camera pose as condition
information. However, the one-to-one manner adopted in Zero-1-to-3 incurs
challenges for building geometric and visual consistency across views,
especially for complex objects. We propose a cascade generation framework
constructed with two Zero-1-to-3 models, named Cascade-Zero123, to tackle this
issue, which progressively extracts 3D information from the source image.
Specifically, a self-prompting mechanism is designed to generate several nearby
views at first. These views are then fed into the second-stage model along with
the source image as generation conditions. With self-prompted multiple views as
the supplementary information, our Cascade-Zero123 generates more highly
consistent novel-view images than Zero-1-to-3. The promotion is significant for
various complex and challenging scenes, involving insects, humans, transparent
objects, and stacked multiple objects etc. The project page is at
this https URL

从一张单一图像中综合多视图 3D 是一项重要且具有挑战性的任务。为了解决这个问题，Zero-1-to-3 方法旨在将二维潜在扩散模型推广到三维范围。我们提出了一个由两个 Zero-1-to-3 模型构建的级联生成框架，名为 Cascade-Zero123，用于解决构建视图之间的几何和视觉一致性的问题。利用自发多视图作为补充信息，我们的 Cascade-Zero123 生成比 Zero-1-to-3 更一致的新视图图像，对于各种复杂和具有挑战性的场景非常有效，包括昆虫、人类、透明物体和堆叠多个物体等。

Cascade-Zero123: 单幅图像到高度一致的 3D 图像的自提近景方法

Cascade-Zero123: One Image to Highly Consistent 3D with Self-Prompted  Nearby Views

We tackle the problem of generating novel-view images from collections of 2D
images showing refractive and reflective objects. Current solutions assume
opaque or transparent light transport along straight paths following the
emission-absorption model. Instead, we optimize for a field of 3D-varying Index
of Refraction (IoR) and trace light through it that bends toward the spatial
gradients of said IoR according to the laws of eikonal light transport.

本文研究如何利用变化的折射率（IoR）优化射线跟随路径，从而生成展示折射和反射对象的新视角图像。