We present a real-time method for robust estimation of multiple instances of
geometric models from noisy data. Geometric models such as vanishing points,
planar homographies or fundamental matrices are essential for 3D scene
analysis. Previous approaches discover distinct model instances in an iterative
manner, thus limiting their potential for speedup via parallel computation. In
contrast, our method detects all model instances independently and in parallel.
A neural network segments the input data into clusters representing potential
model instances by predicting multiple sets of sample and inlier weights. Using
the predicted weights, we determine the model parameters for each potential
instance separately in a RANSAC-like fashion. We train the neural network via
task-specific loss functions, i.e. we do not require a ground-truth
segmentation of the input data. As suitable training data for homography and
fundamental matrix fitting is scarce, we additionally present two new synthetic
datasets. We demonstrate state-of-the-art performance on these as well as
multiple established datasets, with inference times as small as five
milliseconds per image.

我们提出了一种实时方法，用于从嘈杂数据中稳健地估计多个几何模型实例。

PARSAC: 并行样本一致性加速健壮的多模型拟合

PARSAC: Accelerating Robust Multi-Model Fitting with Parallel Sample  Consensus

We present the first real-time method for inserting a rigid virtual object
into a neural radiance field, which produces realistic lighting and shadowing
effects, as well as allows interactive manipulation of the object. By
exploiting the rich information about lighting and geometry in a NeRF, our
method overcomes several challenges of object insertion in augmented reality.
For lighting estimation, we produce accurate, robust and 3D spatially-varying
incident lighting that combines the near-field lighting from NeRF and an
environment lighting to account for sources not covered by the NeRF. For
occlusion, we blend the rendered virtual object with the background scene using
an opacity map integrated from the NeRF. For shadows, with a precomputed field
of spherical signed distance field, we query the visibility term for any point
around the virtual object, and cast soft, detailed shadows onto 3D surfaces.
Compared with state-of-the-art techniques, our approach can insert virtual
object into scenes with superior fidelity, and has a great potential to be
further applied to augmented reality systems.

我们提出了第一个实时方法，用于将刚性虚拟对象插入到神经辐射场中，它能够产生逼真的光照和阴影效果，并且允许对对象进行交互操作。通过利用 NeRF 中有关光照和几何的丰富信息，我们的方法克服了增强现实中对象插入的几个挑战。对于光照估计，我们产生了精确、稳健和具有三维空间变化的入射光，结合了 NeRF 的近场光照和环境光，以考虑 NeRF 未覆盖的光源。对于遮挡，我们使用从 NeRF 中整合的不透明度图将渲染的虚拟对象与背景场景混合。对于阴影，我们使用一个预计算的球形符号距离场查询视点周围的任意点的可见性，并在三维表面上投射软、详细的阴影。与现有技术相比，我们的方法能够以更高的保真度将虚拟对象插入场景中，并有很大潜力进一步应用于增强现实系统。

一种实时插入虚拟物体到神经辐射场的方法

A Real-time Method for Inserting Virtual Objects into Neural Radiance  Fields

Recent progress in NeRF-based GANs has introduced a number of approaches for
high-resolution and high-fidelity generative modeling of human heads with a
possibility for novel view rendering. At the same time, one must solve an
inverse problem to be able to re-render or modify an existing image or video.
Despite the success of universal optimization-based methods for 2D GAN
inversion, those, applied to 3D GANs, may fail to produce 3D-consistent
renderings. Fast encoder-based techniques, such as those developed for
StyleGAN, may also be less appealing due to the lack of identity preservation.
In our work, we introduce a real-time method that bridges the gap between the
two approaches by directly utilizing the tri-plane representation introduced
for EG3D generative model. In particular, we build upon a feed-forward
convolutional encoder for the latent code and extend it with a
fully-convolutional predictor of tri-plane numerical offsets. As shown in our
work, the renderings are similar in quality to optimization-based techniques
and significantly outperform the baselines for novel view. As we empirically
prove, this is a consequence of directly operating in the tri-plane space, not
in the GAN parameter space, while making use of an encoder-based trainable
approach.

该论文介绍了一种基于三平面表示的实时方法，该方法直接利用编码器进行潜在编码，并将其扩展为三平面数值偏移的全卷积预测器，从而弥合了当今用于高质量变形和重新渲染存在的方法之间的差距。

TriPlaneNet: EG3D 反演编码器

TriPlaneNet: An Encoder for EG3D Inversion

We present the first real-time method to capture the full global 3D skeletal
pose of a human in a stable, temporally consistent manner using a single RGB
camera. Our method combines a new convolutional neural network (CNN) based pose
regressor with kinematic skeleton fitting. Our novel fully-convolutional pose
formulation regresses 2D and 3D joint positions jointly in real time and does
not require tightly cropped input frames. A real-time kinematic skeleton
fitting method uses the CNN output to yield temporally stable 3D global pose
reconstructions on the basis of a coherent kinematic skeleton. This makes our
approach the first monocular RGB method usable in real-time applications such
as 3D character control---thus far, the only monocular methods for such
applications employed specialized RGB-D cameras. Our method's accuracy is
quantitatively on par with the best offline 3D monocular RGB pose estimation
methods. Our results are qualitatively comparable to, and sometimes better
than, results from monocular RGB-D approaches, such as the Kinect. However, we
show that our approach is more broadly applicable than RGB-D solutions, i.e. it
works for outdoor scenes, community videos, and low quality commodity RGB
cameras.

采用单一 RGB 相机，我们提出了第一个实时方法来稳定地连续捕捉人类的全局三维骨架姿态，并结合卷积神经网络和运动学骨骼拟合，成功实现了单目 RGB 方法，避免了使用专门的 RGB-D 相机，同时具有优越的性能和实用价值。