We introduce InseRF, a novel method for generative object insertion in the
NeRF reconstructions of 3D scenes. Based on a user-provided textual description
and a 2D bounding box in a reference viewpoint, InseRF generates new objects in
3D scenes. Recently, methods for 3D scene editing have been profoundly
transformed, owing to the use of strong priors of text-to-image diffusion
models in 3D generative modeling. Existing methods are mostly effective in
editing 3D scenes via style and appearance changes or removing existing
objects. Generating new objects, however, remains a challenge for such methods,
which we address in this study. Specifically, we propose grounding the 3D
object insertion to a 2D object insertion in a reference view of the scene. The
2D edit is then lifted to 3D using a single-view object reconstruction method.
The reconstructed object is then inserted into the scene, guided by the priors
of monocular depth estimation methods. We evaluate our method on various 3D
scenes and provide an in-depth analysis of the proposed components. Our
experiments with generative insertion of objects in several 3D scenes indicate
the effectiveness of our method compared to the existing methods. InseRF is
capable of controllable and 3D-consistent object insertion without requiring
explicit 3D information as input. Please visit our project page at
this https URL

InseRF 是一种基于用户提供的文本描述和参考视点中的 2D 边界框，在 3D 场景的 NeRF 重建中生成新对象的新方法。

InseRF: 文本驱动的神经三维场景中物体插入生成

InseRF: Text-Driven Generative Object Insertion in Neural 3D Scenes

While the voxel-based methods have achieved promising results for
multi-person 3D pose estimation from multi-cameras, they suffer from heavy
computation burdens, especially for large scenes. We present Faster VoxelPose
to address the challenge by re-projecting the feature volume to the three
two-dimensional coordinate planes and estimating X, Y, Z coordinates from them
separately. To that end, we first localize each person by a 3D bounding box by
estimating a 2D box and its height based on the volume features projected to
the xy-plane and z-axis, respectively. Then for each person, we estimate
partial joint coordinates from the three coordinate planes separately which are
then fused to obtain the final 3D pose. The method is free from costly 3D-CNNs
and improves the speed of VoxelPose by ten times and meanwhile achieves
competitive accuracy as the state-of-the-art methods, proving its potential in
real-time applications.

本论文提出了 Faster VoxelPose 方法，通过在三个二维坐标平面上重投影特征体积并分别从其中估计 X、Y、Z 坐标，以解决多人三维姿态估计的计算负担过重问题。该方法可以在不需要昂贵的三维卷积神经网络的情况下，使 VoxelPose 的速度提高十倍，同时达到与最先进方法相当的准确性，显示了其在实时应用中的潜力。

更快的 VoxelPose：通过正交投影实时进行三维人体姿态估计

Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic  Projection

We present structured domain randomization (SDR), a variant of domain
randomization (DR) that takes into account the structure and context of the
scene. In contrast to DR, which places objects and distractors randomly
according to a uniform probability distribution, SDR places objects and
distractors randomly according to probability distributions that arise from the
specific problem at hand. In this manner, SDR-generated imagery enables the
neural network to take the context around an object into consideration during
detection. We demonstrate the power of SDR for the problem of 2D bounding box
car detection, achieving competitive results on real data after training only
on synthetic data. On the KITTI easy, moderate, and hard tasks, we show that
SDR outperforms other approaches to generating synthetic data (VKITTI, Sim
200k, or DR), as well as real data collected in a different domain (BDD100K).
Moreover, synthetic SDR data combined with real KITTI data outperforms real
KITTI data alone.

介绍了一种结构化领域随机化方法，将对象和干扰因素随机放置并生成图像以使神经网络考虑对象周围的上下文，可通过使用合成数据来训练在真实数据上获得良好表现的 2D 边界框车辆检测神经网络。