We propose Text2Scene, a method to automatically create realistic textures
for virtual scenes composed of multiple objects. Guided by a reference image
and text descriptions, our pipeline adds detailed texture on labeled 3D
geometries in the room such that the generated colors respect the hierarchical
structure or semantic parts that are often composed of similar materials.
Instead of applying flat stylization on the entire scene at a single step, we
obtain weak semantic cues from geometric segmentation, which are further
clarified by assigning initial colors to segmented parts. Then we add texture
details for individual objects such that their projections on image space
exhibit feature embedding aligned with the embedding of the input. The
decomposition makes the entire pipeline tractable to a moderate amount of
computation resources and memory. As our framework utilizes the existing
resources of image and text embedding, it does not require dedicated datasets
with high-quality textures designed by skillful artists. To the best of our
knowledge, it is the first practical and scalable approach that can create
detailed and realistic textures of the desired style that maintain structural
context for scenes with multiple objects.

通过引用图像和文本描述，我们提出了一种名为 Text2Scene 的方法，可以自动创建由多个物体组成的虚拟场景的逼真纹理，这种方法在房间的标记三维几何体上增加了详细纹理，使生成的颜色尊重经常由类似材料组成的分层结构或语义部分。

Text2Scene：基于文本的室内场景风格化与局部细节

Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Details

In this paper, we propose a weakly-supervised approach for 3D object
detection, which makes it possible to train a strong 3D detector with
position-level annotations (i.e. annotations of object centers). In order to
remedy the information loss from box annotations to centers, our method, namely
Back to Reality (BR), makes use of synthetic 3D shapes to convert the weak
labels into fully-annotated virtual scenes as stronger supervision, and in turn
utilizes the perfect virtual labels to complement and refine the real labels.
Specifically, we first assemble 3D shapes into physically reasonable virtual
scenes according to the coarse scene layout extracted from position-level
annotations. Then we go back to reality by applying a virtual-to-real domain
adaptation method, which refine the weak labels and additionally supervise the
training of detector with the virtual scenes. Furthermore, we propose a more
challenging benckmark for indoor 3D object detection with more diversity in
object sizes to better show the potential of BR. With less than 5% of the
labeling labor, we achieve comparable detection performance with some popular
fully-supervised approaches on the widely used ScanNet dataset. Code is
available at: this https URL

提出了一种基于虚拟场景的弱监督方法用于 3D 物体检测，利用合成的 3D 形状将弱标签转换成更强的虚拟场景，并再次利用虚拟标签去改善真实标签和更好地训练检测器，同时提出了更具挑战性的室内 3D 物体检测基准测试。