We study the use of parametric building information modeling (BIM) to
automatically generate training data for artificial neural networks (ANNs) to
recognize building objects in photos. Teaching artificial intelligence (AI)
machines to detect building objects in images is the foundation toward
AI-assisted semantic 3D reconstruction of existing buildings. However, there
exists the challenge of acquiring training data which is typically
human-annotated, that is, unless a computer machine can generate high-quality
data to train itself for a certain task. In that vein, we trained ANNs solely
on realistic computer-generated images of 3D BIM models which were
parametrically and automatically generated using the BIMGenE program. The ANN
training result demonstrated generalizability and good semantic segmentation on
a test case as well as arbitrary photos of buildings that are outside the range
of the training data, which is significant for the future of training AI with
generated data for solving real-world architectural problems.

本研究研究了使用参数化建筑信息模型（BIM）自动生成人工神经网络（ANN）的培训数据，以识别照片中的建筑物对象。

使用参数生成的虚拟建筑进行深度学习，实现现实世界中物体识别

Deep Learning from Parametrically Generated Virtual Buildings for Real-World Object Recognition

While 3D reconstruction is a well-established and widely explored research
topic, semantic 3D reconstruction has only recently witnessed an increasing
share of attention from the Computer Vision community. Semantic annotations
allow in fact to enforce strong class-dependent priors, as planarity for ground
and walls, which can be exploited to refine the reconstruction often resulting
in non-trivial performance improvements. State-of-the art methods propose
volumetric approaches to fuse RGB image data with semantic labels; even if
successful, they do not scale well and fail to output high resolution meshes.
In this paper we propose a novel method to refine both the geometry and the
semantic labeling of a given mesh. We refine the mesh geometry by applying a
variational method that optimizes a composite energy made of a state-of-the-art
pairwise photo-metric term and a single-view term that models the semantic
consistency between the labels of the 3D mesh and those of the segmented
images. We also update the semantic labeling through a novel Markov Random
Field (MRF) formulation that, together with the classical data and smoothness
terms, takes into account class-specific priors estimated directly from the
annotated mesh. This is in contrast to state-of-the-art methods that are
typically based on handcrafted or learned priors. We are the first, jointly
with the very recent and seminal work of [M. Blaha et al arXiv:1706.08336,
2017], to propose the use of semantics inside a mesh refinement framework.
Differently from [M. Blaha et al arXiv:1706.08336, 2017], which adopts a more
classical pairwise comparison to estimate the flow of the mesh, we apply a
single-view comparison between the semantically annotated image and the current
3D mesh labels; this improves the robustness in case of noisy segmentations.

本文提出了一种新颖的方法，通过应用变分方法和 MRF 来细化给定网格的几何和语义标注，并针对通过半监督学习直接估计类特定先验的方法进行了改进。