Computer graphics can not only generate synthetic images and ground truth but
it also offers the possibility of constructing virtual worlds in which: (i) an
agent can perceive, navigate, and take actions guided by AI algorithms, (ii)
properties of the worlds can be modified (e.g., material and reflectance),
(iii) physical simulations can be performed, and (iv) algorithms can be learnt
and evaluated. But creating realistic virtual worlds is not easy. The game
industry, however, has spent a lot of effort creating 3D worlds, which a player
can interact with. So researchers can build on these resources to create
virtual worlds, provided we can access and modify the internal data structures
of the games. To enable this we created an open-source plugin UnrealCV
(this http URL) for a popular game engine Unreal Engine 4 (UE4). We
show two applications: (i) a proof of concept image dataset, and (ii) linking
Caffe with the virtual world to test deep network algorithms.

介绍了利用计算机图形学技术创建虚拟世界的方法和困难，详细介绍了基于游戏资源的虚拟世界构建方法和拓展工具 UnrealCV，并给出了两个应用实例：基于虚拟世界的证明性图像数据集和测试深度学习算法。