Neural rendering is a new image and video generation method based on deep
learning. It combines the deep learning model with the physical knowledge of
computer graphics, to obtain a controllable and realistic scene model, and
realize the control of scene attributes such as lighting, camera parameters,
posture and so on. On the one hand, neural rendering can not only make full use
of the advantages of deep learning to accelerate the traditional forward
rendering process, but also provide new solutions for specific tasks such as
inverse rendering and 3D reconstruction. On the other hand, the design of
innovative hardware structures that adapt to the neural rendering pipeline
breaks through the parallel computing and power consumption bottleneck of
existing graphics processors, which is expected to provide important support
for future key areas such as virtual and augmented reality, film and television
creation and digital entertainment, artificial intelligence and the metaverse.
In this paper, we review the technical connotation, main challenges, and
research progress of neural rendering. On this basis, we analyze the common
requirements of neural rendering pipeline for hardware acceleration and the
characteristics of the current hardware acceleration architecture, and then
discuss the design challenges of neural rendering processor architecture.
Finally, the future development trend of neural rendering processor
architecture is prospected.

神经渲染是一种基于深度学习的新型图像和视频生成方法，将深度学习模型与计算机图形学的物理知识相结合，以获得可控和逼真的场景模型，并实现对光照、相机参数、姿态等场景属性的控制。本文回顾了神经渲染的技术内涵、主要挑战和研究进展，在此基础上分析了神经渲染流程对硬件加速的常见需求和当前硬件加速架构的特点，并讨论了神经渲染处理器架构的设计挑战，最后展望了神经渲染处理器架构的未来发展趋势。