Differentiable rendering is a technique to connect 3D scenes with
corresponding 2D images. Since it is differentiable, processes during image
formation can be learned. Previous approaches to differentiable rendering focus
on mesh-based representations of 3D scenes, which is inappropriate for medical
applications where volumetric, voxelized models are used to represent anatomy.
We propose a novel Projective Spatial Transformer module that generalizes
spatial transformers to projective geometry, thus enabling differentiable
volume rendering. We demonstrate the usefulness of this architecture on the
example of 2D/3D registration between radiographs and CT scans. Specifically,
we show that our transformer enables end-to-end learning of an image processing
and projection model that approximates an image similarity function that is
convex with respect to the pose parameters, and can thus be optimized
effectively using conventional gradient descent. To the best of our knowledge,
this is the first time that spatial transformers have been described for
projective geometry. The source code will be made public upon publication of
this manuscript and we hope that our developments will benefit related 3D
research applications.

提出了基于 Projective Spatial Transformer module 的可微分体绘制新方法，适用于医学应用。通过这种方法，可以 end-to-end 训练图像处理和投影模型，以近似与姿势参数相关的图像相似度函数，并通过梯度下降进行有效优化。这是空间变换器第一次用于投影几何，代码将在出版后公开。