Advances in NERFs have allowed for 3D scene reconstructions and novel view
synthesis. Yet, efficiently editing these representations while retaining
photorealism is an emerging challenge. Recent methods face three primary
limitations: they're slow for interactive use, lack precision at object
boundaries, and struggle to ensure multi-view consistency. We introduce IReNe
to address these limitations, enabling swift, near real-time color editing in
NeRF. Leveraging a pre-trained NeRF model and a single training image with
user-applied color edits, IReNe swiftly adjusts network parameters in seconds.
This adjustment allows the model to generate new scene views, accurately
representing the color changes from the training image while also controlling
object boundaries and view-specific effects. Object boundary control is
achieved by integrating a trainable segmentation module into the model. The
process gains efficiency by retraining only the weights of the last network
layer. We observed that neurons in this layer can be classified into those
responsible for view-dependent appearance and those contributing to diffuse
appearance. We introduce an automated classification approach to identify these
neuron types and exclusively fine-tune the weights of the diffuse neurons. This
further accelerates training and ensures consistent color edits across
different views. A thorough validation on a new dataset, with edited object
colors, shows significant quantitative and qualitative advancements over
competitors, accelerating speeds by 5x to 500x.

通过在预先训练的 NERF 模型和单一训练图像中实时调整网络参数，IReNe 能够快速、准确地调整颜色编辑，同时控制物体边界和视图效果，从而实现了在 NERF 中近实时的颜色编辑和物体控制。