Fully unsupervised 3D representation learning has gained attention owing to
its advantages in data collection. A successful approach involves a
viewpoint-aware approach that learns an image distribution based on generative
models (e.g., generative adversarial networks (GANs)) while generating various
view images based on 3D-aware models (e.g., neural radiance fields (NeRFs)).
However, they require images with various views for training, and consequently,
their application to datasets with few or limited viewpoints remains a
challenge. As a complementary approach, an aperture rendering GAN (AR-GAN) that
employs a defocus cue was proposed. However, an AR-GAN is a CNN-based model and
represents a defocus independently from a viewpoint change despite its high
correlation, which is one of the reasons for its performance. As an alternative
to an AR-GAN, we propose an aperture rendering NeRF (AR-NeRF), which can
utilize viewpoint and defocus cues in a unified manner by representing both
factors in a common ray-tracing framework. Moreover, to learn defocus-aware and
defocus-independent representations in a disentangled manner, we propose
aperture randomized training, for which we learn to generate images while
randomizing the aperture size and latent codes independently. During our
experiments, we applied AR-NeRF to various natural image datasets, including
flower, bird, and face images, the results of which demonstrate the utility of
AR-NeRF for unsupervised learning of the depth and defocus effects.

本文提出了一种基于光圈渲染神经辐射场 (AR-NeRF) 的方法，该方法能够在统一的射线跟踪框架中利用视点和景深线索，以无监督的方式学习深度和景深效果，为少视点数据集提供了潜在的应用。

AR-NeRF: 使用光圈渲染神经辐射场从自然图像中无监督学习深度和景深效应

AR-NeRF: Unsupervised Learning of Depth and Defocus Effects from Natural Images with Aperture Rendering Neural Radiance Fields

Understanding the 3D world from 2D projected natural images is a fundamental
challenge in computer vision and graphics. Recently, an unsupervised learning
approach has garnered considerable attention owing to its advantages in data
collection. However, to mitigate training limitations, typical methods need to
impose assumptions for viewpoint distribution (e.g., a dataset containing
various viewpoint images) or object shape (e.g., symmetric objects). These
assumptions often restrict applications; for instance, the application to
non-rigid objects or images captured from similar viewpoints (e.g., flower or
bird images) remains a challenge. To complement these approaches, we propose
aperture rendering generative adversarial networks (AR-GANs), which equip
aperture rendering on top of GANs, and adopt focus cues to learn the depth and
depth-of-field (DoF) effect of unlabeled natural images. To address the
ambiguities triggered by unsupervised setting (i.e., ambiguities between smooth
texture and out-of-focus blurs, and between foreground and background blurs),
we develop DoF mixture learning, which enables the generator to learn real
image distribution while generating diverse DoF images. In addition, we devise
a center focus prior to guiding the learning direction. In the experiments, we
demonstrate the effectiveness of AR-GANs in various datasets, such as flower,
bird, and face images, demonstrate their portability by incorporating them into
other 3D representation learning GANs, and validate their applicability in
shallow DoF rendering.

本文引入了孔径渲染生成对抗网络（AR-GANs），使用焦点提示学习深度及深度场效应，并且通过深度场混合学习减少了无监督学习的歧义，实现了多样化的图像生成。

利用光圈渲染生成对抗网络从自然图像中无监督学习深度和虚化效果

Unsupervised Learning of Depth and Depth-of-Field Effect from Natural  Images with Aperture Rendering Generative Adversarial Networks

We present a novel method to train machine learning algorithms to estimate
scene depths from a single image, by using the information provided by a
camera's aperture as supervision. Prior works use a depth sensor's outputs or
images of the same scene from alternate viewpoints as supervision, while our
method instead uses images from the same viewpoint taken with a varying camera
aperture. To enable learning algorithms to use aperture effects as supervision,
we introduce two differentiable aperture rendering functions that use the input
image and predicted depths to simulate the depth-of-field effects caused by
real camera apertures. We train a monocular depth estimation network end-to-end
to predict the scene depths that best explain these finite aperture images as
defocus-blurred renderings of the input all-in-focus image.

本文提出了一种新方法，通过使用相机光圈所提供的信息作为监督来训练机器学习算法从单幅图像中估计场景深度。为了使学习算法能够使用光圈效应作为监督，我们引入了两个可微的光圈渲染函数，使用输入图像和预测深度来模拟实际相机光圈引起的景深效应。然后我们通过端到端地训练单目深度估计网络来预测场景深度，使其最好地解释这些有限光圈图像作为输入全焦距图像的虚焦模糊渲染。