The task of novel view synthesis aims to generate unseen perspectives of an
object or scene from a limited set of input images. Nevertheless, synthesizing
novel views from a single image still remains a significant challenge in the
realm of computer vision. Previous approaches tackle this problem by adopting
mesh prediction, multi-plain image construction, or more advanced techniques
such as neural radiance fields. Recently, a pre-trained diffusion model that is
specifically designed for 2D image synthesis has demonstrated its capability in
producing photorealistic novel views, if sufficiently optimized on a 3D
finetuning task. Although the fidelity and generalizability are greatly
improved, training such a powerful diffusion model requires a vast volume of
training data and model parameters, resulting in a notoriously long time and
high computational costs. To tackle this issue, we propose Efficient-3DiM, a
simple but effective framework to learn a single-image novel-view synthesizer.
Motivated by our in-depth analysis of the inference process of diffusion
models, we propose several pragmatic strategies to reduce the training overhead
to a manageable scale, including a crafted timestep sampling strategy, a
superior 3D feature extractor, and an enhanced training scheme. When combined,
our framework is able to reduce the total training time from 10 days to less
than 1 day, significantly accelerating the training process under the same
computational platform (one instance with 8 Nvidia A100 GPUs). Comprehensive
experiments are conducted to demonstrate the efficiency and generalizability of
our proposed method.

我们提出了一种简单而有效的框架 Efficient-3DiM，用于学习单张图像的新视角合成器，通过减小训练开销，包括精心设计的时间步长采样策略、更优质的三维特征提取器和增强的训练方案，实现了将总训练时间从 10 天缩短到不到 1 天，并在相同计算平台上（8 个 Nvidia A100 GPU 实例）加速训练过程。我们进行了全面的实验来证明我们提出方法的高效性和广泛适用性。

Efficient-3DiM：一天学会通用的单图像新视角综合器

Efficient-3DiM: Learning a Generalizable Single-image Novel-view  Synthesizer in One Day

This paper proposes a novel neural network architecture, that we call an
auto-precoder, and a deep-learning based approach that jointly senses the
millimeter wave (mmWave) channel and designs the hybrid precoding matrices with
only a few training pilots. More specifically, the proposed machine learning
model leverages the prior observations of the channel to achieve two
objectives. First, it optimizes the compressive channel sensing vectors based
on the surrounding environment in an unsupervised manner to focus the sensing
power on the most promising spatial directions. This is enabled by a novel
neural network architecture that accounts for the constraints on the RF chains
and models the transmitter/receiver measurement matrices as two complex-valued
convolutional layers. Second, the proposed model learns how to construct the RF
beamforming vectors of the hybrid architectures directly from the projected
channel vector (the received sensing vector). The auto-precoder neural network
that incorporates both the channel sensing and beam prediction is trained
end-to-end as a multi-task classification problem. Thanks to this design
methodology that leverages the prior channel observations and the implicit
awareness about the surrounding environment/user distributions, the proposed
approach significantly reduces the training overhead compared to classical
(non-machine learning) solutions. For example, for a system of 64 transmit and
64 receive antennas, with 3 RF chains at both sides, the proposed solution
needs only 8 or 16 channel training pilots to directly predict the RF
beamforming/combining vectors of the hybrid architectures and achieve
near-optimal achievable rates. This highlights a promising solution for the
channel estimation and hybrid precoding design problem in mmWave and massive
MIMO systems.

本论文提出了一种新型的神经网络架构 (auto-precoder)，基于少量的训练数据，这种深度学习方法可以同时感知毫米波通道 (channel sensing) 和设计混合预编码矩阵 (hybrid precoding matrices)，从而降低了训练开销，实验结果表明，在 64 个发射天线和 64 个接收天线的系统中仅需 8 或 16 个通道训练数据即可实现近乎最优的速率，这是毫米波和大规模 MIMO 系统信道估计和混合预编码设计问题的一种有前途的解决方案。

毫米波大规模 MIMO 系统直接混合预编码的深度学习

Deep Learning for Direct Hybrid Precoding in Millimeter Wave Massive  MIMO Systems

Supporting high mobility in millimeter wave (mmWave) systems enables a wide
range of important applications such as vehicular communications and wireless
virtual/augmented reality. Realizing this in practice, though, requires
overcoming several challenges. First, the use of narrow beams and the
sensitivity of mmWave signals to blockage greatly impact the coverage and
reliability of highly-mobile links. Second, highly-mobile users in dense mmWave
deployments need to frequently hand-off between base stations (BSs), which is
associated with critical control and latency overhead. Further, identifying the
optimal beamforming vectors in large antenna array mmWave systems requires
considerable training overhead, which significantly affects the efficiency of
these mobile systems. In this paper, a novel integrated machine learning and
coordinated beamforming solution is developed to overcome these challenges and
enable highly-mobile mmWave applications. In the proposed solution, a number of
distributed yet coordinating BSs simultaneously serve a mobile user. This user
ideally needs to transmit only one uplink training pilot sequence that will be
jointly received at the coordinating BSs using omni or quasi-omni beam
patterns. These received signals draw a defining signature not only for the
user location, but also for its interaction with the surrounding environment.
The developed solution then leverages a deep learning model that learns how to
use these signatures to predict the beamforming vectors at the BSs. This
renders a comprehensive solution that supports highly-mobile mmWave
applications with reliable coverage, low latency, and negligible training
overhead. Simulation results show that the proposed deep-learning coordinated
beamforming strategy approaches the achievable rate of the genie-aided solution
that knows the optimal beamforming vectors with no training overhead.

本文提出一种整合了机器学习和协调波束成形技术的解决方案，以克服毫米波系统中窄波束的使用以及高度移动用户在基站之间切换等挑战，并支持高度移动的毫米波应用，同时实现可靠的覆盖、低延迟和可忽略的训练开销。

高移动毫米波系统的深度学习协同波束成形

Deep Learning Coordinated Beamforming for Highly-Mobile Millimeter Wave  Systems

In high mobility applications of millimeter wave (mmWave) communications,
e.g., vehicle-to-everything communication and next-generation cellular
communication, frequent link configuration can be a source of significant
overhead. We use the sub-6 GHz channel covariance as an out-of-band side
information for mmWave link configuration. Assuming: (i) a fully digital
architecture at sub-6 GHz; and (ii) a hybrid analog-digital architecture at
mmWave, we propose an out-of-band covariance translation approach and an
out-of-band aided compressed covariance estimation approach. For covariance
translation, we estimate the parameters of sub-6 GHz covariance and use them in
theoretical expressions of covariance matrices to predict the mmWave
covariance. For out-of-band aided covariance estimation, we use weighted sparse
signal recovery to incorporate out-of-band information in compressed covariance
estimation. The out-of-band covariance translation eliminates the in-band
training completely, whereas out-of-band aided covariance estimation relies on
in-band as well as out-of-band training. We also analyze the loss in the
signal-to-noise ratio due to an imperfect estimate of the covariance. The
simulation results show that the proposed covariance estimation strategies can
reduce the training overhead compared to the in-band only covariance
estimation.

本文提出一种利用子 6 GHz 信道协方差作为 mmWave 链路配置的外带侧面信息的方法，可以降低训练开销。