We present a conditional estimation (CEST) framework to learn 3D facial
parameters from 2D single-view images by self-supervised training from videos.
CEST is based on the process of analysis by synthesis, where the 3D facial
parameters (shape, reflectance, viewpoint, and illumination) are estimated from
the face image, and then recombined to reconstruct the 2D face image. In order
to learn semantically meaningful 3D facial parameters without explicit access
to their labels, CEST couples the estimation of different 3D facial parameters
by taking their statistical dependency into account. Specifically, the
estimation of any 3D facial parameter is not only conditioned on the given
image, but also on the facial parameters that have already been derived.
Moreover, the reflectance symmetry and consistency among the video frames are
adopted to improve the disentanglement of facial parameters. Together with a
novel strategy for incorporating the reflectance symmetry and consistency, CEST
can be efficiently trained with in-the-wild video clips. Both qualitative and
quantitative experiments demonstrate the effectiveness of CEST.

本研究提出了一种条件评估框架（CEST），通过对视频进行自监督训练，从 2D 单视图图像中学习 3D 面部参数。该框架可高效地利用面部参数的统计依赖性进行评估，采用了反射对称性和一致性等方法，以提高面部参数的分离效果，并通过定量和定性实验证明了其有效性。

基于条件估计的自监督三维人脸重建

Self-Supervised 3D Face Reconstruction via Conditional Estimation

Video sequences contain rich dynamic patterns, such as dynamic texture
patterns that exhibit stationarity in the temporal domain, and action patterns
that are non-stationary in either spatial or temporal domain. We show that an
energy-based spatial-temporal generative ConvNet can be used to model and
synthesize dynamic patterns. The model defines a probability distribution on
the video sequence, and the log probability is defined by a spatial-temporal
ConvNet that consists of multiple layers of spatial-temporal filters to capture
spatial-temporal patterns of different scales. The model can be learned from
the training video sequences by an "analysis by synthesis" learning algorithm
that iterates the following two steps. Step 1 synthesizes video sequences from
the currently learned model. Step 2 then updates the model parameters based on
the difference between the synthesized video sequences and the observed
training sequences. We show that the learning algorithm can synthesize
realistic dynamic patterns. We also show that it is possible to learn the model
from incomplete training sequences with either occluded pixels or missing
frames, so that model learning and pattern completion can be accomplished
simultaneously.

使用能量基础的时空生成 ConvNet 建立一个概率分布模型，可用于学习和合成存在于视频序列中的动态纹理和运动模式，该模型可从不完整的训练序列中学习和完成动态模式，并具有较好的生成能力。

学习基于能量的时空生成性卷积神经网络以获取动态模式

Learning Energy-based Spatial-Temporal Generative ConvNets for Dynamic  Patterns

In this paper, we present a novel and efficient architecture for addressing
computer vision problems that use `Analysis by Synthesis'. Analysis by
synthesis involves the minimization of the reconstruction error which is
typically a non-convex function of the latent target variables.
State-of-the-art methods adopt a hybrid scheme where discriminatively trained
predictors like Random Forests or Convolutional Neural Networks are used to
initialize local search algorithms. While these methods have been shown to
produce promising results, they often get stuck in local optima. Our method
goes beyond the conventional hybrid architecture by not only proposing multiple
accurate initial solutions but by also defining a navigational structure over
the solution space that can be used for extremely efficient gradient-free local
search. We demonstrate the efficacy of our approach on the challenging problem
of RGB Camera Relocalization. To make the RGB camera relocalization problem
particularly challenging, we introduce a new dataset of 3D environments which
are significantly larger than those found in other publicly-available datasets.
Our experiments reveal that the proposed method is able to achieve
state-of-the-art camera relocalization results. We also demonstrate the
generalizability of our approach on Hand Pose Estimation and Image Retrieval
tasks.

本文提出一种新颖高效的架构，用于解决使用 “分析与合成” 方法的计算机视觉问题，并在 RGB 相机重定位、手姿估计和图像检索任务上展示了其有效性。