Human vision demonstrates higher robustness than current AI algorithms under
out-of-distribution scenarios. It has been conjectured such robustness benefits
from performing analysis-by-synthesis. Our paper formulates triple vision tasks
in a consistent manner using approximate analysis-by-synthesis by
render-and-compare algorithms on neural features. In this work, we introduce
Neural Textured Deformable Meshes, which involve the object model with
deformable geometry that allows optimization on both camera parameters and
object geometries. The deformable mesh is parameterized as a neural field, and
covered by whole-surface neural texture maps, which are trained to have spatial
discriminability. During inference, we extract the feature map of the test
image and subsequently optimize the 3D pose and shape parameters of our model
using differentiable rendering to best reconstruct the target feature map. We
show that our analysis-by-synthesis is much more robust than conventional
neural networks when evaluated on real-world images and even in challenging
out-of-distribution scenarios, such as occlusion and domain shift. Our
algorithms are competitive with standard algorithms when tested on conventional
performance measures.

本文提出了一种通过近似合成分析的方法来训练神经网络，利用可变形网格模型来优化图像与三维模型之间的关系，从而实现对数据的稳健泛化，本方法在测试集中表现良好，尤其在数据分布不均时表现更优。

基于合成分析的神经纹理可变形网格

Neural Textured Deformable Meshes for Robust Analysis-by-Synthesis

This paper addresses the challenge of novel view synthesis for a human
performer from a very sparse set of camera views. Some recent works have shown
that learning implicit neural representations of 3D scenes achieves remarkable
view synthesis quality given dense input views. However, the representation
learning will be ill-posed if the views are highly sparse. To solve this
ill-posed problem, our key idea is to integrate observations over video frames.
To this end, we propose Neural Body, a new human body representation which
assumes that the learned neural representations at different frames share the
same set of latent codes anchored to a deformable mesh, so that the
observations across frames can be naturally integrated. The deformable mesh
also provides geometric guidance for the network to learn 3D representations
more efficiently. To evaluate our approach, we create a multi-view dataset
named ZJU-MoCap that captures performers with complex motions. Experiments on
ZJU-MoCap show that our approach outperforms prior works by a large margin in
terms of novel view synthesis quality. We also demonstrate the capability of
our approach to reconstruct a moving person from a monocular video on the
People-Snapshot dataset. The code and dataset are available at
this https URL

本文提出了 Neural Body，一种人体新的神经表示方法，通过假设不同帧的学习神经表示共享相同的潜在代码来集成视频帧之间的观察，从而解决了高度稀疏的视图的表征学习问题，并在多视点数据集 ZJU-MoCap 上得到了验证。