We propose Filtering Inversion (FINV), a learning framework and optimization
process that predicts a renderable 3D object representation from one or few
partial views. FINV addresses the challenge of synthesizing novel views of
objects from partial observations, spanning cases where the object is not
entirely in view, is partially occluded, or is only observed from similar
views. To achieve this, FINV learns shape priors by training a 3D generative
model. At inference, given one or more views of a novel real-world object, FINV
first finds a set of latent codes for the object by inverting the generative
model from multiple initial seeds. Maintaining the set of latent codes, FINV
filters and resamples them after receiving each new observation, akin to
particle filtering. The generator is then finetuned for each latent code on the
available views in order to adapt to novel objects. We show that FINV
successfully synthesizes novel views of real-world objects (e.g., chairs,
tables, and cars), even if the generative prior is trained only on synthetic
objects. The ability to address the sim-to-real problem allows FINV to be used
for object categories without real-world datasets. FINV achieves
state-of-the-art performance on multiple real-world datasets, recovers object
shape and texture from partial and sparse views, is robust to occlusion, and is
able to incrementally improve its representation with more observations.

本文提出了 Filtering Inversion（FINV），采用 3D 生成模型和优化过程的学习框架，是从一个或少量的部分视图预测可渲染的 3D 物体表示，成功地解决了从部分观察中合成新的对象视图的挑战，实验表明 FINV 在多个真实世界数据集上实现了最先进的性能，并能够从部分稀疏视图中恢复对象形状和纹理，并且能够随着观察量的增加而逐步改进其表示。

通过筛选反演实现部分视野对象视图合成

Partial-View Object View Synthesis via Filtered Inversion

Robot control policies learned in simulation do not often transfer well to
the real world. Many existing solutions to this sim-to-real problem, such as
the Grounded Action Transformation (GAT) algorithm, seek to correct for or
ground these differences by matching the simulator to the real world. However,
the efficacy of these approaches is limited if they do not explicitly account
for stochasticity in the target environment. In this work, we analyze the
problems associated with grounding a deterministic simulator in a stochastic
real world environment, and we present examples where GAT fails to transfer a
good policy due to stochastic transitions in the target domain. In response, we
introduce the Stochastic Grounded Action Transformation(SGAT) algorithm,which
models this stochasticity when grounding the simulator. We find experimentally
for both simulated and physical target domains that SGAT can find policies that
are robust to stochasticity in the target domain

分析了用 Grounded Action Transformation 算法来解决 sim-to-real 问题的局限性，因为它们没有明确考虑到目标环境中的随机性，提出了 Stochastic Grounded Action Transformation 算法，这种算法在基于模拟的控制器在真实环境中无法成功迁移时有很好的应用。