Inferring programs which generate 2D and 3D shapes is important for reverse
engineering, editing, and more. Training models to perform this task is
complicated because paired (shape, program) data is not readily available for
many domains, making exact supervised learning infeasible. However, it is
possible to get paired data by compromising the accuracy of either the assigned
program labels or the shape distribution. Wake-sleep methods use samples from a
generative model of shape programs to approximate the distribution of real
shapes. In self-training, shapes are passed through a recognition model, which
predicts programs that are treated as "pseudo-labels" for those shapes. Related
to these approaches, we introduce a novel self-training variant unique to
program inference, where program pseudo-labels are paired with their executed
output shapes, avoiding label mismatch at the cost of an approximate shape
distribution. We propose to group these regimes under a single conceptual
framework, where training is performed with maximum likelihood updates sourced
from either Pseudo-Labels or an Approximate Distribution (PLAD). We evaluate
these techniques on multiple 2D and 3D shape program inference domains.
Compared with policy gradient reinforcement learning, we show that PLAD
techniques infer more accurate shape programs and converge significantly
faster. Finally, we propose to combine updates from different PLAD methods
within the training of a single model, and find that this approach outperforms
any individual technique.

本研究提出了一种基于最大似然估计和伪标签或近似分布的培训策略 PLAD，用于训练模型进行程序推断，达到更准确，更简单的结果。

PLAD: 学习使用伪标签和近似分布推断形状程序

PLAD: Learning to Infer Shape Programs with Pseudo-Labels and  Approximate Distributions

The choice of approximate posterior distribution is one of the core problems
in variational inference. Most applications of variational inference employ
simple families of posterior approximations in order to allow for efficient
inference, focusing on mean-field or other simple structured approximations.
This restriction has a significant impact on the quality of inferences made
using variational methods. We introduce a new approach for specifying flexible,
arbitrarily complex and scalable approximate posterior distributions. Our
approximations are distributions constructed through a normalizing flow,
whereby a simple initial density is transformed into a more complex one by
applying a sequence of invertible transformations until a desired level of
complexity is attained. We use this view of normalizing flows to develop
categories of finite and infinitesimal flows and provide a unified view of
approaches for constructing rich posterior approximations. We demonstrate that
the theoretical advantages of having posteriors that better match the true
posterior, combined with the scalability of amortized variational approaches,
provides a clear improvement in performance and applicability of variational
inference.

本论文提出了一种新的变分推断方法，通过应用一系列可逆转变，构造了一种更为复杂的、灵活的、可拓展的近似后验分布，与简单的后验分布相比，提升了推断性能和适用性。

具有归一化流的变分推断

Variational Inference with Normalizing Flows

This paper presents a novel theoretical study of the general problem of
multiple source adaptation using the notion of Renyi divergence. Our results
build on our previous work [12], but significantly broaden the scope of that
work in several directions. We extend previous multiple source loss guarantees
based on distribution weighted combinations to arbitrary target distributions
P, not necessarily mixtures of the source distributions, analyze both known and
unknown target distribution cases, and prove a lower bound. We further extend
our bounds to deal with the case where the learner receives an approximate
distribution for each source instead of the exact one, and show that similar
loss guarantees can be achieved depending on the divergence between the
approximate and true distributions. We also analyze the case where the labeling
functions of the source domains are somewhat different. Finally, we report the
results of experiments with both an artificial data set and a sentiment
analysis task, showing the performance benefits of the distribution weighted
combinations and the quality of our bounds based on the Renyi divergence.

通过 Renyi 离散度的概念，本文对多源适配的一般问题进行了新颖的理论研究，扩大了之前的多源损失保证的范围，并分析了多种情况，包括未知目标分布和标注函数不同的情况，通过实验表明基于 Renyi 离散度的保证的性能优越。