Discrete structures play an important role in applications like program
language modeling and software engineering. Current approaches to predicting
complex structures typically consider autoregressive models for their
tractability, with some sacrifice in flexibility. Energy-based models (EBMs) on
the other hand offer a more flexible and thus more powerful approach to
modeling such distributions, but require partition function estimation. In this
paper we propose ALOE, a new algorithm for learning conditional and
unconditional EBMs for discrete structured data, where parameter gradients are
estimated using a learned sampler that mimics local search. We show that the
energy function and sampler can be trained efficiently via a new variational
form of power iteration, achieving a better trade-off between flexibility and
tractability. Experimentally, we show that learning local search leads to
significant improvements in challenging application domains. Most notably, we
present an energy model guided fuzzer for software testing that achieves
comparable performance to well engineered fuzzing engines like libfuzzer.

本文提出一种名为 ALOE 的算法，该算法可以学习用于离散结构数据的有条件和无条件能量模型，其参数梯度使用模拟局部搜索的学习取样器进行估计，并且通过一种新的变分幂迭代形式有效地训练能量函数和取样器。实验结果表明，在软件测试等应用领域中，学习局部搜索可以取得显著的改进。

通过辅助变量的局部探索学习离散能量模型

Learning Discrete Energy-based Models via Auxiliary-variable Local  Exploration

Detecting out-of-distribution (OOD) inputs is critical for safely deploying
deep learning models in the real world. Existing approaches for detecting OOD
examples work well when evaluated on benign in-distribution and OOD samples.
However, in this paper, we show that existing detection mechanisms can be
extremely brittle when evaluating on in-distribution and OOD inputs with
minimal adversarial perturbations which don't change their semantics. Formally,
we extensively study the problem of Robust Out-of-Distribution Detection on
common OOD detection approaches, and show that state-of-the-art OOD detectors
can be easily fooled by adding small perturbations to the in-distribution and
OOD inputs. To counteract these threats, we propose an effective algorithm
called ALOE, which performs robust training by exposing the model to both
adversarially crafted inlier and outlier examples. Our method can be flexibly
combined with, and render existing methods robust. On common benchmark
datasets, we show that ALOE substantially improves the robustness of
state-of-the-art OOD detection, with 58.4% AUROC improvement on CIFAR-10 and
46.59% improvement on CIFAR-100.

本文主要研究深度学习模型在检测 in-distribution 和 OOD inputs 时受到极小对抗扰动的影响，并提出了一种名为 ALOE 的有效算法，该算法通过将模型暴露于经过对抗训练的 inlier 和 outlier examples 中的方式，可以灵活地与现有方法相结合，提高现有的 Out-of-distribution detection 方法的鲁棒性，CIFAR-10 和 CIFAR-100 数据集的 AUROC 分别提高了 58.4% 和 46.59%。