Large language models generate code one token at a time. Their autoregressive
generation process lacks the feedback of observing the program's output.
Training LLMs to suggest edits directly can be challenging due to the scarcity
of rich edit data. To address these problems, we propose neural diffusion
models that operate on syntax trees of any context-free grammar. Similar to
image diffusion models, our method also inverts ``noise'' applied to syntax
trees. Rather than generating code sequentially, we iteratively edit it while
preserving syntactic validity, which makes it easy to combine this neural model
with search. We apply our approach to inverse graphics tasks, where our model
learns to convert images into programs that produce those images. Combined with
search, our model is able to write graphics programs, see the execution result,
and debug them to meet the required specifications. We additionally show how
our system can write graphics programs for hand-drawn sketches.

大型语言模型通过对语法树进行迭代编辑，结合搜索以满足规定的规范，将图像转化为程序，并能够输出执行结果进行调试。

语法树扩散在程序合成中的应用

Diffusion On Syntax Trees For Program Synthesis

Diffusion models have shown remarkable performance on many generative tasks.
Despite recent success, most diffusion models are restricted in that they only
allow linear transformation of the data distribution. In contrast, broader
family of transformations can potentially help train generative distributions
more efficiently, simplifying the reverse process and closing the gap between
the true negative log-likelihood and the variational approximation. In this
paper, we present Neural Diffusion Models (NDMs), a generalization of
conventional diffusion models that enables defining and learning time-dependent
non-linear transformations of data. We show how to optimise NDMs using a
variational bound in a simulation-free setting. Moreover, we derive a
time-continuous formulation of NDMs, which allows fast and reliable inference
using off-the-shelf numerical ODE and SDE solvers. Finally, we demonstrate the
utility of NDMs with learnable transformations through experiments on standard
image generation benchmarks, including CIFAR-10, downsampled versions of
ImageNet and CelebA-HQ. NDMs outperform conventional diffusion models in terms
of likelihood and produce high-quality samples.

传播模型是生成任务中表现出色的模型，然而大部分传播模型仅允许对数据分布进行线性变换，相比之下，更广泛的转换可能有助于更高效地训练生成分布并消除真实负对数似然和变分近似之间的差距。在本文中，我们介绍了神经传播模型 (NDMs)，这是传统传播模型的泛化，可以定义和学习数据的时间相关的非线性转换。我们展示了如何在无需模拟的情况下使用变分上界来优化 NDMs。此外，我们导出了 NDMs 的时间连续形式，可以使用现成的数值 ODE 和 SDE 求解器进行快速可靠的推断。最后，我们通过在标准图像生成基准测试上的实验，包括 CIFAR-10、ImageNet 的降采样版本和 CelebA-HQ，展示了可学习转换的 NDMs 的实用性。NDMs 在似然度和生成高质量样本方面优于传统传播模型。

神经扩散模型

Neural Diffusion Models

Generative models are becoming popular for the synthesis of medical images.
Recently, neural diffusion models have demonstrated the potential to generate
photo-realistic images of objects. However, their potential to generate medical
images is not explored yet. In this work, we explore the possibilities of
synthesis of medical images using neural diffusion models. First, we use a
pre-trained DALLE2 model to generate lungs X-Ray and CT images from an input
text prompt. Second, we train a stable diffusion model with 3165 X-Ray images
and generate synthetic images. We evaluate the synthetic image data through a
qualitative analysis where two independent radiologists label randomly chosen
samples from the generated data as real, fake, or unsure. Results demonstrate
that images generated with the diffusion model can translate characteristics
that are otherwise very specific to certain medical conditions in chest X-Ray
or CT images. Careful tuning of the model can be very promising. To the best of
our knowledge, this is the first attempt to generate lungs X-Ray and CT images
using neural diffusion models. This work aims to introduce a new dimension in
artificial intelligence for medical imaging. Given that this is a new topic,
the paper will serve as an introduction and motivation for the research
community to explore the potential of diffusion models for medical image
synthesis. We have released the synthetic images on
this https URL.

本文探讨使用神经扩散模型合成医学图像的可能性，结果表明扩散模型生成的图像可以翻译一些胸部 X 射线或 CT 图像中特定医学情况的特征，这是一项新的人工智能医学图像研究，展示了扩散模型在医学图像合成领域的潜力。