Many protein design applications, such as binder or enzyme design, require
scaffolding a structural motif with high precision. Generative modelling
paradigms based on denoising diffusion processes emerged as a leading candidate
to address this motif scaffolding problem and have shown early experimental
success in some cases. In the diffusion paradigm, motif scaffolding is treated
as a conditional generation task, and several conditional generation protocols
were proposed or imported from the Computer Vision literature. However, most of
these protocols are motivated heuristically, e.g. via analogies to Langevin
dynamics, and lack a unifying framework, obscuring connections between the
different approaches. In this work, we unify conditional training and
conditional sampling procedures under one common framework based on the
mathematically well-understood Doob's h-transform. This new perspective allows
us to draw connections between existing methods and propose a new variation on
existing conditional training protocols. We illustrate the effectiveness of
this new protocol in both, image outpainting and motif scaffolding and find
that it outperforms standard methods.

通过统一条件训练和条件采样程序，本文基于数学上理解的 Doob 的 h 转换方法提出了一个新的视角，揭示了现有方法之间的联系，并提出了一种新的改进方法，通过在图像外延和结构基元搭建方面的实验证明了其有效性。

一种用于条件扩散建模的框架及其在蛋白设计中的应用

A framework for conditional diffusion modelling with applications in  motif scaffolding for protein design

Language models (LMs) are pretrained to imitate internet text, including
content that would violate human preferences if generated by an LM: falsehoods,
offensive comments, personally identifiable information, low-quality or buggy
code, and more. Here, we explore alternative objectives for pretraining LMs in
a way that also guides them to generate text aligned with human preferences. We
benchmark five objectives for pretraining with human feedback across three
tasks and study how they affect the trade-off between alignment and
capabilities of pretrained LMs. We find a Pareto-optimal and simple approach
among those we explored: conditional training, or learning distribution over
tokens conditional on their human preference scores given by a reward model.
Conditional training reduces the rate of undesirable content by up to an order
of magnitude, both when generating without a prompt and with an
adversarially-chosen prompt. Moreover, conditional training maintains the
downstream task performance of standard LM pretraining, both before and after
task-specific finetuning. Pretraining with human feedback results in much
better preference satisfaction than standard LM pretraining followed by
finetuning with feedback, i.e., learning and then unlearning undesirable
behavior. Our results suggest that we should move beyond imitation learning
when pretraining LMs and incorporate human preferences from the start of
training.

通过在预训练中引入人类的反馈，实现对于语言模型的生成文本的可控和可导向性，减少哪些偏离人类喜好的内容的生成，并且在标准的预训练和任务特定的微调中保持下游任务表现。推荐在训练开始时，就结合人类反馈，不再使用模仿学习的方式预训练语言模型。

使用人类偏好进行语言模型预训练

Pretraining Language Models with Human Preferences

For many large undirected models that arise in real-world applications, exact
maximumlikelihood training is intractable, because it requires computing
marginal distributions of the model. Conditional training is even more
difficult, because the partition function depends not only on the parameters,
but also on the observed input, requiring repeated inference over each training
example. An appealing idea for such models is to independently train a local
undirected classifier over each clique, afterwards combining the learned
weights into a single global model. In this paper, we show that this piecewise
method can be justified as minimizing a new family of upper bounds on the log
partition function. On three natural-language data sets, piecewise training is
more accurate than pseudolikelihood, and often performs comparably to global
training using belief propagation.

本文介绍了一种基于单个团集合的独立训练方法，以在训练大规模无向图模型时提高准确性，并通过对三个自然语言数据集的实验，证明了其比伪似然更准确，并且通常与使用信念传播的全局训练相当。