Offline model-based optimization (MBO) aims to maximize a black-box objective
function using only an offline dataset of designs and scores. A prevalent
approach involves training a conditional generative model on existing designs
and their associated scores, followed by the generation of new designs
conditioned on higher target scores. However, these newly generated designs
often underperform due to the lack of high-scoring training data. To address
this challenge, we introduce a novel method, Design Editing for Offline
Model-based Optimization (DEMO), which consists of two phases. In the first
phase, termed pseudo-target distribution generation, we apply gradient ascent
on the offline dataset using a trained surrogate model, producing a synthetic
dataset where the predicted scores serve as new labels. A conditional diffusion
model is subsequently trained on this synthetic dataset to capture a
pseudo-target distribution, which enhances the accuracy of the conditional
diffusion model in generating higher-scoring designs. Nevertheless, the
pseudo-target distribution is susceptible to noise stemming from inaccuracies
in the surrogate model, consequently predisposing the conditional diffusion
model to generate suboptimal designs. We hence propose the second phase,
existing design editing, to directly incorporate the high-scoring features from
the offline dataset into design generation. In this phase, top designs from the
offline dataset are edited by introducing noise, which are subsequently refined
using the conditional diffusion model to produce high-scoring designs. Overall,
high-scoring designs begin with inheriting high-scoring features from the
second phase and are further refined with a more accurate conditional diffusion
model in the first phase. Empirical evaluations on 7 offline MBO tasks show
that DEMO outperforms various baseline methods.

离线基于模型的优化（MBO）旨在最大化黑箱目标函数，仅使用设计和分数的离线数据集。我们引入一种名为离线模型优化设计编辑（DEMO）的新方法，通过在离线数据集上应用训练好的代理模型的梯度上升来产生一个合成数据集，用于训练条件扩散模型以生成高分设计，并通过直接将离线数据集中的高分特征加入到设计生成过程中进行了改进。实证评估表明，DEMO 在 7 个离线 MBO 任务中优于各种基准方法。

离线基于模型优化的设计编辑

Design Editing for Offline Model-based Optimization

In this work we theoretically show that conservative objective models (COMs)
for offline model-based optimisation (MBO) are a special kind of contrastive
divergence-based energy model, one where the energy function represents both
the unconditional probability of the input and the conditional probability of
the reward variable. While the initial formulation only samples modes from its
learned distribution, we propose a simple fix that replaces its gradient ascent
sampler with a Langevin MCMC sampler. This gives rise to a special
probabilistic model where the probability of sampling an input is proportional
to its predicted reward. Lastly, we show that better samples can be obtained if
the model is decoupled so that the unconditional and conditional probabilities
are modelled separately.

本文研究了保守的客观模型（COM）在离线基于模型的优化（MBO）中的应用；提出了用 Langevin MCMC 采样器代替渐近梯度采样器的方法，使得采样概率正比于所预测的奖励；进一步表明，通过将无条件概率和条件概率分别建模，可以获得更好的采样效果。

保守目标模型是一种特殊的对比离差能量模型

Conservative objective models are a special kind of contrastive  divergence-based energy model

In the research area of offline model-based optimization, novel and promising
methods are frequently developed. However, implementing such methods in
real-world industrial systems such as production lines for process control is
oftentimes a frustrating process. In this work, we address two important
problems to extend the current success of offline model-based optimization to
industrial process control problems: 1) how to learn a reliable dynamics model
from offline data for industrial processes? 2) how to learn a reliable but not
over-conservative control policy from offline data by utilizing existing
model-based optimization algorithms? Specifically, we propose a dynamics model
based on ensemble of conditional generative adversarial networks to achieve
accurate reward calculation in industrial scenarios. Furthermore, we propose an
epistemic-uncertainty-penalized reward evaluation function which can
effectively avoid giving over-estimated rewards to out-of-distribution inputs
during the learning/searching of the optimal control policy. We provide
extensive experiments with the proposed method on two representative cases (a
discrete control case and a continuous control case), showing that our method
compares favorably to several baselines in offline policy learning for
industrial process control.

利用条件生成对抗网络集成的动态模型和基于信息不确定性的奖励评估函数，实现离线建模优化在工业过程控制中的精确应用。