The tasks of designing messenger RNAs and non-coding RNAs are discrete
optimization problems, and several versions of these problems are NP-hard. As
an alternative to commonly used local search methods, we formulate these
problems as continuous optimization and develop a general framework for this
optimization based on a new concept of "expected partition function". The basic
idea is to start with a distribution over all possible candidate sequences, and
extend the objective function from a sequence to a distribution. We then use
gradient descent-based optimization methods to improve the extended objective
function, and the distribution will gradually shrink towards a one-hot sequence
(i.e., a single sequence). We consider two important case studies within this
framework, the mRNA design problem optimizing for partition function (i.e.,
ensemble free energy) and the non-coding RNA design problem optimizing for
conditional (i.e., Boltzmann) probability. In both cases, our approach
demonstrate promising preliminary results. We make our code available at
this https URL

设计信使 RNA 和非编码 RNA 是离散优化问题，我们提出一种基于 “期望分割函数” 的连续优化框架，通过梯度下降优化方法逐渐缩小分布，以解决这些问题。在 mRNA 设计和非编码 RNA 设计两个重要案例中，我们的方法取得了有希望的初步结果。

利用预期分区函数和连续优化设计信使和非编码 RNA

Messenger and Non-Coding RNA Design via Expected Partition Function and  Continuous Optimization

Large language models (LLMs) have exhibited an emergent in-context learning
(ICL) ability. However, the ICL models that can solve ordinary cases are hardly
extended to solve more complex tasks by processing the demonstration examples
once. This single-turn ICL is incoordinate with the decision making process of
humans by learning from analogy. In this paper, we propose an effective and
efficient two-stage framework to boost ICL in LLMs by exploiting a dual form
between Transformer attention and gradient descent-based optimization.
Concretely, we divide the ICL process into "Deep-Thinking" and inference
stages. The "Deep-Thinking" stage performs iterative forward optimization of
demonstrations, which is expected to boost the reasoning abilities of LLMs at
test time by "thinking" demonstrations multiple times. It produces accumulated
meta-gradients by manipulating the Key-Value matrices in the self-attention
modules of the Transformer. Then, the inference stage only takes the test query
as input without concatenating demonstrations and applies the learned
meta-gradients through attention for output prediction. In this way,
demonstrations are not required during the inference stage since they are
already learned and stored in the definitive meta-gradients. LLMs can be
effectively and efficiently adapted to downstream tasks. Extensive experiments
on ten classification and multiple-choice datasets show that our method
achieves substantially better performance than standard ICL in terms of both
accuracy and efficiency.

本研究提出了一种有效的、高效的两阶段方法来增强大型语言模型中的上下文学习，该方法利用 Transformer 注意力和梯度下降优化之间的双重形式，将上下文学习过程分为 'Thinking' 和推理阶段，通过递归前向优化演示来增强 LLLs 的推理能力，并且将得到的元梯度通过注意力应用于最终的输出预测中，从而有效地、高效地适应下游任务。