This paper proposes a new 3D molecule generation framework, called GOAT, for
fast and effective 3D molecule generation based on the flow-matching optimal
transport objective. Specifically, we formulate a geometric transport formula
for measuring the cost of mapping multi-modal features (e.g., continuous atom
coordinates and categorical atom types) between a base distribution and a
target data distribution. Our formula is solved within a unified, equivalent,
and smooth representation space. This is achieved by transforming the
multi-modal features into a continuous latent space with equivalent networks.
In addition, we find that identifying optimal distributional coupling is
necessary for fast and effective transport between any two distributions. We
further propose a flow refinement and purification mechanism for optimal
coupling identification. By doing so, GOAT can turn arbitrary distribution
couplings into new deterministic couplings, leading to a unified optimal
transport path for fast 3D molecule generation. The purification filters the
subpar molecules to ensure the ultimate generation performance. We
theoretically prove the proposed method indeed reduced the transport cost.
Finally, extensive experiments show that GOAT enjoys the efficiency of solving
geometric optimal transport, leading to a double speedup compared to the
sub-optimal method while achieving the best generation quality regarding
validity, uniqueness, and novelty.

该论文提出了一种名为 GOAT 的新的 3D 分子生成框架，基于流匹配最优传输目标，实现快速有效的 3D 分子生成。

快速三维分子生成基于统一的几何最优输运

Fast 3D Molecule Generation via Unified Geometric Optimal Transport

Offline goal-conditioned RL (GCRL) offers a way to train general-purpose
agents from fully offline datasets. In addition to being conservative within
the dataset, the generalization ability to achieve unseen goals is another
fundamental challenge for offline GCRL. However, to the best of our knowledge,
this problem has not been well studied yet. In this paper, we study
out-of-distribution (OOD) generalization of offline GCRL both theoretically and
empirically to identify factors that are important. In a number of experiments,
we observe that weighted imitation learning enjoys better generalization than
pessimism-based offline RL method. Based on this insight, we derive a theory
for OOD generalization, which characterizes several important design choices.
We then propose a new offline GCRL method, Generalizable Offline
goAl-condiTioned RL (GOAT), by combining the findings from our theoretical and
empirical studies. On a new benchmark containing 9 independent identically
distributed (IID) tasks and 17 OOD tasks, GOAT outperforms current
state-of-the-art methods by a large margin.

本文研究了离线目标导向增强学习算法的越界泛化问题，提出了一种基于加权模仿学习的离线学习算法（GOAT），在 9 项独立同分布任务和 17 项越界任务测试中显著优于现有的算法。

离线目标条件强化学习的未见目标泛化必要因素是什么？

What is Essential for Unseen Goal Generalization of Offline  Goal-conditioned RL?

We introduce Goat, a fine-tuned LLaMA model that significantly outperforms
GPT-4 on a range of arithmetic tasks. Fine-tuned on a synthetically generated
dataset, Goat achieves state-of-the-art performance on BIG-bench arithmetic
sub-task. In particular, the zero-shot Goat-7B matches or even surpasses the
accuracy achieved by the few-shot PaLM-540B. Surprisingly, Goat can achieve
near-perfect accuracy on large-number addition and subtraction through
supervised fine-tuning only, which is almost impossible with previous
pretrained language models, such as Bloom, OPT, GPT-NeoX, etc. We attribute
Goat's exceptional performance to LLaMA's consistent tokenization of numbers.
To tackle more challenging tasks like large-number multiplication and division,
we propose an approach that classifies tasks based on their learnability, and
subsequently decomposes unlearnable tasks, such as multi-digit multiplication
and division, into a series of learnable tasks by leveraging basic arithmetic
principles. We thoroughly examine the performance of our model, offering a
comprehensive evaluation of the effectiveness of our proposed decomposition
steps. Additionally, Goat-7B can be easily trained using LoRA on a 24GB VRAM
GPU, facilitating reproducibility for other researchers. We release our model,
dataset, and the Python script for dataset generation.

Goat，用 LLaMA 模型细调后，在一系列算术任务中显著优于 GPT-4，特别是在大数字加法和减法方面表现出了非常高的准确性。 对于更具挑战性的任务，如大数字乘法和除法，我们提出了一种根据其可学习性分类任务的方法，并通过利用基本算术原则将不可学习的任务（如多位数乘法和除法）分解为一系列可学习的任务。

Goat：Fine-tuned LLaMA 在算术任务上优于 GPT-4

Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks

Graph representation learning (GRL) is a powerful technique for learning
low-dimensional vector representation of high-dimensional and often sparse
graphs. Most studies explore the structure and metadata associated with the
graph using random walks and employ an unsupervised or semi-supervised learning
schemes. Learning in these methods is context-free, resulting in only a single
representation per node. Recently studies have argued on the adequacy of a
single representation and proposed context-sensitive approaches, which are
capable of extracting multiple node representations for different contexts.
This proved to be highly effective in applications such as link prediction and
ranking.
However, most of these methods rely on additional textual features that
require complex and expensive RNNs or CNNs to capture high-level features or
rely on a community detection algorithm to identify multiple contexts of a
node.
In this study we show that in-order to extract high-quality context-sensitive
node representations it is not needed to rely on supplementary node features,
nor to employ computationally heavy and complex models. We propose GOAT, a
context-sensitive algorithm inspired by gossip communication and a mutual
attention mechanism simply over the structure of the graph. We show the
efficacy of GOAT using 6 real-world datasets on link prediction and node
clustering tasks and compare it against 12 popular and state-of-the-art (SOTA)
baselines. GOAT consistently outperforms them and achieves up to 12% and 19%
gain over the best performing methods on link prediction and clustering tasks,
respectively.

本文提出了一种基于 gossip 通信和 mutual attention 机制的 GOAT 算法，无需任何附加文本特征，仅仅通过图的结构就能提取高质量的上下文敏感节点表示。我们使用 6 个真实数据集对 GOAT 进行了比较，并与 12 个流行和最先进的基线进行了比较，发现 GOAT 在链接预测和节点聚类方面的表现均优于其他方法，相对最佳性能方法的提高率达到 12% 和 19%。