In this paper, we introduce a method for unifying language, action, and state
information in a shared embedding space to facilitate a range of downstream
tasks in robot learning. Our method, Contrastive Language, Action, and State
Pre-training (CLASP), extends the CLIP formulation by incorporating
distributional learning, capturing the inherent complexities and one-to-many
relationships in behaviour-text alignment. By employing distributional outputs
for both text and behaviour encoders, our model effectively associates diverse
textual commands with a single behaviour and vice-versa. We demonstrate the
utility of our method for the following downstream tasks: zero-shot
text-behaviour retrieval, captioning unseen robot behaviours, and learning a
behaviour prior for language-conditioned reinforcement learning. Our
distributional encoders exhibit superior retrieval and captioning performance
on unseen datasets, and the ability to generate meaningful exploratory
behaviours from textual commands, capturing the intricate relationships between
language, action, and state. This work represents an initial step towards
developing a unified pre-trained model for robotics, with the potential to
generalise to a broad range of downstream tasks.

本文介绍了一种名为 Contrastive Language, Action, and State Pre-training (CLASP) 的方法，采用分布式输出使文本命令和行为单元之间的对齐变得更加准确，从而帮助了解决机器人学习中的相关问题。该模型在对未知数据集进行检索和图像描述生成等任务中表现出了优秀的性能。

对比语言、动作和状态预训练用于机器人学习

Contrastive Language, Action, and State Pre-training for Robot Learning

In this paper we study the approximate learnability of valuations commonly
used throughout economics and game theory for the quantitative encoding of
agent preferences. We provide upper and lower bounds regarding the learnability
of important subclasses of valuation functions that express
no-complementarities. Our main results concern their approximate learnability
in the distributional learning (PAC-style) setting. We provide nearly tight
lower and upper bounds of $\tilde{\Theta}(n^{1/2})$ on the approximation factor
for learning XOS and subadditive valuations, both widely studied superclasses
of submodular valuations. Interestingly, we show that the
$\tilde{\Omega}(n^{1/2})$ lower bound can be circumvented for XOS functions of
polynomial complexity; we provide an algorithm for learning the class of XOS
valuations with a representation of polynomial size achieving an $O(n^{\eps})$
approximation factor in time $O(n^{1/\eps})$ for any $\eps > 0$. This
highlights the importance of considering the complexity of the target function
for polynomial time learning. We also provide new learning results for
interesting subclasses of submodular functions.
Our upper bounds for distributional learning leverage novel structural
results for all these valuation classes. We show that many of these results
provide new learnability results in the Goemans et al. model (SODA 2009) of
approximate learning everywhere via value queries.
We also introduce a new model that is more realistic in economic settings, in
which the learner can set prices and observe purchase decisions at these prices
rather than observing the valuation function directly. In this model, most of
our upper bounds continue to hold despite the fact that the learner receives
less information (both for learning in the distributional setting and with
value queries), while our lower bounds naturally extend.

本文研究了经济学和博弈论中常用于量化代理人偏好的估值函数的近似可学习性。我们提供了关于表达无补充性的重要估值函数子类的可学习性的上下界。我们的主要结果涉及在分布式学习（PAC 样式）设置中他们的近似可学习性。此外，我们还为一些子模函数提供了新的学习结果，同时介绍了一种更现实的经济设置下的学习模型。