Autoregressive language models have demonstrated a remarkable ability to
extract latent structure from text. The embeddings from large language models
have been shown to capture aspects of the syntax and semantics of language. But
what {\em should} embeddings represent? We connect the autoregressive
prediction objective to the idea of constructing predictive sufficient
statistics to summarize the information contained in a sequence of
observations, and use this connection to identify three settings where the
optimal content of embeddings can be identified: independent identically
distributed data, where the embedding should capture the sufficient statistics
of the data; latent state models, where the embedding should encode the
posterior distribution over states given the data; and discrete hypothesis
spaces, where the embedding should reflect the posterior distribution over
hypotheses given the data. We then conduct empirical probing studies to show
that transformers encode these three kinds of latent generating distributions,
and that they perform well in out-of-distribution cases and without token
memorization in these settings.

通过将自回归预测目标与构建预测充分统计量的思想联系起来，我们确定了三种情况下嵌入的最佳内容：独立同分布数据、潜在状态模型和离散假设空间，并进行实证研究表明 Transformers 编码了这三种潜在生成分布，并在这些情况下表现良好。

嵌入式表示应该嵌入什么？自回归模型表示潜在的生成分布

What Should Embeddings Embed? Autoregressive Models Represent Latent  Generating Distributions

Developing semi-supervised task-oriented dialog (TOD) systems by leveraging
unlabeled dialog data has attracted increasing interests. For semi-supervised
learning of latent state TOD models, variational learning is often used, but
suffers from the annoying high-variance of the gradients propagated through
discrete latent variables and the drawback of indirectly optimizing the target
log-likelihood. Recently, an alternative algorithm, called joint stochastic
approximation (JSA), has emerged for learning discrete latent variable models
with impressive performances. In this paper, we propose to apply JSA to
semi-supervised learning of the latent state TOD models, which is referred to
as JSA-TOD. To our knowledge, JSA-TOD represents the first work in developing
JSA based semi-supervised learning of discrete latent variable conditional
models for such long sequential generation problems like in TOD systems.
Extensive experiments show that JSA-TOD significantly outperforms its
variational learning counterpart. Remarkably, semi-supervised JSA-TOD using 20%
labels performs close to the full-supervised baseline on MultiWOZ2.1.

本文提出了一种名为 JSA-TOD 的算法，将联合随机逼近应用到半监督学习的潜在状态任务导向对话模型中，结果表明该算法在 MultiWOZ2.1 数据集上的性能显著优于基于变分学习的半监督学习算法，并在使用 20% 标签的情况下接近于完全监督学习的基准。

离散潜变量模型的 JSA 学习提升半监督面向任务的对话系统

Advancing Semi-Supervised Task Oriented Dialog Systems by JSA Learning of Discrete Latent Variable Models

Meta-reinforcement learning algorithms can enable autonomous agents, such as
robots, to quickly acquire new behaviors by leveraging prior experience in a
set of related training tasks. However, the onerous data requirements of
meta-training compounded with the challenge of learning from sensory inputs
such as images have made meta-RL challenging to apply to real robotic systems.
Latent state models, which learn compact state representations from a sequence
of observations, can accelerate representation learning from visual inputs. In
this paper, we leverage the perspective of meta-learning as task inference to
show that latent state models can \emph{also} perform meta-learning given an
appropriately defined observation space. Building on this insight, we develop
meta-RL with latent dynamics (MELD), an algorithm for meta-RL from images that
performs inference in a latent state model to quickly acquire new skills given
observations and rewards. MELD outperforms prior meta-RL methods on several
simulated image-based robotic control problems, and enables a real WidowX
robotic arm to insert an Ethernet cable into new locations given a sparse task
completion signal after only $8$ hours of real world meta-training. To our
knowledge, MELD is the first meta-RL algorithm trained in a real-world robotic
control setting from images.

本文介绍了一种基于元学习的，使用潜在状态模型从图像中加速学习的方法，该方法被用于实现机器人在执行任务时快速获得新技能，该算法优于以前的元再强化学习方法，并且已经在真实世界的机器人控制设置中进行了训练。