We generalize active learning to address real-world settings where sampling
is restricted to an accessible region of the domain, while prediction targets
may lie outside this region. To this end, we propose ITL, short for
information-based transductive learning, an approach which samples adaptively
to maximize the information gained about specified prediction targets. We show,
under general regularity assumptions, that ITL converges uniformly to the
smallest possible uncertainty obtainable from the accessible data. We
demonstrate ITL in two key applications: Few-shot fine-tuning of large neural
networks and safe Bayesian optimization, and in both cases, ITL significantly
outperforms the state-of-the-art.

我们将主动学习推广到应对真实世界中的情况，其中采样被限制在域的可访问区域内，而预测目标可能位于该区域之外。为此，我们提出了 ITL，即信息驱动的转导学习，一种自适应采样的方法，旨在最大限度地提高对指定预测目标的信息获取。在普遍的规则性假设下，我们证明 ITL 会一致地收敛到可从可访问数据中获得的最小不确定性。我们在两个关键应用中展示了 ITL 的应用：大型神经网络的少样本微调和安全贝叶斯优化，ITL 在这两种情况下明显优于现有技术。

基于信息的迁移式主动学习

Information-based Transductive Active Learning

Value estimation is a critical component of the reinforcement learning (RL)
paradigm. The question of how to effectively learn value predictors from data
is one of the major problems studied by the RL community, and different
approaches exploit structure in the problem domain in different ways. Model
learning can make use of the rich transition structure present in sequences of
observations, but this approach is usually not sensitive to the reward
function. In contrast, model-free methods directly leverage the quantity of
interest from the future, but receive a potentially weak scalar signal (an
estimate of the return). We develop an approach for representation learning in
RL that sits in between these two extremes: we propose to learn what to model
in a way that can directly help value prediction. To this end, we determine
which features of the future trajectory provide useful information to predict
the associated return. This provides tractable prediction targets that are
directly relevant for a task, and can thus accelerate learning the value
function. The idea can be understood as reasoning, in hindsight, about which
aspects of the future observations could help past value prediction. We show
how this can help dramatically even in simple policy evaluation settings. We
then test our approach at scale in challenging domains, including on 57 Atari
2600 games.

本文提出了利用表征学习中的先验信息直接进行值函数预测的方法，即结合模型学习和模型自由方法的优势，确定哪些未来轨迹特征提供有用信息，从而为任务提供可操作的预测目标，加速值函数的学习。