In reinforcement learning, temporal abstraction in the action space,
exemplified by action repetition, is a technique to facilitate policy learning
through extended actions. However, a primary limitation in previous studies of
action repetition is its potential to degrade performance, particularly when
sub-optimal actions are repeated. This issue often negates the advantages of
action repetition. To address this, we propose a novel algorithm named
Uncertainty-aware Temporal Extension (UTE). UTE employs ensemble methods to
accurately measure uncertainty during action extension. This feature allows
policies to strategically choose between emphasizing exploration or adopting an
uncertainty-averse approach, tailored to their specific needs. We demonstrate
the effectiveness of UTE through experiments in Gridworld and Atari 2600
environments. Our findings show that UTE outperforms existing action repetition
algorithms, effectively mitigating their inherent limitations and significantly
enhancing policy learning efficiency.

通过使用 ensemble 方法，我们提出了一种名为 Uncertainty-aware Temporal Extension（UTE）的新算法，它能够准确测量在行为扩展期间的不确定性，这使得策略可以在探索和不确定性规避之间做出有针对性的选择，从而有效地提高了策略学习的效率。

学习具有不确定性感知的时间延续动作

Learning Uncertainty-Aware Temporally-Extended Actions

We present an approach for estimating the period with which an action is
repeated in a video. The crux of the approach lies in constraining the period
prediction module to use temporal self-similarity as an intermediate
representation bottleneck that allows generalization to unseen repetitions in
videos in the wild. We train this model, called Repnet, with a synthetic
dataset that is generated from a large unlabeled video collection by sampling
short clips of varying lengths and repeating them with different periods and
counts. This combination of synthetic data and a powerful yet constrained
model, allows us to predict periods in a class-agnostic fashion. Our model
substantially exceeds the state of the art performance on existing periodicity
(PERTUBE) and repetition counting (QUVA) benchmarks. We also collect a new
challenging dataset called Countix (~90 times larger than existing datasets)
which captures the challenges of repetition counting in real-world videos.
Project webpage: this https URL .

本文介绍了一种利用时间自相似性作为中间表征的方法，通过一定的限制来预测视频中重复动作的周期，结果表明，该模型在现有周期性和重复计数基准测试任务上显著超越现有技术，并在新的数据集 Countix 上显示了鲁棒性。