We present a temporally layered architecture (TLA) for temporally adaptive
control with minimal energy expenditure. The TLA layers a fast and a slow
policy together to achieve temporal abstraction that allows each layer to focus
on a different time scale. Our design draws on the energy-saving mechanism of
the human brain, which executes actions at different timescales depending on
the environment's demands. We demonstrate that beyond energy saving, TLA
provides many additional advantages, including persistent exploration, fewer
required decisions, reduced jerk, and increased action repetition. We evaluate
our method on a suite of continuous control tasks and demonstrate the
significant advantages of TLA over existing methods when measured over multiple
important metrics. We also introduce a multi-objective score to qualitatively
assess continuous control policies and demonstrate a significantly better score
for TLA. Our training algorithm uses minimal communication between the slow and
fast layers to train both policies simultaneously, making it viable for future
applications in distributed control.

我们提出了一种时间分层框架 (TLA)，以最小化能量消耗进行时间自适应控制。 TLAlayer 了一个快速和一个慢速策略以实现时间抽象，使每层专注于不同的时间尺度。我们的设计借鉴了人脑的节能机制，根据环境要求在不同的时间尺度下执行动作。我们在一套连续控制任务上评估了我们的方法，并展示了 TLA 在多个重要指标上的显着优势，包括持续探索，所需决策减少，抖动减少和操作重复增加。我们还引入了一个多目标评分，以定性评估连续控制策略，并展示了 TLA 的得分显着更高。我们的训练算法在慢速和快速层之间使用最少的通信来同时训练两个策略，使其适用于分布式控制的未来应用。