Learning tabula rasa, that is without any prior knowledge, is the prevalent
workflow in reinforcement learning (RL) research. However, RL systems, when
applied to large-scale settings, rarely operate tabula rasa. Such large-scale
systems undergo multiple design or algorithmic changes during their development
cycle and use ad hoc approaches for incorporating these changes without
re-training from scratch, which would have been prohibitively expensive.
Additionally, the inefficiency of deep RL typically excludes researchers
without access to industrial-scale resources from tackling
computationally-demanding problems. To address these issues, we present
reincarnating RL as an alternative workflow or class of problem settings, where
prior computational work (e.g., learned policies) is reused or transferred
between design iterations of an RL agent, or from one RL agent to another. As a
step towards enabling reincarnating RL from any agent to any other agent, we
focus on the specific setting of efficiently transferring an existing
sub-optimal policy to a standalone value-based RL agent. We find that existing
approaches fail in this setting and propose a simple algorithm to address their
limitations. Equipped with this algorithm, we demonstrate reincarnating RL's
gains over tabula rasa RL on Atari 2600 games, a challenging locomotion task,
and the real-world problem of navigating stratospheric balloons. Overall, this
work argues for an alternative approach to RL research, which we believe could
significantly improve real-world RL adoption and help democratize it further.
Open-sourced code and trained agents at
this https URL.

提出转世强化学习理论作为强化学习（RL）研究的另一种工作流程或问题类别，以便能够在设计迭代之间或不同代理之间重复使用或转移先前的计算工作（例如，学习策略）; 研究关注现有子优政府的有效传输，并提出了一种简单的算法来解决其限制，最后，在 Atari 2600 游戏，挑战性的互动任务以及导航平流层气球等问题上展现了转世 RL 与 tabula rasa RL 相比的优势，证明这是一种可能提高 RL 实际应用和进一步民主化 RL 的另一种方法。

重生强化学习：重复利用过去计算以加速进展

Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress

The AI City Challenge was created with two goals in mind: (1) pushing the
boundaries of research and development in intelligent video analysis for
smarter cities use cases, and (2) assessing tasks where the level of
performance is enough to cause real-world adoption. Transportation is a segment
ripe for such adoption. The fifth AI City Challenge attracted 305 participating
teams across 38 countries, who leveraged city-scale real traffic data and
high-quality synthetic data to compete in five challenge tracks. Track 1
addressed video-based automatic vehicle counting, where the evaluation being
conducted on both algorithmic effectiveness and computational efficiency. Track
2 addressed city-scale vehicle re-identification with augmented synthetic data
to substantially increase the training set for the task. Track 3 addressed
city-scale multi-target multi-camera vehicle tracking. Track 4 addressed
traffic anomaly detection. Track 5 was a new track addressing vehicle retrieval
using natural language descriptions. The evaluation system shows a general
leader board of all submitted results, and a public leader board of results
limited to the contest participation rules, where teams are not allowed to use
external data in their work. The public leader board shows results more close
to real-world situations where annotated data is limited. Results show the
promise of AI in Smarter Transportation. State-of-the-art performance for some
tasks shows that these technologies are ready for adoption in real-world
systems.

这篇研究论文介绍了人工智能城市挑战赛及其关于交通智能化方面的研究，用于推动智能视频分析在城市交通等领域的发展和提高算法性能以促进其在真实应用中广泛使用。