Advances in artificial intelligence (AI) have led to its application in many
areas of everyday life. In the context of control engineering, reinforcement
learning (RL) represents a particularly promising approach as it is centred
around the idea of allowing an agent to freely interact with its environment to
find an optimal strategy. One of the challenges professionals face when
training and deploying RL agents is that the latter often have to run on
dedicated embedded devices. This could be to integrate them into an existing
toolchain or to satisfy certain performance criteria like real-time
constraints. Conventional RL libraries, however, cannot be easily utilised in
conjunction with that kind of hardware. In this paper, we present a framework
named LExCI, the Learning and Experiencing Cycle Interface, which bridges this
gap and provides end-users with a free and open-source tool for training agents
on embedded systems using the open-source library RLlib. Its operability is
demonstrated with two state-of-the-art RL-algorithms and a rapid control
prototyping system.

在控制工程的背景下，人工智能的发展使得其应用于日常生活的许多领域。在强化学习中，RL（reinforcement learning）是一种特别有前景的方法，其核心思想是允许代理与环境自由交互以找到最佳策略。然而，传统的 RL 库很难与嵌入式硬件结合使用。本文介绍了一种名为 LExCI（Learning and Experiencing Cycle Interface）的框架，它弥合了这一差距，并为终端用户提供了一个在嵌入式系统上使用开源库 RLlib 训练代理的免费工具。通过两种最先进的 RL 算法和一种快速控制原型系统来证明其可操作性。

LExCI: 嵌入式系统的强化学习框架

LExCI: A Framework for Reinforcement Learning with Embedded Systems

Reinforcement learning (RL) algorithms involve the deep nesting of highly
irregular computation patterns, each of which typically exhibits opportunities
for distributed computation. We argue for distributing RL components in a
composable way by adapting algorithms for top-down hierarchical control,
thereby encapsulating parallelism and resource requirements within
short-running compute tasks. We demonstrate the benefits of this principle
through RLlib: a library that provides scalable software primitives for RL.
These primitives enable a broad range of algorithms to be implemented with high
performance, scalability, and substantial code reuse. RLlib is available at
this https URL

通过采用自上而下的分层控制算法，对 RL 组件进行可组合的分布式处理，RLlib 提供可扩展的软件基元，从而将并行性和资源需求封装在短暂的计算任务中，实现高性能、可扩展性和大量代码重用的广泛算法的实现。