In spite of the new opportunities brought about by the Open RAN, advances in
ML-based network automation have been slow, mainly because of the
unavailability of large-scale datasets and experimental testing infrastructure.
This slows down the development and widespread adoption of Deep Reinforcement
Learning (DRL) agents on real networks, delaying progress in intelligent and
autonomous RAN control. In this paper, we address these challenges by proposing
practical solutions and software pipelines for the design, training, testing,
and experimental evaluation of DRL-based closed-loop control in the Open RAN.
We introduce ColO-RAN, the first publicly-available large-scale O-RAN testing
framework with software-defined radios-in-the-loop. Building on the scale and
computational capabilities of the Colosseum wireless network emulator, ColO-RAN
enables ML research at scale using O-RAN components, programmable base
stations, and a "wireless data factory". Specifically, we design and develop
three exemplary xApps for DRL-based control of RAN slicing, scheduling and
online model training, and evaluate their performance on a cellular network
with 7 softwarized base stations and 42 users. Finally, we showcase the
portability of ColO-RAN to different platforms by deploying it on Arena, an
indoor programmable testbed. Extensive results from our first-of-its-kind
large-scale evaluation highlight the benefits and challenges of DRL-based
adaptive control. They also provide insights on the development of wireless DRL
pipelines, from data analysis to the design of DRL agents, and on the tradeoffs
associated to training on a live RAN. ColO-RAN and the collected large-scale
dataset will be made publicly available to the research community.

这篇论文介绍了 ColO-RAN，是一个能够解决缺少数据和实验测试基础设施的具有大规模的 O-RAN 测试框架，用于设计，训练，测试和评估基于深度强化学习 (DRL) 的闭环控制。它可以在真实网络上延迟进展，并提供了关于开发无线 DRL 管道的见解，从数据分析到 DRL 代理的设计，并在实际环境中展示了 ColO-RAN 的可移植性。