In Federated Learning (FL), the limited accessibility of data from diverse
locations and user types poses a significant challenge due to restricted user
participation. Expanding client access and diversifying data enhance models by
incorporating diverse perspectives, thereby enhancing adaptability. However,
challenges arise in dynamic and mobile environments where certain devices may
become inaccessible as FL clients, impacting data availability and client
selection methods. To address this, we propose an On-Demand solution, deploying
new clients using Docker Containers on-the-fly. Our On-Demand solution,
employing Deep Reinforcement Learning (DRL), targets client availability and
selection, while considering data shifts, and container deployment
complexities. It employs an autonomous end-to-end solution for handling model
deployment and client selection. The DRL strategy uses a Markov Decision
Process (MDP) framework, with a Master Learner and a Joiner Learner. The
designed cost functions represent the complexity of the dynamic client
deployment and selection. Simulated tests show that our architecture can easily
adjust to changes in the environment and respond to On-Demand requests. This
underscores its ability to improve client availability, capability, accuracy,
and learning efficiency, surpassing heuristic and tabular reinforcement
learning solutions.

我们提出了一个利用深度强化学习技术的按需解决方案，通过在运行时使用 Docker 容器部署新的客户端，以解决动态和移动环境中的数据可用性和客户端选择的挑战。我们的架构能够适应环境变化，并对按需请求做出响应，从而提高客户端的可用性、能力、准确性和学习效率，超越启发式和表格型强化学习解决方案。