We consider a dynamic multichannel access problem, where multiple correlated
channels follow an unknown joint Markov model. A user at each time slot selects
a channel to transmit data and receives a reward based on the success or
failure of the transmission. The objective is to find a policy that maximizes
the expected long-term reward. The problem is formulated as a partially
observable Markov decision process (POMDP) with unknown system dynamics. To
overcome the challenges of unknown system dynamics as well as prohibitive
computation, we apply the concept of reinforcement learning and implement a
Deep Q-Network (DQN) that can deal with large state space without any prior
knowledge of the system dynamics. We provide an analytical study on the optimal
policy for fixed-pattern channel switching with known system dynamics and show
through simulations that DQN can achieve the same optimal performance without
knowing the system statistics. We compare the performance of DQN with a Myopic
policy and a Whittle Index-based heuristic through both simulations as well as
real-data trace and show that DQN achieves near-optimal performance in more
complex situations. Finally, we propose an adaptive DQN approach with the
capability to adapt its learning in time-varying, dynamic scenarios.

研究多通道接入的问题，应用深度 Q 网络实现对于未知系统动态的自适应学习，以最大化长期回报，通过仿真与实际数据跟踪比较表明该方法在更为复杂的情境下具有最佳性能。