Deep Reinforcement Learning (DRL) is widely used to optimize the performance
of multi-UAV networks. However, the training of DRL relies on the frequent
interactions between the UAVs and the environment, which consumes lots of
energy due to the flying and communication of UAVs in practical experiments.
Inspired by the growing digital twin (DT) technology, which can simulate the
performance of algorithms in the digital space constructed by coping features
of the physical space, the DT is introduced to reduce the costs of practical
training, e.g., energy and hardware purchases. Different from previous
DT-assisted works with an assumption of perfect reflecting real physics by
virtual digital, we consider an imperfect DT model with deviations for
assisting the training of multi-UAV networks. Remarkably, to trade off the
training cost, DT construction cost, and the impact of deviations of DT on
training, the natural and virtually generated UAV mixing deployment method is
proposed. Two cascade neural networks (NN) are used to optimize the joint
number of virtually generated UAVs, the DT construction cost, and the
performance of multi-UAV networks. These two NNs are trained by unsupervised
and reinforcement learning, both low-cost label-free training methods.
Simulation results show the training cost can significantly decrease while
guaranteeing the training performance. This implies that an efficient decision
can be made with imperfect DTs in multi-UAV networks.

通过引入数字孪生技术来减少无人机网络实际训练的成本，该方法使用两个级联神经网络进行优化，同时保证了训练性能。