Communication is a crucial factor for the big multi-agent world to stay organized and productive. Recently, deep reinforcement learning (DRL) has been applied to learn the communication strategy and the control policy for multiple agents. However, the practical \emph{\textbf{→