Opponent modeling has benefited a controlled agent's decision-making by
constructing models of other agents. Existing methods commonly assume access to
opponents' observations and actions, which is infeasible when opponents'
behaviors are unobservable or hard to obtain. We propose a novel multi-agent
distributional actor-critic algorithm to achieve imaginary opponent modeling
with purely local information (i.e., the controlled agent's observations,
actions, and rewards). Specifically, the actor maintains a speculated belief of
the opponents, which we call the \textit{imaginary opponent models}, to predict
opponents' actions using local observations and makes decisions accordingly.
Further, the distributional critic models the return distribution of the
policy. It reflects the quality of the actor and thus can guide the training of
the imaginary opponent model that the actor relies on. Extensive experiments
confirm that our method successfully models opponents' behaviors without their
data and delivers superior performance against baseline methods with a faster
convergence speed.

提出了一种基于分布式 Actor-Critic 算法的多智能体虚拟对手建模方法，通过构建虚拟模型来模拟未观察到对手的行为，使用本地可得信息实现虚拟对手模型的训练和优化。实验证明该方法能有效准确地模拟对手行为，相较于基线方法具有更快的收敛速度并显著提升了性能表现。