self-play is a common paradigm for constructing solutions in markov games that can yield optimal policies in collaborative settings. However, these policies often adopt highly-specialized conventions that make pl