Recent work has exhibited the surprising cross-lingual abilities of multilingual bert (M-BERT) -- surprising since it is trained without any cross-lingual objective and with no aligned data. In this work, we provide a comprehensive study of the contribution of different components in M