Recent years have seen breakthroughs in neural language models that capture
nuances of language, culture, and knowledge. Neural networks are capable of
translating between languages -- in some cases even between two languages where
there is little or no access to parallel translations, in what is known as
Unsupervised Machine Translation (UMT). Given this progress, it is intriguing
to ask whether machine learning tools can ultimately enable understanding
animal communication, particularly that of highly intelligent animals. Our work
is motivated by an ambitious interdisciplinary initiative, Project CETI, which
is collecting a large corpus of sperm whale communications for machine
analysis.
We propose a theoretical framework for analyzing UMT when no parallel data
are available and when it cannot be assumed that the source and target corpora
address related subject domains or posses similar linguistic structure. The
framework requires access to a prior probability distribution that should
assign non-zero probability to possible translations. We instantiate our
framework with two models of language. Our analysis suggests that accuracy of
translation depends on the complexity of the source language and the amount of
``common ground'' between the source language and target prior.
We also prove upper bounds on the amount of data required from the source
language in the unsupervised setting as a function of the amount of data
required in a hypothetical supervised setting. Surprisingly, our bounds suggest
that the amount of source data required for unsupervised translation is
comparable to the supervised setting. For one of the language models which we
analyze we also prove a nearly matching lower bound.
Our analysis is purely information-theoretic and as such can inform how much
source data needs to be collected, but does not yield a computationally
efficient procedure.

我们提出了一个理论框架，用于分析无平行数据可用且源语料库与目标语料库不相干或具有相似语言结构时的 UMT，并且研究表明翻译准确性取决于源语言的复杂性和源语言与目标先验之间的 “共同基础”。