This paper simulates a low-resource setting across 17 languages in order to
evaluate embedding similarity, stability, and reliability under different
conditions. The goal is to use corpus similarity measures before training to
predict properties of embeddings after training. The main contribution of the
paper is to show that it is possible to predict downstream embedding similarity
using upstream corpus similarity measures. This finding is then applied to
low-resource settings by modelling the reliability of embeddings created from
very limited training data. Results show that it is possible to estimate the
reliability of low-resource embeddings using corpus similarity measures that
remain robust on small amounts of data. These findings have significant
implications for the evaluation of truly low-resource languages in which such
systematic downstream validation methods are not possible because of data
limitations.

本研究通过模拟 17 种语言的低资源情境，评估嵌入式相似性、稳定性和可靠性在不同条件下的表现，并利用语料相似度来预测训练后的嵌入式属性，结果表明可以利用小数据集下的语料相似度估计低资源情境中嵌入式的可靠性。这些结果对于那些数据受限制无法进行系统下游验证的低资源语言的评估具有重要意义。