Borrowing ideas from {\em Production functions} in micro-economics, in this
paper we introduce a framework to systematically evaluate the performance and
cost trade-offs between machine-translated and manually-created labelled data
for task-specific fine-tuning of massively multilingual language models. We
illustrate the effectiveness of our framework through a case-study on the
TyDIQA-GoldP dataset. One of the interesting conclusions of the study is that
if the cost of machine translation is greater than zero, the optimal
performance at least cost is always achieved with at least some or only
manually-created data. To our knowledge, this is the first attempt towards
extending the concept of production functions to study data collection
strategies for training multilingual models, and can serve as a valuable tool
for other similar cost vs data trade-offs in NLP.

本研究引用微观经济学中的生产函数理论，在机器翻译和手动标注数据的建模中提出一种系统评估模型性能和成本折衷的框架，并通过 TyDIQA-GoldP 数据集的案例研究证明了该框架的有效性和结论为手动标注数据的性价比优于或至少等于机器翻译。此外，这也是第一次尝试将生产函数理论扩展到 NLP 中的数据收集策略研究，并能为类似的成本 vs 数据折衷问题提供有价值的工具。