Using crowdsourcing, we collected more than 10,000 URL pairs (parallel top page pairs) of bilingual websites that contain parallel documents and created a Japanese-Chinese parallel corpus of 4.6M sentence pairs from these websites. We used a Japanese-Chinese bilingual dictionary of 160K word pairs for document and sentence alignment. We then used high-quality 1.2M Japanese-Chinese sentence pairs to train a parallel corpus filter based on statistical language models and word translation probabilities. We compared the translation accuracy of the model trained on these 4.6M sentence pairs with that of the model trained on Japanese-Chinese sentence pairs from CCMatrix (12.4M), a parallel corpus from global web mining. Although our corpus is only one-third the size of CCMatrix, we found that the accuracy of the two models was comparable and confirmed that it is feasible to use crowdsourcing for web mining of parallel data.

使用众包的方法，我们从包含平行文档的双语网站收集了超过10,000个URL对（平行顶级页面对），并从这些网站创建了一个包含4.6M个句对的日语-中文平行语料库。我们使用了一个包含160K个词对的日语-中文双语词典进行文档和句子对齐。随后，我们使用1.2M条高质量的日语-中文句对训练了一个基于统计语言模型和词汇翻译概率的平行语料库过滤器。我们将在这4.6M个句对上训练的模型的翻译准确度与在全球网络挖掘的平行语料库CCMatrix（12.4M）上训练的模型的准确度进行了比较。尽管我们的语料库只有CCMatrix的三分之一大小，但我们发现这两个模型的准确度相当，证实了使用众包进行平行数据的网络挖掘是可行的。

利用众包进行网络挖掘的日中平行语料库