Large language models (LLMs) exhibit emerging geospatial capabilities,
stemming from their pre-training on vast unlabelled text datasets that are
often derived from the Common Crawl corpus. However, the geospatial content
within CC remains largely unexplored, impacting our understanding of LLMs'
spatial reasoning. This paper investigates the prevalence of geospatial data in
recent Common Crawl releases using Gemini, a powerful language model. By
analyzing a sample of documents and manually revising the results, we estimate
that between 1 in 5 and 1 in 6 documents contain geospatial information such as
coordinates and street addresses. Our findings provide quantitative insights
into the nature and extent of geospatial data within Common Crawl, and web
crawl data in general. Furthermore, we formulate questions to guide future
investigations into the geospatial content of available web crawl datasets and
its influence on LLMs.

分析大型语言模型对 Common Crawl 数据集的地理空间数据的利用率，发现 1/5 至 1/6 的文档中包含有经纬度和街道地址等地理空间信息，为进一步研究有关地理空间内容对大型语言模型的影响提供了定量的见解。

在 Common Crawl 语料库中量化地理空间

Quantifying Geospatial in the Common Crawl Corpus

Cross-lingual document alignment aims to identify pairs of documents in two
distinct languages that are of comparable content or translations of each
other. In this paper, we exploit the signals embedded in URLs to label web
documents at scale with an average precision of 94.5% across different language
pairs. We mine sixty-eight snapshots of the Common Crawl corpus and identify
web document pairs that are translations of each other. We release a new web
dataset consisting of over 392 million URL pairs from Common Crawl covering
documents in 8144 language pairs of which 137 pairs include English. In
addition to curating this massive dataset, we introduce baseline methods that
leverage cross-lingual representations to identify aligned documents based on
their textual content. Finally, we demonstrate the value of this parallel
documents dataset through a downstream task of mining parallel sentences and
measuring the quality of machine translations from models trained on this mined
data. Our objective in releasing this dataset is to foster new research in
cross-lingual NLP across a variety of low, medium, and high-resource languages.

利用 URL 标签指导对网络文章进行跨语言文档对齐，构建了一个大规模的网络文档数据集，使用该数据集通过基线模型进行文本内容的对齐，最终通过挖掘平行语句和测量基于挖掘数据训练的模型的机器翻译质量来证明了数据集的价值，旨在促进跨语言 NLP 研究的发展。

CCAligned：大规模跨语言网页文档对集合

CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs

In this paper we present a preliminary analysis over the largest publicly
accessible web dataset: the Common Crawl Corpus. We measure nine web
characteristics from two levels of granularity using MapReduce and we comment
on the initial observations over a fraction of it. To the best of our knowledge
two of the characteristics, the language distribution and the HTML version of
pages have not been analyzed in previous work, while the specific dataset has
been only analyzed on page level.

本研究使用 MapReduce 对 Common Crawl Corpus 进行了初步分析，总结出语言分布和 HTML 版本是该数据集中尚未被研究的网页特征之一。